PREDICTION OF A REPRESENTATION OF AN INVESTIGATION AREA OF AN INVESTIGATION OBJECT IN A STATE OF A SEQUENCE OF STATES

Description

[0001] The present invention relates to the technical field of radiology, in particular to supporting radiologists in radiological examinations using artificial intelligence methods. The present invention relates to training a machine learning model and using the trained model to predict representations of an examination area in one or more states of a sequence of states during a radiological examination.

[0002] L. Zhang et al. reveal spatio-temporal convolutional LSTMs for tumor growth prediction by learning from 4D longitudinal patient data (Spatio-Temporal Convolutional LSTMs for Tumor Growth Prediction by Learning 4D Longitudinal Patient Data, IEEE Trans Med Imaging. 2020 April ; 39(4): 1114-1126).

[0003] P. Luc et al. reveal methods for predicting future semantic segmentations (Predicting Deeper into the Future of Semantic Segmentation, Proceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017, 648-657).

[0004] The temporal tracking of processes within the body of a human or animal using imaging techniques plays an important role in the diagnosis and / or therapy of diseases, among other things.

[0005] One example is the detection and differential diagnosis of focal liver lesions using dynamic contrast-enhancing magnetic resonance imaging (MRI) with a hepatobiliary contrast agent.

[0006] A hepatobiliary contrast agent such as Primovist® can be used to detect tumors in the liver. The blood supply to healthy liver tissue is primarily provided by the portal vein (vena portae), while the hepatic artery (arteria hepatica) supplies most primary tumors. Therefore, after an intravenous bolus injection of a contrast agent, a time delay can be observed between the signal increase in healthy liver parenchyma and that in the tumor.

[0007] In addition to malignant tumors, benign lesions such as cysts, hemangiomas, and focal nodular hyperplasia (FNH) are frequently found in the liver. For appropriate treatment planning, these must be differentiated from malignant tumors. Primovist® can be used to detect benign and malignant focal liver lesions. It provides information about the nature of these lesions using T1-weighted MRI. Differentiation is achieved by utilizing the different blood supply to the liver and tumors, as well as the temporal course of contrast enhancement.

[0008] The contrast enhancement achieved with Primovist® during the onset phase reveals typical perfusion patterns that provide information for characterizing the lesions. Visualizing the vascularization helps to characterize the lesion types and determine the spatial relationship between the tumor and blood vessels.

[0009] In T1-weighted MRI scans, Primovist® < 10-20 minutes after injection (in the hepatobiliary phase) leads to a marked signal enhancement in healthy liver parenchyma, while lesions containing no or few hepatocytes, e.g. metastases or moderately to poorly differentiated hepatocellular carcinomas (HCCs), appear as darker areas.

[0010] Monitoring the distribution of the contrast agent over time offers a valuable tool for the detection and differential diagnosis of focal liver lesions; however, the examination takes a relatively long time. During this period, patient movement should be largely avoided to minimize motion artifacts in the MRI scans. This prolonged restriction of movement can be uncomfortable for the patient.

[0011] Disclosure WO2021 / 052896A1 proposes that one or more MRI scans during the hepatobiliary phase should not be generated using measurement techniques, but rather calculated (predicted) based on MRI scans from one or more previous phases in order to shorten the patient's stay in the MRI scanner.

[0012] The approach described in the patent application WO2021 / 052896A1 involves training a machine learning model to predict an MRI scan of the same area at a later time, based on MRI scans taken before and / or immediately after the administration of a contrast agent. The model is thus trained to map multiple MRI scans as input data to a single MRI scan as output data. mapping ) .

[0013] Based on the described state of the art, the task was to improve the prediction quality of the machine learning model and / or to create a model that learns the dynamics of the contrast agent distribution in order to be able to use it in a variety of ways.

[0014] This problem is solved by the subject matter of the present independent claims. Preferred embodiments are found in the dependent claims, as well as in the present description and in the drawings.

[0015] A first object is a computer-implemented method according to claim 1.

[0016] Another object of the present invention is a computer system according to claim 9. an input unit, a control and arithmetic unit, and an output unit, wherein the control and arithmetic unit is configured as a representation R p to receive a study area, where the representation R p the investigation area in a state Z p a sequence of states Z 1 to Z n represented, whereby p an integer that is less than n is, whereby n an integer greater than two, the representation R p to feed into a trained machine learning model, where the machine learning model has been trained on the basis of training data,

[0017] Another object of the present invention is a computer program product according to claim 10.

[0018] Another object of the present invention is the use of a contrast agent in a radiological examination method according to claim 11.

[0019] Another object of the present invention is a kit according to claim 12.

[0020] The invention is explained in more detail below, without distinguishing between the subject matter of the invention (method, computer system, computer program product, use, kit). Rather, the following explanations are intended to apply analogously to all subject matter of the invention, regardless of the context in which they are made (method, computer system, computer program product, use, kit).

[0021] If steps are mentioned in a sequence in the present description or in the claims, this does not necessarily mean that the invention is limited to that sequence. Rather, it is conceivable that the steps could also be performed in a different sequence or even in parallel; unless one step builds upon another, which makes it essential that the building step be performed subsequently (which will be clear in each individual case). The sequences mentioned thus represent preferred embodiments of the invention. With the aid of the present invention, representations of an investigation area of ​​an object under investigation can be predicted.

[0022] According to the invention, the area of ​​investigation is the liver or a part of the liver of a human being.

[0023] The examination area, also called the recording volume (English: field of view, The field of view (FOV) is a volume that is depicted in radiological images. The area under examination is typically selected by a radiologist, for example, on a panoramic radiograph (or overview image). localizer ). Alternatively or additionally, the scope of investigation can also be defined automatically, for example based on a selected protocol.

[0024] According to the invention, a representation of the area under investigation is a radiological image.

[0025] Radiology is the branch of medicine that deals with the application of primarily electromagnetic radiation and (including, for example, ultrasound diagnostics) mechanical waves for diagnostic, therapeutic, and / or scientific purposes. In addition to X-rays, other ionizing radiation such as gamma rays or electrons are also used. Since a key application is imaging, other imaging techniques such as sonography and magnetic resonance imaging (MRI) are also considered part of radiology, even though these techniques do not use ionizing radiation. The term "radiology" as used in the present invention therefore includes, in particular, the following examination methods: computed tomography, magnetic resonance imaging, and sonography.

[0026] In a preferred embodiment of the present invention, the radiological examination is a magnetic resonance imaging examination.

[0027] Magnetic resonance imaging, abbreviated MRI (English: MRI: Magnetic Resonance Imaging ), is an imaging technique used primarily in medical diagnostics to depict the structure and function of tissues and organs in the human or animal body.

[0028] In magnetic resonance imaging (MRI), the magnetic moments of protons in a sample are aligned in a baseline magnetic field, resulting in macroscopic magnetization along a longitudinal axis. This magnetization is then displaced from its resting position by the application of radiofrequency (RF) pulses (excitation). The return of the excited states to their resting positions (relaxation), or the magnetization dynamics, is subsequently detected as relaxation signals using one or more RF receiver coils.

[0029] For spatial encoding, rapidly switched magnetic gradient fields are superimposed on the fundamental magnetic field. The acquired relaxation signals or the detected MR data initially exist as raw data in the frequency domain and can be transformed into the spatial domain (image space) by subsequent inverse Fourier transformation.

[0030] A representation of the area under investigation within the meaning of the present invention may be an MRI scan, a computed tomography scan, an ultrasound image or the like.

[0031] A representation of the investigation area within the meaning of the present invention can be a representation in spatial space (image space), a representation in frequency space, or another representation. Preferably, representations of the investigation area are in a spatial representation or in a form that can be converted (transformed) into a spatial representation.

[0032] A representation in accordance with the present invention shows the investigation area in one state of a sequence of states.

[0033] The sequence of states is preferably a temporal sequence. Within a temporal sequence, immediately successive states can always have the same time interval between them, or they can have varying time intervals. Mixed forms are also conceivable.

[0034] This was demonstrated using the following: Fig. 1 explained.

[0035] Fig. 1 shows two timelines ( t = time), a first timeline (a) and a second timeline (b). Defined points in time are shown on each timeline. t 1 , t 2 , t 3 and t 4 marked. At any given time, the area under investigation can have a different state, i.e., each point in time can represent a state of the area under investigation.

[0036] The times t 1 , t 2 , t 3 and t 4 form a chronological sequence for each timeline:t 1 → t 2 → t 3 → t 4. The timing t 2 immediately follows time t 1, the time interval between t 1 and t 2 is t 2 -t 1 ; the time t 3 follows immediately after the point in time t 2 , the time interval between t 2 and t 3 is t 3 -t 2; the time t 4 follows immediately after the point in time t 3, the time interval between the points in time t 3 and t 4 is t 4 -t 3 .

[0037] In the case of the first timeline (a), the time intervals between immediately consecutive points in time are the same for all immediately consecutive points in time: t 2 -t 1 = t 3 -t 2 = t 4 -t 3 .

[0038] In the case of the second timeline (b), the time intervals between immediately consecutive time points vary for all immediately consecutive time points: t 2 -t 1 ≠ t 3 -t 2 ≠ t 4 -t 3 . In the present example, the time intervals increase in the case of the second time axis (b) for immediately consecutive time points: t 2 -t 1 < t 3 -t 2 < t 4 -t 3 . However, it is also conceivable that they decrease, or initially increase and then decrease, or initially decrease and then increase, or are distributed differently along the time axis.

[0039] It should be noted that the time axis does not necessarily have to run in the direction of increasing time. This means that the point in time t 1 from the time t2. From the perspective of this point, it does not necessarily have to be in the past; it is also conceivable that the time axis shows a decreasing time and the point in time t 1 from the time t 2. From the perspective of the first state, it lies in the future. In other words, if there is a temporal sequence of states, then a second state immediately following a first state, viewed from the time of the first state, can lie in the future or it can lie in the past.

[0040] The (preferably temporal) sequence can be determined in a first step. The temporal sequence defines which training data is used to train the machine learning model and which representations the machine learning model can generate (predict). In other words, to train the machine learning model, reference representations of the domain of study are required in the states defined by the sequence of states, and the trained machine learning model can typically only generate representations of such states that have been part of the training (exceptions to this rule are listed later in the description).

[0041] According to the invention, the sequence of states defines different states of the investigation area before and / or during and / or after one or more applications of a contrast agent.

[0042] "Contrast agents" are substances or mixtures of substances that improve the visualization of the body's structures and functions in radiological imaging procedures.

[0043] Examples of contrast agents can be found in the literature (see, e.g., ASL Jascinth et al.: Contrast Agents in computed tomography: A Review, Journal of Applied Dental and Medical Sciences, 2016, Vol. 2, Issue 2, 143–149; H. Lusic et al.: X-ray-Computed Tomography Contrast Agents, Chem. Rev. 2013, 113, 3, 1641–1666; https: / / www.radiology.wisc.edu / wp-content / uploads / 2017 / 10 / contrast-agents-tutorial.pdf, MR Nough) et al.: Radiographie and magnetic resonances contrast agents: Essentials and tips, for safe practices, World J Radiol. 2017 Sep 28; 9(9): 339-349; L. C. Abonyi et al.: Intravascular Contrast Media in Radiography: Historical Development & Review ofRisk Factors for Adverse Reactions, South American Journal of Clinical Research, 2016, Vol. 3, Issue 1, 1-10; ACR Manual on Contrast Media, 2020, ISBN: 978-1-55903-012-0; A. Ignee et al.: Ultrasound contrast agents, Endosc Ultrasound. 2016 Nov-Dec; 5(6): 355-362).

[0044] Preferably, the contrast agent is an MR contrast agent. MR contrast agents exert their effect by altering the relaxation times of the structures that absorb the contrast agent. Two groups of substances can be distinguished: para- and superparamagnetic substances. Both groups possess unpaired electrons that induce a magnetic field around the individual atoms or molecules. Superparamagnetic contrast agents lead to a predominantly T2 shortening, while paramagnetic contrast agents lead primarily to a T1 shortening. The effect of these contrast agents is indirect, since the contrast agent itself does not emit a signal, but only influences the signal intensity in its surroundings. An example of a superparamagnetic contrast agent is iron oxide nanoparticles (SPIOs). superparamagnetic iron oxide ) .Examples of paramagnetic contrast agents include gadolinium chelates such as gadopentetate dimeglumine (trade name: Magnevist ®< etc.), gadoteric acid (Dotarem ®< , Dotagita ®< , Cyclolux ®< ), gadodiamide (Omniscan ®< ), gadoteridol (ProHance ®< ) and gadobutrol (Gadovist ®< ).

[0045] According to the invention, the MR contrast agent is a hepatobiliary contrast agent. A hepatobiliary contrast agent is characterized by the fact that it is specifically taken up by liver cells, the hepatocytes, accumulates in the functional tissue (parenchyma), and enhances the contrast in healthy liver tissue. An example of a hepatobiliary contrast agent is the disodium salt of gadoxetic acid (Gd-EOB-DTPA disodium), which is described in US Patent No. 6,039,931A and is commercially available under the brand names Primovist® and Eovist®.

[0046] Following the intravenous administration of a hepatobiliary contrast agent as a bolus into a vein in the arm, the contrast agent initially reaches the liver via the arteries. These are shown with contrast enhancement in the corresponding MRI scans. The phase in which the hepatic arteries appear with contrast enhancement in MRI scans is referred to as the "arterial phase."

[0047] The contrast agent then reaches the liver via the hepatic veins. While the contrast in the hepatic arteries is already decreasing, it reaches its maximum in the hepatic veins. The phase in which the hepatic veins appear contrast-enhanced in MRI scans is called the "portal venous phase."

[0048] The portal venous phase is followed by the "transition phase" (English: transition phase). transitional phase ) in which the contrast in the hepatic arteries continues to decrease, and the contrast in the hepatic veins also decreases. When using a hepatobiliary contrast agent, the contrast in healthy liver cells gradually increases during the transition phase.

[0049] The arterial phase, the portal venous phase and the transition phase are collectively referred to as the "dynamic phase".

[0050] Ten to twenty minutes after injection, a hepatobiliary contrast agent leads to a marked increase in signal intensity in healthy liver parenchyma. This phase is called the "hepatobiliary phase." The contrast agent is eliminated from the liver cells only slowly; accordingly, the hepatobiliary phase can last two hours or more.

[0051] The phases mentioned are described in more detail in the following publications, for example: J. Magn. Reson. Imaging, 2012, 35(3): 492-511, doi:10.1002 / jmri.22833; Clujul Medical, 2015, Vol. 88 no. 4: 438-448, DOI: 10.15386 / cjmed-414; Journal of Hepatology, 2019, Vol. 71: 534-542, http: / / dx.doi.org / 10.1016 / j.jhep.2019.05.005).

[0052] The conditions resulting from the sequence of conditions could, for example, include the condition of the liver as an area of ​​investigation. act before the application of a hepatobiliary contrast agent, in the arterial phase, in the portal venous phase, in the transition phase and / or in the hepatobiliary phase.

[0053] It is conceivable that there are more or fewer states.

[0054] The aforementioned phases are described below using the following examples: Fig. 2 explained in more detail. Fig. 2 schematically shows the time course ( t (= time) of the signal intensities I, which are caused by a hepatobiliary contrast agent in hepatic arteries A, hepatic veins V and healthy liver cells L during a dynamic contrast-enhanced MRI examination with a hepatobiliary contrast agent. The signal intensity I The concentration of the contrast agent in the aforementioned areas correlates positively with the concentration of the contrast agent. Following an intravenous bolus injection into a person's arm, the concentration of the contrast agent in the hepatic arteries A rises first (dashed curve). The concentration reaches a maximum and then decreases. The concentration in the hepatic veins V rises more slowly than in the hepatic arteries and reaches its maximum later (dotted curve). The concentration of the contrast agent in the liver cells L rises slowly (solid curve) and reaches its maximum only at a much later time (in the Figur 2 (not shown). Several characteristic time points can be defined: At time point TP1, contrast medium is administered intravenously as a bolus. Since the administration of a contrast medium itself takes a certain amount of time, time point TP1 preferably defines the time at which the administration is complete, i.e., when the contrast medium has been completely introduced into the subject of the examination. At time point TP2, the signal intensity of the contrast medium reaches its maximum in the hepatic arteries A. At time point TP3, the signal intensity curves of the hepatic arteries A and the hepatic veins V intersect. At time point TP4, the signal intensity of the contrast medium reaches its maximum in the hepatic veins V. At time point TP5, the signal intensity curves of the hepatic arteries A and the healthy liver cells L intersect.At time TP6, the concentrations in the hepatic arteries A and the hepatic veins V have decreased to a level at which they no longer cause measurable contrast enhancement.

[0055] The sequence of states can include a first state that exists before time TP1, a second state that exists at time TP2, a third state that exists at time TP3, a fourth state that exists at time TP4, a fifth state that exists at time TP5 and / or a sixth state that exists at time TP6 and / or thereafter.

[0056] In general A first state is the state of an examination area at a first time point before the administration of a contrast agent; a second state is the state of the examination area at a second time point, after the administration of the contrast agent; a third state is the state of the examination area at a third time point, after the administration of the contrast agent; a fourth state is the state of the examination area at a fourth time point, after the administration of the contrast agent. and so on.

[0057] In this context, consecutive points in time as described above can have a constant time interval between them, either wholly or partially, and / or a variable time interval between them.

[0058] According to the invention, there are at least three states, preferably the number of states is between 3 and 100. However, the number of states is not limited to 100.

[0059] For each state in the sequence of states, there can be one or more representations that represent the area of ​​investigation in that particular state.

[0060] Typically, for a first state there is at least one first representation that represents the domain of investigation in the first state, for a second state there is at least one second representation that represents the domain of investigation in the second state, for a third state there is at least one third representation that represents the domain of investigation in the third state, and so on.

[0061] For the sake of clarity, the present invention is described primarily on the basis of one representation per state; however, this should not be understood as a limitation of the invention. The actual scope of protection is defined exclusively by the accompanying claims.

[0062] Returning to the example described above, it could, for example, Provide one or more first representations representing the liver before the application of a hepatobiliary contrast agent, provide one or more second representations representing the liver during the arterial phase, provide one or more third representations representing the liver during the portal venous phase, provide one or more fourth representations representing the liver during the transition phase, and / or provide one or more fifth representations representing the liver during the hepatobiliary phase.

[0063] With the aid of the present invention, one or more representations of an investigation area of ​​an investigation object can be predicted, which represent the investigation area in one state of a sequence of states.

[0064] The prediction is made using a machine learning model.

[0065] A "machine learning model" can be understood as a computer-implemented data processing architecture. The model can receive input data and provide output data based on this input data and model parameters. Through training, the model can learn a relationship between the input data and the output data. During training, model parameters can be adjusted to produce a desired output for a given input.

[0066] When training such a model, it is presented with training data from which it can learn. The trained machine learning model is the result of the training process. In addition to input data, the training data includes the correct output data (target data) that the model is to generate based on the input data. During training, patterns are recognized that map the input data to the target data.

[0067] During the training process, the input data for the training data is fed into the model, and the model generates output data. This output data is then compared to the target data. Model parameters are adjusted to reduce the deviations between the output data and the target data to a (defined) minimum.

[0068] The deviations can be analyzed using an error function (English: loss function ) can be quantified. Such an error function can be used to determine an error value (English: loss value The goal of the training process is to calculate the error value for a given pair of output and target data. This can involve modifying (adjusting) the parameters of the machine learning model to reduce the error value for all pairs in the training dataset to a (defined) minimum.

[0069] If the output and target data are numbers, for example, the error function can be the absolute difference between these numbers. In this case, a high absolute error value may mean that one or more model parameters need to be changed significantly.

[0070] For output data in the form of vectors, difference metrics between vectors such as the mean squared error, a cosine distance, a norm of the difference vector such as a Euclidean distance, a Chebyshev distance, an Lp norm of a difference vector, a weighted norm, or any other type of difference metric of two vectors can be chosen as the error function.

[0071] For higher-dimensional outputs, such as two-dimensional, three-dimensional, or even higher-dimensional outputs, an element-wise difference metric can be used. Alternatively or additionally, the output data can be transformed before calculating an error value, for example, into a one-dimensional vector.

[0072] In the present case, the machine learning model is trained using training data to generate a representation of a domain of investigation in a state of a sequence of states based on representations of the domain of investigation in previous states of the sequence of states, whereby the model generates representations of the domain of investigation in subsequent states starting from a first state, with the generation of each representation of the domain of investigation in each state being at least partially based on a (predicted) representation of the previous state.

[0073] This will be explained using an example of three states, without limiting the invention to this embodiment. This explanation will refer to... Fig. 3 Reference made to.

[0074] Fig. 3 The diagram shows three representations: a first representation R1, a second representation R2, and a third representation R3. The first representation R1 represents a domain of investigation of an object in a first state Z1, the second representation R2 represents the domain of investigation in a second state Z2, and the third representation R3 represents the domain of investigation in a third state Z3. The three states form a sequence of states: first state → second state → third state.

[0075] The three representations R1, R2, and R3 form a dataset within training data TD. The training data TD comprises a multitude of such datasets. The term "multitude" preferably means more than 100. Each dataset typically (but not necessarily) contains three representations of the domain of study for the three states, one representation for each state. The domain of study is usually always the same, and each dataset contains representations of the domain of study in the three states, which are also usually the same for all datasets: first state, second state, third state. Only the object of study can vary. Each dataset usually originates from a different object of study. The statements made in this paragraph apply generally, not only with regard to the one in Fig. 3 Example shown.

[0076] The representations used to train the machine learning model are also referred to as reference representations in this description; for better understanding / readability, the explanation will use the term "reference representations". Fig. 3 and Fig. 4 The addition of "reference" was omitted.

[0077] Representations generated (predicted) using the machine learning model are predominantly marked in this description by a superscript asterisk symbol *.

[0078] In a first step (A), the first representation R1 is fed into a machine learning model M. The machine learning model M is configured to generate an output R2* based on the first representation R1 and model parameters MP (step (B)). The output R2* should be as close as possible to the second representation R2; ideally, the output R2* should be indistinguishable from the second representation R2. In other words, the output R2* is a predicted second representation. The output R2* is compared with the (actual) second representation R2, and the deviations are quantified using an error function LF2. For each pair of output R2* and second representation R2, an error value LV2 can be calculated using the error function LF2.

[0079] Examples of error functions that are general (not limited to the example in Fig. 3 ) can be used to carry out the present invention, are L1 fault function ( L1 loss ), L2 error function ( L2 loss ), Lp loss function, structural similarity index measure ( structural similarity index measure (SSIM)), VGG loss, perceptual loss ( perceptual loss ) or a combination of the above-mentioned functions or other error functions. Further details on error functions can be found, for example, in the scientific literature (see, e.g., R. Mechrez et al.: The Contextual Loss for Image Transformation with Non-Aligned Data, 2018, arxiv:1803.02077v4, H. Zhao et al.: Loss Functions for Image Restoration with Neural Networks, 2018, arxiv:1511.08861v3).

[0080] The output R 2 * is fed back into the machine learning model M in step (C). Even if Fig. 3 This might give a different impression: the machine learning model M, which in step (A) receives the first representation R1 and in step (C) the predicted second representation R2*, is the same model. That is, the machine learning model is not only configured to generate a predicted second representation based on the first, but it is also configured to generate an output R3* from a (predicted and / or actual) second representation that is as close as possible to the third representation R3. The output R3* is a predicted third representation. The output R3* is then compared to the third representation R3.Using an error function LF 3, the deviation between the output R 3 * and the third representation can be quantified; an error value LV 3 can be determined for each pair of a third representation and a predicted third representation.

[0081] Preferably, the machine learning model is end-to-end (English: end-to-end). end-to-end ) trained. This means that the machine learning model is trained simultaneously to generate a predicted second representation based on the first representation and a predicted third representation based on the predicted second representation. An error function is used for this purpose, which takes into account both the deviations between the predicted second representation and the second representation, as well as the deviations between the predicted third representation and the third representation.

[0082] It is possible to quantify the deviations between the second representation and the predicted second representation using the error function LF2, and the deviations between the third representation and the predicted third representation using the error function LF3. An error function LF that takes both deviations into account can, for example, be the sum of the individual error functions: LF = LF2 + LF3. It is also possible to weight the contributions of the individual error functions LF2 and LF3 to the overall error function LF differently: LF = w2 · LF2 + w3 · LF3, where w2 and w3 are weighting factors that can take values ​​between 0 and 1, for example. A weighting factor of zero can be used, for instance, if a representation is missing from a dataset (more on this can be found in the description later).

[0083] When the trained machine learning model is used (later) to predict new representations, the goal may be to predict a third representation based on a first representation. In principle, a model could be trained to generate the third representation directly based on the first. However, according to the invention, the machine learning model is trained not to generate the third representation in one step based on the first representation, but in two steps. In the first step, a second representation is predicted, and the third representation is then predicted based on this second representation.The advantage of the approach according to the invention compared to the aforementioned "direct training" is, among other things, that additional training data (second representations representing the domain of investigation in the second state) can be used, thus enabling higher prediction accuracy. Furthermore, the model does not learn a mapping from one representation to another (or, as in the case of WO2021 / 052896A1, a mapping from several representations to one representation), but rather it learns the dynamic behavior of the domain of investigation, i.e., the dynamics of how the states are traversed from one to another. The more states the training data covers, the more accurately the model can learn the dynamics from one state to the subsequent states.Furthermore, a model trained in this way can be used to predict representations of states for which no training data was available by using the model multiple times (iteratively) to predict representations of subsequent states. This extrapolation to states not considered during training is described in more detail later in this document.

[0084] The in Fig. 3 The described method is a preferred embodiment of the present invention and comprises, in summary, the following steps: Receiving a first representation R1 of a domain of investigation, wherein the first representation R1 represents the domain of investigation in a first state Z1 of a sequence of states Z1, Z2, Z3; generating a predicted second representation R2* of the domain of investigation using a machine learning model, at least partially based on the first representation R1, wherein the second representation R2* represents the domain of investigation in the second state Z2, where the second state Z2 immediately follows the first state Z1 in the sequence of states; generating a predicted third representation R3* of the domain of investigation using the machine learning model, at least partially based on the predicted second representation R2*, wherein the predicted third representation R3* represents the domain of investigation in the third state Z3.where the third state Z 3 immediately follows the second state Z 2 in the sequence of states, output and / or storage and / or transmission of the third representation R 3 *, , wherein the machine learning model was trained on the basis of training data, wherein the training data comprised a multitude of datasets, each dataset comprising a first reference representation, a second reference representation and a third reference representation, wherein the first reference representation represented the domain of study in the first state Z 1 The sequence of states is represented, the second reference representation the investigation domain in the second state Z. 2 the sequence of states is represented, and the third reference representation represents the domain of investigation in the third state Z. 3 the sequence of states is represented, whereby the machine learning model was trained, at least partially, on the basis of the first reference representation. to predict the second reference representation and to predict the third reference representation, at least partially, based on the predicted second reference representation.

[0085] In general, the inventive machine learning model can be trained to create a representation of an investigation domain in a state Z. i a sequence of states Z 1 to Z n to predict, whereby n an integer greater than 2 and i an index that represents the numbers from 2 to nThe process runs through the machine learning model, starting with an initial representation of the domain of investigation in state Z1, followed by a successive sequence of predicted representations of the domain of investigation in states Z2 to Z3. n to generate, whereby each representation of a state is generated at least partially based on the predicted representation of the immediately preceding state.

[0086] This can be illustrated by the example of Fig. 4 explained in more detail. Fig. 4 can be considered an extension of the in Fig. 3 shown scheme of three states on n States are understood, whereby n a whole number greater than two. Preferably the number n in the range of 3 to 100.

[0087] Fig. 4 The image shows a machine learning model M. The model is shown three times; however, it is always the same model. The machine learning model M is trained, a number n- 1. To predict representations. In the present example, nAn integer greater than 3. The model is fed a first representation R1 as input data. The first representation R1 represents a domain of investigation of an object in a first state Z1. The machine learning model M is configured to generate an output R2*, at least partially, based on the first representation R1 and model parameters MP. The output R2* represents a predicted second representation that represents the domain of investigation in a second state Z2. The model is further configured to generate an output R3*, at least partially, based on the output R2* and model parameters MP. The output R3* represents a predicted third representation that represents the domain of investigation in a third state Z3.The model is further configured to generate, at least partially, another output based on the output R3* and model parameters MP, which shows a predicted representation of the investigation domain in a state following state Z3. This scheme continues up to an output R. n * continued. Issue R n * represents a predicted n -th representation, which represents the area of ​​investigation in a n -ten state Z n represented.

[0088] The states Z i (with i = 1 to n ) form a sequence of states (Z 1 → Z 2 → Z 3 → ... → Z n ) .

[0089] The in Fig. 4 The described procedure for training the machine learning model comprises, in summary, the following steps: Receiving training data, wherein the training data comprises a plurality of datasets of a plurality of reference investigation objects, wherein the datasets comprise reference representations of an investigation domain, and wherein each dataset contains one or more reference representations of a number n comprises reference representations, where each reference representation is R i the investigation area in a state Z i represented, whereby i an index that lists the numbers from 1 to n passes through, passing through the states Z 1 , ..., Z n Form a sequence of states, train the machine learning model, whereby the model is trained, starting from the reference representation R 1, successively predicted representations R 2 * to R n * to generate, wherein the generation of the predicted representation R 2 * is at least partially based on the reference representation R 1 and the generation of each subsequent predicted representation R k * at least partially based on the predicted representation R k -1* is done, whereby k an index that lists the numbers from 3 to n runs through, with the predicted representation R k * the investigation area in state Z k represented and predicted representation R k -1 * the investigation area in state Z k -1 represents, and the state Z k -l the state Z k immediately preceding.

[0090] In the case of the in Fig. 4 In the example shown, besides the first representation R 1, there are further (real, metrologically generated) representations R j (with j = 2 to n) of the investigation area in states Z j (with j = 2 to n ) which are presented as target data ( ground truth ) can be used to train the machine learning model. This allows the deviation between an output R to be determined. j * (one predicted j -th representation of the investigation area in state Z j represents) and the representation R j with an error function LF j can be quantified: an error function LF 2 can quantify the deviation between the representation R 2 and the predicted representation R 2 *, an error function LF 3 can quantify the deviation between the representation R 3 and the predicted representation R 3 *, and so on.

[0091] In end-to-end training, a total error function (LF) can be used that takes all individual deviations into account; this could, for example, be the sum of the individual deviations: LF = LF 2 + LF 3 + … + LF n = ∑ j = 2 n LF j

[0092] However, it could also be a weighted sum: LF = w 2 ⋅ LF 2 + w 3 ⋅ LF 3 + … + w n ⋅ LF n = ∑ j = 2 n w j ⋅ LF j where the weight factors w j for example, they can take values ​​from 0 to 1.

[0093] The weighting has the advantage that when generating a predicted representation R n *, which places the investigation area in a state Z n represents one or more predicted representations that represent a state prior to state Z n lies / lie, is given more or less weight than other representations. For example, it is possible that the weighting factors w i in the series w 2 , w 3 , ..., w n increase, and thus representations whose states are closer to state Z n Later states are weighted more heavily. The increase can be logarithmic, linear, quadratic, cubic, exponential, or some other type of increase. It is also conceivable to assign less weight to later states by changing the weighting factors in the series w₂, w₃, ..., w₁. n The weight decreases, thereby assigning more weight to representations whose states are closer to the initial state Z1. Such a decrease can also be logarithmic, linear, quadratic, cubic, exponential, or some other type of decrease.

[0094] It is also conceivable that the training data includes incomplete datasets. This means that some datasets of the subjects under investigation do not contain all reference representations R1 to R2. n include. As explained later in the description, incomplete datasets can also be used to train the machine learning model. If a reference representation R p If missing, the weight factor w p , which defines the error function for the reference representation R p and the predicted reference representation R p * weights should be set to zero, whereby p an integer that has the values ​​2 to n can assume.

[0095] It is also conceivable that random numbers 1 ≤ were used in the training. j < k ≤ n and the associated subsequences of states Z j , Z j +1 , ..., Z k- 1 , Z k These can be considered. The learning problem described above can then be solved on these (optionally varying from time to time) subsequences. For example, it is conceivable that a random initial state Z j (1 ≤ j ≤ n- 2) is determined and the model on the basis of which always the representations of the following two states Z j +1 , Z j+ 2 synthesize.

[0096] In Fig. 4 It is also schematically shown that to generate a predicted representation R j *, which places the investigation area in a state Z j represented, in addition to the predicted representation R j -1 *, which places the investigation area in a state Z j -1 represents, also other (predicted) representations R j -2 * and / or R j -3 * and / or ... up to R 1, which defines the scope of investigation in further preceding states Z j -2 , Z j -3 ... to Z 1 can be used to represent, where j an integer that has the values ​​2 to nThis can be assumed. This is expressed by the dashed arrows. Thus, to predict the third representation, in addition to the predicted second representation, the first representation can also be fed into the machine learning model. To predict a fourth representation, in addition to the predicted third representation, the first representation and / or the predicted second representation can also be fed into the machine learning model.

[0097] Additional information can be incorporated into the machine learning model to predict representations, such as information about the state of the domain being studied, whose representation is used to predict a representation in a subsequent state. In other words, in addition to the first representation, information about the state of the domain represented by the first representation can also be used to predict the second representation. Similarly, the predicted third representation can be generated based on the predicted second representations and information about the second state. If information about the current state of a representation is provided, the machine learning model "knows" its position within the sequence of states.

[0098] A state can be represented, for example, by a number or a vector. The states can be numbered, so that the first state is represented by the number 1, the second state by the number 2, and so on.

[0099] Instead of or in addition to information about the current state, a time that can be assigned to the respective state can also be included in the prediction of a representation. For example, the first state in the sequence of states could be assigned the time . t =0 will be assigned. The time tThe value = 0 can be fed into the machine learning model along with the first representation to predict the second representation. The second state can be assigned a number of seconds, minutes, or another unit of time difference indicating how much time elapsed between the first and second states. This time difference, along with the predicted second representation (and optionally the first representation), can be fed into the machine learning model to predict the third representation. Similarly, the third state can be assigned a time difference indicating how much time elapsed between the first and third states.This time difference, along with the predicted third representation (and optionally the first representation and / or the predicted second representation), can be fed into the machine learning model to predict the third representation. And so on.

[0100] The machine learning model according to the invention can also be understood as a transformation that can be applied to a representation of a domain of investigation in one state of a sequence of states in order to predict a representation of the domain of investigation in a subsequent state of the sequence of states. The transformation can be applied simply (once) to predict a representation of the domain of investigation in the immediately following state, or applied multiple times (multiple, iteratively) to predict a representation of the domain of investigation in a state further down the sequence of states.

[0101] Is there a consequence of n States Z1 to Z n present, and is available for every state Z i a representation R i before, which the investigation area in state Z i represented, the machine learning model M can be based on the representation R 1 q -fold applied to predict a representation R 1+ q * to generate the investigation area in the state Z 1+ q represented, whereby q the values ​​1 to n -1 can be assumed: M q R 1 = R 1 + q * q = 1 : M R 1 = R 2 * q = 2 : M R 2 * = M M R 1 = M 2 R 1 = R 3 * q = 3 : M R 3 * = M M R 2 * = M M M R 1 = M 3 R 1 = R 4 * q = n − 1 : M R n − 1 * = M M R n − 2 * = … = M n − 1 R 1 = R n *

[0102] An error function that accounts for all deviations between predicted representations of R q * and real (measurably generated) representations R q Quantified, for example, can be expressed by the following formula: LV = w 2 ⋅ d M R 1 , R 2 + w 3 ⋅ d M 2 R 1 , R 3 + … + w n ⋅ d M n − 1 R 1 , R n

[0103] Here, LV is the error value that applies to a data set encompassing the reference representations R1, R2, ..., R. n results. d is an error function that measures the deviations between a predicted representation M (Rq -1 ) and the reference representation R q quantified. As described earlier, this can be, for example, one of the following error functions: L1 loss, L2 loss, Lp loss, structural similarity index measure (SSIM), VGG loss, perceptual loss or a combination thereof.

[0104] w 2 , w 3 , ..., w n These are weight factors that have already been described earlier in the description.

[0105] n indicates the number of states.

[0106] It is also conceivable that the error value is the maximum deviation calculated for a data set: LV = max w 2 ⋅ d M R 1 , R 2 , w 3 ⋅ d M 2 R 1 , R 3 , … , w n ⋅ d M n − 1 R 1 , R n where, in this formula as well, different weights can be assigned to the individual deviations through the weighting factors.

[0107] As mentioned earlier in the description, it should be noted that for training the machine learning model, it is not necessary for each training data set to contain representations of all the states the model is to learn. This will be illustrated with an example. Suppose the machine learning model is to be trained to predict representations of a domain of study in the states of a sequence of 6 states, Z1 to Z6. Assume that training data comprising 10 data sets from 10 objects of study is sufficient for training the machine learning model. Each data set contains representations of the domain of study in different states, e.g.: Dataset 1: R1, R3, R4, R5, R6 Dataset 2: R1, R2, R4, R6 Dataset 3: R1, R2, R3, R4, R5 Dataset 4: R1, R2, R3, R3, R6 Dataset 5: R2, R3, R5, R6 Dataset 6: R2, R3, R4, R5, R6 Dataset 7: R2, R3, R5, R6 Dataset 8: R3, R5, R6 Dataset 9: R3, R4, R5, R6 Dataset 10: R3, R4, R6

[0108] In this example, there is no single "complete" dataset, i.e., no dataset that includes all possible representations R1, R2, R5, R4, R5, and R6. Nevertheless, it is possible to train the machine learning model based on such training data to predict a representation of each state. This is an advantage of the present invention over the "direct training" described above.

[0109] Once the machine learning model is trained, new (i.e., not used in training) representations of a domain of study in a state of a sequence of states can be fed into the model, and the model can predict (generate) one or more representations of the domain of study in a subsequent state or in several subsequent states of the sequence of states.

[0110] This is an example and a schematic representation in Fig. 5 depicted. In the Fig. 5 The example shown is based on a representation R p , which the investigation area is in that condition Z p represented, a sequence of representations R p +1 *, R p +2 *, R p +3 * and R p +4 * generated, which expands the investigation area to states Z p +1 , Z p +2 , Z p+3 and Z p +4 represent. The states Z p +1 , Z p+2 , Z p+3 and Z p +4 form a sequence of states: Z p +1 → Z p +2 → Z p+3 → Z p +4 .

[0111] In step (A), the machine learning model M is assigned the representation R. p supplied. In addition to the representation R p The machine learning model M can also receive information about the state Z. p and / or other information will be provided, as described in this description.

[0112] In step (B), the machine learning model M generates the predicted representation R based on the input data. p +1 *, which defines the investigation area in state Z p +1 represents.

[0113] In step (C), the previously generated representation R is added to the machine learning model M. p +1 * added. Besides the representation R p+1 * can also provide the machine learning model M with information about the state Z p +1 and / or further information can be added. Furthermore, the machine learning model can also be given the representation R. p and / or information about the condition Z p be supplied.

[0114] Preferably, the machine learning model M comprises a memory S that stores input data (and preferably also output data), so that input data and / or generated output data do not need to be received and / or entered again, but are already available to the machine learning model. This applies not only to the use of the trained machine learning model for prediction described in this example, but also to the training of the machine learning model according to the invention.

[0115] In step (D), the machine learning model M generates the predicted representation R based on the input data. p +2 *, which defines the investigation area in state Z p +2 represents.

[0116] In step (E), the previously generated representation R is added to the machine learning model M. p +2 * supplied. In addition to the predicted representation R p +2 * can also provide the machine learning model M with information about the state Z p +2 and / or further information can be added. Furthermore, the machine learning model can also be given the representation R. p and / or the predicted representation R p +1 * and / or information about the condition Z p and / or condition Z p +1 will be added.

[0117] In step (F), the machine learning model M generates the predicted representation R based on the input data. p +3 *, which expands the investigation area to the state Z p+3 represented.

[0118] In step (G), the previously generated representation R is added to the machine learning model M. p +3 * added. In addition to the predicted representation R p +3 * can also provide the machine learning model M with information about the state Z p+3 and / or further information can be added. Furthermore, the machine learning model can also be represented by R. p and / or the predicted representation R p +1 * and / or the predicted representation R p +2 * and / or information about the condition Z p and / or condition Z p +1 and / or state Z p +2 will be added.

[0119] In step (H), the machine learning model M generates the predicted representation R based on the input data. p +4 *, which defines the investigation area in state Z p +4 represents.

[0120] The generated representations R p +1 *, R p +2 *, R p+3 * and / or R p +4 * can be output (e.g. displayed on a monitor and / or printed with a printer) and / or stored in a data storage device and / or transmitted to a (separate) computer system.

[0121] The machine learning model according to the invention was trained, starting from a first representation R 1, which represents the investigation domain in a first state Z 1, a sequence of predicted representations R 2 *, ..., R . n * to generate, where each representation R j * the investigation area in a state Z j represented, whereby jIf an index is used that cycles through the numbers from 2 to n, then such a model can be given a new representation R. p are supplied, which put the investigation area in state Z p represented, and the machine learning model can predict the representations R p +1 *, R p +2 *, ... , R n * generate, whereby p a number that has the values ​​2 to n can assume.

[0122] This means that the trained model does not necessarily need to be provided with a new representation R 1 that represents the domain of investigation in the first state Z 1, and the trained machine learning model is also not limited to only the representation R n *, which defines the investigation area in its last state Z n represented, predicting. Instead, one can use the sequence of states Z1 to Z2. n "Enter" at any point and predict a representation of any other state based on the representation fed into the model.

[0123] It is even possible to create representations that represent states that were not the subject of the training at all. So instead of using the representation R n * to stop, the representations R can also n +1 *, R n +2 * and so on can be predicted. The trained machine learning model can therefore be used to continue the learned dynamics and to compute representations of states that have never been metrologically generated. In this respect, the trained machine learning model can be used to extrapolate to new states.

[0124] Furthermore, the predictions are not limited to subsequent states. It is also possible to predict representations of the domain under investigation in a previous state of the sequence of states. Firstly, as described earlier, the machine learning model can, in principle, be trained in both directions: towards subsequent states and towards previous states. Secondly, the machine learning model performs a transformation on an input representation, which can, in principle, also be reversed. By analyzing the mathematical functions of the model that transform an input representation into an output representation, inverse functions can be determined that reverse the process and transform the previous output representation back into the previous input representation.The inverse functions can then be used to predict representations of previous states, even if the model has been trained to predict representations of subsequent states, and vice versa.

[0125] Fig. 6 shows an extension of the in Fig. 5 shown scheme. While in Fig. 5 based on a representation R p , which defines the investigation area in state Z p represented, one after the other, the predicted representations R p +1 *, R p +2 *, R p +3 * and R p +4 * are generated, which define the investigation area in the states Z p +1 , Z p +2 , Z p+ 3 and Z p +4 represent is in Fig. 6 shown how, based on the representation R p , which defines the investigation area in state Z p represented, one after the other, the predicted representations R p +1 * to R p+q * are generated, whereby q an integer greater than 1. In Fig. 6 The iterative nature of the machine learning model M becomes particularly evident. The machine learning model M is developed starting from the representation R. p q applied -times, whereby ( q -1)-times the output data is fed back into the machine model.

[0126] It should be noted that the (trained or untrained) machine learning model does not have to be applied to a complete radiological image (e.g., an MRI scan, a CT scan, or the like). It is possible to apply the machine learning model to only a portion of a radiological image. For example, it is possible to first segment a radiological image to identify an area of ​​interest. region of interest ) to identify / select. The model can then be applied exclusively to the area of ​​interest, for example.

[0127] Furthermore, the application of the machine learning model can include one or more preprocessing and / or postprocessing steps. For example, it is conceivable to first subject a received representation of the domain of investigation to one or more transformations, such as motion correction, color space conversion, normalization, segmentation, and / or the like. In a further step, the transformed representation can be fed into the machine learning model, which then undergoes a series of iterations (cycles) (as in Fig. 6 (schematically represented), in order to generate a series of further (subsequent) representations of the domain of investigation in a series of further (subsequent) states, starting from the transformed representation.

[0128] The machine learning model according to the invention may, for example, be an artificial neural network, or it may comprise one.

[0129] An artificial neural network comprises at least three layers of processing elements: a first layer with input neurons (nodes), an N-th layer with at least one output neuron (node), and N-2 inner layers, where N is a natural number and greater than 2.

[0130] The input neurons are used to receive the representations. Typically, there is one input neuron for each pixel or voxel of a representation if the representation is a spatial representation in the form of a raster graphic. Additional input neurons may be present for additional input values ​​(e.g., information about the area under investigation, the object under investigation, the conditions that prevailed when the representation was generated, information about the state that the representation represents, and / or information about the time or time period in which the representation was generated).

[0131] The output neurons can be used to output a predicted representation that represents the area under investigation in a subsequent state.

[0132] The processing elements of the layers between the input neurons and the output neurons are interconnected in a predetermined pattern with predetermined connection weights.

[0133] Preferably, the artificial neural network is a so-called Convolutional Neural Network (CNN) or it includes one.

[0134] A CNN typically consists essentially of filters (convolutional layer) and aggregation layers (pooling layer) that alternate and repeat, and ultimately of one or more layers of "normal" fully connected neurons (dense / fully connected layer).

[0135] Training a neural network can be performed, for example, using a backpropagation method. This involves the network making the most reliable possible prediction of the dynamics of the domain under investigation, from a given state through at least one intermediate state to a final state. The quality of the prediction is described by an error function. The goal is to minimize this error function. In the backpropagation method, the artificial neural network is trained by changing the connection weights.

[0136] In the trained state, the connection weights between the processing elements contain information regarding the dynamics of state changes, which can be used to predict, based on an initial representation representing the domain of investigation in a first state, one or more representations representing the domain of investigation in one or more subsequent states.

[0137] A cross-validation method can be used to split the data into training and validation datasets. The training dataset is used for backpropagation training of the network weights. The validation dataset is used to verify the prediction accuracy of the trained network when applied to unknown data.

[0138] The artificial neural network can have an autoencoder architecture; for example, the artificial neural network can have an architecture like the U-Net (see, for example, O. Ronneberger et al.: U-net: Convolutional networks for biomedical image segmentation, International Conference on Medical image computing and computer-assisted intervention, pages 234-241, Springer, 2015, https: / / doi.org / 10.1007 / 978-3-319-24574-4_28).

[0139] The artificial neural network can be a Generative Adversarial Network (GAN).

[0140] The artificial neural network can be a recurrent neural network or comprise one. Recurrent or feedback neural networks are neural networks that, unlike feedforward networks, are characterized by connections between neurons of one layer and neurons of the same or a preceding layer. The artificial neural network can, for example, be a Long short-term memory (LSTM) (see e.g. Y. Gao et al.: Fully convolutional structured LSTM networks for joint 4D medical image segmentation, DOI: 10.1109 / ISBI.2018.8363764).

[0141] The artificial neural network can be a transformer network (see e.g. D. Karimi et al.: Convolution-Free Medical Image Segmentation using Transformers, arXiv:2102.13645 [eess.IV]).

[0142] Fig. 7 shows an exemplary and schematic computer system according to the invention.

[0143] A "computer system" is a system for electronic data processing that processes data using programmable instructions. Such a system typically comprises a "computer," the unit containing a processor for performing logical operations, as well as peripherals.

[0144] In computer technology, "peripherals" refers to all devices connected to a computer that are used to control the computer and / or as input and output devices. Examples include monitors (screens), printers, scanners, mice, keyboards, drives, cameras, microphones, speakers, etc. Internal ports and expansion cards are also considered peripherals in computer technology.

[0145] The in Fig. 7 The computer system shown (1) comprises an input unit (10), a control and computing unit (20) and an output unit (30).

[0146] The control and computing unit (20) serves to control the computer system (1), to coordinate the data flows between the units of the computer system (1) and to perform calculations.

[0147] The control and computing unit (20) is configured to receive a representation via the input unit (10) that defines an investigation area in a first state Z 1 of a sequence of states Z 1 to Z . n represented, whereby n an integer greater than two, to feed the representation to a trained machine learning model, where the machine learning model has been trained on the basis of training data, starting from a first reference representation R 1 a number n -1 of reference representations R 2 * to R n to predict, where the first reference representation R 1 represents the investigation domain in state Z 1 and each predicted reference representation R j * the investigation area in state Z j represented, whereby j an index that represents the numbers from 2 to n runs through, whereby the generation of the predicted reference representation R 2 * is at least partially based on the reference representation R 1, and the generation of each further predicted reference representation R k * at least partially based on the predicted reference representation R k -1 * is done, whereby k an index that lists the numbers from 3 to nby performing a process to receive one or more predicted representations of the domain of investigation from the machine learning model, wherein each of the one or more predicted representations represents the domain of investigation in a state following the first state Z 1, and to cause the output unit (30) to output and / or store and / or transmit the one or more predicted representations to a separate computer system.

[0148] Fig. 8 shows, by way of example and schematically, another embodiment of the computer system according to the invention.

[0149] The computer system (1) comprises a processing unit (21) connected to a memory (22). The processing unit (21) and the memory (22) form a control and calculation unit, as described in Fig. 7 shown.

[0150] The processing unit (21) (English: processing unitThe processing unit (21) may comprise one or more processors alone or in combination with one or more memories. The processing unit (21) may be ordinary computer hardware capable of processing information such as digital images, computer programs, and / or other digital information. The processing unit (21) typically consists of an arrangement of electronic circuits, some of which may be implemented as an integrated circuit or as several interconnected integrated circuits (an integrated circuit is sometimes referred to as a "chip"). The processing unit (21) may be configured to execute computer programs, which may be stored in a working memory of the processing unit (21) or in the memory (22) of the same or another computer system.

[0151] The memory (22) can be ordinary computer hardware capable of storing information such as digital images (e.g., representations of the study area), data, computer programs, and / or other digital information, either temporarily and / or permanently. The memory (22) can be volatile and / or non-volatile and can be permanently installed or removable. Examples of suitable memory include RAM (Random Access Memory), ROM (Read-Only Memory), a hard disk, flash memory, a removable computer disk, an optical disc, a magnetic tape, or a combination of the above. Optical discs can include read-only compact discs (CD-ROM), read / write compact discs (CD-R / W), DVDs, Blu-ray discs, and similar media.

[0152] In addition to the memory (22), the processing unit (21) can also be connected to one or more interfaces (11, 12, 31, 32, 33) to display, transmit, and / or receive information. The interfaces can include one or more communication interfaces (32, 33) and / or one or more user interfaces (11, 12, 31). The one or more communication interfaces can be configured to send and / or receive information, for example, to and / or from an MRI scanner, a CT scanner, an ultrasound camera, other computer systems, networks, data storage devices, or the like. The one or more communication interfaces can be configured to transmit and / or receive information via physical (wired) and / or wireless communication links.The one or more communication interfaces may include one or more interfaces for connecting to a network, e.g., using technologies such as cellular, Wi-Fi, satellite, cable, DSL, fiber optic, and / or the like. In some examples, the one or more communication interfaces may include one or more near-field communication interfaces configured to connect devices using near-field communication technologies such as NFC, RFID, Bluetooth, Bluetooth LE, ZigBee, infrared (e.g., IrDA), or similar technologies.

[0153] The user interfaces may include a display (31). A display (31) may be configured to show information to a user. Suitable examples include a liquid crystal display (LCD), a light-emitting diode (LED) display, a plasma display (PDP), or the like. The user input interface(s) (11, 12) may be wired or wireless and may be configured to receive information from a user into the computer system (1), for example, for processing, storage, and / or display. Suitable examples of user input interfaces include a microphone, an image or video recording device (e.g., a camera), a keyboard or keypad, a joystick, a touch-sensitive surface (separate from or integrated into a touchscreen), or the like.In some examples, the user interfaces may include automatic identification and data capture (AIDC) technology for machine-readable information. This could include barcodes, radio frequency identification (RFID), magnetic stripes, optical character recognition (OCR), integrated circuit cards (ICC), and similar technologies. The user interfaces may also include one or more interfaces for communication with peripheral devices such as printers and the like.

[0154] One or more computer programs (40) can be stored in memory (22) and executed by the processing unit (21), which is programmed to perform the functions described in this description. The retrieval, loading, and execution of instructions from the computer program (40) can be sequential, with one instruction being retrieved, loaded, and executed at a time. However, the retrieval, loading, and / or execution can also be performed in parallel.

[0155] The memory (22) can also store the machine learning model according to the invention.

[0156] The system according to the invention can be designed as a laptop, notebook, netbook and / or tablet PC; it can also be a component of an MRI scanner, a CT scanner or an ultrasound diagnostic device.

Claims

1. Computer-implemented method comprising the steps of: - receiving training data, where the training data comprise a multiplicity of data sets of a multiplicity of reference examination objects, where the data sets comprise reference representations of an examination region, where the examination region is the liver or part of the liver of a human, where the reference representations are radiological images, where each data set comprises one or more reference representations of a number n of reference representations, where n is an integer greater than 2, where each reference representation Ri represents the examination region in one state Zi, where i is an index passing through the numbers from 1 to n, where the states Z1, ..., Zn form a sequence of states, - training the machine-learning model, ∘ where the model is trained to generate, starting from the reference representation R1, predicted representations R2* to Rn* one after the other, ∘ where the predicted representation R2* is generated at least partly on the basis of the reference representation R1 and each subsequent predicted representation Rk* is generated at least partly on the basis of the predicted representation Rk-1*, where k is an index passing through the numbers from 3 to n, ∘ where the predicted representation R2* is generated by inputting the reference representation R1 into the model during training, and each subsequent predicted representation Rk* is generated by inputting the predicted representation Rk-1* into the model during training, ∘ where the predicted representation Rk* represents the examination region in the state Zk and the predicted representation Rk-1* represents the examination region in the state Zk-1, and the state 2k-1 directly precedes the state Zk, where training the machine-learning model further comprises the steps of: - for each predicted representation Rj*: calculating a loss value for each pair composed of a reference representation Rj and the predicted representation Rj* with the aid of a loss function, where j is an index passing through the numbers from 2 to n, - calculating a total loss value with the aid of a total loss function, where the total loss function is a function of the loss values in which the loss values are weighted with weight factors, - modifying parameters of the machine-learning model, such that the total loss value is reduced to a defined minimum, - storing the trained machine-learning model and / or utilizing the machine-learning model for prediction, where the sequence of states comprises one or more of the following states: - the liver or part of the liver before the administration of a hepatobiliary contrast agent, - the liver or part of the liver during the arterial phase after the administration of the hepatobiliary contrast agent, - the liver or part of the liver during the portal venous phase after the administration of the hepatobiliary contrast agent, - the liver or part of the liver during the transitional phase after the administration of the hepatobiliary contrast agent, - the liver or part of the liver during the hepatobiliary phase after the administration of the hepatobiliary contrast agent.

2. Method according to Claim 1, further comprising the steps of: • receiving a representation Rp of the examination region, where the representation Rp represents the examination region in one state Zp of the sequence of states Z1 to Zn, where p is an integer less than n, • feeding the representation Rp to the trained machine-learning model, • receiving from the machine-learning model one or more predicted representations Rp+q* of the examination region, where each of the one or more predicted representations Rp+q* represents the examination region in one state Zp+q*, where q is an index passing through the numbers from 1 to m, where m is an integer less than n-1 or equal to n-1 or greater than n-1, • outputting and / or storing and / or transmitting the one or more predicted representations Rp+q*.

3. Method according to Claim 2, wherein each predicted representation Rp+q* generated by the machine-learning model is fed back to the trained machine-learning model in order to generate a predicted representation Rp+q+1*.

4. Method according to Claims 3, wherein m is in the range from n to n+2.

5. Method according to any of Claims 1 to 4, wherein the reference representations and the predicted representations and the representation Rp and the predicted representations Rp+q* and Rp+q+1* are CT images, MRI images or ultrasound images.

6. Method according to Claim 1, wherein the total loss function has the following formula: LV = ∑ j = 2 n w j ⋅ LF j where LV is the total loss value, where LFj are the loss functions for calculating the loss values for the differences between the reference representation Rj and the predicted representation Rj*, and wj are weight factors, where j is an index passing through the numbers from 2 to n.

7. Method according to any of Claims 1 to 6, wherein each predicted representation Rp+q* is calculated according to the following formula: M q R 1 = R 1 + q * where M represents the machine-learning model, where Mq means the q-times application of the machine-learning model, and q is an integer which can assume the values of 1 to n-1.

8. Method according to any of Claims 1 to 7, wherein the sequence of states comprises one or more of the following states: - a first state of the examination region at a first time point before the administration of a contrast agent, - a second state of the examination region at a second time point, after the administration of the contrast agent, - a third state of the examination region at a third time point, after the administration of the contrast agent, - a fourth state of the examination region at a fourth time point, after the administration of the contrast agent, - a fifth state of the examination region at a fifth time point, after the administration of the contrast agent, where the first time point, the second time point, the third time point, the fourth time point and the fifth time point form a chronological sequence.

9. Computer system comprising • an input unit, • a control and calculation unit and • an output unit, wherein the control and calculation unit is configured • to receive a representation Rp of an examination region, where the representation Rp represents the examination region in one state Zp of a sequence of states Z1 to Zn, where p is an integer less than n, where n is an integer greater than two, where the examination region is the liver or part of the liver of a human, where the representation Rp is a radiological image, • to feed the representation Rp to a trained machine-learning model, ∘ where the machine-learning model has been trained on the basis of training data to generate, starting from a first reference representation R1, a number n-1 of predicted representations R2* to Rn*, ∘ where the first reference representation R1 represents the examination region in a first state Z1 and each predicted representation Rj* represents the examination region in the state Zj, where j is an index passing through the numbers from 2 to n, ∘ where the predicted representation R2* is generated at least partly on the basis of the reference representation R1 and each further predicted representation Rk* is generated at least partly on the basis of the predicted representation Rk-1*, where k is an index passing through the numbers from 3 to n, ∘ where the predicted representation R2* is generated by inputting the reference representation R1 into the model during training, and each subsequent predicted representation Rk* is generated by inputting the predicted representation Rk-1* into the model during training, o where training the machine-learning model further comprises the steps of: - for each predicted representation Rj*: calculating a loss value for each pair composed of a reference representation Rj and the predicted representation Rj* with the aid of a loss function, where j is an index passing through the numbers from 2 to n, - calculating a total loss value with the aid of a total loss function, where the total loss function is a function of the loss values in which the loss values are weighted with weight factors, - modifying parameters of the machine-learning model, such that the total loss value is reduced to a defined minimum, • to receive from the machine-learning model one or more predicted representations Rp+q* of the examination region, where each of the one or more predicted representations Rp+q* represents the examination region in one state Zp+q*, where q is an index passing through the numbers from 1 to m, where m is an integer less than n-1 or equal to n-1 or greater than n-1, to output the one or more predicted representations Rp+q* and / or to store them and / or to transmit them to a separate computer system, where the sequence of states comprises one or more of the following states: - the liver or part of the liver before the administration of a hepatobiliary contrast agent, - the liver or part of the liver during the arterial phase after the administration of the hepatobiliary contrast agent, - the liver or part of the liver during the portal venous phase after the administration of the hepatobiliary contrast agent, - the liver or part of the liver during the transitional phase after the administration of the hepatobiliary contrast agent, - the liver or part of the liver during the hepatobiliary phase after the administration of the hepatobiliary contrast agent.

10. Computer program product comprising a computer program which can be loaded into a working memory of a computer system, where it causes the computer system to execute the following steps: • receiving a representation Rp of an examination region, where the representation Rp represents the examination region in one state Zp of a sequence of states Z1 to Zn, where p is an integer less than n, where n is an integer greater than two, where the examination region is the liver or part of the liver of a human, where the representation Rp is a radiological image, • feeding the representation Rp to a trained machine-learning model, ∘ where the machine-learning model has been trained on the basis of training data to generate, starting from a first reference representation R1, a number n-1 of predicted representations R2* to Rn*, ∘ where the first reference representation R1 represents the examination region in a first state Z1 and each predicted representation Rj* represents the examination region in the state Zj, where j is an index passing through the numbers from 2 to n, ∘ where the predicted representation R2* is generated at least partly on the basis of the reference representation R1 and each further predicted representation Rk* is generated at least partly on the basis of the predicted representation Rk-1*, where k is an index passing through the numbers from 3 to n, ∘ where the predicted representation R2* is generated by inputting the reference representation R1 into the model during training, and each subsequent predicted representation Rk* is generated by inputting the predicted representation Rk-1* into the model during training, o where training the machine-learning model further comprises the steps of: - for each predicted representation Rj*: calculating a loss value for each pair composed of a reference representation Rj and the predicted representation Rj* with the aid of a loss function, where j is an index passing through the numbers from 2 to n, - calculating a total loss value with the aid of a total loss function, where the total loss function is a function of the loss values in which the loss values are weighted with weight factors, - modifying parameters of the machine-learning model, such that the total loss value is reduced to a defined minimum, • receiving from the machine-learning model one or more predicted representations Rp+q* of the examination region, where each of the one or more predicted representations Rp+q* represents the examination region in one state Zp+q*, where q is an index passing through the numbers from 1 to m, where m is an integer less than n-1 or equal to n-1 or greater than n-1, • outputting and / or storing and / or transmitting the one or more predicted representations Rp+q*, where the sequence of states comprises one or more of the following states: • the liver or part of the liver before the administration of a hepatobiliary contrast agent, • the liver or part of the liver during the arterial phase after the administration of the hepatobiliary contrast agent, • the liver or part of the liver during the portal venous phase after the administration of the hepatobiliary contrast agent, • the liver or part of the liver during the transitional phase after the administration of the hepatobiliary contrast agent, • the liver or part of the liver during the hepatobiliary phase after the administration of the hepatobiliary contrast agent.

11. Use of a contrast agent in a radiological examination method, wherein the radiological examination method comprises the following steps: • receiving a representation Rp of an examination region, where the representation Rp represents the examination region before or after administration of the contrast agent in one state Zp of a sequence of states Z1 to Zn, where p is an integer less than n, where n is an integer greater than two, where the examination region is the liver or part of the liver of a human, where the representation Rp is a radiological image, • feeding the representation Rp to a trained machine-learning model, ∘ where the machine-learning model has been trained on the basis of training data to generate, starting from a first reference representation R1, a number n-1 of predicted representations R2* to Rn*, ∘ where the first reference representation R1 represents the examination region in a first state Z1 and each predicted representation Rj* represents the examination region in the state Zj, where j is an index passing through the numbers from 2 to n, ∘ where the predicted representation R2* is generated at least partly on the basis of the reference representation R1 and each further predicted representation Rk* is generated at least partly on the basis of the predicted representation Rk-1*, where k is an index passing through the numbers from 3 to n, ∘ where the predicted representation R2* is generated by inputting the reference representation R1 into the model during training, and each subsequent predicted representation Rk* is generated by inputting the predicted representation Rk-1* into the model during training, o where training the machine-learning model further comprises the steps of: - for each predicted representation Rj*: calculating a loss value for each pair composed of a reference representation Rj and the predicted representation Rj* with the aid of a loss function, where j is an index passing through the numbers from 2 to n, - calculating a total loss value with the aid of a total loss function, where the total loss function is a function of the loss values in which the loss values are weighted with weight factors, - modifying parameters of the machine-learning model, such that the total loss value is reduced to a defined minimum, • receiving from the machine-learning model one or more predicted representations Rp+q* of the examination region, where each of the one or more predicted representations Rp+q* represents the examination region in one state Zp+q*, where q is an index passing through the numbers from 1 to m, where m is an integer less than n-1 or equal to n-1 or greater than n-1, • outputting and / or storing and / or transmitting the one or more predicted representations Rp+q*, where the sequence of states comprises one or more of the following states: • the liver or part of the liver before the administration of a hepatobiliary contrast agent, • the liver or part of the liver during the arterial phase after the administration of the hepatobiliary contrast agent, • the liver or part of the liver during the portal venous phase after the administration of the hepatobiliary contrast agent, • the liver or part of the liver during the transitional phase after the administration of the hepatobiliary contrast agent, • the liver or part of the liver during the hepatobiliary phase after the administration of the hepatobiliary contrast agent.

12. Kit comprising a contrast agent and a computer program product, wherein the computer program product comprises a computer program that can be loaded into a working memory of a computer system, where it causes the computer system to execute the following steps: • receiving a representation Rp of an examination region, where the representation Rp represents the examination region before or after administration of the contrast agent in one state Zp of a sequence of states Z1 to Zn, where p is an integer less than n, where n is an integer greater than two, where the examination region is the liver or part of the liver of a human, where the representation Rp is a radiological image, • feeding the representation Rp to a trained machine-learning model, ∘ where the machine-learning model has been trained on the basis of training data to generate, starting from a first reference representation R1, a number n-1 of predicted representations R2* to Rn*, ∘ where the first reference representation R1 represents the examination region in a first state Z1 and each predicted representation Rj* represents the examination region in the state Zj, where j is an index passing through the numbers from 2 to n, ∘ where the predicted representation R2* is generated at least partly on the basis of the reference representation R1 and each further predicted representation Rk* is generated at least partly on the basis of the predicted representation Rk-1*, where k is an index passing through the numbers from 3 to n, ∘ where the predicted representation R2* is generated by inputting the reference representation R1 into the model during training, and each subsequent predicted representation Rk* is generated by inputting the predicted representation Rk-1* into the model during training, o where training the machine-learning model further comprises the steps of: - for each predicted representation Rj*: calculating a loss value for each pair composed of a reference representation Rj and the predicted representation Rj* with the aid of a loss function, where j is an index passing through the numbers from 2 to n, - calculating a total loss value with the aid of a total loss function, where the total loss function is a function of the loss values in which the loss values are weighted with weight factors, - modifying parameters of the machine-learning model, such that the total loss value is reduced to a defined minimum, • receiving from the machine-learning model one or more predicted representations Rp+q* of the examination region, where each of the one or more predicted representations Rp+q* represents the examination region in one state Zp+q*, where q is an index passing through the numbers from 1 to m, where m is an integer less than n-1 or equal to n-1 or greater than n-1, • outputting and / or storing and / or transmitting the one or more predicted representations Rp+q*, where the sequence of states comprises one or more of the following states: • the liver or part of the liver before the administration of a hepatobiliary contrast agent, • the liver or part of the liver during the arterial phase after the administration of the hepatobiliary contrast agent, • the liver or part of the liver during the portal venous phase after the administration of the hepatobiliary contrast agent, • the liver or part of the liver during the transitional phase after the administration of the hepatobiliary contrast agent, • the liver or part of the liver during the hepatobiliary phase after the administration of the hepatobiliary contrast agent.