Prediction of the representation of the inspection area in the inspection target in one state of a sequence consisting of multiple states.

Abstract

The present invention relates to the technical field of radiology, in particular to the field of assisting radiologists performing radiological examinations by using artificial intelligence methods. The present invention also relates to the training of a machine learning model and the use of the trained model for predicting a representation for an examination region in one or more states of a sequence of states in a radiological examination.

Description

[Technical Field]

[0001] The present invention relates to the technical field of radiology, and more particularly to the technical field of assisting radiologists performing radiological examinations by using artificial intelligence methods. The present invention relates to training a machine learning model and using a trained model in relation to predicting a representation of an examination area in one or more states of a sequence of multiple states in a radiological examination. [Background technology]

[0002] Using imaging techniques to track processes within the human or animal body over time plays a crucial role, particularly in the diagnosis and / or treatment of diseases.

[0003] One example that can be mentioned is the detection and differential diagnosis of localized liver lesions using dynamic contrast-enhanced magnetic resonance imaging (MRI) with hepatobiliary contrast agents.

[0004] Hepatobiliary contrast agents such as Primovist (registered trademark) can be used to detect tumors within the liver. While blood is supplied to healthy liver tissue primarily via the hepatic portal vein, most primary tumors are supplied by the hepatic artery. Therefore, after a bolus of contrast agent is injected intravenously, a time delay can be observed between the increased signal in healthy liver parenchyma and the increased signal in tumors.

[0005] Besides malignant tumors, benign lesions such as cysts, hemangiomas, and focal nodular hyperplasia (FNH) are commonly found in the liver. To develop an appropriate treatment plan, it is necessary to distinguish these from malignant tumors. Primovist® can be used to differentiate between benign and malignant focal liver lesions. T1-weighted MRI provides information about the characteristics of the lesions. Differentiation is achieved by utilizing the difference in blood supply to the liver and the tumor, and by utilizing the contrast-enhanced time profile.

[0006] If contrast enhancement is obtained with Primovist® during the wash-in phase, what is observed is a typical perfusion pattern that provides information for characterizing the lesion. Visualizing vascularization helps to characterize the type of lesion and determine the spatial relationship between the tumor and blood vessels.

[0007] In T1-weighted MRI images, Primovist® clearly enhances the signal in healthy liver parenchyma 10 to 20 minutes after injection (hepatocyte contrast phase), while lesions that do not contain hepatocytes or contain only a few hepatocytes, such as metastatic or moderately to poorly differentiated hepatocellular carcinomas (HCCs), appear as darker areas.

[0008] Therefore, while tracking the diffusion of contrast agent over time provides an excellent method for detecting and differentiating localized liver lesions, the examination requires a relatively long duration. During this period, patient movement should be minimized to reduce motion artifacts in the MRI images. Being restricted from moving for extended periods can be uncomfortable for the patient.

[0009] International Publication No. 2021 / 052896 proposes that, in order to reduce the time patients spend in the MRI scanner, one or more MRI images during the hepatocyte contrast-enhanced phase should be calculated (predicted) based on MRI images from one or more preceding phases, rather than being generated by measurement.

[0010] The approach described in the previous international publication 2021 / 052896 involves training a machine learning model to use MRI images of the examination area before and / or immediately after contrast agent administration as a basis for predicting MRI images of the examination area at a later time. Thus, the model is trained to map multiple MRI images as input data onto an MRI image as output data. [Overview of the project]

[0011] Building upon the prior art described above, an object of the present invention is to improve the predictive quality of machine learning models so that they can be used in a variety of embodiments, and / or to create a model that learns the dynamics of contrast agent diffusion in an examination area.

[0012] This objective is achieved by the subject matter of the independent claim. Preferred embodiments may be found in the dependent claims, as well as in this specification and the drawings.

[0013] In a first view, the present invention provides a computer implementation method for training a machine learning model. The training method is: - This involves receiving training data. • Training data represents representations of the test area across a large number of test subjects. T R1~ T R n This includes, where n is an integer greater than or equal to 3, ·Each expression T R i This refers to multiple states Z1~Z n One state Z of a sequence consisting of irepresents the inspection area at, where i is an index that passes through integers from 1 to n, and - training a machine learning model, · The model is represented by T Starting from R1, for each inspection target among a number of inspection targets, the representation T R2 * ~ T R n * is trained to be generated one after another, and the representation T R2 * is generated at least partially based on the representation T R1, and each further representation T R k * is generated at least partially based on the previously generated corresponding representation T R k-1 * where k is an index that passes through numbers from 3 to n, and the representation T R k * represents the inspection area in state Z k and the representation T R k-1 * represents the inspection area in state Z k-1 and state Z k-1 is directly prior to state Z k · The difference between the generated representation T R j * and the representation T j R is quantified by a loss function, where j is an index that passes through numbers from 2 to n, · The difference is minimized by an optimization method by changing the model parameters of the machine learning model, - storing the trained machine learning model and / or using the machine learning model for prediction.

[0014] The present invention further provides a computer implementation method for predicting one or more representations relating to an inspection area in an object under inspection. The prediction method is: - Expressions related to the testing area in the subject of testing R p It means receiving, ·Expression R p This refers to multiple states Z1~Z n One state Z of a sequence consisting of p This represents the inspection area. Here, p is an integer smaller than n. Here, n is an integer greater than or equal to 3, -Expression R p This involves supplying it to a pre-trained machine learning model. • A trained machine learning model, based on the training data, creates a first representation T Starting from R1, n-1 representations T R2 * ~ T R n * It is trained to generate them one after another, ·1st expression T R1 represents the examination region in the first state Z1, and each generated representation T R j * State Z j This represents the examination region, where j is an index that passes through numbers from 2 to n. ·Expression T R2 * This is, at least in part, an expression. T Generated based on R1, and further each representation T R k * These are, at least in part, corresponding to previously generated expressions. T R k-1 * It is generated based on the following, where k is an index that passes through numbers from 3 to n, - From the machine learning model, one or more representations of the inspection domain R p+q * It means receiving, • One or more expressions R p+q * Each of these is in state Z p+q This represents the test region, where q is an index passing through numbers from 1 to m, and m is an integer less than n-1, an integer equal to n-1, or an integer greater than n-1. -1 or more expressions R p+q * This includes outputting, and / or storing, and / or transmitting.

[0015] The present invention further relates to a computer system, • Input unit and, • Control calculation unit, • Provides a computer system including an output unit, The control calculation unit is, - The input unit contains an expression R relating to the inspection area of ​​the object being inspected. p It is configured to receive, where, expression R p This refers to multiple states Z1~Z n One state Z of a sequence consisting of p This represents the test region, where p is an integer less than n, and n is an integer greater than or equal to 3. -Expression R p It is configured to supply the first representation to a trained machine learning model, where the trained machine learning model uses the training data to generate the first representation T Starting from R1, n-1 representations T R2 * ~ T R n * It is trained to generate them one after another, and here, the first expression T R1 represents the examination region in the first state Z1, and each generated representation T R j * State Z j This represents the examination region, where j is an index that passes through numbers from 2 to n, and the representation T R2* is, at least in part, generated based on the expression T R1, and each further expression T R k * is, at least in part, generated based on the previously generated corresponding expression T R k-1 * respectively, where k is an index passing through the numbers from 3 to n, - configured to receive from a machine learning model one or more expressions R p+q * relating to the inspection area, where each of the one or more expressions R p+q * represents the inspection area in the state Z p+q where q is an index passing through the numbers from 1 to m, and m is an integer less than n - 1, or equal to n - 1, or greater than n - 1, - configured to output one or more expressions R p+q * to an output unit and / or store them and / or transmit them to another computer system.

[0016] The present invention further provides a computer program product including a data memory storing a computer program that can be loaded into the working memory of a computer system, the computer program causing the computer system to - receive an expression R p relating to the inspection area in the object to be inspected, where the expression R p represents the inspection area in one state Z n from a sequence consisting of a plurality of states Z1 to Z p where p is an integer less than n and n is an integer greater than or equal to 3, and - supply the expression R <000,0106>to a trained machine learning model, where the machine learning model, based on training data, a first expressionT Starting from R1, n-1 representations T R2 * ~ T R n * It is trained to generate them one after another, and here, the first expression T R1 represents the examination region in the first state Z1, and each generated representation T R j * State Z j This represents the examination region, where j is an index that passes through numbers from 2 to n, and the representation T R2 * This is, at least in part, an expression. T Generated based on R1, and further each representation T R k * These are, at least in part, corresponding to previously generated expressions. T R k-1 * It is generated based on the following, where k is the step, which is the index that passes through the numbers from 3 to n, - From the machine learning model, one or more representations of the inspection domain R p+q * This is a step in which one or more expressions R are received. p+q * Each of these is in state Z p+q This represents the examination region, where q is the index passing through numbers from 1 to m, and m is an integer less than n-1, an integer equal to n-1, or an integer greater than n-1, and -1 or more expressions R p+q * The process involves the steps of outputting, and / or storing, and / or transmitting the data.

[0017] The present invention further provides the use of contrast agents in radiological examinations, and radiological examinations are, - Expressions related to the testing area in the subject of testing R p This is the step of receiving the expression R pThis refers to multiple states Z1~Z n One state Z of a sequence consisting of p This represents the examination area before or after the administration of the contrast agent, where p is an integer less than n and n is an integer greater than or equal to 3. -Expression R p This is a step of supplying the first representation to a trained machine learning model, where the machine learning model uses the training data to determine the first representation T Starting from R1, n-1 representations T R2 * ~ T R n * It is trained to generate them one after another, and here, the first expression T R1 represents the examination region in the first state Z1, and each generated representation T R j * State Z j This represents the examination region, where j is an index that passes through numbers from 2 to n, and the representation T R2 * This is, at least in part, an expression. T Generated based on R1, and further each representation T R k * These are, at least in part, corresponding to previously generated expressions. T R k-1 * It is generated based on the following, where k is the step, which is the index that passes through the numbers from 3 to n, - From the machine learning model, one or more representations of the inspection domain R p+q * This is the step of receiving, where one or more expressions R p+q * Each of these is in state Z p+q This represents the examination region, where q is the index passing through numbers from 1 to m, and m is an integer less than n-1, an integer equal to n-1, or an integer greater than n-1, and -1 or more expressions R p+q* This includes the steps of outputting, and / or storing, and / or transmitting, the data.

[0018] The present invention further provides a contrast agent for use in radiological examinations, and radiological examinations are, - Expressions related to the testing area in the subject of testing R p This is the step of generating the representation R p This refers to multiple states Z1~Z n One state Z of a sequence consisting of p This represents the examination area before or after the administration of the contrast agent, where p is an integer less than n and n is an integer greater than or equal to 3. -Expression R p This is a step of supplying the first representation to a trained machine learning model, where the machine learning model uses the training data to determine the first representation T Starting from R1, n-1 representations T R2 * ~ T R n * It is trained to generate them one after another, and here, the first expression T R1 represents the examination region in the first state Z1, and each generated representation T R j * State Z j This represents the examination region, where j is an index that passes through numbers from 2 to n, and the representation T R2 * This is, at least in part, an expression. T Generated based on R1, and further each representation T R k * These are, at least in part, corresponding to previously generated expressions. T R k-1 * It is generated based on the following, where k is the step, which is the index that passes through the numbers from 3 to n, - From the machine learning model, one or more representations of the inspection domain R p+q* This is the step of receiving, where one or more expressions R p+q * Each of these is in state Z p+q This represents the examination region, where q is the index passing through numbers from 1 to m, and m is an integer less than n-1, an integer equal to n-1, or an integer greater than n-1, and -1 or more expressions R p+q * This includes the steps of outputting, and / or storing, and / or transmitting, the data.

[0019] The present invention further provides a kit comprising a contrast agent and a computer program product, wherein the computer program product includes a computer program that can be loaded into the working memory of a computer system, and the computer program is installed on the computer system. - Expressions related to the testing area in the subject of testing R p This is the step of receiving the expression R p This refers to multiple states Z1~Z n One state Z of a sequence consisting of p This represents the examination area before or after the administration of the contrast agent, where p is an integer less than n and n is an integer greater than or equal to 3. -Expression R p This is a step in which the machine learning model supplies the first representation based on the training data. T Starting from R1, n-1 representations T R2 * ~ T R n * It is trained to generate them one after another, and here, the first expression T R1 represents the examination region in the first state Z1, and each generated representation T R j * State Z jThis represents the examination region, where j is an index that passes through numbers from 2 to n, and the representation T R2 * This is, at least in part, an expression. T Generated based on R1, and further each representation T R k * These are, at least in part, corresponding to previously generated expressions. T R k-1 * It is generated based on the following, where k is the step, which is the index that passes through the numbers from 3 to n, - From the machine learning model, one or more representations of the inspection domain R p+q * This is a step in which one or more expressions R are received. p+q * Each of these is in state Z p+q This represents the examination region, where q is the index passing through numbers from 1 to m, and m is an integer less than n-1, an integer equal to n-1, or an integer greater than n-1, and -1 or more expressions R p+q * The process involves the steps of outputting, and / or storing, and / or transmitting the data. [Modes for carrying out the invention]

[0020] Hereinafter, the present invention will be described more specifically without distinguishing between its subject matter (training methods, prediction methods, computer systems, computer program products, uses, contrast agents for use, kits). Rather, the following description is intended to apply similarly to all subject matter of the present invention, regardless of the context in which they appear (training methods, prediction methods, computer systems, computer program products, uses, contrast agents for use, kits).

[0021] In this specification or the claims, if each step is described in a certain order, this does not necessarily mean that the invention is limited to the described order. Rather, it is envisioned that each step may be executed in a different order or that each step may be executed in parallel with each other, except in the case where one step is based on another step, in which case the step based on the previous step must be executed subsequently (however, this will be clear in individual cases). Thus, the described order is a preferred embodiment of the invention.

[0022] With the assistance of the present invention, it is possible to predict the representation regarding the inspection region in the inspection object.

[0023] The "inspection object" is usually a living being, preferably a mammal, and most preferably a human.

[0024] The "inspection region" is a part of the inspection object, such as an organ or a part thereof, such as the liver, brain, heart, kidney, lung, stomach, intestine, or a part of those organs, or a plurality of organs or another part of the body.

[0025] In a preferred embodiment of the present invention, the inspection region is a human liver or a part thereof.

[0026] The inspection region is also referred to as the field of view (FOV), and in particular, it is the volume imaged in a radiographic image. The inspection region is typically defined by a radiologist, for example, on a localizer image. Of course, the inspection region can also be defined in an automated manner, alternatively or additionally, based on, for example, a selected protocol.

[0027] The "representation regarding the inspection region" is preferably a medical image. The representation regarding the inspection region is preferably the result of a radiological examination.

[0028] "Radiology" is a branch of medicine that primarily uses electromagnetic and mechanical waves (including, for example, ultrasound) for diagnostic, therapeutic, and / or scientific purposes. Besides X-rays, other ionizing radiations, such as gamma radiation or electrons, are also used. Because imaging is a key application, other imaging techniques, such as ultrasound and magnetic resonance imaging (nuclear magnetic resonance imaging), are also counted as radiology, even though they do not use ionizing radiation. Therefore, in the context of this invention, the term "radiology" encompasses imaging techniques, particularly computed tomography, magnetic resonance imaging, and ultrasound.

[0029] In a preferred embodiment of the present invention, the radiological examination is computed tomography or magnetic resonance imaging.

[0030] Computed tomography (CT) is an X-ray imaging technique that creates cross-sectional images of the human body. Compared to conventional X-ray images, which can usually only distinguish coarse structures and bones, CT images capture even soft tissues with small contrast differences in detail. The X-ray tube generates a so-called X-ray fan beam, which penetrates the body and is attenuated to varying degrees by various structures such as organs and bones. A receiving detector located on the opposite side of the X-ray emitter receives signals of varying intensities and transmits these signals to a computer, which then compiles cross-sectional images of the body from the received data. Computed tomography images (CT images) can be observed in both 2D and 3D. To make internal structures of the human body (e.g., blood vessels) more clearly identifiable, a contrast agent can be injected, for example, intravenously, before generating the CT image.

[0031] Magnetic resonance imaging, or MRI for short, is an imaging technique used particularly in medical diagnosis to visualize the structure and function of tissues and organs within the body of humans or animals.

[0032] In MRI, the magnetic moments of protons within the examination area are aligned within the fundamental magnetic field, resulting in the generation of macroscopic magnetization along the longitudinal direction. This magnetization is then deflected from its stationary position by irradiation (excitation) with a high-frequency (HF) pulse. Subsequently, the return from the excited state to the stationary state (relaxation), i.e., the magnetization dynamics, is detected as a relaxation signal by one or more high-frequency receiving coils.

[0033] For spatial coding, rapidly switching gradient magnetic fields are superimposed onto the fundamental magnetic field. The captured relaxation signal, i.e., the detected MRI data, initially exists as raw data in frequency space and can be transformed into real space (image space) by a subsequent inverse Fourier transform.

[0034] In the context of this invention, expressions relating to the examination area may include MRI images, computed tomography images, ultrasound images, or similar images.

[0035] In the context of this invention, the representation of the inspection area can be a representation in real space (image space), a representation in frequency space, or several other representations. Preferably, the representation of the inspection area exists in a form that exists within a real-space representation, or that can be converted into a real-space representation. A representation in real space is often also referred to as a pictorial representation or image.

[0036] In this specification, a representation in real space, also referred to as a real-space depiction or real-space representation, is typically represented by a large number of image elements (pixels or voxels), which may be, for example, a raster array, where each image element represents a portion of the examination area, and each image element may be assigned a color value or a gray value. The DICOM format is a format widely used in radiology for storing and processing representations in real space. DICOM (Digital Imaging and Communications in Medicine) is an open standard for storing and exchanging information in medical image data management.

[0037] A representation in real space can be transformed (converted) into a representation in frequency space, for example, by the Fourier transform. Conversely, a representation in frequency space can be transformed (converted) into a representation in real space, for example, by the inverse Fourier transform.

[0038] In a representation within frequency space, also referred to herein as frequency-space plotting or frequency-space representation, the test region is represented by a superposition of fundamental frequencies. For example, the test region may be represented by the sum of sine and / or cosine functions having different amplitudes, different frequencies, and different phases. The amplitude and phase may be plotted as functions of frequency, for example, in a two-dimensional or three-dimensional representation. Typically, the lowest frequency (origin) is located in the center. As you move away from this center, the frequency increases. Each frequency can be assigned an amplitude representing the frequency in the frequency-space plotting and a phase indicating the degree of wave shift relative to the sine or cosine wave, respectively.

[0039] Details regarding real-space and frequency-space mapping, as well as their corresponding interconversions, are described in numerous publications; for example, see https: / / see.stanford.edu / materials / lsoftaee261 / book-fall-07.pdf.

[0040] In the context of the present invention, "representation" refers to an inspection region in one state of a sequence consisting of multiple states.

[0041] A sequence consisting of multiple states is preferably a time series. States directly linked within a time series may always have the same time interval between them, or they may have different time intervals. Mixed forms are also conceivable.

[0042] This will be explained with reference to Figure 1.

[0043] Figure 1 shows two timelines (t = time), namely, the first timeline (a) and the second timeline (b). Both timelines are labeled by defined time points t1, t2, t3, and t4, respectively. At each time point, the test area can have different states; that is, each time point can represent one state with respect to the test area.

[0044] Points t1, t2, t3, and t4 form a time series t1→t2→t3→t4 with respect to each timeline. Point t2 is directly following point t1, and the time interval between t1 and t2 is t2-t1. Point t3 is directly following point t2, and the time interval between t2 and t3 is t3-t2. Point t4 is directly following point t3, and the time interval between point t3 and t4 is t4-t3.

[0045] In the first timeline (a), the time intervals between directly connected points in time are the same for all directly connected points in time, i.e., t2-t1=t3-t2=t4-t3.

[0046] In the case of the second timeline (b), the time intervals between directly consecutive time points are different for all directly consecutive time points, i.e., t2 - t1 (not equal to) t3 - t2 (not equal to) t4 - t3. In this example, for directly consecutive time points along the second time axis (b), the time intervals are increasing, i.e., t2 - t1 < t3 - t2 < t4 - t3. However, it is also assumed that the time intervals may decrease, or the time intervals may first increase and then decrease, or the time intervals may first decrease and then increase, or the time intervals may have some other distribution along the time axis.

[0047] It should be noted that the time axis does not necessarily have to extend in the direction of increasing time. This means that when viewed from time point t2, time point t1 does not necessarily have to be in the past, i.e., it is also assumed that the time axis indicates decreasing time and time point t1 is in the future when viewed from time point t2. In other words, when there are a plurality of states forming a time series, the second state directly following the first state may be located in the future or in the past when viewed from the time point of the first state.

[0048] In the first step, a sequence consisting of a plurality of (preferably time - series) states may be defined. The sequence consisting of a plurality of states defines both what training data to use for training a machine - learning model and what representations the machine - learning model can generate (predict). In other words, what is required for training a machine - learning model is the representation of the inspection area in a number of inspection objects in a plurality of states defined by the sequence consisting of a plurality of states, and a trained machine - learning model can usually generate only the representation related to the state that was part of the training (exceptions to this rule are listed later in this specification).

[0049] Preferably, a sequence consisting of multiple states defines different states of the examination area before, / or during, and / or after administration of the contrast agent, either once or multiple times.

[0050] A "contrast agent" is a substance or mixture of substances used in radiographic imaging to improve the visualization of the structure and function of the body.

[0051] Examples of contrast agents can be found in the literature (e.g., ASL Jascinth et al.: Contrast Agents in computed tomography: A Review, Journal of Applied Dental and Medical Sciences, 2016, vol. 2, issue 2, 143-149; H. Lusic et al.: X-ray-Computed Tomography Contrast Agents, Chem. Rev. 2013, 113, 3, 1641-1666, https: / / www.radiology.wisc.edu / wp-content / uploads / 2017 / 10 / contrast-agents-tutorial.pdf; MR Nough et al.: Radiographic and magnetic resonances contrast agents: Essentials and tips for safe practices, World J Radiol. 2017 Sep. 28; 9(9): 339-349; LC Abonyi et al.: Intravascular Contrast Media in Radiography: See Historical Development & Review of Risk Factors for Adverse Reactions, South American Journal of Clinical Research, 2016, vol. 3, issue 1, 1-10; ACR Manual on Contrast Media, 2020, ISBN: 978-1-55903-012-0; and A. Ignee et al.: Ultrasound contrast agents, Endosc Ultrasound. 2016 Nov-Dec; 5(6): 355-362.

[0052] Preferably, the contrast agent is an MRI contrast agent. MRI contrast agents exert their effects by altering the relaxation time of structures that take up the contrast agent. Two groups of materials can be identified: paramagnetic materials and superparamagnetic materials. Both groups of materials have unpaired electrons that induce a magnetic field around individual atoms or molecules. Superparamagnetic contrast agents result in a significant shortening of T2, while paramagnetic contrast agents mainly result in a shortening of T1. The effect of contrast agents is indirect because they do not emit a signal themselves, but merely affect the signal intensity of nearby objects. An example of a superparamagnetic contrast agent is iron oxide nanoparticles (SPIO, superparamagnetic iron oxide). Examples of paramagnetic contrast agents include gadolinium chelates, such as dimeglumine gadopentetate (trade name: Magnevist®, and others), gadoteric acid (Dotarem®, Dotagita®, Cyclolux®), gadodiamide (Omniscan®), gadoteridol (ProHance®), and gadobutrol (Gadovist®).

[0053] Preferably, the contrast agent for MRI is a hepatobiliary contrast agent. Hepatobiliary contrast agents are specifically taken up by hepatocytes (hepatic parenchymal cells), accumulate in functional tissue (parenchymal tissue), and enhance the contrast of healthy liver tissue. An example of a hepatobiliary contrast agent is the disodium salt of gadoxetinic acid (Gd-EOB-DTPA disodium), which is described in U.S. Patent No. 6,039,931A and is marketed under the trade names Primovist® and Eovist®.

[0054] After intravenous administration of a hepatobiliary contrast agent in the form of a bolus into a vein in the arm, the contrast agent first reaches the liver through the arteries. These are then depicted with enhanced contrast in the corresponding MRI images. The phase in the MRI image in which the hepatic artery is depicted with enhanced contrast is called the "arterial phase."

[0055] Subsequently, the contrast agent reaches the liver through the hepatic veins. While the contrast of the hepatic arteries has already decreased, the contrast of the hepatic veins reaches its maximum. The phase in the MRI image in which the hepatic veins are depicted with enhanced contrast is called the "portal phase."

[0056] The portal venous phase is followed by a "transitional phase," during which the contrast of the hepatic arteries further decreases, as does the contrast of the hepatic veins. When hepatobiliary contrast agents are used, the contrast of healthy hepatocytes gradually increases during the transitional phase.

[0057] The arterial phase, portal venous phase, and transitional phase are also collectively referred to as the "dynamic phase."

[0058] Hepatobiliary contrast agents produce a clear signal enhancement in healthy liver parenchyma 10 to 20 minutes after injection. This phase is called the "hepatocyte contrast phase." Because the contrast agent is slowly eliminated from hepatocytes, the hepatocyte contrast phase can continue for more than two hours.

[0059] Each of the phases described can be found in more detail in the following publications, for example: J. Magn. Reson. Imaging, 2012, 35(3): 492-511, doi:10.1002 / jmri.22833; Clujul Medical, 2015, Vol.88 No.4: 438-448, DOI: 10.15386 / cjmed-414; Journal of Hepatology, 2019, Vol.71: 534-542, http: / / dx.doi.org / 10.1016 / j.jhep.2019.05.005).

[0060] Each state in a sequence consisting of multiple states is, for example, - Before administration of hepatobiliary contrast agent, -In the arterial phase, -In the portal venous phase, - In the transition phase, and / or -In the hepatocyte contrast-enhanced phase, The condition of the liver as an area of ​​examination is also acceptable.

[0061] It is assumed that there are either a greater number of states or a smaller number of states.

[0062] The following provides a more detailed explanation of each phase described, with reference to Figure 2. Figure 2 schematically shows the time profile (t=time) of signal intensity I induced in hepatic artery A, hepatic vein V, and healthy hepatocytes L by a hepatobiliary contrast agent in a dynamic contrast-enhanced MRI. Signal intensity I is positively correlated with the contrast agent concentration in each region described. After intravenous bolus injection into the human arm, the contrast agent concentration first rises in hepatic artery A (dashed curve). The concentration passes a maximum value and then decreases. The concentration in hepatic vein V rises more slowly than in the hepatic artery and reaches a maximum value at a later point (dotted curve). The contrast agent concentration in hepatocytes L rises slowly (solid curve) and reaches its maximum value much later (not shown in Figure 2). Several characteristic time points can be defined, namely, at time point TP1, the contrast agent is administered intravenously as a bolus. Since the administration of the contrast agent itself requires a certain time interval, time point TP1 preferably defines the time when the administration is completed, that is, the time when the contrast agent has been fully introduced into the area being examined. At time point TP2, the signal intensity of the contrast agent in hepatic artery A reaches its maximum value. At time point TP3, the signal intensity curve for hepatic artery A and the signal intensity curve for hepatic vein V intersect. At time point TP4, the signal intensity of the contrast agent in hepatic vein V passes its maximum value. At time point TP5, the signal intensity curve for hepatic artery A and the signal intensity curve for healthy hepatocytes L intersect. At time point TP6, the concentrations in hepatic artery A and hepatic vein V have decreased to a level that no longer causes measurable contrast enhancement.

[0063] Each state in a sequence of multiple states may include a first state existing before time point TP1, a second state existing at time point TP2, a third state existing at time point TP3, a fourth state existing at time point TP4, a fifth state existing at time point TP5, and / or a sixth state existing at and / or after time point TP6.

[0064] Generally speaking, -The first state may be the state of the examination area at the first time point before the administration of the contrast agent. -The second state may be the state of the examination area at the second time point after the administration of the contrast agent. -The third state may also be the state of the examination area at the third time point after the administration of the contrast agent. -The fourth state may also be the state of the examination area at the fourth time point after the administration of the contrast agent. And so on.

[0065] As described above, all points in time that are directly connected to each other, or some of them, may have a fixed time interval between them and / or a variable time interval between them.

[0066] According to the present invention, at least three states exist, and preferably, the number of states is 3 to 100. However, the number of states is not limited to 100.

[0067] In a sequence consisting of multiple states, each state may have one or more representations that indicate the inspection area for that corresponding state.

[0068] Typically, the first state has at least one first representation representing the test area in the first state, the second state has at least one second representation representing the test area in the second state, the third state has at least one third representation representing the test area in the third state, and so on.

[0069] For the sake of easier understanding of this specification, the present invention will be described primarily based on a single expression relating to each state, but this should not be understood as limiting the invention.

[0070] Returning to the example above, for instance, - One or more first expressions representing the liver before administration of hepatobiliary contrast agent, - One or more second representations of the liver during the arterial phase, - One or more third representations of the liver during the portal venous phase, - One or more fourth representations of the liver during the transitional phase, and / or - There may be one or more fifth representations of the liver during the hepatocyte contrast-enhanced phase.

[0071] With the support of the present invention, it is possible to predict one or more representations of an inspection area in an object under inspection, such as representing an inspection area in one state of a sequence consisting of multiple states.

[0072] The predictions are made with the help of machine learning models.

[0073] A "machine learning model" can be understood as a data processing architecture implemented by a computer. The model can receive input data and, based on this input data and further based on model parameters, can supply output data. Through training, the model can learn the relationship between input and output data. During training, model parameters can be adjusted to supply a desired output for a specific input.

[0074] During training for such a model, the model is presented with training data that it can learn from. A trained machine learning model is the result of the training process. In addition to the input data, the training data includes appropriate output data (target data) that the model should generate based on the input data. During training, patterns that map the input data onto the target data are recognized.

[0075] In the training process, input data from the training dataset is fed into the model, and the model generates output data. The output data is compared to the target data. Model parameters are modified to reduce the difference between the output data and the target data to a (predetermined) minimum value.

[0076] This difference can be quantified using a loss function. This type of loss function allows us to calculate the loss for a given set of output and target data. The goal of the training process is to modify (fit) the machine learning model parameters so that the loss for all pairs of the training dataset is reduced to a (specified) minimum. Adjusting the model parameters to reduce the loss can be done using optimization techniques such as gradient descent.

[0077] For example, if the output data and target data are numerical, the loss function can be the absolute difference between those numbers. In this case, a large absolute loss means that one or more model parameters need to be significantly changed.

[0078] For output data in vector format, the loss function can be selected from among the difference metrics between the vectors, such as the mean squared error, cosine distance, norm of the difference vector (such as Euclidean distance or Chebyshev distance), Lp norm of the difference vector, weighted norm, or any other type of difference metric relating to the two vectors.

[0079] For high-dimensional outputs, such as 2D, 3D, or higher-dimensional outputs, an element-wise difference metric can be used, for example. Alternatively or additionally, the output data can be converted to, for example, a 1D vector before calculating the loss.

[0080] In this case, the machine learning model is trained on training data to generate a representation of the check region in one state of a sequence of multiple states based on the representation of the check region in a preceding state in the sequence of multiple states. The model starts from the first state and iteratively generates representations of the check region in subsequent states, with each representation of the check region in each state being generated at least partially based on the (predicted) representation of the preceding state.

[0081] With regard to this, and without any intention to limit the present invention to this embodiment, we will explain it based on an example of three states. This explanation will refer to Figure 3.

[0082] In this explanation, please note that the representation of the inspection area is identified by the letter R. The letter R can be followed by an index, for example, R1, R2, R3, or R i This can be done as follows. The index added as a suffix indicates what is represented by the corresponding expression. Expression R1 represents the inspection area of ​​the object being inspected in the first state Z1 of a sequence consisting of multiple states, expression R2 represents the inspection area of ​​the object being inspected in the second state Z2 of a sequence consisting of multiple states, expression R3 represents the inspection area of ​​the third state Z3 of a sequence consisting of multiple states, and so on. In general, expression R i Z is one state in a sequence of multiple states. i This represents the test region within the object being tested, where i is the index through which integers i to n pass, and n is an integer greater than or equal to 3. The expression "index i passes through integers a to b" means that i takes the values ​​a to b in succession, that is, i first takes the value a (i=a), then the value a+1 (i=a+1), and so on, until i reaches the value b (i=b). i * However, there are multiple states Z1~Z n One state Z of a sequence consisting of iThe expression "represents the inspection area in state Z1~Z" means n When (in a mathematical sense) they form a sequence, the representation R1 * This represents the inspection area in state Z1, and the representation R2 * This represents the inspection area in state Z2, and so on. In particular, in the claims and drawings, the representation used for training and the representation generated in training are, for example, T R1, T R2, and T R k-1 * It is added as a prefix, as in the case of T It is identified by R1. In particular, in the claims and drawings, the representations generated by the machine learning model are R1 * , R2 * , and T R k-1 * As in the case of a superscript asterisk * Identified by [the specified label]. The labels described herein serve solely for clarification, and in particular to avoid clarity-related objections in the patent granting procedure.

[0083] Figure 3 shows three representations, namely the first representation. T R1 and the second expression T R2 and the third expression T This indicates R3. First representation T R1 represents the inspection area in the inspection target in the first state Z1, and is the second representation. T R2 represents the examination region in the second state Z2, and is the third representation. T R3 represents the examination region in the third state Z3. The three states form a sequence consisting of multiple states, namely, the first state (Z1) → second state (Z2) → third state (Z3).

[0084] Three expressions T R1, T R2, TR3 forms the datasets within the training data TD. The training data TD contains multiple such datasets. The term "multiplicity" preferably means more than 100. Each dataset typically (but not necessarily) contains one representation for each state, and for three states, it contains three representations of the test region. The test region is usually always the same, and each dataset contains representations of the test region for three states, and these states are also usually the same for all datasets, namely the first state, the second state, and the third state. Only the test subjects may differ. Each dataset is usually derived from different test subjects. The descriptions in this paragraph are not limited to the example shown in Figure 3 and are generally applicable.

[0085] Step 1 (A) is the first expression T R1 is supplied to the machine learning model M. The machine learning model M is the first representation T Based on R1 and model parameter MP, output T R2 * It is configured to generate (step (B)) the output. T R2 * is the second expression T The intention is to approximate R2 as closely as possible, and ideally, the output T R2 * is the second expression T It cannot be distinguished from R2. In other words, the output T R2 * This is the predicted second representation. Output T R2 * is the (actual) second expression T It is compared with R2, and the difference is quantified using the loss function LF2. If there are multiple test subjects, the loss value LV2 is output by the loss function LF2. T R2 * and second expression T For each pair consisting of R², this can be calculated.

[0086] Examples of loss functions that can be commonly used to carry out the present invention (not limited to the example in Figure 3) include the L1 loss function, L2 loss function, Lp loss function, Structural Similarity Index Scale (SSIM), VGG loss function, Perceptual Loss function, or combinations of the above functions or combinations with other loss functions. Further details regarding loss functions can be found, for example, in scientific literature (see, for example, R. Mechrez et al.: The Contextual Loss for Image Transformation with Non-Aligned Data, 2018, arXiv:1803.02077v4 and H. Zhao et al.: Loss Functions for Image Restoration with Neural Networks, 2018, arXiv:1511.08861v3).

[0087] output T R2 * In step (C), the first representation is resupplied to the machine learning model M. Figure 3 may suggest otherwise, but in step (A), the first representation T R1 is supplied with a machine learning model M and the generated representation T R2 * The machine learning model M supplied in step (C) is the same model. That is, the machine learning model M is configured not only to generate a predicted second representation based on the first representation, but also to generate a third representation from the (predicted and / or actual) second representation. T Output that approximates R3 as closely as possible. T R3 * It is also configured to generate the output. T R3 * This is the predicted third representation. Output T R3 * This is the third expression T It is compared with R3. Using the loss function LF3, the output T R3 * and the third expression TThe difference from R3 can be quantified. When there are multiple test subjects, the loss value LV3 is the third representation. T R3 and generated representation T R3 * For each pair consisting of and , a decision can be made.

[0088] Preferably, the machine learning model is trained end-to-end. That is, the machine learning model is simultaneously trained to generate a predicted second representation based on a first representation, and then to generate a predicted third representation based on the predicted second representation. Preferably, this is achieved by using a loss function that takes into account both the difference between the predicted second representation and the second representation, and the difference between the predicted third representation and the third representation.

[0089] The difference between the second representation and the predicted second representation can be quantified using the loss function LF2, and the difference between the third representation and the predicted third representation can be quantified using the loss function LF3. The loss function LF, which takes both differences into account, may be, for example, the sum of the individual loss functions, i.e., LF = LF2 + LF3. Furthermore, different weights can be assigned to the components formed by the individual loss functions LF2 and LF3 in the overall loss function LF, i.e., LF = w2·LF2 + w3·LF3, where w2 and w3 are weight coefficients that can take values ​​from, for example, 0 to 1. For example, if representations are missing in the dataset, a value of zero may be used for the weight coefficient (further details can be found below in this specification).

[0090] The training method shown in Figure 3 is a preferred embodiment of the present disclosure, - This is the step of receiving training data. • Training data represents representations of the test area across a large number of test subjects. T R1~ T Including R3, ·Expression T R1 represents the inspection region in the first state Z1, and is expressed as TR2 represents the inspection region in the second state Z2, and is expressed as T R3 represents the inspection area in the third state Z3, and states Z1, Z2, and Z3 form a step that constitutes a sequence of multiple states. -This is the step of training a machine learning model, and the training for each of the numerous test subjects is performed. ·Expression T This involves supplying R1 to a machine learning model, in which case the machine learning model is at least partially represented T Based on R1 and model parameters, representation T R2 * It is configured to generate, ·Expression T R2 * This involves supplying a machine learning model with a representation, in which case the machine learning model is at least partially represented. T R2 * Based on, and based on model parameters, representation T R3 * It is further configured to generate, ·Expression T R2 and the generated representation T R2 * Quantify the difference and express T R3 and generated representation T R3 * Quantifying the difference between and The steps include minimizing the difference between the model parameters by changing them, -Includes the steps of storing a trained machine learning model and / or using the machine learning model for prediction.

[0091] When a trained machine learning model is used (later) to predict a new representation, it may predict a third representation based on a first representation. For this purpose, in principle, it is possible to train the model to directly generate a third representation based on a first representation. However, according to the present invention, the machine learning model is trained to generate a third representation in two steps, such that the second representation is predicted in the first step, and then the third representation is predicted based on the second representation, rather than generating a third representation in a single step based on the first representation. The advantage of the approach according to the present invention compared to the "direct training" described above is, in particular, that it can achieve more accurate predictions because additional training data (a second representation representing the examination domain in the second state) can be used. Furthermore, instead of learning how one representation maps onto another (or, as in International Publication No. 2021 / 052896, how multiple representations map onto one representation), the model learns the dynamic behavior of the examination domain, that is, the dynamics through which the states pass. The more states covered by the training data, the more accurately the model can learn the dynamics from one state to a subsequent state. Furthermore, a model trained in this manner can be used to predict representations of states for which the training data is unavailable, by predicting representations of even more subsequent states by using the model multiple times (iteratively). This extrapolation to states not considered during training is described in more detail below.

[0092] Once the machine learning model shown in Figure 3 has been trained, it can be used for prediction. This is shown in Figure 4. In the first step (A), a first representation R1 is supplied to the trained machine learning model M. The first representation R1 represents the test region in a new test subject in the first state Z1. The term "new" means that no representation of the test subject was used when training the machine learning model M. In step (B), the model M generates a second representation R2 based on the first representation R1 and the model parameter MP. * Generates the second representation R2. * This represents the inspection region in the object being inspected in the second state Z2. In the third step (C), the generated second representation R2 * However, this is supplied to the machine learning model M. Model M, in step (D), the second representation R2 * Based on and model parameter MP, the third representation R3 * Generates the third representation R3. * This represents the inspection area of ​​the object being inspected in the third state Z3.

[0093] The prediction method shown in Figure 4, and the prediction method based on the training method shown in Figure 3, are preferred embodiments of this disclosure. - A step of receiving a representation R1 relating to the inspection region, where representation R1 represents the inspection region in the first state Z1 of a sequence consisting of multiple states Z1, Z2, Z3, - Using a machine learning model, we can determine the representation R2 of the inspection region, at least partially based on the representation R1. * This is the step to generate the representation R2 * This represents the inspection region in the second state Z2, where the second state Z2 is a step that directly follows the first state Z1 in a sequence of multiple states, - Using a machine learning model, at least partially represent R2 * Based on this, the expression R3 regarding the inspection area. * This is the step of generating the expression R3 *This represents the inspection region in the third state Z3, where the third state Z3 is a step that directly follows the second state Z2 in a sequence of multiple states, - Expression R3 * Steps to output and / or representation R3 * Steps to store and / or represent R3 * The steps include sending a machine learning model, where the machine learning model is trained on training data, and the training data includes a large number of datasets, each dataset representing T R1 and expression T R2 and expression T R3, including, and here expressed T R1 represents the inspection region in the first state Z1 of a sequence consisting of multiple states, and is expressed as T R2 represents the inspection region in the second state Z2 of a sequence consisting of multiple states, and is expressed as T R3 represents the inspection region in the third state Z3 of a sequence consisting of multiple states, where the machine learning model represents, at least partially, each of the numerous inspection targets. T Expressed based on R1 T R2 * To generate, further, express at least partially T R2 * Expressed based on T R3 * It is trained to generate, where training is the representation T Expressed as R2 T R2 * The difference between and, and the expression T Expressed as R3 T R3 * This includes quantifying the difference and modifying the model parameters to minimize that difference.

[0094] In general, the machine learning model according to the present invention has multiple states Z1 to Z n One state Z of a sequence consisting of iA model may be trained to predict a representation of the test domain in the test object, where n is an integer greater than or equal to 3, and i is an index passing through the numbers from 2 to n.

[0095] The machine learning model is a first representation of the inspection region in state Z1. T Starting from R1, state Z2~Z n A series of expressions regarding the inspection area T R1 * ~ T R i * The system may be trained to generate a succession of representations, where each representation for a given state is generated, at least in part, based on the corresponding representation previously generated for a directly preceding state.

[0096] This will be explained in more detail using the example in Figure 5. Figure 5 can be understood as an extension of the method shown in Figure 3 from 3 states to n states, where n is an integer greater than or equal to 3. Preferably, the number n is in the range of 3 to 100.

[0097] Figure 5 shows the machine learning model M. Although model M is specified three times, it is always the same model. The machine learning model M is trained to predict n-1 representations in succession. In this example, n is an integer greater than or equal to 4. The input data for the model is the first representation T R1 is supplied. First expression T R1 represents the test region in the test object in the first state Z1. The machine learning model M is at least partially the first representation T Based on R1 and based on the model parameter MP, the output T R2 * It is configured to generate the output. T R2 * This is the predicted second representation, which represents the inspection region in the second state Z2. The model, at least partially, outputs T R2 *Based on and based on the model parameter MP, the output T R3 * It is further configured to generate the output. T R3 * This is the predicted third representation, which represents the inspection region in the third state Z3. The model, at least partially, outputs T R3 * Based on and model parameter MP, it is further configured to generate a further output showing a predictive representation of the inspection region in one state following state Z3. This scheme outputs T R n * It continues until... Output T R n * is the nth state Z n This is the predicted nth representation that represents the examination area.

[0098] State Z i (Here, i=1~n) is a sequence consisting of multiple states (Z1→Z2→Z3→...→Z n ) forms.

[0099] In the example shown in Figure 5, the first expression T Not only R1, but also state Z j Further (actual, measurement-generated) representations of the test region (where j=2~n) T R j (Here, j=2~n) also exists, and these expressions T R j This can be used as ground truth data for training machine learning models. For example, the output T R j * (This is state Z) j (This is the predicted j-th representation of the inspection area) and expressed T R j The difference is the loss function LF j It may be quantified using, i.e., the loss function LF2 is expressed as T R2 and the generated representation T R2* The difference can be quantified, and the loss function LF3 expresses T R3 and generated representation T R3 * This allows for the quantification of the difference, and so on.

[0100] In end-to-end training, a total loss function LF that takes into account all individual differences may be used; for example, this may be the sum of the individual differences.

[0101]

number

[0102] Alternatively, the sum may be a weighted sum.

[0103]

number

[0104] Here, the weight coefficient w j It can take values ​​between 0 and 1, for example.

[0105] The advantage of weighting is that one state Z n Expressions representing the inspection area T R n * When generating state Z, n This means that one or more (generated) representations representing the state immediately preceding a state are assigned a larger or smaller weight compared to other representations. For example, the weight coefficient w j w2, w3, ..., w n By increasing in this order, a larger weight is assigned to state Z n A weight can be given to an expression related to a state that is closer to the original state. This increase may be logarithmic, linear, squared, cubed, or exponential, or it may be some other increase. It is also conceivable to give smaller weights to subsequent states, in which case the weight coefficients may be w2, w3, ..., wn Since they decrease in the order of, a larger weight is given to the expression related to the state closer to the initial state Z1. Even in such a case of decrease, the decrease may be logarithmic, linear, quadratic, cubic, or exponential, or may be some other decrease.

[0106] Also, it is assumed that the training data includes an incomplete data set. This means that some data sets regarding the inspection target do not include all the expressions T R1~ T R n As will be described later in this specification, an incomplete data set may also be used for training a machine learning model. When the expression T R p is missing, the weight coefficient w T R p used to weight the difference between the generated expression R p * and R p may be set to zero, where p is an integer that can take values from 2 to n.

[0107] Also, it is assumed that the training includes considering random numbers 1 <= j < k <= n and related partial sequences Z j 、Z j+1 、...、Z k-1 、Z k consisting of a plurality of states. The above-mentioned problems of learning may be solved based on these partial sequences (optionally, changing from moment to moment). For example, it is assumed that a random initial state Z j (1 <= j <= n - 2) is determined, and the model always synthesizes expressions regarding the subsequent two states Z j+1 、Z j+2 based on such an initial state.

[0108] In FIG. 5, also, the expression j representing the inspection area at the state Z T R j *However, the preceding state Z j-1 Expressions representing the inspection area T R j-1 * Furthermore, an even earlier state Z j-2 , Z j-3 ..., a further (generated) representation of the inspection area in Z1 T R j-2 * and / or T R j-3 * and / or... T R1 * It has also been shown in general terms that it can be generated by using up to, where j is an integer that can take values ​​from 2 to n. This is represented by a dashed arrow. For example, representation T R3 * is an expression T R2 * not only expression T R1 can also be generated by supplying it to the machine learning model M. T R4 * is an expression T R3 * not only, expression T R2 * and / or expression T R1 can also be generated by supplying it to the machine learning model M.

[0109] To generate representations, additional information may also be incorporated into the machine learning model, such as information about the state in which the inspection region is located, and by using representations of that state, representations for subsequent states are generated. In other words, representations T R2 * is an expression T Not only R1, but also expression T The representation can be generated by using information about the state in which the test region represented by R1 is located. T R3 * Also, expression T R2 *Based on and information regarding the second state Z2, it can be generated.

[0110] During training, and / or when using a trained machine learning model for prediction, if information about the states corresponding to the representations is provided, the machine learning model will "know" where each state is located within a sequence of multiple states.

[0111] For example, a state may be represented by a number or a vector. For example, multiple states may be numbered sequentially, so that the first state is represented by number 1, the second state by number 2, and so on.

[0112] Instead of, or in addition to, information about the corresponding states, time that can be assigned to each corresponding state may also be used in generating the representation. For example, the first state in a sequence of states may be assigned time t=0. Time t=0 may be input into the machine learning model along with the first representation to predict the second representation. The second state may be assigned seconds, minutes, or some other unit of time difference indicating how much time has passed between the first and second states. This time difference may be supplied to the machine learning model along with the predicted second representation (and optionally the first representation) to predict the third representation. Similarly, the third state may be assigned a time difference indicating how much time has passed between the first and third states. This time difference may be supplied to the machine learning model along with the predicted third representation (and optionally the first and / or predicted second representation) to predict the third representation, and so on.

[0113] The machine learning model according to the present invention can also be understood as meaning a transformation that can be applied to a representation of a check region in one state of a sequence of multiple states in order to predict a representation of a check region in a subsequent state of the sequence of multiple states. This transformation may be applied alone (only once) to predict a representation of a check region in a subsequent state, or it may be applied multiple times (multiple times, iteratively) to predict a representation of a check region in a further downstream state within the sequence of multiple states.

[0114] n states Z1~Z n If a sequence consisting of exists, then further, each state Z i In state Z, i R is an expression representing the inspection area. i If state Z exists, 1+q Predicted representation of the inspection area R 1+q * To generate the expression R1, a machine learning model M may be applied q times, starting from the expression R1, where q can take values ​​from 1 to n-1. M q (R1) = R 1+q * q=1:M(R1)=R2 * q=2:M(R2 * )=M(M(R1))=M 2 (R1) = R3 * q=3:M(R3 * )=M(M(R2 * ))=M(M(M(R1)))=M 3 (R1) = R4 * ... q=n-1:M(R n-1 * )=M(M(R n-2 * ))=...=M n-1 (R1) = R n *

[0115] This also applies to training methods, where the transformation M over q steps may be expressed by the following formula. M q ( T R1) = T R 1+q *

[0116] Generated expression T R q * and the actual (measurement-generated) representation T R q The loss function that quantifies all differences between and may be expressed, for example, by the following equation: LV = w²·d(M( T R1), T R2)+w3·d(M 2 ( T R1), T R3)+...+w n ·d(M n-1 ( T R1), T R n )

[0117] Here, LV is the (actual, measurement-generated) representation. T R1, T R2, ... T R n This is the loss value generated for the dataset containing . d is the predicted representation M( T R q-1 ) and expressed as T R q This is a loss function that quantifies the difference between the two. As already described herein, this may be, for example, one of the following loss functions: L1 loss function, L2 loss function, Lp loss function, Structural Similarity Index Scale (SSIM), VGG loss function, Perceived loss function, or a combination thereof.

[0118] w2, w3, ..., w n This is a weighting coefficient, similarly described in this specification.

[0119] n represents the number of states.

[0120] Furthermore, it is assumed that the loss value is the maximum difference calculated for the dataset, that is, LV=max(w2·d(M( T R1), T R2);w3·d(M 2 ( T R1), T R3);...;w n ·d(M n-1 ( T R1), T R n )), In this formula as well, by using weighting coefficients, different weights can be assigned to individual differences.

[0121] As already shown herein, it should be noted that in training a machine learning model, each training dataset does not need to contain representations of all states that the model should learn. In principle, two representations per dataset are sufficient. This will be illustrated with an example. Suppose a machine learning model is to be trained to predict representations of the check region in a sequence of six states Z1 to Z6. Suppose it is sufficient to train the machine learning model with training data containing 10 datasets, each containing a representation of the check region in a different state. Dataset 1: T R1, T R3, T R4, T R5, T R6 Dataset 2: T R1, T R2, T R4, T R6 Dataset 3: T R1, T R2, T R3, T R4, T R5 Dataset 4: T R1, T R2, T R3, T R5, T R6 Dataset 5: T R2, T R3, T R5, T R6 Dataset 6: T R2, T R3, T R4, T R5, T R6 Dataset 7: T R2, T R3, T R5, T R6 Dataset 8: T R3, T R5, T R6 Dataset 9: T R3, T R4, T R5, T R6 Dataset 10: T R3, T R4, T R6

[0122] In the example, there is no "complete" single dataset; that is, any dataset represents all possible representations. T R1, T R2, T R3, T R4, T R5, and T It does not include R6. Nevertheless, a machine learning model can be trained on such training data to predict the representation in each state. This is another advantage of the present invention over the "direct training" described above.

[0123] After a machine learning model has been trained, it can be supplied with new (i.e., not used in training) representations of the test region in one state of a sequence of multiple states, and the model can predict (generate) one or more representations of the test region in one subsequent state or multiple subsequent states of the sequence of multiple states.

[0124] This is schematically illustrated in Figure 6 as an example. In the example shown in Figure 6, state Z p R is an expression representing the inspection area. p Starting from, the generated state is state Z p+1 , Z p+2 , Z p+3 , Z p+4 R is an expression representing the inspection area. p+1 * , R p+2 * , R p+3 * , R p+4 * This is a sequence consisting of state Z. p+1 , Z p+2 , Z p+3 , Z p+4 This forms a sequence consisting of multiple states, namely, Z p+1 →Z p+2 →Z p+3 →Z p+4 It forms a structure.

[0125] In step (A), the machine learning model M is given a representation R. p It is supplied. Expression R p In addition, for the machine learning model M, and as described herein, state Z p Information relating to and / or additional / other information may be provided.

[0126] In step (B), the machine learning model M determines the state Z based on the supplied input data. p+1 R is an expression representing the inspection area. p+1 * Generates.

[0127] In step (C), the previously generated representation R is applied to the machine learning model M. p+1 * It is supplied. Expression R p+1 * In addition, for the machine learning model M, and also for the state Z p+1 Information regarding and / or additional information may be provided for the machine learning model M and the representation R. p However, and / or state Z p Information regarding this may be provided.

[0128] Preferably, the machine learning model M includes memory S for storing input data (and preferably output data as well), so that the input data and / or generated output data, once input, do not need to be received or input again, and are already available to the machine learning model. This applies not only to the use of a trained machine learning model for prediction as described in this example, but also to the training of a machine learning model according to the present invention.

[0129] In step (D), the machine learning model M determines the state Z based on the supplied input data. p+2 R is an expression representing the inspection area. p+2 * Generates.

[0130] In step (E), the previously generated representation R is applied to the machine learning model M. p+2 * It is supplied. Expression R p+2 * In addition, for the machine learning model M, also, state Z p+2 Information regarding and / or additional information may be provided. Furthermore, for the machine learning model M, also, representation R p and / or representation R p+1 * However, as well as / or state Z p and / or state Z p+1Information regarding this may be provided.

[0131] The machine learning model M, in step (F), determines the state Z based on the supplied input data. p+3 R is an expression representing the inspection area. p+3 * Generates.

[0132] In step (G), the previously generated representation R is applied to the machine learning model M. p+3 * It is supplied. Expression R p+3 * In addition, for the machine learning model M, also, state Z p+3 Information regarding and / or additional information may be provided. Furthermore, for the machine learning model M, also, representation R p and / or representation R p+1 * and / or representation R p+2 * However, as well as / or state Z p and / or state Z p+1 and / or state Z p+2 Information regarding this may be provided.

[0133] The machine learning model M, in step (H), determines the state Z based on the supplied input data. p+4 R is an expression representing the inspection area. p+4 * Generates.

[0134] The generated representation R p+1 * , R p+2 * , R p+3 * , and / or R p+4 * It may be output (for example, displayed on a monitor and / or printed by a printer), and / or stored in data memory, and / or transmitted to another computer system.

[0135] The machine learning model represents the first representation of the inspection region in the first state Z1. T Starting from R1, a series of expressions T R2 * ... T R n * If trained to generate, here each expression T R j * is one state Z j This represents the inspection region, where j is an index that passes through numbers from 2 to n, and for such a model, state Z p A new representation R for the inspection area p It can supply, and the machine learning model can express R p+1 * , R p+2 * ,..., R n * This can be generated, where p is a number that can take values ​​from 2 to n.

[0136] This means that a trained model does not necessarily need to be supplied with a new representation R1 that represents the examination region in the first state Z1. Furthermore, a trained machine learning model does not necessarily need R p+1 * From R n * It is not necessary to generate all representations up to state Z1~Z. n You can "drop in" and "drop out" at any point in the sequence consisting of R p Based on this, one or more representations can be generated that represent the inspection area in one or more subsequent states.

[0137] It is even possible to generate representations that represent states that were never covered in training. Therefore, representation R n * Instead of stopping at R n+1 * , R n+2 *Predictions may also be made regarding expressions such as "etc." Therefore, by using a trained machine learning model, the trained dynamics can be maintained, and representations of states that can never be generated by measurement can be computed. In this respect, the trained machine learning model may be used to extrapolate to new states.

[0138] Furthermore, predictions are not limited to subsequent states. It is also possible to predict representations of the inspection region in preceding states of a sequence of multiple states. Firstly, as already described herein, machine learning models can essentially be trained in both directions: toward subsequent states and toward preceding states. Secondly, machine learning models perform transformations on input representations that are, in principle, also possible in reverse. By analyzing the mathematical function of the model that transforms the input representation into an output representation, it is possible to reverse the process and determine an inverse function that returns the previous output representation back to the previous input representation. Then, using the inverse function, even if the model was trained to predict representations of subsequent states, it may also predict representations of preceding states, and vice versa.

[0139] Figure 7 shows an extension of the method shown in Figure 6. Figure 6 shows state Z p R is an expression representing the inspection area. p Based on this, state Z p+1 , Z p+2 , Z p+3 , Z p+4 R is an expression representing the inspection area. p+1 * , R p+2 * , R p+3 * , R p+4 * While Figure 7 shows how states Z are generated one after another, Figure 7 shows the state Z p R is an expression representing the inspection area. p Based on this, expression R p+1 * ~R p+q *This shows how the following are generated one after another, where q is an integer greater than or equal to 2. In Figure 7, the iterative properties of the machine learning model M are particularly clearly shown. The machine learning model M is represented by R p Starting from there, it is applied q times, and the output data is returned to the machine learning model M((q-1)x) (q-1) times.

[0140] It should be noted that machine learning models (trained or untrained) do not need to be applied to the entire radiographic image (e.g., an MRI image, or a CT scan, or similar). Machine learning models can be applied to only a portion of the radiographic image. For example, the radiographic image can first be segmented to identify / select a region of interest. The model can then be applied, for example, only to that region of interest.

[0141] Furthermore, the application of the machine learning model may include one or more preprocessing steps and / or one or more postprocessing steps. For example, it is conceivable that the received representation of the inspection domain is first subjected to one or more transformations, such as motion correction, color space conversion, normalization, segmentation, Fourier transform (e.g., for conversion from image space representation to frequency space representation), inverse Fourier transform (e.g., for conversion from frequency space representation to image space representation), and / or similar. In a further step, the transformed representation may be fed to the machine learning model, and then, through a series of iterations (cycles) (as schematically illustrated in Figure 7), a series of further (subsequent) representations of the inspection domain in a series of further (subsequent) states are generated, starting from the transformed representation.

[0142] The machine learning model according to the present invention may be, for example, an artificial neural network, or may include such a network.

[0143] An artificial neural network includes at least three processing layers: a first layer having input neurons (input nodes), an Nth layer having at least one output neuron (output node), and N-2 hidden layers, where N is a natural number greater than or equal to 3.

[0144] Input neurons function to receive representations. For example, when the representation is a real-space depiction in the form of a raster graphic, there may be one input neuron for each pixel or voxel of the representation, or when the representation is a frequency-space depiction, there may be one input neuron for each frequency present in the representation. Additional input neurons may be provided for additional input values ​​(e.g., information about the test area, information about the object being tested, information about the state when the representation was generated, information about the state represented by the representation, and / or information about the time when the representation was generated or the period during which the representation was generated).

[0145] The output neuron may function to output a generated representation of the test area in the subsequent state.

[0146] The processing elements in the layer between the input neuron and the output neuron are connected to each other in a predetermined pattern with predetermined connection weights.

[0147] Preferably, the artificial neural network is a so-called convolutional neural network (CNN), or includes such a network.

[0148] A CNN typically consists of alternating layers of filters (convolutional layers) and aggregation layers (pooling layers), and one or more layers of fully connected neurons (densely / fully connected layers) at the end.

[0149] Artificial neural networks may be trained, for example, by backpropagation. The goal of the network is to predict, as reliably as possible, the dynamics of a test domain from a given state through at least one intermediate state to the final state. The quality of the prediction is described by the loss function. The goal is to minimize the loss function. In the case of backpropagation, the artificial neural network is trained by changing the connection weights.

[0150] In a trained state, the connection weights between processing elements contain information about the dynamics of state changes, and this information can be used to predict one or more representations representing the check regions in one or more subsequent states, based on a first representation representing the check region in the first state.

[0151] Cross-validation may be used to classify the data into a training dataset and a validation dataset. The training dataset is used for training the backpropagation of the network weights. The validation dataset is used to check the prediction accuracy when the trained network is applied to unknown data.

[0152] An artificial neural network may have an autoencoder architecture, for example, an artificial neural network may have an architecture such as U-Net (see, for example, O. Ronneberger et al.: U-net: Convolutional networks for biomedical image segmentation, International Conference on Medical image computing and computer-assisted intervention, pages 234-241, Springer, 2015, https: / / doi.org / 10.1007 / 978-3-319-24574-4_28).

[0153] The artificial neural network may be a generative adversarial network (GAN) (see, for example, M.-Y. Liu et al.: Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications, arXiv:2008.02793; J. Henry et al.: Pix2Pix GAN for Image-to-Image Translation, DOI: 10.13140 / RG.2.2.32286.66887).

[0154] An artificial neural network may be, or may include, a recurrent neural network. A recurrent neural network, or feedback neural network, is, in contrast to a feedforward network, a neural network in which neurons in one layer are distinguished by their connections to neurons in the same layer, or by their connections to neurons in the preceding layer. An artificial neural network may include, for example, long short-term memory (LSTM) (see, for example, Y. Gao et al.: Fully convolutional structured LSTM networks for joint 4D medical image segmentation, DOI: 10.1109 / ISBI.2018.8363764).

[0155] The artificial neural network may also be a transformer network (see, for example, D. Karimi et al.: Convolution-Free Medical Image Segmentation using Transformers, arXiv:2102.13645 [eess.IV]).

[0156] Figure 8 schematically illustrates the computer system described herein as an example.

[0157] A "computer system" is an electronic data processing system that processes data according to programmable computational rules. Such a system typically includes a "computer," which is a unit that includes a processor and peripherals for performing logical operations.

[0158] In computer technology, "peripheral devices" refer to all devices that are connected to a computer and used to control the computer, and / or used as input / output devices. Examples of peripheral devices include monitors (screens), printers, scanners, mice, keyboards, drives, cameras, microphones, speakers, etc. Internal ports and expansion cards are also considered peripheral devices in computer technology.

[0159] The computer system (1) shown in Figure 8 includes an input unit (10), a control calculation unit (20), and an output unit (30).

[0160] The control calculation unit (20) functions to control the computer system (1), to coordinate the data flow between units of the computer system (1), and to perform calculations.

[0161] The control calculation unit (20) is -Through the input unit (10), multiple states Z1~Z n One state Z of a sequence consisting of p R is an expression representing the test area within the subject of the test. p It is configured to receive, where n is an integer greater than or equal to 3, and p is an integer less than n. - Received expression R p It is configured to supply the first representation to a trained machine learning model, where the machine learning model uses the training data to generate the first representation T Starting from R1, n-1 representations TR2 * ~ T R n * It is trained to generate them one after another, and here, the first expression T R1 represents the examination region in state Z1, and each generated representation T R j * State Z j This represents the inspection area, where j is an index that passes through numbers from 2 to n, and represents T R2 * This is, at least in part, an expression. T Generated based on R1, and further each representation T R k * This is, at least partially, the generated expression T R k-1 * It is generated based on the following, where k is an index that passes through numbers from 3 to n, - The machine learning model is configured to receive one or more representations of the inspection domain, where each of the one or more representations is state Z. p This represents the inspection area in one state that follows. -The system is configured to output one or more representations received from a machine learning model to an output unit (30), and / or store them, and / or transmit them to another computer system.

[0162] Figure 9 schematically illustrates a further embodiment of the computer system according to the present invention as an example.

[0163] The computer system (1) includes a processing unit (21) connected to a memory (22). The processing unit (21) and the memory (22) form a control calculation unit, as shown in Figure 8.

[0164] The processing unit (21) may include one or more processors, either alone or in combination with one or more memories. The processing unit (21) may also be ordinary computer hardware capable of processing information such as digital images (e.g., representations relating to inspection areas), computer programs, and / or other digital information. The processing unit (21) typically consists of an array of electronic circuits, some of which can be designed as an integrated circuit or as a group of interconnected integrated circuits (integrated circuits are sometimes also referred to as "chips"). The processing unit (21) may be configured to execute computer programs that can be stored in the working memory of the processing unit (21), or computer programs that can be stored in the memory (22) of the same computer system or a different computer system.

[0165] Memory (22) may be ordinary computer hardware and may temporarily and / or permanently store information such as digital images (e.g., representations relating to an inspection area), data, computer programs, and / or other digital information. Memory (22) may include volatile memory and / or non-volatile memory and may be non-removable or removable. Suitable examples of memory include RAM (random access memory), ROM (read-only memory), hard disks, flash memory, replaceable computer floppy disks, optical disks, magnetic tapes, or combinations thereof. Optical disks may include compact disks with read-only memory (CD-ROMs), compact disks with read / write capabilities (CD-R / Ws), DVDs, Blu-ray discs, and similar types.

[0166] The processing unit (21) may be connected not only to the memory (22), but also to one or more interfaces (11, 12, 31, 32, 33) for displaying, transmitting, and receiving information. These interfaces may include one or more communication interfaces (32, 33) and / or one or more user interfaces (11, 12, 31). One or more communication interfaces (32, 33) may be configured to transmit and / or receive information over, for example, an MRI scanner, CT scanner, ultrasound camera, other computer systems, networks, data memory, or the like. One or more communication interfaces (32, 33) may be configured to transmit and / or receive information over physical (wired) communication connections and / or wireless communication connections. One or more communication interfaces (32, 33) may include one or more interfaces for connecting to a network using, for example, cellular, Wi-Fi, satellite, cable, DSL, optical fiber, and / or the like. In some examples, one or more communication interfaces (32, 33) may include one or more near-field communication interfaces configured to connect devices having near-field communication technologies such as NFC, RFID, Bluetooth, Bluetooth LE, ZigBee, infrared (e.g., IrDA), or the like.

[0167] The user interface (11, 12, 31) may include a display (31). The display (31) may be configured to display information to the user. Preferred examples of the display are liquid crystal displays (LCDs), light-emitting diode displays (LEDs), plasma display panels (PDPs), or the like. The user input interface(s)(11, 12) may be wired or wireless and may be configured to receive information from the user within the computer system(1) for processing, storage, and / or display. Preferred examples of the user input interface are a microphone, an image or video recording device (e.g., a camera), a keyboard or keypad, a joystick, a touch surface (separate from or integrated with a touchscreen), or the like. In some examples, the user interface(s)(11, 12, 31) may include an automated identification data capture (AIDC) technique for machine-readable information. This may include barcodes, radio frequency identification (RFID), magnetic strips, optical character recognition (OCR), integrated circuit cards (ICC), and the like. The user interface (11, 12, 31) may further include one or more interfaces for communicating with peripheral devices such as printers and the like.

[0168] One or more computer programs (40) may be stored in memory (22), and such computer programs may be executed by a processing unit (21), in which case the processing unit (21) is programmed by such computer programs to perform the functions described herein. The acquisition, loading, and execution of instructions of the computer programs (40) may be performed sequentially, such that each instruction is acquired, loaded, and executed. However, the acquisition, loading, and / or execution may also be performed in parallel.

[0169] The machine learning model of this disclosure may also be stored in memory (22).

[0170] The computer system of this disclosure may be designed as a laptop, notebook, netbook, and / or tablet PC, or it may be a component of an MRI scanner, CT scanner, or ultrasound diagnostic device.

[0171] Figure 10 schematically illustrates one embodiment of a method for training a machine learning model in the form of a flowchart.

[0172] Method (100) is, (110) Receiving training data, • Training data represents representations of the test area across a large number of test subjects. T R1~ T R n This includes, where n is an integer greater than or equal to 3, ·Each expression T R i This refers to multiple states Z1~Z n One state Z of a sequence consisting of i This represents the inspection region, where i is an index passing through integers from 1 to n. (120) Training a machine learning model, • Models are representations T Starting from R1, express T R2 * ~ T R n * Trained to generate them one after another, ·Expression T R2 * This is, at least in part, an expression. T Generated based on R1, and further each representation T R k * This is, at least in part, an expression. T R k-1 * It is generated based on the following, where k is an index that passes through numbers from 3 to n, ·Expression T R k * State Z k This represents the inspection area and expression T R k-1 * State Z k-1 This represents the inspection area in state Z. k-1 State Z k It directly precedes, • Generated expression T R j * and express T R j The difference between this and the result is quantified by the loss function, where j is an index that passes through numbers from 2 to n. The difference can be minimized by changing the model parameters of the machine learning model. (130) including storing a trained machine learning model and / or using a machine learning model for prediction.

[0173] Figure 11 schematically illustrates, in flowchart form, one embodiment of a computer implementation method for generating a representation of a certain state.

[0174] Method (200) is, (210) Expressions related to the test area in the subject of testing R p It means receiving, ·Expression R p This refers to multiple states Z1~Z n One state Z of a sequence consisting of p This represents the test region, where p is an integer less than n, and n is an integer greater than or equal to 3. (220) Expression R p This involves supplying it to a pre-trained machine learning model. • The machine learning model was trained according to method (100), (230) From the machine learning model, one or more representations of the inspection domain R p+q *It means receiving, • One or more expressions R p+q * Each of these is a single state Z p+q This represents the inspection area. • q is an index that passes through the numbers 1 to m. The conditions are that m is an integer less than n-1, an integer equal to n-1, or an integer greater than n-1. (240) One or more expressions R p+q * This includes outputting, and / or storing, and / or transmitting.

Claims

1. A computer implementation method for training machine learning models, - This involves receiving training data. - The aforementioned training data represents multiple representations of the test area in a large number of test subjects. Ｔ R １ ~ Ｔ R ｎ It includes n, where n is an integer greater than or equal to 3. ・Each expression Ｔ R ｉ represents the inspection area in one state Z of a sequence consisting of a plurality of radiation inspection states Z １ ~Z ｎ and the sequence consisting of the plurality of radiation inspection states is a time-series sequence consisting of each time point before and / or during and / or after administration of a contrast agent, and i is an index that passes through integers from 1 to n ｉ ​ To receive, - This involves training the aforementioned machine learning model. - The above model represents Ｔ R １ Starting from, with respect to each of the numerous test subjects mentioned above, multiple expressions Ｔ R ２ ＊ ~ Ｔ R ｎ ＊ It generates these sequentially and is trained to learn the temporal contrast enhancement dynamics corresponding to the diffusion of the administered contrast agent within the examination area. ・Expression Ｔ R ２ ＊ This is, at least in part, an expression. Ｔ R １ Generated based on, and each further expression Ｔ R ｋ ＊ These are, at least in part, corresponding to previously generated expressions. Ｔ R ｋ－１ ＊ It is generated based on the following, where k is an index that passes through numbers from 3 to n. ・Expression Ｔ R ｋ ＊ State Z ｋ This represents the aforementioned inspection area and expresses Ｔ R ｋ－１ ＊ State Z ｋ－１ This represents the inspection area in state Z. ｋ－１ The above-mentioned multiple radiation examination states Z １ ~Z ｎ State Z in the sequence consisting of ｋ It directly precedes the generated expression Ｔ R ｋ ＊ This depicts the temporal contrast enhancement dynamics corresponding to the diffusion of the administered contrast agent within the examination area across a series of consecutive states. Training and -Storing the trained machine learning model and / or using the machine learning model for prediction, Computer implementation methods including

2. - Each generated expression Ｔ R ｊ ＊ Regarding the received expression Ｔ R ｊ and the generated expression Ｔ R ｊ ＊ For a pair consisting of and , the step is to calculate the loss value using a loss function, where j is an index that passes through numbers from 2 to n, and the step is to calculate - A step of calculating the total loss value using a total loss function, wherein the total loss function is a function of the loss value, - A step of minimizing the total loss value by changing the parameters of the machine learning model, The method according to claim 1, further comprising:

3. The total loss function is given by the following equation [Math 1] It has, LV is the total loss value, LF ｊ The received expression Ｔ R ｊ and the generated expression Ｔ R ｊ ＊ This is a loss function for calculating the loss value related to the difference between the two, and w ｊ This is the weighting coefficient, The method according to claim 2.

4. A computer implementation method for predicting one or more representations relating to a test area in a test object, - Expression R relating to the inspection area ｐ It means receiving, ・Expression R ｐ This refers to multiple radiation examination conditions Z １ ~Z ｎ One state Z of a sequence consisting of ｐ The above represents the examination area, and the sequence consisting of the multiple radiographic examination states is a time-series sequence consisting of the points in time before and / or during and / or after the administration of the contrast agent, where p is an integer smaller than n, and n is an integer of 3 or more, and the representation R ｐ This is the aforementioned state Z １ ~Z ｎ Representation of training data used to train a machine learning model with respect to the representation format and state index within a sequence consisting of Ｔ R ｐ It supports To receive, -Expression R ｐ This involves supplying it to a pre-trained machine learning model. - The trained machine learning model, based on the training data, determines the multiple radiation examination states Z １ ~Z ｎ In a sequence consisting of the above, the first representation Ｔ R １ Starting from, n-1 expressions Ｔ R ２ ＊ ~ Ｔ R ｎ ＊ It generates these sequentially and is trained to learn the temporal contrast enhancement dynamics corresponding to the diffusion of the administered contrast agent within the examination area. ・First expression Ｔ R １ This is the first state Z １ This represents the aforementioned inspection area, and each generated expression Ｔ R ｊ ＊ State Z ｊ This represents the aforementioned inspection area, where j is an index that passes through numbers from 2 to n. ・Expression Ｔ R ２ ＊ is at least partially generated based on the expression Ｔ R １ Each further expression Ｔ R ｋ ＊ is at least partially generated based on the previously generated corresponding expression Ｔ R ｋ－１ ＊ where k is an index that passes through the numbers from 3 to n, and each generated expression Ｔ R ｋ ＊ depicts the temporal contrast enhancement dynamics corresponding to the diffusion of the administered contrast agent within the examination region over a plurality of consecutive states The aforementioned training data is the result of a radiation examination. To supply, - From the machine learning model, one or more representations R relating to the inspection area ｐ＋ｑ ＊ It means receiving, - The one or more expressions R ｐ＋ｑ ＊ Each of these is in state Z ｐ＋ｑ The aforementioned inspection region is represented, where q is an index passing through numbers from 1 to m, and m is an integer smaller than n-1, an integer equal to n-1, or an integer greater than n-1, and the one or more representations R ｐ＋ｑ ＊ Each of them is in state Z ｐ＋ｑ To depict the temporal contrast enhancement dynamics corresponding to the diffusion of the administered contrast agent within the examination area, To receive, - The one or more expressions R ｐ＋ｑ ＊ To output and / or store and / or transmit, Computer implementation methods including

5. The aforementioned method, - Expression R relating to the inspection area ｐ Receiving, expression R ｐ The plurality of states Z １ ~Z ｎ State Z in the sequence consisting of the above ｐ This represents the aforementioned inspection area, where p is an integer smaller than n, and it is received. -Expression R ｐ The means to supply the said trained machine learning model, the said trained machine learning model being trained by the method of claim 1, - From the trained machine learning model, one or more representations R relating to the inspection area ｐ＋ｑ ＊ This means receiving the aforementioned one-term expression R ｐ＋ｑ ＊ Each of these is in state Z ｐ＋ｑ The aforementioned inspection area is represented, where q is an index passing through numbers from 1 to m, and m is an integer smaller than n-1, an integer equal to n-1, or an integer greater than n-1, and is received. - The one or more expressions R ｐ＋ｑ ＊ To output and / or store and / or transmit, The method according to claim 4, including the method described in claim 4.

6. Each expression R １＋ｑ ＊ The following formula M ｑ (R １ )=R １＋ｑ ＊ It is calculated according to, M represents the transformation applied by the machine learning model to the input data of the machine learning model, ｑ This means that the transformation is applied q times, and in the first step, the transformation is applied to the first representation R １ The process is applied to the first step, and in the second step, the transformation is applied to the result of the first step, in which case the result is returned to the machine learning model, and the procedure is repeated for each of the corresponding further transformation results until the transformation is applied a total of q times, where q is an integer that can take values ​​from 1 to m, and m is an integer less than n-1, or an integer equal to n-1, or an integer greater than n-1. The method according to claim 1.

7. The sequence consisting of the plurality of states is - The first state of the examination area at the first time point before administration of the contrast agent, - The second state of the examination area at a second time point after administration of the contrast agent, - The third state of the examination area at the third time point after administration of the contrast agent, - The fourth state of the examination area at the fourth time point after administration of the contrast agent, - The fifth state of the examination area at the fifth time point after administration of the contrast agent, This includes one or more of the following states: The method according to claim 1, wherein the first time point, the second time point, the third time point, the fourth time point, and the fifth time point form a time series.

8. The method according to claim 1, wherein the area being examined is a human liver or a part of the human liver.

9. The sequence consisting of the plurality of states is - Before administration of hepatobiliary contrast agent, the liver or a part of the liver, - The liver or a portion of the liver during the arterial phase after administration of the hepatobiliary contrast agent. - During the portal venous phase following administration of the hepatobiliary contrast agent, the liver or a portion of the liver - During the transition phase following administration of the hepatobiliary contrast agent, the liver or a portion of the liver - During the hepatocyte contrast phase following administration of the hepatobiliary contrast agent, the liver or a portion of the liver The method according to claim 8, comprising one or more of the following states.

10. Each representation R generated by the aforementioned machine learning model ｐ＋ｑ ＊ and / or Ｔ R ｐ＋ｑ ＊ is, expression R ｐ＋ｑ＋１ ＊ and / or Ｔ R ｐ＋ｑ＋１ ＊ The method according to claim 1, wherein the trained machine learning model is returned to generate the following.

11. The method according to claim 4, wherein m is located in the range of n to n+2.

12. The method according to claim 1, wherein the received representation is a CT image, an MRI image, or an ultrasound image.

13. A computer system, - Input unit and, • Control calculation unit, • Output unit and, Includes, The control calculation unit is - The input unit receives an expression R relating to the inspection area of ​​the object to be inspected. ｐ It is configured to receive, and expression R ｐ This refers to multiple radiation examination conditions Z １ ~Z ｎ One state Z of a sequence consisting of ｐ The above represents the examination area, and the sequence consisting of the multiple radiographic examination states is a time-series sequence consisting of the points in time before and / or during and / or after the administration of the contrast agent, where p is an integer smaller than n, and n is an integer of 3 or more, and the representation R ｐ This is the aforementioned state Z １ ~Z ｎ Representation of training data used to train a machine learning model with respect to the representation format of a sequence consisting of and state index Ｔ R ｐ It supports, -Expression R ｐ The system is configured to supply the trained machine learning model with the multiple radiation examination states Z based on the training data. １ ~Z ｎ In a sequence consisting of the above, the first representation Ｔ R １ Starting from, n-1 expressions Ｔ R ２ ＊ ~ Ｔ R ｎ ＊ It generates these one after another and is trained to learn the temporal contrast enhancement dynamics corresponding to the diffusion of the administered contrast agent within the examination area, and the first expression Ｔ R １ This is the first state Z １ This represents the aforementioned inspection area, and each generated expression Ｔ R ｊ ＊ State Z ｊ This represents the aforementioned inspection area, where j is an index that passes through numbers from 2 to n, and represents Ｔ R ２ ＊ This is, at least in part, an expression. Ｔ R １ Generated based on, and each further expression Ｔ R ｋ ＊ These are, at least in part, corresponding to previously generated expressions. Ｔ R ｋ－１ ＊ It is generated based on, where k is an index that passes through numbers from 3 to n, and each generated representation Ｔ R ｋ ＊ This depicts the temporal contrast enhancement dynamics corresponding to the diffusion of the administered contrast agent within the examination area across a series of consecutive states. - From the machine learning model, one or more representations R relating to the inspection area ｐ＋ｑ ＊ It is configured to receive the one or more expressions R ｐ＋ｑ ＊ Each of these is in state Z ｐ＋ｑ The aforementioned inspection region is represented, where q is an index passing through numbers from 1 to m, and m is an integer smaller than n-1, an integer equal to n-1, or an integer greater than n-1, and the one or more representations R ｐ＋ｑ ＊ Each of them is in state Z ｐ＋ｑ The temporal contrast enhancement dynamics corresponding to the diffusion of the administered contrast agent within the examination area are depicted. - The one or more expressions R ｐ＋ｑ ＊ It is configured to output to the output unit, and / or store, and / or transmit to another computer system. Computer system.

14. A computer-readable recording medium that stores a computer program that can be loaded into the working memory of a computer system, The computer program is installed on the computer system. - Expression R regarding the test area in the subject of the test ｐ This is a step to receive the expression R ｐ This refers to multiple radiation examination conditions Z １ ~Z ｎ One state Z of a sequence consisting of ｐ The above represents the examination area, and the sequence consisting of the multiple radiographic examination states is a time-series sequence consisting of the points in time before and / or during and / or after the administration of the contrast agent, where p is an integer smaller than n, and n is an integer of 3 or more, and the representation R ｐ This is the aforementioned state Z １ ~Z ｎ Representation of training data used to train a machine learning model with respect to the representation format and state index within a sequence consisting of Ｔ R ｐ The steps to receive the item, which are compatible with this process, -Expression R ｐ This is a step of supplying the above to a trained machine learning model, and the machine learning model, based on the training data, determines the plurality of radiation examination states Z １ ~Z ｎ In a sequence consisting of the above, the first representation Ｔ R １ Starting from, n-1 expressions Ｔ R ２ ＊ ~ Ｔ R ｎ ＊ It generates these one after another and is trained to learn the temporal contrast enhancement dynamics corresponding to the diffusion of the administered contrast agent within the examination area, and the first expression Ｔ R １ This is the first state Z １ This represents the aforementioned inspection area, and each generated expression Ｔ R ｊ ＊ State Z ｊ This represents the aforementioned inspection area, where j is an index that passes through numbers from 2 to n, and represents Ｔ R ２ ＊ This is, at least in part, an expression. Ｔ R １ Generated based on, and each further expression Ｔ R ｋ ＊ These are, at least in part, corresponding to previously generated expressions. Ｔ R ｋ－１ ＊ It is generated based on, where k is an index that passes through numbers from 3 to n, and each generated representation Ｔ R ｋ ＊ The process involves supplying a contrast agent to depict the temporal contrast enhancement dynamics corresponding to the diffusion of the administered contrast agent within the examination area across a series of consecutive states. - From the machine learning model, one or more representations R relating to the inspection area ｐ＋ｑ ＊ This is a step of receiving the one or more expressions R ｐ＋ｑ ＊ Each of these is in state Z ｐ＋ｑ The aforementioned inspection region is represented, where q is an index passing through numbers from 1 to m, and m is an integer smaller than n-1, an integer equal to n-1, or an integer greater than n-1, and the one or more representations R ｐ＋ｑ ＊ Each of them is in state Z ｐ＋ｑ A step of depicting and receiving the temporal contrast enhancement dynamics corresponding to the diffusion of the administered contrast agent within the examination area, - The one or more expressions R ｐ＋ｑ ＊ The steps include causing it to output and / or store and / or transmit, A recording medium that enables execution.

15. The use of contrast agents in radiological examinations, The aforementioned radiation examination method is, - Expression R regarding the test area in the subject of the test ｐ This means receiving the above expression R ｐ This refers to multiple radiation examination conditions Z １ ~Z ｎ One state Z of a sequence consisting of ｐ This represents the examination area before and / or during and / or after the administration of the contrast agent, and the sequence of the plurality of radiographic examination states is a time-series sequence consisting of the points in time before and / or during and / or after the administration of the contrast agent, where p is an integer smaller than n, and n is an integer of 3 or more, and the representation R ｐ This is the aforementioned state Z １ ~Z ｎ Representation of training data used to train a machine learning model with respect to the representation format and state index within a sequence consisting of Ｔ R ｐ It corresponds to receiving, -Expression R ｐ This involves supplying the trained machine learning model with the multiple radiation examination states Z based on the training data. １ ~Z ｎ In a sequence consisting of the above, the first representation Ｔ R １ Starting from, n-1 expressions Ｔ R ２ ＊ ~ Ｔ R ｎ ＊ It generates these one after another and is trained to learn the temporal contrast enhancement dynamics corresponding to the diffusion of the administered contrast agent within the examination area, and the first expression Ｔ R １ This is the first state Z １ This represents the aforementioned inspection area, and each generated expression Ｔ R ｊ ＊ State Z ｊ This represents the aforementioned inspection area, where j is an index that passes through numbers from 2 to n, and represents Ｔ R ２ ＊ This is, at least in part, an expression. Ｔ R １ Generated based on, and each further expression Ｔ R ｋ ＊ These are, at least in part, corresponding to previously generated expressions. Ｔ R ｋ－１ ＊ It is generated based on, where k is an index that passes through numbers from 3 to n, and each generated representation Ｔ R ｋ ＊ This involves supplying and depicting the temporal contrast enhancement dynamics corresponding to the diffusion of the administered contrast agent within the examination area across a series of consecutive states. - From the machine learning model, one or more representations R relating to the inspection area ｐ＋ｑ ＊ This means receiving the aforementioned one-term expression R ｐ＋ｑ ＊ Each of these is in state Z ｐ＋ｑ The aforementioned inspection region is represented, where q is an index passing through numbers from 1 to m, and m is an integer smaller than n-1, an integer equal to n-1, or an integer greater than n-1, and the one or more representations R ｐ＋ｑ ＊ Each of them is in state Z ｐ＋ｑ To depict and receive the temporal contrast enhancement dynamics corresponding to the diffusion of the administered contrast agent within the examination area, - The one or more expressions R ｐ＋ｑ ＊ To output and / or store and / or transmit, Use of contrast agents containing [specific components].

16. A kit comprising a contrast agent and a computer program product, The computer program product includes a computer program that can be loaded into the working memory of a computer system, and the computer program is loaded into the computer system. - Expression R regarding the test area in the subject of the test ｐ This is a step to receive the expression R ｐ This refers to multiple radiation examination conditions Z １ ~Z ｎ One state Z of a sequence consisting of ｐ This represents the examination area before and / or during and / or after the administration of the contrast agent, and the sequence of the plurality of radiographic examination states is a time-series sequence consisting of the points in time before and / or during and / or after the administration of the contrast agent, where p is an integer smaller than n, and n is an integer of 3 or more, and the representation R ｐ This is the aforementioned state Z １ ~Z ｎ Representation of training data used to train a machine learning model with respect to the representation format and state index within a sequence consisting of Ｔ R ｐ The steps to receive the item, which are compatible with this process, -Expression R ｐ This is a step of supplying the above to a trained machine learning model, and the machine learning model, based on the training data, determines the plurality of radiation examination states Z １ ~Z ｎ In a sequence consisting of the above, the first representation Ｔ R １ Starting from, n-1 expressions Ｔ R ２ ＊ ~ Ｔ R ｎ ＊ It generates these one after another and is trained to learn the temporal contrast enhancement dynamics corresponding to the diffusion of the administered contrast agent within the examination area, and the first expression Ｔ R １ This is the first state Z １ This represents the aforementioned inspection area, and each generated expression Ｔ R ｊ ＊ State Z ｊ This represents the aforementioned inspection area, where j is an index that passes through numbers from 2 to n, and represents Ｔ R ２ ＊ This is, at least in part, an expression. Ｔ R １ Generated based on, and each further expression Ｔ R ｋ ＊ These are, at least in part, corresponding to previously generated expressions. Ｔ R ｋ－１ ＊ It is generated based on, where k is an index that passes through numbers from 3 to n, and each generated representation Ｔ R ｋ ＊ The process involves supplying a contrast agent to depict the temporal contrast enhancement dynamics corresponding to the diffusion of the administered contrast agent within the examination area across a series of consecutive states. - One or more expressions R relating to the inspection area ｐ＋ｑ ＊ This is a step of receiving the one or more expressions R ｐ＋ｑ ＊ Each of these is in state Z ｐ＋ｑ The aforementioned inspection region is represented, where q is an index passing through numbers from 1 to m, and m is an integer smaller than n-1, an integer equal to n-1, or an integer greater than n-1, and the one or more representations R ｐ＋ｑ ＊ Each of them is in state Z ｐ＋ｑ A step of depicting and receiving the temporal contrast enhancement dynamics corresponding to the diffusion of the administered contrast agent within the examination area, - The one or more expressions R ｐ＋ｑ ＊ The steps include causing it to output and / or store and / or transmit, A kit to perform the task.

17. The method according to claim 1, wherein each subject of examination is a human being, and the examination area is a part of the human being.

Images