Identifying artifacts in synthetic medical records

Abstract

The present invention relates to the art of generating synthetic medical records, and more particularly to a computer-readable storage medium including a method, a computer system, and a computer program for identifying artifacts in synthetic medical records.

Description

[Technical Field]

[0001] Copyright Notice Parts of the disclosure in this patent document include material protected by copyright. The copyright holder does not object to the facsimile reproduction of the patent document as it appears in the patent files or records of the Japan Patent Office, but reserves all other copyrights and rights. Copyright 2023 Bayer Ltd.

[0002] This disclosure relates to the technical field of generating synthetic medical images. The subject matter of this disclosure is a computer-readable storage medium including a method, computer system, and computer program for detecting artifacts in synthetic medical images.

[0003] Introduction Artificial intelligence is increasingly being adopted in medicine. Machine learning models are being used not only to identify signs of disease in medical images of human or animal bodies (see, for example, International Publication No. 2018202541 and International Publication No. 2020229152), but also increasingly to generate synthetic (artificial) medical images.

[0004] For example, International Publication Nos. 2021052896 and 2021069338 describe a method for generating artificial medical images showing the examination area of ​​a subject to be examined during a first period (stage). The artificial medical images are generated using a trained machine learning model based on medical images showing the examination area during a second period (stage). This method can be used, for example, to speed up radiological examinations. Instead of measuring radiological images over a relatively long period, measurements are performed only within a portion of a certain period, and one or more radiological images are predicted for the rest of that period using a trained model.

[0005] For example, International Publication Nos. 2019 / 074938 and 2022184297 describe a method for generating an artificial radiographic image showing the examination area after administration of a standard dose of contrast agent, even when only a smaller dose than the standard dose is administered. The standard dose is the amount recommended by the manufacturer and / or distributor of the contrast agent, and / or the amount approved by the regulatory authority, and / or the amount specified in the contrast agent's package leaflet. Therefore, the amount of contrast agent can be reduced using the method described in International Publication Nos. 2019 / 074938 and 2022184297.

[0006] Medical images generated by trained machine learning models may contain errors (see, for example, K. Schwarz et al., On the Frequency Bias of Generative Models, https: / / doi.org / 10.48550 / arXiv.2111.02447). [Overview of the Initiative] [Problems that the invention aims to solve]

[0007] Such errors can be problematic because physicians may make diagnoses and / or initiate treatment based on artificial medical images. When reviewing artificial medical images, physicians need to know whether the features in the artificial medical image are due to the actual features of the subject being examined, or whether the features in the artificial medical image are artifacts resulting from prediction errors by trained machine learning models. [Means for solving the problem]

[0008] These and other issues are addressed by the subject matter of this disclosure.

[0009] This disclosure, in the first embodiment, The steps include receiving at least one image of the area to be examined, A step of generating a number of sub-images based on at least one received image, wherein each sub-image represents a sub-region of the inspection area to be inspected, and the sub-regions represented by different sub-images partially overlap but do not completely overlap. A step of generating multiple composite subimages by a generative model based at least partially on the generated subimage, A step of determining the color values ​​of corresponding image elements in a composite partial image, wherein the corresponding image elements represent the same sub-region of the inspection area. The steps include determining the scale of the variance of the color values ​​of the corresponding image elements, The steps include determining the confidence value based on the measure of variance, Steps to output confidence values, The present invention provides a computer implementation method for generating at least one confidence value of a composite image, including [specific data / features].

[0010] This disclosure is, Processor and Memory that stores application programs configured to perform actions when executed by the processor and The operation is provided, The steps include receiving at least one image of the area to be examined, A step of generating a number of sub-images based on at least one received image, wherein each sub-image represents a sub-region of the inspection area to be inspected, and the sub-regions represented by different sub-images partially overlap but do not completely overlap. A step of generating multiple composite subimages by a generative model based at least partially on the generated subimage, A step of determining the color values ​​of corresponding image elements in a composite partial image, wherein the corresponding image elements represent the same sub-region of the inspection area. The steps include determining the scale of the variance of the color values ​​of the corresponding image elements, The steps include determining the confidence value based on the measure of variance, Steps to output confidence values, We will further provide computer systems, including the following.

[0011] This disclosure further provides a computer-readable storage medium containing a computer program that can be loaded into the working memory of a computer system, the computer program being loaded into the computer system The steps include receiving at least one image of the area to be examined, A step of generating a number of sub-images based on at least one received image, wherein each sub-image represents a sub-region of the inspection area to be inspected, and the sub-regions represented by different sub-images partially overlap but do not completely overlap. A step of generating multiple composite subimages by a generative model based at least partially on the generated subimage, A step of determining the color values ​​of corresponding image elements in a composite partial image, wherein the corresponding image elements represent the same sub-region of the inspection area. The steps include determining the scale of the variance of the color values ​​of the corresponding image elements, The steps include determining the confidence value based on the measure of variance, Steps to output confidence values, Make it run. [Brief explanation of the drawing]

[0012] [Figure 1] This diagram schematically illustrates the generation of a partial image based on the received image of the area being inspected. [Figure 2] This paper schematically illustrates the generation of composite sub-images based on sub-images using a generative model, and the merging of composite sub-images to form a composite image. [Figure 3] This diagram schematically illustrates the process of combining composite images to form a combined composite image. [Figure 4] This diagram schematically illustrates the process of combining composite images to form a combined composite image. [Figure 5a]A schematic example of a method for generating partial images from each of multiple received images, and a method for generating a composite partial image based on the generated partial images, is provided. [Figure 5b] A schematic example of a method for forming a composite image by merging composite partial images, and a method for forming a combined composite image by combining composite images, are shown. [Figure 6] The determination of at least one confidence value and the generation of a confidence expression are schematically shown as an example. [Figure 7] One embodiment of the method disclosed herein is shown in flowchart form. [Figure 8] A schematic example of a method for training such a generative machine learning model is provided. [Figure 9] A schematic example of a computer system provided in this disclosure is shown below. [Figure 10] Further embodiments of the computer system of this disclosure are schematically shown as examples. [Modes for carrying out the invention]

[0013] The present invention will be described in more detail below without distinguishing between the subject matter of this disclosure (methods, computer systems, computer-readable storage media). Rather, the following descriptions shall apply mutatis mutandis to all subject matter of the present invention, regardless of the context in which they are described (methods, computer systems, computer-readable storage media).

[0014] Where steps are described in order in this specification or in the claims, this does not necessarily mean that the invention is limited to the order described. Instead, steps may be performed in a different order or in parallel with each other, unless it is absolutely necessary that one step builds upon another and that the step built upon the previous step is subsequently performed (however, this will become apparent in the individual cases). Thus, the order described is a preferred embodiment of this disclosure.

[0015] In certain places, the present invention will be described in more detail with reference to the drawings. The drawings show specific embodiments having particular features and combinations of features, which are primarily illustrative. The present invention should not be understood as being limited to the features and combinations of features shown in the drawings. Furthermore, the descriptions made in the description of the drawings in relation to features and combinations of features are intended to be generally applicable, that is, applicable to other embodiments and not limited to the embodiments shown.

[0016] This disclosure describes a means for determining the reliability of a composite image of an inspection area to be inspected.

[0017] The term "reliability" is understood to mean that a person examining a composite image can trust that the structures and / or forms and / or textures depicted in the composite image are attributable to the actual structures and / or forms and / or textures of the area being examined, and are not artifacts.

[0018] The term “synthesized” means that the synthesized image is not a direct result of measurements on an actual object being examined, but is artificially generated (calculated). However, the synthesized image can be based on images of an actual object being examined; that is, one or more images of an actual object being examined can be used to generate the synthesized image. Examples of synthesized images are described in the introduction and further description of this disclosure. According to this disclosure, the synthesized image is generated by a machine learning model. The generation of a synthesized image using a machine learning model is also referred to herein as “prediction.” The terms “synthesized” and “prediction” are used synonymously in this disclosure. In other words, the synthesized image is an image generated by a (trained) machine learning model based on input data that may include one or more images generated by measurements.

[0019] The "subject of examination" is preferably a human or animal, preferably a mammal, and most preferably a human.

[0020] A “test area” is a part of the object being tested, for example, an organ of a human or animal, such as the liver, brain, heart, kidney, lung, stomach, intestine, pancreas, thyroid, prostate, breast, or a part of one of the aforementioned organs, or multiple organs, or another part of the object being tested. A test area may also include multiple organs and / or parts of multiple organs.

[0021] In one embodiment, the examination area includes the liver or a portion of the liver, or the examination area is the liver or a portion of the liver of a mammal, preferably a human.

[0022] In further embodiments, the examination area includes the brain or a portion of the brain, or the examination area is the brain or a portion of the brain of a mammal, preferably a human.

[0023] In further embodiments, the examination area includes the heart or a portion of the heart, or the examination area is the heart or a portion of a mammal, preferably a human heart.

[0024] In further embodiments, the examination area includes the chest or a portion of the chest, or the examination area is the chest or a portion of the chest of a mammal, preferably a human.

[0025] In further embodiments, the examination area includes the stomach or a portion of the stomach, or the examination area is the stomach or a portion of the stomach of a mammal, preferably a human.

[0026] In further embodiments, the examination area includes the pancreas or a portion of the pancreas, or the examination area is the pancreas or a portion of the pancreas of a mammal, preferably a human.

[0027] In further embodiments, the examination area includes a kidney or a portion of a kidney, or the examination area is a mammalian, preferably human, kidney or a portion of a kidney.

[0028] In further embodiments, the examination area includes one or both lungs or a portion of the lungs of a mammal, preferably a human.

[0029] In further embodiments, the examination area includes a breast or a portion of a breast, or the examination area is a breast or a portion of a breast of a female mammal, preferably a female human.

[0030] In further embodiments, the examination area includes the prostate or a portion of the prostate, or the examination area is the prostate or a portion of the prostate of a male mammal, preferably a male human.

[0031] The examination area, also known as the field of view (FOV), is specifically the volume imaged in a radiographic image. The examination area is typically defined by a radiologist, for example, on a localizer image. Of course, the examination area can also be defined alternatively or additionally in an automated manner, for example, based on a selected protocol.

[0032] The term "image" refers to a data structure that constitutes the spatial distribution of a physical signal. The spatial distribution can have any dimension, e.g., 2D, 3D, 4D, or higher dimensions. The spatial distribution can have any form, for example, it can form a grid that can be irregular or regular, thereby defining pixels or voxels. The physical signal can be any signal, e.g., proton density, echo intensity, transmittance, absorptive capacity, relaxation degree, information about rotating hydrogen nuclei in a magnetic field, color, density level, depth, surface, or volume occupancy.

[0033] The term “image” is preferably understood to mean a visually captureable representation of the inspection area being inspected in two, three, or more dimensions. The received image is typically a digital image. The term “digital” means that the image can be processed by a machine, generally a computer system. “Processing” is understood to mean known methods for electronic data processing (EDP).

[0034] Digital images can be processed, edited, played back, and converted into standardized data formats such as JPEG (Joint Photographic Experts Group Graphics Format), PNG (Portable Network Graphics), or SVG (Scalable Vector Graphics) by computer systems and software. Digital images can be visualized by appropriate display devices such as computer monitors, projectors, and / or printers.

[0035] In digital images, image content is typically represented and stored as integers. Most often, images are two-dimensional or three-dimensional and can be binary-encoded and optionally compressed. Digital images are typically raster graphics, where image information is stored in a uniform raster grid. Raster graphics consist of a raster array of so-called pixels in two-dimensional representations and volume elements (voxels) in three-dimensional representations. In four-dimensional representations, the term doxel (dynamic voxel) is commonly used for image elements. In higher-dimensional representations, or more generally, the term "n-xel" may also be used, where n indicates a specific dimension. This disclosure generally uses the term image element. Therefore, an image element can be an image element (pixel) in two-dimensional representations, a volume element (voxel) in three-dimensional representations, a dynamic voxel (doxel) in four-dimensional representations, or a higher-dimensional image element in higher-dimensional representations.

[0036] Each image element within an image is typically assigned at least one color value. The color value indicates how the image element should be visually displayed (for example, on a monitor or printer) (e.g., in what color).

[0037] The simplest case is a binary image where image elements are displayed as either white or black. A color value of "0" is typically "black," and a color value of "1" is "white."

[0038] In a grayscale image, each image element is assigned a gray level ranging from black to white across a defined number of shades of gray. These gray levels are also called gray values. The number of shades can range, for example, from 0 to 255 (i.e., 256 gray levels / gray values), where a value of "0" is typically "black" and the highest gray value (255 in this example) is "white".

[0039] In the case of color images, the color coding used for image elements is defined, in particular, with respect to the color space and color depth. For an image where the color is defined in terms of the so-called RGB color space (RGB representing the primary colors red, green, and blue), each pixel is assigned three color values: one for red, one for green, and one for blue. The color of an image element is produced by the superposition (additive color mixing) of these three color values. Each individual color value can be discretized into, for example, 256 distinguishable levels, called tonal values, which typically range from 0 to 255. A tonal value of "0" for each color channel is usually the darkest nuance of color. If all three color channels have a tonal value of 0, the corresponding image element appears black; if all three color channels have a tonal value of 255, the corresponding image element appears white.

[0040] Regardless of whether the image is a binary image, a grayscale image, or a color image, the term “color value” is used in this disclosure to indicate the color (including “color,” “black,” and “white,” as well as all shades of gray) that an image element should display. Thus, a color value can be a color channel, a shade of gray, or a tonal value for “black” or “white.”

[0041] The color values ​​of images (especially medical images) typically represent the intensity of the physical signal (see above). Note that "color values" can also refer to the values ​​of the physical signal itself.

[0042] There are numerous possible digital image formats and color codings. For simplicity, this description assumes that the current image is a raster graphic with a certain number of image elements. However, this assumption should never be understood as limiting. To those skilled in image processing, it will be clear how the teachings herein may apply to image files existing in other image formats and / or with differently encoded color values.

[0043] In the context of this disclosure, “image” may also be one or more excerpts from a video sequence.

[0044] In the first step, at least one image of the area to be examined is received.

[0045] The term “received” encompasses both the retrieval of an image and the reception of an image transmitted to, for example, a computer system of this disclosure. At least one image may be received from a computed tomography scanner, a magnetic resonance imaging scanner, an ultrasound scanner, a camera, and / or any other device for generating an image. At least one image may be read from data memory and / or transmitted from a separate computer system.

[0046] Preferably, at least one received image is a two-dimensional or three-dimensional representation of the inspection area to be inspected.

[0047] In one embodiment of this disclosure, at least one received image is a medical image.

[0048] "Medical imaging" is a visual representation of a human or animal examination area that can be used for diagnostic and / or therapeutic purposes.

[0049] There are numerous techniques that can be used to generate medical images. Examples of such techniques include radiography, computed tomography (CT), fluoroscopy, magnetic resonance imaging (MRI), ultrasound (sonography), endoscopy, elastography, tactile imaging, thermography, microscopy, positron emission tomography (POST), optical coherence tomography (OCT), and fundus photography.

[0050] Examples of medical images include CT images, X-ray images, MRI images, fluorescence angiography images, OCT images, histological images, ultrasound images, and fundus images.

[0051] At least one received image may be a CT image, MRI image, ultrasound image, OCT image, and / or any other representation of the area being examined.

[0052] At least one received image may also contain representations of different modalities, such as CT and MRI images.

[0053] Preferably, at least one received image is the result of a radiological examination. "Radiology" is a branch of medicine relating to the use of electromagnetic and mechanical waves (e.g., ultrasound) for diagnostic, therapeutic and / or scientific purposes. In addition to X-rays, other ionizing radiation such as gamma rays or electrons is also used. In imaging, which is an important application, other imaging methods such as ultrasound and magnetic resonance imaging (nuclear magnetic resonance imaging) are also counted as radiology, even though these methods do not use ionizing radiation. Therefore, the term "radiology" in the context of this disclosure includes computed tomography, magnetic resonance imaging, and ultrasound in particular.

[0054] In one embodiment of this disclosure, the radiological examination is magnetic resonance imaging.

[0055] In a further embodiment, the radiological examination is computed tomography.

[0056] In further embodiments, the radiological examination is an ultrasound examination.

[0057] In radiological examinations, contrast agents are commonly used to enhance the contrast effect.

[0058] A "contrast agent" is a substance or mixture of substances that enhances the depiction of the structure and function of the body in radiological examinations.

[0059] In computed tomography, iodine-containing solutions are typically used as contrast agents. In magnetic resonance imaging (MRI), superparamagnetic materials (e.g., iron oxide nanoparticles, superparamagnetic iron-platinum particles (SIPP)) or paramagnetic materials (e.g., gadolinium chelate, manganese chelate, hafnium chelate) are typically used as contrast agents. In ultrasound examinations, liquids containing gas-filled microbubbles are usually administered intravenously. Examples of contrast agents can be found in the literature (e.g., ASL Jascinth et al.: Contrast Agents in computed tomography: A Review, Journal of Applied Dental and Medical Sciences, 2016, vol.2, issue 2, 143-149; H. Lusic et al.: X-ray-Computed Tomography Contrast Agents, Chem. Rev. 2013, 113, 3, 1641-1666; https: / / www.radiology.wisc.edu / wp-content / uploads / 2017 / 10 / contrast-agents-tutorial.pdf; MRNouh et al.: Radiographic and magnetic resonances contrast agents: Essentials and tips for safe practices, World J Radiol. 2017 Sep.28, 9(9):339-349; LCAbonyi et al.: Intravascular Contrast Media in Radiography: Historical Development & Review of Risk Factors for Adverse Reactions, South This can be found in American Journal of Clinical Research, 2016, vol.3, issue 1, 1-10; ACR Manual on Contrast Media, 2020, ISBN:978-1-55903-012-0; A. Ignee et al.: Ultrasound contrast agents, Endosc Ultrasound. 2016 Nov-Dec;5(6):355-362.

[0060] MRI contrast agents exert their effects in MRI examinations by altering the relaxation time of structures that absorb the contrast agent. Two groups of materials can be distinguished: paramagnetic and superparamagnetic. Both groups of materials have unpaired electrons that induce a magnetic field around individual atoms or molecules. Superparamagnetic contrast agents primarily shorten T2, while paramagnetic contrast agents primarily shorten T1. The effect of the contrast agent is indirect because the contrast agent itself does not emit a signal, but rather affects the intensity of signals in its vicinity. An example of a superparamagnetic contrast agent is iron oxide nanoparticles (SPIO, superparamagnetic iron oxide). Examples of paramagnetic contrast agents include gadopentetate dimeglumine (trade name: Magnevist®, etc.), gadoteric acid (Dotarem®, Dotagita®, Cyclolux®), gadodiamide (Omniscan®), gadoteridol (ProHance®), gadobutrol (Gadovist®), gadopicrenol (Elucirem, Vueway), and gadoxetic acid (Primovist® / Eovist®), which are gadolinium chelates.

[0061] In one embodiment, the radiological examination is an MRI examination in which an MRI contrast agent is used.

[0062] In a further embodiment, the radiological examination is a CT scan in which a CT contrast agent is used.

[0063] In a further embodiment, the radiological examination is a CT scan in which an MRI contrast agent is used.

[0064] In one embodiment of the present disclosure, the contrast agent is a drug comprising gadolinium(III)2-[4,7,10tris(carboxymethyl)-1,4,7,10tetrazacyclododeca-1-yl]acetic acid (also known as gadolinium-DOTA or gadoteric acid).

[0065] In further embodiments, the contrast agent is a drug containing gadolinium(III) ethoxybenzyldiethylenetriaminepentaacetic acid (Gd-EOB-DTPA). Preferably, the contrast agent contains the disodium salt of gadolinium(III) ethoxybenzyldiethylenetriaminepentaacetic acid (also known as gadoxetic acid).

[0066] In one embodiment of the present disclosure, the contrast agent is a drug comprising gadolinium(III)2-[3,9-bis[1-carboxylate-4-(2,3-dihydroxypropylamino)-4-oxobutyl]-3,6,9,15tetrazabicyclo[9.3.1]pentadeca-1(15),11,13trien-6-yl]-5-(2,3-dihydroxypropylamino)-5-oxopentanoate (also known as gadopicrenol) (see, for example, International Publication No. 2007 / 042504 and International Publication No. 2020 / 030618 and / or International Publication No. 2022 / 013454).

[0067] In one embodiment of the present disclosure, the contrast agent is a drug comprising gadolinate(2-)dihydrogen[(±)-4-carboxy-5,8,11tris(carboxymethyl)-1-phenyl-2-oxa-5,8,11triazatridecane-13-oato(5-)] (also known as gadobenic acid).

[0068] In one embodiment of the present disclosure, the contrast agent is a drug comprising tetragadolinium acetate [4,10bis(carboxylatomethyl)-7-{3,6,12,15tetraoxo-16-[4,7,10tris-(carboxylatomethyl)-1,4,7,10tetraazacyclododecane-1-yl]-9,9-bis({[({2-[4,7,10tris-(carboxylatomethyl)-1,4,7,10tetraazacyclododecane-1-yl]propanoyl}aminoacetyl]amino}methyl)-4,7,11,14tetraazaheptadecan-2-yl}-1,4,7,10tetraazacyclododecane-1-yl] (also known as gadoquatran). (For example, see J. Lohrke et al.: Preclinical Profile of Gadoquatrane: A Novel Tetrameric, Macrocyclic High Relaxivity Gadolinium-Based Contrast Agent. Invest Radiol., 2022, 1, 57(10):629-638; International Publication No. 2016193190).

[0069] In one embodiment of the present disclosure, the contrast agent is a compound of formula (I) Gd 3+ It is a drug containing a complex, [ka] Here, Ar is [ka] It is a base selected from, Here, # This is a combination with X, X is CH2, (CH2)2, (CH2)3, (CH2)4 and *-(CH2)2-O-CH2- # It is a base selected from, In the formula, * represents a bond with Ar, # This is a bond with an acetate residue, R 1 , R 2 and R 3is, independently of each other, a group selected from a hydrogen atom or C1-C3 alkyl, -CH2OH, -(CH2)2OH and -CH2OCH3, R 4 is a group selected from C2-C4 alkoxy, (H3C-CH2)-O-(CH2)2-O-, (H3C-CH2)-O-(CH2)2-O-(CH2)2-O- and (H3C-CH2)-O-(CH2)2-O-(CH2)2-O-(CH2)2-O-, R 5 is a hydrogen atom, and R 6 is a hydrogen atom, or a stereoisomer, tautomer, hydrate, solvate or salt thereof, or a mixture thereof.

[0070] In one embodiment of the present disclosure, the contrast agent is a drug containing a Gd 3+ complex of a compound of formula (II),

Chemical formula

Chemical formula

[0071] The term "C1-C3 alkyl" refers to a linear or branched saturated monovalent hydrocarbon group having one, two, or three carbon atoms, such as methyl, ethyl, n-propyl, or isopropyl. The term "C2-C4 alkyl" refers to a linear or branched saturated monovalent hydrocarbon group having two, three, or four carbon atoms.

[0072] The term "C2-C4 alkoxy" refers to a linear or branched saturated monovalent group of the formula (C2-C4 alkyl)-O-, such as a methoxy, ethoxy, n-propoxy, or isopropoxy group, as defined above by the term "C2-C4 alkyl".

[0073] In one embodiment of the present disclosure, the contrast agent is a drug comprising gadolinium 2,2',2"-(10-{1-carboxy-2-[2-(4-ethoxyphenyl)ethoxy]ethyl}-1,4,7,10tetraazacyclododecane-1,4,7-triyl)triacetate (see, for example, International Publication No. 2022 / 194777, Example 1).

[0074] In one embodiment of the present disclosure, the contrast agent is a drug comprising gadolinium 2,2',2"-{10-[1-carboxy-2-{4-[2-(2-ethoxyethoxy)ethoxy]phenyl}ethyl]-1,4,7,10tetraazacyclododecane-1,4,7-triyl}triacetate (see, for example, International Publication No. 2022 / 194777, Example 2).

[0075] In one embodiment of the present disclosure, the contrast agent is a drug comprising gadolinium 2,2',2"-{10-[(1R)-1-carboxy-2-{4-[2-(2-ethoxyethoxy)ethoxy]phenyl}ethyl]-1,4,7,10tetraazacyclododecane-1,4,7-triyl}triacetate (see, for example, International Publication No. 2022 / 194777, Example 4).

[0076] In one embodiment of the present disclosure, the contrast agent is a drug comprising gadolinium(2S,2'S,2"S)-2,2',2"-{10-[(1S)-1-carboxy-4-{4-[2-(2-ethoxyethoxy)ethoxy]phenyl}butyl]-1,4,7,10tetraazacyclododecane-1,4,7triyl}tris(3-hydroxypropanoate) (see, for example, International Publication No. 2022 / 194777, Example 15).

[0077] In one embodiment of the present disclosure, the contrast agent is a drug comprising gadolinium 2,2',2"-{10-[(1S)-4-(4-butoxyphenyl)-1-carboxybutyl]-1,4,7,10tetraazacyclododecane-1,4,7-triyl}triacetate (see, for example, International Publication No. 2022 / 194777, Example 31).

[0078] In one embodiment of the present disclosure, the contrast agent is a drug comprising gadolinium 2,2',2"-{(2S)-10-(carboxymethyl)-2-[4-(2-ethoxyethoxy)benzyl]-1,4,7,10tetraazacyclododecane-1,4,7-triyl}triacetate.

[0079] In one embodiment of the present disclosure, the contrast agent is a drug comprising gadolinium 2,2',2"-[10-(carboxymethyl)-2-(4-ethoxybenzyl)-1,4,7,10tetraazacyclododecane-1,4,7-triyl]triacetate.

[0080] In one embodiment of the present disclosure, the contrast agent is a drug comprising gadolinium(III)5,8-bis(carboxylatomethyl)-2-[2-(methylamino)-2-oxoethyl]-10-oxo-2,5,8,11tetraazadodecane-1-carboxylate hydrate (also known as gadodiamide).

[0081] In one embodiment of the present disclosure, the contrast agent is a drug comprising gadolinium(III)2-[4-(2-hydroxypropyl)-7,10bis(2-oxide-2-oxoethyl)-1,4,7,10tetrazacyclododeca-1-yl]acetate (also known as gadoteridol).

[0082] In one embodiment of the present disclosure, the contrast agent is a drug comprising gadolinium(III)2,2',2"-(10-((2R,3S)-1,3,4trihydroxybutan-2-yl)-1,4,7,10tetraazacyclododecane-1,4,7triyl)triacetate (also known as gadobutrol or Gd-DO3A-butrol).

[0083] At least one received image may also include representations of the examination region generated under different measurement conditions, e.g., T1-weighted MRI images and / or T2-weighted MRI images and / or diffusion-weighted MRI images and / or any other MRI images and / or one or more dual-energy CT images and / or one or more spectral CT images.

[0084] At least one received image may also include multiple radiographic images generated after administration of different amounts of contrast agent and / or after administration of different contrast agents, e.g., a native radiographic image and / or radiographic images after administration of a first amount of contrast agent and / or one or more radiographic images after administration of a second contrast agent and / or virtual non-contrast (VNC) representations.

[0085] At least one received image may also include multiple radiographic images generated at different times before and / or after the administration of one or more contrast agents, and / or representing different stages and / or states of the examination area.

[0086] Each received image contains a number of image elements. Each of these image elements represents a sub-region of the inspection area being examined. The term "number of image elements" means at least 1000, preferably at least 10000, and more preferably more than 100000. The received image may contain one or more image elements that do not represent the inspection area being examined, but represent any other region, such as adjacent regions and / or surrounding regions.

[0087] In the first step, multiple sub-images are generated based on at least one image. The term “multiple sub-images” means at least two, preferably at least ten, and more preferably more than twenty sub-images.

[0088] Each sub-image represents a sub-region of the examination area being examined. Sub-regions represented by different sub-images partially overlap but do not completely overlap. In other words, there are at least two sub-images, each representing a sub-region of the examination area, and some of these sub-regions overlap but do not completely overlap. In other words, there are sub-regions of the examination area represented by multiple sub-regions (at least two sub-regions), and sub-images representing the same sub-region represent different sub-regions.

[0089] Preferably, each sub-region of the examination area is represented by several (at least two) sub-images with configurations different from those of other sub-regions.

[0090] A sub-region of the examination area is represented by one or more image elements of a partial image. Therefore, there are at least two partial images, each having at least one common image element, but at least one different image element. A "common image element" here represents the same sub-region, while a "different image element" represents a different sub-region.

[0091] Preferably, for each partial image, there exists at least one other partial image having at least one common image element and at least one different image element. More preferably, for each partial image, there exists a plurality of other partial images having at least one common image element and at least one different image element.

[0092] Preferably, the number of identical (common) image elements and / or different image elements is greater than 10, and more preferably greater than 100.

[0093] Preferably, for each image element of at least one received image, there exist multiple subimages, each of which is also an image element, and each subimage is distinguished from the other subimages by at least one other image element.

[0094] A partial image can be generated, for example, by dividing at least one received image into partial images by cutting, where the cutting line (in the case of a 2D image) or cutting plane (in the case of a 3D image) extends at different angles to at least one of the received images.

[0095] One embodiment for generating a partial image is described below in more detail with reference to Figure 1, and is not intended to limit the present invention to the embodiment shown in Figure 1. Any descriptions made in relation to the embodiments shown in the drawings of this disclosure shall apply by analogy to all other embodiments.

[0096] Figure 1 shows the received image I of the area being examined. i This shows the received image I. iIt contains numerous image elements, with three image elements representing IE1, IE2, and IE3 indicated by dots.

[0097] Received image I i Numerous copies I1, I2, I3, and I4 are generated from this. Note that one of copies I1, I2, I3, or I4 is the same as the received image I i It may be the image itself. The number of copies generated for each received image (per image) is usually equal to the number of composite images that can be generated and then combined to form a combined composite image.

[0098] Each copy I1, I2, I3, and I4 is divided into numerous sub-images. In this example, this is done by cutting and dividing each copy along a cutting plane. The cutting plane extends at different angles to copies I1, I2, I3, and I4. For example, in the case of copy I2, the cutting plane extends parallel to the xz plane, resulting in sub-image PI 21 PI 22 PI 23 PI 24 PI 25 , and PI 26 This results in, for example, copy I4, the cross-section plane extends parallel to the yz plane, and the partial image PI 41 PI 42 PI 43 PI 44 PI 45 PI 46 , and PI 47 This results in the division of copies I1 and I3, which is shown only in Figure 1, where the cross-sections are shown, and the partial images resulting from the corresponding cuts along the cross-sections are not explicitly shown in Figure 1.

[0099] In this example, the copy is divided into partial images by a plane. This is a preferred embodiment of the present invention. However, it is also possible to divide the copy into partial images by a curved surface or other cuts.

[0100] In this example, the cross-sections within each copy are at the same distance from one another. This is a preferred embodiment of the present invention. However, the distances between cross-sections (or generally between cross-sectional surfaces) may differ.

[0101] In copies I2 and I4, all sub-images are the same size (they have the same number of image elements). In other words, the sub-regions of the inspection area they represent in each case are the same size for all sub-images. In copies I1 and I3, only some sub-images are the same size. It is possible to make all sub-images the same size by padding the smaller sub-image regions with zero (or different values).

[0102] The number of subimages generated from each copy may be the same or different. In this example, the number of subimages generated from each copy is different: 6 for copy I1, 6 for copy I2, 9 for copy I3, and 7 for copy I4.

[0103] Preferably, the copy is divided into subimages such that all obtained subimages are the same size (i.e., have the same number of image elements). This can be achieved by padding in each case. This has the advantage that subimages of the same size can always be supplied to the generative model.

[0104] The generation of the partial image shown in Figure 1 satisfies the above requirements. - There are at least two sub-images representing the same sub-region of the inspection area, but each represents yet another sub-region. The sub-region represented by image element IE1 is sub-image PI 25 and partial image PI 41 It is represented by both. However, each subimage PI 25 and PI 41 Furthermore, it represents other sub-regions of the examination area that are not represented by the other partial image. Therefore, partial image PI 25For example, using the image element IE3, a partial image PI 41 Represents a sub-region not represented by the main image. Partial image P 47 This uses the image element IE3 to create a partial image PI. 25 It also represents a sub-region, but uses the image element IE2 to represent a partial image PI 25 It represents a sub-region not represented by the other region. - Partial image PI 25 and PI 47 These images share the common image element IE3, but differ in at least one other image element, and the partial image PI 25 For example, partial image PI 47 Includes image element IE1 which is not included in the partial image PI 47 This is a partial image PI 25 Includes image elements IE2 that are not included in the standard.

[0105] Furthermore, the generation of the partial image shown in Figure 1 satisfies one of the further conditions of the embodiments described above. -For each sub-image, there exists at least one other sub-image that has at least one common image element and at least one different image element. In this example, for each sub-image, there are at least three sub-images that have at least one common image element and at least one different image element. A "common image element" here is an image element that represents the same sub-region, and a "different image element" is an image element that represents a different sub-region.

[0106] In the next step, a composite partial image is generated based on the partial images. The composite partial image is generated by a model referred to in this disclosure as the generative model.

[0107] A generative model can be a trained machine learning model. A “machine learning model” can be considered a computer-implemented data processing architecture. Such a model can receive input data and supply output data based on the input data and model parameters. Through training, such a model can learn the relationship between input and output data. During training, model parameters can be adjusted to supply a desired output for a particular input.

[0108] During training of such a model, the model is presented with training data from which it can learn. A trained machine learning model is the result of the training process. The training data includes input data, as well as the correct output data (target data) that the model generates based on the input data. During training, patterns that map the input data to the target data are recognized.

[0109] In the training process, training data is input to the model, and the model generates output data. The output data is compared to the target data. Model parameters are modified to reduce the deviation between the output data and the target data to a (defined) minimum value. To modify the model parameters to reduce the deviation, optimization procedures such as gradient procedures can be used.

[0110] The deviation can be quantified using a loss function. This type of loss function can be used to calculate the loss for a given pair of output and target data. The goal of the training process may be to modify (tune) the parameters of the machine learning model so that the loss for all pairs in the training dataset is reduced to a (defined) minimum.

[0111] For example, if the output data and target data are numbers, the loss function can be the absolute difference between these numbers. In this case, a high absolute loss may mean that one or more model parameters need to be changed to a considerable extent.

[0112] For example, in the case of output data in vector format, the difference metric between vectors can be selected as the loss function, such as the mean squared error, cosine distance, Euclidean distance, Chebyshev distance, Lp norm of the difference vector, weighted norm, or any other type of difference metric between two vectors.

[0113] For higher-dimensional outputs, such as 2D, 3D, or higher-order outputs, an element-wise difference metric can be used. Alternatively or additionally, the output data can be converted to a one-dimensional vector, for example, before the loss value is calculated.

[0114] Figure 8 schematically illustrates an example of training a machine learning model, which will be explained in more detail below.

[0115] A generative model may include one or more algorithms that specify how to generate a composite sub-image based on one or more sub-images. Typically, one or more sub-images are fed to the generative model, and the model generates a composite sub-image based on one or more sub-images, model parameters, and optionally further input data.

[0116] The generative model is used, for example, in the following publications: International Publication No. 2019 / 074938, International Publication No. 2022 / 253687, International Publication No. 2022 / 207443, International Publication No. 2022 / 223383, International Publication No. 20227184297, International Publication No. 2022 / 179896, International Publication No. 2021 / 069338, European Patent Application Publication No. 22209510.1, European Patent Application Publication No. 2 It may also be a machine learning model such as one of the following: Patent No. 3159288.2, PCT / EP2023 / 053324, PCT / EP2023 / 050207, Chinese Published Patent No. 110852993, Chinese Published Patent No. 110853738, US Patent Application Publication No. 2021150671, arXiv:2303.15938v1, doi:10.1093 / jrr / rrz030.

[0117] The generation model can be configured, for example, to generate a composite radiographic image after the administration of a second amount of contrast agent, based on at least one received radiographic image of the examination area before and / or after the administration of a first amount of contrast agent, wherein the second amount is preferably greater than the first amount (for example, as described in International Publication No. 2019 / 074938 or International Publication No. 2022184297).

[0118] At least one received radiation image can be, for example, an MRI image, and the composite radiation image can be a composite MRI image.

[0119] At least one received radiographic image may also include CT images before and / or after administration of a first amount of MRI contrast agent, and the composite radiographic image may be a composite CT image after administration of a second amount of MRI contrast agent, the second amount being preferably greater than the first amount and preferably greater than the standard amount of MRI contrast agent for an MRI examination (e.g., as described in PCT / EP2023 / 053324). The “standard amount” is typically the amount recommended by the manufacturer and / or distributor of the contrast agent, and / or the amount approved by the regulatory authority, and / or the amount specified in the contrast agent's package leaflet.

[0120] The generation model can be configured to generate a composite radiographic image representing the examination area in a second period after administration of contrast agent, based on at least one radiographic image of the examination area representing the examination area in a first period before and / or after administration of contrast agent (as described, for example, in International Publication No. 2021052896).

[0121] However, in contrast to the methods described in the publications cited above, in this disclosure, not only is a composite image generated, but multiple composite images are generated, and these multiple composite images can be combined in a later step to form a single composite image, which is referred to in this description as a combined composite image.

[0122] The term "multiple composite images" means at least two, preferably at least five, and more preferably at least ten composite images.

[0123] The composite image of multiple composite images differs from one another in that it is generated based at least partially on different sub-images of at least one received image. The sub-images differ in the configuration of the image elements that constitute them, and different sub-images may contain the same image elements, but these same image elements may be contained in a different configuration from the other image elements.

[0124] In other words, each composite image in a group of composite images is generated based on a partial image containing a different configuration of image elements. Therefore, a generative model is supplied with partially different input data and is used to generate multiple composite images based on that partially different input data.

[0125] Next, the difference between the multiple composite images can be used to determine a confidence value indicating the reliability of the combined composite image obtained by combining the multiple composite images. Further details on how to determine the confidence value will be explained in the following section.

[0126] Figure 2 schematically illustrates the generation of a composite partial image based on a partial image using a generative model, and the merging of composite partial images to form a composite image. The partial image is the partial image PI shown in Figure 1. 21 PI 22 PI 23 PI 24 PI 25 PI 26 That is the case.

[0127] Figure 2 is a partial image PI from Figure 1. 21 PI 22 PI 23 PI 24 PI 25 , and PI 26 This could suggest that the partial image PI is supplied to the generative model GM together with the partial image PI.21 , PI 22 , PI 23 , PI 24 , PI 25 , and PI 26 are separately (e.g., continuously) supplied to the generation model GM. In the example shown in FIG. 2, the generation model GM is configured to generate a synthetic partial image PS 2m based on the partial image PI 2m , where m is an index that is an integer ranging from 1 to 6 in this example. Thus, in this example, the synthetic partial image PS 21 is generated based on the partial image PI 21 , the synthetic partial image PS 22 is generated based on the partial image PI 22 , the synthetic partial image PS 23 is generated based on the partial image PI 23 , the synthetic partial image PS 24 is generated based on the partial image PI 24 , the synthetic partial image PS 25 is generated based on the partial image PI 25 , and the synthetic partial image PS 26 is generated based on the partial image PI 26 .

[0128] Each synthetic partial image preferably represents the same part of the inspection area as the partial image generated based on it.

[0129] Further, in FIG. 2, it is shown that the synthetic partial images PS 21 , PS 22 , PS 23 , PS 24 , PS 25 , PS 26 can be merged in a further step to form a synthetic image S2. The synthetic image S2 preferably represents the entire inspection area (similar to the case of at least one received image I i in FIG. 1).

[0130] The partial images PS 21 , PS 22 , PS 23PS 24 PS 25 PS 26 The method using this as an example is carried out similarly for the other remaining copies I1, I3, and I4.

[0131] Therefore, for each copy I1, I2, I3, and I4, this yields composite images S1, S2, S3, and S4 in each case. This is shown in Figure 3.

[0132] As schematically shown in Figure 3, the composite images S1, S2, S3, and S4 can be combined in a further step to form a combined composite image S.

[0133] The composite images S1, S2, S3, and S4 represent the same inspection area and preferably contain the same number of image elements. Thus, each sub-area of ​​the inspection area is quadruple-represented by the composite images S1, S2, S3, and S4.

[0134] The process of combining composite images S1, S2, S3, and S4 to form a combined composite image is based on the color values ​​of the corresponding image elements.

[0135] Image elements representing the same sub-region of an inspection area are referred to in this disclosure as "corresponding image elements," or simply "corresponding image elements." Corresponding image elements may be, for example, image elements having the same coordinates when each composite image is a raster graphic.

[0136] For each k-tuple of corresponding image elements in the generated composite image, a color value is determined. Here, k represents the number of corresponding image elements. In the example shown in Figure 3, in all cases, the four image elements of the composite images S1, S2, S3, and S4 correspond to each other.

[0137] An average value (e.g., arithmetic mean, geometric mean, root mean square, or another average value) can be calculated from the color values. The average of the color values of the tuple of corresponding image elements can be set as the color value of the corresponding image element of the combined composite image.

[0138] In the case of multiple color values (e.g., three color values as in the RGB color model), an average can be calculated for each color channel. Then, each average value can be set as the color value of the corresponding color channel of the corresponding image element of the combined composite image.

[0139] For the corresponding image elements, instead of the average value, the maximum or minimum color value can be determined, and the combined composite image can be formed from the image elements having each maximum or minimum color value. Instead of the maximum / minimum value, other statistical values can also be determined and used to generate the combined composite image.

[0140] FIG. 4 schematically shows, as an example, combining composite images to form a combined composite image.

[0141] Three composite images S1, S2, and S3 are shown. The composite images S1, S2, S3 shown in FIG. 4 are 2D raster graphics. Each of the composite images S1, S2, S3 represents the same inspection area of the inspection object.

[0142] Each of the three composite images S1, S2, and S3 includes 10·10 = 100 image elements. The image elements are arranged in a grid, and numbers are assigned to each row and each column, so that each image element can be clearly specified by its coordinates (row value, column value).

[0143] The composite images S1, S2, and S3 are binary images, that is, either the color value "white" or the color value "black" is assigned to each image element.

[0144] The combined synthetic image S is generated by combining synthetic images S1, S2, and S3. The combination is performed based on corresponding image elements. The corresponding image elements represent the same sub-region of the inspection area to be inspected in any case. In this example, the coordinates of the corresponding image elements are the same. For example, the image element at the coordinates (1,1) of the synthetic image S1 corresponds to the image element with the coordinates (1,1) of the synthetic image S2 and the image element with the coordinates (1,1) of the synthetic image S3. The image elements with the coordinates (1,1) of the synthetic images S1, S2, and S3 form a tuple of corresponding image elements.

[0145] For each tuple of corresponding image elements, a color value is determined, and based on the determined color value, the color value of the corresponding image element of the combined synthetic image is determined.

[0146] In this example, the synthetic images are combined according to the following rule to form a combined synthetic image: The color value of each image element of the combined synthetic image S corresponds to the color value of the majority of the color values of the corresponding image elements of the synthetic images S1, S2, and S3.

[0147] For example, the color value of the image element with the coordinates (1,1) of the synthetic image S1 is "white". The color value of the corresponding image element with the coordinates (1,1) of the synthetic image S2 is also "white". The color value of the corresponding image element with the coordinates (1,1) of the synthetic image S3 is also "white". The majority of the corresponding image elements (i.e., all the image elements) have the color value "white". Therefore, the color value of the image element with the coordinates (1,1) of the combined synthetic image is also set to "white".

[0148] For example, the color value of the image element with the coordinates (1,4) of the synthetic image S1 is "white". The color value of the corresponding image element with the coordinates (1,4) of the synthetic image S2 is "black". The color value of the corresponding image element with the coordinates (1,4) of the synthetic image S3 is "white". The majority of the corresponding image elements have the color value "white". Therefore, the color value of the image element with the coordinates (1,4) of the combined synthetic image is set to "white".

[0149] For example, the color value of the image element at coordinates (7,10) in composite image S1 is "black". Similarly, the color value of the corresponding image element at coordinates (7,10) in composite image S2 is also "black". The color value of the corresponding image element at coordinates (7,10) in composite image S3 is "white". Most of the corresponding image elements have a color value of "black". Therefore, the color value of the image element at coordinates (7,10) in the combined composite image is set to "black".

[0150] Further possibilities exist for combining individual composite images to form a combined composite image. For example, a machine learning model (e.g., an artificial neural network) can be trained to generate a combined composite image from composite images of multiple composite images according to specified factors. If training data is available that includes not only composite images as input data but also images that can be used as target data, the machine learning model can be trained in a supervised learning manner to combine composite images of multiple composite images to form a combined composite image. For example, an attention mechanism can be used (see, e.g., arXiv:2203.14263) that assigns different weights to the individual composite images of multiple composite images when they are combined to form a combined composite image.

[0151] The method for generating a combined composite image may not be the same for each tuple of corresponding image elements. Different methods can be used to generate combined composite images for different subregions of the subject being examined. For example, image elements representing a particular tissue and / or organ and / or lesion may use a different method for combining color values ​​with image elements representing a different tissue and / or organ and / or subregion. Subregions for which there are different methods for combining corresponding image elements can be identified, for example, by segmentation.

[0152] The term "segmentation" refers to the process of dividing an image into multiple segments, also called image segments, image regions, or image objects. Segmentation is commonly used to locate objects and boundaries (lines, curves, etc.) within an image. In a segmented image, the locced objects can be separated from the background, visually highlighted (e.g., in color), measured, counted, or otherwise quantified. In segmentation, each image element of an image is assigned an identifier (e.g., a number), so that image elements with the same identifier have a common specific characteristic, such as representing the same tissue (e.g., bone tissue or adipose tissue or healthy tissue or diseased tissue (e.g., tumor tissue) or muscle tissue) and / or the same organ. For corresponding image elements with a specific identifier, their color values ​​can be combined using specific calculation rules to produce a combined composite image. For corresponding image elements with different (specific) identifiers, their color values ​​can be combined using different (specific) calculation rules.

[0153] It is also possible to generate two or more combined composite images, for example, two, three, four, or more than four combined composite images. For example, it is possible to generate a first combined composite image containing the maximum value of each color value, and a second combined composite image containing the minimum value of each color value. It is also possible to generate a third combined composite image containing the average value of the color values.

[0154] The combined composite image can be output (e.g., displayed on a monitor and / or printed) and / or stored in data memory and / or transmitted to a separate computer system via a network connection, for example. Based on the outputted combined composite image, a physician can, for example, make a diagnosis and / or initiate treatment.

[0155] It is also possible that the combined composite image is not generated and / or output. A subsequent analysis of the corresponding image elements of the composite image can reveal that the combined composite image is unreliable. For example, the determined confidence value, which positively correlates with reliability, may be lower than a predetermined threshold. In some cases, a diagnosis should not be made and / or treatment should not be initiated based on the combined composite image due to the determined confidence value, and therefore the reliability being very low. In such cases, the combined composite image may be worthless, or even misleading, and therefore potentially dangerous. Therefore, the generation and / or output of such unreliable combined composite images can be omitted. A warning can be output to draw the user's attention to the fact that an unreliable composite image has been generated by the generative model based on the received image.

[0156] In the examples shown in Figures 1, 2, and 3, one received image I i Based on this, only one combined composite image S is generated. However, it is also possible to generate only one combined composite image S based on two or more received images.

[0157] In such cases, partial images are generated from each received image, as described in this disclosure. The partial images generated from different received images are then fed together to a generation model, which generates a composite partial image based on the supplied partial images. Preferably, the partial images supplied to the generation model represent the same sub-region of the inspection area. The composite partial images can then be merged to form multiple composite images, and the multiple composite images can be combined to form a combined composite image. This is schematically shown in Figures 5a and 5b in an example having two received images.

[0158] Figure 5a schematically illustrates a method for generating partial images from each of multiple received images, and a method for generating a composite partial image based on the generated partial images. Figure 5b schematically illustrates a method for merging composite partial images to form a composite image, and a method for combining composite images to form a combined composite image.

[0159] The starting point of the method shown in FIG. 5a is two received images, a first image I1 and a second image I2. Each image preferably represents the same inspection area of the same inspection object.

[0160] From each received image, a plurality of copies are generated. In the example shown in FIG. 5a, three copies of each received image are generated, and from the first image I1, copies I 11 、I 12 、I 13 are generated, and from the second image I2, copies I 21 、I 22 、I 23 are generated. One of the copies I 11 、I 12 or I 13 can be the first image I1 itself. Similarly, one of the copies I 21 、I 22 or I 23 can be the second image I2 itself.

[0161] Each copy is divided into partial images, and each partial image represents a sub-region of the inspection area of the inspection object. From copy I 11 ,partial images PI 111 、PI 112 、PI 113 、PI 114 、PI 115 、PI 116 are generated, and from copy I 12 ,partial images PI 121 、PI 122 [[ID=5b]]、PI 123 、PI 124 、PI 125 、PI 126 are generated, and from copy I 13 ,partial images PI 131 、PI 132 、PI 133 、PI 134 、PI 135 、PI 136 are generated, and from copy I 21 ,partial images PI 211 、PI 212 、PI 213 、PI214 PI 215 PI 216 A copy I is generated. 22 From, partial image PI 221 PI 222 PI 223 PI 224 PI 225 , and PI 226 A copy I is generated. 23 From, partial image PI 231 PI 232 PI 233 PI 234 PI 235 PI 236 This is generated.

[0162] Subregions represented by different subimages partially overlap but not completely. One example of this is subimage PI. 111 This is the partial image PI. 111 , PI 121 PI 131 PI 132 PI 133 PI 134 PI 135 , and PI 136 It partially overlaps, but not completely.

[0163] Corresponding subimages arising from different received images are fed together to the generative model GM. "Corresponding subimages" are subimages representing the same sub-region of the examination area.

[0164] The generative model GM is shown three times in Figure 5a for better understanding, but in each case it is the same generative model.

[0165] In the example shown in Figure 5a, the partial image PI 111 This is a partial image PI 211 Corresponds to partial image PI 112 This is a partial image PI 212 Corresponds to partial image PI 113 This is a partial image PI 213 Corresponds to partial image PI 114 This is a partial image PI 214corresponds to the partial image PI 115 is the partial image PI 215 corresponds to the partial image PI 116 is the partial image PI 216 corresponds to the partial image PI 121 is the partial image PI 221 corresponds to the partial image PI 122 is the partial image PI 222 corresponds to the partial image PI 123 is the partial image PI 223 corresponds to the partial image PI 124 is the partial image PI 224 corresponds to the partial image PI 125 is the partial image PI 225 corresponds to the partial image PI 126 is the partial image PI 226 corresponds to the partial image PI 131 is the partial image PI 231 corresponds to the partial image PI 132 is the partial image PI 232 corresponds to the partial image PI 133 is the partial image PI 233 corresponds to the partial image PI 134 is the partial image PI 234 corresponds to the partial image PI 135 is the partial image PI 235 corresponds to the partial image PI 136 is the partial image PI 236 corresponds.

[0166] In any case, the mutually corresponding partial images are supplied together to the generation model GM, and the generation model generates a synthesized partial image based on the supplied partial images in each case. Each synthesized partial image corresponds to the partial image on which it is generated, that is, it represents a sub-region of the same inspection area as the partial image on which it is generated.

[0167] In the example shown in FIG. 5a, the generation model GM generates a synthesized partial image PS 111 and PI 211 based on the synthesized partial image PS 11 from the partial image PI 112 and PI 212 based on the synthesized partial image PS 12Partial image PI 113 and PI 213 Based on the composite partial image PS 13 Partial image PI 114 and PI 214 Based on the composite partial image PS 14 Partial image PI 115 and PI 215 Based on the composite partial image PS 15 Partial image PI 116 and PI 216 Based on the composite partial image PS 16 Partial image PI 121 and PI 221 Based on the composite partial image PS 21 Partial image PI 122 and PI 222 Based on the composite partial image PS 22 Partial image PI 123 and PI 223 Based on the composite partial image PS 23 Partial image PI 124 and PI 224 Based on the composite partial image PS 24 Partial image PI 125 PI 225 Based on the composite partial image PS 25 Partial image PI 126 PI 226 Based on the composite partial image PS 26 Partial image PI 131 PI 231 Based on the composite partial image PS 31 Partial image PI 132 PI 232 Based on the composite partial image PS 32 Partial image PI 133 PI 233 Based on the composite partial image PS 33 Partial image PI 134 PI 234 Based on the composite partial image PS 34 Partial image PI 135 PI 235 Based on the composite partial image PS 35 Partial image PI 136 PI 236Based on the composite partial image PS 36 Generates. The composite sub-images generated based on the same division of the copy are merged in the next step to form the composite image.

[0168] In the example shown in Figure 5b, partial image PS 11 PS 12 PS 13 PS 14 PS 15 PS 16 These are merged to form a composite image S1, and a partial image PS 21 PS 22 PS 23 PS 24 PS 25 PS 26 These are merged to form a composite image S2, and a partial image PS 31 PS 32 PS 33 PS 34 PS 35 PS 36 These are merged to form the composite image S3.

[0169] The composite images S1, S2, and S3 are then combined in a further step to form a combined composite image S. This operation corresponds to the operation shown in Figures 3 and 4. The combined composite image S can be output (e.g., displayed on a monitor and / or printed) and / or stored in data memory and / or transmitted to a separate computer system via a network connection, for example.

[0170] As already explained, it is also possible to receive three or more images and generate a combined composite image based on these received images.

[0171] The number of received images can be m, where m is a positive integer. m images I1, I2, ... m It is possible to generate p copies from each of the elements, where p is a positive integer. This results in a total of m·p copies:I 11 , I 12 ,..., I 1p, I 21 , I 22 ,...I 2p ,...I m1 , I m2 ,..., I mp This occurs.

[0172] Each copy is divided into sub-images. For example, the number of sub-images into which each copy is divided can be q, where q is a positive integer. As mentioned above, the number of sub-images into which a copy is divided may differ from copy to copy. If it is equal for all copies, then this is m·p·q sub-images: PI 111 PI 112 ,..., PI 11q PI 121 PI 122 ,..., PI 12q ,...,P 1p1 , P 1p2 ,...,P 1pq ,...,P 211 , P 212 ,...,P 21q , P 221 , P 222 ,...,P 22q ,...,P 2p1 , P 2p2 ,...,P 2pq ,...,P m11 , P m12 ,...,P m1q , P m21 , P m22 ,...,P m2q ,...,P mp1 , P mp2 ,...,P mpq This occurs.

[0173] Copies arising from the same received image are divided into sub-images in different ways, and as a result, for each sub-image of a copy, there exists at least one other sub-image of another copy that completely overlaps but does not completely overlap the sub-image. When sub-images partially overlap, the overlapping region represents the same sub-region of the inspection region.

[0174] In other words, copy I 11 , I 12,..., I 1p It is divided into sub-images in different ways, and as a result, for each sub-image, there is at least one other sub-image that partially overlaps but does not completely overlap the sub-image. Similarly, copy I 21 , I 22 ,...I 2p The image is divided into sub-images such that for each sub-image, there exists at least one other sub-image that partially overlaps but does not completely overlap the sub-image. As a generalization, copy I j1 , I j2 ,..., I jp For each subimage, the subimage is divided into subimages such that there is at least one other subimage that partially overlaps but does not completely overlap the subimage, and j is an integer that can take values ​​in the range of 1 to m.

[0175] The method by which a copy of the received image is divided into sub-images is preferably the same for different received images. This yields corresponding sub-images. "Corresponding sub-images" are sub-images that represent the same sub-region of the examination area.

[0176] This results in copy I 11 This is copy I 21 , copy I 31 ..., copy I m1 Similarly, it is divided into partial images. Similarly, copy I 12 This is copy I 22 , copy I 32 ..., copy I m2 Similarly, it is divided into partial images. As a generalization, copy I rs The image is similarly divided into sub-images, where r is an integer index ranging from 1 to m. Here, r is a number ranging from 1 to m, and the value s remains constant, where s is an integer that can take values ​​ranging from 1 to p.

[0177] Therefore, partial image PI 111 This is a partial image PI 211 , partial image PI 311 ..., partial image PI m11This corresponds to the partial image PI. 112 PI 212 PI 312 ,..., PI m12 These correspond to each other. Similarly, the partial image PI 113 PI 213 PI 313 ,..., PI m13 These correspond to each other. As a general rule, partial image PI rst These correspond to each other, and r is an integer index in the range 1 to m. Here, r is a number in the range 1 to m, and the values ​​s and t remain constant, where s is an integer that can take values ​​in the range 1 to p, and t is an integer that can take values ​​in the range 1 to q.

[0178] A composite subimage is generated using a generative model based on the subimages. In this operation, the corresponding subimages are supplied together to the generative model. The generative model generates the composite subimage based on each tuple of the corresponding subimages. The generative model generates the composite subimage based on the corresponding subimages PI 111 PI 211 PI 311 ,..., PI m11 Based on this, composite partial image PS 11 The generation model generates the corresponding subimages PI. 112 PI 212 PI 312 ,..., PI m12 Based on this, composite partial image PS 12 The generation model generates the corresponding subimages PI. 113 PI 213 PI 313 ,..., PI m13 Based on this, composite partial image PS 13 It generates the following. In general, the generative model generates the corresponding subimages PI. rst Based on this, composite partial image PS stThis generates a function where r is an integer index ranging from 1 to m. Here, r is a number ranging from 1 to m, and the values ​​s and t remain constant, where s is an integer that can take values ​​ranging from 1 to p, and t is an integer that can take values ​​ranging from 1 to q.

[0179] Next, the composite partial images can be merged to form a composite image. In this operation, the composite partial images PS 11 , P 12 ,...,PS 1q These are merged to form a composite image S1, and a composite partial image PS 21 , P 22 ,...,PS 2q These are merged to form the composite image S2, and the composite partial image PS 31 , P 32 ,...,PS 2q These are merged to form composite image S3, and so on. As a general rule, composite partial image PS vu This is a composite image S v The images are merged to form a single combined composite image, where u is an integer index ranging from 1 to q. Here, u is a number ranging from 1 to q, and the value v remains constant, being an integer that can take values ​​ranging from 1 to p. All the generated composite images can be joined together to form a single combined composite image.

[0180] The determination of at least one confidence level is explained in more detail below.

[0181] At least one confidence value can be a value that indicates the degree to which a composite image (e.g., a combined composite image) is reliable. The confidence value can be positively correlated with the reliability of the composite image; that is, a low confidence value indicates low reliability, and a high confidence value indicates high reliability. However, the confidence value can also be negatively correlated with reliability; that is, a low confidence value indicates high reliability, and a high confidence value indicates low reliability. In the case of a negative correlation, another term that can be used instead of confidence value is uncertainty value, and a high uncertainty value indicates that the composite image exhibits a high degree of uncertainty. It is possible that a composite image may have one or more artifacts, and that structures and / or forms and / or textures within the composite image do not actually correspond, i.e., structures and / or forms and / or textures within the composite image cannot be attributable to actual structures and / or actual forms and / or actual textures within the examination area. In contrast, a low uncertainty value indicates that the composite image exhibits a low degree of uncertainty. Features within the composite image do actually correspond, the composite image is reliable, a medical diagnosis can be made based on the composite image, and / or medical treatment can be initiated based on the composite image.

[0182] A confidence value that is positively correlated with reliability can, in principle, be converted into a confidence value (uncertainty value) that is negatively correlated with reliability, for example, by taking its reciprocal (or multiplication reciprocal). Conversely, a confidence value (uncertainty value) that is negatively correlated with reliability can be converted into a confidence value that is positively correlated with reliability.

[0183] At least one confidence value can be determined based on the corresponding image elements of the composite image.

[0184] For each k-tuple of corresponding image elements in the generated composite image, a color value is determined, where k represents the number of corresponding image elements. In the example shown in Figure 4, in all cases, the three image elements of the composite images S1, S2, and S3 are corresponding to each other.

[0185] The greater the difference in color values ​​between corresponding image elements within a composite image, the greater the influence on the sub-images that form the basis for generating the composite image. However, when the sub-images that form the basis of the composite image are significantly affected, the composite image becomes somewhat uncertain, and the greater the difference, the less reliable it becomes.

[0186] Therefore, the degree to which the color values ​​of corresponding image elements differ can be used as a measure of reliability / uncertainty. The greater the difference in color values ​​of corresponding image elements, the lower the reliability and the higher the uncertainty. The smaller the difference in color values ​​of corresponding image elements, the lower the uncertainty and the higher the reliability.

[0187] Therefore, reliability / uncertainty can be determined for each tuple of corresponding image elements in the composite image of multiple composite images, and then represent (i) reliability / uncertainty for each composite image of the multiple composite images, (ii) reliability / uncertainty for all composite images of the multiple composite images, and (iii) reliability / uncertainty for the combined composite image.

[0188] In other words, for each individual image element of a combined composite image, it is possible to determine a confidence value that indicates the degree to which the color values ​​of that image element are reliable.

[0189] Such a confidence value can be, for example, the range of color values ​​in a tuple of corresponding image elements. The range is defined as the difference between the maximum and minimum values ​​of the variable. Therefore, for each tuple of corresponding image elements, the maximum and minimum color values ​​can be determined, and the difference between the maximum and minimum color values ​​can be calculated. The result is the range of color values ​​in a tuple of corresponding image elements that can be used as a confidence value.

[0190] When there are multiple color values ​​(for example, three color values ​​as in an image, specified according to the RGB color model), the maximum and minimum color values ​​can be determined for each color channel, and the difference can be calculated for each color channel. This results in three ranges. Each range can be used as a separate confidence value for each color channel. However, it is also possible to combine the ranges of the color channels to form a single value. The maximum range can be used as the confidence value. The mean of the ranges (e.g., arithmetic mean, geometric mean, root mean square, or another mean) can be used as the confidence value. The length of the vector specified by the ranges in 3D space (or higher-order space if more than three color channels are used) can be used as the confidence value. Further possibilities are conceivable.

[0191] The confidence value of a tuple of corresponding image elements may be the variance and / or standard deviation of the color values ​​of the corresponding image elements. Variance is defined as the mean squared deviation from the expected value of the variable, and the standard deviation is defined as the square root of the variance.

[0192] Confidence values ​​can also be other measures of variance, such as the sum of squared deviations, coefficient of variation, mean absolute deviation, quantile range, interquantile range, mean absolute deviation from the median, median absolute deviation, and / or geometric standard deviation. It is also possible for more than one confidence value to exist for a tuple of corresponding image elements.

[0193] The method for calculating confidence values ​​may not be the same for each tuple of corresponding image elements. Different methods can be used to calculate confidence values ​​for different subregions under examination. For example, an image element representing a particular tissue and / or organ and / or lesion may use a different method for calculating confidence values ​​than an image element representing a different tissue and / or organ and / or subregion. Subregions for which different calculation rules for confidence values ​​exist can be identified, for example, by segmentation.

[0194] In segmentation, each image element of an image can be assigned an identifier (e.g., a number), and thus image elements with the same identifier share a common characteristic, such as representing the same tissue (e.g., bone tissue or adipose tissue or healthy tissue or diseased tissue (e.g., tumor tissue) or muscle tissue) and / or the same organ. For corresponding image elements with a specific identifier, a confidence value can be calculated using a specific calculation rule. For corresponding image elements with different (specific) identifiers, a confidence value can be calculated using a different (specific) calculation rule.

[0195] The confidence values ​​determined for the corresponding image element tuples can be output (e.g., displayed on a monitor or printed to a printer), stored in data memory, and / or transmitted to a separate computer system, for example, over a network.

[0196] The confidence values ​​determined for the corresponding image element tuples can also be displayed graphically.

[0197] Therefore, in addition to the combined composite image, a further representation of the inspection area can be output (e.g., displayed on a monitor) to indicate the reliability of each image element. Such a representation is also referred to herein as a confidence representation. The confidence representation preferably has the same dimensions and size as the combined composite image. Preferably, each image element of the combined composite image is assigned an image element in the confidence representation.

[0198] Such confidence representations can be used by users (e.g., physicians) to identify the degree to which the color values ​​of image elements are reliable for each individual image element. The confidence representations can be fully or partially overlaid on the combined composite image and / or received image. The overlay can be configured by the user to be hidden or visible. The user can view the combined composite image and / or received image layer by layer, as is customary for, for example, computed tomography representations, magnetic resonance imaging representations, and other three-dimensional or multi-dimensional representations. For each layer, the user can view the corresponding layer of the confidence representation to check whether the image elements in the layer representing structure, morphology, and / or texture in the combined composite image are reliable or uncertain. This allows the user to identify the level of risk that the structure, morphology, and / or texture are actual characteristics or artifacts of the examination area.

[0199] For example, image elements with low reliability (high degree of uncertainty) can be displayed brightly and / or in signal colors (e.g., red, orange, or yellow), while image elements with high reliability (low degree of uncertainty) can be displayed darkly, less conspicuously, or in muted colors (e.g., green or blue). In the case of superposition, it is also possible to display only image elements whose confidence value is above or below a predetermined threshold. If the confidence value is positively correlated with reliability, for example, it is possible to display only image elements with confidence representations where the confidence value is below a predetermined threshold. In such cases, the user (e.g., a doctor) will only recognize image elements that should not be trusted.

[0200] It is also possible to determine the confidence values ​​of sub-regions (partial images) (e.g., layers within the combined image) and / or the entire combined image of a combined image. Such confidence values ​​can be determined for sub-regions or entire images based on the confidence values ​​of the image elements that constitute them. For example, the confidence value of a layer can be determined by taking into account the confidence values ​​of all the image elements within that layer. However, it is also possible to take into account adjacent image elements (e.g., image elements of layers above and / or below the layer under consideration). The confidence value of a sub-region or entire region can be determined, for example, by calculating the mean (e.g., arithmetic mean, geometric mean, root mean square, or another mean). It is also possible to determine the maximum (e.g., in the case of confidence values ​​that are negatively correlated with confidence) or minimum (e.g., in the case of confidence values ​​that are negatively correlated with confidence) of the image elements of a sub-region or entire region and use that as the confidence value of the sub-region or entire region. Further methods can be considered for determining the confidence value of a sub-region or entire region based on the confidence values ​​of individual image elements.

[0201] Such confidence values ​​for a sub-region or an entire region can also be output (e.g., displayed on a monitor or printed), stored in data memory, and / or transmitted to a separate computer system. They can also be represented graphically (e.g., by color), as described for individual confidence values.

[0202] If the confidence value of a sub-region or an entire region that is positively correlated with reliability falls below a predetermined threshold, the corresponding sub-region or entire region may not be trustworthy. Such sub-regions or entire regions may be not output at all (e.g., not displayed at all), as described above, or they may be displayed with a warning indicating that the user should exercise caution when interpreting the displayed data due to the uncertainty of the displayed data.

[0203] It is also possible to provide users of the computer system / computer program of this disclosure with the option to navigate to less reliable subregions within the combined composite image via a user interface. For example, the user may be shown a list of the least reliable subregions (e.g., in the form of a list having l subregions with the lowest confidence values ​​positively correlated with confidence, where l is a positive integer). By clicking on a list entry, the user is shown the corresponding subregion in the form of the combined composite image, confidence representation and / or the received image and / or details thereof.

[0204] Figure 6 schematically illustrates the determination of at least one confidence value as an example. At least one confidence value is determined based on the corresponding image elements of the composite images S1, S2, and S3 already shown in Figure 4.

[0205] For each tuple of corresponding image elements, a confidence value is determined. In the first step, the color values ​​of all image elements are determined. In this example, as is generally customary, the color "black" is assigned a color value of "0", and as is generally customary, the color "white" is assigned a color value of "1".

[0206] The confidence value calculated for each tuple of corresponding image elements in the composite images S1, S2, and S3 is the range of color values.

[0207] For example, the color value of the image element at coordinate (1,1) in composite image S1 is "1" (white). Similarly, the color value of the corresponding image element at coordinate (1,1) in composite image S2 is also "1" (white). Similarly, the color value of the corresponding image element at coordinate (1,1) in composite image S3 is also "1" (white). Therefore, the range of the tuple for the corresponding image elements is 1-1=0.

[0208] For example, the color value of the image element at coordinates (1,4) in composite image S1 is "1" (white). The color value of the corresponding image element at coordinates (1,4) in composite image S2 is "0" (black). The color value of the corresponding image element at coordinates (1,4) in composite image S3 is "1" (white). Therefore, the range of the tuple for the corresponding image elements is 1-0=1.

[0209] For example, the color value of the image element at coordinates (7,10) in composite image S1 is "0" (black). Similarly, the color value of the corresponding image element at coordinates (7,10) in composite image S2 is also "0" (black). The color value of the corresponding image element at coordinates (7,10) in composite image S3 is "1" (white). Therefore, the range of the tuple for the corresponding image elements is 1-0=1.

[0210] The confidence values ​​are listed in the CV table.

[0211] The confidence values ​​determined in this way are negatively correlated with reliability.

[0212] Based on confidence values, a confidence representation can be determined. In the example shown in Figure 6, the color value of each image element in the confidence representation SR is set to the corresponding confidence value of the corresponding image element tuple. For example, the image element with coordinates (1,1) in the confidence representation receives black, and the image elements with coordinates (1,4) and (7,10) receive white. Using the confidence representation SR, a user (e.g., a doctor) can immediately identify which image elements are certain (black) and which are uncertain (white). Regions in the confidence representation SR where many white image elements occur should be considered low confidence by the user.

[0213] Figure 7 shows one embodiment of the method of this disclosure in flowchart form.

[0214] Method (100) is, (110) The step of receiving at least one image of the area to be examined, (120) A step of generating a number of sub-images based on at least one received image, wherein each sub-image represents a sub-region of the inspection area to be inspected, and the sub-regions represented by different sub-images partially overlap but do not completely overlap, (130) A step of generating multiple composite partial images based at least partially on the generated partial image, (140) A step of determining the color values ​​of corresponding image elements in a composite partial image, wherein the corresponding image elements represent the same sub-region of the inspection area, (150) A step of determining the scale of the variance of the color values ​​of the corresponding image elements, (160) A step of determining confidence values ​​based on a measure of variance, (170) Steps to output confidence values, Includes.

[0215] As explained herein, the generative models described herein can be trained machine learning models. Figure 8 schematically illustrates an example of a method for training a machine learning model.

[0216] The generative model GM is trained using the training data TD. For each of the numerous reference objects, the training data TD includes at least one reference image of the reference region of the reference object in at least one first state as input data, and one reference image of the reference region of the reference object in at least one state different from the first state. The term “numerous reference objects” preferably means more than 10, and more preferably more than 100 reference objects.

[0217] The term "criteria" is used here to distinguish the training phase from the phase in which a trained model is used to generate a composite image.

[0218] The "reference image" is an image used to train the model. The "reference object" is the object from which the reference image originates. The reference object is usually an animal or a human, preferably a human, similar to the object being examined. The reference region is a part of the reference object. Preferably, the reference region is the same part as the object being examined.

[0219] However, the term “reference” is not limited to other meanings. Any description made in this explanation with respect to at least one received image applies similarly to each reference image, any description made in this explanation with respect to the object being examined applies similarly to each reference object, and any description made in this explanation with respect to the area being examined applies similarly to the reference area.

[0220] In the example shown in Figure 8, only one set of training data TD for one reference object is shown. Typically, training data TD contains many such datasets for many reference objects. In the example shown in Figure 8, training data TD includes the first reference image RI1, the second reference image RI2, and the third reference image RI3.

[0221] The first reference image RI1 represents the reference region of the reference object in the first state; the second reference image RI2 represents the reference region of the reference object in the second state; and the third reference image RI3 represents the reference region of the reference object in the third state. The first, second, and third states are usually different from each other. For example, a state can represent the amount of contrast agent introduced into or that has been introduced into the reference region. For example, a state can represent the time before and / or after administration of the contrast agent.

[0222] For example, the first reference image RI1 may represent no contrast agent administration or after administration of a first dose; the second reference image RI2 may represent the reference area after administration of a second dose of contrast agent; and the third reference image RI3 may represent the reference area after administration of a third dose of contrast agent. The first dose may be less than the second dose, and the second dose may be less than the third dose (see, for example, International Publication No. 2019 / 074938 and International Publication No. 2022184297).

[0223] For example, the first reference image RI1 may represent the reference region during a first period before or after administration of the contrast agent, the second reference image RI2 may represent the reference region during a second period after administration of the contrast agent, and the third reference image RI3 may represent the reference region during a third period after administration of the contrast agent (see, for example, International Publication Nos. 2021052896 and 2021069338).

[0224] In the example shown in Figure 8, the first reference image RI1 and the second reference image RI2 serve as input data, which are supplied to the generation model GM. The generation model GM is configured to generate a composite image S based on the first reference image RI1 and the second reference image RI2, and based on the model parameter MP. The composite image S should approximate the third reference image RI3 as closely as possible. In the example shown in Figure 8, this means that the third reference image RI3 acts as the target data (ground truth).

[0225] The composite image S generated by the generative model GM is compared to a third reference image RI3. The loss function LF is used to quantify the difference between the composite image S and the third reference image RI3. For each pair of composite image and third reference image, the loss value can be calculated using the loss function LF.

[0226] In the optimization procedure, the loss value, and therefore the difference, between the synthesized image S generated by the generative model and the third reference image RI3 can be reduced by modifying the model parameter MP.

[0227] This process is repeated for a large number of reference objects.

[0228] Training can be terminated if the loss value reaches a predetermined minimum value, or if the loss value cannot be further reduced by modifying the model parameters. The trained model can be stored, transmitted to a separate computer system, and / or used to generate a composite image of a (new) subject (object under examination).

[0229] Figure 9 schematically illustrates the computer system described herein as an example.

[0230] A "computer system" is an electronic data processing system that processes data using programmable computing rules. Such a system typically comprises a "computer," which is a unit equipped with a processor for performing logical operations, and peripheral devices.

[0231] In computer technology, "peripheral devices" refer to all devices connected to a computer and used for controlling the computer and / or as input / output devices. Examples include monitors (screens), printers, scanners, mice, keyboards, drives, cameras, microphones, and speakers. Internal ports and expansion cards are also considered peripheral devices in computer technology.

[0232] The computer system (10) shown in Figure 9 comprises a receiving unit (11), a control and calculation unit (12), and an output unit (13).

[0233] The control and calculation unit (12) is used to control the computer system (10), coordinate the data flow between units of the computer system (10), and perform calculations.

[0234] The control and computing unit (12) The receiving unit (11) receives at least one image of the inspection area to be inspected. Based on at least one received image, a number of sub-images are generated, each sub-image representing a sub-region of the area being examined, and the sub-regions represented by different sub-images partially overlap but not completely. Multiple composite subimages are generated based at least partially on the generated subimages. Determine the color values ​​of the corresponding image elements in the composite subimage, and the corresponding image elements represent the same sub-region of the inspection area. Determine the measure of the variance of the color values ​​of the corresponding image elements. The confidence value is determined based on the measure of variance. The output device outputs a reliable value. It is configured in this way.

[0235] Figure 10 schematically illustrates a further embodiment of the computer system. The computer system (10) comprises a processing unit (21) connected to a memory (22). The processing unit (21) and the memory (22) constitute a control and calculation unit, as shown in Figure 9.

[0236] The processing unit (21) may comprise one or more processors, either alone or in combination with one or more memories. The processing unit (21) may be standard computer hardware capable of processing information such as digital image recordings, computer programs and / or other digital information. The processing unit (21) typically consists of an electronic circuit configuration, some of which may be designed as an integrated circuit or as a group of interconnected integrated circuits (integrated circuits are sometimes called "chips"). The processing unit (21) may be configured to execute computer programs that can be stored in the main memory of the processing unit (21) or in the memory (22) of the processing unit or in the memory of another computer system.

[0237] Memory (22) can be standard computer hardware capable of temporarily and / or permanently storing information such as, for example, digital image recordings (e.g., representations of areas under investigation), data, computer programs, and / or other digital information. Memory (22) can include volatile and / or non-volatile memory and can be fixedly installed or removable. Suitable examples of memory include RAM (random access memory), ROM (read-only memory), hard disks, flash memory, replaceable computer floppy disks, optical disks, magnetic tape, or a combination of the above. Optical disks can include compact disks with read-only memory (CD-ROM), compact disks with read / write capabilities (CD-R / W), DVDs, Blu-ray discs, and the like.

[0238] The processing unit (21) can be connected not only to the memory (22) but also to one or more interfaces (11, 12, 31, 32, 33) for displaying, transmitting, and / or receiving information. The interfaces may comprise one or more communication interfaces (11, 32, 33) and / or one or more user interfaces (12, 31). One or more communication interfaces may be configured to transmit and / or receive information to, for example, an MRI scanner, CT scanner, ultrasound camera, other computer systems, networks, data memory, etc. One or more communication interfaces may be configured to transmit and / or receive information via physical (wired) and / or wireless connections. One or more communication interfaces may include one or more interfaces for connecting to a network using, for example, cellular, Wi-Fi, satellite, cable, DSL, or optical fiber. In some examples, one or more communication interfaces may comprise one or more near-field communication interfaces configured to connect devices using short-range communication technologies such as NFC, RFID, Bluetooth, Bluetooth LE, ZigBee, or infrared (e.g., IrDA).

[0239] The user interface may include a display (31). The display (31) may be configured to display information to the user. Suitable examples include liquid crystal displays (LCDs), light-emitting diode displays (LEDs), plasma display panels (PDPs), etc. One or more user input interfaces (11, 12) may be wired or wireless and may be configured to receive information from the user to the computer system (1) for processing, storage and / or display, for example. Suitable examples of user input interfaces include microphones, image or video recording devices (e.g., cameras), keyboards or keypads, joysticks, touch-sensitive surfaces (separate from or integrated into the touchscreen), etc. In some examples, the user interface may include automatic identification and data acquisition (AIDC) technology for machine-readable information. This may include barcodes, radio frequency identification (RFID), magnetic strips, optical character recognition (OCR), integrated circuit cards (ICCs), etc. The user interface may also include one or more interfaces for communicating with peripheral devices such as printers.

[0240] One or more computer programs (40) can be stored in memory (22) and executed by a processing unit (21), thereby programmed to perform the functions described herein. The retrieval, loading, and execution of instructions of the computer programs (40) can be carried out sequentially, one instruction at a time, so as to be retrieved, loaded, and executed. However, the retrieval, loading, and / or execution can also be carried out in parallel.

[0241] The computer systems of this disclosure may be designed as laptops, notebooks, netbooks and / or tablet PCs, and may be components of MRI scanners, CT scanners, or ultrasound diagnostic devices. [Explanation of symbols]

[0242] 10 Computer Systems 11. User input interface, receiving unit 12. User input interface, calculation unit 13 Output devices, output units 14 User Interface 17 Communication Interface 18 Communication Interfaces 21 Processing Units 22 Working Memory 31. User Interface, Display 32 Interfaces 33 Interfaces 40 Computer Programs

Claims

1. At least one image of the area to be examined (I i The steps include receiving ) and The at least one received image (I i ) based on numerous partial images (PI 21 PI 22 PI 23 PI 24 PI 25 PI 26 A step of generating a sub-image, wherein each sub-image represents a sub-region of the inspection area to be inspected, and the sub-regions represented by different sub-images partially overlap but do not completely overlap. The generated partial image (PI 21 , PI 22 , PI 23 , PI 24 , PI 25 , PI 26 ) to generate a number of composite partial images (PS 21 , PS 22 , PS 23 , PS 24 , PS 25 , PS 26 ) based at least partially on the step of generating, Composite partial image (PS 21 PS 22 PS 23 PS 24 PS 25 PS 26 A step of determining the color value of the corresponding image element of the inspection area, wherein the corresponding image element represents the same sub-region of the inspection area, A step of determining the scale of the variance of the color values ​​of the corresponding image elements, A step of determining the confidence value based on the aforementioned measure of variance, The steps include outputting the aforementioned confidence value, Computer implementation methods, including those mentioned above.

2. Composite partial image (PS 21 PS 22 PS 23 PS 24 PS 25 PS 26 For each tuple of the corresponding image elements of ), the scale of the variance of the color values ​​is determined in each case, and the composite partial image (PS 21 PS 22 PS 23 PS 24 PS 25 PS 26 The method according to claim 1, wherein for each tuple of the corresponding image elements of ), a confidence value is determined in each case based on the respective scale of variance.

3. The method according to claim 1 or 2, wherein the measure of the variance is the range of the color values ​​of the corresponding image element, the standard deviation, the variance, the sum of the squared deviations, the coefficient of variation, the mean absolute deviation, the quantile range, the interquantile range, the mean absolute deviation from the median, the absolute deviation from the median, and / or the geometric standard deviation, or derived therefrom.

4. The method according to any one of claims 1 to 3, wherein for each partial image, there exists at least one other partial image having at least one common image element and at least one different image element.

5. The at least one received image (I i The method according to any one of claims 1 to 4, wherein for each image element of ), there exist a plurality of subimages including this image element, and each of the plurality of subimages is different from the plurality of subimages by at least one other image element.

6. The aforementioned numerous partial images (PI) 21 PI 22 PI 23 PI 24 PI 25 PI 26 The above generation of ) is The at least one received image (I i Numerous copies of (I 1 , I 2 , I 3 , I 4 The steps to generate ) and Each copy (I 1 , I 2 , I 3 , I 4 ) partial image (PI 21 PI 22 PI 23 PI 24 PI 25 PI 26 ) is a step of dividing into all parts, and all parts images of all copies are in different steps from each other. The method according to any one of claims 1 to 5, including the method described in any one of claims 1 to 5.

7. The aforementioned numerous partial images (PI) 21 PI 22 PI 23 PI 24 PI 25 PI 26 The above generation of ) is The at least one received image (I i Numerous copies of (I 1 , I 2 , I 3 , I 4 The steps to generate ) and Each copy (I 1 , I 2 , I 3 , I 4 ) is cut to create a partial image (PI 21 PI 22 PI 23 PI 24 PI 25 PI 26 A step of dividing into ) wherein the cutting is performed differently for each copy of the copy, The method according to any one of claims 1 to 6, including the method described in any one of claims 1 to 6.

8. Synthesize partial images (PS 21 、PS 22 、PS 23 、PS 24 、PS 25 、PS 26 ), and merge them to form a composite image (S 1 、S 2 、S 3 、S 4 ), and the step of The synthetic image (S 1 , S 2 , S 3 , S 4 ), a step of generating a combined synthetic image (S), wherein the generation of the combined synthetic image (S) includes the step of combining the color values of the corresponding image elements of the synthetic image (S 1 , S 2 , S 3 , S 4 ), a step and, The method according to any one of claims 1 to 7, including the method described in any one of claims 1 to 7.

9. The generation of the combined composite image (S) is as follows: The aforementioned composite image (S 1 S 2 S 3 S 4 For each tuple of corresponding image elements in the combined image (S), the average color value is determined by averaging the color values ​​of the corresponding image elements, and the average color value is set as the color value of the corresponding image element in the combined composite image (S). The method according to claim 8, including the method described in claim 8.

10. Steps of outputting the combined composite image (S) and / or transmitting the combined composite image (S) to a separate computer system. The method according to any one of claims 1 to 9, further comprising:

11. A step of generating a confidence representation (SR), wherein the confidence representation (SR) includes a number of image elements, each of the image elements represents a sub-region of the inspection area, each image element has a color value, and the color value correlates with the confidence value of the respective tuple of the corresponding image elements of the composite image. The steps include: outputting the confidence expression (SR) preferably superimposed on the combined composite image (S), and / or transmitting the confidence expression (SR) to a separate computer system; The method according to any one of claims 1 to 10, further comprising:

12. A step of determining the confidence value of one or more subregions of the combined composite image (S) and / or the entire combined composite image (S), The steps include outputting the aforementioned confidence value, The method according to any one of claims 8 to 11, further comprising:

13. The method according to any one of claims 1 to 12, wherein the subject of the examination is a human or an animal, preferably a mammal, most preferably a human.

14. The at least one received image (I i ) is at least one medical image, and each composite image (S 1 S 2 S 3 S 4 The method according to any one of claims 1 to 13, wherein the combined composite image (S) is a synthetic medical image.

15. The at least one received image (I i ) includes a first radiographic image and a second radiographic image, wherein the first radiographic image represents the examination area of ​​the subject being examined without contrast agent or after administration of a first amount of contrast agent, and the second radiographic image represents the examination area of ​​the subject being examined after administration of a second amount of contrast agent. Each composite image (S 1 S 2 S 3 S 4 ) and / or the combined composite image (S) is a composite radiation image, and each composite image (S) 1 S 2 S 3 S 4 ) and / or the combined composite image (S) represents the examination area of ​​the subject after administration of a third amount of the contrast agent, wherein the second amount is different from the first amount and preferably greater than the first amount, and the third amount is different from the first and second amounts and preferably greater than the first and second amounts. The method according to any one of claims 1 to 14.

16. The at least one received image (I i ) includes a first radiographic image and a second radiographic image, wherein the first radiographic image represents the examination area of ​​the subject to be examined during a first period before or after the administration of the contrast agent, and the second radiographic image represents the examination area of ​​the subject to be examined during a second period after the administration of the contrast agent. Each composite image (S 1 S 2 S 3 S 4 ) and / or the combined composite image (S) is a composite radiation image, and each composite image (S) 1 S 2 S 3 S 4 ) and / or the combined composite image (S) represents the examination area of ​​the subject being examined during a third period after the administration of the contrast agent, wherein the second period preferably follows the first period, and the third period preferably follows the second period. The method according to any one of claims 1 to 15.

17. Receiving unit (11), Control and computing unit (12), Output unit (13), Equipped with, The control and computing unit (12) At least one image of the area to be examined (I i The receiving unit (11) receives the following: The at least one received image (I i ) based on numerous partial images (PI 21 PI 22 PI 23 PI 24 PI 25 PI 26 ) is generated, and each partial image represents a sub-region of the inspection area of ​​the subject being inspected, and the sub-regions represented by different partial images partially overlap but do not completely overlap. The generated partial image (PI) 21 PI 22 PI 23 PI 24 PI 25 PI 26 ) based at least partially on a number of composite partial images (PS 21 PS 22 PS 23 PS 24 PS 25 PS 26 ) generates, The color values ​​of the corresponding image elements in the composite partial image are determined, and the corresponding image elements represent the same sub-region of the inspection area. Determine the scale of the variance of the color values ​​of the corresponding image elements, The confidence value is determined based on the aforementioned measure of variance, The output device (13) is made to output the confidence value. A computer system (10) configured in such a way.

18. A computer-readable storage medium comprising a computer program (40), wherein when the computer program (40) is loaded into the working memory (22) of a computer system (10), the computer system (10) At least one image of the area to be examined (I i The steps include receiving ) and The at least one received image (I i ) based on numerous partial images (PI 21 PI 22 PI 23 PI 24 PI 25 PI 26 A step of generating a sub-image, wherein each sub-image represents a sub-region of the inspection area to be inspected, and the sub-regions represented by different sub-images partially overlap but do not completely overlap. The generated partial image (PI) 21 PI 22 PI 23 PI 24 PI 25 PI 26 ) based at least partially on a number of composite partial images (PS 21 PS 22 PS 23 PS 24 PS 25 PS 26 The steps to generate ) and A step of determining the color values ​​of corresponding image elements in a composite partial image, wherein the corresponding image elements represent the same sub-region of the inspection region. A step of determining the scale of the variance of the color values ​​of the corresponding image elements, A step of determining the confidence value based on the aforementioned measure of variance, The steps include outputting the aforementioned confidence value, A computer-readable storage medium that enables execution of [something].

Images