Provides classification explanations and generative functions

Abstract

The present invention relates to providing classification explanations and generating functions. The present invention relates to a method for providing classification explanations, comprising: receiving an acquired medical image corresponding to an examination volume; receiving or determining an actual classification value, in particular by applying a classification function to the acquired medical image to generate the actual classification value, wherein the actual classification value classifies an anatomical structure within the acquired medical image; receiving or determining a first classification value different from the actual classification value, wherein the first classification value is a potential classification value of the anatomical structure within the acquired medical image; applying a first generating function to the acquired medical image to generate a first composite image, wherein the first generating function is based on the first classification value, wherein the first generating function is configured to modify the anatomical structure to display the first classification value; providing a visualization based on the acquired medical image and the first composite image. The present invention also relates to a system and a computer program for providing classification explanations.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to providing classification interpretation and generation functions. Background Art

[0002] In recent years, clinical decision support using deep learning (DL) has become a major research topic. This includes applications in microscopy, histopathology, time-continuous biosignal analysis, and other types of medical image analysis, including magnetic resonance imaging (MRI), positron emission tomography (PET), and computed tomography (CT). Although recent work has repeatedly demonstrated that evaluation performance is comparable to or even exceeds gold standard human assessment, such tools have rarely been used in clinical practice. A major obstacle to such applications lies in the fact that deep learning-based systems are generally considered difficult to understand and opaque, raising issues of error-proneness and liability. Therefore, automated decision interpretation systems based on deep learning systems would be a major step towards clinical application.

[0003] In the context of medical image classification, recent work has proposed methods for explaining decisions. However, these methods produce unsatisfactory results when used on classifiers trained with few samples or with samples of low input resolution or quality. This is a commonly encountered type of data for various nontrivial clinical tasks, such as lung lesion malignancy estimation.

[0004] Known methods in the prior art can be categorized as model-dependent methods (involving explicit modifications to the underlying model architecture) and model-agnostic methods (mostly applicable to arbitrary architectures) that backpropagate target class activations through the architecture back into the input space.

[0005] Well-known model-dependent solutions include deconvolutional networks (DeconvNet), (gradient-based) class activation maps (CAM / GradCAM), or attention-gated networks. However, model-dependent methods only generate coarse, blob-like explanations that emphasize only coarse regions. As a result, they do not allow for the identification of decision-relevant and clinically important fine structures, such as spicules or vascularization in lung lesions.

[0006] On the other hand, model-independent methods include Layered Relevance Propagation (LRP), Locally Interpretable Model-Independent Explanations (LIME), (Deep) Shapley Additive Explanations (SHAP / DeepSHAP) and variants (e.g., DeepTaylor), or (Deep) Learning Important Features (LIFT / DeepLIFT). Model-independent methods work by directly backpropagating decisions through the network. Because deep neural networks apply highly nonlinear decision functions to the data, such backpropagation cannot be applied explicitly. Therefore, all of these methods apply approximations based on assumptions about the visual meaning of feature interpretations, and their results are typically evaluated using user studies. However, these assumptions do not necessarily hold, depending on the model architecture (e.g., when using LRP). When applied to small amounts of data, or data of low quality or resolution, all of the above methods produce highly noisy explanations, which makes it very difficult to identify decision-relevant regions and does not help or even undermine clinical acceptance. Summary of the Invention

[0007] The problem underlying the present invention is to provide a better interpretation of the results of machine learning or deep learning algorithms, in particular classification results. This problem is solved by a computer-implemented method, system, computer program product, and computer-readable storage medium according to the independent claims. Advantageous embodiments and additional features are set forth in the dependent claims and the following description.

[0008] Below, the solution according to the present invention is described with respect to the claimed system and with respect to the claimed method. Features, advantages, or alternative embodiments described herein may be assigned to other corresponding claimed objects, and vice versa. In other words, features described or claimed in the context of the corresponding method may be used to improve the system. In such cases, the functional features of the method are embodied by the target unit of the system.

[0009] Furthermore, the solutions according to the present invention are described with respect to methods and systems for providing classification explanations and with respect to methods and systems for providing generating functions for use in methods and systems for providing classification explanations. Features, advantages or alternative embodiments herein may be assigned to other corresponding claimed objects, and vice versa. In other words, features described or claimed in the context of methods and systems for providing classification explanations may be used to improve methods and systems for providing classification explanations. In particular, the same term (potentially up to the addition of the adjective "training") may have the same features and embodiments in both types of methods and systems.

[0010] According to a first aspect, the present invention relates to a computer-implemented method for providing a classification interpretation, the method comprising receiving an acquired medical image corresponding to an examination volume.

[0011] Furthermore, an actual classification value is received or determined. Here, the actual classification value classifies the anatomical structure within the acquired medical image. Specifically, the actual classification value can be determined by applying a classification function to the acquired medical image. In other words, the actual classification value is based on or equal to the output of the classification function when applied to the acquired medical image. When applying the classification function to the acquired medical image, additional input data for the classification function may be present.

[0012] Another step is receiving or determining a first classification value different from the actual classification value, wherein the first classification value is a potential classification value for the structure within the acquired medical image. In other words, the first classification value is a potential output of the classification function. The first classification value can be based on input from a user or a system external to the method. Alternatively, the first classification value can be determined without external input. In particular, the first classification value can be determined based on the actual classification value and / or the acquired medical image.

[0013] Another step of the method is applying a first generation function to the acquired medical image to generate a first composite image, wherein the first generation function is based on the first classification value and wherein the first generation function is configured to modify the anatomical structure to exhibit the first classification value. In other words, the first generation function is configured to generate the first composite image including a modified structure that is a modification of the anatomical structure that exhibits the first classification value. In other words, the first generation function is configured to generate the composite image including a composite structure that is similar to the anatomical structure but exhibits the first classification value.

[0014] The final step of the method is providing a visualization based on the acquired medical image and the first composite image. The visualization may include simultaneous display of the acquired medical image and the first composite image, particularly wherein the acquired medical image and the first composite image are spatially aligned. The visualization may also include modifying the acquired medical image based on the first composite image, or including additional elements (e.g., arrows or circles) within the acquired medical image based on the first composite image.

[0015] According to yet another possible aspect, the present invention relates to a computer-implemented method for providing a classification explanation, comprising:

[0016] - receiving an acquired medical image corresponding to an examination volume,

[0017] - applying the classification function to the acquired medical image, thereby generating an actual classification value that classifies the anatomical structure within the acquired medical image,

[0018] - receiving or determining a first classification value that is different from the actual classification value,

[0019] The first classification value is a potential classification value of the anatomical structure in the acquired medical image,

[0020] - applying a first generating function to the acquired medical image, thereby generating a first composite image,

[0021] Wherein, the first generating function is based on the first classification value,

[0022] wherein the first generating function is configured to modify the anatomical structure to exhibit a first classification value,

[0023] - providing a visualization based on the acquired medical image and the first composite image.

[0024] According to yet another possible aspect, the present invention relates to a computer-implemented method for providing a classification explanation, comprising:

[0025] - receiving an acquired medical image corresponding to an examination volume, the acquired medical image comprising an anatomical structure, in particular wherein the anatomical structure is classified by an actual classification value,

[0026] - receiving a first classification value, wherein the first classification value is a potential classification value of an anatomical structure within the acquired medical image,

[0027] - applying a first generating function to the acquired medical image, thereby generating a first composite image,

[0028] Wherein, the first generating function is based on the first classification value,

[0029] wherein the first generating function is configured to modify the anatomical structure to exhibit a first classification value,

[0030] - providing a visualization based on the acquired medical image and the first composite image.

[0031] In particular, the acquired medical image and the first composite image are medical images. In particular, the medical image is an X-ray image, a computed tomography image (abbreviated as "CT image"), a magnetic resonance image (abbreviated as "MR image"), a positron emission tomography image (abbreviated as "PET image"), a single photon emission computed tomography image (abbreviated as "SPECT image"), and / or an ultrasound image (abbreviated as "US image"). Furthermore, the medical image may be a microscopic image, a histopathological image, and / or a time-continuous biosignal analysis image.

[0032] In particular, the medical image may be a two-dimensional image, a three-dimensional image or a four-dimensional image. In particular, in a four-dimensional image, there are three spatial dimensions and one time dimension.

[0033] Medical images may include a plurality of pixels or voxels, where the term "pixel" is primarily used for the building blocks of two-dimensional images, and the term "voxel" is used for the building blocks of images with arbitrary dimensions (primarily three or more dimensions). Hereinafter, the term "voxel" will be used as a synonym for the term "pixel or voxel." Each voxel may include at least one intensity value.

[0034] A medical image may be identical to or encapsulated in one or more DICOM files. Whenever DICOM is mentioned herein, it should be understood that this refers to the "Digital Imaging and Communications in Medicine" (DICOM) standard, such as the current DICOM PS3.1 2020c standard (or any later or earlier version of the standard). Several medical images may also be encapsulated in a single DICOM file.

[0035] In particular, the acquired medical image is a medical image created using a medical imaging device. In particular, the acquired medical image corresponds to an examination volume (or region of interest) within a patient's body. In particular, intensity values ​​of voxels of the acquired medical image correspond to physical properties of tissue within the examination volume or region of interest within the patient, subject to measurement inaccuracies.

[0036] In particular, the synthesized image does not correspond to an examination volume (or region of interest) within the patient's body. In particular, the intensity values ​​of the voxels of the synthesized image do not directly correspond to physical properties of tissue within the examination volume of interest within the patient. The synthesized image is based on an acquired medical image and includes a modification of the acquired medical image. In particular, the synthesized image resembles the acquired medical image, in particular in that an absolute difference between intensity values ​​of corresponding voxels of the acquired medical image and the synthesized image does not exceed a predetermined value and / or a ratio of said absolute difference to a corresponding intensity value of the acquired medical image does not exceed a predetermined value.

[0037] In particular, the first synthesized image has the same dimensions as the acquired medical image. In particular, if the acquired medical image is a two-dimensional image, the first synthesized image is a two-dimensional image; if the acquired medical image is a three-dimensional image, the first synthesized image is a three-dimensional image; and if the acquired medical image is a four-dimensional image, the first synthesized image is a four-dimensional image. In particular, the first synthesized image has the same size as the acquired medical image, wherein the size of the medical image corresponds to the extension of the medical image measured in the number of voxels with respect to each dimension.

[0038] In particular, an anatomical structure is a complex anatomical part of a patient's body. The terms "anatomical structure," "body structure," and "complex body part" may be used as synonyms. Examples of anatomical structures are organs, bones, joints, cartilage, muscles, vessels (e.g., blood vessels or lymphatic vessels), the brain, a neoplasm (also referred to as a "tumor," in particular a benign tumor, a tumor in situ, a malignant tumor, and / or a tumor of uncertain / unknown behavior), a nodule, and / or parts of the aforementioned structures.

[0039] In particular, the classification value is related to the classification of the anatomical structure. In particular, for a given anatomical structure, there are at least two possible classification values.

[0040] The multiple possible classification values ​​for an anatomical structure can be a finite set (i.e., there are a finite number of possible classification values, such as "true" and "false" or "benign" and "malignant"), a discrete set (e.g., there are only countably many possible classification values, such as natural numbers), or a continuous set (e.g., real numbers corresponding to the size of a particular anatomical structure).

[0041] The plurality of possible classification values ​​for the anatomical structure may be comparable, ie, for each pair of two possible classification values, the first classification value is smaller than, equal to, or larger than the second classification value.

[0042] In contrast to a regression process, nothing in this application should be construed as limiting categorical values ​​to the results of a classification process. In particular, a categorical value can be the result of a regression process. For example, a categorical value can be an estimate of a patient's weight based on a medical imaging dataset, such as a whole-body scan. Furthermore, nothing in this application should be construed as limiting categorical values ​​to numerical values. In particular, a categorical value can be given by a class label, a vector containing numerical values, or even an imaging dataset.

[0043] In particular, the classification function takes as input a medical image comprising an anatomical structure and gives as output a classification value corresponding to the anatomical structure within the input medical image. In particular, the classification function may be a trained function.

[0044] For example, a classification function can be used to assess the malignancy of a tumor within a medical image. Other examples of possible classification functions relate to survival estimation, brain hemorrhage detection, or medical image segmentation.

[0045] In particular, a generation function takes as input a medical image (particularly an acquired medical image) and generates as output a composite image, the medical image comprising an anatomical structure and a classification value. In particular, the generation function is configured to modify the anatomical structure such that a result of the classification function applied to the composite image produces a classification value used as input to the generation function. In particular, the generation function may be a trained function.

[0046] Typically, trained functions mimic cognitive functions that humans associate with other human minds. Specifically, by being trained on training data, trained functions are able to adapt to new environments and detect and infer patterns.

[0047] Typically, the parameters of a trained function can be adjusted by means of training. In particular, supervised training, semi-supervised training, unsupervised training, reinforcement learning, and / or active learning can be used. Furthermore, representation learning (an alternative term is "feature learning") can be used. In particular, the parameters of a trained function can be iteratively adjusted over several training steps.

[0048] In particular, the trained function may include a neural network, a support vector machine, a decision tree, and / or a Bayesian network, and / or the trained function may be based on k-means clustering, Qlearning, a genetic algorithm, and / or association rules. In particular, the neural network may be a deep neural network, a convolutional neural network, or a convolutional deep neural network. Furthermore, the neural network may be an adversarial network, a deep adversarial network, and / or a generative adversarial network.

[0049] The inventors have recognized that the described method provides high-quality, intuitively understandable explanations of classification decisions, improving understanding of the decision strategy. In particular, this allows users to understand the internal state of the decision system (or classification system) and to examine the quality of the decisions themselves. Furthermore, the inventors have recognized that direct access to the classification function is not required to use the described method. In particular, the described method can be used independently of the classification function and / or as a decision explanation for an externally given classification function.

[0050] In a double-blind, randomized, in-house study conducted by eight participants with many years of experience in the field of biomedical image processing, the described method significantly outperformed three state-of-the-art methods (DeepSHAP, DeepTaylor, LRP) on multiple test criteria (visual impression, significance regarding typical signs of malignancy of lung lesions, and overall image quality).

[0051] According to a further possible aspect of the invention, the first generating function is based on the first classification value by using the first classification value as an input parameter of the first generating function and / or by selecting the first generating function from a plurality of generating functions based on the first classification value.

[0052] According to a further aspect of the invention, the method comprises the step of determining a modified medical image based on the acquired medical image and the first composite image. Furthermore, providing the visualization then comprises providing the modified medical image.

[0053] In particular, the modified medical image can include data from the acquired medical image and / or the first composite image. In particular, the modified medical image can include all intensities of voxels of the acquired medical image, and / or additional data derived from the first composite image can be included as an overlay. For example, a marker having a position based on data derived from the first composite image (and optionally also the acquired medical image) can be included as an overlay. Examples of such a marker are arrows or bounding boxes.

[0054] In particular, the modified medical image has the same dimensions as the acquired medical image. In particular, if the acquired medical image is a two-dimensional image, the modified medical image is a two-dimensional image; if the acquired medical image is a three-dimensional image, the modified medical image is a three-dimensional image; and if the acquired medical image is a four-dimensional image, the modified medical image is a four-dimensional image. In particular, the modified medical image has the same size as the acquired medical image, wherein the size of the medical image corresponds to an extension of the medical image measured in terms of the number of voxels with respect to each dimension.

[0055] The inventors have recognized that by determining the modified medical image, spatial correlations and / or deviations between the acquired medical image and the first composite image can be included in the visualization. Specifically, regions of the acquired medical image corresponding to low correlations and / or high deviations can be directly labeled. Because these regions correspond to areas where different classification values ​​are expected to differ, information about these regional labels can be determined regarding regions that result in a particular outcome for the classification function.

[0056] According to yet another aspect of the present invention, the modified medical image is based on a difference between the first synthesized image and the acquired medical image. In particular, the modified medical image is based on an absolute value of the difference between the first synthesized image and the acquired medical image. In particular, the difference between the synthesized image and the acquired medical image is a voxel-by-voxel difference in intensity values ​​of corresponding voxels.

[0057] The inventors have realized that higher values ​​of such a difference correspond to larger deviations between the acquired medical image and the first synthetic image, which means that the areas with these deviations have a large influence on the result of the classification function.

[0058] According to another aspect of the present invention, each voxel of the modified medical image includes a first intensity value and a second intensity value, wherein the first intensity value corresponds to the intensity value of the corresponding voxel of the acquired medical image, and the second intensity value corresponds to the intensity value of the corresponding voxel of the difference between the first composite image and the acquired medical image.

[0059] In particular, the first intensity values ​​of the modified medical image may form a first color channel of the modified medical image, and the second intensity values ​​of the modified medical image may form a second color channel of the modified medical image. In particular, the first color channel may correspond to a grayscale, wherein high intensity values ​​correspond to white, low intensity values ​​correspond to black, or vice versa, and intermediate intensity values ​​correspond to black. In particular, the second color channel may correspond to a color scale between transparent and a predetermined color (e.g., red), wherein low intensity values ​​(corresponding to small differences) correspond to full transparency, and wherein high intensity values ​​(corresponding to large differences) correspond to a predetermined color with no or less than full transparency.

[0060] The inventors have recognized that by using two different intensity values ​​within the modified medical image, the deviations between the first synthetic image and the acquired medical image (corresponding to the regions that primarily influence the result of the classification function) can be displayed in a fine-grained manner. In particular, by using transparency to display the second intensity, the complete information of the acquired medical image can be displayed in the visualization.

[0061] According to yet another potential aspect of the present invention, the actual classification value and the first classification value are comparable. In this aspect, the first classification value is less than the actual classification value, or the first classification value is greater than the actual classification value.

[0062] In particular, two values ​​are comparable if it can be determined whether they are equal, or which of the two values ​​is less than the other. In particular, a set of values ​​is comparable if all pairs of its elements are comparable.

[0063] The inventors realized that by using the proposed method on comparable categorical values, decision explanations can also encompass step-wise differences between categorical values.

[0064] According to another aspect of the present invention, the actual classification value and the first classification value are elements of a plurality of possible classification values ​​of the anatomical structure, the plurality of possible classification values ​​being a discrete and comparable set. In addition, the first classification value is the smallest classification value among the plurality of possible classification values ​​that are greater than the actual classification value, alternatively, the first classification value is the largest classification value among the plurality of possible classification values ​​that are less than the actual classification value. In other words, there is a smallest positive or negative difference between the actual classification value and the first classification value. In other words, the first classification value is the next or adjacent value of the actual classification value. In particular, in this regard, the first classification value is determined based on the actual classification value by finding the corresponding first classification value among the plurality of possible classification values.

[0065] The inventors have realised that by using a first classification value that is adjacent to the actual classification value, an explanation can be provided as to why the actual classification is not actually higher or lower than that output by the classification function.

[0066] According to yet another aspect of the present invention, the method further includes receiving or determining a second classification value that is different from the actual classification value and different from the first classification value. Here, the second classification value is a potential classification value for an anatomical structure within the acquired medical image. The method further includes applying a second generation function to the acquired medical image to generate a second composite image, wherein the second generation function is based on the second classification value, and wherein the second generation function is configured to modify the anatomical structure to exhibit the second classification value. Furthermore, the visualization is based on the second composite image.

[0067] In particular, the second classification value may comprise all advantageous features and embodiments described with respect to the first classification value.The second composite image may comprise all advantageous features and embodiments as described with respect to the second classification value.

[0068] The inventors have realised that by using a second composite image based on a second classification value, the output of the classification function can be interpreted with respect to two different factors, eg with respect to higher and lower classification values, or with respect to orthogonal aspects of multiple classification values.

[0069] According to a further possible aspect of the invention, the second generating function is based on the second classification value by using the second classification value as an input parameter of the second generating function and / or by selecting the second generating function from a plurality of generating functions based on the second classification value.

[0070] According to a further aspect of the invention, the method comprises the step of determining a modified medical image based on the acquired medical image, the first composite image and the second composite image. Furthermore, providing the visualization then comprises providing the modified medical image.

[0071] In particular, the modified medical image may include data from the acquired medical image, the first composite image, and / or the second composite image. In particular, the modified medical image may include all intensities of voxels of the acquired medical image, and / or additional data originating from the first composite image and the second composite image may be included as an overlay. For example, a marker having a position based on data originating from the first composite image and / or the second composite image (and optionally also the acquired medical image) may be included as an overlay. Examples of such markers are arrows or bounding boxes.

[0072] The inventors have recognized that by determining a modified medical image, spatial correlations and / or deviations between the acquired medical image, the first composite image, and the second composite image can be included in the visualization. Specifically, regions of the acquired medical image corresponding to low correlations and / or high deviations can be directly labeled. Because these regions correspond to areas where different classification values ​​are expected to differ, information about these regional labels can be determined regarding regions that result in a particular outcome for the classification function.

[0073] According to yet another aspect of the present invention, a modified medical image is based on a difference between the first composite image and the acquired medical image, and a difference between the second composite image and the acquired medical image. Specifically, the modified medical image is based on an absolute value of the difference between the first composite image and the acquired medical image, and an absolute value of the difference between the second composite image and the acquired medical image. Specifically, the difference between the composite image and the acquired medical image is a voxel-by-voxel difference in intensity values ​​of corresponding voxels.

[0074] The inventors have realized that higher values ​​of such a difference correspond to larger deviations between the acquired medical image and the first synthetic image, which means that the areas with these deviations have a large influence on the result of the classification function.

[0075] According to another aspect of the present invention, each voxel of the modified medical image includes a first intensity value, a second intensity value, and a third intensity value. Here, the first intensity value corresponds to the intensity value of the corresponding voxel of the acquired medical image, the second intensity value corresponds to the intensity value of the corresponding voxel of the difference between the first composite image and the acquired medical image, and the third intensity value corresponds to the intensity value of the corresponding voxel of the difference between the second composite image and the acquired medical image.

[0076] In particular, the first intensity values ​​of the modified medical image may form a first color channel of the modified medical image, the second intensity values ​​of the modified medical image may form a second color channel of the modified medical image, and the third intensity values ​​of the modified medical image may form a third color channel of the modified medical image. In particular, the first color channel may correspond to a grayscale, wherein high intensity values ​​correspond to white, low intensity values ​​correspond to black, or vice versa, and intermediate intensity values ​​correspond to black. In particular, the second color channel may correspond to a color scale between transparent and a predetermined first color (e.g., red), wherein low intensity values ​​(corresponding to small differences) correspond to full transparency, and wherein high intensity values ​​(corresponding to large differences) correspond to the predetermined first color with no or less than full transparency. In particular, the third color channel may correspond to a color scale between transparent and a predetermined second color (e.g., blue), wherein low intensity values ​​(corresponding to small differences) correspond to full transparency, and wherein high intensity values ​​(corresponding to large differences) correspond to the predetermined second color with no or less than full transparency.

[0077] The inventors recognized that by using three different intensity values ​​within the modified medical image, the deviations between the first synthetic image, the second synthetic image, and the acquired medical image (corresponding to the regions that have the greatest impact on the result of the classification function) can be displayed in a fine-grained manner. In particular, by using transparency to display the second and third intensities, the complete information of the acquired medical image can be displayed in the visualization, and small values ​​of the second intensity do not hide large values ​​of the third intensity, and vice versa.

[0078] According to yet another aspect of the present invention, the actual classification value, the first classification value, and the second classification value are comparable. In addition, the first classification value is smaller than the actual classification value, and the second classification value is larger than the actual classification value.

[0079] The inventors have recognized that by using a first classification value that is smaller than the actual classification value and a second classification value that is larger than the actual classification value, a decision explanation or classification explanation can be given as to why the actual classification value is neither higher nor lower than the actual classification.

[0080] According to yet another aspect of the present invention, the actual classification value, the first classification value, and the second classification value are elements of a plurality of possible classification values ​​for the anatomical structure, the plurality of possible classification values ​​being a discrete and comparable set, wherein the first classification value is a maximum classification value of the plurality of possible classification values ​​that is less than the actual classification value, and wherein the second classification value is a minimum classification value of the plurality of possible classification values ​​that is greater than the actual classification value.

[0081] In other words, the plurality of possible classification values ​​may be arranged in ascending order, the first classification value and the second classification value are two neighboring values ​​of the actual classification value, the first classification value is smaller than the actual classification value, and the second classification value is larger than the actual classification value.

[0082] The inventors have recognized that by using the described selection of first and second classification values, the classification explanation may include information about why the classification function selected the actual classification value rather than one of the other closest classification values ​​in the stepwise sequence.

[0083] According to another aspect of the present invention, the anatomical structure in the acquired medical image is a lesion. Furthermore, the actual classification value, the first classification value, and / or the second classification value correspond to the benignity and / or malignancy of the lesion. In particular, the classification value corresponding to the benignity and / or malignancy of the lesion indicates the benignity and / or malignancy of the lesion. In particular, the indication can be given according to a standardized classification scheme, such as the TNM classification of malignant tumors, BI-RADS (an acronym for "Breast Imaging Reporting and Data System"), Ann Arbor staging, modified Ann Arbor staging, or CIN staging (an acronym for "Cervical Intraepithelial Neoplasia").

[0084] In general, a lesion is an injury or abnormal change in the tissues of an organism, usually caused by disease or trauma. Lesions can occur anywhere in the body. Lesions can often be classified based on their pattern, size, location, or cause.

[0085] In particular, a lesion can be a tumor. In particular, a tumor is an abnormal and excessive growth of a tissue. When such an abnormal growth can be referred to as a tumor, such abnormal growth typically forms a mass. In particular, tumors can be classified as benign tumors, in situ tumors, malignant tumors, and / or tumors of uncertain or unknown behavior. Malignant tumors can also be denoted as "cancer."

[0086] According to a potential further aspect of the present invention, if the actual classification value corresponds to a benign lesion, the first classification value and / or the second classification value may correspond to a malignant lesion; alternatively, if the actual classification value corresponds to a malignant lesion, the first classification value and / or the second classification value may correspond to a benign lesion.

[0087] According to a potential further aspect of the invention, the actual classification value, the first classification value and / or the second classification value correspond to a degree of benignity and / or malignancy of the lesion, wherein the first classification value corresponds to a degree one level lower than the actual classification value, and / or wherein the second classification value corresponds to a degree one level higher than the actual classification value.

[0088] According to a potential further aspect of the invention, the actual classification value, the first classification value and / or the second classification value correspond to a degree of benignity and / or malignancy of the lesion, wherein the first classification value corresponds to a degree one level higher than the actual classification value, and / or wherein the second classification value corresponds to a degree one level lower than the actual classification value.

[0089] The inventors have recognized that by using the lesion-related method, a classification explanation regarding the benignity and / or malignancy of a particular lesion can be provided by visualizing the differences between the actual anatomical structure and hypothetical structures having different benign and / or malignant characteristics.

[0090] According to another aspect of the present invention, the first generating function and / or the second generating function are based on a generative adversarial algorithm, in particular, a cycle-consistent generative adversarial algorithm. In particular, the first generating function and the second generating function are based on a cycle-consistent generative adversarial algorithm.

[0091] Generative adversarial algorithms (GA algorithms, by their acronym) consist of a generator function and a decision function. Here, the generator function creates synthetic data, and the decision function distinguishes between synthetic data and real data. In particular, what is achieved by training the generator function and / or the decision function is that, on the one hand, the generator function creates synthetic data that is incorrectly classified as real by the decision function, and on the other hand, the decision function can distinguish between real data and synthetic data as much as possible. In game theory, generative adversarial algorithms can also be interpreted as zero-sum games. The training of the generator function and / or the decision function is based in particular on minimizing the cost function in each case.

[0092] By making the first generating function the same as the generating function of the generative adversarial algorithm, the first generating function can be based on the generative adversarial algorithm.The first generating function and the second generating function can be based on the same generative adversarial algorithm, or on different generative adversarial algorithms.

[0093] The cycle-consistent generative adversarial algorithm (acronym "CC-GA algorithm") is based on a first GA algorithm and a second GA algorithm. The generating function of the first GA algorithm converts input data from the first domain into output data from the second domain, wherein the generating function of the second GA algorithm converts input data from the second domain into output data from the first domain. In particular, by training the generating and / or decision functions, it is achieved that the cascade of the generating function of the first GA algorithm and the generating function of the second GA algorithm approaches the identity function, and vice versa. The training of the generating and / or decision functions is in particular based on the minimization of a cost function that includes a cycle consistency term in each case. In particular, the first domain and the second domain can be images comprising anatomical structures having a specific classification value.

[0094] The inventors have recognized that the GA algorithm is suitable for generating realistic synthetic images. In particular, by using the GA algorithm, the synthetic images can contain realistic anatomical structures associated with different classification values. Furthermore, the inventors have recognized that by using the CC-GA algorithm, the anatomical structures associated with different classification values ​​can have shapes that are very similar to the original anatomical structures in the acquired medical image, differing only in the portions necessary to simulate the different classification values.

[0095] According to yet another aspect of the invention, the first generating function is a generating part of a first generative adversarial algorithm, and / or wherein the second generating function is a generating part of a second generative adversarial algorithm.

[0096] According to another aspect of the present invention, the first generative function is the generative part of a first generative adversarial algorithm, wherein the second generative function is the generative part of a second generative adversarial algorithm, and wherein the first generative adversarial algorithm and the second generative adversarial algorithm form a cycle-consistent generative adversarial algorithm.

[0097] According to another aspect of the present invention, the first generating function and / or the second generating function is an artificial neural network. In particular, the corresponding universal adversarial algorithm is a universal adversarial network.

[0098] In particular, if the generator function and the decision function are given by a network, in particular an artificial neural network, the GA algorithm is denoted as a GA network (acronym "GAN"), and the GA algorithm is also called a GA network (also called "GAN", which is an acronym for "Generative Adversarial Network"). This is known in particular from the publication "Generative Adversarial Networks" by Ian J. Goodfellow, arxiv 1406.2661 (2014), the entire content of which is hereby incorporated by reference. The cost function can be minimized in particular by backpropagation.

[0099] According to another aspect of the present invention, the classification function is based on a generative adversarial algorithm, in particular, a cycle-consistent generative adversarial algorithm. The inventors realized that by connecting the classification function and the generative adversarial algorithm, the decision explanation can be closely linked to the classification function.

[0100] According to another aspect of the present invention, the classification function is based on a decision portion of a generative adversarial algorithm or a cycle-consistent generative adversarial algorithm. In particular, the decision portion of the generative adversarial algorithm, when applied to an input image, can provide two results, namely, a first result corresponding to whether the input image is a real / acquired medical image or a synthesized image, and a second result corresponding to a classification value of an anatomical structure within the input image.

[0101] According to another aspect of the present invention, the classification function is an artificial neural network, in particular, the classification function is a deep artificial neural network, in particular, the classification function is a convolutional deep neural network. The inventors have recognized that classification functions based on neural networks, in particular based on deep neural networks, in particular based on deep convolutional neural networks, are particularly suitable for classifying (medical) images.

[0102] According to a second aspect, the present invention relates to an interpretation system for providing classification interpretations, the system comprising an input interface, an output interface and a calculation unit.

[0103] - wherein the input interface is configured for receiving an acquired medical image corresponding to the examination volume,

[0104] wherein the input interface and / or the calculation unit is configured for receiving or determining an actual classification value, in particular by applying a classification function to the acquired medical image to generate the actual classification value,

[0105] The actual classification value classifies the anatomical structure in the acquired medical image.

[0106] wherein the input interface or the computing unit is configured to receive or determine a first classification value that is different from the actual classification value,

[0107] The first classification value is a potential classification value of the anatomical structure in the acquired medical image,

[0108] - wherein the computing unit is configured to apply a first generating function to the acquired medical image, thereby generating a first synthetic image,

[0109] Wherein, the first generating function is based on the first classification value,

[0110] wherein the first generating function is configured to modify the anatomical structure to exhibit a first classification value,

[0111] - wherein the output interface is configured for providing a visualization based on the acquired medical image and the first composite image.

[0112] According to another aspect, the present invention relates to an explanation system for providing classification explanations, the system comprising an input interface, an output interface, and a calculation unit.

[0113] - wherein the input interface is configured for receiving an acquired medical image corresponding to the examination volume,

[0114] - wherein the computing unit is configured for applying the classification function to the acquired medical image, thereby generating an actual classification value classifying the anatomical structure within the acquired medical image,

[0115] wherein the input interface and / or the calculation unit is configured to receive or determine a first classification value that is different from the actual classification value,

[0116] The first classification value is a potential classification value of the anatomical structure in the acquired medical image,

[0117] - wherein the computing unit is configured to apply a first generating function to the acquired medical image, thereby generating a first synthetic image,

[0118] Wherein, the first generating function is based on the first classification value,

[0119] wherein the first generating function is configured to modify the anatomical structure to exhibit a first classification value,

[0120] - wherein the output interface is configured for providing a visualization based on the acquired medical image and the first composite image.

[0121] According to another aspect, the present invention relates to an explanation system for providing classification explanations, the system comprising an input interface, an output interface, and a calculation unit.

[0122] wherein the input interface is configured for receiving an acquired medical image corresponding to the examination volume, the acquired medical image comprising an anatomical structure, in particular wherein the anatomical structure is classified by an actual classification value,

[0123] - wherein the input interface is configured for receiving a first classification value, wherein the first classification value is a potential classification value of an anatomical structure within the acquired medical image,

[0124] - wherein the computing unit is configured to apply a first generating function to the acquired medical image, thereby generating a first synthetic image,

[0125] Wherein, the first generating function is based on the first classification value,

[0126] wherein the first generating function is configured to modify the anatomical structure to exhibit a first classification value,

[0127] - wherein the output interface is configured for providing a visualization based on the acquired medical image and the first composite image.

[0128] In particular, the interpretation system is configured to perform the previously described method for providing classification interpretations. The interpretation system is configured to perform the previously described method for providing classification interpretations and aspects thereof through their interfaces and their computing units configured to perform the corresponding method steps.

[0129] The interpretation system can be implemented as a data processing system or part of a data processing system. Such a data processing system may include, for example, a cloud computing system, a computer network, a computer, a tablet computer, a smartphone, a microprocessor, etc. The interpretation system may include hardware and / or software. The hardware may include, for example, a processor system, a memory system, or a combination thereof. The hardware may be configured and / or operated by software.

[0130] Whenever a module such as an interface, a computing unit or a memory unit is mentioned herein, it should be understood that this may refer to a module implemented as hardware and / or software. Modules are mentioned and described in an easy-to-understand manner, and it will be apparent that all functions of all modules can be implemented by one and the same body of computer program instructions (or: computer code). In addition, without departing from the present disclosure, the functions of any or all modules may overlap, and some modules may be integrated into other modules. In addition, each module may also be a logical combination of several (same or different) physical submodules, and submodules may also be distributed to different spatial locations.

[0131] According to a third aspect, the present invention relates to a medical imaging system comprising an interpretation system according to aspects of the present invention. In particular, the medical imaging system may be an X-ray imaging system, a computed tomography imaging system, a magnetic resonance imaging system, a PET imaging system, a SPECT imaging system, and / or an ultrasound imaging system. Furthermore, the medical imaging system may be a microscopic imaging system, a histopathology imaging system, and / or a time-continuous biosignal analysis imaging system.

[0132] According to a fourth aspect, the present invention relates to a computer-implemented method for providing a first generating function, comprising:

[0133] - receiving a first training image, the first training image comprising a first training anatomical structure, the first training anatomical structure being classified by a first training classification value,

[0134] - receiving or determining a second training classification value different from the first training classification value,

[0135] wherein the second training classification value is a potential classification value of the first anatomical structure in the first training medical image,

[0136] - applying the first generating function to the first training image, thereby generating a modified first training image,

[0137] Wherein, the first generating function is based on the second training classification value,

[0138] - receiving a first input image, the first input image comprising a first input anatomical structure, the first input anatomical structure being classified by a second trained classification value,

[0139] - applying a first decision function to the first input image, thereby generating a first decision value,

[0140] - applying the first decision function to the modified first training image to generate a second decision value,

[0141] - adjusting parameters of the first generation function and / or the first decision function based on the generative adversarial loss function,

[0142] The generative adversarial loss function is based on the first discriminator value and the second discriminator value,

[0143] - providing a first generation function and / or a first decision function.

[0144] In particular, the training image may be an acquired medical image. The training image may include all advantageous features and possible embodiments of the acquired medical image. In particular, the input image may be an acquired medical image. The input image may include all advantageous features and possible embodiments of the acquired medical image. In particular, the modified training image may be the first composite image. The modified training image may include all advantageous features and possible embodiments of the first composite image.

[0145] In particular, the first training classification value can be an actual classification value for the anatomical structure within the first training image. In particular, the first training classification value can include all advantageous features and possible embodiments of the actual classification value. In particular, the first training classification value can be determined by applying a classification function to the first training image. In particular, the second training classification value is the classification value according to which the modified first training image is to be classified. In particular, the second training classification value can be determined by applying the classification function to the first input image.

[0146] In particular, the decision function is a trained function. In particular, the decision function is trained to distinguish real or acquired input data from synthetic input data. In particular, by making the first generating function the same as the generating function of the generative adversarial network, and by making the first decision function the same as the decision function of the generative adversarial algorithm, the first generating function and the first decision function can be based on the generative adversarial algorithm.

[0147] The discriminator value is the result of applying the discriminator function to the imaging data. In particular, the discriminator value comprises a probability value, wherein, ideally, the probability value is 1 for real imaging data used as input to the discriminator function and the probability value is 0 for synthetic data used as input to the discriminator function.

[0148] In particular, adjusting a parameter of the trained function can be based on minimizing a loss function. In particular, the loss function can include an adversarial loss function. In particular, minimizing the loss function can be based on stepwise (gradient) descent. In particular, if the trained function is an artificial neural network, the parameter can correspond to at least one edge weight of the corresponding network, and minimizing the parameter can be based on a backpropagation algorithm.

[0149] In particular, the adversarial loss function includes a term that causes a loss to the decision function if the decision function incorrectly determines that the synthesized input image is a real image and / or a term that causes a loss to the decision function if the decision function incorrectly determines that the real input image is a synthesized image. In particular, the adversarial loss function includes a term that causes a loss to the generator function.

[0150] The inventors realized that, based on the described approach, a first generating function can be provided that allows explaining the classification decision.

[0151] In this training method, the first training image and the first input image are acquired medical images (including real anatomical structures), and the first modified training image is a synthetic image (not including real anatomical structures but synthesized anatomical structures). The anatomical structures in the first training image correspond to a first classification value, while the anatomical structures in the second input image correspond to a second classification value.

[0152] The first decision value is the result of the first decision function applied to the first input image. Using an ideal first decision function, the first decision value is 1, corresponding to the result of the first decision function indicating that the first input image is a true image. The second decision value is the result of the first decision function applied to the modified first training image. Using an ideal first decision function, the first decision value is 0, corresponding to the result of the first decision function indicating that the modified first training image is a true training image.

[0153] By using the adversarial loss function, the parameters of the first decision function are adjusted so that the output of the first decision function approaches the ideal value. At the same time, the parameters of the first generator function are adjusted so that the output of the first generator function causes the first decision function to mistakenly classify it as a real image. This means that the first generator function is trained by the proposed method to mimic real medical images that include anatomical structures with the second classification value.

[0154] According to yet another aspect, a method for providing a first generating function comprises:

[0155] - applying the second generating function to the modified first training image to generate a comparison first training image,

[0156] Wherein, the second generating function is based on the first training classification value,

[0157] - adjusting parameters of the first generating function and / or the second generating function based on a cycle consistency loss function,

[0158] Among them, the cycle consistency loss function is based on comparing the difference between the first training image and the second training image.

[0159] In particular, the cycle consistency loss function includes a term that causes a loss of the first generating function and / or the second generating function for comparing the deviation between the first training image and the second training image.

[0160] The inventors have recognized that by using a cycle consistency loss, the first and / or second generator functions can be trained to provide synthetic images that are very similar to the inputs to the generator functions, yet simulate different classification values. Such generator functions are well suited for decision interpretation in the image field, particularly in the medical image field.

[0161] According to yet another aspect, a method for providing a first generating function comprises:

[0162] - receiving a second training image, the second training image comprising a second anatomical structure, the second anatomical structure being classified by a second training classification value,

[0163] - applying the second generating function to the second training image, thereby generating a modified second training image,

[0164] Wherein, the second generating function is based on the first training classification value,

[0165] - receiving a second input image, the second input image comprising a second input anatomical structure, the second input anatomical structure being classified by the first trained classification values,

[0166] - applying the second decision function to the second input image, thereby generating a third decision value,

[0167] - applying the second decision function to the modified second training image to generate a fourth decision value,

[0168] - adjusting parameters of the second generation function and / or the second decision function based on the generative adversarial loss function,

[0169] The generative adversarial loss function is based on the third discriminator value and the fourth discriminator value.

[0170] The inventors recognized that by using generative adversarial training, two generating functions (or equivalently, the same generating function based on different classification values) can be trained to mimic an input image.

[0171] According to a potential further aspect, a method for providing a first generating function comprises:

[0172] - applying the first generating function to the modified second training image to generate a comparison second training image,

[0173] Wherein, the first generating function is based on the second training classification value,

[0174] - adjusting parameters of the first generating function and / or the second generating function based on a cycle consistency loss function,

[0175] Among them, the cycle consistency loss function is based on comparing the difference between the first training image and the second training image.

[0176] The inventors have recognized that by using a cycle consistency loss, the first and / or second generating functions can be trained to provide synthetic images that closely resemble the inputs to the generating functions, yet simulate different classification values. By applying a cycle consistency loss to the possible orders of the first and second generating functions, the consistency of the generating functions can be even better. Such generating functions are well-suited for decision interpretation in the field of images, particularly medical images.

[0177] According to another aspect of the present invention, the parameters of the first generation function and / or the second generation function are also adjusted based on the classification loss function, wherein the classification loss function is based on the difference between the second classification value and the result of using the classification function on the modified first medical image, and / or wherein the classification loss function is based on the difference between the first classification value and the result of using the classification function on the modified second medical image.

[0178] The inventors have recognized that by using a classification loss during training, or by using a classification function during training, the output of the generator function can be better adapted to the performance of the classification function. In particular, the generator functions are trained so that their outputs will actually be classified by the classification function as the classification values ​​on which they are based. In particular, through this feature, the generator functions are trained to operate in the classification space given by the classification function, rather than in an externally given classification value space.

[0179] According to another aspect, the present invention relates to a training system for providing a first generating function, the system comprising an input interface, an output interface and a calculation unit.

[0180] wherein the input interface is configured for receiving a first training image, the first training image comprising a first training anatomical structure, the first training anatomical structure being classified by a first training classification value,

[0181] wherein the input interface and / or the calculation unit is configured to receive or determine a second training classification value that is different from the first training classification value,

[0182] wherein the second training classification value is a potential classification value of the first anatomical structure in the first training medical image,

[0183] wherein the computing unit is configured to apply the first generating function to the first training image, thereby generating a modified first training image,

[0184] Wherein, the first generating function is based on the second training classification value,

[0185] - wherein the input interface is configured for receiving a first input image, the first input image comprising a first input anatomical structure, the first input anatomical structure being classified by a second trained classification value,

[0186] - wherein the calculation unit is configured for applying a first decision function to the first input image, thereby generating a first decision value,

[0187] wherein the computing unit is configured to apply the first decision function to the modified first training image, thereby generating a second decision value,

[0188] wherein the computing unit is configured to adjust parameters of the first generation function and / or the first decision function based on a generative adversarial loss function,

[0189] The generative adversarial loss function is based on the first discriminator value and the second discriminator value,

[0190] - wherein the output interface is configured to provide the TPROV first generation function and / or the first decision function.

[0191] In particular, the training system is configured to perform the previously described method for providing a first generating function.The training system is configured to perform the previously described method for providing a classification explanation and aspects thereof via its interface and its computing units configured to perform the corresponding method steps.

[0192] The training system can be implemented as a data processing system or part of a data processing system. Such a data processing system may include, for example, a cloud computing system, a computer network, a computer, a tablet computer, a smartphone, a microprocessor, etc. The interpretation system may include hardware and / or software. The hardware may include, for example, a processor system, a memory system, or a combination thereof. The hardware may be configured and / or operated by software.

[0193] According to another aspect of the present invention, in the method for providing classification interpretation according to the present invention and aspects thereof, the first generating function and / or the second generating function has been provided by the method for providing the first generating function according to the present invention and aspects thereof.

[0194] According to a further aspect, the present invention relates to a system comprising an interpretation system according to the present invention and aspects thereof or a medical imaging system according to the present invention and aspects thereof, and a training system according to the present invention and aspects thereof, wherein a first generating function provided by the training system is used within the interpretation system.

[0195] According to a further aspect, the invention relates to a computer program or a computer program product comprising instructions for causing an interpretation system to perform the method for providing classification interpretations according to the invention and aspects thereof, when the program is executed by the interpretation system.

[0196] According to yet another aspect, the present invention relates to a computer-readable medium comprising instructions which, when executed by an interpretation system, cause the interpretation system to perform a method for providing classification interpretations according to the present invention and aspects thereof.

[0197] According to a further aspect, the invention relates to a computer program or a computer program product comprising instructions for causing a training system to perform the method for providing a first generating function according to the invention and aspects thereof, when the program is executed by the training system.

[0198] According to a further aspect, the invention relates to a computer-readable medium comprising instructions which, when executed by a training system, cause the training system to perform the method for providing a first generating function according to the invention and aspects thereof.

[0199] Implementing the invention by means of a computer program product and / or a computer-readable medium has the advantage that already existing production systems can be easily adapted by means of a software update in order to operate according to the proposal of the invention.

[0200] A computer program product may be, for example, a computer program or include another element in addition to the computer program. The other element may be hardware, such as a storage device storing the computer program, a hardware key for using the computer program, etc., and / or software, such as documentation or a software key for using the computer program.

[0201] The characteristics, features, and advantages of the present invention described above, as well as the manner in which they are implemented, will become clearer and more readily understood from the following description and examples, which are described in detail in the context of the accompanying drawings. This following description does not limit the present invention to the included embodiments. Identical components or parts may be labeled with the same reference numerals in different figures. In general, the figures are not drawn to scale.

[0202] The numbering and / or order of the method steps is intended to facilitate understanding and, unless otherwise explicitly stated or implicitly clear, should not be interpreted as meaning that a given step must be performed according to the number of its reference numerals and / or the order in which it appears in the drawings. In particular, several or even all method steps may be performed simultaneously, in an overlapping manner, or sequentially. BRIEF DESCRIPTION OF THE DRAWINGS

[0203] Figure 1 shows a first data flow diagram according to aspects of the present invention,

[0204] Figure 2 shows a second data flow diagram according to aspects of the present invention,

[0205] Figure 3 shows a first data flow diagram associated with an aspect of the present invention for providing a first generating function,

[0206] Figure 4 shows a second data flow diagram associated with aspects of the present invention for providing a first generating function,

[0207] Figure 5 shows a third data flow diagram relating to aspects of the present invention for providing a first generating function,

[0208] Figure 6 A first embodiment of a method for providing classification interpretation according to the present invention is shown,

[0209] Figure 7 A second embodiment of the method for providing classification interpretation according to the present invention is shown,

[0210] Figure 8 A third embodiment of the method for providing classification interpretation according to the present invention is shown,

[0211] Figure 9 Schematically showing potential outcomes of the method for providing classification explanations,

[0212] Figure 10 Additional potential outcomes of the method for providing classification explanations are shown,

[0213] Figure 11 Additional potential outcomes of the method for providing classification explanations are shown,

[0214] Figure 12 shows a comparison of different generating functions used in the context of the present invention,

[0215] Figure 13 The results of the following study comparing the results of the proposed method based on different types of generating functions are shown,

[0216] Figure 14 shows a flow chart of a first embodiment of a method for providing a first generating function,

[0217] Figure 15 shows a flow chart of a second embodiment of a method for providing a first generating function,

[0218] Figure 16 shows a flow chart of a third embodiment of a method for providing a first generating function,

[0219] Figure 17 shows a flow chart of a fourth embodiment of a method for providing a first generating function,

[0220] Figure 18 An interpretation system, a medical imaging system, and a training system are shown. DETAILED DESCRIPTION

[0221] Figure 1 shows a first data flow diagram according to aspects of the present invention, Figure 2 A second data flow graph according to an aspect of the present invention is shown. Compared to the first data flow graph, a second generating function GF.2 appears in the second data flow graph. Figure 2 (Described in the context of the second generating function GF.2) Figure 1 and Figure 2 However, the following description can also be understood as Figure 1 Related.

[0222] The seed element of the first data flow graph and the second data flow graph is the acquired medical image IMG.A. By applying the classification function CF to the medical image IMG.A acquired by APPL-CV, the actual classification value CV.A is determined or generated. The acquired medical image IMG.A is represented by x, and c A represents the actual classification value CV.A, which can be determined as c A =CF(x).

[0223] In the data flow diagram, there is also a first classification value CV.1 and a second classification value CV.2. These additional classification values ​​CV.1 and CV.2 can be received as user input, or they can be determined based on known quantities (for example, based on the actual classification value CV.A, as shown by the dotted line).

[0224] Furthermore, a first synthetic image IMG.1 and a second synthetic image IMG.2 are determined by using a first generation function GF.1 that uses the acquired medical image IMG.A as input and is based on a first classification value CV.1, and a second generation function GF.2 that uses the acquired medical image IMG.A as input and is based on a second classification value CV.2.

[0225] The generating functions GF.1, GF.2 can be based on the categorical values ​​CV.1, CV.2 by using them as additional input parameters. Denoting GF1 and GF2 as generating functions GF.1, GF.2, y1 and y2 as composite images IMG.1, IMG.2, and c1 and c2 as categorical values ​​CV.1, CV.2, will result in y1 = GF1(x, c1) and y2 = GF2(x, c2). In this alternative, the generating functions GF.1, GF.2 are based on GF. 1 / 2 :R n ×C→R nMap images and classification values ​​to images, where R represents a real number, n represents the dimension of the images IMG.A, IMG.1, IMG.2, and C represents the set of possible classification values ​​CV.1, CV.2, CV.A. In particular, the generating functions GF.1 and GF.2 can be the same.

[0226] Alternatively, the generating functions GF.1, GF.2 may be based on the classification values ​​CV.1, CV.2 by selecting said generating functions GF.1, GF.2 from a plurality of generating functions based on the classification values ​​CV.1, CV.2. In this alternative, the generating functions GF.1, GF.2 are selected according to GF.1, GF.2. 1 / 2 :R n →R n Mapping an image to an image. In particular, in this alternative, the possible first classification value CV.1 and the second classification value CV.2 are restricted based on the available generating functions. For example, if the classification values ​​CV.1, CV.2, CV.A are comparable and discrete, and if the plurality of available generating functions includes a first available generating function (denoted as G ) that transforms the anatomical structure in the medical image at a higher level according to the classification values ​​CV.1, CV.2, CV.A. + ) and a second available generating function (denoted as G ) for transforming the anatomical structure in the lower level medical image according to the classification values ​​CV.1, CV.2, CV.A - ), then if c 1 / 2 =c a +1 for G 1 / 2 =G + And if c 1 / 2 =c a –1 G 1 / 2 =G - (Here, +1 and -1 are not necessarily arithmetic operations, but represent going to the next level or the previous level of the classification values ​​CV.1, CV.2, and CV.A).

[0227] Optionally, a modified medical image IMG.M is present in the data flow diagram. The modified medical image IMG.M may be determined based on the acquired medical image IMG.A and the first synthesized image IMG.1; or the modified medical image IMG.M may be determined based on the acquired medical image IMG.A, the first synthesized image IMG.1, and the second synthesized image IMG.2.

[0228] Figure 3 、 Figure 4 and Figure 5A data flow diagram related to aspects of the present invention for providing a first generating function GF.1 is shown. It must be understood that objects that appear twice in the same flow diagram (e.g., the first generating function GF.1) do refer to the same object and are only drawn twice for better visibility of the data flow. Furthermore, the dashed lines do not represent direct data flow, but rather indicate which loss functions are used to adjust the parameters of which trained functions GF.1, GF.2, DF.1, and DF.2.

[0229] Figure 3 and Figure 4 The seed elements of the data flow graph shown in FIG are a first training image IMG.T1 and a first input image IMG.S1. The first training image IMG.T1 includes a first anatomical structure, and the first input image IMG.S1 includes a first input anatomical structure. The first anatomical structure and the first input anatomical structure are of the same type (e.g., both are lesions, such as lung nodules), but the first anatomical structure is classified by a first training classification value CV.T1, while the first input anatomical structure is classified by a second training classification value CV.T2, and the first training classification value CV.T1 and the second training classification value CV.T2 are different.

[0230] exist Figure 5 In the embodiment shown in FIG, the additional seed elements are a second training image IMG.T2 and a second input image IMG.S2. The second training image IMG.T2 includes a second anatomical structure, and the second input image IMG.S2 includes a second input anatomical structure. The second anatomical structure and the second input anatomical structure are of the same type as the first anatomical structure and the first input anatomical structure (e.g., both are lesions, such as lung nodules), wherein the second anatomical structure is classified by a second training classification value CV.T2, while the second input anatomical structure is classified by a first training classification value CV.T1, and the first training classification value CV.T1 and the second training classification value CV.T2 are different.

[0231] Based on the training images IMG.T1 and IMG.T2, by applying the first generating function GF.1 and the second generating function GF.2, the modified training images IMG.T1' and IMG.T2' can be calculated. 1 / 2 Denote the first training image IMG.T1 and the second training image IMG.T2, with c 1 / 2 Denote the first training classification value CV.T1 and the second training classification value CV.T2, and use GF 1 / 2Denoting the first generating function GF.1 and the second generating function GF.2, the modified training images IMG.T1' and IMG.T2' can be calculated as t1' = GF1(t1, c2) and t2' = GF2(t2, c1). The training classification values ​​CV.T1 and CV.T2 can be given as input parameters or obtained by applying the classification function CF to the first input image IMG.S1 (denoted by s1) according to c2 = CF(s1), and / or applying the classification function CF to the second input image IMG.S2 (denoted by s2) according to c1 = CF(s2). Alternatively, the training images IMG.T1 and IMG.T2 can also be used here.

[0232] Here, the generating functions GF.1, GF.2 are based on the classification values ​​CV.T1, CV.T2 by using them as additional input parameters. In particular, the first generating function GF.1 and the second generating function GF.2 can be equivalent and differ only in the value of the additional input parameter used. Alternatively, as with respect to Figure 1 and Figure 2 As described, the generating functions GF.1, GF.2 can be selected from a plurality of possible generating functions.

[0233] exist Figures 3 to 5 In the embodiment shown, a first decision function DF.1 (denoted as DF1) and a second decision function DF.2 (denoted as DF2) are also used. The decision functions DF.1 and DF.2 take an image as input and output a decision value. In particular, the decision value can be a probability value between 0 and 1 indicating whether the input image IMG.S1 and IMG.S2 is a real / acquired image (as the first input image IMG.S1 or the second input image IMG.S2) or a synthesized / generated image (as the modified first training image IMG.T1' or the modified second training image IMG.T2') according to the decision functions GF.1 and GF.2.

[0234] In these embodiments, similar to the generating functions GF.1 and GF.2, the decision functions DF.1 and DF.2 are based on the classification values ​​CV.T1 and CV.T2 by using the classification values ​​CV.T1 and CV.T2 as additional input parameters or by selecting from a plurality of decision functions based on the classification values ​​CV.T1 and CV.T2. In the former case, the first decision function DF.1 and the second decision function DF.2 may be equivalent and differ only in the value of the additional input parameter used. DF1 and DF2 are represented as decision functions DF.1 and DF.2. In a first alternative, the decision functions DF.1 and DF.2 are based on DF. 1 / 2 :R n×C→[0,1] maps the image and classification values ​​CV.T1, CV.T2 to real numbers (as probability values), where R represents a real number, n represents the dimension of the input images IMG.S1, IMG.S2 and C represents the set of possible classification values ​​CV.T1, CV.T2. In the second alternative, the decision functions DF.1, DF.2 are mapped to real numbers (as probability values). 1 / 2 :R N →[0,1] maps the image to real numbers (as probability values).

[0235] The decision functions DF.1 and DF.2 are used to determine the decision values ​​used in the adversarial loss L.ADV. The first decision value can be calculated as d1 = DF1(s1, c2), and the second decision value can be calculated as d2 = DF1(t1', c2) = DF1(GF1(t1, c2), c2). In addition, Figure 5 In the embodiment shown in FIG, the third determination value can be calculated as d3=DF2(s2,c1), and the fourth determination value can be calculated as d4=DF2(t2',c1)=DF2(GF2(t2,c1),c1).

[0236] exist Figures 3 to 5 In the embodiment shown in FIG, the adversarial loss L.ADV has a component L for the first decision function DF.1. (DF1) adV -logDF1(s1,c2)-log[1-DF1(GF1(t1,c2),c2)], and the component L for the first generating function GF.1 (GF1) adV ~log[1-DF1(GF1(t1,c2),c2)]. Figure 5 In the embodiment shown in FIG, the adversarial loss L.ADV has a component L for the second decision function DF.2. (DF2) adv ~-logDF2(s2,c1)-log[1–DF2(GF2(t2,c1),c1)], and the component L for the second generating function GF.2 (GF2) adv ~log[1–DF2(GF2(t2,c1),c1)].

[0237] The first and second decision values ​​can be used to adjust the parameters of the first generating function GF.1 and the first decision function DF.1 by minimizing the loss function including the adversarial loss L.ADV. The third and fourth decision values ​​can be used to adjust the parameters of the second generating function GF.2 and the second decision function DF.2 by minimizing the loss function including the adversarial loss L.ADV.

[0238] exist Figure 4 and Figure 5 In the embodiment shown in FIG, the second generating function GF.2 is further applied to the modified first training image IMG.T1′ to determine a comparative first training image IMG.T1″ denoted by t1″. In particular, the comparative first training image IMG.T1″ is determined as t1″=GF2(t1′,c1)=GF2(GF1(t1,c2),c1). Figure 5 In the embodiment shown in , the first generating function GF.1 is also applied to the modified second training image IMG.T2′ to determine a comparative second training image IMG.T2″ represented by t2″. In particular, the comparative second training image IMG.T2″ is determined as t2″=GF1(t2′,c2)=GF1(GF2(t2,c1),c2).

[0239] Comparing the training images IMG.T1” and IMG.T2” can be used for the cycle-consistency loss L.CYC, which has components L cc ~║t1–t1”║ m =║t1–GF2(GF1(t1,c2),c1)║ m and / or component L cc ~║t2–t2”║ m =║t2–GF1(GF2(t2,c1),c2)║ m , where ║x║ m Denotes the m-norm (usually m=1 or m=2). These loss functions can be minimized by adjusting the parameters of the first generating function GF.1 and the second generating function GF.2.

[0240] Figure 6 A first embodiment of a method for providing classification interpretation according to the present invention is shown.

[0241] A first embodiment comprises the following steps: receiving a REC-IMG.A corresponding to an acquired medical image IMG.A of an examination volume. This step is performed by an interface ESYS.IF of an interpretation system ESYS. In this embodiment, the acquired medical image IMG.A is based on a computed tomography imaging process. In particular, the acquired medical image IMG.A is a two-dimensional slice through a lesion within a computed tomography dataset. Alternatively, the acquired medical image IMG.A can also be a three-dimensional medical image including the lesion. In particular, the acquired medical image IMG.A is contained in a DICOM image dataset. The DICOM image dataset may include additional information, such as the patient's name, the patient's age, and / or information related to the underlying imaging examination, such as the imaging protocol used.

[0242] Another step of the illustrated embodiment is receiving or determining a REC-DET-CV.A actual classification value CV.A, wherein the actual classification value CV.A classifies an anatomical structure within the acquired medical image IMG.A. In the illustrated embodiment, the anatomical structure within the acquired medical image IMG.A is a lesion, and the actual classification value CV.A corresponds to benignity and / or malignancy of the lesion within the acquired medical image IMG.A.

[0243] In this embodiment, the actual classification value CV.A is determined by applying the classification function CF to the acquired medical image IMG.A. The corresponding steps are performed by the computing unit ESYS.CU of the interpretation system ESYS. This means that the same system that evaluates the acquired medical image IMG.A (by applying the classification function CF to the acquired medical image IMG.A) also provides an interpretation of the actual classification value CV.A. Alternatively, the actual classification value CV.A can be determined outside of the method step and received within the method step (in particular, via the interface ESYS.IF of the interpretation system ESYS). This means that a different system provides an interpretation of the actual classification value CV.A rather than actually determining the actual classification value CV.A.

[0244] In this embodiment, the actual classification value CV.A is a real number between 0 and 1, where a value of 0 corresponds to a benign nodule and a value of 1 corresponds to a malignant nodule. The values ​​between 0 and 1 correspond to different levels of certainty of the classification function CV (values ​​0 and 1 correspond to maximum certainty, while a value of 0.5 corresponds to maximum uncertainty). Alternatively, the actual classification value CV.A can correspond to a discrete classification of the anatomical structure (e.g., a BI-RADS score for breast imaging).

[0245] Different anatomical structures and different classification values ​​can also be used. An example of a classification value that can be used is a survival estimate. The survival estimate can be given in terms of an amount of time (e.g., months or years) depending on the severity of a particular anatomical structure (e.g., a particular tumor). In such a case, discrete categories of survival estimates can also be used (e.g., "less than 1 month," "two or three months," "four to five months," "six to twelve months," "more than a year"). Another possible classification value can be used for cerebral hemorrhage detection, such as a probability value for the presence of a cerebral hemorrhage. Another possible classification task can be related to medical image segmentation, where the classification value can correspond to a segmentation mask (i.e., a set of voxels associated with a particular anatomical structure).

[0246] Another step of this embodiment is receiving or determining a first classification value CV.1 REC-DET-CV.1 that is different from the actual classification value CV.A, wherein the first classification value CV.1 is a potential classification value of the anatomical structure within the acquired medical image IMG.A. In particular, the potential classification value is given by a potential output of the classification function.

[0247] In this embodiment, the first classification value CV.1 is then also a real number between 0 and 1, like the actual classification value CV.A. Alternatively, if the actual classification value CV.A can be an element of a discrete set of potential classification values ​​(e.g., a BI-RADS score), the first classification value CV.1 can also be selected or determined from this discrete set.

[0248] In this embodiment, the first classification value CV.1 is determined by the calculation unit ESYS.CU of the interpretation system ESYS based on the actual classification value CV.A. Specifically, the first classification value CV.1 can be selected as 0 (to visually explain why the classification did not lead to a definitive benign classification) or as 1 (to visually explain why the classification did not lead to a definitive malignant classification), depending on whether the actual classification value CV.A is less than or greater than 0.5. Alternatively, the first classification value CV.1 can be received, for example, as user input, using the interface ESYS.IF of the interpretation system ESYS. This latter alternative is particularly useful for providing interactive explanations to users, particularly in the field of teaching radiologists.

[0249] If the set of potential classification values ​​is discrete and comparable, the first classification value CV.1 can be the next potential classification value greater than the actual classification value CV.A, or the first classification value CV.1 can be the next potential classification value less than the actual classification value CV.A. This setting enables visualization of why a particular actual classification value CV.A is assigned to an anatomical structure rather than an adjacent classification value.

[0250] A further step of the illustrated embodiment is to apply a first generation function GF.1 to the medical image IMG.A acquired by APPL-GF.1, thereby generating a first composite image IMG.S1. Here, the first generation function GF.1 is based on a first classification value CV.1 and is configured to modify the anatomical structure to exhibit the first classification value CV.1.

[0251] In particular, the first generating function GF.1 is configured or trained to change the shape and / or texture of anatomical structures within the acquired medical image IMG.A without changing the surroundings of the anatomical structures in the acquired medical image IMG.A, thereby creating a first synthesized image IMG.S1 that differs from the acquired medical image IMG.A only in the extent to which the anatomical structures within the synthesized image IMG.S1 will be classified by the first classification value CV.1. In particular, the first synthesized image IMG.S1 has the same dimensionality as the acquired medical image IMG.A, and the first synthesized image IMG.S1 has the same size, measured in number of voxels, as the acquired medical image IMG.A with respect to each dimension.

[0252] In this embodiment, to achieve this goal, the first generating function GF.1 is based on a cycle-consistent generative adversarial algorithm. In particular, the first generating function GF.1 corresponds to the generative part of the cycle-consistent generative adversarial algorithm. In particular, the first generating function GF.1 is an artificial neural network.

[0253] In this embodiment, the first generating function GF.1 is selected from the generating function GF (0) And the generating function GF (1) , the generating function GF (0) is configured to create a composite image including anatomical structures classified with a classification value of 0, the generating function GF (1) is configured to create a composite image comprising an anatomical structure classified with a classification value of 1. In an alternative, if the actual classification value CV.A and the first classification value CV.1 are from a discrete and sortable set, the first generating function GF.1 is selected from the generating function GF (+) And the generating function GF (-) , the generating function GF (+) is configured to create a synthetic image including anatomical structures classified with a classification value one level greater than the classification value of the anatomical structures in the input image, the generating function GF (-) The method is configured to create a composite image including an anatomical structure classified with a classification value one level less than a classification value of the anatomical structure in the input image.

[0254] Alternatively, the first classification value CV.1 may be used as an additional parameter of the first generating function GF.1.

[0255] The final step of the illustrated embodiment is to provide PROV-VIS visualization based on the acquired medical image IMG.A and the first synthesized image IMG.S1. In this embodiment, the acquired medical image IMG.A and the first synthesized image IMG.S1 are displayed side by side and spatially aligned. Alternatively, the first synthesized image IMG.S1 can be displayed as an overlay on the acquired medical image IMG.A, which can be inserted and removed via user interaction. Alternatively, markers can be displayed in the acquired medical image IMG.A based on the first synthesized image IMG.S1.

[0256] Figure 7 A second embodiment of the method for providing classification interpretation according to the present invention is shown.

[0257] The steps of receiving the medical image IMG.A acquired by REC-IMG.A, receiving or determining the actual classification value CV.A of REC-DET-CV.A, receiving or determining the first classification value CV.1 of REC-DET-CV.1 different from the actual classification value CV.A, and applying the first generating function GF.1 to the medical image IMG.A acquired by APPL-GF.1 are as follows: Figure 6 The corresponding steps of the first embodiment shown in FIG. 3 are identical. In particular, they can exhibit all advantageous features and implement the embodiments described therein.

[0258] The second embodiment further comprises the step of determining a DET-IMG.M modified medical image IMG.M based on the acquired medical image IMG.A and the first synthesized image IMG.S1.

[0259] In this embodiment, each voxel of the modified medical image IMG.M comprises a first intensity value and a second intensity value. The first intensity value corresponds to the intensity value of the corresponding voxel of the acquired medical image IMG.A. The second intensity value corresponds to the voxel-wise difference between the modified medical image IMG.M and the first synthesized image IMG.S1. Figure 1 Annotation, use x m Represents the modified medical image IMG.M, which is composed of x m =(x,s1–x) or x m =(x,|s1–x|) is given in voxels.

[0260] Furthermore, providing the PROV-VIS visualization comprises providing the modified medical image IMG.M. In this embodiment, the modified medical image IMG.M comprises two color channels, a first color channel being associated with a first intensity and being a grayscale channel, and a second color channel being associated with a transparent to red scale.

[0261] Figure 8 A third embodiment of the method for providing classification explanations according to the present invention is shown.

[0262] The steps of receiving the medical image IMG.A acquired by REC-IMG.A, receiving or determining the actual classification value CV.A of REC-DET-CV.A, receiving or determining the first classification value CV.1 of REC-DET-CV.1 different from the actual classification value CV.A, and applying the first generating function GF.1 to the medical image IMG.A acquired by APPL-GF.1 are as follows: Figure 6 The corresponding steps of the first embodiment shown in FIG. 3 are identical. In particular, they can exhibit all advantageous features and implement the embodiments described therein.

[0263] A further step of this embodiment is receiving or determining REC-DET-CV.2 a second classification value CV.2 different from the actual classification value CV.A and different from the first classification value CV.1, wherein the second classification value CV.2 is a potential classification value of the anatomical structure within the acquired medical image IMG.A.

[0264] In this embodiment, the second classification value CV.2 is then also a real number between 0 and 1, just like the actual classification value CV.A. Alternatively, if the actual classification value CV.A can be an element of a discrete set of potential classification values ​​(e.g., a BI-RADS score), the second classification value CV.2 can also be selected or determined from this discrete set.

[0265] In this embodiment, the second classification value CV.2 is determined based on the actual classification value CV.A by the calculation unit ESYS.CU of the interpretation system ESYS. Specifically, the first classification value CV.1 is set to 0, and the second classification value CV.2 is set to 1, or vice versa. Alternatively, the interface ESYS.IF of the interpretation system ESYS can be used to receive the second classification value CV.2, for example, as user input. This latter alternative is particularly useful for providing interactive interpretations to users, particularly in the field of educating radiologists.

[0266] If the set of potential classification values ​​is discrete and comparable, the first classification value CV.1 can be the next potential classification value greater than the actual classification value CV.A, while the second classification value CV.2 can be the next potential classification value less than the actual classification value CV.A, and vice versa. This setting makes it possible to visualize why a specific actual classification value CV.A is assigned to an anatomical structure rather than an adjacent classification value.

[0267] A further step in the illustrated embodiment is to apply a second generation function GF.2 to the medical image IMG.A acquired by APPL-GF.2, thereby generating a second composite image IMG.S2. Here, the second generation function GF.2 is based on a second classification value CV.2 and is configured to modify the anatomical structure to exhibit the second classification value CV.2.

[0268] In particular, the second generating function GF.2 is configured or trained to change the shape and / or texture of the anatomical structures within the acquired medical image IMG.A without changing the surroundings of the anatomical structures in the acquired medical image IMG.A, thereby creating a second synthesized image IMG.S2 that differs from the acquired medical image IMG.A only in the extent to which the anatomical structures within the synthesized image IMG.S2 will be classified by the second classification value CV.2. In particular, the second synthesized image IMG.S2 has the same dimensions as the acquired medical image IMG.A, and the second synthesized image IMG.S2 has the same size as the acquired medical image IMG.A in terms of the number of voxels with respect to each dimension.

[0269] In this embodiment, to achieve this goal, the second generating function GF.2 is based on a cycle-consistent generative adversarial algorithm. In particular, the second generating function GF.2 corresponds to the generative part of the cycle-consistent generative adversarial algorithm, and in particular, corresponds to a different generative part of the cycle-consistent generative adversarial algorithm. In particular, the second generating function GF.2 is an artificial neural network.

[0270] In this embodiment, the second generating function GF.2 is selected from the generating function GF (0) And the generating function GF (1) , the generating function GF (0) is configured to create a composite image including anatomical structures classified with a classification value of 0, the generating function GF (1) is configured to create a synthetic image comprising an anatomical structure classified with a classification value of 1. In an alternative, if the actual classification value CV.A, the first classification value CV.1 and the second classification value CV.2 are from a discrete and sortable set, the second generating function GF.2 is selected from the generating function GF (+) And the generating function GF (-) , the generating function GF (+) is configured to create a synthetic image including anatomical structures classified with a classification value one level greater than the classification value of the anatomical structures in the input image, the generating function GF (-) The method is configured to create a composite image comprising anatomical structures classified with a classification value one level lower than the classification value of the anatomical structures in the input image. In particular, in both cases, the first generating function GF.1 and the second generating function GF.2 are chosen to be different.

[0271] Alternatively, the second classification value CV.2 may be used as an additional parameter of the second generating function GF.2.

[0272] The third embodiment further comprises the step of determining a DET-IMG.M modified medical image IMG.M based on the acquired medical image IMG.A, the first synthesized image IMG.S1 and the second synthesized image.

[0273] In this embodiment, each voxel of the modified medical image IMG.M includes a first intensity value, a second intensity value, and a third intensity value. The first intensity value corresponds to the intensity value of the corresponding voxel of the acquired medical image IMG.A. The second intensity value corresponds to the difference in voxels between the modified medical image IMG.M and the first synthesized image IMG.S1. The third intensity value corresponds to the difference in voxels between the modified medical image IMG.M and the second synthesized image IMG.S2. Figure 1 Annotation, use x m Represents the modified medical image IMG.M, which is composed of x m =(x,s1–x,s2–x) or x m =(x,|s1–x|,|s2–x|) is given in voxels.

[0274] Furthermore, providing PROV-VIS visualization includes providing a modified medical image IMG.M. In this embodiment, the modified medical image IMG.M includes three color channels, a first color channel associated with a first intensity and being a grayscale channel, a second color channel associated with a ratio of transparent to red, and a third color channel associated with a ratio of transparent to blue.

[0275] Figure 9 Schematically shows Figure 6 The first embodiment and Figure 7 Potential results of the second embodiment.

[0276] Figure 9 The acquired medical image IMG.A is shown as a two-dimensional image containing an anatomical structure as a lesion. Figure 9 An example of a first synthesized image IMG.S1 corresponding to a more malignant version of the lesion is shown, since the boundaries of the lesion are less regular than the boundaries of the lesion in the acquired medical image IMG.A.

[0277] The acquired medical image IMG.A and the first synthesized image IMG.S1 are displayed aligned in space and form Figure 6 Potential visualization results of the first embodiment. By spatially aligning, direct comparison is possible.

[0278] also, Figure 9 A difference image IMG.D corresponding to the difference between the acquired medical image IMG.A and the first synthesized image IMG.S1 is shown. The difference image IMG.D has non-vanishing values ​​in voxels where the acquired medical image IMG.A and the first synthesized image IMG.S1 do not match, where the non-zero values ​​are located at the boundaries of the lesion.

[0279] in addition, Figure 9 A modified image IMG.M is displayed. The modified image is based on the acquired medical image IMG.A and displays the values ​​of the difference image IMG.D in another channel (e.g., in another color). Alternatively, the modified image may include markings such as arrows pointing to voxel regions where the difference image IMG.D has high values ​​(corresponding to regions where the difference between the acquired medical image IMG.A and the first synthesized image IMG.S1 is large).

[0280] Figure 10 The method according to the invention is shown, in particular, according to Figures 6 to 8 The data shown corresponds to the use of the described method on real medical data.

[0281] The acquired medical image IMG.A is a two-dimensional X-ray image that includes an anatomical structure as a lesion. The lesion has an actual classification value CV.A of 0.73, which can be calculated by applying the classification function CF to the acquired medical image IMG.A. Here, the classification value is a real number in the interval [0, 1] that is associated with the malignancy of the lesion.

[0282] Furthermore, a first synthetic image IMG.S1 and a second synthetic image IMG.S2 are shown, corresponding to a more benign lesion and a more malignant lesion. The first synthetic image IMG.S1 and the second synthetic image IMG.S2 have been generated using a first generating function GF.1 and a second generating function GF.2. Applying the classification function CF to the first synthetic image IMG.S1 yields a classification value of 0.37, while applying the classification function CF to the second synthetic image IMG.S2 yields a classification value of 0.93. It should be noted that these calculated classifications are not necessarily equivalent to the first and second classification values ​​CV.1 and CV.2, as imperfections in the first and second generating functions GF.1 and GF.2 may cause deviations between the calculated classification values ​​and the desired first and second classification values ​​CV.1 and CV.2.

[0283] Figure 11 The method according to the invention is shown for other example lesions, in particular according to Figures 6 to 8More results for one of the embodiments of FIG. There are four different rows, each row relating to a different lesion. Furthermore, there are four columns. The first column includes the corresponding acquired medical image IMG.A, the second column includes the corresponding first synthesized image IMG.S1, the third column includes the corresponding second synthesized image IMG.S2, and the fourth column includes the corresponding modified medical image IMG.M.

[0284] Figure 12 The different generating functions GF.1, GF.2 used in the context of the present invention are shown for six different lesions (corresponding to Figure 12 Comparison of the six rows of images in .

[0285] The first column includes the corresponding acquired medical image IMG.A. The second column includes the modified medical image IMG.M based on the cycle-consistent generative adversarial network. The third column includes the modified medical image based on Shapley Additive Explanations. The fourth column includes the modified medical image based on Deep Taylor. The last column includes the modified medical image based on Layerwise Relevance Propagation.

[0286] Figure 13 The results of a study comparing the proposed method based on different types of generating functions, GF.1 and GF.2, are presented. This study was conducted as a double-blind, randomized, in-house study involving eight participants with several years of experience in biomedical image processing. Based on the acquired medical image IMG.A, a questionnaire was used to assess the intuitive impression of the modified medical image IMG.M, its significance regarding typical signs of malignancy in lung lesions, and its overall image quality.

[0287] The top row shows the overall frequency distribution of the evaluations given by the participants, and the top row shows a comparison between cases with benign nodules (left) and malignant pulmonary nodules (right). The left column relates to the rating of intuitive impressions, the middle column relates to the significance of typical criteria for rating malignancy and benignity, and the right column relates to overall image quality.

[0288] The first distribution DB.1 is based on the generating functions GF.1 and GF.2 based on cycle-consistent generative adversarial networks, the second distribution DB.2 is based on Deep Shapley Additive Explanations, the third distribution DB.3 is based on Deep Taylor, and the fourth distribution DB.4 is based on hierarchical correlation propagation.

[0289] Figure 14A flow chart of a first embodiment of a method for providing a first generating function GF.1 is shown.

[0290] The method includes receiving a TREC-T1 first training image IMG.T1, which includes a first training anatomical structure, which is classified by a first training classification value CV.T1. In the present embodiment, the first training image IMG.T1 is a two-dimensional medical image based on X-rays. The first training image IMG.T1 includes lesions, in particular lung nodules, as anatomical structures. The first training image IMG.T1 can be received by an interface TSYS.IF of a training system TSYS. The first training image IMG.T1 includes 512x512 voxels, each of which includes at least one intensity value. The first training classification value CV.T1 is a real number in the interval [0, 1] that classifies the malignancy of the lung nodules in the first training image IMG.T1.

[0291] A further step in the illustrated embodiment is receiving or determining a second training classification value CV.T2 of TREC-TDET-CV.1 that is different from the first training classification value CV.T1, wherein the second training classification value CV.T2 is a potential classification value for a first anatomical structure within the first training medical image IMG.T1. In this embodiment, the second training classification value CV.T2 is also a real number in the interval [0, 1] that classifies the malignancy of a pulmonary nodule. In particular, similar to how the first classification value CV.1 was determined based on the actual classification value CV.A, the second training classification value CV.T2 can be determined based on the first training classification value CV.T1.

[0292] The method further comprises applying a first generating function GF.1 to the TAPPL-GF.1-T1 first training image IMG.T1, thereby generating a modified first training image IMG.T1'. Here, the first generating function GF.1 is based on the second training classification value CV.T2. Figures 3 to 5 Introducing notation, the modified first training image IMG.T1' can be determined as t1 = GF.1(t1, c2). The modified first training image IMG.T1' is a composite image and has the same dimensions and size as the first training image IMG.T1. In this embodiment, the modified first training image IMG.T1' is a two-dimensional composite image comprising 512x512 voxels.

[0293] Another step in the illustrated embodiment is receiving a TREC-S1 first input image IMG.S1 comprising a first input anatomical structure to be classified by a second training classification value CV.T2. Generally, the first input image IMG.S1 has the same dimensions and size as the first training image IMG.T1. In this embodiment, the first input image IMG.S1 is a two-dimensional X-ray-based medical image comprising 512x512 voxels.

[0294] The illustrated method includes the additional steps of applying the first decision function DF.1 to the first input image IMG.S1 of TAPPL-DF.1-TS to generate a first decision value, and applying the first decision function DF.1 to the first training image IMG.T1' modified by TAPPL-DF.1-T1' to generate a second decision value. Specifically, the decision value corresponds to a real number in the interval [0, 1] that is related to the probability that the input image to the first decision function is a real medical image rather than a synthetic image. In this embodiment, the first decision function DF.1 is also based on the second training classification value CV.T2, but a first decision function DF.1 not based on a classification value could also be used.

[0295] The next step of the shown embodiment is to adjust parameters of the TADPT first generator function GF.1 and / or the first decision function DF.1 based on a generative adversarial loss function, wherein the generative adversarial loss function is based on the first and second decision value.

[0296] In this example, the following loss function is used:

[0297] L (DF1) ~-logDF1(s1,c2)-log[1-DF1(GF1(t1,c2),c2)]

[0298] L (GF1) ~log[1-DF1(GF1(t1,c2),c2)]

[0299] In this embodiment, both the first generating function GF.1 and the first decision function DF.1 are artificial neural networks, where the edge weights of the artificial neural networks correspond to adjustable parameters. The weights are adjusted by minimizing a loss function based on gradient descent using a backpropagation algorithm. In particular, the first generating function GF.1 and the first decision function DF.1 may include convolutional layers.

[0300] The last step of the method shown is providing TPROV the first generation function GF.1 and / or the first decision function DF.1. Providing the first generation function GF.1 and / or the first decision function DF.1 may include storing, displaying and / or transmitting the first generation function and / or the first decision function DF.1.

[0301] Figure 15 A flow chart of a second embodiment of a method for providing a first generating function GF.1 is shown. The second embodiment comprises Figure 14 All steps of the first embodiment shown in FIG, all of which may include Figure 14 Describes advantageous options or alternative features.

[0302] The second embodiment further includes the step of applying a second generating function GF.2 to the first training image IMG.T1′ modified by APPL-GF.2-T1′, thereby generating a comparative first training image IMG.T1″, wherein the second generating function GF.2 is based on the first training classification value CV.T1. The comparative first training image IMG.T1″ is a composite image having the same dimensions and the same size as the first training image IMG.T1. In this embodiment, the comparative first training image IMG.T1″ is a two-dimensional composite image comprising 512x512 voxels.

[0303] In this embodiment, the step of adjusting the TADPT involves parameters of a first generating function GF.1 and / or a second generating function GF.2 based on a cycle consistency loss function. Here, the cycle consistency loss function L.CC is based on comparing the first training image IMG.T1″ with the first training image IMG.T1. Additionally, the step of adjusting the TADPT involves parameters of the first generating function GF.1 and / or the first decision function DF.1 based on a generative adversarial loss function, wherein the generative adversarial loss function is based on a first decision value and a second decision value.

[0304] In this example, the following loss function is used:

[0305] L (DF1) ~-logDF1(s1,c2)-log[1-DF1(GF1(t1,c2),c2)]

[0306] L (GF1) ~log[1-DF1(GF1(t1,c2),c2)]+║t1–GF2(GF1(t1,c2),c1)║2

[0307] L (GF2) ~║t1–GF2(GF1(t1,c2),c1)║2

[0308] In this embodiment, the first generating function GF.1, the first decision function DF.1, and the second generating function GF.2 are artificial neural networks, where the edge weights of the artificial neural networks correspond to adjustable parameters. The weights are adjusted by minimizing a loss function based on gradient descent using a backpropagation algorithm. In particular, the first generating function GF.1, the first decision function DF.1, and the second generating function GF.2 may include convolutional layers.

[0309] Figure 16 A flow chart of a third embodiment of a method for providing a first generating function GF.1 is shown. The third embodiment comprises Figure 14 All steps of the first embodiment shown in FIG, all of which may include Figure 14 Describes advantageous options or alternative features.

[0310] A third embodiment includes receiving a TREC-T2 second training image IMG.T2, which includes a second anatomical structure classified by a second training classification value CV.T2. This step is similar to the step of receiving a TREC-T1 first training image IMG.T1, and the second training image IMG.T2 may include all the features and advantageous embodiments of the first training image IMG.T1.

[0311] A further step is to apply a second generating function GF.2 to the TAPPL-GF.2-T2 second training image IMG.T2, thereby generating a modified second training image IMG.T2', wherein the second generating function GF.2 is based on the first training classification values ​​CV.T1. This step is similar to the step of applying the first generating function GF.1 to the TAPPL-GF.2-T1 first training image IMG.T1.

[0312] A further step of the third embodiment is receiving a TREC-S2 second input image IMG.S2, which includes a second input anatomical structure classified by the first trained classification value CV.T1. This step is similar to the step of receiving the TREC-S1 first input image IMG.S1, and the second input image IMG.S2 may include all the features and advantageous embodiments of the first input image IMG.S1.

[0313] The additional steps are applying the second decision function DF.2 to the TAPPL-DF.2-T2 second input image IMG.S2 to generate a third decider value, and applying the second decision function DF.2 to the TAPPL-DF.2-T2' modified second training image IMG.T2' to generate a fourth decider value.

[0314] In this embodiment, the step of adjusting the TADPT involves parameters of a second generating function GF.2 and / or a second decision function DF.2 based on a generative adversarial loss function, wherein the generative adversarial loss function is based on a third arbiter value and a fourth arbiter value.

[0315] In this example, the following loss function is used:

[0316] L (DF1) ~-log DF1(s1,c2)-log[1-DF1(GF1(t1,c2),c2)]

[0317] L (GF1) ~log[1-DF1(GF1(t1,c2),c2)]

[0318] L (DF2) ~-log DF2(s2,c1)-log[1–DF2(GF2(t2,c1),c1)]

[0319] L (GF2) ~log[1–DF2(GF2(t2,c1),c1)]

[0320] In this embodiment, the generating functions GF.1 and GF.2 and the decision functions DF.1 and DF.2 are artificial neural networks, where the edge weights of the artificial neural networks correspond to adjustable parameters. The weights are adjusted by minimizing a loss function based on gradient descent using a backpropagation algorithm. In particular, the generating functions GF.1 and GF.2 and the decision functions DF.1 and DF.2 may include convolutional layers.

[0321] Figure 17 A flow chart of a fourth embodiment of a method for providing a first generating function GF.1 is shown. The fourth embodiment comprises Figure 14 All steps of the first embodiment shown in Figure 15 and Figure 16 Additional steps of the second and third embodiments shown. All of these steps may include Figures 14 to 16 Describes advantageous options or alternative features.

[0322] The fourth embodiment further comprises applying a first generating function GF.1 to the second training image IMG.T2′ modified by TAPPL-GF.1-T2′, thereby generating a comparative second training image IMG.T2″, wherein the first generating function GF.1 is based on the second training classification value CV.T2. The step of applying the first generating function GF.1 to the second training image IMG.T2′ modified by TAPPL-GF.1-T2′ is similar to the step of applying the second generating function GF.2 to the first training image IMG.T1′ modified by TAPPL-GF.2-T1′, and comparing the second training image IMG.T2″ may include all features and advantageous embodiments of comparing the first training image IMG.T1″.

[0323] In this embodiment, the step of adjusting the TADPT involves parameters of the first generating function GF.1 and / or the second generating function GF.2 based on a cycle consistency loss function, wherein the cycle consistency loss function is based on comparing the differences between the second training image IMG.T2″ and the second training image IMG.T2.

[0324] In this example, the following loss function is used:

[0325] L (DF1) ~-log DF1(s1,c2)-log[1-DF1(GF1(t1,c2),c2)]

[0326] L (DF2) ~-log DF2(s2,c1)-log[1–DF2(GF2(t2,c1),c1)]

[0327] L (GF1) ~log[1-DF1(GF1(t1,c2),c2)]+║t1–GF2(GF1(t1,c2),c1)║2+║t2–GF1(GF2(t2,c1),c2)║2

[0328] L (GF2) ~log[1–DF2(GF2(t2,c1),c1)]+║t1–GF2(GF1(t1,c2),c1)║2+║t2–GF1(GF2(t2,c1),c2)║2

[0329] Alternatively, only one norm term may be used in the loss functions of the first generating function GF.1 and the second generating function GF.2. In this embodiment, the generating functions GF.1 and GF.2 and the decision functions DF.1 and DF.2 are artificial neural networks, where the edge weights of the artificial neural networks correspond to adjustable parameters. The weights are adjusted by minimizing the loss function based on gradient descent using a backpropagation algorithm. In particular, the generating functions GF.1 and GF.2 and the decision functions DF.1 and DF.2 may include convolutional layers.

[0330] Figure 18 The interpretation system ESYS, the medical imaging system MISYS, and the training system TSYS are shown. The interpretation system ESYS includes an input interface ESYS.IIF, an output interface ESYS.OIF, a computing unit ESYS.CU, and a storage unit ESYS.MU. The medical imaging system MISYS includes a training system TSYS. The training system TSYS includes an input interface TSYS.IIF, an output interface TSYS.OIF, a computing unit TSYS.CU, and a storage unit TSYS.MU.

[0331] The interpretation system ESYS and / or the training system TSYS can be a (personal) computer, a workstation, a virtual machine running on host hardware, a microcontroller, or an integrated circuit. In particular, the interpretation system ESYS and / or the training system TSYS can be a mobile device, such as a smartphone or a tablet. Alternatively, the interpretation system ESYS and / or the training system TSYS can be a real or virtual computer group (the technical term for a real computer group is a "cluster" and the technical term for a virtual computer group is a "cloud").

[0332] The interpretation system ESYS and / or the training system TSYS can be connected to a network NETW. The network can be implemented as a LAN (acronym for "Local Area Network"), in particular a WiFi network, or any other local connection, for example via Bluetooth or USB (acronym for "Universal Serial Bus"). Alternatively, the network can also be implemented as a VPN (acronym for "Virtual Private Network").

[0333] The input interfaces ESYS.IIF and TSYS.IIF and the output interfaces ESYS.OIF and TSYS.OIF can be implemented as hardware interfaces or software interfaces (e.g., PCIBus, USB, or Firewire). In particular, the input interfaces ESYS.IIF and TSYS.IIF and / or the output interfaces ESYS.OIF and TSYS.OIF can be a combination of several other interfaces. In particular, the input interfaces ESYS.IIF and TSYS.IIF and / or the output interfaces ESYS.OIF and TSYS.OIF can include one or more interfaces as subcomponents. The input interfaces ESYS.IIF and TSYS.IIF and the output interfaces ESYS.OIF and TSYS.OIF can be identical. In particular, the output interface ESYS.OIF of the interpretation system ESYS can be a display.

[0334] In general, a computing unit ESYS.CU, TSYS.CU may include hardware and software components, such as a microprocessor, a CPU (the acronym for "Central Processing Unit"), a GPU (the acronym for "Graphics Processing Unit"), a Field Programmable Gate Array (the acronym is "FPGA"), or an ASIC (the acronym for "Application-Specific Integrated Circuit"). The computing unit ESYS.CU, TSYS.CU may be configured as multithreaded, meaning that the computing unit ESYS.CU, TSYS.CU may simultaneously host different computing processes, executing active and passive computing processes in parallel or switching between them. In particular, the computing unit ESYS.CU, TSYS.CU may be a combination of several other computing units; in particular, the computing unit ESYS.CU, TSYS.CU may include one or more computing units as subcomponents. The storage unit ESYS.MU, TSYS.MU may be, for example, a non-volatile main memory (e.g., random access memory) or a persistent mass storage device (e.g., a hard drive, USB stick, SD card, solid-state drive).

[0335] The present invention, its aspects and embodiments may also be described by the following clauses, which all form part of this disclosure:

[0336] Clause 1: A computer-implemented method for providing a classification explanation, comprising:

[0337] - receiving REC-IMG.A corresponding to an acquired medical image IMG.A of the examination volume,

[0338] - receiving or determining REC-DET-CV.A an actual classification value CV.A, in particular by applying a classification function CF to the acquired medical image IMG.A to generate said actual classification value CV.A,

[0339] The actual classification value CV.A classifies the anatomical structure in the acquired medical image IMG.A.

[0340] - receiving or determining REC-DET-CV.1 a first classification value CV.1 which differs from said actual classification value CV.A,

[0341] The first classification value CV.1 is a potential classification value of the anatomical structure in the acquired medical image IMG.A.

[0342] - applying a first generation function GF.1 to the medical image IMG.A acquired by APPL-GF.1, thereby generating a first composite image IMG.S1,

[0343] The first generating function GF.1 is based on the first classification value CV.1,

[0344] wherein the first generating function GF.1 is configured to modify the anatomical structure to exhibit the first classification value CV.1,

[0345] - providing PROV-VIS based on a visualization of the acquired medical image IMG.A and the first composite image IMG.S1.

[0346] Clause 2: A computer-implemented method for providing a classification explanation, comprising:

[0347] - receiving REC-IMG.A corresponding to an acquired medical image IMG.A of the examination volume,

[0348] - applying the classification function CF to the acquired medical image IMG.A, thereby generating an actual classification value CV.A classifying the anatomical structure within the acquired medical image IMG.A,

[0349] - receiving or determining REC-DET-CV.1 a first classification value CV.1 which differs from said actual classification value CV.A,

[0350] The first classification value CV.1 is a potential classification value of the anatomical structure in the acquired medical image IMG.A.

[0351] - applying a first generation function GF.1 to the medical image IMG.A acquired by APPL-GF.1, thereby generating a first composite image IMG.S1,

[0352] The first generating function GF.1 is based on the first classification value CV.1,

[0353] wherein the first generating function GF.1 is configured to modify the anatomical structure to exhibit the first classification value CV.1,

[0354] - providing PROV-VIS based on a visualization of the acquired medical image IMG.A and the first composite image IMG.S1.

[0355] Clause 3: A computer-implemented method for providing a classification explanation, comprising:

[0356] - receiving REC-IMG.A corresponding to an acquired medical image IMG.A of the examination volume, the acquired medical image IMG.A comprising anatomical structures, in particular wherein said anatomical structures are classified by actual classification values ​​CV.A,

[0357] - receiving a first classification value CV.1, wherein the first classification value CV.1 is a potential classification value of an anatomical structure within the acquired medical image IMG.A,

[0358] - applying a first generation function GF.1 to the medical image IMG.A acquired by APPL-GF.1, thereby generating a first composite image IMG.S1,

[0359] The first generating function GF.1 is based on the first classification value CV.1,

[0360] wherein the first generating function GF.1 is configured to modify the anatomical structure to exhibit the first classification value CV.1,

[0361] - providing PROV-VIS based on a visualization of the acquired medical image IMG.A and the first composite image IMG.S1.

[0362] Clause 4: A method according to one of the preceding clauses, wherein the first generating function GF.1 is based on the first classification value CV.1 by making the first classification value CV.1 an input value of the first generating function GF.1 and / or by selecting the first generating function GF.1 from a plurality of generating functions GF.1, GF.2 based on the first classification value CV.1.

[0363] Clause 5: Method according to one of the preceding clauses, wherein the first generation function GF.1 is configured to generate a synthetic image comprising a synthetic structure, the synthetic structure being similar to the anatomical structure but exhibiting the first classification value CV.1.

[0364] Clause 6: The method according to one of the preceding clauses, further comprising:

[0365] - determining DET-IMG.M a modified medical image IMG.M based on the acquired medical image IMG.A and the first synthesized image IMG.S1,

[0366] Wherein, providing PROV-VIS said visualization comprises providing said modified medical image IMG.M.

[0367] Clause 7: The method of clause 6, wherein the modified medical image IMG.M is based on a difference between the first synthesized image IMG.S1 and the acquired medical image IMG.A.

[0368] Clause 8: The method of clause 7, wherein each voxel of the modified medical image IMG.M comprises a first intensity value and a second intensity value,

[0369] wherein the first intensity value corresponds to the intensity value of the corresponding voxel of the acquired medical image IMG.A,

[0370] And wherein the second intensity value corresponds to the intensity value of the corresponding voxel of the difference between the first synthetic image IMG.S1 and the acquired medical image IMG.A.

[0371] Clause 9: The method according to one of the preceding clauses, wherein the actual classification value CV.A and the first classification value CV.1 are comparable,

[0372] The first classification value CV.1 is smaller than the actual classification value CV.A, or the first classification value CV.1 is larger than the actual classification value CV.A.

[0373] Clause 10: The method according to one of the preceding clauses, wherein the actual classification value CV.A and the first classification value CV.1 are elements of a plurality of possible classification values ​​of the structure, the plurality of possible classification values ​​being a discrete and comparable set, and

[0374] The first classification value CV.1 is the smallest classification value among multiple possible classification values ​​greater than the actual classification value CV.A, or the first classification value CV.1 is the largest classification value among multiple possible classification values ​​less than the actual classification value CV.A.

[0375] Clause 11: The method according to one of the preceding clauses, further comprising:

[0376] - receiving or determining REC-DET-CV.2 a second classification value CV.2 which differs from said actual classification value CV.A and from said first classification value CV.1,

[0377] The second classification value CV.2 is a potential classification value of the anatomical structure in the acquired medical image IMG.A.

[0378] - applying the second generation function GF.2 to the medical image IMG.A acquired by APPL-GF.2, thereby generating a second composite image IMG.S2,

[0379] The second generating function GF.2 is based on the second classification value CV.2,

[0380] wherein the second generating function GF.2 is configured to modify the anatomical structure to exhibit the second classification value CV.2,

[0381] Therein, the visualization is also based on the second composite image IMG.S2.

[0382] Clause 12: A method according to clause 11, wherein the second generating function GF.2 is based on the second classification value CV.2 by making the second classification value CV.2 an input value of the second generating function GF.2 and / or by selecting the second generating function GF.2 from a plurality of generating functions GF.1, GF.2 based on the second classification value CV.2.

[0383] Clause 13: The method according to clause 11 or clause 12, further comprising:

[0384] - determining a DET-IMG.M modified medical image IMG.M based on the acquired medical image IMG.A, the first synthesized image IMG.S1 and the second synthesized image IMG.S2,

[0385] Therein, providing the visualization comprises providing the modified medical image IMG.M.

[0386] Clause 14: The method of clause 13, wherein the modified medical image is based on differences between the first composite image and the medical image and differences between the second composite image and the medical image.

[0387] Clause 15: The method of clause 14, wherein each voxel of the modified medical image IMG.M comprises a first intensity value, a second intensity value, and a third intensity value,

[0388] wherein the first intensity value corresponds to the intensity value of the corresponding voxel of the acquired medical image IMG.A,

[0389] wherein the second intensity value corresponds to the intensity value of the corresponding voxel of the difference between the first synthetic image IMG.S1 and the acquired medical image IMG.A, and

[0390] Therein, the third intensity value corresponds to the intensity value of the corresponding voxel of the difference between the second synthesized image IMG.S2 and the acquired medical image IMG.A.

[0391] Clause 16: The method according to one of clauses 11 to 15, wherein the actual classification value CV.A, the first classification value CV.1 and the second classification value CV.2 are comparable,

[0392] wherein the first classification value CV.1 is less than the actual classification value CV.A, and

[0393] The second classification value CV.2 is greater than the actual classification value CV.A.

[0394] Clause 17: The method of clause 16, wherein the actual classification value CV.A, the first classification value CV.1, and the second classification value CV.2 are elements of a plurality of possible classification values ​​of the structure, the plurality of possible classification values ​​being a discrete and comparable set,

[0395] wherein the first classification value CV.1 is the largest classification value among a plurality of possible classification values ​​that are smaller than the actual classification value CV.A, and

[0396] The second classification value CV.2 is the smallest classification value among multiple possible classification values ​​that are greater than the actual classification value CV.A.

[0397] Clause 18: The method according to one of the preceding clauses, wherein the anatomical structure in the acquired medical image IMG.A is a lesion,

[0398] The actual classification value CV.A, the first classification value CV.1 and / or the second classification value CV.2 correspond to the benignity and / or malignancy of the lesion.

[0399] Clause 19: The method according to clause 18, wherein, in the case where the actual classification value CV.A corresponds to a benign lesion, the first classification value CV.1 and / or the second classification value CV.2 corresponds to a malignant lesion, and

[0400] In the case where the actual classification value CV.A corresponds to a malignant lesion, the first classification value CV.1 and / or the second classification value CV.2 corresponds to a benign lesion.

[0401] Clause 20: A method according to Clause 18, wherein the actual classification value CV.A, the first classification value CV.1 and / or the second classification value CV.2 correspond to the degree of benignity and / or malignancy of the lesion, wherein the first classification value CV.1 corresponds to a degree one level lower than the actual classification value CV.A, and / or wherein the second classification value CV.2 corresponds to a degree one level higher than the actual classification value CV.A.

[0402] Clause 21: Method according to one of the preceding clauses, wherein the first generating function GF.1 and / or the second generating function GF.2 is based on a generative adversarial algorithm, in particular a cycle-consistent generative adversarial algorithm.

[0403] Clause 22: The method of clause 21, wherein the first generating function GF.1 is the generating part of a first generative adversarial algorithm, and / or wherein the second generating function GF.2 is the generating part of a second generative adversarial algorithm.

[0404] Clause 23: A method according to clause 21, wherein the first generation function GF.1 is the generation part of a first generative adversarial algorithm, wherein the second generation function GF.2 is the generation part of a second generative adversarial algorithm, and wherein the first generative adversarial algorithm and the second generative adversarial algorithm form a cycle-consistent generative adversarial algorithm.

[0405] Clause 24: Method according to one of clauses 21 to 23, wherein the first generating function GF.1 and / or the second generating function GF.2 is an artificial neural network.

[0406] Clause 25: The method according to one of clauses 21 to 24, wherein the classification function CF is based on the generative adversarial algorithm, in particular on the cycle-consistent generative adversarial algorithm.

[0407] Clause 26: The method of clause 25, wherein the classification function CF is based on a decision part of a generative adversarial algorithm or a cycle-consistent generative adversarial algorithm.

[0408] Clause 27: The method according to one of the preceding clauses, wherein the classification function is an artificial neural network, in particular, wherein the classification function is a deep artificial neural network, in particular, wherein the classification function is a convolutional deep neural network.

[0409] Clause 28: An interpretation system ESYS for providing classification interpretations, said system comprising an input interface ESYS.IIF, an output interface ESYS.OIF and a calculation unit ESYS.CU,

[0410] wherein the input interface ESYS.IIF is configured to receive REC-IMG.A corresponding to an acquired medical image IMG.A of the examination volume,

[0411] wherein the input interface ESYS.IIF and / or the calculation unit ESYS.CU are configured to receive or determine a REC-DET-CV.A actual classification value CV.A, in particular by applying a classification function CF to the acquired medical image IMG.A to generate the actual classification value CV.A,

[0412] The actual classification value CV.A classifies the anatomical structure in the acquired medical image IMG.A.

[0413] wherein the input interface ESYS.IIF or the calculation unit ESYS.CU is configured to receive or determine a first classification value CV.1 REC-DET-CV.1 different from the actual classification value CV.A,

[0414] The first classification value CV.1 is a potential classification value of the anatomical structure in the acquired medical image IMG.A.

[0415] wherein the calculation unit ESYS.CU is configured to apply a first generation function GF.1 to the medical image IMG.A acquired by APPL-GF.1, thereby generating a first synthetic image IMG.S1,

[0416] The first generating function GF.1 is based on the first classification value CV.1,

[0417] wherein the first generating function GF.1 is configured to modify the anatomical structure to exhibit the first classification value CV.1,

[0418] - wherein the output interface ESYS.OIF is configured for providing a PROV-VIS visualization based on the acquired medical image IMG.A and the first composite image IMG.S1.

[0419] Clause 29: An interpretation system ESYS for providing classification interpretations, said system comprising an input interface ESYS.IIF, an output interface ESYS.OIF and a calculation unit ESYS.CU,

[0420] wherein the input interface ESYS.IIF is configured for receiving REC-IMG.A corresponding to an acquired medical image IMG.A of the examination volume,

[0421] wherein the calculation unit ESYS.CU is configured to apply a classification function CF to the acquired medical image IMG.A, thereby generating an actual classification value CV.A for classifying an anatomical structure within the acquired medical image IMG.A,

[0422] wherein the input interface ESYS.IIF and / or the calculation unit ESYS.CU are configured to receive or determine a first classification value CV.1 REC-DET-CV.1 which differs from the actual classification value CV.A,

[0423] The first classification value CV.1 is a potential classification value of the anatomical structure in the acquired medical image IMG.A.

[0424] wherein the calculation unit ESYS.CU is configured to apply a first generation function GF.1 to the medical image IMG.A acquired by APPL-GF.1, thereby generating a first synthetic image IMG.S1,

[0425] The first generating function GF.1 is based on the first classification value CV.1,

[0426] wherein the first generating function GF.1 is configured to modify the anatomical structure to exhibit the first classification value CV.1,

[0427] - wherein the output interface ESYS.OIF is configured for providing a PROV-VIS visualization based on the acquired medical image IMG.A and the first composite image IMG.S1.

[0428] Clause 30: An interpretation system ESYS for providing classification interpretations, said system comprising an input interface ESYS.IIF, an output interface ESYS.OIF and a calculation unit ESYS.CU,

[0429] wherein the input interface ESYS.IIF is configured for receiving REC-IMG.A corresponding to an acquired medical image IMG.A of an examination volume, the acquired medical image IMG.A comprising anatomical structures, in particular wherein the anatomical structures are classified by actual classification values ​​CV.A,

[0430] wherein the input interface ESYS.IIF is configured to receive a first classification value CV.1, wherein the first classification value CV.1 is a potential classification value of an anatomical structure within the acquired medical image IMG.A,

[0431] wherein the calculation unit ESYS.CU is configured to apply a first generation function GF.1 to the medical image IMG.A acquired by APPL-GF.1, thereby generating a first synthetic image IMG.S1,

[0432] The first generating function GF.1 is based on the first classification value CV.1,

[0433] wherein the first generating function GF.1 is configured to modify the anatomical structure to exhibit the first classification value CV.1,

[0434] - wherein the output interface ESYS.OIF is configured for providing a PROV-VIS visualization based on the acquired medical image IMG.A and the first composite image IMG.S1.

[0435] Clause 31: The interpretation system ESYS according to one of clauses 28 to 30, further configured to perform the method according to one of clauses 1 to 27.

[0436] Clause 32: A medical imaging system comprising an interpretation system ESYS according to one of clauses 28 to 31.

[0437] Clause 33: A computer-implemented method for providing a first generating function GF.1, comprising:

[0438] - receiving a TREC-T1 first training image IMG.T1, said first training image IMG.T1 comprising a first training anatomical structure, said first training anatomical structure being classified by a first training classification value CV.T1,

[0439] - receiving or determining a second training classification value CV.T2 TREC-TDET-CV.1 different from said first training classification value CV.T1,

[0440] The second training classification value CV.T2 is a potential classification value of the first anatomical structure in the first training medical image IMG.T1.

[0441] - applying a first generating function GF.1 to TAPPL-GF.1-T1 said first training image IMG.T1, thereby generating a modified first training image IMG.T1',

[0442] The first generating function GF.1 is based on the second training classification value CV.T2,

[0443] - receiving a TREC-S1 first input image IMG.S1, said first input image IMG.S1 comprising a first input anatomical structure, said first input anatomical structure being classified by said second training classification values ​​CV.T2,

[0444] - applying a first decision function DF.1 to TAPPL-DF.1-TS said first input image IMG.S1, thereby generating a first decision value,

[0445] - applying the first decision function DF.1 to the modified first training image IMG.T1' of TAPPL-DF.1-T1', thereby generating a second decision value,

[0446] - adjusting the parameters of the TADPT first generator function GF.1 and / or the first decision function DF.1 based on the generative adversarial loss function,

[0447] wherein the generative adversarial loss function is based on the first discriminator value and the second discriminator value,

[0448] - providing TPROV said first generation function GF.1 and / or said first decision function DF.1.

[0449] Clause 34: The method according to clause 33, further comprising:

[0450] - applying the second generating function GF.2 to the modified first training image IMG.T1' of APPL-GF.2-T1', thereby generating a comparison first training image IMG.T1'

[0451] The second generating function GF.2 is based on the first training classification value CV.T1.

[0452] - adjusting the parameters of the first generating function GF.1 and / or the second generating function GF.2 of TADPT based on the cycle consistency loss function,

[0453] The cycle consistency loss function is based on the difference between the first training image IMG.T1″ and the first training image IMG.T1.

[0454] Clause 35: A method according to clause 33 or clause 34, further comprising:

[0455] - receiving a TREC-T2 second training image IMG.T2, said second training image IMG.T2 comprising a second anatomical structure, said second anatomical structure being classified by said second training classification values ​​CV.T2,

[0456] - applying a second generating function GF.2 to TAPPL-GF.2-T2 said second training image IMG.T2, thereby generating a modified second training image IMG.T2',

[0457] The second generating function GF.2 is based on the first training classification value CV.T1.

[0458] - receiving a TREC-S2 second input image IMG.S2, said second input image IMG.S2 comprising a second input anatomical structure, said second input anatomical structure being classified by said first trained classification value CV.T1,

[0459] - applying a second decision function DF.2 to TAPPL-DF.2-T2 said second input image IMG.S2, thereby generating a third decider value,

[0460] - applying the second decision function DF.2 to the second training image IMG.T2' modified by TAPPL-DF.2-T2', thereby generating a fourth decider value,

[0461] - adjusting the parameters of the second generation function GF.2 and / or the second decision function DF.2 based on the generative adversarial loss function,

[0462] The generative adversarial loss function is based on the third discriminator value and the fourth discriminator value.

[0463] Clause 36: The method according to clause 35, further comprising:

[0464] - applying the first generating function GF.1 to the modified second training image IMG.T2' described in APPL-GF.2, thereby generating a comparative second training image IMG.T2'

[0465] The first generating function GF.1 is based on the second training classification value CV.T2,

[0466] - adjusting parameters of the first generating function GF.1 and / or the second generating function DF.2 based on a cycle consistency loss function,

[0467] The cycle consistency loss function is based on the difference between the second training image IMG.T2″ and the second training image IMG.T2.

[0468] Clause 37: The method according to one of clauses 33 to 36, wherein the parameters of the first generating function GF.1 and / or the second generating function GF.2 are further adjusted based on a classification loss function,

[0469] wherein the classification loss function is based on the difference between the second classification value CV.2 and the result of applying the classification function CF to the modified first medical image IMG.T1', and / or wherein the classification loss function is based on the difference between the first classification value CV.1 and the result of applying the classification function CF to the modified second medical image IMG.T2'.

[0470] Clause 38: A training system TSYS for providing a first generating function GF.1, said system comprising an interface TSYS.IF and a calculation unit TSYS.CU,

[0471] wherein the interface TSYS.IF is configured to receive a TREC-T1 first training image IMG.T1 comprising a first training anatomical structure classified by a first training classification value CV.T1,

[0472] wherein the interface TSYS.IF and / or the calculation unit TSYS.CU are configured to receive or determine a second training classification value CV.T2 TREC-TDET-CV.1 different from the first training classification value CV.T1,

[0473] The second training classification value CV.T2 is a potential classification value of the first anatomical structure in the first training medical image IMG.T1.

[0474] wherein the calculation unit TSYS.CU is configured to apply a first generating function GF.1 to the first training image IMG.T1 TAPPL-GF.1-T1, thereby generating a modified first training image IMG.T1′,

[0475] The first generating function GF.1 is based on the second training classification value CV.T2,

[0476] wherein said interface TSYS.IF is configured for receiving a TREC-S1 first input image IMG.S1 comprising a first input anatomical structure, said first input anatomical structure being classified by said second training classification values ​​CV.T2,

[0477] wherein the calculation unit TSYS.CU is configured to apply a first decision function DF.1 to the first input image IMG.S1 TAPPL-DF.1-TS, thereby generating a first decider value,

[0478] wherein the calculation unit TSYS.CU is configured to apply the first decision function DF.1 to the first training image IMG.T1 ′ modified by TAPPL-DF.1-T1 ′, thereby generating a second decider value,

[0479] wherein the computing unit TSYS.CU is configured to adjust parameters of the first generating function GF.1 and / or the first decision function DF.1 of TADPT based on a generative adversarial loss function,

[0480] wherein the generative adversarial loss function is based on the first discriminator value and the second discriminator value,

[0481] - wherein the interface TSYS.IF is configured to provide TPROV the first generation function GF.1 and / or the first decision function DF.1.

[0482] Clause 39: The method according to one of clauses 1 to 27, wherein the first generating function GF.1 has been provided by the method according to one of clauses 33 to 37.

[0483] Clause 40: A system comprising an interpretation system ESYS according to one of clauses 28 to 30 or a medical imaging system according to clause 31 and a training system TSYS according to clause 38, wherein a first generating function GF.1 provided by the training system TSYS is used in the interpretation system ESYS.

[0484] Clause 41: A computer program product comprising instructions for causing an interpretation system ESYS to perform a method according to one of clauses 1 to 27 or clause 39 when the program is executed by the interpretation system ESYS, and / or comprising instructions for causing a training system TSYS to perform a method according to one of clauses 33 to 37 when the program is executed by the training system TSYS.

[0485] Item 42: A computer-readable medium comprising instructions that, when executed by an interpretation system ESYS, cause the interpretation system ESYS to perform a method according to one of items 1 to 27 or item 39; and / or comprising instructions that, when executed by a training system TSYS, cause the training system TSYS to perform a method according to one of items 33 to 37.

[0486] Clause 43: A computer program product comprising instructions which, when the program is executed by an interpretation system ESYS, cause the interpretation system ESYS to perform the method according to one of clauses 1 to 27 or clause 39.

[0487] Clause 44: A computer program product comprising instructions which, when executed by a training system TSYS, cause the training system TSYS to perform the method according to one of clauses 33 to 37.

[0488] Clause 45: A computer-readable medium comprising instructions which, when executed by an interpretation system ESYS, cause the interpretation system ESYS to perform the method according to one of clauses 1 to 27 or clause 39.

[0489] Clause 46: A computer readable medium comprising instructions which, when executed by a training system TSYS, cause the training system TSYS to perform the method according to one of clauses 33 to 37.

[0490] Where not explicitly described, individual embodiments or their individual aspects and features may be combined or exchanged with one another without limiting or expanding the scope of the described invention, provided such combination or exchange makes sense and is within the meaning of the invention. Advantages described with respect to one embodiment of the invention are also advantages of the other embodiments of the invention, where applicable.

Claims

1. A computer-implemented method for providing a classification explanation, comprising: - receiving an acquired medical image (IMG.A) corresponding to an examination volume, - receiving or determining the actual classification value (CV.A), The actual classification value (CV.A) classifies the anatomical structure in the acquired medical image (IMG.A), - receiving or determining a first classification value (CV.1) different from said actual classification value (CV.A), wherein the first classification value (CV.1) is a potential classification value related to the classification of the anatomical structure within the acquired medical image (IMG.A), wherein the anatomical structure in the acquired medical image (IMG.A) is a lesion, wherein the actual classification value (CV.A) and the first classification value (CV.1) correspond to the benignity and / or malignancy of the lesion, - applying a first generating function (GF.1) to the acquired medical image (IMG.A) thereby generating a first synthetic image (IMG.S1), wherein the first generating function (GF.1) is configured to modify the anatomical structure to exhibit the first classification value (CV.1), - providing a visualization based on the acquired medical image (IMG.A) and said first composite image (IMG.S1), Therein, the first classification value (CV.1) is used as an input parameter of the first generating function (GF.1).

2. The method according to claim 1, wherein: The actual classification value (CV.A) is generated by applying a classification function (CF) to the acquired medical image (IMG.A).

3. The method according to claim 1, further comprising: - determining a modified medical image (IMG.M) based on the acquired medical image (IMG.A) and said first composite image (IMG.S1), Therein, providing the visualization comprises providing the modified medical image (IMG.M).

4. The method according to claim 3, wherein: The modified medical image (IMG.M) is based on the difference between the first synthesized image (IMG.S1) and the acquired medical image (IMG.A).

5. The method according to claim 4, wherein: Each voxel of the modified medical image (IMG.M) comprises a first intensity value and a second intensity value, wherein the first intensity value corresponds to the intensity value of the corresponding voxel of the acquired medical image (IMG.A), And wherein the second intensity value corresponds to the intensity value of the corresponding voxel of the difference between the first synthetic image (IMG.S1) and the acquired medical image (IMG.A).

6. The method according to claim 1, wherein: The actual classification value (CV.A) and the first classification value (CV.1) are elements of a plurality of classification values ​​of the anatomical structure, the plurality of classification values ​​being a discrete and comparable set, and Wherein, the first classification value (CV.1) is the smallest classification value among multiple classification values ​​greater than the actual classification value (CV.A), or wherein the first classification value (CV.1) is the largest classification value among multiple classification values ​​less than the actual classification value (CV.A).

7. The method according to one of claims 1 to 5, further comprising: - receiving or determining a second classification value (CV.2) different from said actual classification value (CV.A) and different from said first classification value (CV.1), wherein the second classification value (CV.2) is a potential classification value related to the classification of the anatomical structure within the acquired medical image (IMG.A), - applying a second generating function (GF.2) to the acquired medical image (IMG.A) thereby generating a second synthetic image (IMG.S2), wherein the second generating function (GF.2) is based on the second classification value (CV.2), wherein the second generating function (GF.2) is configured to modify the anatomical structure to exhibit the second classification value (CV.2), Therein, the visualization is also based on the second synthetic image (IMG.S2).

8. The method according to claim 7, wherein: the actual classification value (CV.A), the first classification value (CV.1) and the second classification value (CV.2) are comparable, wherein the first classification value (CV.1) is less than the actual classification value (CV.A), and Wherein, the second classification value (CV.2) is greater than the actual classification value (CV.A).

9. The method according to claim 8, wherein: The actual classification value (CV.A), the first classification value (CV.1) and the second classification value (CV.2) are elements of a plurality of classification values ​​of the anatomical structure, the plurality of classification values ​​being a discrete and comparable set, wherein the first classification value (CV.1) is the largest classification value among a plurality of classification values ​​that are smaller than the actual classification value (CV.A), and The second classification value (CV.2) is the smallest classification value among multiple classification values ​​that are greater than the actual classification value (CV.A).

10. The method according to claim 8 or 9, wherein: The second classification value (CV.2) corresponds to benignity and / or malignancy of the lesion.

11. The method according to claim 8 or 9, wherein: The first generating function (GF.1) and / or the second generating function (GF.2) are based on a generative adversarial algorithm.

12. The method according to claim 11, wherein: The first generating function (GF.1) and / or the second generating function (GF.2) are based on a cycle-consistent generating adversarial algorithm.

13. An interpretation system (ESYS) for providing classification interpretations, comprising an input interface (ESYS.IIF), an output interface (ESYS.OIF) and a calculation unit (ESYS.CU), -in, The input interface (ESYS.IIF) is configured for receiving an acquired medical image (IMG.A) corresponding to an examination volume, wherein the input interface (ESYS.IIF) and / or the calculation unit (ESYS.CU) are configured to receive or determine an actual classification value (CV.A), The actual classification value (CV.A) classifies the anatomical structure in the acquired medical image (IMG.A), - wherein the input interface (ESYS.IIF) and / or the calculation unit (ESYS.CU) are configured to receive or determine a first classification value (CV.1) different from the actual classification value (CV.A), wherein the first classification value (CV.1) is a potential classification value related to the classification of the anatomical structure within the acquired medical image (IMG.A), wherein the anatomical structure in the acquired medical image (IMG.A) is a lesion, wherein the actual classification value (CV.A) and the first classification value (CV.1) correspond to the benignity and / or malignancy of the lesion, - wherein the calculation unit (ESYS.CU) is configured to apply a first generation function (GF.1) to the acquired medical image (IMG.A) so as to generate a first synthetic image (IMG.S1), wherein the first generating function (GF.1) is configured to modify the anatomical structure to exhibit the first classification value (CV.1), - wherein the output interface (ESYS.OIF) is configured for providing a visualization based on the acquired medical image (IMG.A) and the first synthetic image (IMG.S1), Therein, the first classification value (CV.1) is used as an input parameter of the first generating function (GF.1).

14. An interpretation system (ESYS) according to claim 13, wherein: The actual classification value (CV.A) is generated by applying a classification function (CF) to the acquired medical image (IMG.A).

15. A medical imaging system comprising an interpretation system (ESYS) according to claim 13 or 14.

16. A computer-implemented method for providing a first generating function (GF.1), comprising: - receiving a first training image (IMG.T1), said first training image (IMG.T1) comprising a first training anatomical structure, said first training anatomical structure being classified by a first training classification value (CV.T1), - receiving or determining a second training classification value (CV.T2) different from said first training classification value (CV.T1), wherein the second training classification value (CV.T2) is a potential classification value related to the classification of the first training anatomical structure within the first training image (IMG.T1), wherein the first training anatomical structure in the first training image (IMG.T1) is a lesion, The first training classification value (CV.T1) and the second training classification value (CV.T2) correspond to the benignity and / or malignancy of the lesion, - applying a first generating function (GF.1) to said first training image (IMG.T1) so as to generate a modified first training image (IMG.T1'), - receiving a first input image (IMG.S1), said first input image (IMG.S1) comprising a first input anatomical structure, said first input anatomical structure being classified by said second training classification values ​​(CV.T2), - applying a first decision function (DF.1) to said first input image (IMG.S1) thereby generating a first decision value, - applying a first decision function (DF.1) to said modified first training image (IMG.T1') so as to generate a second decision value, - adjusting parameters of the first generation function (GF.1) and / or the first decision function (DF.1) based on a generative adversarial loss function, wherein the generative adversarial loss function is based on the first discriminator value and the second discriminator value, - providing said first generating function (GF.1) and / or said first decision function (DF.1), Therein, the second training classification value (CV.T2) is used as an input parameter of the first generating function (GF.1).

17. The method according to claim 16, further comprising: - applying a second generating function (GF.2) to said modified first training image (IMG.T1′) so as to generate a comparison first training image (IMG.T1″), wherein the second generating function (GF.2) is based on the first training classification value (CV.T1), - adjusting the parameters of the first generating function (GF.1) and / or the second generating function (DF.2) based on a cycle consistency loss function, Wherein, the cycle consistency loss function is based on the difference between the comparison of the first training image (IMG.T1″) and the first training image (IMG.T1).

18. A computer program product comprising instructions for causing an interpretation system (ESYS) to perform a method according to one of claims 1 to 12 when the program is executed by the interpretation system (ESYS) and / or for causing a training system (TSYS) to perform a method according to claim 16 or 17 when the program is executed by the training system (TSYS).

19. A computer-readable medium comprising instructions that, when the program is executed by an interpretation system (ESYS), cause the interpretation system (ESYS) to perform a method according to one of claims 1 to 12, and / or comprising instructions that, when the program is executed by a training system (TSYS), cause the training system to perform a method according to claim 16 or 17.

Images