Lesion classification and classification function providing method, classification and imaging system and medium

Abstract

The present invention relates to a lesion classification and classification function providing method, classification and imaging system and medium. In particular, the present invention relates to a computer-implemented method for classifying lesions, comprising: - receiving a first medical image of an examination volume, wherein the first medical image corresponds to a first examination time, - receiving a second medical image of the examination volume, wherein the second medical image corresponds to a second examination time different from the first examination time, - determining a first lesion region corresponding to a lesion within the first medical image, - determining a registration function based on a comparison of the first medical image and the second medical image, - determining a second lesion region within the second medical image based on the registration function and the first lesion region, - classifying the lesion within the first medical image based on the second lesion region. The present invention also relates to a computer-implemented method for providing a trained classification function, a classification system, a computer program product and a computer-readable medium.

Description

Technical Field

[0001] The present invention relates to classifying lesions based on longitudinal studies, and more particularly, to a computer-implemented method for classifying lesions, a computer-implemented method for providing a trained classification function, a classification system, a computer program product and a computer-readable medium. Background Art

[0002] Patients with certain risk factors should undergo regular screening tests. For example, lung cancer screening based on low-dose computed tomography (the acronym is "CT") is recommended for heavy smokers between the ages of 55 and 80.

[0003] Based on the results detected in each screening (typically lung nodules between 6mm and 30mm), follow-up screening tests (also denoted "longitudinal studies") are scheduled to confirm the malignancy prediction before the patient is sent for pathology evaluation. Due to the small size of lung nodules, manual detection of lung nodules by radiologists is a time-consuming task. In addition to the difficulty of detecting nodules in a single CT scan, it is also challenging for radiologists to track and compare multiple nodules between longitudinal CT scans to grade the malignancy of lung nodules. Although there are different guidelines to guide radiologists in making decisions for arranging future follow-up and calling patients for pathology testing, such decisions are still highly subjective.

[0004] It is known from the paper “Class-Aware Adversarial Lung Nodule Synthesis in CT Images.” ISBI 2019, IEEE, 2019 by Yang, Jie et al. to treat nodule malignancy grading as a 3D image patch classification problem. Nodule candidates are detected by radiologists or computer-aided diagnosis systems. 3D image patches centered on these candidates are extracted from the CT images and sent to a classifier based on a deep convolutional neural network. The malignancy probability of each nodule is then used to make a diagnostic decision. Some related methods also use features extracted using image processing-based techniques to compensate the neural network. In an alternative approach, it is suggested to train auxiliary tasks, such as nodule type classification, together with the malignancy classification task to slightly improve the accuracy.

[0005] Although information on differences between longitudinal studies is the primary source of evidence used by radiologists for decision making, these prior works were unable to use such information. Summary of the invention

[0006] The problem of the invention is to provide an improvement in the diagnosis and characterization of lesions in longitudinal medical images. This problem is solved by a method for classifying lesions, a method for providing a trained classification function, a classification system, medical imaging, a computer program product and a computer readable medium according to the invention and its aspects. Advantageous embodiments are described in the invention and its aspects and in the following description.

[0007] Below, the solution according to the invention is described for the claimed system and for the claimed method. Features, advantages or alternative embodiments herein can be assigned to other corresponding claimed objects, and vice versa. In other words, the features described or claimed in the context of the corresponding method can be used to improve the system. In this case, the functional features of the method are realized by the target unit of the system.

[0008] Furthermore, solutions according to the invention are described with respect to a method and a system for providing a medical data record, with respect to a method and a system for classifying lesions, and with respect to a method and a system for providing a trained classification function. Features, advantages or alternative embodiments herein may be assigned to other corresponding claimed objects, and vice versa. In other words, features of the method and system for classifying lesions may be improved with features described or claimed in the context of a method and a system for providing a trained classification function. In particular, a trained classification function provided by a method or a system for providing a trained classification function may be used within a method or a system for classifying lesions.

[0009] According to a first aspect, the present invention relates to a computer-implemented method for classifying lesions, comprising receiving a first medical image of an examination volume, wherein the first medical image corresponds to a first examination time, and also comprising receiving a second medical image of the examination volume, wherein the second medical image corresponds to a second examination time, the second examination time being different from the first examination time, and also comprising determining a first lesion region corresponding to the lesion in the first medical image, and also comprising determining a registration function based on a comparison of the first medical image and the second medical image, determining a second lesion region in the second medical image based on the registration function and the first lesion region, and also comprising classifying the lesion in the first medical image based on the second lesion region.

[0010] In particular, the steps of receiving the first medical image and receiving the second medical image are performed by an interface, in particular, by an interface of a classification system. In particular, the steps of determining a first lesion region, determining a registration function, determining a second lesion region, and classifying the lesion are performed by a computing unit, in particular, by a computing unit of a classification system.

[0011] In particular, a lesion corresponds to an injury or change in the tissue of an organism (in particular a mammal, in particular a human) usually caused by disease or trauma. In particular, a lesion can be caused by a tumor and is classified according to its benignity and / or its malignancy.

[0012] The medical image corresponds to a result of a medical imaging examination. The medical image may include additional data (particularly metadata). In particular, the medical image may be an X-ray image, a computed tomography image, an ultrasound image, a magnetic resonance image, a positron emission tomography image, a single photon emission computed tomography image and / or a digital pathology image. In particular, the first medical image and the second medical image are of the same type.

[0013] In particular, the medical image is a two-dimensional medical image, a three-dimensional medical image and / or a four-dimensional medical image. The medical image may include a plurality of pixels or voxels (the terms "pixel" and "voxel" are used as synonyms in this specification and correspond to the basic building blocks of an image). In particular, each pixel or voxel includes an assigned intensity value. In particular, the intensity value of a pixel may correspond to an X-ray attenuation value of a tissue mapped by the corresponding pixel or voxel.

[0014] In particular, each medical image corresponds to an examination time, which is a point in time at which the medical image was created. The first medical examination time may be earlier in time than the second medical examination time, or vice versa.

[0015] In particular, the registration function is a function that maps an input medical image to an output medical image. In particular, the registration function assigns a subset of pixels or voxels of the input medical image to corresponding pixels or voxels in the output medical image. In particular, the input medical image and the output medical image have the same dimension. In particular, the first medical image is the input medical image and the second medical image is the output medical image.

[0016] The registration function can be an intensity-based registration function and / or a feature-based registration function. The registration function can be based on a linear (or affine) transformation or on a nonlinear transformation. The nonlinear transformation can be based on a radial basis function, a physical continuum model and / or a large deformation model (e.g., a differential homeomorphism). The registration function can be based on a frequency domain representation of the first medical image and / or the second medical image, such as a Fourier or Laplace transform of the first medical image and / or the second medical image. The registration function can be determined manually, interactively, semi-automatically or automatically. In particular, the registration function can be determined by applying a trained registration function (e.g., a convolutional or non-convolutional neural network) based on a known training registration of paired training images.

[0017] In particular, the lesion region is a region or part of the medical image corresponding to the lesion. In particular, the lesion region may include all pixels or voxels of the medical image corresponding to the lesion. In particular, the first lesion region is a lesion region within the first medical image. In particular, the second lesion region is a lesion region within the second medical image. In particular, the first lesion region and the second lesion region correspond to the same lesion. In particular, the lesion region may be a mask or segmentation of the lesion within the corresponding medical image.

[0018] The step of classifying the lesion may include determining a classification value corresponding to the lesion (or the first lesion area and the second lesion area) and / or providing the classification value. Providing the classification value may include storing, displaying or transmitting the classification value.

[0019] The inventors have realized that by the classification being based on the second lesion region and the second medical image, longitudinal information of the disease path can be taken into account. By using a registration function to determine the second lesion based on the first lesion region, additional data can be used even if the lesion region cannot be (completely) detected from the second medical image alone by conventional methods.

[0020] According to another possible aspect of the invention, determining the second lesion region is based on applying the registration function to the first lesion region. In particular, pixels or voxels corresponding to the first lesion region are mapped to pixels or voxels within the second medical image, and the set of these pixels or voxels within the second medical image defines the second lesion region within the second medical image. In particular, the second lesion region defined by the set of pixels and voxels can be the convex hull of the set of pixels or voxels. Alternatively, pixels or voxels corresponding to the boundary of the first lesion region are mapped to pixels or voxels within the second medical image, and the convex hull of the set of these pixels or voxels within the second medical image defines the second lesion within the second medical image.

[0021] The inventors have realised that by applying the registration function to the first lesion region, the second lesion region can be determined in a fast and efficient manner.By relying on the convex hull, an efficient error correction algorithm can be used for situations where some pixels or voxels of the second medical image are not mapped targets due to numerical or rounding errors.

[0022] According to another possible aspect of the present invention, the step of classifying the lesion is also based on the first lesion region. The inventors have realized that by classifying the lesion based on the first lesion region and the second lesion region, longitudinal information can be incorporated into the classification process very efficiently.

[0023] According to another aspect of the present invention, the step of classifying the lesion comprises the following sub-steps: applying a trained classification function to first input data and second input data, thereby generating output data, wherein the first input data is based on the first lesion region, wherein the second input data is based on the second lesion region, and wherein the step of classifying the lesion comprises the sub-step of determining the lesion classification based on the output data. In particular, the lesion classification may be the same as the output data.

[0024] A trained classification function is a trained function. Typically, a trained function simulates the cognitive functions of humans associated with other human thinking. In particular, through training based on training data, the trained function is able to adapt to new environments and is able to detect and extrapolate patterns.

[0025] Typically, the parameters of the trained function can be adaptively adjusted by training. In particular, supervised training, semi-supervised training, unsupervised training, reinforcement learning and / or active learning can be used. In addition, representation learning (an alternative term is "feature learning") can be used. In particular, the parameters of the trained function can be iteratively adaptively adjusted through several training steps.

[0026] In particular, the trained function may include a neural network, a support vector machine, a decision tree and / or a Bayesian network, and / or the trained function may be based on k-means clustering, Q-learning, a genetic algorithm and / or an association rule. In particular, the neural network may be a deep neural network, a convolutional neural network or a convolutional deep neural network. In addition, the neural network may be an adversarial network, a deep adversarial network and / or a generative adversarial network.

[0027] The inventors have realized that, using the trained classification function, all features corresponding to the first lesion region and the second lesion region can be considered to create output data and classification values. In particular, correlations not identified by human experts can also be used to infer output data and classification values.

[0028] By the input of the trained classification function being based on the first lesion region and the second lesion region, the correlation between different stages of lesion development can be considered. By considering such correlation, lesion identification can be better and more accurate than only based on medical images at different time points.

[0029] According to another aspect of the present invention, the trained classification function is a recurrent neural network, wherein the first input data and the second input data are independently used as input data of the recurrent neural network.

[0030] In particular, a recurrent neural network is an artificial neural network in which the connections between nodes form a directed graph along a time series. In particular, a recurrent neural network can be interpreted as a directed acyclic graph. In particular, a recurrent neural network can be a finite impulse recurrent neural network or an infinite impulse recurrent neural network (wherein a finite impulse network can be unrolled and replaced by a strictly feedforward neural network, while an infinite impulse network cannot be unrolled and replaced by a strictly feedforward neural network). In particular, a recurrent neural network can include additional network structures incorporating time delays or including feedback loops and additional storage states.

[0031] Equivalently, a recurrent neural network can also be defined as a neural network whose output depends not only on the input values ​​and edge weights, but also on a hidden state vector, where the hidden state vector is based on previous inputs used on the recurrent neural network.

[0032] The inventors have realised that, using a recurrent neural network, variable length input sequences can be used. In particular, this means that the method cannot be used for only a fixed number of medical images or lesion regions (and would need to be trained differently for each additional number of medical images or lesion regions used as input), but can be used for an arbitrary number of input medical images or input lesion regions. This means that, independently of the number of consecutive medical images or lesion regions, the entire set of training data can be used within the training of the trained classification function, and the training data is not reduced to training data corresponding to a certain number of consecutive medical images or lesion regions.

[0033] In particular, by independently using the first input data and the second input data as inputs to the recurrent neural network, information from applying the recurrent neural network to the second input data can be stored within the hidden state and can be used as additional information when applying the recurrent network to the first input data. This behavior results in output data and classification values ​​being based on a sequence of lesion regions, so that classification of lesions can be accomplished with high accuracy and taking into account the temporal development of the lesions.

[0034] According to another aspect of the present invention, the recurrent neural network includes at least one long short-term memory ("long short-term memory" abbreviated as LSTM) block.

[0035] In particular, the LSTM block includes a cell, an input gate, an output gate, and a forget gate, wherein the cell corresponds to a hidden vector, and the input gate, the output gate, and the forget gate regulate the flow of information into and out of the cell. In particular, by using cells, the LSTM block can prevent the explosive and vanishing gradient problems that may be encountered when training other types of recurrent neural networks.

[0036] The inventors realized that LSTM blocks are suitable for input data separated by different or even unknown time intervals, and are therefore suitable for use in longitudinal medical imaging studies, because the time difference between different images in a longitudinal study is usually not constant and may even be unknown.

[0037] According to another aspect of the present invention, the classification step further comprises the following sub-step: determining the first input data by applying the trained preprocessing function to at least a portion of the first medical image containing the first lesion area. In addition, the classification step comprises the following sub-step: determining the second input data by applying the trained preprocessing function to at least a portion of the second medical image containing the second lesion area. In particular, the trained preprocessing function is a trained function.

[0038] At least a portion of the first medical image containing the first lesion region is a subset of the first medical image, wherein the subset includes the first lesion region. At least a portion of the second medical image containing the second lesion region is a subset of the second medical image, wherein the subset includes the second lesion region.

[0039] In particular, the trained preprocessing function is applied to at least a portion of the first medical image by applying the trained preprocessing function to a pixel-by-pixel or voxel-by-voxel product of the first medical image and a mask defined by the first lesion region. In particular, the trained preprocessing function is applied to at least a portion of the second medical image by applying the trained preprocessing function to a product of a mask defined by the second lesion region and the second medical image.

[0040] The inventors have recognized that by using a trained preprocessing function, relevant features of the first lesion region and the second lesion region and / or the first medical image and the second medical image can be extracted from the corresponding object before using the data as input to the trained classification network. In particular, the trained preprocessing function can be used for standardization of input data (e.g., to obtain input data with the same brightness and contrast).

[0041] According to another aspect of the invention, a trained preprocessing function is configured to classify lesions within a single medical image by training. In particular, the trained preprocessing function is trained to classify lesions within a single medical image by adaptively adjusting at least one of the parameters of the trained preprocessing function based on a comparison of a training output and a true classification value of the lesion, wherein the training output is a result of applying the trained preprocessing function to the training medical image. In particular, classification based on a single medical image means that only medical images acquired at a specific time are used as input, rather than a longitudinal study.

[0042] The inventors have realised that by using a trained pre-processing function configured to classify lesions within a single medical image, a known image classification function may be used as the trained pre-processing function in order to reduce the training effort for the trained classification function.

[0043] According to another possible aspect of the present invention, the trained preprocessing function is a convolutional neural network. In particular, the trained preprocessing function is a deep convolutional network. The inventors recognize that convolutional networks and deep convolutional networks are very suitable for image processing and extracting features from images.

[0044] According to another aspect of the invention, the registration function is a non-rigid registration function. A synonym for "non-rigid registration function" is "elastic registration function".

[0045] Typically, a rigid registration function stores the value of the Euclidean distance between two points, such that d(TF(x), TF(y))d(x,y), where d(x,y) is the distance between the two points and TF(x) is the result of applying the registration function to the points. Non-rigid registration functions do not store the value of the Euclidean distance between two points.

[0046] In particular, the non-rigid registration functions may be radial basis functions (in particular, selected from the group of thin plate transforms or surface spline transforms, multiple quadratic transforms and compactly supported transforms), physical continuum models (e.g. viscous fluid models) and large deformation models (diffeomorphic transforms).

[0047] The inventors have recognized that by using a non-rigid registration function, local geometric differences between the first medical image and the second medical image can be taken into account. In particular, those local geometric differences may occur due to physical changes in the examination volume between the first medical imaging examination and the second medical imaging examination, or due to different postures of the patient when performing the first medical imaging examination and the second medical imaging examination. In particular, a non-rigid transformation may be helpful if a growing lesion causes changes in the adjacent examination volume.

[0048] According to another aspect of the invention, the registration function is determined to be based on a vector momentum parameterized stationary velocity field (“vector momentum-parameterized stationary velocity field” abbreviated as “vSVF”). The method of vSVF is known, for example, from Z. Shen et al., “Networks for Joint Affine and Non-Parametric Image Registration” (20019) 4219-4228.10.1109 / CVPR.2019.00435. In particular, vSVF is a fluid dynamics method for deforming an image according to a smooth velocity field, wherein the deformation map can be accumulated over time.

[0049] The inventors have realised that using the vSVF technique, the registration function can be determined with better transformation control regularity and in a faster manner than other comparable registration techniques.

[0050] According to another aspect of the present invention, the first medical image and the second medical image are two-dimensional or three-dimensional X-ray based medical images.

[0051] In particular, an X-ray-based medical image is a medical image recorded by means of X-ray radiation. In particular, a three-dimensional X-ray-based medical image can be a computed tomography image.

[0052] The inventors have recognised that the described method is particularly suitable for X-ray based medical images.

[0053] According to a second aspect, the present invention relates to a computer-implemented method for providing a trained classification function. The method is based on receiving a first medical training image of a training examination volume, wherein the first medical training image corresponds to a first examination time, and based on receiving a second medical training image of the training examination volume, wherein the second medical training image corresponds to a second examination time, the second examination time being different from the first examination time. Furthermore, the method is based on determining a registration function based on a comparison of the first medical training image and the second medical training image. Furthermore, the method is based on determining a first lesion area corresponding to a lesion within the first medical training image. Furthermore, the method is based on receiving a training classification corresponding to the first lesion area within the first medical training image. Furthermore, the method is based on determining a second lesion area within the second medical training image based on the registration function and the first medical training image. Furthermore, the method is based on applying the trained classification function to first training input data and second training input data, thereby generating training output data, wherein the first training input data is based on the first lesion area, and wherein the second training input data (TID.2) is based on the second lesion area (LA.2). Furthermore, the method is based on adjusting at least one parameter of the trained classification function based on a comparison of the training classification and the output data. Furthermore, the method is based on providing a trained classification function.

[0054] In particular, the steps of receiving the first medical training image, receiving the second medical training image, receiving the training classification and providing the trained classification function are performed by an interface, in particular, by an interface of a providing system. In particular, the steps of determining the registration function, determining the second lesion area, applying the trained classification function and adjusting at least one parameter of the trained classification function are performed by a computing unit, in particular, by a computing unit of a providing system.

[0055] The inventors realized that based on this method, a trained classification function can be provided in an efficient manner, wherein at the same time the trained classification function can produce high-quality results when classifying lesions in longitudinal studies.

[0056] According to a third aspect, the present invention relates to a classification system comprising:

[0057] - an interface configured to receive a first medical image of an examination volume, wherein the first medical image corresponds to a first examination time,

[0058] is further configured to receive a second medical image of the examination volume, wherein the second medical image corresponds to a second examination time that is different from the first examination time; and

[0059] - a computing unit configured to determine a first lesion region corresponding to a lesion within the first medical image;

[0060] is further configured to determine a registration function based on a comparison of the first medical image and the second medical image;

[0061] is further configured to determine a second lesion region within the second medical image based on the registration function and the first lesion region;

[0062] The method is further configured to classify a lesion in the first medical image based on the second lesion region.

[0063] In particular, the classification system is configured to perform the method for classifying lesions according to the invention and aspects thereof. In particular, the classification system is configured to perform the method for classifying lesions via an interface, and the computing unit is configured to perform individual steps of the method for classifying lesions.

[0064] According to a fourth aspect, the invention relates to a medical imaging system comprising a classification system according to the invention and its aspects. In particular, the medical imaging system may be an X-ray based medical imaging system, such as a computed tomography system or a fluoroscopy system.

[0065] According to a fifth aspect, the invention relates to a providing system comprising an interface and a computing unit:

[0066] - wherein the interface is configured for receiving a first medical training image of a training examination volume, wherein the first medical training image corresponds to a first examination time,

[0067] - wherein the interface is further configured to receive a second medical training image of the training examination volume, wherein the second medical training image corresponds to a second examination time, the second examination time being different from the first examination time,

[0068] - wherein the computing unit is configured to determine a registration function based on a comparison of the first medical training image with the second medical training image,

[0069] - wherein the computing unit is further configured to receive a training classification corresponding to a first lesion region in a first medical training image,

[0070] - wherein the computing unit is further configured to determine a second lesion region within the second medical training image based on the registration function and the first medical training image,

[0071] - wherein the computing unit is further configured to apply the trained classification function to first training input data and second training input data, thereby generating training output data, wherein the first training input data is based on the first lesion area, wherein the second training input data is based on the second lesion area,

[0072] - wherein the computing unit is further configured to adjust at least one parameter of the trained classification function based on a comparison of the training classification with the output data,

[0073] - wherein the interface is further configured to provide a trained classification function (TCF).

[0074] According to a sixth aspect, the present invention relates to a computer program comprising instructions which, when executed by a classification system, cause the classification system to perform the method for classifying lesions according to the invention and aspects thereof.

[0075] According to a possible seventh aspect, the invention relates to a computer program comprising instructions which, when executed by a providing system, cause the providing system to perform the method for providing a trained classification function according to the invention and aspects thereof.

[0076] According to an eighth aspect, the present invention relates to a computer-readable medium comprising instructions which, when executed by a classification system, cause the classification system to perform a method for classifying lesions according to the present invention and aspects thereof.

[0077] According to a possible ninth aspect, the invention relates to a computer-readable medium comprising instructions which, when executed by a providing system, cause the providing system to perform a method for providing a trained classification function according to the invention and aspects thereof.

[0078] Realizing the invention by means of a computer program product and / or a computer-readable medium has the advantage that already existing provisioning systems can be easily adapted by means of a software update to operate as proposed by the invention.

[0079] The computer program product may be, for example, a computer program or may include another element in addition to the computer program. The other element may be hardware, such as a storage device storing the computer program, a hardware key for using the computer program, etc., and / or software, such as a document or software key for using the computer program.

[0080] According to a tenth aspect, the present invention relates to a computer-readable storage medium comprising a trained classification function provided by the method for providing a trained classification function according to the present invention and aspects thereof, for use in a method for classifying a lesion.

[0081] According to a possible eleventh aspect, the present invention relates to a computer-readable storage medium comprising a trained classification function provided by a method for providing a trained classification function according to the invention and aspects thereof.

[0082] According to a possible twelfth aspect, the present invention relates to a method for classifying lesions using a computer-readable storage medium comprising a trained classification function provided by the method for providing a trained classification function according to the invention and aspects thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] The characteristics, features and advantages of the present invention described above and the manner in which they are achieved become clearer and more understandable from the following description and embodiments, which will be described in detail in the context of the accompanying drawings. The following description does not limit the present invention to the embodiments included. In different figures, the same components or parts may be marked with the same reference numerals. Generally, the drawings are not drawn to scale. In the following:

[0084] Figure 1 A timeline indicating the relationship between the medical image and the registration function is shown.

[0085] Figure 2 A data flow diagram showing an embodiment of a method for classifying lesions,

[0086] Figure 3 shows a recurrent neural network including an LSTM block as a trained classification function,

[0087] Figure 4 A first embodiment of a method for classifying lesions is shown,

[0088] Figure 5 A second embodiment of a method for classifying lesions is shown,

[0089] Figure 6 A third embodiment of a method for classifying lesions is shown,

[0090] Figure 7 A fourth embodiment of a method for classifying a lesion is shown,

[0091] Figure 8 An embodiment of a method for providing a trained classification function is shown,

[0092] Fig. 9 The classification system is shown.

[0093] Fig.10 The training system is shown. DETAILED DESCRIPTION

[0094] Figure 1 The relationship between the medical images IMG.1, IMG.2, IMG.3 and the registration functions TF(t.2, t.1), TF(t.3, t.1) is shown.

[0095] The first medical image IMG.1 corresponds to a first examination time t.1, the second medical image IMG.2 corresponds to a second examination time t.2, and the third medical image IMG.3 corresponds to a third examination time t.3. The first medical examination time t.1 is later than the second medical examination time t.2, and the second medical examination time t.2 is later than the third medical examination time t.3.

[0096] Each registration function TF(t.2, t.1), TF(t.3, t.1) is based on a pair of medical images IMG.1, IMG.2, IMG.3. In particular, each registration function TF(t.2, t.1), TF(t.3, t.1) is based on a pair of medical images IMG.1, IMG.2, IMG.3 comprising a first medical image IMG.1 and another medical image IMG.2, IMG.3. In particular, the registration function maps the coordinates of the first medical image IMG.1 to the coordinates of the other medical image IMG.2, IMG.3. The coordinates may be represented by real numbers and / or by integers corresponding to pixels. In particular, the registration function maps pixels or voxels of the first medical image to pixels or voxels of the second medical image.

[0097] For example, if the first medical image IMG.1 and the second medical image IMG.2 are two-dimensional medical images, the registration function TF(t.2, t.1) maps the pixel P.1 of the first medical image IMG.1 coordinated by coordinates P.1 = (i.1, j.1) to the pixel P.2 of the second medical image IMG.2 coordinated by coordinates P.2 = (i.2, j.2) through (i.2, j.2) = TF(t.2, t.1)(i.1, j.1).

[0098] If the first medical image IMG.3 and the second medical image IMG.3 are three-dimensional medical images, the registration function TF(t.2, t.1) maps the voxel P.1 of the first medical image IMG.1 coordinated by coordinates P.1 = (i.1, j.1, k.1) to the voxel P.2 of the second medical image IMG.2 coordinated by coordinates P.2 (i.2, j.2, k.2) through (i.2, j.2, k.2) = TF(t.2, t.1)(i.1, j.1, k.1).

[0099] The registration function TF(t.2, t.1) may also be used to map a first region within the first medical image IMG.1 to a second region within the second medical image IMG.2, wherein the region is determined, for example, by a set of pixels or voxels of the respective images. For example, a first region comprising pixels of the first medical image IMG.1 is mapped to a set of pixels in the second medical image IMG.2, which set of pixels forms a second region in the second medical image IMG.2, wherein the second region in the second medical image comprises pixels resulting from the pixel-by-pixel application of the registration function TF(t.2, t.1) to the pixels of the first region.

[0100] Figure 2 A data flow diagram of some embodiments of the method for classifying lesions is shown. In the data flow diagram, three medical images IMG.1, IMG.2, IMG.3 are used as input, where Figure X As shown in , a registration function TF(t.2, t.1) for the first medical image IMG.1 and the second medical image IMG.2 and a registration function TF(t.3, t.1) for the first medical image IMG.1 and the third medical image IMG.3 are calculated.

[0101] The first medical image IMG.1 comprises a first lesion area LA.1, which comprises a number of pixels or voxels of the first medical image IMG.1. The registration functions TF(t.2, t.1), TF(t.3, t.1) are used to determine a second lesion area LA.2 in the second medical image IMG.2 and a third lesion area LA.3 in the third medical image IMG.3, in particular by LA.2=TF(t.2, t.1)(LA.1) and LA.3=TF(t.3, t.1)(LA.1).

[0102] In this embodiment, the lesion areas LA.1, LA.2, LA.3 are used as inputs of the trained preprocessing function TPF so that the input data ID.1, ID.2, ID.3 are determined by ID.i=TPF(LA.i). In particular, the trained preprocessing function TPF is a trained convolutional neural network. The lesion areas LA.1, LA.2, LA.3 can also be used as inputs of the trained preprocessing function TPF in combination with the corresponding medical images IMG.1, IMG.2, IMG.3, for example, by using the pixel-by-pixel or voxel-by-voxel product of the lesion areas LA.1, LA.2, LA.3 and the corresponding medical images IMG.1, IMG.2, IMG.3 as mask images by ID.i=(LA.i×IMG.i), where × represents the pixel-by-pixel or voxel-by-voxel product. Alternatively, there is no need to use a trained pre-processing function TPF, in which case the input data ID.1, ID.2, ID.3 are equivalent to the lesion areas LA.1, LA.2, LA.3 (or, for example, are equivalent to the pixel-by-pixel or voxel-by-voxel multiplication of the lesion areas LA.1, LA.2, LA.3 as masks with the corresponding medical images IMG.1, IMG.2, IMG.3).

[0103] The input data ID.1, ID.2, ID.3 are then used as input to the trained classification function TCF. In this embodiment, the trained classification function TCF is a recurrent neural network comprising recurrent neural network blocks RNB.1, RNB.2, RNB.3. In particular, each input data ID.1, ID.2, ID.3 is used as input to a different recurrent neural network block in the recurrent neural network blocks RNB.1, RNB.2, RNB.3.

[0104] Each recurrent neural network block RNB.1, RNB.2, RNB.3 determines output data OD.1, OD.2, OD.3 and intermediate data BD.1, BD.2, BD.3, wherein the intermediate data BD.1, BD.2, BD.3 are used as additional input for the next one of the consecutive recurrent neural network blocks RNB.1, RNB.2, RNB.3.

[0105] Then, the classification value of the lesion can be based on all the output data OD.1, OD.2, OD.3, or only on some of the output data OD.1, OD.2, OD.3. In particular, the classification value of the lesion is based on the output data OD.1 related to the first medical image IMG.1, and possibly on the further output data OD.2, OD.3. By being based at least on the output data OD.1 corresponding to the first medical image IMG.1, the nearest one of the medical images IMG.1, IMG.2, IMG.3 can be used with a high weight in the classification of the lesion, which means a better and more accurate classification.

[0106] Figure 3 A detailed view of an LSTM network comprising several recurrent neural network blocks RNB.i, RNB.j is shown. Each recurrent neural network block RNB.i, RNB.j uses input data ID.i, ID.j to generate or calculate output data OD.i, OD.j. In addition, each recurrent neural network block RNB.i, RNB.j takes intermediate data IBD.i, IBD.j as additional input and produces intermediate data OBD.i, OBD.j as additional output, wherein the output intermediate data OBD.i, OBD.j can be used as input intermediate data IBD.i, IBD.j in the next step.

[0107] It is important to understand that Figure 3 An iterative process is shown, which is presented for two inputs. In order to accommodate more input data, the iteration can be extended to cover an arbitrary number of input data ID.i, ID.j. Furthermore, up to a number of internal states IG.i, IG.j, OG.i, OG.j, FG.i, FG.j, the recurrent neural network blocks RNB.i, RNB.j are identical. In particular, this means that the output of the neural network blocks RNB.i, RNB.j depends only on the input data ID.i, ID.j, the additional input intermediate data IBD.i, IBD.j and the internal states IG.i, IG.j, OG.i, OG.j, FG.i, FG.j.

[0108] In this embodiment, the neural network is an LSTM network, and the recurrent neural network blocks RNB.i, RNB.j have internal states represented as input gates IG.i, IG.j, output gates OG.i, OG.j, and forget gates FG.i, FG.j. In particular, the values ​​of these internal states can be calculated as

[0109] i j =σ(W (x，I) *x j +W (y，I) *y i +W (c，I) ·c i +b (I) )

[0110] f j =σ(W (x，F) *x j +W (y，F) *y i +W (c，F) ·c i +b (F) )

[0111] O j =σ(W (x，O) *xj +W (y，O) *y i +W (c，O) ·c j +b (O) )

[0112] c j =f j ·c i +i j tanh(W (x，C) *x j +W (y，C) *y i +b (C) )

[0113] y j =O j tanh(C j )

[0114] In this iteration, the operation "·" is a point-wise multiplication, "*" is a convolution operation, and "σ" represents a sigmoid function. j , o j and f j Corresponds to the values ​​of the input gate IG.j, output gate OG.j, and forget gate FG.j. The value x j and j Corresponds to the input data ID,j and output data OD.j of the corresponding block. Value c i and c j corresponds to the intermediate input intermediate data IBD.i and the output intermediate data OBD.i, OBD.j, and is usually denoted as “unit state”. The values ​​W and b correspond to the weights of the network, which are fixed by training the recurrent neural network.

[0115] In an alternative implementation, the update can be simplified by not letting the cell state affect the update of the input gates IG.i, IG.j, the output gates OG.i, OG.j, and the forget gates FG.i, FG.j:

[0116] i j =σ(W (x，I) *x j +W (y，I) *y i +b (I) )

[0117] f j =σ(W (x，F) *x j +W (y，F) *y i +b (F) )

[0118] O j=σ(W (x，O) *x j +W (y，O) *y i +b (O) )

[0119] C j =f j ·c i +i j tanh(W (x，C) *x j +W (y，C) *y i +b (C) )

[0120] y j =O j tanh(C j )

[0121] In another alternative implementation, the calculation of the cell state may be modified as follows:

[0122] c j =f j ·c i +(1-f j )·tanh(W (x，C) *x j +W (y，C) *y i +b (C) )

[0123] Figure 4 A first embodiment of a computer-implemented method for classifying lesions is shown.

[0124] The first two steps of the embodiment shown are receiving a first medical image IMG.1 of the examination volume REC-IMG.1, and receiving a second medical image IMG.2 of the examination volume REC-IMG.2. The first medical image IMG.1 corresponds to a first examination time t.1 (which is the time at which the first medical image IMG.1 was determined based on a medical imaging examination), and the second medical image IMG.2 corresponds to a second examination time t.2 (which is the time at which the second medical image IMG.2 was determined based on another medical imaging examination). Here, the second examination time t.2 is earlier than the first examination time t.1, and the second examination time is different from the first examination time. In particular, the first medical image IMG.1 and the second medical image IMG.2 may be part of a longitudinal study related to a certain patient.

[0125] In this embodiment, both the first medical image IMG.1 and the second medical image IMG.2 are three-dimensional images generated by means of a computed tomography imaging examination. Alternatively, the first medical image IMG.1 and / or the second medical image IMG.2 may be based on other known medical imaging methods, such as magnetic resonance imaging, positron emission tomography or single photon emission computed tomography. In particular, both the first medical image IMG.1 and the second medical image IMG.2 are digital imaging and communications images in medicine (abbreviated as DICOM). In particular, both the first medical image IMG.1 and the second medical image IMG.2 have the same size (measured in terms of the number of voxels) with respect to each of the dimensions of the respective medical images IMG.1, IMG.2.

[0126] The next step of the first embodiment shown is to determine DET-LA.1 a first lesion area LA.1 corresponding to the lesion within the first medical image IMG.1. In this embodiment, the lesion area corresponds to a segmentation of the lesion within the first medical image IMG.1. In particular, all voxels that are part of the segmentation corresponding to the lesion are considered to be part of the first lesion area LA.1, and all other voxels are considered not to be part of the first lesion area LA.1.

[0127] There are several possibilities known to those skilled in the art for determining the first lesion area LA.1 within the first medical image IMG.1 DET-LA.1. In this embodiment, the method described in "Faster R-CNN: Towards real-time object detection with region proposal networks" by S. Ren et al. in Advances in Neural Information Processing Systems 28 (NIPS 2015) is used.

[0128] A further step of the embodiment shown is determining a DET-RF registration function RF(t.2, t.1) based on a comparison of the first medical image IMG.1 and the second medical image IMG.2. This further step can be performed before, after or in parallel with the step of determining the DET-LA.1 first lesion area LA.1.

[0129] In this embodiment, the registration function RF (t.2, t.1) is based on a non-rigid registration function of a static velocity field parameterized by vector momentum. The method for determining such a registration function RF (t.2, t.1) is known, for example, from the paper "Networks for Joint Affine and Non-Parametric Image Registration" (20019) 4219-4228, 10.1109 / CVPR.2019.00435 by Z. Shen et al.

[0130] In particular, if v represents a vector field, Φ -1 represents the registration function RF(t.2, t.1) (also referred to as the registration map), I 1 represents the first medical image IMG.1 and I 2 Denoting the second medical image IMG.2, the registration function RF(t.2, t.1) can be determined by minimizing the following:

[0131]

[0132] The initial or boundary conditions are given by:

[0133] Φ -1 t +DΦ -1 v=0;Φ -1 (0) = Φ -1 (0) ;v 0 =(L + L) -1 m 0 .

[0134] Here, D represents the Jacobian matrix, and v0 = <L + Lv,v> is obtained by specifying the differential operator L and its adjoint L + The spatial norm is defined by L. Choosing a particular L means choosing the desired deformation model. In vSVF, the differential operator is spatially invariant and is predefined to encode the desired level of smoothness. The vector-valued momentum m is time-space invariant and is equal to m=L + Lv. The optimization takes place on m, where the velocities are smoothed from m. The resulting transformation is guaranteed to be diffeomorphic.

[0135] The next step of the first embodiment shown is to determine the second lesion area LA.2 within the DET-LA.2 second medical image IMG.2 based on the registration function RF(t.2, t.1) and the first lesion area LA.1. In this first embodiment, each voxel of the first medical image IMG.1 corresponding to the first lesion area LA.1 within the first medical image IMG.1 is mapped to at least one voxel of the second medical image IMG.2 using the registration function RF(t.2, t.1). All voxels of the second medical image IMG.2 as a result of such mapping are at least part of the second lesion area LA.2 within the second medical image IMG.2. A further normalization process may be applied, for example, the second lesion area LA.2 may be defined as the convex hull of all voxels of the second medical image IMG.2 as a result of such mapping.

[0136] The last step of the second embodiment shown is to classify the lesions within the CLF first medical image IMG.1 based on the second lesion area LA.2. In the second embodiment, the step of classifying the CLF lesions is also based on the first lesion area LA.1. In this embodiment, the first lesion area LA.1 and the second lesion area LA.2 (or, equivalently, the first medical image IMG.1 and the second medical image IMG.2 in which the first lesion area LA.1 and the second lesion area LA.2 are used as masks) are used as input to a trained classification function TCF, which creates a classification value of the lesion (e.g. a binary value of whether the lesion is benign or malignant) as output.

[0137] Figure 5 A second embodiment of a computer-implemented method for classifying lesions is shown. The steps of receiving a first medical image IMG.1 REC-IMG.1 and receiving a second medical image IMG.2 REC-IMG.2, determining a DET-RF registration function RF(t.2, t.1) and determining a first lesion area LA.1 DET-LA.1 and determining a second lesion area LA.2 DET-LA.2 are equivalent to the corresponding steps of the first embodiment and may include all advantageous embodiments of the corresponding steps.

[0138] In a second embodiment, the step of classifying the lesion CLF comprises the following sub-steps: applying the trained classification function TCF APL-TCF to the first input data ID.1 and the second input data ID.2, thereby generating output data OD.1, OD.2. Here, the first input data ID.1 is based on the first lesion area LA.1, and the second input data ID.2 is based on the second lesion area LA.2. In particular, the first input data ID.1 is based on the mask defined by the first medical image IMG.1 and the first lesion area LA.1, and the second input data ID.2 is based on the mask defined by the second medical image IMG.2 and the second lesion area LA.2.

[0139] In this embodiment, the trained classification function TCF is a recurrent neural network. The recurrent network can be applied to a sequence of input data ID.1, ID.2 by iteratively applying a block (comprising or defined by a neural network) to each input data ID.1, ID.2, maintaining the internal state. For example, if B represents such a block, x n represents the nth input data ID.1, ID.2, then (y n ,h n )=B(x n ,h n+1 ), where y n Indicates the nth output data OD.1, OD.2, and h n Denotes the hidden state vector after the nth application of block B of the recurrent neural network. Symbol B HV (x n ,h n+1 )=h n It may refer to the hidden vector portion of the output of block B. The initial value h of the hidden state vector may be initialized randomly or by some sequence (eg, all entries may be set to 0 or 1).

[0140] Note that the iteration is defined “backwards” because the first input data ID.1 (denoted as x 1 ) corresponds to the last data in the time series of the longitudinal study.

[0141] In particular, if there are a first medical image IMG.1 and a second medical image IMG.2, the second output data OD.2 can be obtained by (y 2 ,h 2 )=B(x 2 , h) calculation, and the first output data OD.1 can be obtained by (y 1 ,h 1 )=B(x 1 ,h 2 )=B(x 1 , BHV (x 2 , h)) calculation.

[0142] The step of classifying the lesion further comprises the following sub-step: determining a DET-LC lesion classification based on the output data OD.1, OD.2. In this embodiment, the lesion classification is a real number between 0 and 1 corresponding to the probability that the classified lesion is malignant. Alternatively, the lesion classification may be a class label indicating the type of lesion. In particular, the lesion classification may be equivalent to the output data OD.1 corresponding to the first medical image IMG.1.

[0143] Figure 6 A third embodiment of a computer-implemented method for classifying lesions is shown. The steps of receiving REC-IMG.1 a first medical image IMG.1 and receiving REC-IMG.2 a second medical image IMG.2, determining a DET-RF registration function RF(t.2, t.1), determining DET-LA.1 a first lesion area LA.1 and determining DET-LA.2 a second lesion area LA.2, and the sub-steps of applying a trained classification function TCF APL-TCF to the first input data ID.1 and the second input data ID.2 and determining DET-LC lesion classification are equivalent to the corresponding steps of the first embodiment and / or the second embodiment and may include all advantageous embodiments of the corresponding steps.

[0144] In a third embodiment, the step of classifying lesions CLF includes the following sub-steps: determining DET-ID.1 first input data ID.1 by applying the trained preprocessing function TPF to at least a portion of the first medical image IMG.1 containing the first lesion area LA.1; and determining DET-ID.2 second input data ID.2 by applying the trained preprocessing function TPF to at least a portion of the second medical image IMG.2 containing the second lesion area LA.2.

[0145] In this embodiment, the trained preprocessing function TPF is a convolutional deep neural network. In particular, the trained preprocessing function TPF is applied to the pixel-by-pixel or voxel-by-voxel multiplication of the respective lesion area LA.1, LA.2 interpreted as a mask and the respective medical image IMG.1, IMG.2.

[0146] If I n Represents the nth medical image IMG.1, IMG.2 and L n Corresponding to the nth lesion area LA.1, LA.2, where L n With I n The same dimensions and the same size, and for pixels or voxels that are not part of the corresponding lesion area, L n= 0, and for pixels or voxels that are part of the corresponding lesion area, L n =1, then the nth input data ID.1 and ID.2 can be calculated as x n =TPF(I n ·L n ).

[0147] In particular, in the notation of the second embodiment, if there are a first medical image IMG.1 and a second medical image IMG2, the second output data OD.2 can be obtained by (y 2 ,h 2 )=B(TPF(I 2 ·L 2 ), h), and the first output data OD.1 can be calculated by (y 1 ,h 1 )=B(TPF(I 1 ·L 1 ), h2) = B(TPF(I 1 ·L 1 ), B HV (TPF(I 2 ·L 2 , h)) to calculate.

[0148] Figure 7 A fourth embodiment of a computer-implemented method for classifying lesions is shown. The steps of receiving REC-IMG.1 a first medical image IMG.1 and receiving REC-IMG.2 a second medical image IMG.2, determining DET-LA.1 a first lesion area LA.1, determining DET-LA.2 a second lesion area LA.2 and classifying the lesions may comprise all alternatives and advantageous embodiments of the first, second and third embodiments of the method for classifying lesions described above.

[0149] The fourth embodiment shows an extension of the method for three medical images IMG.1, IMG.2, IMG.3 within a longitudinal study. The third medical image IMG.3 is of the same type as the first medical image IMG.1 and the second medical image IMG.2. The third medical image IMG.3 corresponds to a third time t.3, wherein the third time t.3 is earlier than the second time.

[0150] Compared with the previous embodiment, the fourth embodiment further comprises the step of receiving the REC-IMG.3 third medical image IMG.3. If there are additional medical images, there will be an additional receiving step for each of the additional medical images.

[0151] In addition, compared with the previous embodiment, in the fourth embodiment, the step of determining the DET-RF registration functions RF(t.2, t.1), RF(t.3, t.1) includes the following sub-steps: for each of the medical images IMG.2, IMG.3 other than the first medical image IMG.1, the DET-RF.2, DET-RF.3 registration functions RF(t.2, t.1), RF(t.3, t.1) are determined based on the first medical image IMG.1 and the corresponding other medical images IMG.2, IMG.3. In this fourth embodiment, the step of determining the DET-RF registration functions RF(t.2, t.1), RF(t.3, t.1) includes the following sub-steps: determining the first registration function RF(t.2, t.1) based on the first medical image IMG.1 and the second medical image IMG.2; and determining the second registration function RF(t.3, t.1) based on the first medical image IMG.1 and the third medical image IMG.3. If further medical images are present, there is a further sub-step of determining a registration function based on the first medical image and the respective further medical image.

[0152] The fourth embodiment further comprises the additional step of determining a third lesion area LA.3 within the third medical image IMG.3 DET-LA.3. The step of determining the third lesion area LA.3 DET-LA.3 is equivalent to the step of determining the second lesion area LA.2 DET-LA.2, wherein the second medical image IMG.2 is replaced by the third medical image IMG.3 (and using the corresponding registration function RF(t.3, t.1)). If there will be additional medical images, there will be an additional step of determining the lesion area for each additional medical image.

[0153] Furthermore, the step of classifying CLF is based on all lesion areas LA.2, LA.3.

[0154] Figure 8 A flow chart of an embodiment of a computer-implemented method for providing a trained classification function TCF is shown.

[0155] The first step of the illustrated embodiment is receiving a first medical training image of a TREC-TIMG.1 training examination volume, wherein the first medical training image corresponds to a first examination time, and receiving a second medical training image of the training examination volume, wherein the second medical training image corresponds to a second examination time different from the first examination time. The first medical training image and the second medical training image may include Figure 4 All advantageous features described in the context of the first medical training image and the second medical training image.

[0156] Particularly, both the first medical training image and the second medical training image are medical images based on two-dimensional or three-dimensional X-rays.

[0157] The further steps of the embodiment shown are determining a first lesion region corresponding to a lesion in the first medical training image by TDET-LA.1, determining a TDET-RF registration function based on a comparison of the first medical training image and the second medical training image, and determining a second lesion region in the second medical training image based on the registration function and the first medical training image. These three steps may include utilizing Figures 4 to 7 All advantageous features and embodiments described for the corresponding steps of the embodiment of the method for classifying lesions shown in one of the drawings.

[0158] Advantageously, the registration function is a non-rigid registration function. In particular, the TDET-RF registration function is determined to be based on a static velocity field parameterized by vector momentum. In particular, it can be determined as Figure 4 The registration function is determined as described for the method for classifying lesions.

[0159] Another step of the illustrated embodiment is receiving a training classification corresponding to a first lesion region in a first medical training image by TREC-TCL. In particular, the training classification may be a binary value corresponding to whether the lesion corresponding to the first lesion region is a malignant lesion or a benign lesion. Alternatively, the training classification may be a class label corresponding to a class of the lesion corresponding to the first lesion region, such as a BI-RADS ("Breast Imaging Reporting and Data System" abbreviated as BI-RADS) classification value.

[0160] Another step of the illustrated embodiment is applying the trained classification function TCF TAPL-TCF to first training input data and second training input data to generate training output data, wherein the first training input data is based on the first lesion region and wherein the second training input data is based on the second lesion region.

[0161] The relationship between the first training input data, the lesion region, and the first medical training image is equivalent to the relationship between the first input data, the first lesion region, and the first medical image described in relation to the method for classifying lesions. The relationship between the second training input data, the second lesion region, and the second medical training image is equivalent to the relationship between the second input data, the second lesion region, and the second medical image described in relation to the method for classifying lesions.

[0162] Furthermore, the trained classification function may include all advantageous features and embodiments as the trained classification function described with respect to the method of classifying a lesion.

[0163] Advantageously, the trained classification function TCF is a recurrent neural network and the first training input data and the second training input data are independently used as training input data for the recurrent neural network. Advantageously, the recurrent neural network comprises at least one LSTM block.

[0164] Advantageously, the step of applying the TAPL-TCF trained classification function TCF further comprises the following sub-steps: determining first training input data by applying the trained preprocessing function TPF to at least a portion of a first medical training image containing a first lesion area, and determining second training input data by applying the trained preprocessing function TPF to at least a portion of a second medical training image containing a second lesion area. In particular, the trained preprocessing function is configured by training to classify lesions within a single medical image, and / or, wherein the trained preprocessing function TPF is a convolutional neural network.

[0165] In particular, the trained pre-processing function TPF may be pre-trained based on a comparison between the trained classifications of lesions within single medical images and the results of applying the trained pre-processing function TPF to these single medical images.

[0166] Another step of the displayed embodiment of the method for providing a trained classification function is to adjust at least one parameter of the TADJ-TCF trained classification function TCF based on the comparison of the training classification with the training output data. In particular, the parameters are adaptively adjusted by minimizing the cost function, wherein the training classification and the trained output parameters are used as input values ​​of the cost function. In particular, if the training classification is equal to the training output data, the cost function has a (local or global) extreme value. In particular, adjusting at least one parameter of the TADJ-TCF trained classification function TCF may include a gradient descent algorithm and / or a back propagation algorithm.

[0167] The last step of the shown embodiment is providing TPROV-TCF the trained classification function TCF. In particular, providing TPROV-TCF the trained classification function TCF may comprise displaying, storing and / or sending the trained classification function TCF.

[0168] Fig. 9 The classification system CSYS is shown. Fig.10 The training system TSYS is shown. The classification system CSYS includes an interface CSYS.IF, a calculation unit CSYS.CU, and a storage unit CSYS.MU. The training system TSYS includes an interface TSYS.IF, a calculation unit TSYS.CU, and a storage unit TSYS.MU.

[0169] The classification system CSYS and / or the training system TSYS can be a (personal) computer, a workstation, a virtual machine running on host hardware, a microcontroller or an integrated circuit. In particular, the classification system CSYS and / or the training system TSYS can be a mobile device, such as a smartphone or a tablet. As an alternative, the classification system CSYS and / or the training system TSYS can be a real or virtual group of computers (the technical term for a real group of computers is a "cluster", the technical term for a virtual group of computers is a "cloud").

[0170] The interfaces CSYS.IF, TSYS.IF can be implemented as hardware interfaces or software interfaces (e.g. PCIBus, USB or FireWire). Typically, computing units CU.1, CU.2, CU.3 may include hardware elements and software elements, such as a microprocessor, a central processing unit (abbreviated as CPU), a graphics processing unit (abbreviated as GPU), a field programmable gate array (abbreviated as FPGA) or an application-specific integrated circuit (abbreviated as ASIC). The computing units CSYS.CU, TSYS.CU may be configured for multithreading, i.e., the computing units CSYS.CU, TSYS.CU may carry different computing processes at the same time, execute in parallel or switch between active and inactive computing processes. The storage units CSYS.MU, TSYS.MU may be, for example, non-permanent main memory (e.g. random access memory) or permanent mass storage devices (e.g. hard disk, USB stick, SD card, solid state disk).

[0171] The interfaces CSYS.IF, TSYS.IF may include several (possibly spatially) separated sub-interfaces, each of which has the characteristics of the above-described interfaces. The computing units CSYS.CU, TSYS.CU may include several (possibly spatially) separated computing sub-units, each of which has the characteristics of the computing units described above. The storage units CSYS.MU, TSYS.MU may include several (possibly spatially) separated storage sub-units, each of which has the characteristics of the storage units described above.

[0172] The classification system CSYS is configured to perform one of the embodiments of the method for classifying lesions. In particular, the classification system CSYS is configured to perform the method via an interface CSYS.IF and a calculation unit CSYS.CU configured to perform the corresponding steps of the method. The training system TSYS is configured to perform one of the embodiments of the method for providing a trained classification function TCF. In particular, the training system TSYS is configured to perform the method via an interface TSYS.IF and a calculation unit TSYS.CU configured to perform the corresponding steps of the method.

[0173] Where not explicitly described, individual embodiments or their individual aspects and features may be combined or exchanged with one another without limiting or expanding the scope of the described invention, provided that such combination or exchange makes sense and is within the meaning of the invention. Wherever applicable, the advantages described with respect to one embodiment of the invention are also advantages of other embodiments of the invention.

Claims

1. A computer-implemented method for classifying a lesion, comprising: - receiving a first medical image of an examination volume, wherein the first medical image corresponds to a first examination time, - receiving a second medical image of the examination volume, wherein the second medical image corresponds to a second examination time, the second examination time being different from the first examination time, - determining a first lesion region corresponding to said lesion in said first medical image, - determining a registration function based on a comparison of said first medical image and said second medical image, - determining a second lesion region in the second medical image based on the registration function and the first lesion region, - applying the trained classification function to the first input data and the second input data to generate output data, Wherein, the first input data is based on the first lesion area, Wherein, the second input data is based on the second lesion area, Wherein, the trained classification function is a recurrent neural network, wherein the first input data and the second input data are independently used as input data of the recurrent neural network, and wherein information from applying the recurrent neural network to the second input data is stored in a hidden state and is used as additional information when applying the recurrent neural network to the first input data, - determining a lesion classification based on the output data.

2. The method according to claim 1, wherein: The recurrent neural network includes at least one LSTM block.

3. The method according to claim 1 or 2, wherein: - determining said first input data by applying a trained pre-processing function to at least a portion of said first medical image comprising said first lesion area, - determining said second input data by applying said trained pre-processing function to at least a portion of said second medical image comprising said second lesion area.

4. The method according to claim 3, wherein: The trained pre-processing function is configured through training to classify lesions within a single medical image.

5. The method according to any one of claims 1, 2 and 4, wherein: The registration function is a non-rigid registration function.

6. The method according to claim 3, wherein: The registration function is a non-rigid registration function.

7. The method according to claim 5, wherein: Determining the registration function is based on a stationary velocity field parameterized by a vector momentum.

8. The method according to any one of claims 1, 2, 4, 6 and 7, wherein: The first medical image and the second medical image are two-dimensional or three-dimensional X-ray based medical images.

9. The method according to claim 3, wherein: The first medical image and the second medical image are two-dimensional or three-dimensional X-ray based medical images.

10. The method according to claim 5, wherein: The first medical image and the second medical image are two-dimensional or three-dimensional X-ray based medical images.

11. A computer-implemented method for providing a trained classification function, comprising: - receiving a first medical training image of a training examination volume, wherein the first medical training image corresponds to a first examination time, - receiving a second medical training image of the training examination volume, wherein the second medical training image corresponds to a second examination time, the second examination time being different from the first examination time, - determining a first lesion region corresponding to a lesion in said first medical training image, - determining a registration function based on a comparison of said first medical training image and said second medical training image, - receiving a training classification corresponding to a first lesion region in said first medical training image, - determining a second lesion region within the second medical training image based on the registration function and the first medical training image, - applying the trained classification function to the first training input data and the second training input data to generate training output data, Wherein, the first training input data is based on the first lesion area, Wherein, the second training input data is based on the second lesion area, Wherein, the trained classification function is a recurrent neural network, wherein the first training input data and the second training input data are independently used as input data of the recurrent neural network, and wherein information from applying the recurrent neural network to the second training input data is stored in a hidden state and used as additional information when applying the recurrent neural network to the first training input data, - adjusting at least one parameter of the trained classification function based on a comparison of the training classification with the training output data, - providing said trained classification function.

12. A classification system for classifying lesions, comprising: - an interface configured to receive a first medical image of an examination volume, wherein the first medical image corresponds to a first examination time, is further configured to receive a second medical image of the examination volume, wherein the second medical image corresponds to a second examination time, the second examination time being different from the first examination time; and - a computing unit configured to determine a first lesion region corresponding to a lesion within the first medical image; is further configured to determine a registration function based on a comparison of the first medical image and the second medical image; is further configured to determine a second lesion region within the second medical image based on the registration function and the first lesion region; is further configured to apply the trained classification function to the first input data and the second input data to generate output data, Wherein, the first input data is based on the first lesion area, Wherein, the second input data is based on the second lesion area, Wherein, the trained classification function is a recurrent neural network, wherein the first input data and the second input data are independently used as input data of the recurrent neural network, and wherein information from applying the recurrent neural network to the second input data is stored in a hidden state and used as additional information when applying the recurrent neural network to the first input data; The system is further configured to determine a lesion classification based on the output data.

13. A medical imaging system comprising the classification system according to claim 12.

14. A computer program product comprising instructions which, when the computer program is executed by a classification system, cause the classification system to perform the method according to one of claims 1 to 10.

15. A computer-readable medium comprising instructions which, when executed by a classification system, cause the classification system to perform the method according to one of claims 1 to 10.

16. A computer-readable storage medium comprising a trained classification function provided by the method according to claim 11, for use in a method for classifying lesions according to one of claims 1 to 10.

Images