Image processing apparatus and method

Abstract

The present invention relates to an image processing apparatus and method that can be used for comparative brain imaging. In order to improve detection of anomalies with a sparse scan history having a considerable time difference between available images, the image processing device comprises: an input (30) configured to obtain a series of two or more images of an object of interest of an object and a reference image of the object of interest of the object, the input image has been acquired at a different point in time, and the reference image has been acquired at a later point in time than the input image; a processing unit (31) configured to generate an estimated image representing an estimated representation of the object of interest at an estimated point in time by applying a trained algorithm or computing system to the input image, the trained algorithm or computing system having been trained on a plurality of training images, the plurality of training images show objects of the same type as the object of interest at different points in time so that the object of interest shown in the input image artificially ages for a desired period of time, and determining a deviation of the estimated image from the reference image; and an output section (32) configured to output the generated estimated image and the determined deviation.

Description

Technical Field

[0001] The present invention relates to an image processing device and method. Background Art

[0002] Detection of brain lesions and other abnormalities can be accomplished through comparative imaging, in which previous and current brain MR images are registered and can be compared. For example, comparative brain imaging can be used to track the progression of multiple sclerosis (MS) patients in order to simplify the visual detection of new white matter lesions. MS patients are scanned frequently during patient care, so that differences between baseline and follow-up images are primarily associated with new white matter lesions.

[0003] Except for specific disease areas like multiple sclerosis or brain tumors, subjects are scanned infrequently. Therefore, typically, the scanning history of an object is sparse, i.e., only a few scans (also referred to herein as "images") for the object are available.

[0004] If the time difference between the baseline image and the subsequent images is large, e.g. many years, the contrast difference is not only related to abnormalities but also to normal aging, e.g. normal atrophy in the case of brain aging or normal vascular effects. Therefore, finding abnormalities is a difficult task, especially in the case of sparse scanning histories with considerable time differences between the available images. The images may generally be any kind of images of the object of interest, such as X-ray images, CT images, MR images, PET images, ultrasound images, etc.

[0005] TIAN XIA ET AL.: "Learning to synthesize the ageing brain without longitudinal data" (ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 2010 CORNELL UNIVERSITY ITHACA, NY 14853, May 11, 2021) discloses a deep learning-based approach that learns to model subject-specific brain aging trajectories without relying on longitudinal data. The method synthesizes images modulated on two factors: age and Alzheimer's disease status. Using an adversarial formulation, a joint distribution of brain appearance, age, and AD status is learned.

[0006] RAVIDANIELE et al.: “Degenerative adversarial neuroimage nets for brainscan simulations: Application in ageing and dementia” (MEDICAL IMAGE ANALYSIS, OXFORD UNIVERSITY PRESS, OXOFRD, GB, Vol. 75, October 14, 2021) discloses a deep learning framework, the 4D degenerative adversarial neuroimage network, to generate high-resolution longitudinal MRI scans that simulate subject-specific neurodegenerative diseases in aging and dementia based on a single MRI baseline scan and their healthy state.

[0007] WEGMAYR VIKTOR et al.: "Generative Aging Of Brain MRI For Early Prediction Of MCI-AD Conversion" (2019IEEE 16TH INTERNATIONAL SYMPOSIUM ONBIOMEDICAL IMAGING (ISBI 2019), IEEE, April 8, 2019) discloses a deep learning-based approach to predict the transition of mild cognitive impairment (MCI) to Alzheimer's disease (AD) based on widely available TI-weighted MR brain images. Given a single baseline image, a synthetic aged brain image is generated. The aged image is passed to an MCI / AD discriminator, which determines the future disease state. Summary of the invention

[0008] The object of the present invention is how to improve the detection of anomalies in case of sparse scan histories with considerable time differences between available images.

[0009] In a first aspect of the present invention, an image processing device is provided, comprising:

[0010] an input unit configured to obtain a series of two or more images of an object of interest of a subject and a reference image of the object of interest of the subject, wherein the input images have been acquired at different points in time and the reference image has been acquired at a later point in time than the input images;

[0011] A processing unit configured to:

[0012] generating an estimated image by applying a trained algorithm or computing system to the input image, the estimated image representing an estimated representation of the object of interest at an estimated point in time, the trained algorithm or computing system having been trained on a plurality of training images showing objects of the same type as the object of interest at different points in time to artificially age the object of interest shown in the input image for a desired period of time, and

[0013] determining a deviation of the estimated image from the reference image; and

[0014] An output unit is configured to output the generated estimated image and the determined deviation.

[0015] In another aspect of the present invention, a system is provided, comprising:

[0016] an image acquisition device configured to acquire a series of two or more input images of an object of interest of a subject and a reference image of the object of interest of the subject, wherein the input images are acquired at different points in time and the reference image is acquired at a second point in time later than the input images; and

[0017] An image processing apparatus as disclosed herein.

[0018] In another aspect of the present invention, a corresponding image processing method is provided; a computer program comprising a program code module, when the computer program is executed on a computer, the program code module is used to cause the computer to perform the steps of the method disclosed herein; and a non-volatile computer-readable recording medium storing a computer program product therein, which, when run by a processor, causes the method disclosed herein to be executed.

[0019] Preferred embodiments of the invention are defined in the dependent claims. It should be understood that the claimed method, system, computer program and medium have similar and / or identical preferred embodiments as the claimed system, in particular as defined in the dependent claims and as disclosed herein.

[0020] The present invention is based on the idea of ​​utilizing a trained algorithm or computing system (e.g., a trained model or neural network) to generate an image of the object of interest (e.g., the brain of a subject) that "simulates" how the object of interest would look at a desired point in time if it were experiencing normal aging without abnormalities. For example, if only a sparse scan history for the subject is available, an estimate of a current hypothetical subsequent image that includes only the effects of healthy aging can be made, and a measure that quantifies the deviation of the sparse scan history from healthy aging can be determined. Thus, the disclosed devices, systems, and methods can be used in comparative imaging applications (e.g., for comparative brain imaging), where visual and / or algorithmic comparisons of hypothetical and true current images can reveal only the effects of abnormalities and not normal aging. Furthermore, they can be used to assess whether changes in the subject's scan history can be explained by normal aging.

[0021] In other words, one of the ideas of the present invention is to estimate a current image based on an individual estimated aging trajectory given two or more input images from the past (also referred to herein as first images), and then calculate the difference with a real image (the reference image, also referred to herein as second image) that has been acquired later than the input images, preferably at the current point in time or at least in the recent past, to show real deviations (such as lesions or other abnormalities) and hide the effects of "normal" aging.

[0022] According to a preferred embodiment, the processing unit is configured to use a learning system, a neural network, a convolutional neural network or a U-net-like network as a trained algorithm or computing system. Generally, the architecture or layout of the trained algorithm or computing system is not required, but for example a learning system or a neural network with a known or conventional architecture can be used to calculate the third image.

[0023] In another embodiment, applying the trained algorithm or computing system by the processing unit to two or more input images comprises: analyzing the input images, in particular analyzing the two or more input images by a convolutional neural network (CNN) to derive image features of the object of interest; and generating the reference image based on the derived image features and information on how these kinds of image features evolve over time, in particular generating the reference image by a generator network based on the derived image features and information on how these kinds of image features evolve over time, in particular age-related atrophy. This provides a simple but effective approach. CNNs operate using convolution kernels with trained weights, but the approach is purely training / data driven.

[0024] The processing unit may also be configured to generate the reference image by aging the object of interest shown in the input image for a time period corresponding to the time difference between the time point at which the corresponding input image has been acquired and the time point at which the reference image has been acquired, in particular the time difference between the time point at which the last input image has been acquired and the time point at which the reference image has been acquired. Thus, in an exemplary application, the input image may be used to generate the estimated image at an estimated time point corresponding to the time point at which the reference image has been acquired. Thus, the reference image and the estimated image show the object of interest at the time period after the time point at which the input image has been acquired, and thus may be directly compared to identify anomalies.

[0025] According to another embodiment, applying the trained algorithm or computing system by the processing unit to the two or more first images comprises: analyzing the input images, in particular analyzing the input images by means of a convolutional neural network, to derive aging information indicating how the object of interest shown in the input images has aged over the time spanned by different time points at which the input images have been acquired; and generating the estimated image based on the derived aging information and the time difference between the time point at which the input image has been acquired and the time point at which the reference image has been acquired, in particular generating the estimated image by means of a generator network based on the derived aging information and the time difference between the time point at which the input image has been acquired and the time point at which the reference image has been acquired.

[0026] Thus, the aging information obtained from the input images helps to understand how a specific object under examination, in particular its object of interest, has changed over time in the past due to "normal" aging, i.e. this aging information can be learned from the input images. This aging information and time information about the input images, in particular when they have been acquired and in particular how much time has passed between the acquisition of different input images and the acquisition of a reference image, can then be used to further improve the generation of the estimated image. For example, if the time difference between, for example, the last input image (i.e. the youngest first image acquired before the acquisition of the reference image) and the reference image is large, the intensity / effect of the aging will generally be greater than if this time difference is smaller. This time information can be taken into account when generating the estimated image.

[0027] According to the invention, the processing unit is configured to determine the deviations of the reference image from the estimated image, and the output unit is configured to output the determined deviations. These deviations are then typically interpreted as anomalies rather than effects of normal aging. Thus, the processing unit can be configured to quantitatively process and / or qualitatively process the deviations (e.g. with respect to size, number, intensity, etc.), and the output unit is configured to output the determined quantitative results and / or qualitative results.

[0028] The time difference between the time point at which the corresponding input image has been acquired and the time point at which the reference image has been acquired and the time difference between the time point at which the corresponding input image has been acquired and the time point at which the estimated image has been estimated are typically longer periods, such as more than a week or more than a month or more than six months or more than a year. The time difference may typically depend on how many images of the object of interest are available and at which intervals the images to be compared have been acquired.

[0029] In another embodiment, the processing unit is configured to perform a registration of the input image with the reference image before generating the estimated image and / or to perform a registration of the estimated image with the reference image after generating the estimated image. This improves the comparison and detection of anomalies.

[0030] Preferably, the processing unit may be configured to train the algorithm or computing system on a plurality of training images showing objects of the same type as the object of interest at different points in time, wherein the plurality of training images include real images and / or synthetic images. Such training images may be obtained from an available database, or may be artificially created for training purposes. Training is preferably performed in advance and offline. The actual generation of the estimated images and the determination of anomalies, particularly if verified by a user such as an experienced clinician, may also be used for training. Therefore, in an embodiment, the calculated estimated images may be used together with the input images and the reference images to train the algorithm or computing system. This may also be used to fine-tune the algorithm or computing system during actual use. For example, an optional online learning component, for example as a function of the processing unit, may be provided for this fine-tuning.

[0031] In an embodiment, the processing unit may be configured to train the algorithm or computing system by analyzing (in particular by a convolutional neural network) the plurality of training images to derive image features describing the normal appearance of the object over training time. Thereby, the image features may be automatically determined as weights for the convolutional neural network so that there is a latent space representation of the image. The weights may be applied not only to the encoding part, but also to the decoding part, i.e. the part used to recreate the image from its latent space representation. The image features may include one or more of the following: brightness, texture, edges, boundaries, patterns, characteristic structures or points, uniformity, area, shape, size, or any other feature that can be learned and used for training. The use of one or more of these image features (or characteristics) enables practical implementation of the disclosed solution.

[0032] In other embodiments, the first layer of the neural network may include image features, which means that the weights of the convolutional filters will constitute image features. In a deep neural network, additional layers with additional weights may follow, where the weights do not correspond to image features. Between the first layer (image features) and the deepest layer (which may output a latent space representation of the image), there may be more layers and weights that correspond neither to latent space nor to image space features.

[0033] The processing unit may also be configured to perform a general training phase and a personalization phase for the neural network as a preprocessing step before generating the third image, wherein a part of the network is retrained, in particular based on data from the object of interest. This also improves the personalized detection of anomalies. Technically, these weights do not differ from each other. In the general training phase, the weights are jointly optimized, and in the personalization phase, only a defined subset of the weights is retrained using the images of the object as training pairs. This is often expressed as "freezing" certain weights during a specific period of training, i.e. the weights can be fixed / frozen during the personalization phase.

[0034] In an embodiment, for retraining in the personalization phase, the input images may be used to improve a trained network based on information about the subject under examination, since the network may thereby learn how the subject, and in particular the object of interest of the subject, has aged in the past. Thus, the retrained network may better predict how the object of interest of the subject ages over time, and may therefore generate a further improved estimated image that more realistically represents the aged object of interest at the estimated point in time.

[0035] For example, in an embodiment, the processing unit is configured to additionally use one or more personal characteristics of the subject of the corresponding image in the plurality of training images to train the algorithm or computing system. Such personal characteristics may include one or more of age, gender, size, weight, medical history.

[0036] In general, the present invention can be applied in different contexts and used with different objects of interest. One application is in the field of medical image processing, where the object of interest is an anatomical organ, such as the brain or the heart. Depending on the anatomical structure, further preprocessing steps may be utilized, such as rigid registration or segmentation of the target anatomical structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiment(s) described hereinafter. In the following drawings:

[0038] Figure 1 A schematic diagram of an embodiment of a system according to the invention is shown.

[0039] Figure 2 A schematic diagram showing an embodiment of an image processing device according to the present invention.

[0040] Figure 3 A schematic diagram showing an embodiment of an image processing method according to the present invention.

[0041] Figure 4 A schematic diagram of an embodiment of a neural network for a processing unit is shown.

[0042] FIG5 schematically illustrates an embodiment of a U-net and its use in the context of the present invention. DETAILED DESCRIPTION

[0043] Figure 1 A schematic diagram of an embodiment of a system 1 according to the present invention for image processing, in particular image processing of medical images acquired by an imaging device of any imaging modality (eg CT, MR, X-ray, PET, ultrasound, etc.) is shown.

[0044] The system 1 comprises an image acquisition device 2 configured to acquire a series of two or more input images (first images) of an object of interest of a subject and a reference image (second image). The first images are acquired at different first points in time (e.g. once every six months or once a year), and the second image is acquired at a second point in time later than the two or more first images (e.g. x months or a year after the last first image). The first and second images may, for example, be MR images of a patient's brain acquired at a difference of a few weeks, months or even a year or more. Thus, the image acquisition device 2 may be an imaging device of one of the modalities mentioned above.

[0045] The system 1 further comprises an image processing device 3 as will be described below.

[0046] The system 1 may also include a display 4 configured to display the estimated image (third image) and / or other information obtained by using the present invention. The display 4 may be, for example, a screen or monitor of a computer, laptop, table or workstation. The third image may be shown together with one or more of the first image and / or the second image, in particular to identify deviations between the different images and illustrate potential anomalies (e.g., anomaly temporal development or anomalies based on comparison with a synthetic "normal" image at a later point in time).

[0047] The system 1 may also include a user interface 5 configured to enable a user (e.g., a clinician, a care provider, etc.) to select images that should be subjected to image processing. The user interface 5 may, for example, include one or more of a touch screen, a keyboard, a computer mouse, a pointer, or any other module that enables a user to perform desired function(s).

[0048] Furthermore, the system 1 may include a storage device 6 for storing images, such as a storage device in a computer, a network or a cloud. Previously acquired images may be stored in the storage device 6 .

[0049] Figure 2 A schematic diagram of an embodiment of a device 3 for processing an image according to the present invention is shown. The device 3 may be implemented as or in a processor or a computer, for example in software and / or hardware.

[0050] The device 3 comprises an input 30 configured to obtain a series of two or more input images (first images) of an object of interest of the subject and a reference image (second image). The input 30 may be directly coupled or connected to the image acquisition device 2 and / or may obtain (i.e. retrieve or receive) images (completely or partially) from a storage device, a buffer, a network, a bus, etc. Thus, the input may be, for example, a (wired or wireless) communication interface or data interface, such as a Bluetooth interface, a WiFi interface, a LAN interface, an HDMI interface, a direct cable connection or any other suitable interface allowing a signal to be transmitted to the device 3.

[0051] The device 3 further comprises a processing unit 31 configured to generate an estimated image (third image) by applying a trained algorithm or computing system on the first image, the estimated image representing an estimated representation of the object of interest at an estimated time point (third time point), the trained algorithm or computing system having been trained on a plurality of training images showing objects of the same type as the object of interest at different time points, so as to artificially age the object of interest shown in the first image for a desired period of time. Furthermore, a deviation of the third image from the second image is determined. The processing unit 31 may be any kind of module configured to process the obtained image data and generate the third image therefrom. It may be implemented in software and / or hardware, for example as a programmed processor or computer or application on a user device such as a smartphone, smartwatch, tablet, laptop, PC, workstation, etc.

[0052] The device 3 further comprises an output 32 configured to output the generated third image. The output 32 may generally be any interface that provides the third image, for example, transmits it to another device or provides it for retrieval by another device (e.g. a smartphone, a computer, a tablet, etc.). Thus, it may generally be any (wired or wireless) communication or data interface.

[0053] Figure 3 A schematic diagram of a first embodiment of an image processing method 100 according to the present invention is shown. The steps of the method 100 may be performed by the device 3, wherein the main steps of the method are performed by the processing unit 31. The method 100 may be implemented, for example, as a computer program running on a computer or a processor.

[0054] In a first step 101, image data (including first and second images) is obtained. In a second step 102, a third image is generated. In a third step 103, the third image is output.

[0055] In a second step 102, the third image may be generated, for example, by artificially aging the object of interest for a desired period of time, in particular by aging the object of interest as shown in the two or more first images for a period of time corresponding to the time difference between the last first point in time (at which the last first image has been acquired) and the second point in time. For example, if the last first image has been acquired one year ago and the second image has been acquired today, the desired period of time by which the last first image should be aged is one year. In other words, the two or more first images are subject to a trained algorithm or computing system to age at least one of the first images, preferably the last first image, by one year to generate the third image.

[0056] Then, in an optional fourth step 104 (which may be performed before, in parallel with, or after the third step 103), the third image may be compared to the second image to identify deviations between the third image and the second image, which deviations may then be interpreted as abnormalities, i.e., changes in anatomical structures not caused by normal aging, such as abnormal brain changes not caused by normal aging. Then, in an optional fifth step 105 (which may be a separate step or part of step 103), these deviations may be output (e.g., highlighted in the second image, or given as annotations or lists of the second image). Thus, the deviations may be processed quantitatively (e.g., how much deviation) and / or qualitatively (e.g., how strong the deviation is), and this information may also be output. For example, a value indicating a deviation from normal may be output.

[0057] Applying the trained algorithm or computing system to two or more first images may include analyzing the first images to derive image features of the object of interest (such as brightness, texture, edges, boundaries, patterns, characteristic structures or points, uniformity, area, shape, size, etc.), and generating a third image based on the derived image features and information on how such image features evolve over time (particularly age-related atrophy). This information has been trained in advance through a plurality of training images, which may be real images and / or artificially created images. For example, if the disclosed method is supposed to be used for comparative brain image analysis, the training images are brain images acquired at a plurality of different time points to train the algorithm or system how the brain ages normally over time. Depending on the time difference between the two actual images, the algorithm or system then knows how to "modify" the first image to create a third image so that it looks like the same brain after the time difference.

[0058] More generally, in an embodiment, a trained algorithm or computing system may derive aging information, i.e., it may "learn" from two or more first images how an object of interest of a subject has aged in the past, and may then use that information to better predict how the object of interest has aged since the last first image has been acquired. This aging information is then used to generate a third image.

[0059] It should be noted that the invention can be used not only with two images, but also with series of images of the same object of interest that have been acquired over time. This can be done, for example, so that non-linear effects of aging (i.e. effects that evolve linearly over time) may also be learned by the network. In other words, having more images available (and therefore more pairs of images at various times), a "personalized training phase" will lead to more robust predictions. If there were only two images (a pair) available, the retrained network might overfit to this one image pair and not produce reasonable results for any other time difference, but there are multiple images to which the predicted aging will generalize better. This may be relevant for the training phase, but may also be useful in the real operation phase.

[0060] For training and practical application of algorithms or systems, learning systems, neural networks, convolutional neural networks or U-net-like networks can be used. For example, in applications such as Figure 4 The architecture of the neural network shown can be used as a processing unit for training and for application in the generation of a third image. The architecture includes a convolutional neural network 40 that can be used to analyze a first image to derive image features and a generator network 41 that can be used to generate a third image based on the derived image features. In many successful implementations for different types of image analysis tasks, a U-net architecture with an encoder part and a decoder part can be found. It is used as a basis for introducing a personalization step. However, alternative methods may also be possible. In other embodiments, the two neural networks 40 and 41 can be combined into a single network, and other networks may also be used alternatively.

[0061] Convolutional networks are generally very efficient (i.e., they do not require as much training data as the simplest architecture which would consist of only connected layers), and also generally generalize much better. They are often the main component of image processing networks. A generator network is generally understood to be a network that gets something as input (e.g., a latent vector that can be either random or meaningful), and creates an output (usually an image or a word / sentence, etc.), which is then (usually during training) classified by a second neural network (usually called a "discriminator"), so that the discriminator trains the generator part (also called the "generating part" or "generating network") to produce realistic outputs. There are other ways of doing that, for example, the decoding part of a variational autoencoder can also be used as a generating network; the generator network can then simply be the decoding branch (e.g., from latent space to image space) of a larger network (e.g., a U-net).

[0062] The processing unit 31 may also be configured to train the algorithm or computing system on a plurality of training images showing objects of the same type as the object of interest at different points in time. For example, it may analyze the plurality of training images, such as by a convolutional neural network, to derive image features describing the normal appearance of the object at the time to be trained.

[0063] In embodiments for training, a training database may be created or used. Given a large longitudinal database of brain scans such as the UE Biobank, NAKO, the ADNI database, or the Rotterdam Scan Study, as examples, available diagnostic and other data (such as cognitive or psychiatric scores and other brain or systemic diseases) may be used to exclude subjects that are considered unsuitable for modeling healthy aging. For example, this includes healthy subjects with a subsequent diagnosis of a neurodegenerative disease based on clinical symptoms, because changes in the object (e.g., the brain) are visible earlier, making the corresponding scans unsuitable for modeling normal aging.

[0064] like Figure 4 As shown, the neural network used for training may include a convolutional neural network 40 that analyzes image data and generates features describing the normal appearance of the brain during training. A generator network 41 fed by the results of the convolutional neural network 40 generates individual brain images for desired time points. Optionally, a recurrent neural network may be used between the two to indicate how the features evolve over time through healthy aging. However, this task may instead be performed by one of the networks 40 and 41.

[0065] The parameters of a neural network type used for image synthesis can be trained using the scans in the training database together with the corresponding age of the subject at the time of image acquisition, so that for given additional age values, the difference between the synthesized output image and the real output image is minimized.

[0066] FIG5 schematically illustrates an embodiment of a U-net and its use in the context of the present invention.

[0067] Figure 5A The generic training phase is illustrated. The numbers given in terms of blocks are specific to the image resolution and size, and the number of layers (blocks) may also vary depending on the task. In the generic training phase, the weights for the U-net are trained by minimizing a loss function related to the deviation of the synthetically generated output image MR O from its true counterpart. The training is based on a pair of two input images MR I at time point T and MR O at a later time point, as well as the corresponding time point information as represented by age I and age O. During training, pairs of two input images from the same object can be randomly pulled from the overall training image set.

[0068] Figure 5B Illustration of the personalization phase with existing scans of an object of interest. This phase differs from the generic training phase for most of the network (marked in black) in that it uses the longitudinal history of the object of interest, i.e. the available images of the object of interest that have been acquired in the past. Here, the (residual) part of the network (marked in grey as blocks connected to the latent space representation at the bottom) is retrained using the same loss function (in the preprocessing step). Therefore, it also requires input image pairs (first and second images) and corresponding time points for the object of interest.

[0069] Figure 5C The inference phase leading to an image for a target age estimate is illustrated.Here, a third image MR X is generated for the current time point given an earlier image MR I and its acquisition date or patient age.

[0070] Compared to the known architecture and usage of U-net, the latent space / bottleneck in the network (at the bottom) includes additional information such as the age of the subject at the time of scanning and Figure 5A The "personalized" weights for the training shown.

[0071] In summary, according to the present invention, although some differences between images are usually caused by normal aging, by comparing the subject's brain MRI with previous scans, detection of abnormalities (e.g., brain abnormalities such as lesions) is possible. A predictive model for personalized normal aging can be established, which can be trained using a database of longitudinal scans of healthy subjects. Given the scan history of healthy individuals, the system is able to (i) estimate a metric that indicates the current likelihood of abnormal aging, and (ii) estimate a hypothetical current scan based on the history of scans. This pseudo scan of the subject will be more suitable for comparison with the real current scan to reveal abnormalities that are not related to age.

[0072] The invention can be used, for example, as software for comparative imaging of medical objects, such as comparative brain imaging or MR brain change assessment in a medical workstation.

[0073] Although the present invention has been illustrated and described in detail in the drawings and the foregoing description, these illustrations and descriptions should be considered illustrative or exemplary rather than restrictive; the present invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and implemented by those skilled in the art in practicing the claimed invention by studying the drawings, the disclosure, and the claims.

[0074] In the claims, the word "comprising" does not exclude other elements or steps, and the word "a" or "an" does not exclude a plurality. A single element or other unit may fulfill the functions of several items recited in the claims. Although certain measures are recited in mutually different dependent claims, this does not indicate that a combination of these measures cannot be used to advantage.

[0075] The computer program may be stored / distributed on suitable media, such as optical storage media or solid-state media provided together with or as part of other hardware, but the computer program may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems.

[0076] Any reference signs in the claims should not be construed as limiting the scope.

Claims

1. An image processing device, include: An input unit (30) configured to obtain a series of two or more images of an object of interest of a subject and a reference image of the object of interest of the subject, wherein the input images have been acquired at different points in time and the reference image has been acquired at a later point in time than the input images; A processing unit (31) configured to: generating an estimated image by applying a trained algorithm or computing system to the input image, the estimated image representing an estimated representation of the object of interest at an estimated point in time, the trained algorithm or computing system having been trained on a plurality of training images showing objects of the same type as the object of interest at different points in time to artificially age the object of interest shown in the input image for a desired period of time, and determining a deviation of the estimated image from the reference image; and An output unit (32) is configured to output the generated estimated image and the determined deviation.

2. The image processing device according to claim 1, in, The processing unit (31) is configured to use a learning system, a neural network, a convolutional neural network or a U-net-like network as a trained algorithm or a computing system.

3. An image processing device according to any one of the preceding claims, in, Applying the trained algorithm or computing system to the input image by the processing unit (31) comprises: Analyzing the input image, in particular by means of a convolutional neural network (40), to derive image features of the object of interest, and The reference image is generated based on the derived image features and information on how these types of image features evolve over time, in particular by a generator network (41) based on the derived image features and information on how these types of image features evolve over time, in particular age-related atrophy.

4. An image processing device according to any one of the preceding claims, in, The processing unit (31) is configured to generate the reference image by aging the object of interest shown in the input image for a time period, the time period corresponding to the time difference between the time point at which the corresponding input image has been acquired and the time point at which the reference image has been acquired, in particular the time difference between the time point at which the last input image has been acquired and the time point at which the reference image has been acquired.

5. Image processing device according to any one of the preceding claims, in, Applying the trained algorithm or computing system to the input image by the processing unit (31) comprises: analyzing the input image, in particular by means of a convolutional neural network (40), to derive aging information indicating how the object of interest shown in the input image has aged over the time spanned by different time points at which the input image has been acquired, The estimated image is generated based on the derived aging information and the time difference between the time point at which the input image has been acquired and the time point at which the reference image has been acquired, in particular by a generator network (41) based on the derived aging information and the time difference between the time point at which the input image has been acquired and the time point at which the reference image has been acquired.

6. Image processing device according to any of the preceding claims, in, The processing unit (31) is configured to perform quantitative processing and / or qualitative processing on the deviation, and Wherein, the output unit (32) is configured to output the determined quantitative result and / or qualitative result.

7. Image processing device according to any of the preceding claims, in, The time difference between the time point when the corresponding input image has been acquired and the time point when the reference image has been acquired, and the time difference between the time point when the corresponding input image has been acquired and the time point when the estimated image has been estimated, exceeds one week, one month, six months or one year.

8. Image processing device according to any of the preceding claims, in, The processing unit (31) is configured to: perform registration of the input image with the reference image before generating the estimated image, and / or perform registration of the estimated image with the reference image after generating the estimated image.

9. Image processing device according to any of the preceding claims, in, The processing unit (31) is configured to train the algorithm or computing system on a plurality of training images, the plurality of training images showing objects of the same type as the object of interest at different points in time, wherein the plurality of training images include real images and / or synthetic images.

10. The image processing device according to claim 9, in, The processing unit (31) is configured to train the algorithm or computing system by analyzing the multiple training images, in particular to train the algorithm or computing system by analyzing the multiple training images using a convolutional neural network to derive image features describing the normal appearance of the object over a training time.

11. Image processing device according to any of the preceding claims, in, The processing unit is configured to perform a generic training phase for a neural network and a personalization phase as a pre-processing step before generating the estimated image, in which pre-processing step parts of the network are retrained.

12. The image processing device according to claim 11, in, The processing unit (31) is configured to additionally use the age of the subject of a corresponding image of the plurality of training images to train the algorithm or computing system.

13. A system, include: An image acquisition device (2) configured to acquire a series of two or more first images of an object of interest of a subject and a second image of the object of interest of the subject, wherein the two or more first images are acquired at different first time points, and the second image is acquired at a second time point later than the two or more first images; and Image processing device (3) according to any of the preceding claims.

14. An image processing method, include: obtaining a series of two or more images of an object of interest of a subject and a reference image of the object of interest of the subject, wherein the input images have been acquired at different points in time and the reference image has been acquired at a later point in time than the input images; generating an estimated image by applying a trained algorithm or computing system to the input image, the estimated image representing an estimated representation of the object of interest at an estimated point in time, the trained algorithm or computing system having been trained on a plurality of training images showing objects of the same type as the object of interest at different points in time to artificially age the object of interest shown in the input image for a desired period of time; determining a deviation of the estimated image from the reference image; and The generated estimation image and the determined deviation are output.

15. A computer program comprising program code means for causing a computer to perform the steps of the method according to claim 14 when the computer program is executed on the computer.

Images