Follow-up analysis of patients

Abstract

The present invention discloses patient follow-up analysis. A computer-implemented method for comparing follow-up medical findings is provided. The method includes the following steps: receiving image data showing a body part of a patient at a first time; and retrieving reference data associated with the image data from a database. The reference data includes reference biometric data and one or more reference medical findings. Thus, the reference biometric data and one or more reference medical findings are extracted from reference image data describing a body part of the patient at a second time different from the first time. The method also includes processing the received image data so as to extract biometric data and one or more medical findings corresponding to the reference biometric data from the image data. In addition, the method includes performing registration of the extracted biometric data with the reference biometric data. Subsequently, the extracted medical findings are compared with the reference medical findings using the registration.

Description

Technical Field

[0001] Embodiments relate to subsequent analysis of patient data, such as medical image data. Background Art

[0002] When a medical diagnosis is derived from medical images acquired by a medical image acquisition modality such as an X-ray device, an ultrasound device, a CT or MR scanner, follow-up readings and assessment of changes are very important. Radiologists usually rely not only on the latest available images when making their decisions, but also on the entire historical record. This historical record may include multiple medical image data sets acquired for the same patient (for the same body part of the patient) at different times and usually using different medical imaging technologies. By tracking such follow-up examinations, changes in medical findings over time can be determined, thereby generating valuable insights for reaching a correct medical diagnosis. Changes in findings can, for example, indicate whether a tumor is growing or a stenosis is worsening and immediately affect treatment decisions.

[0003] In particular, three-dimensional image acquisition techniques such as CT or MR produce a large number of images. Therefore, it becomes very time consuming to browse the images to identify images that are related to medical findings in previous image scans or a small number of images to be able to determine changes in medical findings over time. If data acquired using different imaging parameters or even different imaging modalities are to be compared, the process will be further complicated because these data sets are usually not well aligned. Therefore, the same or similar anatomical features of the subject will appear in different positions in subsequent image data sets. Further difficulties arise from the patient's "normal" anatomical variations due to respiratory motion or other "healthy" anatomical differences, which are always present and mask potentially relevant medical findings.

[0004] In order to assist radiologists in this task, different methods for three-dimensional (image) registration of image volumes have been designed. Such methods essentially derive a coordinate transformation between the coordinate systems of subsequent image datasets based on the original image data. Since these methods usually rely on various image processing steps in order to facilitate spatial co-registration and match the corresponding anatomical structures in the two datasets, they require that the two image volumes be registered to be locally available.

[0005] However, this cannot always be ensured in practice. Firstly, medical image data are often very large. Taking a chest CT dataset as a typical example, it can be safely assumed that a single scan takes up more than 100MB of disk space. Keeping such a dataset readily available for possible later comparisons would require very high storage capacity - with the result that the required data is often already discarded before a subsequent read is needed. Furthermore, new image volumes and previous image volumes are often stored in different storage locations. The image volumes for a newly performed examination may be available locally on the radiologist's reading computer system, while the previous image volumes are stored in a central image database, such as a picture archiving and communication system (PACS). In this case, access rights to the image database may become a problem and the transfer of such large data files may still take a considerable amount of time. This is even more problematic if the previous image volumes are stored in a remotely accessible location with poor or no network access at all.

[0006] The situation is even worse if image registration is to be provided as a cloud service. Here, data security and privacy guidelines often severely restrict the cloud service provider's access to local databases storing medical information. In turn, archiving complete image datasets on cloud platforms for subsequent reading is often not a sustainable option either, as this would require huge data storage capacities and likewise raise important patient data privacy issues. Summary of the invention

[0007] It is therefore an object of the present invention to provide a computer-aided diagnostic tool that supports a user / physician / radiologist / pathologist in deriving a medical diagnosis from medical image data, in particular by implementing an improved automated subsequent reading functionality. In this regard, a specific object of the present invention is to provide an improved computer-implemented method that can be easily implemented in a distributed environment, seamlessly integrated into existing clinical workflows, and improves the availability of comparative data for subsequent reading. Furthermore, it is desirable to provide a method (and related system) that is more easily compliant with existing data privacy regulations, can be flexibly applied to medical image data from a variety of sources, and reduces the amount of data that needs to be kept available.

[0008] This object is solved by a method for comparing subsequent medical examinations, a corresponding system, a corresponding computer program product and a computer-readable storage medium according to the technical solution of the present disclosure.Alternative and / or preferred embodiments are the object of the technical solution of the present disclosure.

[0009] In the following, the technical solution according to the present invention is described with respect to the claimed device and the claimed method. The features, advantages or alternative embodiments described herein can be equally assigned to other claimed objects, and vice versa. In other words, the claims for the method of the present invention can be improved by describing or claiming features with respect to the device. In this case, for example, the functional features of the method are implemented by the target unit or element of the device.

[0010] According to one aspect, a computer-implemented method for comparing subsequent medical examinations (or subsequent medical image data) is provided. The method comprises several steps. A first step involves receiving a medical image dataset showing a body part of a patient at a first time. Subsequently, at least one reference dataset associated with the image dataset is retrieved from a database, wherein the reference dataset comprises reference biometric data extracted from a reference image dataset depicting the body part of the patient at a second time different from the first time. Thus, the reference biometric data defines the location of an anatomical part of the body part in the reference image dataset. A next step involves extracting biometric data from the image dataset, the biometric data defining the location of an anatomical part of the body part in the received image dataset and corresponding at least in part to the reference biometric data previously extracted from the reference image dataset and included in the reference data. Next, the reference dataset is registered with the image dataset based on the reference biometric data and the extracted biometric data, and then the image dataset is compared with the reference dataset based thereon.

[0011] The (medical) image dataset (i.e. the reference image dataset and the received image dataset) may relate to a two-dimensional dataset providing two dimensions in space. Furthermore, the image dataset may relate to a three-dimensional dataset providing three dimensions in space (in other words, the medical image dataset may therefore relate to volumetric image data also referred to as a medical image volume). Furthermore, the image dataset may relate to a time series of two-dimensional or three-dimensional image datasets. The image dataset depicts a body part of the patient in the sense that the image dataset comprises two-dimensional or three-dimensional image data of the body part of the patient (i.e. the reference medical image dataset and the medical image dataset introduced above as image dataset may depict the same body part of the same patient). The medical image dataset may, for example, be in the form of an array of pixels or voxels. Such an array of pixels or voxels may represent an intensity, absorption or other parameter that varies with a two-dimensional or three-dimensional position and may, for example, be obtained by suitable processing of measurement signals obtained by a medical imaging modality. A medical imaging modality corresponds to a system for generating or producing a medical image dataset. For example, the medical imaging modality may be a computed tomography system (CT system), a magnetic resonance system (MR system), an angiography (or C-arm X-ray) system, a positron emission tomography system (PET system), an ultrasound imaging system, and the like. Specifically, computed tomography is a widely used imaging method and utilizes "hard" X-rays generated and detected by a special rotating instrument. The resulting attenuation data (also referred to as raw data) is provided by computer analysis software, which generates a detailed image of the internal structure of the patient's body part. The resulting image set is called a CT scan, which may constitute a plurality of sequences of continuous images to present the internal anatomical structure in a cross section perpendicular to the axis of the human body. To give another example, magnetic resonance imaging (MRI) is an advanced medical imaging technique that utilizes the effect of a magnetic field on the motion of protons. In an MRI machine, the detector is an antenna, and the signal is analyzed by a computer to create a detailed image of the internal structure in any part of the human body. Therefore, the depicted patient's body part will typically include a plurality of anatomical parts and / or organs. Taking a chest image as an example, the reference image data and the subsequent image data may display lung tissue, thorax, lymph nodes, and the like.

[0012] Although the received (medical) image dataset was obtained at a first time of examination, the reference (medical) image dataset relates to an examination performed at a second time different from the first time. The second time may be hours, days, weeks, months or years after or before the first time. In addition, there may be intermediate scans or procedures between the first time and the second time. In one embodiment, the second time is before the first time, so that the received medical image dataset can be considered to be related to the subsequent examination relative to the reference medical image dataset. In one embodiment, the image dataset has been acquired using the same medical imaging modality with the same or similar settings and parameters as the reference image dataset. According to another embodiment, the image dataset has been acquired using different medical imaging modalities. For example, the reference image dataset may have been acquired using a CT system, while the received image dataset was acquired using an ultrasound system.

[0013] "Receiving" may mean that the image dataset is provided by a user by way of uploading. Alternatively, the image dataset may be received by means of another computer-implemented method via an automated workflow outside the presently claimed method. For example, it is conceivable that the image dataset is provided by a computer-aided diagnosis system, which automatically or semi-automatically selects the image dataset for subsequent analysis according to the claimed method. In any case, the receiving step triggers the subsequent processing steps, and if initiated by the user, the receiving step may be regarded as an implicit user request for performing subsequent image analysis according to the subsequent steps. While "receiving" may be regarded as a passive step, the subsequent "retrieval" step (and other subsequent steps) may be regarded as an active step, in which the method actively seeks an appropriate reference dataset for subsequent reading.

[0014] The database may be a database of reference data sets from various patients for comparison with subsequent medical data sets according to the present invention. It may include any storage medium or organizational unit for storing and accessing reference data sets. The database may include multiple separate storage units and repositories, and may in particular include a distributed data architecture. The database may include various data records accessed by appropriate interfaces to manage the delivery of reference data sets. The database may be a local database accessible by the method. The database may be located at the same site from which the medical image data set is received. In addition, the database may be implemented as a cloud storage facility, and thus a platform for centrally archiving reference data sets from multiple institutions (or sites or users) may be provided.

[0015] The reference dataset constitutes reference data or baseline data derived from a reference examination for comparison with the submitted image data. "Associated with the received image dataset" may mean that the two datasets (i.e., the image dataset and the reference dataset) belong to the same patient, but represent data acquired at different points in time. The association may be achieved by a data identifier assigned to the two datasets and unambiguously tying the two datasets together. The data identifier may be an accession number, a patient ID, the patient's name, etc.

[0016] The step of "retrieval" can be considered as a step of automatically querying a database for a reference dataset based on the received image dataset. It is worth noting that this step is not limited to retrieving exactly one reference dataset. On the contrary, there may be other reference datasets in addition to the at least one reference dataset mentioned in the step. Therefore, in other words, this step includes retrieving "at least one" reference dataset or "one or more" reference datasets, each acquired at a different second time. All retrieved reference datasets can undergo the same processing steps, namely extracting corresponding information from the received image dataset, aligning with the image dataset (and optionally other retrieved reference datasets), and comparing with the received image dataset (and optionally other retrieved reference datasets). Therefore, the method also allows the image dataset to be systematically compared with multiple reference datasets, thereby enabling the tracking of disease or health progression through multiple follow-up examinations.

[0017] While the medical image dataset primarily comprises image data, the reference dataset primarily comprises data extracted from the medical image data by one or more image processing steps. Although all or part of the original image data may still be included in the reference dataset, it is preferred that the reference dataset comprises the extracted data in place of the underlying complete image dataset.

[0018] In particular, the reference data set comprises reference biometric data extracted from a reference medical image data set. The biometric data defines the location of the anatomical part in the body part as shown in the (reference) image data set. The biometric data may relate to biologically meaningful points, regions, contours, lines, etc., wherein the correspondence between different image data sets of the examined body part (e.g., in the form of homologous parts) is preserved. In other words, the biometric data relates to unique, measurable geometric identifiers that can be defined in the depicted body part. In particular, the biometric data may be vertices, anchor points, control points, positions, contour points, "sampling" points, nodes, markers, fiducial markers, points in a shape positioned according to certain mathematical or geometric properties, fixed points, markers, features or structures that can be defined in the body part depicted in the (reference) image data set. Although the location of the biometric data is given in the coordinates of the base image data set, the presence of the corresponding features is invariant between the image data sets and depends only on the base body part. Therefore, such biometric data can be used to describe the location relationship of the findings, observations, anatomical parts and organs inside the body part, regardless of the subjective coordinate system spanned by the pixels or voxels of the medical image data set. Biometric data can be considered invariant in the sense that each body part of the same kind includes these biometric data points, even though they are at different locations in the medical image dataset. In other words, biometric data is universal at least to some extent in the sense that each body part of the same kind should have potential features. Specifically, the biometric data may include one or more (anatomical) landmarks, a set or grid of one or more landmarks, one or more anatomical (bio) landmarks, a set or grid of one or more anatomical (bio) landmarks, one or more contours or segmentation masks of organs or anatomical parts included in the body part, one or more surface maps or grids of organs or anatomical parts included in the body part, wherein the surface grid can be regarded as a collection and / or combination of points describing the surface of the object. Since the presence or existence of biometric data is invariant when comparing different medical image datasets, they can be used as reference points or benchmarks for comparing the results of subsequent examinations and provide an excellent basis for the registration of two datasets.

[0019] Several known methods can be used to extract such (reference) biometric data from a reference medical image dataset. For extracting one or more anatomical landmarks from a medical image dataset, for example, reference is made to US 2018 / 0005083A1 and EP 3370206A1, the contents of which are included herein by reference.

[0020] In order to compare the retrieved reference dataset with the received subsequent medical image dataset, the subsequent medical image dataset is processed in a next step so as to extract data from the subsequent medical image data that corresponds to the reference dataset. In other words, this means extracting biometric data from the received image dataset that at least partially corresponds to the reference biometric data. The extraction of biometric data from the image dataset may rely on the same or different processes than the extraction of the reference biometric data. Since the biometric data is associated with a common geometric identifier, it is generally not critical which method is actually used for the extraction of the biometric data. In this context, "at least partially corresponding" may mean that at least a portion of the reference biometric data relates to the same biometric information as at least a portion of the newly extracted biometric data. For example, the reference biometric data may include a first set of anatomical landmarks or surface meshes, and the extracted biometric data may include a second set of landmarks or surface meshes, wherein the first set and the second set at least partially overlap. In this context, it should be noted that the reference biometric data and the extracted biometric data are extracted from different medical image datasets, which may have different imaging parameters or originate from different imaging modalities altogether. Therefore, the available biometric information may naturally differ to some extent, so that a one-to-one correspondence of the biometric information can only rarely be achieved anyway. However, such partial correspondences are usually sufficient for registration-based requirements, as this particular approach is usually also very capable when working with alternative datasets (compared to other techniques such as correlation-based algorithms).

[0021] Using the reference biometric data and its extracted corresponding part, a registration can be performed, wherein the retrieved reference biometric data and the corresponding part of the extracted biometric data are registered to each other. If anatomical landmarks are considered as examples of information included in the biometric data, the registration can be based on a set of corresponding anatomical landmark pairs, each pair of corresponding anatomical landmarks comprising a first anatomical landmark extracted from the reference image dataset and a corresponding second anatomical landmark extracted from the uploaded / received image dataset. The registration can include calculating a (global) transformation function (or transformation matrix) that maps positions in the received image dataset to corresponding positions in the reference (image) dataset and minimizes the spatial distance of the positions of the biometric data in the two medical image datasets. The transformation function can be considered as extrapolating the displacement traceable for the reference biometric data and the extracted biometric data to the entire image volume covered by the image dataset and the reference (image) dataset. The transformation function can be based on rigid, affine, non-rigid transformations and / or any combination thereof. In addition, the calculation of the transformation function can involve the use of a deformation model for soft tissue deformation as shown in the Navier-Cauchy equations. The transformation function obtained in the registration step not only provides information about how the coordinates of the biometric data in one image dataset are transformed into the coordinates of another image dataset, but also provides an indication of how the entire two-dimensional or three-dimensional coordinate system of the reference image and the subsequent image is locally transformed.

[0022] Thus, by means of the registration (i.e. the resulting transformation function), the received image dataset can be directly compared with the reference dataset, since, for example, any local information comprised in the reference dataset (e.g. relating to local observations in the reference image data) can be transformed into the coordinate system of the received medical image dataset and vice versa. This allows spatially corresponding information to be matched and any changes to be quantified on this basis (to this end, the step of comparing may comprise: matching spatially corresponding information in the image dataset and the reference dataset and any changes to be quantified on this basis).

[0023] In summary, the method steps described above synergistically contribute to a method that facilitates the effective comparison of subsequent medical image data (medical examinations) and thus improves the process of subsequent reading. Therefore, a method is provided to assist users in providing medical diagnoses by processing physiological measurements. Therefore, the method allows comparisons to be made in medical image data sets, thereby facilitating the determination of disease progression, treatment response, etc. There is a dual advantage in basing the co-registration of subsequent medical images on a registration based on biometric data (e.g., in the form of anatomical landmarks) rather than on a traditional two-dimensional or three-dimensional image registration. On the one hand, the biometric data described above provide a universal and universal reference point for the registration. This is because the biometric data does not (or at least not primarily) depend on the imaging parameters, but on the physiological characteristics of the body part under consideration. This improves the comparability of results generated using different imaging parameters and facilitates the integration of results from different modalities. Since the use of biometric data allows the comparison of subsequent medical images in a standardized manner, the process is seamlessly integrated into existing workflows and requires less manual interaction. In this regard, the automatic retrieval of reference data sets from a database in particular reduces the workload of the user, since the user does not need to provide the reference data set himself. Furthermore, by automating the retrieval, it can be more reliably ensured that no relevant data for comparison is lost. On the other hand, the method is able to significantly reduce the amount of data that needs to be processed in relation to the reference data. As a result, more reference data sets can be kept available, which increases the likelihood that a suitable reference data set can actually be retrieved. Since the reference data sets remain available in a processed format, only newly received image data sets need to be processed. This reduces processing time and the required computing power and enables comparisons with multiple reference data sets at once to be made more quickly. Furthermore, data traffic and therefore network latency can be reduced. In addition, retrieval from a specified database makes it easier to grant required data access rights and comply with existing data privacy regulations.

[0024] According to one embodiment, the step of performing the registration includes: selecting a biometric region common to the biometric data and the reference biometric data as a region used as a standard for registering the medical image data and the reference data set; and performing a registration step of registering (preferably rigid, non-rigid, affine or any combination thereof) using the biometric region as a registration standard.

[0025] According to one embodiment, the step of performing the registration comprises determining corresponding biometric information in the reference biometric data and the subsequent biometric data, and performing the registration using the corresponding biometric information.

[0026] According to one embodiment, the step of performing registration includes: analyzing the biometric data and the reference biometric data to determine corresponding data points in the biometric data and the reference biometric data; and performing a registration step (preferably rigid, non-rigid, affine or any combination thereof) for the medical image dataset and the reference dataset using the corresponding data points.

[0027] By determining corresponding data points and / or biometric regions and / or corresponding biometric information, it can be ensured that only corresponding data are used for the registration. This can make the registration more efficient, since the computation time can be shortened and the results may be more accurate. In addition, this improves the compatibility of different data sets, which may, for example, show significantly different imaging windows or be related to results from different imaging modalities. The corresponding data are related to the common data in the two data sets. The corresponding data may, for example, relate to the same anatomical landmarks, the same anatomical surface grid, etc.

[0028] According to one embodiment, the reference dataset also includes one or more reference medical findings extracted from a reference image dataset, and the reference medical findings are respectively associated with positions in the reference image dataset, wherein the extraction step also includes: extracting one or more medical findings associated with the positions in the image dataset from the image dataset, wherein the comparison step includes: comparing the extracted medical findings with the reference medical findings, in particular by quantifying the changes in the medical findings.

[0029] A (reference) medical finding can be considered as information about a physiological or pathological feature or observation at a given location in a (reference) medical image dataset. The information included in the medical finding can be digital information. For example, the information can include the volume, cross-section, circumference, diameter, etc. of a lesion, nodule, mass or other pathology-related feature in a body part extracted / measured from a corresponding base image dataset. In addition, the information can include secondary information (e.g., malignancy, determined growth or shrinkage rate); values ​​that quantify observables (e.g., contour state (smooth or spherical), degree of calcification or fat deposition, or degree of firmness). In addition, the information included in the medical finding can be a cut or clipped form of a (reference) image dataset of a pathology-related area (i.e., a medical finding can also include image data), or can be a segmented form in the sense of a lesion form. In addition, the information can be semantic information that describes the medical finding information in words. According to one embodiment, the set of medical findings included in the reference dataset can be a structured report in a human-readable format (e.g., list form). The location of the (reference) medical finding can usually be given in the coordinate system of the base image dataset.

[0030] There are several established methods and approaches for extracting (automatically and / or semi-automatically) one or more of the above-mentioned medical findings from image data sets. These methods may, for example, rely on traditional image processing techniques, such as image segmentation, pattern or feature recognition, etc. and / or may employ machine learning.

[0031] In case the reference dataset already includes the medical findings, this information no longer needs to be recalculated when comparing the uploaded image dataset with the corresponding reference dataset. As a result, the calculation time can be reduced. At the same time, this enables all relevant information to be stored in compressed form for subsequent reading. This significantly reduces the amount of data that needs to be exchanged and stored compared to the (raw) medical image data. Furthermore, it is also easier to anonymize the reference data and thus store this information in an externally accessible database or cloud.

[0032] According to one embodiment, it is also conceivable that the biometric data itself includes one or more of the above-mentioned medical findings, since their relative position within the body part (as the case may be) may also not change from examination to examination.

[0033] Using medical findings as biometric data essentially means that the same information can be used twice, further increasing the effectiveness of the method and reducing the amount of data required.

[0034] According to one embodiment, the step of comparing may include transforming the position of the reference medical finding into the coordinate system of the extracted medical finding or vice versa using registration (or in particular the calculated transformation function).

[0035] Coordinate transformation of the location of the findings to corresponding other coordinate systems may also be specified as a step of aligning or co-aligning the extracted medical findings and the reference medical findings using registration (or in particular the calculated transformation function). The step of aligning helps to identify corresponding findings, thereby improving subsequent readings.

[0036] According to one embodiment, the step of comparing may comprise matching (at least a portion of) the extracted medical finding with (at least a portion of) the reference medical finding based on the steps of registration and / or transformation function and / or alignment.

[0037] The matching can be considered as a step of identifying pairs of related findings in the extracted medical findings and the reference medical findings based on the registration. In an example embodiment, for each pair of a finding in the reference dataset and a finding extracted from the received image dataset, the probability that the two findings describe the same pathology or medical observation can be calculated, for example, taking into account the proximity of the transformed (aligned) positions of the findings, whether they are of the same type, and how similar other parameters are.

[0038] According to one embodiment, the step of comparing may include quantifying changes from the extracted medical findings to the reference medical findings based on the step of comparing and / or the step of aligning and / or the step of matching.

[0039] Automatic determination of changes in medical findings makes it easier for users to track the current status of a case, disease progression, or treatment response.

[0040] According to one embodiment, the reference biometric data and the biometric data respectively comprise one or more anatomical landmarks (preferably a set of anatomical landmarks) and / or a surface mesh of one or more anatomical parts comprised in the body part.

[0041] Anatomical landmarks may be selected, for example, for the chest region, from the group consisting of: aortic root and arch, arterial bifurcations (brachiocephalic branches, bifurcations of carotid or subclavian arteries, celiac trunk, renal bifurcation), nasal bifurcation, left and right kidney tops, left and right lung tops, liver middle and top, pancreas, sternal tip, vertebral bodies. Surface meshes may be selected from the group consisting of: lung lobes (upper and lower left, upper, middle and lower right), heart, aorta, vertebrae.

[0042] Anatomical landmarks or surface meshes are universal for body parts and / or anatomical parts. The use of these data points enables any information associated with them to be compared between different examinations, treatment stages, imaging parameters, imaging modalities, and patients (if necessary). In addition, there are many mature and standardized methods for automatically extracting anatomical landmarks and / or surface meshes from medical image data, which further improves the interchangeability of the results. In addition, anatomical landmarks and surface meshes are ideally suited for use as anchor points for registration because they generally capture the overall anatomical configuration of the body part very well. Taking the set of anatomical landmarks listed above as an example (even just a subset of them), this gives a comprehensive picture of the overall displacement and deformation from one image dataset of a patient to another. It is worth noting that biometric data can combine different kinds of biometric information. For example, biometric data can include a set of anatomical landmarks associated with the chest region and one or more surface meshes for the heart and a segmentation mask associated with the lungs. Depending on the current situation, this combination of "orthogonal" information can provide a very good image of deformation and displacement in subsequent image datasets.

[0043] According to one embodiment, the step of retrieving comprises extracting a data identifier from the received image dataset and using the data identifier to query a reference dataset in a database.

[0044] In other words, the medical image dataset and the reference dataset may be interconnected via a structured association in the form of a data identifier, such that each medical image dataset is unambiguously associated with one or more corresponding reference datasets. The data identifier may be a (unique) electronic data identifier or any other suitable electronic tag. In particular, the data identifier may include a patient's ID or accession number. The data identifier may also include a hash of the aforementioned data and / or any combination thereof. The data identifier serves as an association between the image dataset and the reference dataset. According to one embodiment, all data relating to a particular patient may carry the same data identifier. Thus, an association between the received image dataset and one or more reference datasets may be established based on comparing or matching the data identifier. In practice, for example, the unique identifier may be encoded in the header of the medical image dataset. According to one embodiment, the data identifier is anonymous, so that personal patient information (e.g., patient name) cannot be derived therefrom.

[0045] The use of data identifiers is beneficial for quickly querying the database to obtain the correct reference dataset corresponding to the received image dataset. This therefore improves the integration of the method into existing workflows and ensures that all relevant reference datasets can be found.

[0046] According to one embodiment, the retrieving step includes: retrieving one or more candidate reference data sets from a database; providing one or more candidate reference data sets to a user so as to select at least one user-selected reference data set from the candidate reference data; receiving the user selection; and using at least one reference data set selected by the user as a reference data set.

[0047] In some cases, there may be more than one reference dataset for an image dataset in the database. For example, the available reference datasets may relate to image data acquired at different examination times and / or with different imaging modalities. In this regard, one option is to use all available reference datasets for subsequent alignment and comparison. Another option is to automatically select one or more most likely reference datasets (based on one or more selection criteria - see below). Yet another option is an interactive selection according to the above embodiment, in which candidate reference datasets are presented to the user (the user who first submitted the image dataset). In doing so, the method can be particularly adapted to the needs of the user, as he or she can select which of the available reference datasets constitute the most interesting reference datasets for comparison.

[0048] According to one embodiment, the step of retrieving comprises: setting (or providing) one or more selection criteria for retrieving reference data and retrieving the reference data based on the selection criteria. Therefore, the selection criteria are preferably based on at least one of the following: an indication of the imaging modality used to acquire the received image data and / or the reference data set; a time window between the first time point and the second time point; an indication of the patient's identity (e.g., in the form of the above-mentioned data identifier); an indication of whether the received image data set needs to be compared with a previous examination or a subsequent examination; an indication of one or more previous selections of (the user's) reference data sets; and / or any combination of the above. In this context, the indication may relate to an appropriate data identifier, such as a patient ID or an electronic tag indicating the imaging modality used, etc.

[0049] According to one embodiment, the selection criteria may be interactively set by a user. Alternatively, the selection criteria may be semi-automatically set (e.g., according to initialization with default values), or may be automatically set for a user who needs more help. The time window may be defined as a maximum time period between a first time and a second time. A previous selection may relate to a previous use or a previous user selection of a reference data set for comparison. Information for applying the selection criteria may be recorded in a reference data set and may be extracted from the reference data set. Additional information fragments may be included in the image data set and may be extracted from the reference data set. For example, the information may be attached as metadata, for example in the form of a header, to a header of a medical image data set or a reference data set. Therefore, retrieving a reference data set based on the selection criteria may include accessing and / or reading and / or processing the image data set and the reference data set to extract information corresponding to one or more of the selection criteria.

[0050] Selection criteria can be thought of as filters used to narrow the search for appropriate reference data sets. Therefore, applying selection criteria means that more meaningful results can be produced in a shorter time.

[0051] According to one embodiment, the method further comprises the steps of: using the extracted biometric data to compile (or in other words, generate) a further reference data set; and forwarding the further reference data set to a database for storage in the database. Furthermore, the method may comprise storing the further reference data set in the database.

[0052] According to one embodiment, the DICOM format is used to compile additional reference data sets. According to another embodiment, the reference data sets retrieved from the database are also formatted according to the DICOM format. DICOM (Digital Imaging and Communications in Medicine) is an open standard for the communication and management of medical imaging information and related data in medical informatics. DICOM can be used to store and transmit medical images and related information, thereby realizing the integration of medical imaging equipment (e.g., scanners, servers, workstations, printers, network hardware) and picture archiving and communication systems (PACS). DICOM is widely adopted by clinical groups, hospitals and smaller applications such as doctors' clinics or institutions. DICOM data objects include several attributes, including items such as the patient's name, ID, etc., and also include special attributes containing image pixel data and metadata extracted from the image data. The latter can be used for biometric data and / or medical discovery according to the present invention.

[0053] By including data extracted from the received medical image dataset into the database together with the already existing reference dataset, the shared knowledge included in the system can be enhanced and the comparison of subsequent medical image data can be made more efficient for subsequent queries. The use of the DICOM data format improves the compatibility of the method with existing clinical workflows and equipment.

[0054] According to one embodiment, the database is configured as a cloud platform.

[0055] The advantage of implementing the database as a cloud platform is that individual users can more easily access the database from various sites. This will have the benefit of promoting the availability and accumulation of knowledge databases.

[0056] According to one embodiment, the database is part of a Picture Archiving and Communication System (PACS) - which may be implemented as cloud storage or as local or extended storage.

[0057] The benefit of relying on a PACS system for storing reference data sets is that existing infrastructure can be used, which can facilitate implementation and reduce costs.

[0058] According to one embodiment, the reference data set is anonymized, or in other words, does not include any personal information about the underlying patients.

[0059] "Anonymous" may mean that the reference data set does not reveal or include any personal information that can directly identify the patient (i.e., the patient's name, address, photo, etc.). Instead, the data can be traceable through a unique but anonymous identifier (e.g., an identification number). According to one embodiment, the method may also include the step of anonymizing the medical findings and / or biometric data before archiving the information in the database. The anonymization step may include filtering out any personal information that can directly identify the patient.

[0060] By anonymizing the data, information included in the medical findings and / or biometric data that can be traced back to the corresponding patient can be safely excluded, which is very useful when dealing with personal data privacy issues.

[0061] According to one embodiment, registering the biometric subsequent data with the reference biometric data is based on a rigid and / or affine and / or non-rigid registration and / or any combination thereof.

[0062] In this context, rigid registration may include registration where the coordinates of data points in one dataset (e.g., extracted biometric data) are rotated and translated in order to register them with another dataset (e.g., reference biometric data). In this context, affine registration may include registration where the coordinates of data points in one dataset are subjected to rotation, translation, scaling, and / or shearing in order to register them with corresponding data points in another dataset. Thus, a rigid registration that includes only translation and rotation may be considered a specific type of affine registration. In contrast, non-rigid registration allows for "elastic" transformations, and thus may provide a different displacement for each data point of the datasets to be registered, and may, for example, use non-linear transformations in which the coordinates of data points in one dataset are subjected to flexible deformations in order to register the dataset with another dataset. In some cases, a (non-linear) transformation may be defined using a vector field (e.g., a warp field or other field or function) to define a separate displacement for each data point in a three-dimensional dataset. For more details on image registration, please refer to US 2011 / 0081066 and US 2012 / 0235679, the contents of which are incorporated herein by reference.

[0063] According to one embodiment, the method further comprises the steps of: generating (calculating) a result based on the step of comparing, and forwarding the result to a user and / or archiving the result in a database. Thus, the result is preferably provided in the form of: a structured medical report; one or more trend graphs, such as depicting changes in findings in one or more subsequent examinations; and / or a visualization visible to a user.

[0064] According to another embodiment, the method further comprises the steps of: calculating (generating) results based on the changes in the quantified medical findings; and forwarding the results to a user and / or storing the results in a database. Thus, the results are preferably in the form of one or more trend graphs showing the changes in the quantified medical findings; a structured medical report including an indication of the changes in the quantified medical findings, preferably in the form of one or more DICOM SR objects; and / or a visualization visible to a user that visualizes the changes in the quantified medical findings.

[0065] The generation of results according to the above-described embodiments makes the information extracted during the subsequent reading assisted by the computer according to the invention easily accessible to the user, thus improving the usability of the method. In addition, the results can be more easily archived for later use.

[0066] According to one embodiment, the visualization may be in the form of a supplemental image that includes rendering the received image dataset in a manner that highlights variations of the medical findings.

[0067] By providing the user with rendering the submitted image data set with highlighted changes, the user can immediately infer what changes have occurred and where these changes have occurred. This helps to guide image reading, thereby increasing the usability of the method and providing improved assistance to the user in obtaining a medical diagnosis. Rendering can rely on known rendering processes, such as ray casting, ray tracing, texture rendering, image projection, etc. In this context, the term "highlighting" can mean that the pathological changes are visually enhanced in brightness, color and / or intensity. In addition or alternatively, symbols can be used to highlight the identified pathological changes. Highlighting can be achieved based on information about the identified changes such as the location, volume and amount of the changes. Highlighting can also mean using a heat map, where, for example, the amount of change is color-coded. For example, a different color can be assigned to a contraction nodule than a growth nodule. A heat map can be visualized as an overlay image on the received image data.

[0068] According to one embodiment, the medical image dataset is received from the first site at a central processing system, which is remote from the first site (and is preferably a cloud-based platform), wherein the steps of retrieving, processing, executing and comparing are performed at the central processing system. According to another embodiment, the central processing system also includes a database.

[0069] In other words, the processing of medical image datasets for comparing them with previous or subsequent studies is outsourced from the first site to an external central facility. The advantage of this is that the first site does not need to provide the hardware and software for the processing itself, but can rely on cost-saving external resources. In addition, it is easy to integrate knowledge from multiple sites, for example when setting up a database. This is especially true if the second site is a cloud-based platform. In this regard, the site can be a clinical setting, an institution, a hospital, etc.

[0070] Biometric data may include dedicated data structures that characterize the geometric configuration of the anatomical part in the image data set, such as anatomical landmarks, surface meshes, etc. Such information, independent of the underlying coordinate system, can be used for alignment purposes and for calculating coordinate transformation functions that map the coordinate systems of two image data sets. However, as already mentioned, medical findings themselves are also suitable for providing anchor points for aligning two data sets with position-resolved information to each other (provided that there are enough pairs of medical findings in both data sets). Therefore, alignment can also be based on aligning medical findings and / or a combination of medical findings with biometric data. The following aspects allow the use of this concept. Where appropriate, the optional method steps introduced and explained in conjunction with the previous aspects may also be combined with the following aspects.

[0071] According to another aspect, a computer-implemented method for comparing subsequent medical image data is provided, the method comprising a plurality of steps. A first step involves receiving a medical image dataset showing a body part of a patient at a first time, followed by a step of retrieving a reference dataset associated with the image dataset from a database, wherein the reference dataset is extracted from a reference image dataset depicting the body part of the patient at a second time different from the first time, and the reference dataset comprises a plurality of location-specific reference values ​​respectively relating to locations in the reference image dataset. A further step involves extracting from the image dataset an extracted dataset comprising a plurality of location-specific extracted values, the plurality of location-specific extracted values ​​respectively relating to locations in the body part of the image dataset and at least partially corresponding to the reference dataset. A further step involves performing an alignment of the reference dataset with the extracted dataset by aligning at least a portion of the reference dataset with at least a portion of the extracted dataset. Furthermore, a further step involves comparing the reference dataset with the extracted dataset based on the alignment, in particular by quantifying changes from the reference dataset to the extracted dataset.

[0072] The position specific values ​​are respectively related to position resolved information extracted from the base image data information. By identifying pairs of corresponding position resolved values, the extracted data set and the reference data set can be registered with each other and a transformation function that maps the position specific data in one data set to the corresponding position specific data in the other data set can be calculated. In this way, the two data sets, i.e. the reference data set and the extracted data set, can be compared and any changes between the extracted data and the reference data retrieved from the database can be optionally quantified. Thus, the reference value and the extracted value can respectively include one or medical findings (as described above) and / or biometric data (as described above).

[0073] According to another aspect, a system for comparing subsequent medical examinations is provided, the system comprising an interface unit and a computing unit. The interface unit is configured to receive a medical image dataset from a site external to the system, and to communicate with a database storing a plurality of reference datasets extracted from the medical image dataset. The computing unit is configured to receive, via the interface unit, an image dataset depicting a body part of a patient at a first time, and to retrieve at least one reference dataset associated with the image dataset from the database. Thus, the reference dataset comprises reference biometric data extracted from a reference image dataset depicting a body part of the patient at a second time different from the first time, wherein the reference biometric data defines a location of an anatomical part of the body part in the reference image dataset. The computing unit is further configured to process the image dataset so as to extract biometric data from the image dataset, the biometric data defining a location of an anatomical part of the body part in the image dataset and at least partially corresponding to the reference biometric data, to perform registration of the reference dataset with the image dataset based on the reference biometric data and the extracted biometric data, and to compare the image dataset with the reference dataset based on the registration.

[0074] According to one embodiment, the system is suitable for implementing the method of the present invention for comparing subsequent medical image data. The computing unit may include a retrieval unit configured to retrieve one or more reference data sets from a database of the system or an external database. To this end, the retrieval unit may be configured to query the database and retrieve one or more reference data sets. The retrieval unit may also be configured to extract a suitable data identifier from the received image data set and query the database on this basis. In addition, the computing unit may include an image processing unit that is generally configured to process the received image data set. In particular, the image processing unit may be configured to extract biometric data that at least partially corresponds to the reference biometric data included in the reference data set from the medical image data set. To this end, the image processing unit may be configured to read and analyze the reference biometric data and process the medical image data set on this basis. The computing unit may also include a registration unit configured to register the reference biometric data with the biometric data using at least one image registration. The computing unit may also include a result generation and visualization unit configured to compare the medical image data set with the reference data set and generate a result for display to a user and / or archiving in an appropriate database. The results may be in the form of a visualization (to the user), a structured report or indication, and / or one or more trend graphs highlighting changes identified in the comparison.

[0075] The computing unit can be implemented as a data processing system or a part of a data processing system. Such a data processing system can, for example, include a cloud computing system, a computer network, a computer, a tablet computer, a smart phone, etc. The computing unit can include hardware and / or software. The hardware can include, for example, one or more processors, one or more memories and combinations thereof. One or more memories can store instructions for performing method steps according to the present invention. The hardware can be configured and / or operated by software. Typically, all units, subunits or modules can, at least temporarily, for example, exchange data with each other via a network connection or a corresponding interface. Therefore, each unit can be positioned to be separated from each other.

[0076] The interface unit may comprise an interface for data exchange with a local server or with a central network server via an Internet connection to receive a reference data set and / or a medical image data set. The interface unit may also be adapted to interface with one or more users of the system, for example by displaying the processing results of the computing unit to the user (e.g. in a graphical user interface), selecting a reference data set and / or by adjusting parameters for image processing or visualization.

[0077] According to an embodiment, the system is a cloud-based platform.

[0078] According to an embodiment, the system further comprises a database which may be implemented in the form of cloud storage.

[0079] According to another aspect, the invention relates to a computer program product comprising a program element which, when loaded into a memory of a computing unit of a system for comparing subsequent medical examination results, directs the computing unit to execute the steps according to the above method.

[0080] According to another aspect, the invention relates to a computer-readable medium having stored thereon a program element which, when executed by a computing unit of a system for comparing subsequent medical examination results, can be read and executed by said computing unit to perform the steps of the method of the invention.

[0081] Implementing the invention by means of a computer program product and / or a computer-readable medium has the advantage that already existing provisioning systems can be easily adapted by means of a software update to operate according to the proposal of the invention.

[0082] A computer program product may be, for example, a computer program or a computer program comprising, next to such a computer program, another element. The other element may be: hardware, such as a storage device on which the computer program is stored, a hardware key for using the computer program, etc.; and / or software, such as documentation or a software key for using the computer program. The computer program product may also include development materials, a runtime system and / or a database or library. The computer program product may be distributed among several computer instances. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] The characteristics, features and advantages of the invention described above and their implementation methods become clearer and easier to understand according to the following description and embodiments, which will be described in detail with reference to the accompanying drawings. The following description does not limit the invention to the included embodiments. In different drawings, the same components or parts may be marked with the same reference numerals. Generally, these drawings are not drawn to scale. In the following:

[0084] Figure 1 depicts a distributed environment for comparing subsequent medical examinations according to embodiments;

[0085] Figure 2 depicts a distributed environment for comparing subsequent medical examinations according to embodiments;

[0086] Figure 3 depicts a block diagram corresponding to a computer-implemented method for comparing subsequent medical examinations, according to an embodiment;

[0087] Figure 4schematically illustrates the data flow between system components in conjunction with the main method steps according to an embodiment; and

[0088] Figure 5 Shown according to Figure 3 The order of several optional sub-steps of the method. DETAILED DESCRIPTION

[0089] Figure 1 A distributed environment 1 for identifying and quantifying changes in subsequent medical examinations according to an embodiment of the present invention is depicted. The environment 1 is suitable for performing a method according to one or more embodiments, for example as described in reference Figure 3 and Figure 4 The method is further described.

[0090] The system 1 includes one or more clients 10 and a central processing system 100. The clients 10 and the central processing system 100 are connected via a network 200. The central processing system 100 is generally configured to control and execute comparisons of data obtained from subsequent medical image data in a distributed environment 1. The clients 10 may reside on one or more different local sites. The local sites may, for example, be related to a clinical or medical environment such as a hospital or hospital group, a laboratory, a medical imaging center, a clinic or an institution. In this example, four clients 10 are shown for illustration. However, the distributed environment 1 is not limited to this number. Generally, the distributed environment 1 may include any number of clients 10. In particular, the distributed environment 1 may also include only one client 10.

[0091] The central processing system 100 can be generally considered as a server. Thus, the central processing system 100 can be, for example, a web server. In addition, the central processing system 100 can be, for example, Figure 2 The central processing system 100 may be implemented using any suitable computing device and / or software. The central processing system 100 may have a processor 20 , a database 30 , and an interface unit 40 .

[0092] The network 200 may be implemented as a local area network (LAN), such as an intranet, or a wide area network (WAN) or the Internet. Furthermore, the network 200 may include a combination of different network examples.

[0093] The client 10 may include: one or more imaging systems 11 for acquiring image datasets IM, a user interface 12, a processing system 13, and a database 14 for storing the medical image datasets and / or reference datasets obtained therefrom.

[0094] The medical imaging system 11 is generally configured to acquire, process and store medical image data sets. Therefore, the medical imaging system 11 may include one or more medical imaging modalities, such as a computed tomography system, a magnetic resonance system, an angiography (or C-arm X-ray) system, a positron emission tomography system, a mammography system, a system for acquiring digital pathology images, etc. In addition, the medical image data sets obtained by the medical imaging system 11 can be acquired from a memory, such as from a database 14. In addition, the medical imaging system 11 may include an archive / review station (not shown) for storing the image data set IM. Such an archive / review station may be implemented as cloud storage, or additionally or alternatively, as local or extended storage in the form of, for example, a PACS (Picture Archiving and Communication System).

[0095] The image datasets (image dataset IM and reference image dataset) may be three-dimensional datasets acquired, for example, using a computed tomography system or a magnetic resonance imaging system. Here, the image information is encoded as a three-dimensional array of k×m×n voxels. In addition, the image dataset may be related to a two-dimensional dataset, such as a two-dimensional dataset acquired using an X-ray device, in which the image information is encoded into m×n pixels. In addition, the image dataset may include a three-dimensional medical image or a time lapse of a two-dimensional medical image. When comparing subsequent medical examinations, data extracted from the medical image dataset are compared with data extracted from the reference medical image dataset. Typically, the image dataset IM and the reference medical image dataset show the same body part of the patient, although the same body part at different time points and may be acquired using different imaging modalities. Typically, the depicted body parts will include various anatomical parts and organs. Considering the chest region, the image dataset may, for example, depict a lung lobe, a rib, a heart, a lymph node, etc. Although one of the image datasets (the received image dataset IM itself or the reference image dataset) is obtained in an earlier examination at a first time, the corresponding other image dataset is related to a subsequent examination at a subsequent stage at a second time. The second time may be hours, days, weeks, months or years after the first time. In addition, there may be intermediate scans or processes between the first time and the second time. In one embodiment, the medical image data set has been acquired using the same or similar settings and parameters. Similar settings and parameters may include, for example, the same medical imaging modality 11, similar doses (if available), the same phase timing, x-ray source voltage, contrast agent, MRI protocol, etc. Alternatively, although the image data set depicts the same body part, different imaging modalities 11 and / or different settings of the imaging modality 11 may have been used to acquire the image data set.

[0096] like Figure 1As shown, the user interface 12 may include a display unit 12a and an input unit 12b. The user interface 12 may be implemented by a mobile device such as a smart phone or a tablet computer. In addition, the user interface 12 may be implemented as a workstation in the form of a desktop PC or a laptop computer. The input unit 12b may be integrated in the display unit 12a, for example, in the form of a touch screen. Alternatively or in addition, the input unit may include a keyboard 12b, a mouse or a digital pen and any combination thereof. The display unit 12a is configured to display the image data set IM and / or the result of the image processing performed by the central processing system 100 / processor 20.

[0097] The client 10 also includes a client processing system 13, which is configured to execute at least one software component for serving the display unit 12a and the input unit 12b, so as to provide a graphical user interface for: allowing the user to select an image data set IM to be uploaded to the central processing system 100 and / or to select from the available reference data sets RDS in the graphical user interface. In addition, the client processing system 13 can be configured to communicate with the medical imaging system 11 and / or the central processing system 100, for example, to upload the image data set IM, receive candidate reference data sets RDS for user review and / or receive analysis results provided by the central processing system 100. The user can activate the software component via the user interface 12. He can obtain the software component by downloading it from an Internet application store, for example. According to an example, the software component can also be a client-server computer program in the form of a web application running in a web browser. The client processing system 13 can be a general-purpose processor, a central processing unit, a control processor, a graphics processing unit, a digital signal processor, a three-dimensional rendering processor, an image processor, an application-specific integrated circuit, a field programmable gate array, a digital circuit, an analog circuit, a combination thereof, or other currently known devices for processing image data.

[0098] The database 14 can generally be configured to store a reference data set RDS. The reference data set RDS is obtained from the corresponding image data set IM by applying one or more image processing techniques. Therefore, the reference data set RDS constitutes a compressed or streamlined set of information, which enables quantitative comparison of previous examinations and / or subsequent examinations of the patient even without access to the underlying complete (original) image data. The database 14 can be configured so that it can be queried by the central processing system 100 to retrieve the reference data set RDS. It is worth noting that the database 14 is optional for the distributed environment 1 in the sense that the role of the database 14 can be taken over by the central database 30 included in the central processing system 100. In addition, the database 14 can be implemented as part of an archive / review station included in the medical imaging system 11. The database 14 can be implemented as local or extended storage.

[0099] Specifically, the reference data set RDS may include two types of information. On the one hand, the reference data set RDS may include medical findings EMF, RMF extracted from the basic image data set. On the other hand, the reference data set RDS may include biometric data EBD, RBD. Medical findings EMF, RMF may generally involve physiological observations extracted from image data by image processing, which are related to the conclusion of a medical diagnosis. Examples of such information include the volume, cross-section, perimeter, diameter, etc. of lesions, nodules, masses or other pathology-related features in body parts extracted / measured from the corresponding basic examinations. In addition, the information may include more advanced information, such as the degree of malignancy, known growth or shrinkage rates, quantitative descriptive observables (e.g., contour state (e.g., smooth or spherical), degree of calcification or fat deposition, or degree of firmness, etc.). In addition, the information may be in the form of a cut or clipped (i.e., medical findings EMF, RMF may also include image data) from a (reference) image data set for a pathology-related region. In addition, the information may be semantic information that describes physiological information in words. Physiological information may be automatically extracted from a medical image data set using a (computer-implemented) method known in principle. However, it is also conceivable to use information that has been manually annotated by one or more users, for example in the form of an already existing structured medical report.

[0100] In order to make the medical findings EMF, RMF comparable between different image data sets, the medical findings EMF, RMF included in each image data set need to match each other. In other words, it is necessary to identify the pairs of corresponding findings EMF, RMF. In addition, it should also be determined whether the findings are new or have disappeared. As already explained, it is usually not feasible to do it by directly comparing pixel and / or voxel values, because the coordinate systems of subsequent image data sets are usually not well aligned. This is because imaging parameters such as alignment of imaging windows or patients may vary from examination to examination, and further deviations may be caused due to inevitable anatomical changes and changes. Such anatomical changes may be related to tissue deformation caused by respiratory motion or weight gain or loss, for example. Therefore, a universal reference point or anchor point is needed, which can be used to calculate the coordinate transformation between each coordinate system using a registration technique (see below). These reference points should be easy to extract from a medical image data set and have sufficient versatility in the sense of being applicable to a variety of image data sets that can be derived from various imaging modalities. In this regard, anatomical landmarks and surface grids of anatomical parts and organs have been proven to be very useful. Other options in this regard include segmentation masks or contours of anatomical parts and organs, or even the medical findings EMF, RMF introduced above. Such data as a whole can be envisioned as a biometric fingerprint of the body part depicted in the image dataset and likewise included in the reference dataset RDS as biometric data RBD. According to one embodiment, the reference dataset RDS can be in the form of a structured report in a human-readable format in which the location-specific medical findings RMF are listed together with the biometric data RBD.

[0101] The comparison of subsequent examinations using the reference data set RDS is performed by a processor 20 in the central processing system 100. The processor 20 may include subunits 21 to 24, which are configured to process the received image data set IM, retrieve corresponding reference data sets RDS related to previous or subsequent examinations about the received image data set IM, and quantify pathology-related changes between data extracted from the image data set IM and the reference data set RDS. The processor 20 may be a general-purpose processor, a central processing unit, a control processor, a graphics processing unit, a digital signal processor, a three-dimensional rendering processor, an image processor, an application-specific integrated circuit, a field programmable gate array, a digital circuit, an analog circuit, a combination thereof, or other devices known today for processing image data. The processor 20 is a single device, or may include multiple devices operating in series, in parallel, or separately. The processor 20 may include a real or virtual group of computers, such as a so-called "cluster" or "cloud". Such a server system may be a central server (e.g., a cloud server), or a local server (e.g., a server located at a hospital or a radiation site). The processor 20 may be the main processor of a computer (e.g., a laptop or desktop computer), or may be a processor used to handle certain tasks in a larger system (e.g., an imaging system or server). The processor 20 is configured by instructions, design, hardware, and / or software to perform the steps discussed herein. In addition, the processor 20 may include a memory, such as a RAM for temporarily storing uploaded image datasets IM and / or reference datasets for further processing.

[0102] The central processing system 100 is configured to receive image datasets IM from one or more of the clients 10. To this end, the central processing system 100 may include a suitable interface unit 40 for communicating with the clients 10 via the network 200.

[0103] The subunit 21 is a retrieval unit or module. The subunit 21 is configured to retrieve one or more reference data sets RDS associated (or can be associated) with the image data set IM received from the database 14, 30. The database in question can be related to the database 13 at the corresponding client 10 or to the database 30 of the central processing system. The subunit 21 can be configured to access and read the image data set IM to extract a data identifier that can be used to query the corresponding reference data set RDS from the database 14 or 30. The data identifier can be related to the patient's ID or accession number or file number, for example. In addition, the subunit 21 can be configured to extract metadata from the image data set IM that can be used to narrow the search scope. Such metadata can be related to the time point of acquiring the image data set IM, the type of the image data set IM, the type of the medical imaging modality 11 used to acquire, the body part depicted in the image data set IM, etc. This information can be retrieved from the header of the image data set IM and / or directly extracted from the image data set IM by image processing. Alternatively or in addition, the metadata can be provided by the user when uploading the image data set IM to the central processing system 100. For example, metadata may be input via the user interface 12 at the time of uploading and forwarded to the central processing system 100. In addition, the subunit 21 may be configured to apply one or more selection criteria when querying the reference data set RDS from the database 14 or 30. The selection criteria may be related to the metadata introduced above. For example, one selection criterion may involve searching the database 14 or 30 specifically for reference data sets RDS extracted from image data acquired using the same type of imaging modality 11 as the uploaded image data set IM. In addition, the search for the reference data set RDS may be limited to data related to the same body part as the uploaded image data set IM. For another example, only reference data sets RDS acquired within a set time window of the acquisition time of the image data set IM are retrieved. Another selection criterion may involve the following question: whether to seek reference data from a previous examination or a subsequent examination. The selection criteria may be set by the subunit 21 or interactively set by the user via the user interface 12. The subunit 21 may also be configured to provide the retrieved data set RDS to the user for review, approval and / or selection, for example, by forwarding one or more candidate reference data sets RDS to the user interface 12. If no reference dataset RDS matches the selection criteria and / or no reference dataset RDS is found at all, the subunit 21 may be configured to inform the user accordingly via the user interface 12 .

[0104] The subunit 22 is an image processing unit or module. The subunit 22 is configured to extract information matching the information included in the reference dataset RDS from the uploaded medical image dataset IM so as to enable a comparison between the medical image dataset IM and one or more corresponding reference datasets RDS. Therefore, the subunit 22 is configured to extract biometric data EBD corresponding to the reference biometric data RBD included in the reference dataset RDS from the image dataset IM. It is worth noting that "correspondence" does not require a one-to-one correspondence between the extracted biometric data EBD and the reference biometric data RBD. Taking anatomical landmarks as an example of biometric data, it is sufficient that at least some anatomical landmarks extracted from the medical image dataset IM can be mapped to anatomical landmarks included in the corresponding reference dataset RDS to calculate the coordinate transformation function by the registration unit 23. In order to ensure that the biometric data EBD, RBD fully correspond, the subunit 22 can be configured to analyze the reference biometric data RBD included in the reference dataset RDS and process the image dataset IM accordingly. In addition to extracting appropriate biometric data EBD from the image dataset IM, the subunit 22 may also be configured to extract one or more medical findings EMF from the image dataset IM. Thus, the medical findings EMF may relate to the same kind of information described above in relation to the reference biometric data RBD included in the reference dataset RDS. In order to extract the biometric data EBD and the physiological findings EMF from the image dataset IM, the subunit 22 may be configured to implement the same or similar methods as used to extract this information from the reference medical image dataset when generating the reference dataset RDS.

[0105] Subunit 23 is a registration unit or module. Subunit 23 is configured to perform registration of extracted biometric data EBD and reference biometric data RBD. To this end, subunit 23 can be configured to determine which parts of the extracted biometric data EBD and the reference biometric data RBD correspond and perform registration for the corresponding parts. Taking anatomical landmarks and surface meshes as an example again, subunit 23 can determine those anatomical landmarks and surface meshes included in both the extracted biometric data EBD and the reference biometric data RBD, and use them for registration. Once the corresponding parts of the biometric data and the reference biometric data have been matched (for example, by identifying data pairs in the extracted biometric data EBD and the reference biometric data RBD), subunit 23 can calculate a coordinate transformation, which essentially converts the biometric data EBD into the reference biometric data RBD, or converts the reference biometric data RBD into the biometric data EBD, or in other words, this minimizes the spatial distance between the corresponding parts of the biometric data EBD and the reference biometric data RBD. By construction, such a transformation is equal to any systematic (non-pathological) changes, deformations and other differences in the metrics of the base image data set. Since these changes are usually local and non-isotropic, the transformation function will also be local and non-isotropic. Therefore, the calculation result provided by the subunit 23 can be in the form of a two-dimensional or three-dimensional transformation matrix or deformation field. In order to perform image registration, the subunit 23 can apply one or more image registration techniques including rigid image registration, affine image registration, non-rigid image registration and any combination thereof. In order to improve the result of the registration, the subunit 23 can optionally be configured to mathematically fit the calculation result to one or more motion models for soft tissue deformation. In addition, the subunit 23 can be configured to, for example, divide or cluster the (reference) biometric data EBD, RBD into multiple segments or clusters according to the anatomical part or organ included in the body part. To this end, the subunit 23 can be configured to run one or more segmentation filters, such as a lung segmentation filter, a bone segmentation filter, a liver segmentation filter, etc. Alternatively, the subunit 23 may simply cluster the biometric data EBD, RBD according to the semantic description of the individual data included in the reference biometric data RBD and the extracted biometric data EBD. The subunit 23 may also be configured to generate a respective deformation field for each cluster using a respective image registration and motion model (possibly optimized for the organ or anatomical part included in the corresponding segment or cluster).

[0106] Subunit 24 is a result generation and visualization module or unit. Subunit 24 is configured to compare the received image data set IM with the reference data set RDS using the result provided by subunit 23. In particular, subunit 24 can be configured to apply the result of the registration so that the corresponding medical findings EMF, RMF extracted from the received image data set IM and included in the reference data set RDS match. To this end, subunit 24 can be configured to map the extracted medical findings EMF to the reference medical findings RMF (or map the reference medical findings RMF to the extracted medical findings EMF). Another expression of "mapping" is to align (or co-align) the extracted medical findings EMF and the reference medical findings RMF. This can be accomplished by subunit 24 by transforming the reference medical findings RMF into the coordinate system of the extracted medical findings EMF or transforming the extracted medical findings EMF into the coordinate system of the reference medical findings RMF by applying the transformation function output by subunit 23. Subunit 24 can also be configured to identify the corresponding medical findings EMF, RMF on this basis. This may mean identifying pairs of corresponding medical findings in the extracted medical findings EMF and the reference medical findings RMF. This may also include identifying any medical findings that do not have a corresponding part in the extracted medical findings EMF or the reference medical findings RMF, because this indicates that there are newly formed medical findings or disappeared medical findings. The subunit 24 may also be configured to identify and / or quantify changes in medical findings EMF, RMF based on mapping / alignment, and generate results R on this basis. In addition, the subunit 24 may also be configured to transform or convert these results R into suitable representations to display them to the user. Suitable representations may be in the form of auxiliary images in which changes in medical findings are visually encoded. This may mean that changes are enhanced in visualization. The subunit 24 may, for example, be configured to run or execute an algorithm for rendering a semi-transparent overlay image based on quantified / identified medical findings changes, which is superimposed on the corresponding rendered image of the image data set IM. In addition, the subunit 24 may be configured to highlight changes in the form of symbols in the image data set IM.

[0107] The designation of different subunits 21 to 24 will be explained by way of example and not limitation. Therefore, subunits 21 to 24 can be integrated to form a single unit, or can be implemented by a computer code segment configured to execute the corresponding method steps running on a processor of the processing system 20, etc. The same is true for the client processing system 13. Each subunit 21 to 24 and the client processing system 13 can be separately connected to other subunits and / or other components of the system 1 that need data exchange to perform method steps. For example, subunit 21 can be connected to a database 30 to retrieve a reference data set RDS, and connected to the client processing system 13 and / or the user interface 12 to forward one or more candidate reference data sets RDS to the user for review. The processing system 20 and the interface computing unit 13 together can constitute a computing unit as described above. It is worth noting that the physical layout of the computing unit, that is, the physical distribution of the interface computing unit 13 and subunits 21 to 24, is arbitrary in principle. For example, subunit 24 (or its individual elements or specific algorithm sequences) can be located at the client 10 in the same manner. This is also true for other subunits 21 to 23. Specifically, the processing system 20 may also be integrated into the user interface 10 .

[0108] As already mentioned, the processing system 20 may alternatively be implemented as a server system, such as a cloud server, or a local server, such as a server located at a hospital or radiology site. Figure 2 The implementation in the form of a cloud server is shown in Figure 2 A distributed system 2 for subsequent reading according to an embodiment is depicted. Figure 1 , similar reference numerals denote the same parts. Figure 2 In the illustrated embodiment, the user interface 12 may be designated as the user-facing "front end" or "client," while the central processing system 100 may be considered the "back end." Communication between the user interface 10 and the processing system 20 may occur over a network 200, for example, using the hypertext transfer protocol. The computing power of the system 2 may be distributed between the central processing system 100 and the client 10. In a "thin client" system, most of the computing power resides in the central processing system 100. In a "thick client" system, more computing power, and possibly data, resides on the client 10.

[0109] The various components of the systems 1 and 2 can be connected to each other at least temporarily for data transmission and / or exchange. The client 10 communicates with the central processing system 100 via the interface unit 40 to exchange, for example, image data sets IM, reference data sets RDS and / or processing results R. For example, the central processing system 100 can be activated based on a request, wherein the request is sent by the client 10, for example, by uploading the image data set IM to the central processing system 100. In addition, the central processing system 100 can communicate with the database 14 or 30 or other data storage device via the interface unit 40. Specifically, the connection to the database 14, 30 can include an interface compatible with the DICOM standard (Digital Imaging and Communications in Medicine) (as part of the interface unit 40), and the retrieval of the reference data set RDS can be performed by the DICOM query and retrieval application class. Similarly, the DICOM query and retrieval application class can be used to archive additional reference data sets. The interface unit 40 for data exchange can be implemented as a hardware or software interface, for example, a PCI bus, USB or FireWire. The interface unit 40 can be docked with the network 200. The network connection can be used to implement data transmission via the interface unit 40. The network connection may also be wireless, for example as a wireless LAN (WLAN or WiFi). The interface for data exchange and the components for interfacing with a user may also be considered as part of the above-mentioned interface unit.

[0110] Figure 3 The inventive method for subsequent medical image data according to an embodiment of the present invention is described. The corresponding data flow is as follows Figure 4 The method comprises several steps. The order of the steps does not necessarily correspond to the number of the steps, but may also vary between different embodiments of the invention. Optional steps or sub-steps are Figure 3 Shown in a dotted box.

[0111] In a first step M10 , an image dataset IM for which a subsequent reading is requested is received. The image dataset IM may be provided to the central processing system 100 by a user (may be a radiologist, pathologist or other physician and healthcare personnel) by uploading the image dataset IM to the central processing system 100 .

[0112] The second step M20 involves retrieving at least one reference dataset RDS corresponding to the image dataset IM from the databases 14 and / or 30. To this end, the medical image dataset IM can be read in an optional sub-step M21 to extract information from the image dataset IM, based on which the available databases 14, 30 can be queried for a suitable reference dataset RDS. This information may include, for example, a data identifier in the form of an accession number or patient ID; information indicating the patient, case and / or disease type; the type of medical image dataset (2D, 3D, MR data, CT data, etc.); the imaging modality and imaging parameters used; the time point at which the image dataset was acquired; the treatment administered to the patient, etc. This information can be read from the (DICOM) header or body of the image dataset IM. Alternatively, the user can supplement all or part of this information at the time of uploading (which makes sub-step M21 optional). In any case, the information collected in this way can be used to send a specific query to the available databases 14 and / or 30. For example, the databases 14 and 30 can be queried for all reference datasets related to the same patient as the patient of the image dataset IM. Additionally or alternatively, one or more selection criteria may be defined in optional sub-step M22, with the aid of which the search query may be further specified. Such selection criteria may include: an expected time window between the acquisition of the image dataset IM (at a first time) and the acquisition of a reference image dataset (at a second time) forming the basis of the reference dataset RDS; whether a previous examination and / or a subsequent examination is required for comparison; whether results from different or similar medical imaging modalities should be considered; information on whether and how the reference dataset is used as a reference for subsequent readings, etc. In addition to filtering the available reference datasets RDS, the selection criteria may also be used to calculate a relevance score in optional sub-step M23 and select a reference dataset based on the relevance score. For example, the relevance score may be such that an older reference dataset RDS scores lower than a newer reference dataset. The reference dataset thus retrieved may be forwarded to the client 10 as a "candidate reference dataset" in optional sub-step M24. As in Figure 5As further described in detail in , sub-step M24 can be divided into individual steps M24.1 to M24.4 as follows: in M24.1, one or more candidate reference datasets are retrieved from the database 14 and / or 30; in M24.2, one or more candidate reference datasets are provided to the user to select a user-selected reference dataset from the one or more candidate reference datasets; in M24.3, the user selection is received; and in M24.4, the user-selected reference dataset is used as the reference dataset RDS. At the client 10, the candidate reference datasets can be presented to the user in M24.2, for example, via the user interface 12, and are preferably presented to the user together with an indication of certain key information (timestamps, relevance scores, etc.). The user can then select which reference dataset(s) she or he wants to use for subsequent reading and further processing by the central processing system 100. As for the database from which the reference dataset is retrieved in step M20, this can be a local database 14 (for example, located at the site where the medical image dataset IM has been submitted) or a database 30 of the central processing system 100 (see Figure 4 ). In itself, both databases 14 and 30 or other (external) databases may also be queried in step M20. Step M20 may be performed at least partially on the client 10 side or at the central processing system 100. Where necessary, a corresponding data exchange is included in this step. Preferably, step M20 is mainly performed in the central processing system 100, wherein the actions involving the client are initiated by the central processing system 100.

[0113] Step M30 is an image processing step involving the extraction of information from the image dataset IM for subsequent registration and feature comparison with a reference dataset RDS. The set of data extracted from the image dataset IM in this way is called an extracted dataset EDS. The extracted dataset EDS corresponds at least partially to the reference dataset RDS. Step M30 can be performed in parallel or after step M20. In the latter case, in an optional sub-step M31, the features of the retrieved reference dataset RDS can be read and the image dataset IM can be processed based on the read features. Therefore, the features can be related to the type of biometric data RBD and medical findings RMF included in the reference dataset RDS. In this way, when it is determined in sub-step M31 that the reference dataset RDS includes a given set of anatomical landmarks, the image dataset IM can be carefully processed to automatically extract anatomical landmarks that best correspond to the anatomical landmarks in the reference dataset RDS. However, in particular if the biometric data relied upon are highly standardized, sub-step M31 can also be omitted. Just like the reference dataset RDS, the data extracted from the image dataset IM in step M30 may include two types of information: one is biometric information ("extracted biometric data" EBD), and the other is medical findings ("extracted medical findings" EMF). However, as described above, the extracted biometric data EBD may also include one or more medical findings EMF. Preferably, step M30 is performed in the central processing system 100.

[0114] Once the image processing of step M30 is completed, the data EDS extracted from the image dataset IM is in principle in a good state for comparing it with the reference dataset RDS. In order to quantify the changes from the reference dataset RDS to the extracted dataset EDS, the structured results included respectively must be mapped. In other words, it is necessary to determine which (some) of the medical findings RMF, EMF correspond to each other, which are new and which have disappeared. If the positions of the medical findings RMF, EMF are defined relative to the same coordinate system, the mapping will be equivalent to a simple superposition. However, as already explained, the underlying coordinate systems are far from the same. Therefore, it is necessary to calculate a transformation function that is able to transform the corresponding reference dataset RDS into the coordinate system of the extracted dataset EDS, or to transform the extracted dataset EDS into the coordinate system of the corresponding reference dataset RDS. This is accomplished by performing a registration REG in step M40. In step M40, at least a portion of the corresponding reference dataset RDS is registered with at least a portion of the extracted dataset EDS. In essence, this can include identifying corresponding data points in the corresponding reference dataset RDS and the extracted dataset EDS in an optional sub-step M41. Having identified such corresponding data points, the local offset between these corresponding points can be calculated, which provides an indication of the local offset from the reference data set RDS to the extracted data set EDS in the coordinate system. Performing this operation on a plurality of corresponding data points that have been sufficiently distributed in the base image volume provides a good indication of the displacement and deformation between the corresponding reference data set RDS and the extracted data set EDS. In order to appropriately aggregate these individual contributions into a coherent two-dimensional or three-dimensional transformation function or deformation field, various registration techniques can be used. These techniques may include rigid registration, affine registration, non-rigid registration, non-affine registration and any combination thereof.

[0115] The registration attempts to find a transformation T that minimizes the spatial distance (depending on the implementation: Euclidean norm for points and surface distance metric for meshes) of the landmark locations of the two datasets. Depending on the implementation, the task can be formulated as follows: Assume that N (generalized) landmarks are found in both datasets, and let x n and represents their spatial coordinates, and the transformation consists of M parameters p1,…,p M If yes, the optimization task is:

[0116]

[0117] where w n Represents the weight parameter, and dist nDenotes the distance metric used (eg, Euclidean distance, surface distance). In a preferred embodiment, the choice of T is a translation (simple shift), rigid (shift and rotation), affine (rigid + shear), or deformable transformation.

[0118] Furthermore, the registration may involve a model fitting operation and a segmentation, or a clustering strategy as described above. The calculation of the transformation function may be performed in a designated substep denoted as M42. According to one embodiment, the registration is specifically based on registering the biometric data EBD, RBD respectively included in the reference data set RDS and the extracted data set EDS with each other. Preferably, step M40 is performed in the central processing system 100.

[0119] In a subsequent step M50, the extracted dataset EDS is compared with the reference dataset RDS using the registration REG - and the information extracted from the image dataset IM is then compared with the information extracted from the basic reference medical image dataset. In order to automatically quantify the pathological changes between the basic examinations and help the user deliver a medical diagnosis, the observations (findings, measurements, etc.) extracted from the subsequent image datasets must be matched. This may involve transforming the positions of the medical findings EMF and RMF into a common coordinate system using the results of the registration REG in a sub-step M51. In this way, the medical findings EMF and RMF are mapped to each other, in other words (co)aligned with each other. Subsequently, the corresponding pairs of medical findings EMF and RMF can be identified, and it can be determined whether the medical findings are newly formed or disappeared. In an example embodiment, for each pair of findings from the reference medical findings RMF and the findings from the extracted medical findings EMF, the probability that the two findings describe the same pathology can be calculated, for example, considering their spatial proximity after the mapping in step M51, whether they are of the same type, and how similar other parameters are. In an example embodiment, the pairs that are most likely to describe the same findings are matched / associated until a certain threshold is reached. The remaining findings are likely to be related to, for example, new lesions that did not appear earlier or old lesions that were removed by surgery. Once the medical findings EMF, RMF are matched, the changes in the medical findings can be quantified in an optional sub-step M53. Quantifying the changes may mean comparing the measurement parameters of the medical findings EMF, RMF. For example, the volume occupied by the lesion in the image dataset IM can be compared with the volume of the same lesion extracted from the reference medical image dataset, and the volume difference can be quantified. In this way, an indication of the reduction or growth rate of the lesion can be provided. Preferably, step M50 is performed in the central processing system 100.

[0120] In the subsequent step M60, the result R is generated using the quantitative change of the medical findings. The result R can be in the form of a user-visible result, i.e., a human-readable format. In this way, the result R can be in the form of a structured report indicating the change of the medical findings. For example, the structured report can be in the form of a radiological report pre-filled with the comparison results of the previous step M50 by the system. The report can be configured so that the user can complete the report, for example, by indicating further observations and his / her performance. The generation of the structured report can be performed in an optional sub-step M61. In addition, the result R generated in step M60 can be in the form of visualization. Visualization can be generated in an optional sub-step M62. Visualization can include rendering one or more representations of the image data set IM using changes highlighted for the user, for example, by introducing symbols or numbers near the extracted medical findings EMF, applying a color map or heat map, and / or adjusting the brightness or luminescence value of the rendering. The rendered result can be in the form of one or more auxiliary images, which indicate to the user the location of the change and / or how big the change is. The rendering can be a two-dimensional rendering based on an appropriate representation of the image data set IM, such as a cross section or slice through the image body. The representation can be selected manually by the user, for example by scrolling the image volume, or (semi) automatically by the system. In addition, known volume rendering techniques, such as ray tracing or ray casting, etc., can be used. In this regard, the user can specify parameters such as viewing angle or viewing distance. Preferably, this can be done interactively via the user interface 12 with the help of a suitable graphical user interface. The size of the change can be visualized as a heat map, such as an overlay image on the image data set IM, to help guide the user to read the image. Here, a color gradient with a given color spectrum can be assigned to the size of the change so that large positive changes (growth, new lesions, etc.) are associated with one end of the color spectrum, while large negative changes (shrinkage, disappearance) are associated with the other end of the color spectrum. Another form of the result R can be generated in an optional sub-step M63 by providing or creating a representation or history of the relevant findings in the form of a table or trend graph. It is worth noting that the result R can not only reflect the comparison of the image data set IM with a reference data set RDS, but also reflect the comparison with multiple data sets. Therefore, structured reports, auxiliary images and trend graphs can show the changes of medical findings between multiple subsequent examinations. According to an embodiment, the type of result R (e.g., structured report, auxiliary image, trend graph) can be interactively selected by the user via the user interface 12. Another optional sub-step M64 involves, for example, forwarding the result R to the user via the user interface 12. Step M60 can be performed at least partially on the user interface 12 or on the central processing system 100. If necessary, corresponding data exchanges are included in this step. Preferably, step M60 is mainly performed in the central processing system 100, and the actions involving the client are initiated by the central processing system 100.In other words, step M60 is controlled by the central processing system 100 .

[0121] Optional step M70 is a storage or archiving step. It can process the generated results R (optional sub-step M71) as well as the data extracted from the image dataset IM, i.e. both the extracted biometric data EBD and the extracted medical findings EMF (optional sub-step M72). Figure 4 As shown, archiving in both cases can be performed in the local database 13 as well as in the database 40 of the system. Preferably, step M70 is implemented so that the user has the opportunity to actively decide whether she or he wants to archive the result R and / or the extracted data set EDS. For example, this can be achieved by a corresponding button in a graphical user interface running in the user interface 12. When imported, the extracted data set EDS can be formatted so that it corresponds to the reference data set RDS already stored in the database 14 and / or 30. In addition, the data import can also include automated operations of marking data and mapping the imported data to the archived data. The actions of marking and mapping can be based on any metadata attached to the image data set IM and / or any supplementary information uploaded with the medical image data set. For example, the accession number, patient ID, image modality used and / or time of examination can be extracted from the metadata or any other supplementary information and used to archive the extracted data set EDS in a systematic and easily retrievable manner in the database 14 and / or 30. Before archiving, any data can be subjected to an appropriate filtering process to ensure that the archived data is anonymized. Step M70 may be at least partially performed on the user interface 12 or on the central processing system 100. Where necessary, corresponding data exchange is included in this step. Preferably, step M70 is mainly performed in the central processing system 100, wherein the actions involving the client are initiated by the central processing system 100. In other words, step M70 is controlled by the central processing system 100.

[0122] As long as it makes sense, the various embodiments or their various aspects and features can be combined or exchanged with each other without limiting or expanding the scope of the invention. Where applicable, the advantages described with respect to one embodiment of the invention are also advantageous for other embodiments of the invention.

[0123] The following points are also part of this disclosure:

[0124] 1. A computer-implemented method for comparing subsequent medical examinations, the method comprising:

[0125] receiving a medical image dataset showing a body part of a patient at a first time;

[0126] retrieving at least one reference dataset associated with the image dataset from a database, wherein the reference dataset comprises reference biometric data extracted from a reference image dataset depicting a body part of the patient at a second time different from the first time, the reference biometric data defining a location of an anatomical portion of the body part in the reference image dataset;

[0127] extracting biometric data from the image dataset, the biometric data defining a location of an anatomical portion of the body part in the image dataset and corresponding at least in part to the reference biometric data;

[0128] registering the reference dataset with an image dataset based on the reference biometric data and the extracted biometric data; and

[0129] The image dataset is compared to the reference dataset based on the registration.

[0130] 2. The method according to point 1,

[0131] wherein the reference data set comprises one or more reference medical findings extracted from the reference image data set, the reference medical findings being respectively associated with locations in the reference image data set;

[0132] wherein the step of extracting further comprises extracting from the medical image dataset one or more medical findings associated with the location in the image dataset; and

[0133] The comparing step includes comparing the extracted medical findings with the reference medical findings, and particularly includes quantifying changes in the medical findings.

[0134] 3. A computer-implemented method for comparing subsequent medical image data, the method comprising:

[0135] receiving a medical image dataset showing a body part of a patient at a first time;

[0136] Retrieving at least one reference dataset associated with the image dataset from a database, wherein the reference dataset comprises:

[0137] reference biometric data extracted from a reference image dataset depicting a body part of the patient at a second time different from the first time; and

[0138] one or more reference medical findings extracted from the reference image dataset, the reference medical findings associated with locations in the reference image dataset;

[0139] extracting biometric data and one or more medical findings from the image dataset, the biometric data corresponding at least in part to the reference biometric data, the one or more medical findings associated with locations in the image dataset;

[0140] performing registration of the reference dataset with the image dataset using the reference biometric data and the extracted biometric data; and

[0141] The extracted medical finding is compared with the reference medical finding based on the registration, particularly including quantifying changes in the medical finding.

[0142] 4. The method according to point 2 or 3 further comprises the following steps:

[0143] compiling additional reference data sets based on the extracted biometric data and the extracted medical findings; and

[0144] The further reference data set is forwarded to the database for storing the further reference data set in the database.

[0145] 5. The method according to any one of points 2 to 4, wherein the step of comparing comprises matching the extracted medical findings with the reference medical findings using the registration.

[0146] 6. The method according to any of points 2 to 5, wherein the step of comparing comprises aligning the extracted medical findings with the reference medical findings using the registration.

[0147] 7. The method according to any one of points 2 to 6 further comprises the following steps:

[0148] Calculate results based on quantified changes;

[0149] forwarding said results to the user, and / or

[0150] storing the results in a database;

[0151] The result is preferably in the following form:

[0152] Trend graphs showing changes in quantified medical findings,

[0153] a structured medical report comprising an indication of a change in a quantified medical finding, the indication preferably being in the form of one or more DICOM SR objects,

[0154] A visualization that a user can view that visualizes changes in the quantified medical findings, and / or

[0155] Any combination of the above.

[0156] 8. The method according to any one of points 2 to 7, further comprising the following steps:

[0157] Based on the quantified changes, a result visible to a user is generated, preferably by rendering an image of the medical image dataset in a manner that highlights the changes.

[0158] 9. A method according to any of the preceding points, wherein performing the registration comprises registering at least a portion of the reference biometric data with at least a portion of the extracted biometric data.

[0159] 10. The method according to any one of the preceding points, wherein performing the registration comprises: determining mutually corresponding data in the reference biometric data and the extracted biometric data, respectively, and performing the registration based on the mutually corresponding data.

[0160] 11. The method according to any of the preceding points, wherein the reference data set is formatted according to the DICOM standard.

[0161] 12. The method according to any one of the preceding points, wherein the subsequent medical image data is received by a user in an upload manner.

[0162] 13. The method according to any one of the preceding points, wherein the step of searching comprises:

[0163] extracting a data identifier from the medical image dataset; and

[0164] The reference data set is queried in the database using the data identifier.

[0165] 14. The method according to any one of the preceding points further comprises the following step: providing a database storing a plurality of reference data sets extracted from a medical image data set, wherein, in the retrieval step, the at least one reference data set is retrieved from the database.

[0166] 15. The method according to any of the preceding points, further comprising the step of retrieving from the database a second reference dataset associated with the image dataset, wherein the second reference dataset comprises second reference biometric data extracted from a second reference image dataset depicting a body part of the patient at a third time different from the first time, the reference biometric data defining a position of an anatomical part of the body part in the second reference image dataset;

[0167] extracting biometric data from the image dataset, the biometric data defining a location of an anatomical portion of the body part in the image dataset and corresponding at least in part to the second reference biometric data;

[0168] performing registration of the reference dataset with the image dataset based on the reference biometric data and the extracted second biometric data; and

[0169] The image dataset is compared to the second reference dataset based on the registration.

[0170] 16. The method according to point 15, further comprising:

[0171] performing a second registration of the second reference data set with the reference data set based on the reference biometric data and the extracted second biometric data; and

[0172] The reference data set is compared to the second reference data set based on the registration.

[0173] 17. A computer-implemented method for comparing subsequent medical image data, the method comprising:

[0174] receiving a medical image dataset showing a body part of a patient at a first time;

[0175] Retrieving a reference dataset associated with the image dataset from a database, wherein:

[0176] The reference data set is extracted from a reference image data set depicting a body part of the patient at a second time different from the first time, and

[0177] The reference data set comprises a plurality of location-specific reference values ​​respectively relating to locations in the reference image data set;

[0178] extracting from the image dataset an extracted dataset comprising a plurality of position-dependent extracted values, the plurality of position-dependent extracted values ​​respectively relating to positions of the body part in the image dataset and at least partially corresponding to the reference dataset;

[0179] performing registration of the reference dataset with the extracted dataset by registering at least a portion of the reference dataset with at least a portion of the extracted dataset; and

[0180] The reference data set and the extracted data set are compared based on the registration, in particular by quantifying changes from the reference data set to the extracted data set.

[0181] 18. The method according to point 17, wherein:

[0182] The reference value and the extracted value each include a medical finding respectively associated with a location in the body part; and

[0183] The comparison includes quantifying changes in medical findings from the reference dataset to the extracted dataset.

[0184] 19. The method according to point 17 or 18, wherein:

[0185] The reference values ​​and the extracted values ​​respectively include a plurality of anatomical landmarks and / or a surface mesh of the anatomical portion; and

[0186] The step of performing registration is based on said anatomical landmarks and / or surface mesh.

[0187] 20. The method according to any one of points 17 to 19, further comprising the step of uploading the extracted data set as an additional reference data set to the database.

[0188] 21. A computer-implemented method for comparing subsequent medical image data, the method comprising:

[0189] receiving a medical image dataset showing a body part of a patient at a first time;

[0190] retrieving from a database a reference dataset associated with the image dataset;

[0191] The reference data set includes:

[0192] a set of reference anatomical landmarks extracted from a reference image dataset depicting a body part of the patient at a second time different from the first time, and

[0193] one or more reference medical findings extracted from the reference image dataset, the reference medical findings associated with locations in the reference image dataset;

[0194] extracting from the image dataset a set of anatomical landmarks corresponding at least in part to the reference anatomical landmarks and one or more medical findings associated with locations in the image dataset;

[0195] performing registration of the reference dataset with the image dataset by registering at least a portion of the reference anatomical landmarks with at least a portion of the extracted anatomical landmarks; and

[0196] Based on the registration, the extracted medical finding is compared with the reference medical finding, in particular by quantifying a change in the medical finding.

Claims

1. A computer-implemented method for comparing subsequent medical examinations, the method comprising: receiving (M10) a medical image data set (IM) showing a body part of a patient at a first time; retrieving (M20) from a database (14, 30) at least one reference data set (RDS) associated with the image data set (IM), wherein the reference data set (RDS) comprises reference biometric data (RBD) extracted from a reference image data set depicting a body part of the patient at a second time different from the first time, the reference biometric data (RBD) defining a location of one or more anatomical parts of the body part in the reference image data set; extracting (M30) extracted biometric data (EBD) from the image dataset (IM) by reading and analyzing the reference biometric data (RBD) and processing the image dataset (IM) based thereon, the extracted biometric data (EBD) defining the position of one or more anatomical parts of the body part in the image dataset (IM) and corresponding at least partially to the reference biometric data (RBD), wherein extracting the extracted biometric data (EBD) comprises: reading features of the retrieved reference dataset (RDS), and processing the image dataset (IM) based on the read features to extract the extracted biometric data (EBD); performing a registration (REG) of the reference data set (RDS) with the image data set (IM) based on corresponding data points and / or common biometric areas and / or corresponding biometric information of the reference biometric data (RBD) and the extracted biometric data (EBD); and The image data set (IM) is compared (M50) with the reference data set (RDS) based on the registration (REG).

2. The method according to claim 1, in, The reference data set (RDS) comprises one or more reference medical findings (RMFs) extracted from the reference image data set, the reference medical findings (RMFs) being respectively associated with locations in the reference image data set; Wherein, the step of extracting (M30) further comprises: extracting from the image dataset (IM) one or more medical findings (EMF) associated with the location in the image dataset (IM); and Therein, the step of comparing (M50) comprises: based on the registration (REG), comparing the extracted medical finding (EMF) with the reference medical finding (RMF) by quantifying (M52) the variation of the medical finding.

3. The method according to claim 1 or 2, wherein: The extracted biometric data (EBD) and the reference biometric data (RBD) respectively include: One or more anatomical landmarks; and / or One or more surface meshes of one or more anatomical parts included in the body part.

4. The method according to claim 1 or 2, wherein: The steps of retrieval (M20) include: retrieving (M24.1) one or more candidate reference data sets from the database (14, 30); providing (M24.2) the one or more candidate reference data sets to a user so as to select at least one user-selected reference data set from the one or more candidate reference data sets, receiving (M24.3) user selections; and Using (M24.4) the at least one user-selected reference data set as the at least one reference data set (RDS).

5. The method according to claim 1 or 2, wherein: The steps of retrieval (M20) include: provide one or more selection criteria; and retrieving (M22) said at least one reference data set (RDS) by applying one or more of said selection criteria, The selection criteria are based on at least one of the following: an indication of the patient's identity; an indication of the body part depicted; an imaging modality for acquiring said image dataset (IM) and / or said reference image dataset; a time window between the first time and the second time; an indication whether said reference data set (RDS) relates to a previous examination or a subsequent examination with respect to said image data set (IM); an indication of a previous selection to compare the reference data set (RDS) with the received image data set; and / or Any combination of the above.

6. The method according to claim 1 or 2, wherein: Performing (M40) the registration (REG) comprises calculating (M42) a transformation function mapping corresponding anatomical locations in the reference image dataset and the image dataset (IM) to each other.

7. The method according to claim 1 or 2, wherein: Performing (M40) said registration (REG) is based on a rigid and / or affine and / or non-rigid registration and / or any combination thereof.

8. The method according to claim 1 or 2, further comprising the following steps: compiling a further reference data set (RDS) based on said extracted biometric data (EBD); as well as The further reference data set (RDS) is forwarded (M72) to the database (14, 30) for storing the further reference data set (RDS) in the database (14, 30).

9. The method according to claim 1 or 2, further comprising the following steps: calculating (M60) a result (R) based on the comparison; and forwarding (M64) the result to the user; and / or storing (M71) the result (R) in a database (14, 30), The result (R) includes at least one of the following: Structured medical reports; Trend chart; Visualizations visible to the user; and / or Any combination of the above.

10. The method according to claim 1 or 2, wherein: The image data set (IM) is received from a first site at a central processing system (100), the central processing system (100) being remote from the first site; as well as The steps of searching (M20), extracting (M30), executing (M40) and comparing (M50) are controlled by the central processing system (100).

11. The method according to claim 10, wherein: The central processing system (100) is a cloud-based platform.

12. A system (100) for comparing subsequent medical examinations, comprising a computing unit (20) configured to: receiving an image dataset (IM) depicting a body part of a patient at a first time; at least one reference data set associated with the image data set (IM) is retrieved from a database (14, 30), wherein: the reference data set (RDS) comprising reference biometric data (RBD) extracted from a reference image data set depicting a body part of the patient at a second time different from the first time, the reference biometric data (RBD) defining a location of an anatomical portion of the body part in the reference image data set (RDS); reading and analyzing the reference biometric data (RBD) and processing the image dataset (IM) based thereon so as to extract extracted biometric data (EBD) from the image dataset (IM), the extracted biometric data (EBD) defining the location of one or more anatomical parts of the body part in the image dataset (IM) and corresponding at least partially to the reference biometric data (RBD), wherein extracting the extracted biometric data (EBD) comprises: reading features of the retrieved reference dataset (RDS), and processing the image dataset (IM) based on the read features to extract the extracted biometric data (EBD); performing a registration (REG) of the reference data set (RDS) with the image data set (IM) based on corresponding data points and / or common biometric areas and / or corresponding biometric information of the reference biometric data (RBD) and the extracted biometric data (EBD); and The image data set (IM) is compared with the reference data set (RDS) based on the registration (REG).

13. The system of claim 12, further comprising: The interface unit (40) is configured to: receiving a medical image dataset (IM) from a site (10) external to the system (100); and communicating with the database (14, 30), the database (14, 30) storing a plurality of reference data sets (RDS) extracted from the medical image data set; and / or The database (14, 30) stores a plurality of reference data sets (RDS) extracted from the medical image data set.

14. A computer program product comprising a program element which, when loaded into a memory of a computing unit (20) of a system (100) for comparing subsequent medical examinations, directs the computing unit (20) to perform the steps of the method according to any one of claims 1 to 11.

15. A computer-readable medium storing a program element, which, when executed by a computing unit (20) of a system (100) for comparing subsequent medical examinations, can be read and executed by the computing unit (20) to perform the steps of the method according to any one of claims 1 to 11.

Images