Apparatus and method for visualizing imaging data

Abstract

Various aspects and embodiments relate to a method for mapping additional information related to a 2D medical image of a region of interest to a 3D medical image volume of the region of interest, and an apparatus and computer program product configured to perform such a method. A method according to an embodiment includes: receiving a 2D composite medical image of the region of interest and a 3D medical image volume of the region of interest; decomposing the 2D composite medical image into at least the following: a base image, and an additional image comprising the additional information; identifying a representative 2D slice in the 3D medical image volume corresponding to the base image based on an evaluation of the base image and the 3D medical image volume; evaluating a transformation of the base image that maps the base image to the representative 2D slice; and combining the representative 2D slice with an image formed by applying the evaluated transformation to the additional image to produce a new composite 2D medical image of the region of interest. Various aspects and embodiments recognize that information provided in a 2D medical image may be utilized and mapped onto new imaging information forming portions of a 3D medical image volume. Providing a visualization of this mapping can assist imaging professionals in more efficient evaluation of captured image data representing the region of interest. In particular, mapping additional information to a portion of a 3D medical image volume of a region of interest may assist an imaging professional in performing a technical task to evaluate clinical relevance of captured 3D medical image data. In order to determine an appropriate treatment procedure, or in order to evaluate or diagnose a disease or a chronic condition based on the captured image data, this evaluation can be performed.

Description

Technical Field

[0001] Various example embodiments relate to imaging data. In particular, aspects and embodiments relate to apparatuses and methods for mapping or visualizing additional information related to a 2D medical image of a region of interest to a 3D medical image volume of the region of interest. Background Art

[0002] Patients undergoing chronic conditions or diseases may undergo one or more treatments and associated subsequent treatments or aftercare. The initial investigation, treatment, and subsequent processes that follow can be understood to represent a series of substantially independent interactions with the subject, and each interaction can be associated with capturing or creating one or more images or imaging data sets related to the subject and the condition or disease of interest.

[0003] For example, cancer patients may undergo an initial study and may also typically undergo subsequent imaging studies in the radiology department after receiving cancer treatment. Such studies can be performed for the purpose of tumor response assessment and / or for assessing tissue changes after receiving cancer treatment.

[0004] When imaging a patient, imaging methods generally support capturing 3D imaging data.

[0005] In the case of cancer treatment, for example, imaging data related to, for example, a previous tumor assessment or examination and imaging data capturing details of a previous investigation or treatment (e.g., radiation therapy dose) may be useful to various healthcare professionals interacting with the patient. For example, the correlation of equivalent 3D imaging data from a (one or more) previous assessment or treatment with the imaging data being acquired can support, for example, the measurement or assessment of tumor changes. Such a direct correlation between 3D imaging data sets can also, for example, allow a radiologist to more easily distinguish between tumor progression and radiation therapy side effects.

[0006] Technically, the task of "associating" or image tagging by dose level can be solved by performing non-rigid registration between a subsequent 3D image (CT, MRI, etc.) and a previously acquired 3D planning image and by warping known delivery dose information (or dose level), which is known with respect to the 3D planning image, to the geometry of the subsequent 3D image. Similarly, 3D image registration can be used to associate findings in a current 3D medical image with findings in a previously acquired 3D medical image.

[0007] While technically feasible, such registration between contemporaneous subsequent 3D imaging datasets acquired in a radiology practice and planning 3D imaging data previously acquired in a radiotherapy department may be difficult due to various factors, including, for example, data access restrictions between departments and / or between different institutions that may not share the same image data archives. Additionally, even within the same hospital, searching for and retrieving prior 3D CT and radiotherapy dose distribution data from a PACS archive may be slow.

[0008] Savjani et al., “A Framework for Sharing Radiation Dose Distribution Maps in the Electronic Medical Record for Improving Multidisciplinary Patient Management” [Radiol Imaging Cancer, Mar. 12, 2021, Vol. 3, No. 2, e200075. doi:10.1148 / rycan.2021200075.eCollection, Mar. 2021] recognized that the specialized software and hardware of a radiation oncology practice may render the radiation treatment history inaccessible to other medical subspecialties. The literature pointed out the difficulties that occur when specific data of 3D radiotherapy data from a radiotherapy department is pushed to a hospital's Picture Archiving and Communication System (PACS), and noted that most hospitals' PACSs cannot read the original radiotherapy data.

[0009] Some adjustments to processes and methods related to the sharing of imaging data can support the creation of information that medical professionals are using by providing a way to combine previously acquired information with new imaging datasets. SUMMARY OF THE INVENTION

[0010] The scope of protection sought by the various example embodiments of the present invention is set forth by the independent claims. Example embodiments and features (if any) described in this specification that are not within the scope of the independent claims will be construed as examples useful for understanding the various embodiments of the present invention.

[0011] According to various but not necessarily all example embodiments, there is provided an apparatus configured to map additional information related to a 2D medical image of a region of interest to a 3D medical image volume of the region of interest, the apparatus comprising: at least one processor; and at least one memory storing instructions which, when executed by the at least one processor, cause the apparatus to at least perform the following operations: receive a 2D composite image of the region of interest; receive a 3D image volume of the region of interest; decompose the 2D composite image into at least the following: a base image, and an additional image including the additional information; identify a representative 2D slice in the 3D image volume corresponding to the base image based on an evaluation of the base image and the 3D image volume; evaluate a transformation of the base image that maps the base image to the representative 2D slice; and combine the representative 2D slice with an image formed by applying the evaluated transformation to the additional image to produce a new composite 2D image of the region of interest.

[0012] In one embodiment, the 2D image of the region of interest comprises: a color image.

[0013] In one embodiment, the composite 2D image comprises: a 2D schematic image representing the region of interest in the 3D image volume.

[0014] In one embodiment, the composite 2D image comprises: additional information related to a treatment or procedure performed on the region of interest.

[0015] In one embodiment, wherein the additional information comprises: a geometric distribution of a treatment dose applied to the region of interest.

[0016] In one embodiment, the treatment comprises radiotherapy.

[0017] In one embodiment, the geometric distribution comprises: isodose lines or color-mapped dose information.

[0018] In one embodiment, the additional information comprises: one or more annotations, the one or more annotations comprising: a contour of an interesting feature; and / or a label of an interesting feature; and / or a measurement result of an interesting feature.

[0019] In one embodiment, decomposing the 2D composite image comprises: decomposing the 2D composite image into at least the following: a grayscale base image; and an additional image including the additional information.

[0020] In one embodiment, decomposing the 2D composite image comprises: analyzing pixels of the 2D composite image to determine the grayscale base image and the additional image including the additional information.

[0021] In one embodiment, identifying a representative 2D slice in the 3D image volume corresponding to the base image includes: evaluating one or more planar images forming the 3D image volume for the base image, and selecting the planar image that best matches the base image among the evaluated one or more planar images.

[0022] In one embodiment, the following two items are determined as a combined calculation: identifying the representative 2D slice of the 3D image volume, and evaluating the transformation of the base image that maps the base image to the representative 2D slice.

[0023] In one embodiment, identifying the representative 2D slice of the 3D image volume and evaluating the transformation of the base image that maps the base image to the representative 2D slice includes: using slice-to-volume or 2D-to-2D registration image registration techniques.

[0024] In one embodiment, evaluating the transformation of the base image that maps the base image to the representative 2D slice includes: evaluating one or more differences between those images generated by one or more of rotation, translation, deformation, or scaling.

[0025] According to various but not necessarily all example embodiments, a computer-implemented method for mapping additional information related to a 2D image of a region of interest to a 3D image volume of the region of interest is provided, the method including: receiving a 2D composite image of the region of interest and a 3D image volume of the region of interest; decomposing the 2D composite image into at least the following: a base image, and an additional image including the additional information; identifying a representative 2D slice in the 3D image volume corresponding to the base image based on an evaluation of the base image and the 3D image volume; evaluating the transformation of the base image that maps the base image to the representative 2D slice; and combining the representative 2D slice with an image formed by applying the evaluated transformation to the additional image to generate a new composite 2D image of the region of interest.

[0026] In one embodiment, the 2D medical image of the region of interest includes: a color image.

[0027] In one embodiment, the composite 2D image includes: a 2D summary image representing the region of interest in the 3D image volume.

[0028] In one embodiment, the composite 2D image includes: additional information related to a treatment or process performed on the region of interest.

[0029] In one embodiment, the additional information includes: the geometric distribution of the treatment dose applied to the region of interest.

[0030] In one embodiment, the treatment includes radiotherapy.

[0031] In one embodiment, the geometric distribution includes: dose information of isodose lines or color maps.

[0032] In one embodiment, the additional information includes: one or more annotations, the one or more annotations including: the contour of the feature of interest; and / or the marking of the feature of interest; and / or the measurement result of the feature of interest.

[0033] In one embodiment, decomposing the 2D composite image includes: decomposing the 2D composite image into at least the following: a grayscale base image; and an additional image including the additional information.

[0034] In one embodiment, decomposing the 2D composite image includes: analyzing the pixels of the 2D composite image to determine the grayscale base image and the additional image including the additional information.

[0035] In one embodiment, identifying the representative 2D slice in the 3D image volume corresponding to the base image includes: evaluating one or more planar images forming the 3D image volume for the base image, and selecting the planar image that best matches the base image among the one or more evaluated planar images.

[0036] In one embodiment, the following two items are determined for combined calculation: identifying the representative 2D slice of the 3D image volume, and evaluating the transformation of the base image that maps the base image to the representative 2D slice.

[0037] In one embodiment, identifying the representative 2D slice of the 3D image volume and evaluating the transformation of the base image that maps the base image to the representative 2D slice includes: using slice-to-volume or 2D-to-2D registration image registration techniques.

[0038] In one embodiment, evaluating the transformation of the base image that maps the base image to the representative 2D slice includes: evaluating one or more differences between those images generated by one or more of rotation, translation, deformation, or scaling.

[0039] According to various but not necessarily all exemplary embodiments, a computer program product is provided that is operable to perform a method for mapping additional information related to a 2D image of a region of interest to a 3D image volume of the region of interest when executed on a computer. The method includes: receiving a 2D composite image of the region of interest and the 3D image volume of the region of interest; decomposing the 2D composite image into at least: a base image, and an additional image including the additional information; identifying a representative 2D slice in the 3D image volume corresponding to the base image based on an evaluation of the base image and the 3D image volume; evaluating a transformation of the base image that maps the base image to the representative 2D slice; and combining the representative 2D slice with an image formed by applying the evaluated transformation to the additional image to produce a new composite 2D image of the region of interest.

[0040] According to various but not necessarily all exemplary embodiments, a non-transitory computer-readable medium storing computer program code is provided. The computer program code includes instructions that, when executed by a processor, cause a computer to perform a method for mapping additional information related to a 2D image of a region of interest to the 3D image volume of the region of interest. The method includes: receiving a 2D composite image of the region of interest and the 3D image volume of the region of interest; decomposing the 2D composite image into at least: a base image, and an additional image including the additional information; identifying a representative 2D slice in the 3D image volume corresponding to the base image based on an evaluation of the base image and the 3D image volume; evaluating a transformation of the base image that maps the base image to the representative 2D slice; and combining the representative 2D slice with an image formed by applying the evaluated transformation to the additional image to produce a new composite 2D image of the region of interest.

[0041] Some embodiments recognize that the information provided in a 2D image can be utilized and mapped onto new imaging information to form part of a 3D image volume. Providing a visualization of such mapping in the form of a new composite 2D image can assist imaging professionals in more efficiently evaluating the captured image data representing the region of interest.

[0042] In particular, embodiments recognize that mapping additional information to a portion of the 3D image volume of a region of interest can assist imaging professionals in performing the technical task of evaluating the clinical relevance of the captured 3D image data. Such evaluation may be performed to determine an appropriate treatment process or to evaluate or diagnose a disease or chronic condition based on the captured image data.

[0043] These aspects and other aspects of the invention will be apparent from the (one or more) embodiments and (one or more) arrangements described below.

[0044] Further specific and preferred aspects are set out in the appended independent and dependent claims. The features of the dependent claims may be combined appropriately with the features of the independent claims and may also be combined with features other than those explicitly set out in the claims.

[0045] Where an apparatus feature is described as being operative to provide a function, it is to be understood that this includes both an apparatus feature that provides the function and an apparatus feature that is adapted or configured to provide the function. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Some example embodiments will now be described with reference to the drawings, in which:

[0047] Figure 1A and Figure 1B Two example 2D images of an area of interest of an object are shown;

[0048] Figure 2 Schematically illustrates the input and output of a method according to an arrangement;

[0049] Figure 3 Schematically illustrates the input and output of a method according to a further arrangement;

[0050] Figure 4 Schematically illustrates a system used according to an arrangement; and

[0051] Figure 5 Schematically illustrates the main steps of a method according to an arrangement. DETAILED DESCRIPTION

[0052] Before discussing the example embodiments in more detail, a background for understanding the various aspects and embodiments is first provided.

[0053] A patient experiencing a chronic condition or disease may undergo one or more interactions with various healthcare professionals. The initial investigation, treatment, and subsequent follow-up processes can be understood to represent a series of substantially independent interactions with the subject, and each interaction can be associated with capturing or creating one or more images or imaging datasets related to the subject and the condition or disease of interest.

[0054] For example, a cancer patient may undergo an initial investigation (including imaging of the area of interest, treatment of the area of interest, which may be planned or carried out using an imaging process), and typically may also undergo subsequent imaging studies in, for example, a radiology department after receiving cancer treatment.

[0055] Ongoing imaging studies of a patient can be performed for various purposes, including disease progression monitoring, tumor response assessment, and / or assessment of tissue changes after cancer treatment.

[0056] When imaging a patient, imaging methods are typically used to support the capture of 3D imaging data in the form of a 3D image volume. Examples of 3D imaging methods include computed tomography scans (CT scans) and magnetic resonance imaging scans (MRI scans). Such imaging techniques allow for the capture of a 3D dataset or image volume related to a region of the body.

[0057] In the case of cancer treatment, for example, CT and / or MRI scans can be performed to capture information related to tumor assessment or examination. This CT or MRI imaging data can include details related to previous investigations or treatments (e.g., radiation therapy doses), which are useful to various healthcare professionals who continue to interact with the patient.

[0058] For example, the correlation of equivalent 3D imaging data from (one or more) previous assessments or treatments with newly acquired 3D imaging data can support, for example, the measurement or assessment of tumor changes. Such direct correlation between 3D imaging datasets can, for example, allow a radiologist to more easily distinguish tumor progression from radiation therapy side effects.

[0059] Technically, the task of "associating" or image tagging by dose level can be addressed by performing non-rigid registration between newly acquired 3D images (CT, MRI, etc.) and previously acquired 3D planning images and by warping known delivery dose information (or dose levels), which is known with respect to the 3D planning images, to the geometry of the subsequent 3D images. Similarly, 3D image registration can be used to associate findings in current 3D medical images with findings in previously acquired 3D medical images.

[0060] Although technically feasible, the registration between a new 3D imaging dataset and a previously acquired 3D imaging data (e.g., registration of the new imaging dataset with the dataset used for planning radiation therapy doses) performed in a radiotherapy department can be difficult due to various factors, including, for example, data access restrictions between departments and / or different institutions that may not share the same image data archives. Additionally, even within the same hospital, searching for and retrieving previous 3D CT and radiation therapy dose distribution data from a PACS archive can be slow.

[0061] Full 3D data collected previously may not be available or may be subject to practical constraints. However, for example, when performing a follow-up study, it may be possible to use some simpler information that summarizes the details of the previously acquired imaging data and the details of the administered treatment. For example, it is generally possible to use some 2D information (such as an image that summarizes the details of the previously acquired imaging data, etc.) and the delivered radiotherapy plan, and this 2D information is associated with the records kept about the patient.

[0062] As the patient progresses and is treated or monitored, it is possible to enable parties or healthcare professionals to use information related to the patient. Information that summarizes, for example, the details of the treatment performed on the patient may include soft-copy or hard-copy documents.

[0063] A summary of the information kept about the patient may include one or more summary 2D images of the region of interest. The (one or more) summary images may include, for example, RGB images of representative axial, coronal, and sagittal slices of 3D planning imaging data. For example, such imaging data may include 3D computed tomography (CT) data acquired about the patient. The (one or more) summary 2D images of the region of interest provided in the information summary may be fused with additional information about the treatment or procedure performed on the patient. An example of such treatment-related information includes: the geometric distribution of the planned radiotherapy dose. Such dose information may include, for example, color-coded isodose lines or a color map representing the radiation dose. The color map allows the user to visualize the dose based on the color-coded map. The color-coded map assigns (one or more) colors to different dose ranges. The color map may include a color gradient related to the dose. The summary document may be prepared by a radiation oncologist at the end of the treatment. The summary document is prepared to communicate the treatment details or procedure details to other disciplines and departments.

[0064] Similarly, representative 2D images of previous 3D imaging events (such as tumor assessment examinations) can be summarized in one or more documents, screenshots, or stored as one or more 2D images in a standard format (such as jpeg) in an electronic medical record (EMR). Representative 2D images of previous imaging sessions may include various additional information. For example, they may be annotated with a tumor region of interest (ROI) contour; tumor size measurements (such as lines, arrows, text, etc. marking the tumor diameter). 3D medical images in a picture archiving and communication system (PACS) archive generally do not contain such annotations. For medical professionals (such as radiologists) performing follow-up studies to acquire additional imaging data, it may be time-consuming to load any 3D images of previous examinations or studies from the PACS archive, reproduce any previous measurements that may have been provided in 2D format, and correlate any current findings with such previous measurements.

[0065] While in principle it may seem that an imaging or other healthcare professional could combine information from two sources (a newly acquired 3D data set and a 2D summary of a previous examination, procedure, or intervention) by simply looking at the two sources of information, it should be understood that the complexity of the information and imaging data being acquired and the variations in anatomical structures (e.g., rotation, translation, and / or deformation, different scaling, etc.) mean that it may be difficult for a user to directly correlate the treatment dose or other information from a representative 2D image of the summary or information provided by a previous acquired imaging examination with the 3D patient anatomy in a subsequent CT or MRI scan.

[0066] The background of the various aspects and embodiments to be understood has now been described, and further provides an overview of a method according to some possible embodiments:

[0067] An arrangement can provide a method for mapping additional information related to a 2D image of a region of interest to a 3D image volume of the region of interest. An arrangement can provide a device configured to perform such a method. The device can include, for example, a computer in the form of at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the computer to perform the method.

[0068] A method according to an arrangement includes the steps of: receiving a 2D composite image of a region of interest and a 3D image volume of the region of interest. The method can include the step of decomposing the 2D composite image into at least the following: a base image; and an additional image including additional information. The method can include the step of identifying a representative 2D slice in the 3D image volume corresponding to the base image based on an evaluation of the base image and the 3D image volume. The method can include the step of evaluating a transformation of the base image that maps the base image to the representative 2D slice. The method can include the step of combining the representative 2D slice with an image formed by applying the evaluated transformation to the additional image to produce a new composite 2D image of the region of interest.

[0069] Generally, embodiments recognize that the information provided in 2D medical images can be utilized and mapped onto new imaging information that forms part of a 3D medical image volume. Providing a visualization of such a mapping in the form of a new composite 2D image can assist an imaging professional in more efficiently evaluating the image data captured representing the region of interest. In particular, mapping additional information to a portion of the 3D image volume of the region of interest can assist an imaging professional in performing the technical task of evaluating the clinical relevance of the captured 3D image data. Such an evaluation can be performed to determine an appropriate treatment process or to evaluate or diagnose a disease or chronic condition based on the captured image data.

[0070] As described above, various aspects and embodiments recognize that a summary of information stored about a patient may include one or more summary 2D images of regions of interest. The (one or more) summary images may include, for example, RGB images of representative axial, coronal, and sagittal slices of 3D planning imaging data. For example, such imaging data may include 3D computed tomography (CT) data acquired about the patient. The (one or more) summary 2D images of the regions of interest provided in the information summary may be fused with additional information related to image evaluation and / or a treatment or procedure performed on the patient. An example of such treatment-related information includes the geometric distribution of a planned radiotherapy dose. Such dose information may include, for example, color-coded isodose lines or a color gradient radiation dose. A summary document may be prepared by a radiation oncologist at the end of a treatment. The summary document is prepared to communicate treatment details or procedure details to other disciplines and departments.

[0071] Various aspects and embodiments recognize that such summary information including 2D information can be utilized and combined with, for example, a newly acquired 3D imaging dataset to provide useful information to a user (e.g., a healthcare professional) to enhance and allow evaluation of the newly acquired 3D imaging dataset. The useful information may include enhanced imaging data that provides user assistance in interpreting the newly acquired 3D imaging data. For example, the enhanced imaging data may assist the user in identifying and differentiating (one or more) tissue changes within a radiofrequency treatment area of interest as compared to (one or more) tissue changes outside the radiofrequency treatment area.

[0072] Similarly, various aspects and embodiments recognize that representative 2D images of a previous 3D imaging event (e.g., a tumor assessment examination) can be summarized in one or more documents, screenshots, or stored as one or more 2D images in a standard format (e.g., jpeg) in an electronic medical record (EMR). The representative 2D images of a previous imaging session may include various forms of additional information (which is not part of the originally acquired image data). For example, the additional information included in a 2D image of a region of interest may include one or more annotations indicating the following: a tumor region of interest (ROI) contour; tumor size measurements (e.g., lines, arrows, text, etc. marking the tumor diameter).

[0073] Various aspects and embodiments recognize that summary information related to a previous imaging event (including 2D information) can be utilized and combined with a newly acquired 3D imaging dataset to provide useful information to a user. The useful information may include enhanced imaging data that provides user assistance in interpreting the newly acquired 3D imaging data. For example, the enhanced imaging data may assist the user in aspects such as tumor progression even when no treatment has occurred.

[0074] Aspects and embodiments provide an apparatus and method that support visualization of information derived from 2D images summarizing prior findings and / or treatments on portions of subsequently acquired 3D imaging datasets. Such information can include, for example, one or more geometric features related to prior findings and / or treatments. Representative 2D images (one or more) including additional information can include portions of a treatment summary report or summary information related to prior imaging sessions. Representative 2D images can include one or more images provided in a record associated with a patient or imaging subject. Representative images can include images stored in a standard image format in an electronic medical record (EMR) system.

[0075] Accordingly, in some arrangements, 2D imaging information, 2D imaging datasets, and / or information mapped to 2D images can be received or otherwise obtained such that information related to such 2D images can be used in relation to 3D imaging volumes captured about similar regions of interest.

[0076] To utilize additional information provided in or related to 2D images, it may be necessary to decompose the 2D image into at least two parts. These parts can include: basic image data, and additional information that does not form part of the basic image data.

[0077] According to one example arrangement, a representative 2D RGB summary image of an image including a region of interest containing additional information and / or annotated color dose distribution can be decomposed into:

[0078] a grayscale slice (S) of a planning image, and

[0079] additional features, e.g., a colored geometric dose feature (GDF).

[0080] According to one arrangement, color (e.g., isodose or tumor extent) lines present in the representative 2D summary image can be extracted, for example, using a simple RGB decomposition of the picture and detecting pixels having unequal R, G, and B values. In some examples, mathematical optimization techniques can be used to extract transparent color patches of an alpha-blended image. Thus, the summary image can be suitably divided into at least two images, where one image relates to the basic image of the region of interest and the other image includes additional information related to the basic image.

[0081] After decomposing the summary image into at least two images, in some arrangements, a transformation (T) is calculated that maps the basic image (e.g., the grayscale slice (S) of the planning image) to a substantially equivalent plane or slice of a 3D dataset acquired in the form of, for example, a 3D volume or a subsequent image. The 3D imaging dataset V can include, for example, new CT or MRI information obtained about the region of interest.

[0082] According to some arrangements, the transformation is selected or evaluated such that a plane P or slice in the 3D data set V that substantially corresponds to the 2D base image data is found. For example, the evaluation can include an evaluation of each plane P in the volume V that might correspond to the representative slice S. The evaluation can be such that it aims to determine the index of the slice (or plane) in V that best matches the base image.

[0083] The transformation T can be determined using, for example, slice-to-volume registration techniques or simple 2D-to-2D image registration techniques. Such image registration techniques can adapt the transformation of the base image to align with the plane in the image volume that best matches the base image, thereby taking into account one or more differences between those images caused by one or more of: rotation, translation, deformation, or scaling.

[0084] According to some arrangements, a mapping or transformation that associates the base image of a 2D image with a plane or slice of a 3D image data set has been determined, and this mapping can be applied to an additional information image. The additional information image can, for example, include an image containing representative 2D geometric dose features (GDFs) or other additional information (e.g., one or more indications of dimensions, positions, measurements, etc. related to one or more features included in the base image). In other words, according to some arrangements, the mapping of the available additional information related to the schematic 2D image can be applied such that the additional information can equivalently relate to the appropriate plane or section of the 3D imaging data set. Thus, in some arrangements, T(GDF) is evaluated.

[0085] According to some arrangements, the transformation of the additional information image can be combined with the plane or slice in the 3D data set that has been determined to best correspond to the base image data. In one example, the appropriately transformed or mapped additional information image can be superimposed or combined with the plane or slice of the base image in the 3D data set that has been determined to be substantially equivalent to the 2D schematic image. In some arrangements, for example, T(GDF) is superimposed on P, thereby allowing, for example, direct visualization of the dose features on P. Similarly, by superimposing T(GDF) on P, medical professionals can view the P image, and the P image includes the planned dose level information.

[0086] Decomposition of the 2D image

[0087] Figure 1A and Figure 1B shows two example 2D images of the region of interest of the object.

[0088] Figure 1A shows a schematic representation of an image of a slice of the object's torso. Figure 1AThe 2D image schematically shows an image that may be generated from a subject with histologically proven non-small cell lung cancer. The cancer is marked as contour 100 on the image. The region of general interest is indicated by contour 200 (the entire torso). Figure 1A Various other features of interest (including the lungs 110, spinal cord 120, esophagus 130, and trachea 140) are also marked on the image.

[0089] Figure 1B Shows Figure 1A A slice of the subject's torso, on which Figure 1B The dose distribution of a radiofrequency treatment plan is shown. The treatment plan may include, for example, a stereotactic body radiotherapy plan. The treatment plan involves the VMAT technique, 10 fractions of 5 Gy each, for a total dose of 50 Gy. Isodose lines (percentage of total dose) 300 are shown, and a color overlay shows the dose distribution: regions 400, 410 including high-dose regions around the tumor, gradually decreasing dose regions 420, 430, and a large low-dose region 440.

[0090] It should be understood that Figure 1A Or Figure 1B Any one of them can be used as a 2D image including basic image data (grayscale image dataset) and additional information. Figure 1A Including additional information related to the contours of features of interest. Figure 1B Including additional information related to the contours of features of interest and additional information related to the treatment plan (in this example, isodose lines and color-coded dose information superimposed on the basic image data).

[0091] According to the arrangement, a composite image (e.g., such as the composite image shown in Figure 1A And Figure 1B ) is "unmixed" or "decomposed" to create at least two separate images.

[0092] It should be understood that Figure 1A And Figure 1B The images shown in include basic image data in the form of an image slice of interest generated from a 3D CT image dataset. In the case of Figure 1B , this basic image data has been superimposed with at least a second image with a certain given transparency (i.e., alpha blending). In this example, the second image represents the planned or anticipated radiotherapy treatment dose. Generally, the basic image (in this case, the CT image) includes a grayscale image; the treatment dose image or additional information included in the composite image can usually include one or more color features or color images.

[0093] To solve the problem of image decomposition (i.e., unmixing), if a composite image (e.g., such as Figure 1A AndFigure 1B If the composite image shown in

[0094] is presented, the goal is set to obtain a version of the two original images from the single superimposed composite image. Generally speaking, such decomposition is only possible when certain assumptions are made about the properties of the two original images. In a general configuration, the color value of each pixel in each of the images involved is represented as an RGB - triple. The mapping of CT intensity to the corresponding gray in each pixel of the gray - scale image is achieved by assuming a known gray - scale color map. The mapping of radiotherapy dose levels to the color pixels of the image is achieved via an unknown RGB color mapping.

[0095] Therefore, a possible method for implementing the decomposition of a gray - scale base image superimposed with an RGB image can be based on estimating the unknown color mapping (for generating the original dose - planning image) according to the superimposed image pixel data. For a color map implemented in a subspace orthogonal to the gray - scale of the CT image, this color - map estimation can be straightforward. For other color - map subspaces, this estimation is more complex and usually cannot be solved uniquely.

[0096] In fact, the most commonly used RGB color maps represent connected curves (e.g., broken lines) in the RGB space. By assuming a general linear color map (or a set of multiple linear maps), some appropriate statistical methods for estimating the vector - space basis set (e.g., based on independent component analysis (ICA)) can be applied to estimate the color map as a basis vector according to the RGB values of the image pixels. If the color map has to be assumed to be non - linear, some more complex non - linear vector - space estimation methods are required.

[0097] In general, when no assumptions can be made about the underlying color map (or other image properties), it may be impractical to solve the image unmixing problem. However, the more information is given (e.g., one or both color maps may be known, or the subspace of the color map can be closely provided), the easier it becomes to handle the image unmixing.

[0098] Evaluation of the transformation

[0099] The problem of finding the corresponding cross - sectional slice in a 3D volume for a 2D slice image can be formulated as a slice - to - volume registration problem. Given a 2D slice image S and a 3D volume V, the slice - to - volume registration method can be operated to identify a 2D - to - 2D transformation function and a plane (i.e., a 2D slice from volume V).

[0100] In the most general case, and minimize the following objective function:

[0101]

[0102] Wherein:

[0103] is the "optimal" transformation, and

[0104] is the "optimal" plane;

[0105] C is an image similarity function, and

[0106] R is a regularization term.

[0107] The mapping T can be rigid, affine, or non-rigid. The image similarity function C is a quantitative evaluation of the similarity between the transformed 2D image and the corresponding slice in the 3D volume V.

[0108] Various types of image similarity functions C can be used, e.g., a similarity function defined only using the intensity values in S and V (e.g., normalized cross-correlation, mutual information), or a similarity function defined only using geometric landmarks, or a similarity function defined using a combination of both the image intensity values and geometric landmarks in S and V.

[0109] The regularization term R can be used to well-posed the problem. The regularizer R can impose constraints on the solution (e.g., minimizing out-of-plane deformation, minimizing the magnitude of deformation, imposing elastic constraints, etc.).

[0110] Alternatively, the problem of finding the corresponding cross-sectional slice in the 3D volume for the 2D image can be formulated as an iterative 2D registration problem, wherein the 2D image is iteratively registered with the slices of the 3D volume. The best or optimal corresponding slice can be identified by the index of the slice found to have the lowest value of the similarity function.

[0111] Therefore, it should be understood that the step of identifying the representative slice and the step of evaluating the transformation can be combined or performed separately.

[0112] By implementing a method consistent with the above method, the arrangement facilitates continuous interaction of the user with new imaging data (into which information from the summary has been incorporated). The user can interact, for example, via a graphical user interface (GUI) with some of the outputs of the arrangement. The arrangement allows the user to directly interact with at least one of the following: one or more 2D images provided as a summary of a previous or planned treatment or investigation; or a transformed image (based on which an equivalent plane or slice of a 3D imaging dataset is identified as equivalent to one or more 2D images); or a transformed additional information image (which includes information related to one or more 2D images, including, for example, geometric features of the delivered dose and / or transformed measurements (lines, regions of interest)). The user may be able to interact with various available 2D images and activate or deactivate the overlay of various available 2D images, the various available 2D images including:

[0113] (one or more) original 2D images;

[0114] equivalent transformed images in the form of planes or slices of a 3D imaging dataset, the equivalent transformed images having been evaluated as equivalent to the one or more 2D images;

[0115] transformed additional information image features included in the (one or more) original 2D images.

[0116] Thus, the arrangement and implementation can provide systems and methods for supporting tools to visualize additional information (e.g., one or more geometric features, measurements, annotations, etc. related to (one or more) previous findings and / or treatments) related to a newly acquired imaging dataset of the same region of interest. The additional information can form part of a representative 2D image or image set, thus forming part of a treatment or finding summary report in an electronic medical record system. The 2D image or image set can include one or more files stored in a standard image format (e.g., jpeg, etc.).

[0117] The arrangement can allow a user (e.g., a medical imaging professional) to more quickly evaluate newly acquired imaging information based on a combination of newly acquired imaging information and a summary report of previous findings or treatments provided for one or more representative slices or planes of the region of interest.

[0118] Example 1: Treatment Summary

[0119] Figure 2 Schematically illustrates the input and output of the method according to the arrangement. Having generally described the method, it will now be described in more detail in relation to representative 2D images from radiotherapy treatments or 2D images with annotations from previous tumor assessment examinations, as regarding Figure 3apply such methods to the specific embodiments related to the cases (of those images described).

[0120] Figure 2 A representative 2D image 2000 that can be extracted from a storage medium is shown. The 2D image can include parts reported by the patient, screen capture results, parts of the electronic medical record EMR, etc. In this example, the 2D image can include a treatment plan summary. The image 2000 includes a composite image formed based on the basic image data and the treatment plan related to radiotherapy. Treatment isodose lines 2100 can be seen forming part of the image 2000. The basic image includes one or more features 2200 of the area under study.

[0121] According to the method according to the arrangement, a representative 2D RGB image 2000 (such as a previously acquired CT including a color dose distribution) is decomposed into (i) a grayscale slice (S) representing the planning image, and (ii) a colored geometric dose feature (GDF). For example, a simple RGB decomposition of the 2D RGB image can be used and pixels with unequal R, G, and B values can be detected to extract the colored isodose lines from the representative 2D RGB image. Appropriate mathematical optimization techniques can be used to extract the transparent color patches of the alpha-blended image.

[0122] Figure 2 Schematically illustrates a volume 2500 of 3D imaging data available with respect to the area under study. The volume 2500 of image data can include, for example, a series of image "slices" obtained via appropriate CT or MRI techniques.

[0123] According to the method according to the arrangement, the basic image data from the 2D image 2000 and the 3D volume of subsequent images are processed such that a transformation T can be calculated. T maps S (the basic image data related to the 2D image 2000) into the 3D volume 2500 (V) of the subsequent images. The transformation T is calculated based on determining a plane P in V that can be considered to correspond to the representative slice S. A slice-to-volume registration or a simple 2D-to-2D registration can be used to determine the transformation T, each technique aiming to determine the index of the slice in V that best matches S. It should be understood that it may not be possible to find a slice in V 2500 that exactly matches the feature 2200 of the basic image data of the 2D image. This may be because the capture of the 2D image and the 3D volume of the image data is temporally spaced apart and the feature 2200 has changed. In Figure 2 the example shown, for example, there are additional features 2600. However, according to the arrangement, the best-fitting slice or plane of the image data 2500 is found.

[0124] Once T is calculated, T can be applied to the representative 2D color geometric dose feature 2100 (or other annotations) of the 2D image 2000. In other words, T(GDF) can be calculated.

[0125] T(GDF) can be superimposed on the determined plane or slice P, as shown in Figure 2 the newly created composite image 2700 in. The newly created 2D composite image 2700 allows visualization of the dose features known relative to the 2D image 2000 on P (the representative slice corresponding to the base image of the 2D image 2000 in the 3D volume image data 2500). In addition to the transformation 2800 of dose information such as dose lines 2200, T(GDF) can also include additional annotations or other information, so superimposing T(GDF) on P can support labeling the determined plane or slice P with the planned dose level and / or those additional annotations or other information.

[0126] The arrangement can allow a user (e.g., a radiologist) to see the geometric dose features corresponding to the (one or more) representative CT planning images superimposed on the newly acquired imaging information. For example, according to the arrangement, the user can label the P image forming part of the newly acquired imaging data according to the previously planned and applied treatment dose. Such labeling can allow the radiologist to distinguish regions treated at various dose levels (as indicated by the isodose lines). Superimposing the previously treated isodose lines on subsequent images can improve the radiologist's decision-making.

[0127] Example two: Previous examination

[0128] Although the example has been described with respect to radiotherapy applied to a patient, representative 2D images from previous tumor assessment examinations or imaging events can be treated similarly. Figure 3 Schematically illustrates the input and output of a method according to the arrangement. According to this method, the 2D summary image of the previous examination or object assessment is decomposed into a base image and an additional information image, where the additional information image includes: detected and extracted annotations, which include at least one of the following: text or arrow or line or (one or more) colored region of interest (ROI) contours or similar potentially interesting features.

[0129] Figure 3 Shows a representative 2D image 3000 that can be extracted from a storage medium. The 2D image can include parts of a patient report, screen capture results, parts of an electronic medical record EMR, etc. In this example, the 2D image can include image annotations 3100. The image 3000 includes a composite image in the form of annotations 3100 formed from the base image data and additional information. The base image includes one or more features 3200 of the region under study.

[0130] According to the method according to the arrangement, a representative 2D RGB image 3000 (e.g., of previously acquired CT slices including additional information in the form of image annotations) is decomposed into (i) a grayscale slice (S) representing the planning image and (ii) an additional information image (ADD). For example, a simple RGB decomposition of the 2D RGB image and detection of pixels with unequal R, G, and B values can be used to extract the colored annotations from the representative 2D RGB image.

[0131] Figure 3 Schematically illustrates a volume 3500 of 3D imaging data available with respect to the area under study. The volume 3500 of image data can include, for example, a series of image "slices" obtained via a suitable CT or MRI technique.

[0132] According to the method according to the arrangement, the basic image data from the 2D image 3000 and the 3D volume of subsequent images are processed such that a transformation T can be calculated. T maps S (the basic image data related to the 2D image 3000) into the 3D volume 3500 (V) of subsequent images. T is calculated based on determining a plane P in V that can be considered to correspond to the representative slice S. The transformation T can be determined using slice-to-volume registration or simple 2D-to-2D registration, each technique aiming to determine the index of the slice in V that best matches S. It should be understood that it may not be possible to find a slice in V 3500 that exactly matches the feature 3200 of the basic image data of the 3D image 3000. This may be because the capture of the 3D image and the 3D volume of image data are temporally separated and the feature 3200 has changed. According to the arrangement, the best-fitting slice or plane of the image data 3500 is found.

[0133] Once T is calculated, T can be applied to the representative 2D additional information 3100 (or other annotations) of the 2D image 3000. In other words, T(ADD) can be calculated.

[0134] T(ADD) can be superimposed on the determined plane or slice P, as shown in Figure 3 the newly created composite image 3700. The newly created 2D composite image 3700 allows visualization of the dose features known with respect to the 2D image 3000 (the representative slice in the 3D volume image data 3500 corresponding to the basic image of the 2D image 3000) on P. In addition to the transformation 3800 of the additional information, T(ADD) can also include additional annotations or other information, so superimposing T(ADD) on P can support labeling the determined plane or slice P with those additional annotations or other information known about the 2D image 3000.

[0135] The arrangement can allow a user (e.g., a radiologist) to see geometric dose features corresponding to representative 2D images of previous examinations or evaluations superimposed on newly acquired imaging information. For example, according to the arrangement, the user can label P-images that form part of the newly acquired imaging data based on previously known annotations or measurements. Such labeling can allow the radiologist to evaluate the progression of a disease or changes in the area of interest, thus supporting improved decision-making by medical professionals.

[0136] Figure 4 is a schematic representation of a system for mapping additional information related to a 2D image of an area of interest to a 3D image volume of the area of interest and generating a new composite 2D image of the area of interest as described herein or otherwise contemplated.

[0137] System 4000 includes one or more of the following: a processor 4100, a memory 4200, a user interface 4300, a communication interface 4500, and a storage device 4600, which are interconnected via one or more system buses 4700.

[0138] In some embodiments (e.g., those in which the system is part of or communicates with imaging hardware or an imaging platform), the hardware can include additional imaging hardware (not shown). It should be understood that Figure 4 constitutes an abstraction of system 4000, and the actual organization of the components of system 4000 may differ from what is shown.

[0139] According to an embodiment, system 4000 includes a processor 4100 capable of executing instructions stored in memory 4200 or storage device 4600 or otherwise processing data. Processor 4100 can be configured to execute Figure 5 one or more steps of the method schematically illustrated in and can include one or more modules described herein or otherwise contemplated. Processor 4100 can be formed by one or more modules and can include, for example, memory 4200. Processor 4100 can take any suitable form, including but not limited to: a microprocessor, a microcontroller, multiple microcontrollers, circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a single processor, or multiple processors.

[0140] The memory 4200 can take any suitable form, including non-volatile memory and / or RAM. The memory 4200 may include various memories, such as, for example, a cache memory or a system memory. As such, the memory 4200 may include static random access memory (SRAM), dynamic RAM (DRAM), flash memory, read-only memory (ROM), or other similar memory devices. The memory is capable of storing an operating system, etc. The processor uses the RAM for temporary storage of data. According to an embodiment, the operating system may contain code that, when executed by the processor, controls the operation of one or more components of the system 4000. It is obvious that in embodiments where the processor implements one or more of the functions described herein in hardware, the software described in other embodiments as corresponding to such functions may be omitted.

[0141] The user interface 4300 may include one or more devices for enabling communication with a user (e.g., an administrator, an imaging or medical professional). The user interface may include any device or system that allows for the transmission and / or reception of information, and may include a display, a mouse, and / or a keyboard for receiving user commands. In some embodiments, the user interface 4300 may include a command line interface or a graphical user interface that can be presented to a remote terminal via the communication interface 4500. The user interface 4300 may be co-located with one or more other components of the system or may be located remotely from the system and communicate via a wired and / or wireless communication network.

[0142] The communication interface 4500 may include one or more devices for enabling communication with other hardware devices. For example, the communication interface 4500 may include a network interface card (NIC) configured to communicate according to the Ethernet protocol. Additionally, the communication interface 4500 may implement a TCP / IP stack for communicating according to the TCP / IP protocol. Various alternative or additional hardware or configurations for the communication interface 4500 will be obvious.

[0143] The storage device 4600 may include one or more machine-readable storage media, such as, for example, read-only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, or similar storage media. In various embodiments, the storage device 4600 may store instructions for execution by the processor 4100 or data that the processor 4100 may operate on. For example, the storage device 4600 may store an operating system for controlling various operations of the system 4000.

[0144] Although system 4000 is shown as including one component of each of the described components, various components may be replicated in various embodiments. For example, processor 4100 may include multiple microprocessors that are configured to independently execute the methods described herein or are configured to execute steps or subroutines of the methods described herein such that the multiple processors cooperate to achieve the functions described herein. Additionally, in the case where system 4000 is implemented in a cloud computing system, various hardware components may belong to separate physical systems. For example, processor 4100 may include a first processor in a first server and a second processor in a second server. Many other variations and configurations are also possible.

[0145] According to an embodiment, processor 4100 includes one or more modules to perform one or more functions or one or more steps of the methods described herein or otherwise contemplated. For example, processor 4100 may include: a decomposition module 4700, a slice evaluation module 4750, a transformation module 4800, and / or an image generation module 4850.

[0146] Figure 5 is a flowchart of method 5000 for mapping additional information related to a 2D image of a region of interest to a 3D image volume of the region of interest and generating a new composite 2D image of the region of interest.

[0147] At step 5100, a system for such mapping and image generation is provided. The system may be any system described herein or otherwise contemplated and may include any component or module described herein or otherwise contemplated.

[0148] At step 5200 of the method, the system receives one or more 2D composite images of the region of interest or one or more 2D composite images of the region of interest are provided to the system.

[0149] At step 5300 of the method, the system receives a 3D image volume of the region of interest or a 3D image volume of the region of interest is provided to the system.

[0150] At step 5400 of the method, the system is configured to decompose the 2D composite image into at least the following, for example via steps taken by the decomposition module: a base image; and an additional image including additional information.

[0151] At step 5500 of the method, the system is configured to identify a representative 2D slice in the 3D image volume corresponding to the base image based on an evaluation of the base image and the 3D image volume, for example via steps taken by the slice evaluation module.

[0152] At step 5600 of the method, the system is configured to evaluate a transformation of the base image that maps the base image to representative 2D slices, for example, via steps taken by a transformation module.

[0153] At step 5700 of the method, the system is configured to generate a new composite 2D image of the region of interest by combining the representative 2D slices with an image formed by applying the evaluated transformation to an additional image, for example, via steps taken by an image generation module.

[0154] Those skilled in the art will readily recognize that the steps of the various above-described methods can be performed by a programmed computer. In this document, some embodiments also aim to cover a program storage device (e.g., a digital data storage medium) that is machine or computer-readable and encodes a program of machine-executable or computer-executable instructions, wherein the instructions perform some or all of the steps of the above-described methods. The program storage device can be, for example, a digital memory, a magnetic storage medium (e.g., disks and tapes), a hard disk drive, or an optically readable digital data storage medium. Embodiments also aim to cover a computer programmed to perform the steps of the above-described methods. The term "non-transitory" as used herein is a limitation on the medium itself (i.e., tangible, rather than a signal), rather than a limitation on data storage persistence (e.g., RAM versus ROM).

[0155] While example embodiments of the present invention have been described in the preceding paragraphs with reference to various examples, it should be understood that the examples given can be modified without departing from the scope of the claimed invention.

[0156] The features described in the foregoing description can be used in combinations other than the explicitly described combinations.

[0157] While functions have been described with reference to certain features, these functions can be performed by other features whether or not they are described.

[0158] While features have been described with reference to certain embodiments, these features can also be present in other embodiments whether or not they are described.

[0159] While in the foregoing specification efforts have been made to draw attention to those features of the invention that are considered particularly important, it should be understood that the applicant claims protection for any patentable feature or combination of features mentioned above and / or shown in the drawings, whether or not they have been specifically emphasized.

[0160] By studying the drawings, the disclosure, and the appended claims, those skilled in the art will be able to understand and implement other variations of the disclosed embodiments when practicing the claimed invention. In the claims, the word "comprising" does not exclude other elements or steps, and the words "a" or "an" do not exclude a plurality. A single processor or other unit may implement the functions of several items recited in the claims. The fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used advantageously. A computer program may be stored / distributed on a suitable medium (e.g., an optical storage medium or a solid-state medium provided together with other hardware or as part of other hardware), but may also be distributed in other forms (e.g., via the Internet or other wired or wireless telecommunication systems). Any reference signs in the claims should not be construed as limiting the scope.

Claims

1. An apparatus (4000) configured to map additional information related to a 2D medical image of a region of interest to a 3D medical image volume of the region of interest, the apparatus comprising: at least one processor (4100); and at least one memory (4600, 4200) storing instructions which, when executed by the at least one processor, cause the apparatus to at least perform the following operations: receive (5200) the 2D composite medical image of the region of interest; receive (5300) the 3D medical image volume of the region of interest; decompose (5400) the 2D composite medical image into at least: a base image; and an additional image including the additional information; identify (5500) a representative 2D slice in the 3D medical image volume corresponding to the base image based on an evaluation of the base image and the 3D image volume; evaluate (5600) a transformation of the base image that maps the base image to the representative 2D slice; and combine (5700) the representative 2D slice with an image formed by applying the evaluated transformation to the additional image to produce a new composite 2D medical image of the region of interest.

2. The device according to claim 1, wherein The 2D medical image of the region of interest includes: a color image.

3. The device according to claim 1 or claim 2, wherein, The composite 2D medical image includes: a 2D summary image representing the region of interest in the 3D medical image volume.

4. The apparatus according to any one of the preceding claims, wherein, The composite 2D medical image includes: additional information related to a treatment or procedure performed on the region of interest.

5. The apparatus according to any one of the preceding claims, wherein, The additional information includes: a geometric distribution of a treatment dose applied to the region of interest.

6. The device according to claim 5, wherein, The treatment includes radiotherapy.

7. The device according to claim 5 or claim 6, wherein, The geometric distribution includes: isodose lines or color - mapped dose information.

8. The device according to any one of the preceding claims, wherein, The additional information includes: one or more annotations, the one or more annotations including: a contour of an interesting feature; or a marker of an interesting feature; or a measurement result of an interesting feature.

9. The apparatus according to any one of the preceding claims, wherein, Decomposing the 2D composite medical image (2000, 3000) includes: decomposing the 2D composite image into at least: a grayscale base image; and an additional image (2100, 3100) including the additional information.

10. The device according to any one of the preceding claims, wherein, Decomposing the 2D composite medical image includes: analyzing pixels of the 2D composite medical image to determine the grayscale base image and the additional image including the additional information.

11. The apparatus according to any one of the preceding claims, wherein, Identifying the representative 2D slice (2700, 3700) in the 3D medical image volume (2500, 3500) corresponding to the base image includes: evaluating one or more planar images forming the 3D image volume for the base image, and selecting the planar image that best matches the base image among the evaluated one or more planar images.

12. The apparatus according to any one of the preceding claims, wherein, Identifying the representative 2D slice (2700, 3700) of the 3D medical image volume (2500, 3500) and evaluating the transformation of the base image that maps the base image to the representative 2D slice includes: using slice - to - volume or 2D - to - 2D registration image registration techniques.

13. The apparatus according to any one of the preceding claims, wherein, Evaluating the transformation of the base image that maps the base image to the representative 2D slices (2700, 3700) includes: evaluating one or more differences between those images resulting from one or more of rotation, translation, deformation, or scaling.

14. A computer-implemented method (5000) for mapping additional information related to a 2D image of a region of interest to a 3D medical image volume of the region of interest, the method comprising: Receiving (5200, 5300) the 2D composite medical image of the region of interest and the 3D medical image volume of the region of interest; Decomposing (5400) the 2D composite medical image into at least the following: A base image; and An additional image including the additional information; Identifying (5500) representative 2D slices in the 3D medical image volume corresponding to the base image based on an evaluation of the base image and the 3D medical image volume; Evaluating (5600) the transformation of the base image that maps the base image to the representative 2D slices; And Combining the representative 2D slices with an image formed by applying the evaluated transformation to the additional image to produce (5700) a new composite 2D medical image of the region of interest.

15. A computer program product that is operable to perform the method according to claim 14 when executed on a computer.

Images