Method for generating computer-based visualization of 3D medical image data

Abstract

A method for generating a computer-based visualization of 3D medical image data is described. The method comprises receiving 3D medical image data and performing a selection process to select first image data forming a first portion of the 3D medical image data, the first image data representing a first anatomical object of a given type. An analysis process is performed on the first image data, wherein parameters of the analysis process are based on the given type of the first anatomical object. A visual parameter map is determined with respect to the first portion based at least in part on a result of the analysis process for use in a rendering process to generate the visualization of the 3D medical image data. A method of generating a computer-based visualization of 3D medical image data and a device for performing the method are also described.

Description

TECHNICAL FIELD

[0001] The present invention relates to a method for generating a computer-based visualization of 3D medical image data. BACKGROUND

[0002] A computer-based visualization of a dataset representing a volume can be generated using a technique commonly referred to as volume rendering. Such a dataset can be referred to as a volumetric dataset. For example, volume rendering can be used to visualize a volumetric dataset resulting from a medical imaging process such as a CT scan process. In the medical field, volume rendering can enable a radiologist, surgeon or therapist to visualize data representing anatomical structures and thereby understand and interpret that data. For example, providing a visualization of such data can be used for diagnosis, teaching, patient communication and so on.

[0003] Generally, a volume rendering technique involves applying a mapping of visual parameters to image data forming a volumetric dataset being rendered, for example via classifying the image data by applying a transfer function. The mapping provides one or more visual parameters to be assigned to the image data. The assigned visual parameters can then be used to generate a visualization of the volumetric dataset. For example, a volumetric dataset can comprise a plurality of voxels and a mapping process can be performed to assign visual parameter data such as opacity and color to each voxel. The visual parameters assigned by the mapping process can then be used in a volume rendering technique to generate a visualization of the volume. For example, an integral-based direct volume rendering technique can be performed in which one or more sample rays are projected through the volume for each pixel in the visualization to be generated. In such a technique, each ray can be sampled at a plurality of points to compute an integral based on the visual parameter data and the result of the integral can be used to determine a color value for the corresponding pixel. SUMMARY

[0004] According to a first aspect of the present invention, there is provided a method for generating a computer-based visualization of 3D medical image data, the method comprising: receiving 3D medical image data; performing a selection process to select first image data forming a first portion of the 3D medical image data, the first image data representing a first anatomical object of a given type; performing an analysis process on the first image data, wherein a parameter of the analysis process is based on the given type of the first anatomical object; and determining a mapping of visual parameters for the first portion based at least in part on a result of the analysis process for use in a rendering process to generate a visualization of the 3D medical image data.

[0005] The analysis process can comprise determining one or more features of the first image data. The one or more features of the first image data can be determined based on the parameter of the analysis process.

[0006] The one or more features of the first image data can comprise one or more features of a distribution of first voxel values of the first image data, and the analysis processing can comprise analyzing the distribution of first voxel values to determine the one or more features of the distribution.

[0007] The one or more features of the distribution can comprise a voxel value or a range of voxel values that satisfy a predetermined criterion.

[0008] The predetermined criterion can define a voxel value or a range of voxel values associated with a local maximum or a global maximum in the distribution.

[0009] Determining the visual parameter map can comprise determining a function defining the visual parameter map based on a result of the analysis processing.

[0010] The method can comprise determining parameters of the analysis processing based on a given type of the first anatomical object.

[0011] The parameters of the selection processing can be based on a given type of the first anatomical object represented by the first image data.

[0012] The parameters of the selection processing can be determined based on context information related to the 3D medical image data.

[0013] The parameters of the analysis processing can be determined based on context information related to the 3D medical image data.

[0014] The context information related to the 3D medical image data can for example be one or more of: textual information identifying a medical context of the 3D medical image data; medical history information associated with the 3D medical image data.

[0015] The visual parameter map can be a transfer function for a volume rendering processing.

[0016] The transfer function can be configured to provide opacity values and / or color values of the first image data for use in the volume rendering processing.

[0017] The first anatomical object can comprise an anatomical organ.

[0018] The 3D medical image data can comprise a plurality of 3D medical image data sets, and the selection processing can comprise: selecting a first 3D medical image data set of the plurality of 3D medical image data sets; identifying a portion of the first 3D medical image data set representing the first anatomical object; selecting a second 3D medical image data set of the plurality of 3D medical image data sets; and selecting the first image data from the second 3D medical image data set based on the identified portion of the first 3D medical image data set.

[0019] According to a second aspect of the present invention, there is provided a method of generating a computer-based visualization of 3D medical image data, the method comprising: performing the method according to the first aspect to obtain a visual parameter map for a first portion of the 3D medical image data; and performing a rendering process to generate a visualization of the 3D medical image data, wherein performing the rendering process comprises applying the visual parameter map for the first portion of the 3D medical image data.

[0020] According to a third aspect of the present invention, there is provided a set of machine readable instructions which, when executed by a processor, cause the method according to the first aspect or the second aspect to be performed.

[0021] According to a fourth aspect of the present invention, there is provided a machine readable medium comprising a set of machine readable instructions according to the third aspect.

[0022] According to a fifth aspect of the present invention, there is provided a device comprising a processor and a memory, the memory comprising a set of machine readable instructions which, when executed by the processor, cause the processor to perform the method according to the first aspect or the second aspect.

[0023] According to another aspect, there is provided a computer-implemented method for providing a visualization object. In this case, the visualization object visualizes a three-dimensional anatomical region of a patient for a user, the three-dimensional anatomical region being represented by medical volume data (or 3D medical image data). The method comprises the following steps:

[0024] receiving a selection command of a user, the selection command indicating a patient to be analyzed (or examined or diagnosed);

[0025] based on the selection command, invoking data assigned to the patient;

[0026] determining a medical context information item based on the assigned data;

[0027] selecting suitable volume data of the patient based on the medical context information item and optionally the selection command;

[0028] identifying one or more (anatomical) structures in the selected volume data based on the medical context information item;

[0029] determining a mapping rule for mapping the volume data onto a visualization object of the user, wherein the mapping rule is determined taking into account the medical context information item and / or the identified structures;

[0030] computing the visualization object based on the mapping rule;

[0031] providing the visualization object to the user.

[0032] In other words, when a patient case is called, a visualization of the volume data is automatically generated that is adapted to the situation of the individual case. In particular, the relevant (anatomical) structures can be visualized in a targeted manner by suitable mapping rules. This enables individual structures to be represented individually. At the same time, the adjustments and adaptations to be made by the user are reduced, further improving the usability.

[0033] Thus, a computer-implemented method and a device for providing a visualization object are provided, which is adapted to visualize a three-dimensional anatomical region of a patient represented by volume data for a user. In this process, a medical context information item (or context information) is derived for the individual case, thus facilitating the automatic and targeted definition of a suitable visualization object. In particular, one or more structures to be visualized are identified in the selected volume data on the basis of the context information item, in particular, these structures are relevant to the user on the basis of the context data. Then, when calculating the visualization object, these structures to be visualized can be automatically taken into account on an individual basis, which can improve the result. In particular, it is thus possible to determine individually adjusted visualization parameters for each structure to be visualized, by means of which the individual structures to be visualized can be reproduced in an optimal manner.

[0034] In this context, a structure can in particular be an organ, an anatomical structure, a tissue structure, an implant, a tissue change, etc. in the anatomical region of a patient. In other words, the identified structures can be referred to as structures to be represented or structures to be visualized. Another expression for (anatomical structures) can be "anatomical objects".

[0035] In particular, the user can be the recipient of the visualization (object) and thus the person for whom the visualization is created. In particular, the user can be a doctor or a patient.

[0036] In particular, the visualization object can comprise a two-dimensional visualization image or a time-resolved sequence of a plurality of individual visualization images.

[0037] Volume data can comprise a plurality of voxels. A voxel ("volume pixel" or three-dimensional pixel) is a volume element representing a value on a regular grid in three-dimensional space. A voxel is analogous to a pixel, representing two-dimensional image data. As in the case of pixels, the voxels themselves typically do not contain their position in space (their coordinates), but the coordinates of the voxels are derived based on their position relative to other voxels (i.e. their position in the data structure forming a single volume image). The values of the voxels can represent different physical properties of a three-dimensional object, such as, for example, local density. In computed tomography recordings (CT scans), these values are represented, for example, in Hounsfield units, which represent the opacity of the imaged material relative to X-rays. Thus, the volume data describes a three-dimensional anatomical region in a patient's body. In particular, the volume data can specify the density (in particular, the inhomogeneous density) of the anatomical region. Another expression of the volume data can be 3D medical image data.

[0038] In particular, the volume data can be provided by a medical imaging method. For example, the imaging method can be based on fluoroscopy, computed tomography (CT), magnetic resonance imaging (MRI), ultrasound and / or positron emission tomography (PET). Thus, the three-dimensional object can be a patient's body or a body part. In this case, the three-dimensional object can comprise one or more organs of the patient.

[0039] Furthermore, the volume data can be four-dimensional data having three spatial dimensions and one temporal dimension. Furthermore, the volume data can comprise a plurality of individual volume data recordings, in particular each volume data recording can be generated by a different imaging modality.

[0040] In this case, the mapping of the volume data by means of the mapping rule can be implemented by means of an image synthesis algorithm. In particular, the image synthesis algorithm can be considered a computer program product which is embodied as mapping the volume data onto a two-dimensional projection surface or rendering a volume of a three-dimensional volume or performing volume rendering of a three-dimensional volume. In this case, the projection surface is given by a visualization image. The image synthesis algorithm can have program components in the form of one or more instructions of a processor for computing the visualization image. The visualization image consists of a plurality of visualization pixels. In particular, the resolution of the visualization image in relation to the visualization pixels can be spatially constant or uniform or spatially uniform. Other terms for the image synthesis algorithm include, for example, "renderer", "rendering algorithm" or "volume renderer". The image synthesis algorithm can be provided, for example, typically by means of being kept available in a memory device or being loaded into a main memory of a suitable data processing device or being provided for an application.

[0041] Herein, the image synthesis algorithm can implement various methods for visualizing the volume data record individually or in combination. For example, the image synthesis algorithm can comprise a ray casting module and / or a path tracing module.

[0042] For example, providing the volume data can comprise keeping the volume data available in a memory device, such as a database, and / or calling the volume data from a memory device, such as a database, and / or loading the volume data into, for example, a main memory of a suitable data processing device, or generally making available for application or use.

[0043] For example, providing the assigned data can comprise keeping the context data available in a memory device, such as a database, and / or calling the context data from a memory device, such as a database, or loading the assigned data into, for example, a main memory of a suitable data processing device, or generally making available for application or use. In particular, the assigned data is different from the volume data. The assigned data can relate to information items related to the visualization. For example, the assigned data can indicate which perspectives of an object to be visualized, transfer functions or which parts of an object are of particular relevance for the visualization. In particular, the assigned data can contain natural language. For example, structures of particular relevance for the visualization can be disclosed in writing in the assigned data. For example, if a medical report, as a form of assigned data, contains an explanation related to the liver of a patient, it can be concluded that this organ should be predominantly displayed in the visualization. In particular, the assigned data can comprise non-image data.

[0044] In particular, the assigned data can be assigned to the volume data record by means of a link to the same patient. For example, the assigned data can comprise one or more medical reports, a letter from a physician, a record of a consultation with other users or patients, medical history, laboratory data and / or demographic information items about the patient. The assigned data can be available, for example, in the form of an electronic patient record and can be stored in a suitable information system, for example: a hospital information system. Furthermore, the assigned data can comprise general relevant information items of the patient, for example one or more guidelines and / or one or more electronic textbooks or outlines. Furthermore, the assigned data can relate to the user and may, for example, indicate one or more user preferences.

[0045] In particular, the mapping rule can be understood as an instruction how to properly visualize the volume data in the context of the medical context information item. Another expression of the mapping rule can be a "visual parameter mapping". Going back to the example above, a dedicated view angle, a dedicated scene lighting and a dedicated color scheme for imaging the volume data can be selected for the visualization of the liver. Furthermore, less relevant regions of the volume data in the visualization can be omitted or cut off or represented transparently. The mapping rule can comprise representation parameters (or visual parameters or transfer parameters). The representation parameters are typically very complex. The representation parameters assign certain colors, transparencies, contrasts, brightness, sharpness, etc. to each gray value in the three-dimensional volume. Generally speaking, the representation parameters influence the type of representation of objects of the respective object type in the visualization image output to the user. In other words, the mapping rule can comprise one or more transfer functions.

[0046] In particular, the medical context information item (or context information) can be a medical problem which the user has to solve in the case of the patient. Furthermore, the context information item can comprise an indication of a medical diagnosis, demographic information items about the patient, next steps in a medical guideline, etc.

[0047] According to some examples, the identifying is implemented based on a segmentation of the volume data, preferably based on the medical context information item.

[0048] The segmentation allows for relevant structures to be selected, wherein the relevance can be weighted by the context information item. In particular, a shape segmentation can be used within the scope of the segmentation. For example, a segmentation mask can be used in order to identify structures in the volume data, such as the lungs of the patient. All or only some of the segmented structures can be identified as structures to be represented. Preferably, a dynamic selection can be made from the set of segmented structures based on the context information item in the identifying step.

[0049] According to some examples, the mapping rule comprises a partial mapping rule for each identified structure, and the determining step further comprises specifically optimizing (in other words, or adjusting) the partial mapping rule for each identified structure, wherein the optimization (adjustment) of each partial mapping rule is implemented independently from the respective other partial mapping rules.

[0050] By using multiple partial mapping rules, the visualization of each structure can be specifically optimized, and this can lead to a significantly better representation of the individual structures compared to a global mapping rule for all structures. Furthermore, this facilitates a simple dynamic adjustment of the visualization object when the modified medical context information item indicates different structures to be identified. Each partial mapping rule can comprise individual representation parameters which assign a specific color, transparency, contrast, brightness, sharpness, etc. to each voxel value. In other words, the partial mapping rule can comprise at least one individual transfer function.

[0051] According to some examples, the adjustment / optimization for each identified structure comprises extracting an image information item from the volume data, in particular from the volume data assigned to the respective identified structure, and adjusting the partial mapping rule based on the image information item.

[0052] By evaluating the image information item, it is not only possible to adapt the mapping rule to the clinical environment, but also to the respective condition of the patient and the recording parameters used when recording the volume data, in particular in a selective manner for each structure, since this occurs for each partial mapping rule. According to the application, this makes it possible to take into account the fact that the image information item of a structure behaves differently between different recordings and different patients. This not only facilitates a structure-specific adjustment of the mapping rule, but also a patient-specific or recording-specific adjustment of the mapping rule and thus a multi-dimensional adjustment of the mapping rule.

[0053] According to some examples, the method further comprises the step of providing one or more recording parameters describing one or more conditions under which the volume data was generated, wherein the recording parameters are taken into account when adjusting the partial mapping rule. For example, the recording parameters can comprise a kV specification in the case of a CT recording or an MRI sequence in the case of an MRI recording. Taking the recording parameters into account makes it possible to better adjust the partial mapping rule to be implemented.

[0054] According to some examples, the adjustment of the partial mapping rule can be implemented as an intensity-based segmentation. Taking the image information item into account allows the relevant region of the structure to be better delimited from surrounding structures, for example starting from a shape-based segmentation, and thus to be well calculated for visualization.

[0055] In an alternative, in addition to the volume data of the respective structure, volume data of the structure can also be used which is compared to the remaining anatomical region or other identified structures.

[0056] According to some examples, the image information item comprises a statistical frequency or distribution of image values (voxel values) of the volume pixels belonging to the identified structure. For example, color values or gray scale values can be evaluated as image values here.

[0057] According to some examples, the volume data is generated at least partially using a computed tomography method, and the image information item comprises a statistical frequency or distribution of Hounsfield units (HU).

[0058] This facilitates the simple capture of different contributions in CT imaging and thus the good adjustment of the partial mapping rule, for example by a further intensity-based or contrast-based selection of the voxels to be represented.

[0059] In MR imaging, the contrast varies more significantly than in CT. The contrast behavior is set by the MR sequence and the measurement protocol. The anatomical segmentation has to take into account the modeling or robustness to this variability. Of course, the anatomical shape features of organs or structures are present in MR imaging as well. Shape recognition based segmentation can be directly used in the MRI domain. If the intensity characteristics should be taken into account as well, the segmentation needs to be properly parameterized. Depending on the implementation, the parameterization can be achieved by:

[0060] - existing knowledge about MR imaging;

[0061] - mathematical-physical derivation from the properties of the tissue taking into account the field strength, sequence and protocol and in simple cases with the help of the Bloch equation; and / or

[0062] - available reference images / databases.

[0063] In another case, such segmentation can also be performed with the help of so-called MR fingerprinting methods, which essentially contain the multi-contrast behavior of the imaging sequence itself.

[0064] According to some examples, the adaptation to each identified structure further comprises determining at least two image information contributions in the image information item and adapting the partial mapping rule based on the image information contributions.

[0065] The image information contributions can be produced, for example, by "fitting" one or more characteristic functions to the image information item, such as the distribution of the voxel values. The image information contributions can originate from different tissue types such as bone or soft tissue. By determining the image information contributions, the partial mapping rule can be optimized in a more targeted manner.

[0066] According to some examples, the adaptation to each identified organ / structure further comprises a comparison of the image information item with a reference image information item and an adaptation of the partial mapping rule based on the comparison.

[0067] The reference image information item can be associated with a partial mapping rule that is optimal for this purpose. In a comparative manner, a deviation between the image information item and the reference image information item can be found, which in turn can indicate a possible adaptation of the optimal partial mapping rule to the available volume data.

[0068] According to some examples, determining the mapping rule further comprises selecting one or more partial mapping rules from a pool of partial mapping rules based on the identified structures and / or the assigned data.

[0069] According to some examples, the adjusting of the partial mapping rules with respect to the structure to be displayed in each case comprises selecting preset partial mapping rules assigned to the respective structure to be displayed and adjusting the preset partial mapping rules in order to create the adjusted partial mapping rules.

[0070] Thus, it has become possible to find a good starting point for the adjustment of the partial mapping rules.

[0071] According to some examples, in each case, the partial mapping rules are adjusted taking into account the medical context information items and / or the associated data, thereby ensuring a targeted adjustment of the partial mapping rules.

[0072] According to some examples, the adjustment of the partial mapping rules is implemented by applying a trained function, in particular, which is implemented to provide the partial mapping rules with respect to each identified organ based on the medical context information items and / or the assigned data in particular.

[0073] Generally, a trained function maps input data onto output data. In particular, in this case, the output data can also depend on one or more parameters of the trained function. The one or more parameters of the trained function can be determined and / or adjusted by training. In particular, determining and / or adjusting the one or more parameters of the trained function can be based on a pair of training input data and associated training output data, wherein the trained function is applied to the training input data to generate training imaging data. In particular, the determination and / or adjustment can be based on a comparison of the training imaging data and the training output data. Generally, a trainable function, i.e. a function with parameters that have not yet been adjusted, is also referred to as a trained function.

[0074] Other terms for trained functions include trained mapping rules, mapping rules with trained parameters, functions with trained parameters, artificial intelligence based algorithms and machine learning algorithms. Artificial neural networks are an example of trained functions. Instead of the term "artificial neural network", the term "neural network" can also be used. In principle, neural networks are constructed like biological neural networks, for example the human brain. In particular, artificial neural networks comprise an input layer and an output layer. Artificial neural networks can also comprise a plurality of layers between the input layer and the output layer. Each layer comprises at least one node, preferably a plurality of nodes. Each node can be understood as a biological processing unit, for example a neuron. In other words, each neuron corresponds to an operation that is applied to input data. The nodes of one layer can be connected to the nodes of other layers by edges or connections, in particular by directed edges or connections. These edges or connections define a data flow between the network nodes. The edges or connections are associated with a parameter, often referred to as "weight" or "edge weight". This parameter can adjust the importance of the output of a first node for the input of a second node, wherein the first node and the second node are connected by an edge.

[0075] In particular, neural networks can be trained. In particular, the training of a neural network is performed based on training input data and associated training output data according to a "supervised" learning technique ("supervised learning" is a professional term), wherein known training input data is input into the neural network and the output data generated by the network is compared with the associated training output data. As long as the output data of the last network layer does not sufficiently correspond to the training output data, the artificial neural network learns and independently adjusts the edge weights of the individual nodes.

[0076] In particular, the trained function can also be a deep artificial neural network ("deep neural network" and "deep artificial neural network" are professional terms).

[0077] According to some examples, the computation is implemented using a volume rendering algorithm, in particular a method based on ray casting and / or path tracing, and the mapping rule has one or more transfer functions.

[0078] Such a method enables the generation of particularly realistic visualizations, thereby increasing the practicality of the method. The specified method is complex in application. However, by taking into account the context data according to the application, the specified method can be easily operated and the best possible visualization is automatically provided.

[0079] According to some examples, the mapping rule is implemented such that the identified organ is emphasized in the visualization object to the user. In particular, this can be achieved by making other image components of the volume data non-visualizable.

[0080] According to some examples, the mapping rule has a global mapping rule which defines one or more overall scene properties of the visualized image. For example, these scene properties can relate to perspective, magnification or scene illumination, which are consistently applied to all identified structures, whereby a consistent image impression is generated although individual structures are emphasized.

[0081] According to some examples, the mapping rule is implemented in such a way that the visualized object comprises a time-resolved sequence of multiple individual images. In this case, at least two of the individual images can represent different viewing angles of the volume data record.

[0082] Thus, a user can also be provided with a video material, for example, whereby the temporal relationship or the geometric relationship becomes easier to understand (in particular, the geometric relationship becomes easier to understand when different viewing angles are used in the individual images).

[0083] According to some examples, the method further comprises a step of receiving a user input on the item of medical context information, wherein the item of medical context information is additionally determined based on the user input.

[0084] According to some examples, the user input can comprise a voice input of the user, which can be evaluated by a computer linguistics algorithm. For example, this can enable simple voice control of the method.

[0085] According to some examples, the selected volume data comprises a first volume data record recorded by a first imaging modality and comprises a second volume data record recorded by a second imaging modality different from the first imaging modality. The method then further comprises registering the first volume data record with the second volume data record, wherein the step of determining the mapping rule is additionally implemented based on the registration.

[0086] The registration can be implemented by marking the patient or by means of registration based on the image data. Now, a broader intensity parameter space is available for identification / segmentation. In this multi-dimensional intensity space, the distance of the organ itself is greater than when only one modality is observed. This greater distance can be used to improve and stabilize the identification / segmentation. To this end, a case-specific parameterization can be used to some extent (for example, explained above in connection with MR data).

[0087] The registered multi-dimensional intensity space can then be used with a multi-dimensional partial mapping rule (transfer function). Thus, the specific contribution of a voxel to the entire image can be implemented by one modality only or by any combination. Optionally, an automatic optimization / partial automatic optimization of the partial mapping rule with respect to the structure to be visualized in the intensity space is advantageous.

[0088] According to some examples, the method further comprises a step of providing body data of patients, wherein the suitable body data is selected from the provided body data.

[0089] The body data can also be provided, for example, by appropriately accessing a suitable archive system, such as a PACS system. By selecting the body data related to the medical question, the suitable initial data can be determined automatically, in turn reducing the burden on the user.

[0090] According to another aspect, a system for providing a visualization object is disclosed. The visualization object represents to a user a three-dimensional anatomy of a patient represented by medical body data. The system comprises:

[0091] - an interface for receiving a selection command of a user and for receiving medical body data, the selection command indicating a patient to be analyzed; and

[0092] - a computing unit implemented to:

[0093] - invoke / retrieve data assigned to the patient based on the selection command;

[0094] - determine medical context information items based on the assigned data;

[0095] - select suitable body data of the patient based on the medical context information items and optionally the selection command;

[0096] - identify one or more organs in the selected body data based on the medical context information items;

[0097] - determine mapping rules for mapping the body data onto the visualization object of the user based on the medical context information items and / or the identified organs;

[0098] - compute the visualization object based on the mapping rules; and

[0099] - provide the visualization object to the user.

[0100] The computing unit can be implemented as a centralized computing unit or as a decentralized computing unit. The computing unit can comprise one or more processors. The processors can be implemented as central processing units (CPUs) and / or graphical processing units (GPUs). The computing unit can be implemented as a so-called system on a chip (SoP) which controls all functions of the apparatus. Alternatively, the computing unit can be implemented as a local processing server or as a cloud-based processing server.

[0101] Generally, the interface can be embodied as an interface for the exchange of data between a computing device and other components. The interface can be implemented in the form of one or more separate data interfaces, which can include hardware interfaces and / or software interfaces, such as a PCI bus, a USB interface, a FireWire interface, a ZigBee interface, or a Bluetooth interface. The interface can also include an interface of a communication network, wherein the communication network can have a local area network (LAN), for example an intranet, or a wide area network (WAN). Thus, the one or more data interfaces can include a LAN interface or a wireless LAN interface (WLAN or Wi-Fi).

[0102] The advantages of the proposed device essentially correspond to the advantages of the proposed method. Features, advantages or alternative embodiments can likewise be transferred to other claimed subject matters and vice versa.

[0103] In another aspect, the present application relates to a computer program product comprising a program and being directly loadable into the memory of a programmable computing unit and having program means (e.g. libraries and auxiliary functions) for executing the method for visualizing a three-dimensional object, in particular according to the aforementioned aspects, when the computer program product is executed.

[0104] Furthermore, the present application relates in another aspect to a computer program product comprising a program and being directly loadable into the memory of a programmable computing unit and having program means (e.g. libraries and auxiliary functions) for executing the method for providing a trained function, in particular according to the aforementioned aspects, when the computer program product is executed.

[0105] Here, the computer program product can comprise software with source code which still needs to be compiled and bound or only interpreted or software with executable software code which only needs to be loaded into a processing unit for execution purposes. As a result of the computer program product, these methods can be executed quickly in a likewise repeatable manner and in a robust manner. The computer program products are configured such that they can execute the method steps according to the present application by means of a computing unit. Here, the computing unit must meet the requirements in each case, for example have a suitable main memory, a suitable processor, a suitable graphics card or a suitable logic unit, so that the respective method steps can be executed efficiently.

[0106] For example, the computer program product is stored on a computer-readable storage medium or saved on a network or a server, from which the computer program product can be loaded into the processor of the corresponding computing unit, which can be directly connected to the computing unit or implemented as part of the computing unit. Furthermore, the control information items of the computer program product can be stored on a computer-readable storage medium. The control information items of the computer-readable storage medium can be implemented in such a way that, when the data medium is used in a computing unit, the control information items carry out the method according to the application. Examples of computer-readable storage media include DVDs, magnetic tapes or USB sticks, on which the electronically readable control information items, in particular software, are stored. If these control information items are read from the data medium and stored in a computing unit, all embodiments of the method according to the application described above can be carried out. The application can thus also be continued from the computer-readable medium and / or from the computer-readable storage medium. The advantages of the proposed computer program product or of the associated computer-readable medium essentially correspond to the advantages of the proposed method. BRIEF DESCRIPTION OF DRAWINGS

[0107] The application will now be described by way of example only, with reference to the following drawings, in which:

[0108] Figure 1 A flowchart representation of a method for generating a computer-based visualization of 3D medical image data according to an example of the present disclosure is shown;

[0109] Figure 2 A schematic illustration of an example volume for which 3D medical image data is obtained is shown, the volume comprising an example anatomical object within the anatomy of a patient;

[0110] Figure 3A A schematic representation of an example distribution of voxel values of first image data forming a portion of the 3D medical image data of Figure 2 is shown;

[0111] Figure 3B A schematic representation of an example visual parameter map determined with respect to the first image data of Figure 3B is shown;

[0112] Figure 4 A flowchart representation of a method for visualizing 3D medical image data according to an example of the present disclosure is shown;

[0113] Figure 5 A flowchart representation of a clinical workflow method according to an example of the present disclosure is shown;

[0114] Figure 6A system comprising a volume rendering device for performing certain example methods according to the present disclosure is schematically illustrated. DETAILED DESCRIPTION

[0115] Modeling, reconstructing or visualizing three-dimensional objects has a wide range of applications in the field of medicine (e.g. CT, PET), physics (e.g. electronic structure of large molecules) or geophysics (condition and relative position of the earth's layers). Typically, the object to be examined is illuminated (e.g. by means of electromagnetic or acoustic waves) in order to examine its condition. The scattered radiation is detected and from the detected values the properties of the subject are determined. Usually, the result comprises physical variables (e.g. density, tissue type, elasticity, velocity) whose values are determined for the subject. Usually, a virtual grid is used here, at the grid points of which the values of the variables are determined. These grid points are usually referred to as voxels. The term "voxel" is a hybrid of the terms "volume" and "pixel". A voxel corresponds to the spatial coordinates of a grid point at which a value of a variable is assigned. Here, this is usually a physical variable which can be represented as a scalar or vector field, i.e. a corresponding field value is assigned to a spatial coordinate. By interpolating the voxels, values of the variable or of the field can be obtained at any object point, i.e. at any point of the object to be examined.

[0116] In order to visualize the volume data, a three-dimensional representation of the object or subject to be examined is generated from the voxels on a two-dimensional representation surface (e.g. a screen or panel or lens of a so-called "augmented reality glasses"). In other words, the voxels (defined in three dimensions) are mapped onto the pixels (defined in two dimensions) of the two-dimensional visualization image. The pixels of the visualization image are also referred to as visualization pixels in the following. The mapping is usually referred to as volume rendering. How the information items contained in the voxels are reproduced by means of the pixels depends on how the volume rendering is performed.

[0117] One of the most commonly used methods of volume rendering is the so-called ray casting (see Levoy: "Display of Surfaces from Volume Data", IEEE Computer Graphics and Applications, Issue 8, Volume 3, May 1988, pages 29 to 37). In ray casting, simulated rays are emitted from the eyes of an imaginary observer through the examined body or the examined object. Along the rays, RGBA values are determined for sample points from the voxels and combined by means of alpha blending or alpha compositing to form the pixels of a two-dimensional image. Here, the letters R, G, B in the expression RGBA stand for the color components red, green and blue, which contribute to the color of the respective sample point. A stands for the ALPHA value, which is a measure of the transparency at the sample point. The respective transparency is used for the superposition of the RGB values at the respective sample point to form the pixel. Illumination effects are usually taken into account by means of illumination models within the scope of a method known as "shading".

[0118] Another method of volume rendering is the so-called path tracing method (see Kajiya: "The rendering equation", ACM SIGGRAPH Computer Graphics, Issue 20, Volume 4, August 1986, pages 143 to 150). Here, a plurality of simulated rays are shot into the volume data of each visualization pixel, which then interact with the volume, i.e. the simulated rays are reflected, refracted or absorbed, wherein at least one random ray is generated each time (except in the case of absorption). Thus, each simulated ray finds its path through the volume data. The more virtual rays are used for each visualization pixel, the better the ideal image can be ground out. Here, in particular, the processes and methods described in EP 3 178 068 B1 can be used. The content of EP 3 178 068 B1 is hereby incorporated by reference in its entirety.

[0119] The user, in particular a user in the medical and clinical field, can be effectively assisted by such a visualization method, since such a representation provides a quick overview of complex anatomical relationships. As an example, this enables better planning of surgical interventions. Furthermore, such a visualization image facilitates the creation of meaningful medical reports.

[0120] A major technical obstacle in implementing a system for interactive volume rendering is that the representation has to be adapted to the respective requirements in a targeted manner when establishing a diagnosis for a specific patient for the purpose of answering a specific clinical question. Conventional systems are able to generate a representation that can best reproduce the entire anatomy of a patient represented by the volume data. However, such a global setting is often not well suited for observing individual structures, because the global setting often does not reproduce the details sufficiently. In this context, an individual structure can be, for example, an organ, an implant, a bone, a blood vessel, a spatial requirement or a pathological tissue change such as, for example, a tumor. For example, if an individual structure should be removed from the overall representation of the rendering in order to be able to better assess the tissue located behind the individual structure, the representation errors sometimes reach the extent of removing too much or too little information. This makes it necessary to make corrections for which the user often has neither the time nor the background knowledge. Thus, despite the inherent potential of the technology, the acceptance of volume rendering algorithms for clinical diagnosis is reduced. In the worst case, an inappropriate selection of global visualization parameters conceals pathologically relevant situations and can lead to false decisions.

[0121] It is therefore an object of the present application to provide an improved method and device for visualizing volume data in this respect. In particular, the problem addressed in this process is to provide a visualization method that enables the volume data to be processed in such a way that the visualization of the volume data can be better adapted to the underlying medical question.

[0122] Figure 1 A flowchart representation of an example method 100 for generating a computer-based visualization of 3D medical image data is shown.

[0123] The method 100 comprises receiving 3D medical image data at block 102. The 3D medical image data can be received by loading from a memory, a sensor, and / or other source. In general, the 3D medical image data can be produced using any scanning modality. For example, the scanning modality can include using computed tomography (CT) or using magnetic resonance imaging (MRI). In some examples, scanning modalities including using positron emission tomography (PET), single photon emission computed tomography (SPECT), ultrasound, or another scanning modality can be used. The 3D medical image data can represent an imaged volume within a human or animal anatomy. Briefly turning to Figure 2 Fig. 1 schematically shows an example of an imaged volume 200 of an anatomy of a patient 10.

[0124] The 3D medical image data can have been acquired using a three-dimensional acquisition process, so the 3D medical image data is inherently in 3D format. As an alternative, the 3D medical image data can be derived from a set of two-dimensional (2D) images from one or more imaging planes, where each 2D image is composed of a plurality of pixels. In some examples, the 3D medical image data can include volume data that has been smoothed by interpolation or the like. In another example, the interpolation process can be performed as part of the process of rendering the image data.

[0125] In an example, the 3D medical image data comprises a plurality of data elements, also referred to as voxels, each data element comprising a value of a measured variable, also referred to as a voxel value. Each voxel corresponds to one of a plurality of locations distributed over the imaged volume 200. Typically, the measured variable corresponds to the type of imaging device used to generate the medical image data. For example, for 3D medical image data generated using an X-ray imaging device, the measured variable can relate to radiodensity, e.g. the attenuation measured in Hounsfield scale. As another example, for medical image data generated using an MRI device, the measured variable can be the relaxation time of the protons in the imaged volume, such as the T1 time constant or the T2 time constant. In some cases, each data element within the 3D medical image data can define a value of the measured variable and an associated location. In other examples, the location in the volume 200 to which each data element corresponds is inherent in the structure of the 3D image data (e.g. in the ordering of the data elements). Furthermore, in some cases, the 3D medical image data can define respective values of two or more measured variables for each location. For example, for 3D medical image data generated using an MRI device, values of both the T1 time constant and the T2 time constant can be stored for each location.

[0126] At block 104, the method comprises performing a selection process to select first image data forming a first portion of the 3D medical image data, the first image data representing a first anatomical object of a given type.

[0127] Briefly returning to Figure 2 which schematically illustrates an example of a first anatomical object 210 within an image volume 200 of the anatomy of a patient 10. In this example, the first anatomical object 210 is the skeleton of the patient 10. In this example, the skeleton is the femur of the patient.

[0128] The selection process of block 104 can be configured to identify, within the 3D medical image data, a subset of the 3D medical image data determined to represent an anatomical object of the given type. Thus, this subset of the 3D medical image data can be selected by the selection process as the first image data described above.

[0129] A selection process can be applied to 3D medical image data in order to select regions of the 3D medical image data that represent a given anatomical object (e.g. a single organ, such as a single bone or liver, or a given organ system, e.g. the respiratory system or the skeleton). In one example, the selection process is a segmentation process configured to identify regions of the 3D medical image data that represent a particular organ or a particular organ system. Other examples of anatomical objects that can be selected by the selection process are the liver; the brain; one or more kidneys; and one or more lungs.

[0130] In some examples, the selection process can be configured to select voxels of the image data that can be identified as representing a particular anatomical object due to a characteristic of the voxel values. In some examples, the selection process comprises an intensity-based and / or a shape-based segmentation process. For example, according to one example, a shape-based segmentation can initially be performed to segment a given anatomical object, and later the shape-based segmentation can be refined by an intensity-based segmentation.

[0131] Various parameters of the selection process can be determined according to the particular anatomical feature or anatomical object that the segmentation is intended to select. In some examples, more than one selection process can be performed, each of which can be configured to select a given anatomical object.

[0132] In some examples, parameters of the selection process are based on a given type of first anatomical object to be selected. For example, the parameters of the selection process can define a voxel intensity pattern and / or an object shape used by the selection process to select voxels representing a given anatomical object. For example, the parameters of the selection process can define the selection process as a segmentation process for segmenting a given type of organ, such as a bone, or alternatively, as a segmentation process for segmenting a liver. For example, where the type of first anatomical object is a bone, the parameters can define a characteristic shape and / or intensity pattern used by the segmentation process to identify portions of the volume representing the bone and to segment those portions.

[0133] In some examples, parameters of the selection process can be determined based on contextual information relating to the 3D medical image data. For example, the contextual information can be information relating to a clinical use case of the 3D medical image data. For example, the parameters of the selection process can dictate a type of anatomical object that the selection process is configured to select. For example, the contextual information can determine a shape and / or intensity pattern that the segmentation process is configured to identify in order to segment voxels representing a given anatomical object.

[0134] As a particular example, the context information can indicate that the clinical use case of the 3D medical image data is related to neurology. The context information can then be used to determine parameters for the selection processing that cause the selection processing to select the portion of the 3D medical image data that represents the patient's brain. In another example, the context information can indicate that the clinical use case is related to the liver, and thus the context information can be used to determine parameters for the selection processing that cause image data representing the liver to be selected.

[0135] The context information can be derived from textual information associated with the 3D medical image data. For example, the context information can be derived from textual information associated with the 3D medical image data that is indicative of the medical context of that data. For example, where the 3D medical image data is associated with descriptive meta-information, as is the case where the data is encoded as a Digital Imaging and Communications in Medicine (DICOM) dataset, the context information can be derived from the descriptive meta-information, for example that can be contained in a DICOM header of the DICOM dataset. For example, the textual information can be indicative of the scanning method used to produce the 3D medical image data. Additionally or alternatively, the textual information can be indicative of the anatomical features that the scan was intended to image and / or can identify a medical condition related to the scan.

[0136] In another example, the context information can be derived from medical history information associated with the 3D medical image data, for example from medical data records related to the patient.

[0137] In some cases, the context information that can be used in certain examples of the present method can be obtained via suitable data mining processing. For example, natural language processing analysis can be applied to textual data of the patient data. For example, in certain examples, a trained neural network can be applied to medical data related to the 3D medical image data in order to obtain the context information.

[0138] At block 106, the method comprises performing an analysis processing on the first image data, wherein parameters of the analysis processing are based on the given type of the first anatomical object 210.

[0139] In some examples, the analysis processing comprises determining one or more features of the first image data. In such examples, the parameters of the analysis processing can define the features of the first image data determined by the analysis processing. For example, the parameters of the analysis processing can define a feature of the first image data determined by the analysis processing to be a distribution of the voxel values of the first image data. For example, if the first anatomical object is a bone, the parameter of the analysis processing can be a feature of the image data associated with representing the bone, such as a peak value in the distribution of the voxel values of the first image data or a range of voxel values that satisfy a given predetermined criterion. Determining the features of the image data can for example comprise determining such a peak value and range of voxel values in the distribution. The specific characteristics of the image data determined by the analysis processing can vary between different types of anatomical objects. For example, in the case of bones, the determined features of the image data can differ depending on the specific type of bone. For example, it can be expected that a large bone has more voxel values representing bone marrow than a small bone, and therefore, the regions of the distribution of the voxel values of the large bone and the small bone will have different characteristics. Thus, the way in which the distribution of the voxel values can be decomposed into different components can differ between different types of bones, and therefore, different features can be determined to characterise such different types of distribution of the bone. In another example, in the case where the anatomical object is a liver rather than a bone, the features determined by the analysis processing can be one or more features of the distribution of the voxel values that can be associated with individual components of the liver. In yet another example, in the case where it is expected that there is a high density object such as a metal implant in the first image data, the determined features of the first image data can be related to such a high density object. For example, a peak value in the distribution of the first image data related to the high density object can be determined. For example, in the case of a metal implant, a peak value in a region of approximately 2000 HU can be determined. The voxel value associated with the peak value can thus correspond to the metal implant and be used for the determination of the visual parameter map to allow the metal implant to be appropriately visualised.

[0140] The one or more features of the first image data determined by the analysis processing can for example comprise one or more features of a distribution of first voxel values of the first image data. For example, the analysis processing can comprise analysing the distribution of the first voxel values to determine one or more features of the distribution. In some examples, the distribution of the voxel values is represented as a histogram representing the relative frequency of the voxel values taking various particular voxel values or falling into various sub-ranges of voxel values. In such cases, the one or more features of the voxel values can comprise one or more features of the histogram.

[0141] The one or more features of the distribution of the first voxel values can comprise voxel values or a range of voxel values that satisfy a predetermined criterion. For example, a peak or a mode in the distribution can be determined. The peak can be a voxel value having the highest frequency in the distribution. The peak can be a local maximum or a global maximum in the distribution.

[0142] Various other features of the distribution can be determined. For example, a range of voxel values can be determined whose distribution satisfies a predetermined criterion. For example, a range of voxel values can be determined that have a non-zero frequency or a frequency at or above a predetermined threshold. In some cases, such a range can be limited to a range of values on one side or the other of a given feature in the distribution, such as a peak in the distribution.

[0143] As mentioned above, the features of the first image data to be determined in a given example can depend on the given type of anatomical object that the first image data represents. For example, the features of the distribution to be determined can be features that are relevant to anatomical features of the given type of anatomical object.

[0144] In this regard, the inventors have recognized that image data representing a given type of anatomical object tends to have characteristics that are specific to the given type of anatomical object. For example, a distribution of voxel values of image data representing bone tends to have characteristic features that can be associated with different components of the bone. For example, certain specific features of a distribution of voxel values representing bone are known to be associated with the medullary cavity of the bone, while other certain specific features can be associated with the cortical layer of the bone.

[0145] The features determined by the analysis processing can depend on the given type of anatomical object that the image data represents. For example, where the selection processing performed at block 104 is configured to select cases of bone, the analysis processing can be determined to identify a global peak in the distribution of voxel values and respective ranges of non-zero frequency voxel values that are smaller and larger than the peak. On the other hand, for example, if the selection processing 104 is configured to select image data representing a liver, the analysis processing can be configured to determine different features of the image data as characteristics of the image data representing the liver. Examples of this will be discussed below with reference to Figure 3A and Figure 3B Examples of the case where the first anatomical object is bone are discussed in more detail.

[0146] In some examples, the results of the analysis processing obtained at block 106 can be used to refine the selection processing performed at block 104. For example, if a voxel value does not conform to a typical distribution of the first anatomical object of a given type, the analysis processing may, for example, allow the voxel value comprised in the first image data to be identified as not belonging to the first anatomical object. Such voxel values can be disregarded accordingly when performing the analysis processing on the first image data.

[0147] In some examples, the method comprises determining parameters of the analysis processing based on a given type of the anatomical object.

[0148] In some examples, the context information can be used to determine parameters of the analysis processing. For example, context information can be obtained that is indicative of a type of the first anatomical object. For example, the context information can indicate that the first anatomical object is a liver. The context information can then be used to determine a form that the analysis processing takes. For example, features of the first image data determined by the analysis processing can be determined by the type of the first anatomical object, which in turn can be indicated by the context information. The context information can be obtained in any suitable manner, for example as described above in relation to the selection processing.

[0149] At block 108, the method 100 comprises determining a visual parameter map in respect of the first portion based at least in part on the results of the analysis processing for use in the rendering processing to generate a visualization of the 3D medical image data.

[0150] Using the results of the analysis processing, a visual parameter map can be determined that is suitable for visualizing the anatomical object to which the image data relates.

[0151] For example, in the example described above in which the anatomical object represented by the first image data is a bone, the results of the analysis processing can comprise a voxel value corresponding to a peak in a distribution of voxel values representing the bone. As described above, this voxel value corresponding to the peak and voxel values greater than the peak having a non-zero frequency can be identified as voxel values associated with a cortex of the bone. Thus, the results of the analysis processing can be used to indicate voxel values associated with a specific physical component of the bone, in this example the cortex. Furthermore, in the example in which the first anatomical object is a bone, the results of the analysis processing can indicate a voxel value representing a medulla of the bone, as described above and as will be described in more detail in the context of the examples below, said voxel value being to the left of the peak relating to the cortex of the bone.

[0152] Thus, based on the result of the analysis processing, a visual parameter mapping can be determined for the image data representing the first anatomical object. For example, in case of a bone, since the voxel values representing the cortex are known from the analysis processing, the visual parameter mapping can be configured to assign colors and opacities to these voxel values which, when used in the rendering processing, are suitable for visualizing the cortex. Similarly, since a certain range of voxel values of the image data representing the bone marrow are known from the analysis processing, the visual parameter mapping can be configured to assign colors and opacities to these voxel values which, when used in the rendering processing, are suitable for visualizing the bone marrow. The voxel values corresponding to a particular component of the anatomical object, such as the cortex or the bone marrow, can for example differ from patient to patient due to the age of the patient or other factors. By determining the characteristics of the first image data and matching these characteristics to a particular component of the anatomical object, a visualization of the anatomical object can be provided which is consistent with the given type of anatomical object.

[0153] Determining the visual parameter mapping can involve fitting a function to the distribution of the image data, wherein the form of the function is determined by the type of object represented by the image data. In some examples, the method can comprise selecting a form of a transfer function from a predetermined pool of forms of transfer functions based on the given type of anatomical object. The selected form of the transfer function can be defined by a number of characteristics which are determined by the type of anatomical object to which the form of the transfer function corresponds. For example, the distribution of voxel values representing a given type of organ can have characteristic features such as a peak and / or a region of non-zero frequency which are characteristic for the given type of organ. The selected form of the transfer function can then be adjusted based on the analysis of the first image data to provide a transfer function which is particularly suitable for the first image data. In some examples, the optimization processing can be performed based on a machine learning algorithm which may, for example, use a neural network to adjust a given type of transfer function based on the analysis of the first image data. These features can be associated with anatomical features of the organ and used to derive a transfer function which allows to render these anatomical features in an appropriate manner. In this way, a visual parameter mapping can be obtained for a given anatomical object which provides a visualization of the object which is consistent with the type of anatomical object. For example, a visual parameter mapping can be determined which is configured to provide a consistent representation of a bone in terms of respective colors and opacities for the bone marrow and the cortex representing the bone. Furthermore, the visual parameter mapping can be determined without the need for manual intervention by a user.

[0154] The visual parameter mapping determined by the method 100 can then be applied in a rendering processing for generating a visualization of the 3D medical image data.

[0155] The visual parameter mapping is configured for assigning visual parameter data, such as opacity and / or color, to the first image data. For example, the visual parameter mapping can be a transfer function configured to be applied during a classification process in a direct volume rendering process.

[0156] In certain examples, during rendering, the visual parameter mapping obtained by the method 100 is only applied to the first image data forming a part of the 3D medical image data. That is, image data in the 3D medical image data not within the first part can be assigned visual parameter data in another way, e.g. using a different visual parameter mapping. In other examples, e.g. because only the anatomical object represented by the first image data is desired to be visualized, image data in the 3D medical image data not in the first part can not be assigned visual parameter data at all. Thus, in some examples, the visual parameter mapping of the first image data can be referred to as a partial transfer function, as it provides a transfer function configured for assigning visual parameter data to a given part of a volume.

[0157] Thus, examples of the above-described method provide a visual parameter mapping to be determined that is suitable for visualizing a given anatomical object. By identifying image data representing the anatomical object and analyzing this image data, while taking into account the type of anatomical object, an appropriate visual parameter mapping for the object can be derived.

[0158] The method provides a visual parameter mapping obtained for a part of the image data representing a given anatomical object that is tailored to the specific object. The method does not rely on applying a fixed mapping for a given type of object, where e.g. specific voxel values are mapped to given colors and opacities in a predetermined way. Instead, the present method allows generating a visual parameter mapping that is specifically suitable for visualizing the specific anatomical object represented by the data.

[0159] The above-described method also allows for a quick computation of adjustments to the visualization, e.g. in real-time. Furthermore, the method reduces the input required by the user to determine the visual parameter mapping, as the visual parameter mapping is generated based on the results of a computer analysis of the image data. This allows the method to be fast and not dependent on the user’s ability to adjust the visual parameter mapping to the specific use case at hand. This is advantageous, as the user can typically not be familiar with the subtleties of volume rendering and thus can neither be willing nor able to adjust the parameters of the rendering process.

[0160] The method according to this disclosure enables the determination of a visual parameter mapping for each anatomical object to be visualized. Different visual parameter mappings can then be applied to produce a volume visualization, wherein each object in the volume is visualized using an appropriate, specifically tuned visual parameter mapping. Therefore, unlike some prior art methods that provide a single visual parameter to be applied to an entire volume dataset, the method according to this disclosure enables the visualization of a given anatomical object using different visual parameter mappings. The visual parameter mapping determined for a given object is specifically tuned for visualizing that object based on analysis of image data representing that object, and this visual parameter mapping is configured to be applied locally to visualize the object.

[0161] This method allows multiple visual parameter maps to be obtained for a dataset corresponding to multiple anatomical objects represented in the dataset. These visual parameter maps can then be applied, for example, as partial transfer functions to enable the simultaneous rendering of multiple anatomical objects. For example, in some examples, it may be necessary to implement interactive selection of the anatomical structures to be displayed, for instance, based on a clinical use case of a volume dataset being visualized. For example, the clinical context of the visualization might be related to the liver and therefore require detailed display of the liver, while the lungs might be unrelated and therefore may be less detailed or not shown at all. Furthermore, some clinical use cases may require the simultaneous display of two or more different objects, such as anatomical organs. In some such examples, the opacity, color, or other visual parameters applied to different objects by the visual parameter maps can be determined based on which objects need to be displayed. For example, if the object of interest in the visualization is located behind another object, the other objects in the visualization can be made transparent so as not to obstruct the visibility of the object of interest. Alternatively or alternatively, multiple visual parameter maps can allow the user to switch between which anatomical objects are visualized while providing appropriate visualization regardless of which anatomical objects are being visualized. Furthermore, this method enables visualization of the selected anatomical object and allows for rapid, "on-the-fly" acquisition of visual parameter mappings for that object. This provides users with a workflow that is fast and offers visualizations tailored to the specific use case at hand, without requiring users to adjust rendering parameters appropriately.

[0162] Figure 3A It shows the relationship with Figure 2 The image data corresponding to the first anatomical object 210 is shown in the example below. In this example, the anatomical object 210 is a skeleton. The histogram 300 represents the frequency of voxel values ​​of the selected first image data falling within a given subrange as vertical bars.

[0163] Figure 3AThis schematically illustrates an example of analytical processing performed on the first image data selected by the selection process. Figure 3A The analysis process involves determining the characteristics of the distribution of voxel values ​​in the first image data.

[0164] The characteristics of the first image data to be determined are based on the type of anatomical object represented by the first image data (in this case, skeleton). In this example, the specific characteristics of the image data to be determined are as described below. However, in other examples, for example, one or more different characteristics of the image data can be determined based on the type of anatomical object represented by a given portion of the image data.

[0165] exist Figure 3A In the example shown, the first characteristic identified is the total peak value of 301 in the distribution. In an example where the distribution is in Hornsfield units (HU), the peak value of the histogram may appear around 300 HU to 330 HU, or typically around 320 HU. As mentioned above, the actual voxel value of the peak value can vary depending on various factors, such as, for example, the age of the patient being imaged.

[0166] The second characteristic of the distribution of the determined voxel values ​​is the range of voxel values ​​from the voxel values ​​associated with the peak value 301 to the maximum voxel value of 350.

[0167] The voxel values ​​in the range 350 are used to represent a certain range of voxel values ​​for the cortex. The maximum voxel value in the range 350 can be the maximum voxel value in the distribution of the first image data whose frequency is at or above a predetermined threshold frequency. For example, the maximum voxel value can be the maximum voxel value in the first image data with a non-zero frequency. In other examples, the maximum voxel value can be determined based on contextual information associated with the dataset. For example, the contextual information can indicate the presence of metallic objects or other objects with high radiometric density, and can also indicate that such objects are represented in the first image data. In such cases, the maximum voxel value associated with the cortex of the skeleton can be set to exclude from the range 350 of voxel values ​​corresponding to high cortical voxel values, which may correspond to other such high radiometric density objects. This prevents voxel values ​​associated with objects that are not part of the cortex from being visualized as if they were part of the cortex. In a typical example of measuring voxel values ​​in HU, the voxel values ​​in the range 350 determined to correspond to the cortex can extend from approximately 320 HU to approximately 950 HU. As mentioned above, this range varies between 3D medical image datasets, for example, depending on the age of the patient being imaged or the specific type of bone.

[0168] A third characteristic of the determined distribution of voxel values is a range 360 of voxel values extending from the minimum voxel value for which the frequency is at or above the predetermined threshold frequency to the voxel value associated with the peak value 301. This range 360 of voxel values is related to the bone marrow of the bone. In a typical example where the voxel values are measured in HU, this range can extend from approximately 250 HU to approximately 320 HU.

[0169] A fourth characteristic of the determined distribution of voxel values is a range 362 of voxel values within the range 360 extending from the voxel value associated with the peak value and in which the frequency is at or above a predetermined proportion of the frequency at the peak value 301. For example, the range 362 can include a certain range of voxel values immediately below the peak value 301 having a frequency within 70% of the frequency at the peak value 301. In a typical example where the voxel values are measured in HU, this range can extend from approximately 290 HU to approximately 320 HU.

[0170] Figure 3B It is shown how a schematic representation of a visual parameter map corresponding to the distribution is determined from the results of the analysis processing performed on the image data of the example. Figure 3A

[0171] Figure 3B It is shown how a schematic representation of a visual parameter map corresponding to the distribution is determined from the results of the analysis processing performed on the image data of the example.

[0172] According to this fitting, a transfer function is provided for the first image data. That is, the voxel values falling within the range 350 are mapped to the function values as determined by the function fitted to the distribution of voxel values of the first image data. Figure 3B ​The vertical axis on the right represents opacity values defined by the height of the rectangle 354. In this example, each value determined to represent the cortex is mapped to the same opacity value, which is a maximum opacity value of, for example, 100%. A mapping of color values can be defined, which can be configured to vary in any suitable manner with varying voxel values. For example, each voxel value falling in the range 350 representing the cortex can be assigned the same color value, or alternatively, the color values assigned to these values can vary with the voxel values, for example, so as to convey information about different radiodensities of different portions of the cortex. For example, the color assigned to a given point along the rectangle can be graduated from a first predetermined color at the left side of the rectangle to a second predetermined color at the right side of the rectangle. For example, the color assigned to a given point along the rectangle can be interpolated from the colors assigned at the left and right edges of the rectangle.

[0173] The transfer function defined according to the fit to the distribution also provides a mapping of opacity and color to voxel values of a range 360 corresponding to bone marrow. According to the fit applied in this example, a trapezoid 364 defines the opacity of voxel values of the range 360, which increases linearly from zero at the minimum voxel value in the range 360 to a maximum opacity at the higher voxel values in the range. For example, the maximum opacity of voxel values of the range 360 corresponding to bone marrow can be 70% of the opacity applied by the rectangle 354 for the cortex range 350. In examples, the color assigned by the transfer function to voxel values in the bone marrow region is different from the color assigned to the cortex region. As an example, a red or orange color can be assigned by the transfer function to voxels determined to represent bone marrow, while a white or gray color can be assigned to voxels determined to represent the cortex. The transfer function can also be defined so that the color assigned to voxel values in the bone marrow region varies with the voxel values. For example, a predetermined color can be assigned by different predetermined relative points along the trapezoid in proportion to the width of the trapezoid. Thus, the actual voxel values corresponding to these points along the trapezoid and thus mapped to a given color are determined by the fit of the trapezoid 364 to the image data described above.

[0174] Figure 3A and Figure 3B The example shown in FIGS. 1-3 is based on a particular set of characteristics of the image data determined by the analysis processing to provide a transfer function for the image data representing bone. In other examples, a different set of characteristics of the image data than those shown in FIGS. 1-3 can be determined in order to determine a fit of a transfer function for the image data. For example, a transfer function for the image data shown in FIGS. 1-3 can be determined based on a set of characteristics of the image data determined by the analysis processing to provide a transfer function for the image data representing bone. Figure 3A and Figure 3B The example shown in FIGS. 1-3 is based on a particular set of characteristics of the image data determined by the analysis processing to provide a transfer function for the image data representing bone. In other examples, a different set of characteristics of the image data than those shown in FIGS. 1-3 can be determined in order to determine a fit of a transfer function for the image data. For example, a transfer function for the image data shown in FIGS. 1-3 can be determined based on a set of characteristics of the image data determined by the analysis processing to provide a transfer function for the image data representing bone. Figure 3A and Figure 3BSome but not all of the features described, and fitting can be determined based on the determined features only. In other examples, other features than those shown in Figure 3A and Figure 3B may be determined, or a completely different set of features. In other examples, different forms of transfer function can be provided based on the results of the analysis of the image data. In such examples, the determined features of the distribution of the image data can depend on the form of the function to be fitted to the distribution. For example, in alternative examples, a right-angled triangle rather than a trapezium can be fitted to a range of voxel values corresponding to bone marrow. In such examples, the fourth feature defining the width of the upper edge of the trapezium described above can not be required in the case where a right-angled triangle is fitted to the bone marrow region. Furthermore, in another example, a trapezium can fit the bone marrow region of the distribution configured such that the upper edge of the trapezium has a length that is in a predetermined proportion to the lower edge of the trapezium. For example, the width of the upper edge can be half the width of the lower edge of the trapezium. In such examples, the fourth feature of the voxel values described above can also not be required to be determined.

[0175] Figure 4 A flowchart representation of an example method 400 of generating a computer-based visualization of 3D medical image data is shown.

[0176] At block 402, the method 400 comprises performing a method according to the present disclosure to obtain a visual parameter map in respect of a first portion of the 3D medical image data.

[0177] In some examples, the processing at block 402 can be performed multiple times for each of a plurality of different anatomical objects represented in the 3D medical image data. For example, the above-described method for determining a visual parameter map in respect of first image data in the 3D medical image data can be performed for a plurality of different portions of the 3D medical image data, each different portion comprising a respective different anatomical object. For example, in respect of the example image volume 200 of Figure 2 , the method according to Figure 1 may be performed a first time to determine a first visual parameter map in respect of a first portion of the 3D medical image data representing a first anatomical object 310 (in this case, a bone), and the method according to Figure 1 may be performed a second time to determine a second visual parameter map in respect of a second portion of the 3D medical image data representing a second anatomical object 320 (e.g. the liver of the patient 10). Thus, two or more partial visual parameter maps or partial transfer functions can be obtained for the volume 200, one or both of which can be applied to the rendering process to render a visualization of the volume 200.

[0178] At block 404, the method 400 includes performing a rendering process to generate a visualization of the 3D medical image data, wherein performing the rendering process includes applying a visual parameter map with respect to the first portion of the 3D medical image data.

[0179] The rendering process can include any suitable volume rendering process. For example, the rendering process can be a direct volume rendering process that includes defining a viewpoint with respect to the volume data and causing a plurality of simulated rays originating from the viewpoint to pass through the volume data set. In such examples, each ray that passes through the volume data set can enable a determination of a value or set of values to be displayed by a pixel of an observation plane that intersects the ray. For example, a rendering algorithm can be employed that determines the value to be displayed by a pixel via an integration along the path of the ray of the visual parameter values associated with sample points in the volume.

[0180] It should be noted that at least some sample points can not coincide with voxels, and thus, the calculations related to a particular sample point, such as the visual parameter map or transfer function described above, can employ interpolation to determine a scalar value at that sample point. Then, another example approach of trilinear interpolation or interpolation based on the scalar values of a set of voxels adjacent to the point can be performed to determine an interpolated scalar value for the given sample point. Assigning a visual parameter to the given sample point can then include applying the visual parameter map to the interpolated value of the volume data set at the sample point.

[0181] Various lighting effects can be applied in a given example rendering process. For example, the rendering process can model lighting effects by modeling light sources that illuminate the volume, such as by using a light map.

[0182] Various volume data reconstruction filters can be used during rendering, such as a nearest neighbor filter, a trilinear filter, or a higher order filter such as a B-spline filter. The filter can or can not be interpolating. In the case that the filter is not interpolating, oversmoothing can be used as part of the rendering.

[0183] In some examples, data filtering can be applied separately from the rendering process. Data filtering can include, for example, Gaussian smoothing, non-sharpening masking, thresholding, and various morphological operations.

[0184] According to examples of the present disclosure, a visual parameter map with respect to at least a first portion of a volume representing a first anatomical object in the volume is assigned via examples of the methods as described above. A portion of the volume that is not part of the first portion can be assigned a different visual parameter map than the map with respect to the first portion. For example, the volume can be segmented to form a plurality of segmentation masks using a selection process. During rendering, the segmentation mask into which a given sample point falls can be used to determine a transfer function that is applied to assign visual parameter data at the sample point.

[0185] For example, in an example rendering process for visualizing a volume 200 Figure 2 For example, in an example rendering process for visualizing a volume 200

[0186] For example, assigning visual parameter data to a given sample point during rendering can comprise determining whether the sample point is located within the first portion 210, the second portion 220 or outside the first portion 210 and the second portion 220 and applying a visual parameter mapping based on the determination. In an example, this can be done by determining the segmentation mask a particular sample point belongs to during rendering and applying the visual parameter mapping applicable to that segmentation mask. For example, if it is determined that the sample point is within the first portion 210, the first visual parameter mapping is applied to determine the visual parameter data for the sample point, whereas if it is determined that the sample point is within the second portion 210, the second visual parameter mapping is applied. If it is determined that the sample point is not within the first portion 210 or the second portion 220, a different visual parameter mapping can be applied.

[0187] An advantage of this approach is that visual parameter mappings can be determined for individual portions of the 3D image data, while also defining a global visual parameter mapping for the entire 3D image data. The global visual parameter mapping can be determined by a user manually adjusting or by any other suitable means, while the visual parameter mappings for specific anatomical objects can be determined according to the approach described above. This means that the overall visual parameter mapping is not disturbed by the application of the local visual parameter mapping for a given portion of the data, which in some examples can be a time-consuming process to determine. Instead, as described above, by analyzing the portion of the image data that does not include image data corresponding to a given anatomical organ, the local visual parameter mapping can be determined in an efficient and effective manner.

[0188] Figure 5 A flowchart representation of an example clinical workflow method 500 is shown, which comprises a method of rendering according to examples of Figure 4

[0189] At block 502, patient data comprising image data and non-image data is received. The image data can comprise a plurality of 3D medical image data sets, for example representing the results of a plurality of medical scans of a patient obtained by one or more scanning methods and / or the results of one or more scans obtained by operating one scanning method at different energy levels. Each 3D medical image data set can show the same volume or different volumes of the patient.​

[0190] At block 504, context information is determined from the patient data and used to determine one or more anatomical objects to be visualized. The context information is indicative of a clinical use case for the image to be rendered from the image data. As described above, the clinical use case can be indicative of an organ that is relevant to the clinical use case. Additionally or alternatively, the clinical use case can be indicative of a stage of the patient in a clinical workflow. The context information can thus be used to identify information that should be conveyed to the user by the visualization. For example, the context information can be used to identify which organ or organs should be included in the visualization, and additionally can be used to determine a viewing angle, lighting conditions, and any other factors that can influence the desired information to be conveyed to the user by the image to be rendered. The context information can be obtained in any suitable manner, examples of which have been described above.

[0191] At block 506, a first 3D medical image data set is selected from the image data for segmenting an anatomical object of the one or more anatomical objects to be visualized. The first 3D medical image data set can be selected as a data set that is suitable for segmentation thereon to segment the anatomical object. For example, if the context information obtained at block 504 is indicative of a medical use case for the image to be rendered being an examination of a liver of the patient, at block 506, a first 3D medical image data set that is suitable for segmenting the liver can be selected. For example, a 3D medical image data set that is obtained via a particular series or at a particular energy level can be more suitable for distinguishing between portions of the imaging volume that represent the liver and portions of the imaging volume that do not represent the liver. In some examples, a factor that can influence which data set is suitable for performing the segmentation is a level of filtering or smoothing that has been applied to the data set. For example, a data set in which no filtering has been applied to smooth the voxel values or only a moderate level of such filtering has been applied can be more suitable for performing the segmentation than a data set in which more smoothing has been applied. In some examples, a reconstruction kernel of a particular data set can be indicative of a level of smoothing that has been applied to the data set.

[0192] At block 508, a segmentation process is performed on the first 3D medical image data set to obtain a segmentation of the anatomical object. Examples of such segmentation processes have been described above.

[0193] At block 510, a second 3D medical image data set is selected from the image data. The second 3D medical image is selected as the data set for rendering the visualization of the anatomical object. In some examples, the second 3D medical image data set is different from the first 3D medical image data set. For example, while the first 3D medical image data set (e.g. a data set obtained using a first imaging modality or using a first energy level) can be more suitable for segmenting the anatomical object, the second 3D medical image data set (e.g. a data set obtained using a different imaging modality or a different energy level) can be determined to be more suitable for rendering the visualization of the anatomical object. For example, the first 3D medical image data set and the second 3D medical image data set can be data sets obtained simultaneously. For example, the first 3D medical image data set and the second 3D medical image data set can be data sets representing results of CT scans, and can respectively represent results of a relatively low energy x-ray scan and a relatively high energy x-ray scan. In other examples, the first 3D medical image data set and the second 3D medical image data set can be data sets obtained using different imaging modalities or belonging to different series obtained using the same imaging modality. In other examples, the first 3D medical image data set and the second 3D medical image data set used for obtaining the segmentation and for visualizing the anatomical object can be the same data set.

[0194] At block 512, a portion of the second 3D medical image data set is selected based on the segmentation of the anatomical object obtained from the first 3D medical image data set. For example, a segmentation obtained from the first 3D medical image data set representing a region of an imaged body of a liver can be applied to the second 3D medical image data set to select a portion of the second 3D medical image data set representing the liver and for rendering processing to visualize the liver. Registration between the first 3D medical image data set and the second 3D medical image data set can allow identifying voxels in the data sets corresponding to the same points in the imaged body. Thus, the segmentation of the anatomical object obtained by segmenting the first 3D medical image data set can be applied to select a portion of the second 3D medical image data set representing the same anatomical object. In some examples, the steps performed at blocks 504 to 512 can be considered to form a portion of the selection processing as described above for selecting the first image data for which the visual parameter mapping is to be determined.

[0195] At block 514, a visual parameter mapping is obtained for the selected portion of the second 3D medical image data set. The visual parameter mapping is determined as described in the examples above. According to an example, the determination of the visual parameter mapping is based on context information, wherein the context information determines parameters of an analysis processing performed on a portion of the image data to determine the visual parameter mapping for the portion.

[0196] Finally, at block 516, a rendering process is performed in which the visual parameter maps obtained as described with reference to blocks 506-514 are applied. Examples of such rendering processes have been described above. In some examples, the steps described at blocks 506-514 are each performed multiple times for different anatomical objects. For example, if at block 504 it is determined that the visualization should show the liver and the skeleton of a patient, then blocks 506-514 can be performed to obtain a visual parameter map for the liver and blocks 506-514 can be performed again to obtain a visual parameter map for the skeleton. In such examples, at block 516, each of these obtained visual parameter maps can be applied to the rendering process to visualize the respective anatomical object to which the visual parameter map relates.

[0197] Different 3D medical image data sets can be used to obtain segmentations and / or to visualize different anatomical objects. For example, the above-described second 3D medical image data set used to obtain a visual parameter map and to visualize a given anatomical object can be different for different anatomical objects to be included in the visualization. Similarly, the above-described first 3D medical image data set used to obtain a segmentation of a given anatomical object can be different for different anatomical objects to be included in the visualization. For example, one 3D medical image data set or multiple 3D medical image data sets used to segment and visualize the liver of a patient can be different from the 3D medical image data set used for the skeleton of the patient. In such examples, two or more respective rendered images of different anatomical organs can be combined, e.g., by superimposing one image on another, to provide a combined rendered image depending on the clinical use case. Further, in some examples, two or more rendered images of a single anatomical object obtained using different volume data sets can be used to render a combined image of the single anatomical object.

[0198] In some examples, the workflow according to the present disclosure can allow a user to change the rendered image by, e.g., selecting or deselecting a given organ to be rendered. For example, the user can issue a voice command that can be used to select one or more organs to be visualized.

[0199] Figure 6 A system 600 for processing 3D medical image data is schematically illustrated. In the illustrated example, the system 600 comprises a computing device in the form of a computer 602. The computer 602 comprises one or more processors 604 and a memory 606. The memory 606 can be in the form of a computer-readable storage medium. The memory 606 stores thereon instructions which, when executed by the one or more processors 604, cause the one or more processors to perform the above-described method.

[0200] When system 600 is provided to a user, instructions can be stored on memory 606. Alternatively, the instructions can then be provided to the user, for example, in the form of a computer program product, via a computer-readable storage medium such as an optical disc (CD), digital multifunction disc (DVD), hard disk drive, solid-state drive, flash memory device, etc. Alternatively, the instructions can be downloaded to the storage medium via a data communication network (e.g., the World Wide Web).

[0201] In cases where a method executed by one or more processors 604 involves one or more neural networks, such neural networks can be stored on memory 606. Similar to the instructions stored on memory 606, the neural network can be stored on memory 606 when system 600 is provided to a user, either by means of a computer-readable storage medium or by means of downloading the neural network via a data communication network, or it can be provided later (e.g., in the form of a computer program product).

[0202] Specifically, in cases where the method involves one or more neural networks, the one or more processors 604 may include, for example, one or more graphics processing units (GPUs) or other types of processors. The use of GPUs allows system 600 to be optimized for utilizing neural networks. As will be understood, this is because GPUs can process a large number of threads simultaneously.

[0203] like Figure 6 As shown, in some examples, system 600 may include one or more displays 612 for displaying a view of the region of interest generated by visualization processing to a user.

[0204] For example Figure 6 As shown, system 600 may additionally include imaging device 608 configured to acquire medical image data. For example, system 600 may include an X-ray imaging device or an MRI imaging device.

[0205] In some examples, system 600 may include input interfaces such as a mouse, keyboard (or respective connection interfaces for connecting them), touchscreen interface, voice capture device, etc. Users of system 600 can use the input interfaces to input information into system 600.

[0206] As described above, although the present invention is set within the context of a direct volume rendering algorithm employing a raycasting method, it should be understood that the invention can be applied to other exemplary methods for visualizing volumes. For example, the method described above for determining a composite representation of a volume and its surfaces can be used in other volume rendering techniques. Such a method can be used, for instance, in volume rendering techniques such as path tracing, sputtering, or shear deformation.

[0207] Although in some of the above examples the visual parameter mapping has been described as a transfer function mapping voxel values to opacity and color, the visual parameter mapping can map voxel values to additional or alternative visual parameters. For example, in an example the transfer function can be configured to assign one or more of: a scattering coefficient, a specular reflection coefficient, a diffuse reflection coefficient, a scattering distribution function, a bidirectional transmittance distribution function, a bidirectional reflectance distribution function, and color information. These parameters can be used to derive the transparency, reflectivity, surface roughness, and / or other characteristics of the surface of a given point. These surface material characteristics can be derived based on the scalar values of the volume dataset at the rendering location and / or based on user-specified parameters.

[0208] Although in some of the above examples the method involves determining parameters of the analysis processing based on the type of the anatomical object, such that for example the parameters of the analysis can differ depending on the type of the anatomical object, in other examples the method can be particularly adapted to determine visual parameter mappings for a single type of anatomical object. For example, the method can be provided by a set of computer readable instructions configured to perform a method of selecting image data representing a given type of anatomical object, for example a bone, from 3D medical image data, and performing an analysis processing on the image data that is particularly adapted to determine a visual parameter mapping for the given type of object.

[0209] The above implementations should be understood as illustrative examples of the application. Other implementations are envisaged. It will be appreciated that any feature described in relation to any one implementation can be used alone and / or in combination with other features described, and can also be used in combination with one or more features of any other implementation or any combination of any other implementations. Furthermore, equivalents and modifications not described above can also be employed without departing from the scope of the application, which is defined in the claims below.

[0210] The following points are also part of the present disclosure:

[0211] Point 1. A computer-implemented method for providing a user with a visualization object for visualizing a three-dimensional anatomical region of a patient, the three-dimensional anatomical region of the patient being represented by volume data (3D medical image data), the method comprising the steps of:

[0212] - invoking / retrieving data assigned to the patient (in particular: from a database; in particular, the retrieved data comprises non-image data);

[0213] - determining a medical context information item (or context information) based on the assigned data;

[0214] - selecting suitable volume data of the patient based on the medical context information item (from available image data of the patient);

[0215] - based on the item of medical context information, identifying one or more structures (depicted in the selected volume data; another word for structures is anatomical objects) to be visualized (in the user's visualization object);

[0216] - determining a mapping rule for mapping the volume data onto the visualization object, wherein the mapping rule is determined taking into account the item of medical context information and / or the identified structures;

[0217] - computing the visualization object based on the mapping rule;

[0218] - providing the visualization object.

[0219] Point 2. The method according to point 1, wherein

[0220] the identifying is implemented based on a segmentation of the volume data, wherein the segmentation is preferably implemented based on an item of medical context information, and wherein further preferably the segmentation is a shape-based segmentation.

[0221] Point 3. The method according to any one of the preceding points, wherein

[0222] the mapping rule has a partial mapping rule for each identified structure; and

[0223] the determining step further comprises:

[0224] specifically adjusting (or optimizing) the partial mapping rule for each identified structure, wherein the optimization of each partial mapping rule is in particular implemented independently of the respective other partial mapping rules.

[0225] Point 4. The method according to point 3, wherein

[0226] the adjusting comprises an image value-based or intensity-based adjustment or an adjustment in intensity space, and in particular comprises a image value-based or intensity-based segmentation or a segmentation in intensity space.

[0227] Point 5. The method according to point 3 or 4, wherein

[0228] the adjusting of each identified structure comprises:

[0229] extracting an item of image information from the volume data; and

[0230] adjusting the partial mapping rule based on the item of image information.

[0231] Point 6. The method according to point 5, wherein

[0232] The image information items comprise a statistical frequency (or distribution) of image values of voxels belonging to the identified structures.

[0233] Point 7. The method according to point 6, wherein

[0234] The volume data is generated at least partially using a computed tomography method; and

[0235] The image information items comprise a statistical frequency (or distribution) of Hounsfield units (HU).

[0236] Point 8. The method according to any one of points 4 to 7, wherein

[0237] The adaptation for each identified structure further comprises:

[0238] determining at least two image information contributions in the image information items; and

[0239] adapting the partial mapping rules based on the image information contributions.

[0240] Point 9. The method according to any one of points 4 to 8, wherein

[0241] The adaptation for each identified structure further comprises:

[0242] comparing the image information items to reference image information items; and

[0243] adapting the partial mapping rules based on the comparison.

[0244] Point 10. The method according to any one of points 3 to 9, wherein

[0245] The adaptation of the partial mapping rules comprises:

[0246] selecting a pre-set partial mapping rule assigned to the respective identified structure; and

[0247] adapting the pre-set partial mapping rule to create an adapted partial mapping rule.

[0248] Point 11. The method according to any one of points 3 to 10, wherein

[0249] The adaptation of the partial mapping rules is performed based on the medical context information items.

[0250] Point 12. The method according to any one of points 3 to 11, wherein the partial mapping rules are adapted by applying a trained function, in particular the trained function is implemented to provide a partial mapping rule for each identified structure based on the medical context information items.

[0251] Point 13. The method according to any one of the preceding points, wherein

[0252] the computation is implemented using a volume rendering algorithm, in particular a method based on ray casting and / or path tracing; and

[0253] the mapping rule has one or more transfer functions.

[0254] Point 14. The method according to any one of the preceding points, wherein

[0255] the mapping rule is implemented such that the identified structure is emphasized, in particular optically emphasized, to the user in the visualization object.

[0256] Point 15. The method according to any one of the preceding points, wherein

[0257] the mapping rule comprises a global mapping rule defining one or more overall scene properties of the visualization image.

[0258] Point 16. The method according to any one of the preceding points, wherein

[0259] the mapping rule is implemented in a manner that the visualization object comprises a time-resolved sequence of multiple individual images.

[0260] Point 17. The method according to point 16, wherein

[0261] at least two of the individual images represent different viewing angles of the volumetric data record.

[0262] Point 18. The method according to any one of the preceding points, further comprising the step of:

[0263] receiving a user input regarding the item of medical context information, wherein the item of medical context information is additionally determined based on the user input.

[0264] Point 19. The method according to any one of the preceding points, wherein

[0265] the selected volumetric data comprises a first volumetric data record recorded by a first imaging modality and comprises a second volumetric data record recorded by a second imaging modality different from the first imaging modality; the method further comprises:

[0266] registering the first volumetric data record with the second volumetric data record; and

[0267] the step of additionally further implementing the determination of the mapping rule is based on the registration.

[0268] Point 20. The method according to any of the preceding points, further comprising the steps of:

[0269] providing body data related to the patient; and

[0270] selecting suitable body data from the provided body data.

[0271] Point 21. The method according to any of the preceding points, wherein the assigned data is called from one database or from a plurality of different databases.

[0272] Point 22. A computer-implemented method for providing a visualization object for visualizing a three-dimensional anatomical region of a patient to a user, the three-dimensional anatomical region of the patient being represented by 3D medical image data (or body data), the method comprising the steps of:

[0273] providing the medical image data;

[0274] providing medical data assigned to the patient;

[0275] determining medical context information items based on the medical data and / or the medical image data;

[0276] identifying one or more structures (also denoted as anatomical objects) in the selected body data based on the medical context information items;

[0277] determining mapping rules (also denoted as visual parameter mappings) for mapping the medical image data onto a visualization object of the user, wherein the mapping rules comprise partial mapping rules for each identified structure; and

[0278] the determining step further comprises:

[0279] adjusting the partial mapping rules specific to each identified structure based on medical context information items and the medical image data;

[0280] the adjusting step further comprises:

[0281] adjusting the partial mapping rules specific to each identified structure based on medical context information items and the medical image data;

[0282] computing the visualization object based on the mapping rules;

[0283] providing the visualization object to the user.

[0284] - receiving a selection command of the user, the selection command indicating at least a patient to be analyzed;

[0285] - retrieving medical data assigned to the patient from a database based on the selection command;

[0286] - determining a medical context information item based on the retrieved data;

[0287] - selecting appropriate volume data of the patient based on the medical context information item and optionally the selection command;

[0288] - identifying one or more structures contained in the selected volume data to be visualized based on the medical context information item (also denoted as anatomical objects);

[0289] - determining a mapping rule for mapping the volume data onto a visualization object of a user (also denoted as visual parameter mapping), wherein the mapping rule is determined based on the medical context information and / or the identified structures;

[0290] - computing the visualization object based on the mapping rule;

[0291] - providing the visualization object to the user.

[0292] Point 24. The method according to point 23, wherein

[0293] the mapping rule comprises a partial mapping rule for each structure to be visualized; and

[0294] the determining step further comprises:

[0295] specifically adapting the partial mapping rule for each structure to be visualized, wherein

[0296] the adaptation of each partial mapping rule is in particular independent from the respective other partial mapping rules.

[0297] Point 25. The method according to point 24, wherein

[0298] the adaptation for each structure to be visualized comprises:

[0299] extracting an image information item from the volume data, in particular from the volume data associated with the respective structure to be visualized; and

[0300] adapting the partial mapping rule based on the image information item, in particular by analyzing the image information.

[0301] Point 26. The method according to point 25, wherein

[0302] The image information items comprise statistical frequencies (or distributions) of voxel values and / or intensity values of volume pixels belonging to structures to be visualized.

[0303] Point 27. The method according to point 25 and / or 26, wherein

[0304] The adaptation for each structure to be visualized further comprises:

[0305] determining at least two different image information contributions in the image information items; and adapting the partial mapping rules based on the image information contributions.

[0306] Point 28. The method according to any one of the preceding points, wherein

[0307] the computation is implemented using a volume rendering algorithm, in particular the volume rendering algorithm implements a ray casting and / or path tracing based method; and

[0308] the mapping rules have one or more transfer functions.

[0309] Point 29. The method according to any one of the preceding points, further comprising the step of:

[0310] receiving a user input regarding the medical context information items, wherein the medical context information items are additionally determined based on the user input.

[0311] Point 30. The method according to any one of the preceding points, wherein

[0312] the selected volume data comprises a first volume data record (3D medical image data set) recorded by a first imaging modality and comprises a second volume data record (3D medical image data set) recorded by a second imaging modality different from the first imaging modality; the method further comprises:

[0313] registering the first volume data record with the second volume data record; and

[0314] the step of additionally further implementing determining the mapping rules is based on the registration.

[0315] Point 31. A system for providing a user with a visualization object visualizing a three- dimensional anatomy of a patient represented by medical volume data, the system comprising:

[0316] an interface for receiving a selection command of the user and for receiving medical volume data, the selection command indicating a patient to be analyzed; and

[0317] a computing unit implemented to:

[0318] - receiving and / or invoking data assigned to the patient based on the selection command (from a database);

[0319] - determining a medical context information item based on the assigned data;

[0320] - selecting appropriate volume data of the patient based on the medical context information item and optionally the selection command (from image data available for the patient);

[0321] - identifying one or more structures in the selected volume data to be visualized based on the medical context information item;

[0322] - determining a mapping rule for mapping the volume data onto a visualization object of a user based on the medical context information item and / or the structures to be visualized;

[0323] - computing the visualization object based on the mapping rule; and

[0324] - providing the visualization object to the user.

[0325] Point 32. A computer-implemented method for providing a visualization object to a user visualizing a three-dimensional anatomical region of a patient represented by volume data, the method comprising the steps of:

[0326] - providing the volume data;

[0327] - providing context data assigned to the patient, the context data being different from the volume data;

[0328] - determining a medical context information item based on the assigned context data;

[0329] - identifying one or more structures in the selected volume data to be visualized based on the medical context information item;

[0330] - determining a mapping rule for mapping the volume data onto a visualization object of the user, wherein

[0331] the mapping rule has a partial mapping rule for each identified structure;

[0332] the determining step further comprises:

[0333] adjusting the partial mapping rule specific to each identified structure based on the medical context information item; and

[0334] the adjusting for each structure comprises extracting an image information item from the volume data associated with the respective structure and adjusting the partial mapping rule based on the image information item;

[0335] - computing the visualization object based on the mapping rule;

[0336] - providing the visualization object to the user.

[0337] Point 33. Computer program product having a computer program, which can directly load into the memory of a visualization system, comprising program portions to perform all steps of the method for visualizing a three-dimensional object according to any one of the preceding points and / or as disclosed in the present disclosure when the program portions are executed by the visualization system.

[0338] Point 34. Computer-readable storage medium having stored thereon program portions readable and executable by a visualization system so as to perform all steps of the method for visualizing a three-dimensional object according to any one of the preceding points and / or as disclosed in the present disclosure when the program portions are executed by the visualization system.

Claims

1. A computer-implemented method (100) for generating a computer-based visualization of 3D medical image data, the method comprising: receiving (102) 3D medical image data; performing (104) a selection process to select first image data forming a first portion of the 3D medical image data, the first image data representing a first anatomical object (210, 220) of a given type; performing (106) an analysis process on the first image data; and determining (108) a visual parameter map for the first portion based at least in part on a result of the analysis process to be used in a rendering process to generate a visualization of the 3D medical image data, characterized in that: parameters of the selection process are determined based on context information related to the 3D medical image data, the parameters of the selection process being able to dictate a type of anatomical object the selection process is configured to select, the context information related to the 3D medical image data is one or more of: textual information identifying a medical context of the 3D medical image data; medical history information associated with the 3D medical image data, and parameters of the analysis process are based on the given type of the first anatomical object.

2. The method of claim 1, the analysis process comprises determining one or more features of the first image data, and wherein parameters of the analysis process are determined based on the one or more features of the first image data. wherein 3. The method of claim 2, the one or more features of the first image data comprise one or more features of a distribution (300) of first voxel values of the first image data, and the analysis process comprises analyzing the distribution (300) of first voxel values to determine the one or more features of the distribution (300). wherein 4. The method of claim 3, the one or more features of the distribution (300) comprise a voxel value or a range (350, 360, 362) of voxel values satisfying a predetermined criterion. wherein 5. The method of claim 4, the predetermined criterion defines a voxel value or a range of voxel values associated with a local maximum or a global maximum in the distribution. wherein 6. The method of any one of claims 2 to 5, determining the visual parameter map comprises determining a function defining the visual parameter map based on the result of the analysis process. wherein, 7. The method of any one of claims 1 to 5, comprising: determining parameters of the analysis process based on the given type of the first anatomical object (210, 220).

8. The method of any one of claims 1 to 5, parameters of the selection process are based on the given type of the first anatomical object (210, 220) represented by the first image data. wherein, 9. The method of any one of claims 1 to 5, parameters of the analysis process are determined based on context information related to the 3D medical image data. wherein ​ 10. The method of any of claims 1 to 5, wherein the visual parameter map is a transfer function for a volume rendering process.

11. The method of claim 10, wherein, the transfer function is configured to provide opacity values and / or color values of the first image data for use in the volume rendering process.

12. The method of any of claims 1 to 5, wherein the first anatomical object (210, 220) comprises an anatomical organ.

13. The method of any of claims 1 to 5, wherein the 3D medical image data comprises a plurality of 3D medical image data sets, and the selection process comprises: selecting (506) a first 3D medical image data set of the plurality of 3D medical image data sets; identifying a portion of the first 3D medical image data set representing the first anatomical object; selecting (510) a second 3D medical image data set of the plurality of 3D medical image data sets; and selecting (512) the first image data from the second 3D medical image data set based on the identified portion of the first 3D medical image data set.

14. A method (400) of generating a computer-based visualization of 3D medical image data, the method comprising: performing (402) the method of any of claims 1 to 13 to obtain a visual parameter map for a first portion of 3D medical image data; and performing (404) a rendering process to generate a visualization of the 3D medical image data, wherein performing the rendering process comprises applying the visual parameter map for the first portion of 3D medical image data.

15. A computer program product comprising machine readable instructions which, when executed by a processor, cause the method of any of claims 1 to 14 to be performed.

16. A machine readable medium having stored thereon a computer program product according to claim 15.

17. An apparatus (600) comprising a processor (604) and a memory (606) comprising a set of machine readable instructions which, when executed by the processor (604), cause the processor (604) to perform the method (100, 400) of any of claims 1 to 14.

Images