Apparatus and method for providing enhanced two-dimensional imaging data

Abstract

To provide enhanced two-dimensional imaging data through removal of one or more selected features.SOLUTION: An apparatus for providing enhanced two-dimensional digital imaging data is configured to: obtain three-dimensional digital imaging data 3A indicating at least one physical feature; determine, based on the three-dimensional digital imaging data, data representing a three-dimensional model of at least one element 202, 204, 206, 208 segmented from the three-dimensional digital imaging data; select one or more elements as non-required elements 206, 208; and provide enhanced two-dimensional digital imaging data 3B indicative of the at least one physical characteristic, in which the one or more non-required elements are essentially removed from the two-dimensional imaging data.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates generally to image data, and more particularly to an apparatus and method for providing enhanced two-dimensional image data through the removal of selected features. [Background technology]

[0002] Images used in medical applications such as radiology are of great importance to be able to diagnose and treat different kinds of disorders. To be able to interpret them efficiently by medical personnel, it is crucial to obtain high quality images that provide the relevant information in each case.

[0003] Traditional two-dimensional medical images, such as native X-ray images, can provide only limited information about tissues of particular interest. Although information is provided about all physical features hit by the X-ray beam, such as projection images obtained using information about how the intensity of the X-ray beam attenuates as it passes through tissue, in the resulting projection images, the tissue of interest may be lost, buried, or obscured by information about other redundant physical features, making meaningful analysis of the physical features of interest difficult or impossible. The images generated may not clearly show the tissue of interest due to information related to the redundant tissue or other redundant physical features that confound the projection image formation. In these cases, diagnosis or treatment may be hindered.

[0004] Three-dimensional digital medical image data may be readily utilized to generate two-dimensional images. Although various techniques exist for converting such three-dimensional image data into two-dimensional projection images, these still suffer from the aforementioned problem that the tissue of interest may be affected in a deleterious manner, in whole or in part, by data related to other tissues, and rendering the two-dimensional image may not provide sufficient information about the tissue of interest.

[0005] Although three-dimensional image data may be used to obtain visualizations incorporating three-dimensional aspects, in certain situations, two-dimensional images may be required to provide data related to tissues of interest (e.g., for use by insurance companies). Although three-dimensional imaging techniques may be used to provide two-dimensional images (e.g., computed tomography (CT) slice images), these two-dimensional images may lack information related to three-dimensional aspects, such as, for example, the patient's tissues. Summary of the Invention

[0006] It is an object of the present invention to alleviate at least some of the problems associated with the known prior art. According to one aspect of the present invention, an apparatus for providing enhanced two-dimensional digital image data is provided, the apparatus including at least one processor, the processor acquiring three-dimensional digital image data indicative of at least one physical feature, determining, based on the three-dimensional digital image data, data representative of a three-dimensional model of at least one element segmented from the three-dimensional digital image data, the element including at least one of the physical features. The apparatus further includes: selecting one or more elements as unwanted elements; determining data components of the three-dimensional image data related to the unwanted elements; and providing enhanced two-dimensional digital image data indicative of the at least one physical feature, from which the one or more unwanted elements have been essentially removed from the two-dimensional image data, based on a remaining portion of the at least three-dimensional image data from which the data components related to the unwanted elements have essentially been removed.

[0007] According to another aspect, a method is provided according to independent claim 14 and a computer program product is provided according to independent claim 15.

[0008] Advantageous embodiments of the present invention may provide an apparatus and method for acquiring enhanced two-dimensional image data that allows for important physical features to be more clearly visualized, essentially eliminating elements such as unimportant physical features. In the prior art, two-dimensional image data that may be acquired in similar situations may include data relating to unwanted / uninteresting physical features that may obscure data related to physical features of interest.

[0009] The extended two-dimensional image data is data that can be used to generate projection images that can be advantageously used, for example, in plain radiography. The extended two-dimensional image data is, however, not limited to, for example, x-rays. The extended two-dimensional image data can be considered as projection image data and can be related to various types of three-dimensional image data that can be obtained via various imaging methods.

[0010] The extended two-dimensional image data may correspond / associate with or be used to provide a digitally reconstructed radiograph (DRR) image. The extended two-dimensional image data may be utilized to provide a composite radiographic image.

[0011] Digitally reconstructed radiation image data utilizes three-dimensional image data from which portions of the original data associated with unwanted elements have essentially been removed, and uses the remaining portion of the digital three-dimensional image data in a three-dimensional projection method to obtain two-dimensional projection image data.

[0012] Embodiments of the present invention may provide new and / or expanded possibilities for medical professionals to gain knowledge based on enhanced two-dimensional image data.

[0013] Even now, it may be desirable to utilize two-dimensional images for certain purposes in place of three-dimensional models or typical images that may be obtained by three-dimensional imaging techniques, particularly in cases such as orthopedics, where two-dimensional image projections are typically viewed on a flat X-ray projection screen, whereby the provision of extended projection data and images that may be generated thereby may be facilitated by the present invention, offering the potential for improvements in orthopedic uses.

[0014] The enhanced two-dimensional images or data that may be provided through embodiments of the present invention may also be advantageous for providing insurance companies and / or medical systems, etc. with images or image data that correspond to traditional X-ray projection data.

[0015] In one embodiment, the extended two-dimensional digital image data may include data indicating a selected viewing angle, optionally with respect to the region of interest. The selected viewing angle may be an optimized viewing angle. Thus, using the present invention, for example, digitally reconstructed radiological image data may be acquired, resulting in an extended two-dimensional image at the selected viewing angle, with the viewing angle correct and with unwanted elements removed. Such an image may not be acquired in a conventional manner, where an X-ray image is taken from the patient. In such a case, the patient may not be positioned in a correct position, so that an X-ray image may not be acquired accurately from the desired viewing angle. In other prior art methods where the viewing angle of the projected image may be theoretically selected, unwanted elements that may be present may still obstruct the view of the element / region of interest.

[0016] In embodiments of the present invention, maximum visibility of the region of interest may be ensured. Thus, in advantageous embodiments, an optimized digitally reconstructed radiographic image may be obtained, where the viewing angle of the DRR image is optimized.

[0017] In one embodiment, the digitally reconstructed projection image data may be generated by simulating the attenuation of a beam of electromagnetic radiation passing through the remaining elements (elements not including unwanted elements) based on the remaining portion of the three-dimensional digital image data, where the attenuation of the beam is determined by local intensity data of the three-dimensional digital image data.

[0018] In an embodiment of the present invention, a three-dimensional model of at least one element is generated or determined based on the three-dimensional digital image data, which may be performed by utilizing image segmentation.

[0019] An element herein refers to an entity that can be segmented from the three-dimensional digital image data. An element includes at least one physical feature that is represented by the three-dimensional digital image data. In some embodiments, an element corresponds to or is equivalent to only a physical feature, although in some cases, an element may also include noise data, etc. In some cases, the terms "physical feature" and "element" may be used interchangeably, as they may be essentially equivalent.

[0020] In some embodiments, the image segmentation method may include a method selected from the group of thresholding, clustering, histogram-based methods, edge detection, region growing, model-based methods, semi-automatic segmentation, and trainable segmentation. Of course, various other methods for image segmentation may also be feasible.

[0021] Through the present invention, it is not only from, for example, images or other two-dimensional data that physical features may be removed: using the data representing the determined three-dimensional model, elements, for example, physical features, can be effectively distinguished from one another, and one or more elements can be selected as unwanted elements and removed from the data representing the three-dimensional model, and / or subsequently provided with enhanced two-dimensional image data, without essentially affecting the data or the image associated with the other physical features.

[0022] The remaining portion of the three-dimensional image data may, in some embodiments, be obtained through masking of the three-dimensional image data based on the data representing the three-dimensional model. Masking may be performed by correlating data components of the three-dimensional image data with data points of the data representing the three-dimensional model. Masking may be used in determining which data components of the three-dimensional image data are associated with unwanted elements, and these data components may be removed in the remaining portion of the three-dimensional digital image data.

[0023] If removal of selected unwanted elements is attempted without determining data representing a three-dimensional model of the elements that can be identified from the three-dimensional image data, the quality of the extended image data may be reduced, since the effect of the redundant elements may not be considered or limited to the same extent as can be achieved with the present invention. In the present invention, it may be possible to more accurately determine which elements of the three-dimensional image data are related to or correspond to the unwanted elements, since the elements (e.g., voxels) can be determined, for example, by masking with data representing a three-dimensional model of the elements. Without data representing a three-dimensional model of the elements, it may not be possible to accurately identify which voxels, etc. to remove / ignore from the three-dimensional image data before utilizing the data in the selected (three-dimensional) projection.

[0024] In an embodiment of the invention, providing the enhanced two-dimensional digital image data comprises determining components of the three-dimensional digital image data based on data representing a three-dimensional model of the elements, removing at least a portion of the three-dimensional digital image data associated with one or more unwanted elements, essentially removing / ignoring such data components associated with the unwanted elements, and utilizing the remaining portion of the digital three-dimensional image data in a three-dimensional projection.

[0025] In some embodiments, the three-dimensional projection method may be selected from the group of maximum intensity projection, minimum intensity projection, shaded surface display, volume rendering, and virtual endoscopy. Thus, the examples presented here are methods that can be used to convert three-dimensional image data into a two-dimensional image (or two-dimensional image data), and other methods known to those skilled in the art may also be feasible.

[0026] The physical features herein may include tissues, for example tissues selected from the group of bone, ligament, cartilage, soft tissue. The obtained three-dimensional digital image data may also show other physical features. For example, implants and / or other objects may also be identified from the three-dimensional image data.

[0027] The elements (which can be selected as unwanted elements) can be or correspond to physical features depicted in the three-dimensional image data in various use cases, or the elements can be elements that can be derived as noise data from data related to the physical features.

[0028] In some embodiments, the device can be used to enhance the image quality of two-dimensional image data, such that the elements segmented from the two-dimensional image data may include physical features such as tissue, and one or more elements may include noise data or other data that is a disturbance resulting from the image / characterization system that may be segmented from the three-dimensional image data. One or more elements that constitute the noise data can then be selected as unwanted elements. The noise data can then be removed from the subsequently generated enhanced two-dimensional image data. The enhanced two-dimensional image data may then be of higher quality compared to prior art image data in which the noise data is present.

[0029] In one embodiment, the selection of one or more elements as unwanted elements may be based on predefined criteria, which may indicate one or more elements that are not of interest, and which may be obtained via a user interface included in the device.

[0030] In one use case scenario, a medical professional may be provided with augmented two-dimensional image data, and selected unwanted bones are essentially removed from the augmented two-dimensional image to make the important bones more clearly visible, where the important bones may be occluded by data associated with the unwanted bones in images obtainable via prior art methods / apparatus.

[0031] In embodiments of the invention, enhanced two-dimensional image data may be provided that may be more useful than that which may be obtained with other available methods. Information related to physical features of interest may be more efficiently inferred from the enhanced two-dimensional data. For example, diagnosis may be made more quickly, and the invention may even provide diagnostic or therapeutic possibilities when this is not possible with available methods.

[0032] In some embodiments, the device may be configured to generate an output. The output may include a visualization of the determined data representing a three-dimensional model of the at least one element, and / or a visualization of the enhanced two-dimensional digital image data. The output may, in some embodiments, additionally or alternatively include a computer-readable output including the enhanced two-dimensional digital image data, which may then be used in another device for generating an enhanced two-dimensional image, etc. Another device for generating an enhanced two-dimensional image. In some embodiments, the device may include or be in communication with a printing device, such that the output may be utilized to obtain a printed enhanced two-dimensional image.

[0033] Benefits that may be achieved through embodiments of the present invention may also include reduced radiation exposure to a patient whose image is acquired to obtain image data. For diagnostic purposes, in order to be able to interpret a two-dimensional projection image, the image must be of adequate quality and show relevant features. In many cases, it is determined that additional image data needs to be acquired, for example, due to an X-ray image taken from an incorrect angle for a particular purpose. The present invention may generate extended two-dimensional image data and an extended two-dimensional image derived therefrom, so that the data is essentially always shown from the correct image angle and any selected projection may be acquired. Thus, a minimum number of actual imaging situations may be used, reducing radiation exposure to the patient, since additional imaging data may not need to be acquired via multiple subsequent imaging situations.

[0034] The exemplary embodiments presented herein should not be interpreted as creating limitations on the applicability of the appended claims. In this specification, the verb "comprise" is used as an open-ended limitation that does not exclude the presence of unspecified features. The features recited in the dependent claims can be freely combined with each other, unless expressly stated otherwise.

[0035] The novel features which are believed to be characteristic of the invention are set forth with particularity in the appended claims. The invention itself, however, both as to its structure and its method of operation, together with additional objects and advantages thereof, will best be understood from the following description of certain illustrative embodiments when read in connection with the accompanying drawings.

[0036] The presented discussion regarding various apparatus embodiments can be flexibly applied to method embodiments, mutatis mutandis, as will be appreciated by those skilled in the art, and vice versa. [Brief description of the drawings]

[0037] [Figure 1] FIG. 1 illustrates an exemplary apparatus according to an embodiment of the present invention. [Diagram 2] FIG. 2 illustrates an exemplary visualization of a slice of three-dimensional image data that may be utilized in one use case scenario of an embodiment of the present invention. [Diagram 3] FIG. 3 shows at 3A and 3B example visualizations of data representing a three-dimensional model that may be utilized in one use case scenario of an embodiment of the present invention, with 3B showing a visualization of the three-dimensional model with unwanted physical features removed. [Figure 4] FIG. 4 shows at 4A a visualization of a three-dimensional model based on which unwanted physical features have been removed, and at 4B a visualization of the augmented two-dimensional image data that may be provided in one exemplary use case scenario of an embodiment of the present invention. [Diagram 5] FIG. 5 illustrates an exemplary visualization of a slice of three-dimensional image data that may be utilized in another use case scenario of one of the embodiments of the present invention. [Figure 6] FIG. 6 shows exemplary visualizations at 6A and 6B of data representing a three-dimensional model that may be utilized in another use case scenario of an embodiment of the present invention, where 6B shows a visualization of the three-dimensional model with unnecessary physical features removed. [Figure 7]FIG. 7 shows visualizations of conventional 2D image data at 7A and 7B, and visualizations of augmented 2D image data that can be obtained in a use case scenario via an embodiment of the device at 7B and 7D. [Figure 8] FIG. 8 shows a flow chart of a method according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0038] The invention will now be explained in more detail with reference to exemplary embodiments according to the accompanying drawings, in which: FIG.

[0039] 1 shows an embodiment of an apparatus 100 according to the invention. The apparatus 100 comprises at least one processor 102 configured to acquire three-dimensional digital image data.

[0040] The processor 102 may be configured to execute instructions embodied in the form of computer software stored in a memory, which may refer to, for example, one or more memory chips separate or integrated with the processor 102. The memory may store other data in addition to program instructions. The software may define one or more applications that perform the activities described herein and may implement one or more computational algorithms that perform the operations included in the methods.

[0041] The processor 102 may include or be in communication with one or more databases 104 that may be used to store or retrieve information. The databases 104 may be stored in a memory associated with the processor 102.

[0042] The processor 102 may be a local processor contained in a personal computer-type computing device, or the processor 102 may be one or more remote processors that may be connected, for example, via wireless means.

[0043] A user interface (UI) 106 may provide, for example, a user of the device 100 with control and information access tools, for example, for controlling the device and for providing input data for the processor 102 .

[0044] The three-dimensional digital image data may be provided to the processor 102 as input data, for example in the form of DICOM data. For example, the DICOM data may be provided directly to the processor 102 via an imaging device via a wired or wireless connection (such as WLAN), or the DICOM data may be imported in some other manner, for example via a carrier medium or transferred as a signal or combination of signals from a sending element to a receiving element, either wired or wirelessly, and provided to the processor 102.

[0045] The three-dimensional image data may be three-dimensional image data obtained by a scanning technique such as computed tomography, magnetic resonance imaging, or ultrasound scanning.

[0046] The three-dimensional digital image data is indicative of at least one physical feature of a target entity, such as a human patient, that has been subjected to a selected imaging methodology. The physical feature may be tissue, such as bone, ligament, cartilage, or soft tissue. The three-dimensional digital image data may, in various embodiments, be indicative of multiple physical features of one or more tissues or tissue portions, for example.

[0047] The processor 102 is configured to determine, based on the three-dimensional image data, data representing a three-dimensional model of at least one element that can be segmented from the three-dimensional image data. The at least one element includes at least one physical feature indicated by the three-dimensional image data. In the use case scenario described below, the element essentially corresponds completely to the physical feature indicated by the three-dimensional image data. Thus, the element may also be referred to as a physical feature herein. The data representing the three-dimensional model of the at least one element may include points in three-dimensional space to represent the at least one element, such as a bone or other physical feature, that can be identified from the three-dimensional image data.

[0048] In some cases, the obtained 3D image data may not be fully utilized, i.e. the 3D image data may show some physical features that are not included in the data representing the model of the entity. This is the case, for example, when the 3D image data covers an area of ​​an entity, such as a patient, that is larger than the area of ​​interest in the case.

[0049] In some embodiments, data representing the three-dimensional model may be presented to a user of the device 100 as a visual image or visualization. The device 100 may include or be in communication with one or more display elements 108 that display the creation and visualization.

[0050] Data representing a three-dimensional model of at least one element may be obtained from the three-dimensional digital image data utilizing any type of method available for converting the three-dimensional image data into data representing a three-dimensional model of such element, as a physical feature identifiable from the three-dimensional image data, as known to the skilled person. Image segmentation methods may be used to divide the digital image into a number of segments or image objects. The pixels or voxels of the image may be classified or characterized to group the pixels or voxels or to determine which pixels or voxels have similar characteristics. Thus, the result obtained from the three-dimensional image data via image segmentation methods is data representing a three-dimensional model of at least one element, the element being segmented from the image data, and thus the data representing the three-dimensional model provides information regarding the three-dimensional aspects of the segmented element.

[0051] In some embodiments, one or more additional modeling methods may be utilized to obtain a 3D reconstructed model or three-dimensional representation from the data representing the three-dimensional model of the at least one element. For example, in some use case scenarios, an image-based mesh may be used to further process the data representing the three-dimensional model of the at least one element. This further processed data may be used, for example, in creating a visualization or other visualization of the data representing the three-dimensional model of the at least one element.

[0052] Through the utilized modeling methods, such as image segmentation, one or more elements, e.g., physical features, can be identified / distinguished from the three-dimensional digital image data such that the elements are individuated and presented as entities within the data representing a three-dimensional model of the one or more elements.

[0053] For example, the visualizations disclosed herein can be obtained by using additional modeling methods, such as image-based mesh creation, but such methods need not be used to practice the present invention. Thus, in embodiments of the present invention, visualization of three-dimensional models or 3D reconstructed models is not necessary, but is presented here for ease of understanding.

[0054] One or more elements may then be selected as unwanted elements. The selected unwanted elements may be elements or features that are not of interest in the particular use case scenario, in other words, elements or features that are not of interest / important / relevant. The selected unwanted elements may be, for example, bone or other tissue that may obstruct the display of more interesting elements or physical features. Thus, the selected unwanted elements may also be physical features that are detrimental or disruptive to other physical features that are inspected in the two-dimensional image in the particular use case scenario.

[0055] The selection of the unwanted elements can be based on predetermined criteria. Such predetermined criteria can be provided by a user of the device 100, or the predetermined criteria can be provided or inferred by the device 100 in an automated manner, for example based on other information provided by the user, etc.

[0056] In some embodiments, a user of the device 100 may select unwanted elements through visual inspection of the visualization of the three-dimensional model provided via the display element 108. This may be done via the UI 106, for example, by the user clicking on a visualized element, e.g., a physical feature, in the three-dimensional model. Also, instead of the user explicitly selecting the unwanted elements, the user may select elements of interest, and one or more of the remaining elements may be considered unwanted elements.

[0057] In some embodiments, the user may select unwanted elements, for example, by naming such elements or selecting from a list provided by the device 100 via the UI 106 (or, alternatively, by naming the elements / physical features of interest and the processor 102 considers the remaining elements identified in the three-dimensional model as unwanted elements).

[0058] Automatic recognition of elements based on predefined criteria may be utilized and automatic selection of elements recognized as unnecessary may be performed, for example, via information available in a database. For example, the device 100 may recognize elements as element type, which may identify an element as a particular bone, or tissue type, which may identify tissue as bone tissue or adipose tissue, from properties derivable from the three-dimensional image data (such as intensity data) and / or data representing a three-dimensional model of the element.

[0059] In an alternative embodiment, the user may select (or facilitate the selection of) the unnecessary elements by providing information to the device 100 regarding the use case scenario. For example, the user may provide information indicating a medical condition to be examined. Information available to the device 100, e.g., via the database 104, may then be utilized to select the unnecessary elements / features based on available information relating the information provided by the user to one or more particular tissues, etc., that may be deemed unnecessary. (Or, of course, important physical features may instead be identified, in which case the remaining identified elements may, for example, be deemed unnecessary). As an example, the user of the device 100 may indicate, via the provided augmented two-dimensional image data, that a particular type of fracture should be examined. The device 100 may then select the unnecessary elements based on this information and provide, for example, information regarding a particular bone in the augmented two-dimensional image data.

[0060] In one embodiment, the selected unwanted feature may be a part of a physical feature, for example, in some cases a bone or the like may be identified as a feature from the three-dimensional image data, but the selected unwanted feature may be a part of said bone.

[0061] The apparatus 100 may then provide at least one data representing a three-dimensional model of the at least one physical feature, and enhanced two-dimensional image data indicative of the at least one physical feature, with one or more unnecessary elements omitted from the two-dimensional data based on selection of the at least one unnecessary element by processing a remaining portion of the three-dimensional image data from which data components related to the unnecessary element have been removed.

[0062] Data components of the three-dimensional image data related to the unwanted features may be determined by masking the three-dimensional image data based on data representing the three-dimensional model, whereby a remaining portion of the three-dimensional image data may be determined, which remaining portion is utilized to generate the augmented two-dimensional image data.

[0063] The enhanced two-dimensional image data can then be used to create a two-dimensional image in which one or more physical features are visualized and selected unwanted elements, e.g., physical features, are removed, thus essentially removing or subtracting the unwanted elements from the image (data).

[0064] One or more unwanted elements being removed from the extended two-dimensional image data may mean that data associated with the unwanted elements may be essentially removed from the two-dimensional image data, at least to the extent that the unwanted elements are not clearly visible in a two-dimensional image that may be generated using the extended two-dimensional image data. Thus, the extended two-dimensional image data may, in some embodiments or use case scenarios, include residual data associated with the unwanted elements, but this residual data may be minimal, or at least below a certain threshold.

[0065] Enhanced two-dimensional image data may be provided based on data representing a three-dimensional model of the at least one element, a selection of the at least one unwanted element, and the three-dimensional image data, where a portion of the three-dimensional image data relating to one or more unwanted physical features is removed from the three-dimensional image data and a remaining portion of the digital three-dimensional image data is used in conjunction with the three-dimensional projection.

[0066] Removal of at least a portion of the three-dimensional image data relating to one or more unwanted elements may be performed by comparing or masking the three-dimensional image data with data representing at least one three-dimensional model, recognizing / identifying one element and data components of the three-dimensional image data that correspond to the unwanted elements and removing / ignoring such data components in order to obtain the remaining portion of the three-dimensional digital image data.

[0067] A 3D projection may be any projection method that can be used to convert three-dimensional image data into two-dimensional image data. Thus, it may use a method that obtains points in a two-dimensional plane based on points in three-dimensional space. Thus, a 3D projection may be a method used to map or project three-dimensional data into two-dimensional data.

[0068] In relation to medical imaging data, three-dimensional imaging data, such as DICOM data, provides information about the intensity of an energy beam as it passes through a physical feature, such as tissue, and may experience attenuation of the beam's intensity when encountering the physical feature.

[0069] Thus, the different three-dimensional projection methods used may use different methods of mapping or projecting the intensity data, where the intensity data is equivalent to or included in the three-dimensional image data.

[0070] Typically, a two-dimensional projection image may be obtained from three-dimensional image data, such that all voxels contained in or derivable from the DICOM data, etc., are utilized. The resulting two-dimensional image may have physical features of interest not visualized with sufficient quality due to interference data associated with unwanted physical features.

[0071] Data relating to the intensity of the energy beam as it passes through tissue etc. may be used as three-dimensional image data such that the data is summed over at least one viewing angle in the two-dimensional image data.

[0072] In an advantageous embodiment of the present invention, the extended two-dimensional image data may be digitally extended projection image data, converting the three-dimensional image data into two-dimensional image data, such that voxels contained in or derivable from the three-dimensional image data that represent unwanted physical features may be removed or ignored.

[0073] For example, portions of the three-dimensional image data relating to unwanted elements, such as physical features, may be removed in a sum that may be used to obtain the extended two-dimensional image data.

[0074] In one embodiment, the three-dimensional digital image data may be modified, for example based on masking, to include only the remaining portions (data relating to unwanted elements is removed), and the modified three-dimensional image data may then be utilized in a projection method or the like to obtain extended two-dimensional digital image data.

[0075] In another embodiment, the three-dimensional image data itself may not be altered, but only the remaining portions are utilized, such as through masking, to obtain the extended two-dimensional digital image data.

[0076] In the context of providing radiological images, in a native X-ray scan, a stream of photons arrives from an emitter and penetrates the patient, where some of the photons are absorbed in the tissue (at a rate that correlates with the type of tissue passed through and is at least somewhat known), and the remaining stream of photons reaches a detector on the other side of the patient. The 2D distribution of these transmitted photons produces a clinical 2D X-ray image (conventional). Photon absorption corresponds to the attenuation of the X-ray / electromagnetic radiation as it passes through tissue.

[0077] Then, in connection with three-dimensional digital image data, for example in a CT image, one can obtain three-dimensional image data in which each 3D pixel (voxel) carries an intensity value, which likewise correlates with the tissue type at hand. In the reconstruction of a digital radiograph, one can calculate / determine (for example by ray-casting methods) several "simulated" photons / photon fluxes corresponding to an X-ray passing through the three-dimensional digital image data at a specific location. At each linear increment along this simulated ray of electromagnetic radiation, the three-dimensional digital image data can be considered to attenuate the ray at a rate corresponding to the local (pixel / voxel) intensity of the three-dimensional digital image data (tissue radiopacity, in other words, absorption rate). Thus, once the entire ray has been calculated, the final result may resemble how a real physical photon flux, such as an X-ray, attenuated along a line corresponding to the calculated (ray-cast) ray. Thus, the three-dimensional digital image data may, via local intensity data, indicate the element type (e.g. bone or other tissue) in the (three-dimensional) projection method used, such as the simulated (X-ray) attenuation, and rays may be simulated / calculated such that the final result (DRR) essentially corresponds to a conventional X-ray image obtained from the tissue / element etc.

[0078] The local intensity data or relative density data at each point of the three-dimensional digital image data may be calibrated to a Hounsfield value. On the Hounsfield scale, a value of "1000" corresponds to air and a value of "0" corresponds to water. Using the intensity values ​​and the Hounsfield scale, tissue types may be determined from the three-dimensional digital image data (e.g., dense bone is +300, fat is -100 to -50). The correlation between tissue types and Hounsfield values ​​may be known or may be entered into the device 100 or may be available to the device 100 via a database or the like.

[0079] Having knowledge in 3D terms that the unwanted elements are (precisely) located in the three-dimensional digital image data may be taken into account, for example, during the ray casting step, when adding the contribution of each linear position to the final result (DRR or extended two-dimensional digital image data), or when removing or ignoring the corresponding voxels in the calculation of ray attenuation. Alternatively, data components related to the unwanted elements may essentially be removed from the three-dimensional digital image data, and then a projection is performed (e.g., calculation of simulated attenuation of X-rays, etc. passing through the remaining three-dimensional digital image data). Depending on the implementation details, the final result will essentially be the same.

[0080] If linear attenuation is not the most clinically useful projection representation, other projection methods may be chosen (e.g., maximum intensity projection collects the maximum value along each raycast, such that nothing below the maximum pixel intensity contributes to the final result).

[0081] In calculating the DRR data, the selected viewing angle may also be taken into account so that the simulated light rays are directed accordingly.

[0082] In particular, one aspect of an advantageous embodiment of the present invention may use the generated data representing the three-dimensional model of the element to effectively determine which data components, voxels, etc. contained in or derivable from the three-dimensional image data that are associated with which physical features or elements. Data associated with the selected unwanted elements or physical features may then essentially be removed without removing data associated with the remaining physical features of interest.

[0083] A visualization of the augmented two-dimensional image data and / or other data (such as a three-dimensional model) may be provided to a user of the device via the display element 108.

[0084] 2-4 relate to one use case scenario of an exemplary embodiment of the present invention. FIG. 2 illustrates an exemplary visualization of three-dimensional image data that may be received or utilized by the device 100 in an embodiment of the present invention. The illustrated slice of image data in FIG. 2 illustrates a number of physical features. In FIG. 2, the physical features may be several bones 202, 204, 206, 208 of the imaged patient's foot. Here, only a portion of the physical features in the figure are numbered. Of course, various additional image slices may also be provided, more physical features may be identified, and / or the physical features may be qualified at various depths and / or viewing angles.

[0085] As can be readily appreciated from FIG. 2, the visualization, in other words the two-dimensional image that may be obtained from the three-dimensional image data, may not be very informative in itself, and especially not separately, regarding physical features or any three-dimensional surface.

[0086] Figure 3, at 3A, shows an exemplary visualization of data representing a three-dimensional model of physical features 202, 204, 206, 208 and 210 that may be identified utilizing at least the data of Figure 2. Of course, the slices visualized in Figure 2 are shown by way of example only and are insufficient to obtain data related to the three-dimensional model visualized in Figure 3 (bone 210 is not shown in Figure 2, data related to this physical feature is not feasibly displayed, and thus the features were obtained using supplemental image data that cannot be shown in full here). Data representing the three-dimensional model may be obtained by utilizing image segmentation.

[0087] In some embodiments, the data representing the three-dimensional model may be visualized and provided to a user of the device, and in one embodiment, the user may rotate the three-dimensional model thus depicted (e.g., one corresponding to the model of FIG. 3A) and inspect the physical features.

[0088] One or more physical features identifiable from the three-dimensional model may be selected as unwanted physical features. Selection of unwanted physical features may also refer herein to implicit selection of unwanted physical features by selection of important physical features that should not be included in the group of unwanted physical features.

[0089] In one embodiment, a three-dimensional model may be visualized for inspection by a user of the device 100, and the user may then select unwanted physical features, such as by clicking on one or more visualized physical features, via the UI 106.

[0090] 3A, physical features other than bones 202, 204, and 210 may be selected as unwanted. Thus, for example, bones 206 and 208 may be selected as unwanted physical features along with other, or preferably all, other depicted physical features. This selection may be by, for example, a user of the device indicating selection of bones 206, 208, etc. as unwanted, or indicating an interest in bones 202, 204, and 210, thus implicitly selecting bones 206, 206, etc. as unwanted.

[0091] Data may then be provided that represents a three-dimensional model of at least one physical feature from which the unwanted physical features are removed. FIG. 3B shows a visualization of such data, where only bones 202 and 202 are included. Thus, a three-dimensional model may be provided that includes the physical features identified via the image data and from which the unwanted physical features may be removed. Such a model or visualization (i.e., corresponding to FIG. 3B) does not necessarily have to be created or depicted, as the augmented two-dimensional image data may be obtained through the original three-dimensional model (corresponding to FIG. 3A) and information regarding the selected unwanted physical features.

[0092] In some embodiments, a visualization of the three-dimensional model may be provided to a user of the device, and the user may indicate the viewing angle based on which the augmented two-dimensional image data is provided / generated. For example, the user may rotate the three-dimensional model (perhaps with selected unwanted features removed) on the display 108 screen to arrive at a desired viewing angle and indicate that the augmented two-dimensional image data should correspond to the image as seen from that angle.

[0093] The selected viewing angle for the augmented two-dimensional digital image data (or the augmented two-dimensional digital image) may be selected by a user of the device, or the selected viewing angle may be determined by the device as information characterizing the region of interest (e.g., data regarding a particular joint) based on, for example, such a use case scenario. The selected viewing angle may be an optimized viewing angle that is optimized for the region of interest.

[0094] Visualizations of an exemplary use case scenario are shown in Figures 4A and 4B. 4A shows a visualization of data representing a three-dimensional model of at least one physical feature with unwanted physical features removed (showing only bones 202, 204, and 210 with other bones removed). 4B shows a visualization of the augmented two-dimensional image data (augmented projection image) that essentially includes only bones 202, 204, and 210.

[0095] Related to the use case scenario above, the ankle AP talocrural arthrogram is part of a three view series of the distal tibia, distal fibula, talus, and proximal metatarsal. It is the most appropriate projection to evaluate the articulation of the two malleolar joints with the tibial plafond and the talar dome, also known as the ankle tenon joint.

[0096] In the scenario of Figure 4, the enhanced 2D image in 4B shows the ankle mortise and tenon joint from a selected / optimized viewing angle. The selected viewing angle allows for an optimal view of the joint, but also provides an unobstructed view as unnecessary elements have been removed in the enhanced 2D image.

[0097] Figures 5-7 relate to another possible / example use case scenario of one embodiment of the present invention. Figure 5 shows an example slice of three-dimensional image data that may be acquired by the device 100 showing multiple bones. Figure 6 shows one example visualization of data representing a three-dimensional model of a physical feature, in other words bones that may be identified from the data visualized in Figure 5 (and other slice data associated with the same target three-dimensional image data).

[0098] Figure 6A shows an example visualization of data representing a three-dimensional model of physical features that may be identified from the data visualized in at least Figure 5. Figure 6B shows a visualization of the physical features with selected unwanted physical features removed. As mentioned above, the selection of unwanted physical features may be made explicitly or implicitly, which may be made by a user of the device, or may be made automatically or at least semi-automatically based on information received or available from a user of the device 100, for example.

[0099] Providing visualizations such as the example visualizations presented herein in connection with the example use case scenarios is not necessary for operation of device 100, but in some cases may be advantageous to provide a user of the device with information related to identified physical information, such that the user may more effectively determine which physical features to select as unwanted physical features.

[0100] Figure 7 shows in Figures 7A and 7B a visualization of conventional two-dimensional imaging data that may be acquired with the prior art method. Figure 7A shows a conventional native X-ray image that may be acquired from a target in the same or similar cases corresponding to Figures 5-6 above. Figure 7B shows a two-dimensional projection image that may be obtained from the three-dimensional image data where data representing a three-dimensional model of the physical features is not used and unwanted physical features have not been removed.

[0101] From Figures 7A and 7B, it can be difficult to distinguish between different physical features, to see / distinguish a particular physical feature from an image, and / or to determine some relevant data related to a particular physical feature when other physical features interfere or even hinder the interpretation.

[0102] 7C and 7D show example visualizations of augmented two-dimensional image data (augmented projection images) that may be acquired through an embodiment of the device. 7C and 7D correspond to the use case scenarios shown in FIGS. 5-6 and may be acquired using three-dimensional image data and determined data and selected unwanted physical features representing a three-dimensional model (visualization of data representing a three-dimensional model in FIGS. 6A and 6B, with unwanted physical features removed in FIG. 6B). 7C and 7D show augmented projection images that may be acquired in different ways of acquiring two-dimensional image data from three-dimensional image data in which intensity data is mapped or projected in different ways.

[0103] The unwanted physical features are essentially removed in Figures 7C and 7D. Figures 7C and 7D show that the remaining visualized physical features may be more clearly visible than in Figures 7C and 7D. Using extended two-dimensional image data, such as that corresponding to the visualization of Figures 7C and 7D (in other words, those physical features shown in the obtained three-dimensional data are not among the selected unwanted physical features), a medical professional can more effectively arrive at a diagnosis or a preferred treatment plan related to a medical condition related to the physical features, which is taken into account with the extended two-dimensional image data, compared to when the medical professional could only consider conventional two-dimensional image data such as that shown in Figures 7A or 7B.

[0104] 8 shows a flow chart of a method according to an embodiment of the present invention. At 802, three-dimensional image data indicative of at least one physical feature is obtained. At 804, data representative of a three-dimensional model of at least one entity that can be identified or segmented from the three-dimensional image data is determined, the at least one entity including the at least one physical feature, and at 808, one or more unwanted entities are selected.

[0105] Data components of the three-dimensional image data that relate to the unwanted elements may be determined, while data related to the unwanted elements may be removed from the three-dimensional image data at 810 .

[0106] Finally, at 812, enhanced two-dimensional image data indicative of the at least one physical feature is provided, and selected unwanted elements are essentially removed by utilizing the remaining portion of the three-dimensional image data in a three-dimensional projection.

[0107] In some embodiments, determining data components within the three-dimensional image data that relate to unwanted elements and / or determining remaining portions of the three-dimensional digital image data is performed through masking of the three-dimensional image data based on data representing a three-dimensional model.

[0108] The present invention has been described above with reference to the aforementioned embodiments, and some advantages of the present invention are demonstrated. It is clear that the present invention is not limited to these embodiments, but includes all possible embodiments within the inventive concept and the spirit and scope of the following claims. The features recited in the dependent claims are mutually freely combinable, unless expressly stated otherwise.

Claims

1. An apparatus (100) for providing enhanced two-dimensional digital image data, comprising: at least one processor (102); The processor, acquiring three-dimensional digital image data indicative of at least one physical feature (202, 204, 206, 208); determining, based on the three-dimensional digital image data, data representative of a three-dimensional model of at least one element segmented from the three-dimensional digital image data, the element including at least one of the physical features; Selecting one or more elements as unwanted elements; determining data components of the three-dimensional image data that are associated with the unwanted elements; providing enhanced two-dimensional digital image data indicative of at least one physical feature, the two-dimensional image data having the one or more undesired elements substantially removed therefrom, based on at least a remaining portion of the three-dimensional image data from which the data components associated with the undesired elements have been substantially removed. Device.

2. determining said data representative of a three-dimensional model of at least one element by utilizing an image segmentation method; 2. The apparatus of claim 1.

3. The image segmentation method comprises a method selected from the group of thresholding, clustering, histogram-based methods, edge detection, region growing, model-based methods, semi-automatic segmentation, and trainable segmentation.

3. The apparatus of claim 2.

4. obtaining digitally reconstructed projection image data using the remaining portion of the three-dimensional digital image data to provide the extended two-dimensional digital image data.

4. Apparatus according to any one of claims 1 to 3.

5. the digitally reconstructed projection image data is obtained by simulating the attenuation of a beam of electromagnetic radiation through a remaining element based on the remaining portion of the three-dimensional digital image data; the attenuation of the beam is local intensity data of the three-dimensional digital image data; 5. The apparatus of claim 4.

6. providing the augmented two-dimensional digital image data utilizing the remaining portion of the three-dimensional digital image data in a three-dimensional projection optionally selected from the group of maximum intensity projection, minimum intensity projection, shaded surface display, volume rendering, and virtual endoscopy; 6. Apparatus according to any one of claims 1 to 5.

7. the extended two-dimensional digital image data includes data indicative of a selected viewing angle; Optionally, the selected viewing angle is 7. Apparatus according to any one of claims 1 to 6.

8. providing said remaining portion of said three-dimensional digital image data through masking said three-dimensional image data based on data representing a three-dimensional model; 8. Apparatus according to any one of claims 1 to 7.

9. The at least one physical feature comprises a tissue selected from the group consisting of bone, ligament, cartilage, and soft tissue.

9. Apparatus according to any one of claims 1 to 8.

10. At least one element includes tissue or noise data; 10. Apparatus according to any one of claims 1 to 9.

11. selecting one or more elements as unwanted elements based on predetermined criteria; the predetermined criteria indicating one or more elements of no interest; The predetermined criteria are optionally obtained via a user interface.

11. Apparatus according to any one of claims 1 to 10.

12. the three-dimensional digital image data being acquired by a scanning method selected from the group consisting of computed tomography, magnetic resonance imaging, and ultrasound scanning; 12. Apparatus according to any one of claims 1 to 11.

13. providing an output to a user of said device; The output includes a visualization of the determined data representing a three-dimensional model of the at least one element and / or a visualization of the augmented two-dimensional digital image data.

13. Apparatus according to any one of claims 1 to 12.

14. 1. A computer-implemented method for providing enhanced two-dimensional digital image data, comprising the steps of: obtaining 802 three-dimensional digital image data indicative of at least one physical characteristic; determining (804) data representative of a three-dimensional model of at least one element segmented from the three-dimensional digital imaging data based on the three-dimensional image data, the element including at least one of the physical features; Selecting 808 one or more elements as unwanted elements; determining (812) data components of the three-dimensional image data that are associated with the unwanted elements; providing enhanced two-dimensional digital image data indicative of at least one physical feature, the two-dimensional image data having the one or more undesired elements substantially removed therefrom, based on at least a remaining portion of the three-dimensional image data from which the data components associated with the undesired elements have been substantially removed. method.

15. A computer program product comprising instructions for causing a computer to carry out the method of claim 15.

Images