Apparatus and method for providing enhanced two-dimensional image data

Abstract

To provide enhanced two-dimensional imaging data through removal of one or more selected features.SOLUTION: An apparatus for providing enhanced two-dimensional digital imaging data is configured to: obtain three-dimensional digital imaging data 3A indicating at least one physical feature; determine, based on the three-dimensional digital imaging data, data representing a three-dimensional model of at least one element 202, 204, 206, 208 segmented from the three-dimensional digital imaging data; select one or more elements as non-required elements 206, 208; and provide enhanced two-dimensional digital imaging data 3B indicative of the at least one physical characteristic, in which the one or more non-required elements are essentially removed from the two-dimensional imaging data.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates generally to image data, and more particularly to an apparatus and method for providing enhanced two-dimensional image data through the removal of selected features. [Background technology]

[0002] Images used in medical applications such as radiology are of great importance to be able to diagnose and treat various types of disorders. To enable medical personnel to interpret the images efficiently, it is important to obtain high-quality images that provide the relevant information in each case.

[0003] Traditional two-dimensional medical images, such as native X-ray images, provide limited information about specific tissues of interest. While projection images derived from information about how the intensity of an X-ray beam attenuates as it passes through tissue provide information about all physical features encountered by the beam, the resulting projection images can sometimes lose, obscure, or obscure the tissue of interest with information about other redundant physical features, making meaningful analysis of the physical features difficult or impossible. The resulting images may not clearly depict the tissue of interest due to information related to the unwanted tissue or other redundant physical features that disrupt the projection image formation. In these cases, diagnosis or treatment may be hindered.

[0004] Three-dimensional digital medical image data can be easily used to generate two-dimensional images. While various techniques exist for converting such three-dimensional image data into two-dimensional projection images, these still suffer from the aforementioned problem that the tissue of interest may be adversely affected, in whole or in part, by data related to other tissues, and rendering the two-dimensional image may not provide sufficient information about the tissue of interest.

[0005] Although three-dimensional image data may be used to obtain visualizations incorporating three-dimensional aspects, in certain situations, two-dimensional images may be required to provide data related to the tissue of interest (e.g., for use by insurance companies). Although three-dimensional imaging techniques may be used to provide two-dimensional images (e.g., computed tomography (CT) slice images), these two-dimensional images may lack information related to the three-dimensional aspects of, for example, a patient's tissue. Summary of the Invention

[0006] It is an object of the present invention to alleviate at least some of the problems associated with the known prior art. According to one aspect of the present invention, there is provided an apparatus for providing enhanced two-dimensional digital image data, the apparatus including at least one processor, which acquires three-dimensional digital image data representing at least one physical feature, and determines, based on the three-dimensional digital image data, data representing a three-dimensional model of at least one element segmented from the three-dimensional digital image data, the element including at least one of the physical features. The apparatus further selects one or more elements as unwanted elements, determines data components of the three-dimensional image data associated with the unwanted elements, and provides enhanced two-dimensional digital image data representing the at least one physical feature from the two-dimensional image data, from which the one or more unwanted elements have essentially been removed, based on at least a remaining portion of the three-dimensional image data from which the data components associated with the unwanted elements have essentially been removed.

[0007] According to another aspect, a method is provided according to independent claim 14 and a computer program product is provided according to independent claim 15.

[0008] Advantageous embodiments of the present invention may provide an apparatus and method for acquiring enhanced two-dimensional image data that essentially removes elements such as unimportant physical features, thereby allowing important physical features to be more clearly visualized. In the prior art, two-dimensional image data that can be acquired in similar situations may include data related to unnecessary / uninteresting physical features, which may obscure data related to physical features of interest.

[0009] The extended two-dimensional image data is data that can be used to generate projection images that can be advantageously used, for example, in plain radiography. The extended two-dimensional image data is, however, not limited to, for example, x-rays. The extended two-dimensional image data can be considered projection image data and can be related to various types of three-dimensional image data that can be acquired via various imaging methods.

[0010] The extended two-dimensional image data may correspond / associate or be used to provide a digitally reconstructed radiograph (DRR) image. The extended two-dimensional image data may be utilized to provide a composite radiographic image.

[0011] Digitally reconstructed radiographic image data utilizes three-dimensional image data from which portions of the original data associated with unwanted elements have essentially been removed, and uses the remaining portion of the digital three-dimensional image data in a three-dimensional projection method to obtain two-dimensional projection image data.

[0012] Embodiments of the present invention may offer new and / or expanded possibilities for medical professionals to gain knowledge based on enhanced two-dimensional image data.

[0013] Even now, it may be desirable to utilize two-dimensional images for certain purposes in place of three-dimensional models or typical images that may be obtained by three-dimensional imaging techniques. In particular, in cases such as orthopedics, two-dimensional image projections are typically viewed on a flat X-ray projection screen, whereby the provision of extended projection data and images that can be generated thereby may be facilitated by the present invention, offering potential improvements in orthopedic uses.

[0014] The enhanced two-dimensional images or data that may be provided through embodiments of the present invention may also be advantageous for providing insurance companies and / or medical systems, etc. with images or image data that correspond to traditional X-ray projection data.

[0015] In one embodiment, the extended two-dimensional digital image data may include data indicating a selected viewing angle, optionally relative to the region of interest. The selected viewing angle may be an optimized viewing angle. Thus, using the present invention, for example, digitally reconstructed radiographic image data may be acquired, resulting in an extended two-dimensional image at the selected viewing angle, with the viewing angle correct and with unwanted elements removed. Such an image may not be obtainable using conventional methods in which X-ray images are taken from the patient. In such cases, the patient may not be positioned correctly, making it impossible to accurately obtain the X-ray image from the desired viewing angle. In other prior art methods in which the viewing angle of the projected image can be theoretically selected, unwanted elements may still obstruct the view of the element / region of interest.

[0016] In embodiments of the present invention, maximum visibility of the region of interest can be ensured. Thus, in advantageous embodiments, optimized digitally reconstructed radiographic images can be obtained, where the viewing angle of the DRR image is optimized.

[0017] In one embodiment, the digitally reconstructed projection image data may be generated by simulating the attenuation of a beam of electromagnetic radiation through the remaining elements (elements not including unwanted elements) based on the remaining portion of the three-dimensional digital image data, where the attenuation of the beam is determined by the local intensity data of the three-dimensional digital image data.

[0018] In an embodiment of the present invention, a three-dimensional model of at least one element is generated or determined based on three-dimensional digital image data, which may be performed by utilizing image segmentation.

[0019] An element herein refers to an entity that can be separated from the three-dimensional digital image data. An element includes at least one physical feature that the three-dimensional digital image data represents. In some embodiments, an element corresponds to or is equivalent to only a physical feature, although in some cases, an element may also include noise data, etc. In some cases, the terms "physical feature" and "element" may be used interchangeably, as they may be essentially equivalent.

[0020] In some embodiments, the image segmentation method may include a method selected from the group of thresholding, clustering, histogram-based methods, edge detection, region growing, model-based methods, semi-automatic segmentation, and trainable segmentation. Of course, various other methods for image segmentation may also be feasible.

[0021] Through the present invention, it is not only from, for example, images or other two-dimensional data that physical features may be removed. Using the data representing the determined three-dimensional model, elements, for example, physical features, can be effectively distinguished from one another, and one or more elements can be selected as unwanted elements and removed from the data representing the three-dimensional model, and / or subsequently provide enhanced two-dimensional image data without substantially affecting the data or images associated with the other physical features.

[0022] The remaining portion of the three-dimensional image data may, in some embodiments, be obtained through masking of the three-dimensional image data based on the data representing the three-dimensional model. Masking may be performed by correlating data components of the three-dimensional image data with data points of the data representing the three-dimensional model. Masking may be used to determine which data components of the three-dimensional image data are associated with unwanted features, and these data components may be removed in the remaining portion of the three-dimensional digital image data.

[0023] If removal of selected unwanted elements were attempted without determining data representing three-dimensional models of elements that can be identified from the three-dimensional image data, the effects of the redundant elements may not be considered or limited to the same extent as can be achieved with the present invention, thereby reducing the quality of the extended image data relative to the quality of the data related to the remaining physical features that should be included. In the present invention, elements (e.g., voxels) can be determined, for example, by masking with data representing three-dimensional models of the elements, which may enable more accurate determination of which elements of the three-dimensional image data are related to or correspond to the unwanted elements. Without data representing three-dimensional models of the elements, it may be difficult to accurately identify which voxels, etc., to remove / ignore from the three-dimensional image data before utilizing the data in a selected (three-dimensional) projection.

[0024] In an embodiment of the present invention, providing the augmented two-dimensional digital image data includes determining components of the three-dimensional digital image data based on data representing a three-dimensional model of the elements, removing at least a portion of the three-dimensional digital image data associated with one or more unnecessary elements, essentially removing / ignoring such data components associated with the unnecessary elements, and utilizing the remaining portion of the digital three-dimensional image data in a three-dimensional projection.

[0025] In some embodiments, the three-dimensional projection method may be selected from the group of maximum intensity projection, minimum intensity projection, shaded surface display, volume rendering, and virtual endoscopy. Thus, the examples presented here are methods that can be used to convert three-dimensional image data into two-dimensional images (or two-dimensional image data), and other methods known to those skilled in the art may also be feasible.

[0026] Physical features, as used herein, may include tissues, such as tissues selected from the group consisting of bone, ligament, cartilage, and soft tissue. The obtained three-dimensional digital image data may also show other physical features. For example, implants and / or other objects may also be identified from the three-dimensional image data.

[0027] The elements (which can be selected as unwanted elements) can be or correspond to physical features that the 3D image data shows in various use cases, or the elements can be elements that can be derived as noise data from data related to the physical features.

[0028] In some embodiments, the device can be used to enhance the image quality of two-dimensional image data, such that elements segmented from the two-dimensional image data may include physical features such as tissue, and one or more elements may include noise data or other data that is disturbance resulting from the imaging / characterization system that may be segmented from the three-dimensional image data. One or more elements that constitute the noise data can then be selected as unwanted elements. The noise data can then be removed from the subsequently generated enhanced two-dimensional image data. In this case, the enhanced two-dimensional image data may be of higher quality than prior art image data in which the noise data is present.

[0029] In one embodiment, the selection of one or more elements as unwanted elements may be based on predetermined criteria, which may indicate one or more elements that are not of interest, and which may be obtained via a user interface included in the device.

[0030] In one use case scenario, a medical professional may be provided with augmented two-dimensional image data, and selected unwanted bones may be essentially removed from the augmented two-dimensional image to more clearly view the important bones, where the important bones may be obscured by data associated with the unwanted bones in images obtainable via prior art methods / apparatus.

[0031] Embodiments of the present invention may provide enhanced two-dimensional image data that may be more useful than that obtainable with other available methods. Information related to physical features of interest may be more efficiently inferred from the enhanced two-dimensional data. For example, diagnoses may be made more quickly, and the present invention may even provide diagnostic or therapeutic possibilities where this is not possible with available methods.

[0032] In some embodiments, the device may be configured to generate an output. The output may include a visualization of the determined data representing a three-dimensional model of the at least one element and / or a visualization of the augmented two-dimensional digital image data. In some embodiments, the output may additionally or alternatively include a computer-readable output including the augmented two-dimensional digital image data, which may then be used in another device, such as to generate an augmented two-dimensional image. In some embodiments, the device may include or be in communication with a printing device, such that the output can be utilized to obtain a printed enhanced two-dimensional image.

[0033] Benefits that can be achieved through embodiments of the present invention may also include reduced radiation exposure to patients undergoing imaging to acquire image data. For diagnostic purposes, two-dimensional projection images must be of adequate quality and show relevant features to be interpretable. Often, it is determined that additional image data needs to be acquired due to, for example, an X-ray image taken from the wrong angle for a particular purpose. The present invention may generate augmented two-dimensional image data and augmented two-dimensional images derived therefrom, such that the data is essentially always shown from the correct image angle, allowing any selected projection to be acquired. Therefore, because additional imaging data need not be acquired via multiple subsequent imaging scenarios, a minimal number of actual imaging scenarios may be used, reducing radiation exposure to the patient.

[0034] The exemplary embodiments presented herein should not be construed as imposing limitations on the applicability of the appended claims. In this specification, the verb "comprise" is used as an open-ended limitation that does not exclude the presence of unspecified features. Features recited in dependent claims can be freely combined with each other unless expressly stated otherwise.

[0035] The novel features which are believed to be characteristic of the invention are set forth with particularity in the appended claims. However, the invention itself, both as to its structure and its method of operation, together with additional objects and advantages thereof, will best be understood from the following description of certain illustrative embodiments when read in connection with the accompanying drawings.

[0036] The presented discussion regarding various apparatus embodiments can be flexibly applied mutatis mutandis to method embodiments, and vice versa, as will be understood by those skilled in the art. [Brief explanation of the drawings]

[0037] [Figure 1] FIG. 1 illustrates an exemplary apparatus according to an embodiment of the present invention. [Figure 2] FIG. 2 illustrates an exemplary visualization of a slice of three-dimensional image data that may be utilized in one use case scenario of an embodiment of the present invention. [Figure 3] FIG. 3 shows at 3A and 3B exemplary visualizations of data representing a three-dimensional model that may be utilized in one use case scenario of an embodiment of the present invention, with 3B showing a visualization of the three-dimensional model with unwanted physical features removed. [Figure 4] FIG. 4 shows at 4A a visualization of a three-dimensional model based on which unwanted physical features have been removed, and at 4B a visualization of the augmented two-dimensional image data that may be provided in one exemplary use case scenario of an embodiment of the present invention. [Figure 5] FIG. 5 shows an exemplary visualization of a slice of three-dimensional image data that may be utilized in another use case scenario of one of the embodiments of the present invention. [Figure 6] FIG. 6 shows exemplary visualizations at 6A and 6B of data representing a three-dimensional model that may be utilized in another use case scenario of an embodiment of the present invention, where 6B shows a visualization of the three-dimensional model with unwanted physical features removed. [Figure 7]FIG. 7 shows visualizations of conventional 2D image data at 7A and 7B, and visualizations of augmented 2D image data that can be obtained in a use case scenario via an embodiment of the device at 7B and 7D. [Figure 8] FIG. 8 shows a flowchart of a method according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0038] The invention will now be explained in more detail with reference to exemplary embodiments according to the accompanying drawings, in which: FIG.

[0039] 1 shows an embodiment of an apparatus 100 according to the present invention. The apparatus 100 includes at least one processor 102 configured to acquire three-dimensional digital image data.

[0040] The processor 102 may be configured to execute instructions embodied in the form of computer software stored in memory, which may refer to, for example, one or more memory chips separate from or integrated with the processor 102. The memory may store other data in addition to program instructions. The software may define one or more applications that perform the activities described herein and may implement one or more computational algorithms that perform operations included in methods.

[0041] The processor 102 may include or communicate with one or more databases 104 that may be used to store or retrieve information. The databases 104 may be stored in memory associated with the processor 102.

[0042] The processor 102 may be a local processor included in a personal computer type computing device, or the processor 102 may be one or more remote processors that may be connected, for example, via wireless means.

[0043] A user interface (UI) 106 may, for example, provide a user of the device 100 with control and information access tools, for example, to control the device and provide input data for the processor 102 .

[0044] The three-dimensional digital image data may be provided to the processor 102 as input data, for example, in the form of DICOM data. For example, the DICOM data may be provided directly to the processor 102 via an imaging device over a wired or wireless connection (such as a WLAN), or the DICOM data may be imported in some other manner, for example, via a carrier medium or transferred as a signal or combination of signals from a sending element to a receiving element over a wired or wireless connection, and provided to the processor 102.

[0045] The three-dimensional image data may be obtained by a scanning technique such as computed tomography, magnetic resonance imaging, or ultrasound scanning.

[0046] The three-dimensional digital image data represents at least one physical feature of a target entity, such as a human patient, that has been subjected to a selected imaging method. The physical feature may be tissue, such as bone, ligament, cartilage, or soft tissue. The three-dimensional digital image data may, in various embodiments, represent multiple physical features of, for example, one or more tissues or tissue portions.

[0047] The processor 102 is configured to determine, based on the three-dimensional image data, data representing a three-dimensional model of at least one element that can be segmented from the three-dimensional image data. The at least one element includes at least one physical feature indicated by the three-dimensional image data. In the use case scenario described below, the element essentially fully corresponds to the physical feature indicated by the three-dimensional image data. Thus, the element may also be referred to as a physical feature herein. The data representing the three-dimensional model of the at least one element may include points in three-dimensional space to represent the at least one element, such as a bone or other physical feature, that can be identified from the three-dimensional image data.

[0048] In some cases, the obtained 3D image data may not be fully utilized, i.e., the 3D image data may show multiple physical features that are not all included in the data representing the model of the entity. This is the case, for example, when the 3D image data covers an area of ​​an entity, such as a patient, that is larger than the area of ​​interest in the case.

[0049] In some embodiments, data representing the three-dimensional model may be presented to a user of device 100 as a visual image or visualization. Device 100 may include or be in communication with one or more display elements 108 that display the creation and visualization.

[0050] Data representing a three-dimensional model of at least one element may be obtained from the three-dimensional digital image data using any type of method known to those skilled in the art for converting the three-dimensional image data into data representing a three-dimensional model of such element as a physical feature identifiable from the three-dimensional image data. Image segmentation methods may be used to divide the digital image into multiple segments or image objects. Pixels or voxels of the image may be classified or characterized to group the pixels or voxels or to determine which pixels or voxels have similar characteristics. Thus, the result obtained from the three-dimensional image data via image segmentation methods is data representing a three-dimensional model of at least one element, where the element is segmented from the image data, and the data representing the three-dimensional model thus provides information about the three-dimensional aspects of the segmented element.

[0051] In some embodiments, one or more additional modeling methods can be utilized to obtain a 3D reconstructed model or three-dimensional representation from the data representing the three-dimensional model of the at least one element. For example, in some use case scenarios, an image-based mesh can be used to further process the data representing the three-dimensional model of the at least one element. This further processed data can be used, for example, in creating a visualization or other representation of the data representing the three-dimensional model of the at least one element.

[0052] Through utilized modeling methods such as image segmentation, one or more elements, e.g., physical features, can be identified / distinguished from the three-dimensional digital image data such that the elements are individuated and presented as entities within the data representing a three-dimensional model of the one or more elements.

[0053] For example, the visualizations disclosed herein can be obtained by using additional modeling methods, such as image-based meshing, although such methods need not be used to practice the present invention. Thus, in embodiments of the present invention, visualization of three-dimensional models or 3D reconstructed models is not necessary, but is presented here for ease of understanding.

[0054] Next, one or more elements may be selected as unwanted elements. The selected unwanted elements may be elements or features that are not of interest in a particular use case scenario, in other words, elements or features that are not interesting / important / relevant. The selected unwanted elements may be, for example, bone or other tissue that may obstruct the display of more interesting elements or physical features. Thus, the selected unwanted elements may also be physical features that would be detrimental or disruptive if they were other physical features inspected in the two-dimensional image in a particular use case scenario.

[0055] The selection of unwanted elements can be based on predetermined criteria. Such predetermined criteria can be provided by a user of device 100, or the predetermined criteria can be provided or inferred by device 100 in an automated manner, for example, based on other information provided by the user, etc.

[0056] In some embodiments, a user of device 100 may select unwanted elements through visual inspection of the visualization of the three-dimensional model provided via display element 108. This may be done via UI 106, for example, by the user clicking on a visualized element, e.g., a physical feature, in the three-dimensional model. Also, instead of explicitly selecting unwanted elements, the user may select elements of interest, and one or more of the remaining elements may be considered unwanted elements.

[0057] In some embodiments, the user may select unwanted elements, for example, by naming such elements or selecting from a list provided by the device 100 via the UI 106 (or alternatively, by naming the elements / physical features of interest, with the processor 102 considering the remaining elements identified in the three-dimensional model as unwanted elements).

[0058] Automatic recognition of elements based on predetermined criteria may be utilized, and automatic selection of elements recognized as unnecessary may be performed, for example, via information available in a database. For example, device 100 may recognize elements as element types that identify elements as particular bones, or tissue types that may identify tissue as bone tissue or adipose tissue, from properties derivable from the three-dimensional image data (such as intensity data) and / or data representing a three-dimensional model of the elements.

[0059] In an alternative embodiment, a user may select (or facilitate the selection of) unwanted elements by providing information about the use case scenario to device 100. For example, a user may provide information indicating a medical condition to be examined. Information available to device 100, e.g., via database 104, may then be utilized to select unwanted elements / features based on available information relating the information provided by the user to one or more particular tissues, etc., that may be deemed unwanted. (Or, of course, important physical features may instead be identified, in which case the remaining identified elements may be deemed unwanted, for example.) As an example, a user of device 100 may indicate, via the provided augmented two-dimensional image data, that a particular type of fracture should be examined. Device 100 may then select unwanted elements based on this information and provide information about, for example, a particular bone in the augmented two-dimensional image data.

[0060] In one embodiment, the selected unwanted feature may be a part of a physical feature, for example, in some cases, a bone or the like may be identified as a feature from the three-dimensional image data, and the selected unwanted feature may be a part of the bone.

[0061] The device 100 may then provide at least one data representing a three-dimensional model of the at least one physical feature, and enhanced two-dimensional image data showing the at least one physical feature from which one or more unnecessary elements have been omitted based on selecting at least one unnecessary element by processing the remaining portion of the three-dimensional image data from which data components related to the unnecessary element have been removed.

[0062] Data components of the three-dimensional image data associated with the unwanted features may be determined by masking the three-dimensional image data based on data representing the three-dimensional model, thereby determining a remaining portion of the three-dimensional image data, which is utilized to generate the augmented two-dimensional image data.

[0063] The enhanced two-dimensional image data can then be used to create a two-dimensional image in which one or more physical features are visualized and selected unwanted elements, e.g., physical features, are removed, thus essentially being deleted or subtracted from the image (data).

[0064] Removal of one or more unwanted elements from the augmented two-dimensional image data may mean that data associated with the unwanted elements may be essentially removed from the two-dimensional image data, at least to the extent that the unwanted elements are not clearly visible in a two-dimensional image that may be generated using the augmented two-dimensional image data. Thus, while the augmented two-dimensional image data may, in some embodiments or use case scenarios, include residual data associated with the unwanted elements, this residual data may be minimal, or at least below a certain threshold.

[0065] The augmented two-dimensional image data may be provided based on data representing a three-dimensional model of at least one element, a selection of at least one unwanted element, and the three-dimensional image data, wherein a portion of the three-dimensional image data relating to one or more unwanted physical features is removed from the three-dimensional image data, and the remaining portion of the digital three-dimensional image data is used in conjunction with a three-dimensional projection method.

[0066] Removal of at least a portion of the three-dimensional image data relating to one or more unwanted elements may be performed by comparing or masking the three-dimensional image data with data representing at least one three-dimensional model, recognizing / identifying one element and data components of the three-dimensional image data that correspond to the unwanted element, and removing / ignoring such data components to obtain the remaining portion of the three-dimensional digital image data.

[0067] A 3D projection can be any projection method that can be used to convert three-dimensional image data into two-dimensional image data. Therefore, it can be a method that obtains points in a two-dimensional plane based on points in three-dimensional space. Therefore, a 3D projection can be a method used to map or project three-dimensional data into two-dimensional data.

[0068] In the context of medical imaging data, three-dimensional imaging data, such as DICOM data, provides information about the intensity of an energy beam as it passes through a physical feature, such as tissue, and the intensity of the beam may be attenuated when encountering the physical feature.

[0069] Thus, the different three-dimensional projection methods used may use different methods of mapping or projecting the intensity data, where the intensity data is equivalent to or included in the three-dimensional image data.

[0070] Typically, two-dimensional projection images can be obtained from three-dimensional image data, such that all voxels contained in or derivable from DICOM data, etc. are utilized. The resulting two-dimensional images may have physical features of interest not visualized with sufficient quality due to interference data associated with unwanted physical features.

[0071] Data relating to the intensity of the energy beam as it passes through tissue etc. can be used as three-dimensional image data such that the data is summed over at least one viewing angle in the two-dimensional image data.

[0072] In an advantageous embodiment of the present invention, the extended two-dimensional image data may be digitally extended projection image data, which converts three-dimensional image data into two-dimensional image data, such that voxels contained in or derivable from the three-dimensional image data that show unwanted physical features may be removed or ignored.

[0073] For example, portions of the three-dimensional image data relating to unwanted elements, such as physical features, may be removed in a sum that may be used to obtain the extended two-dimensional image data.

[0074] In one embodiment, the three-dimensional digital image data may be modified to include only the remaining portions, for example based on masking (data relating to unnecessary elements is removed), and the modified three-dimensional image data may then be used in a projection method or the like to obtain extended two-dimensional digital image data.

[0075] In another embodiment, the three-dimensional image data itself may not be altered, but only the remaining portion may be used, such as through masking, to obtain the extended two-dimensional digital image data.

[0076] In a native X-ray scan, in the context of providing a radiological image, a stream of photons arrives from an emitter and penetrates the patient, where some of the photons are absorbed in the tissue (at a rate that correlates with the type of tissue passed through and is at least somewhat known), and the remaining stream of photons reaches a detector on the other side of the patient. The two-dimensional distribution of these transmitted photons produces a clinical two-dimensional X-ray image (traditional). Photon absorption corresponds to the attenuation of the X-ray / electromagnetic radiation as it passes through tissue.

[0077] Next, in connection with three-dimensional digital image data, for example in a CT image, three-dimensional image data can be obtained in which each 3D pixel (voxel) carries an intensity value, which in turn correlates with the tissue type at hand. In digital radiograph reconstruction, several "simulated" photons / photon fluxes corresponding to X-rays passing through the three-dimensional digital image data at a specific location can be calculated / determined (e.g., by ray-casting). At each linear increment along this simulated ray of electromagnetic radiation, the three-dimensional digital image data can be considered to attenuate the ray at a rate corresponding to the local (pixel / voxel) intensity of the three-dimensional digital image data (tissue radiopacity, in other words, absorption rate). Thus, once the entire ray is calculated, the final result can resemble how a real physical photon flux, such as an X-ray, attenuates along a line corresponding to the calculated (ray-cast) ray. Thus, the three-dimensional digital image data may indicate, via local intensity data, the element type (e.g., bone or other tissue) in the (three-dimensional) projection method used, such as the simulated (X-ray) attenuation, and rays may be simulated / calculated so that the final result (DRR) essentially corresponds to a conventional X-ray image obtained from the tissue / element, etc.

[0078] The local intensity data or relative density data at each point in the three-dimensional digital image data may be calibrated to a Hounsfield value. On the Hounsfield scale, a value of "1000" corresponds to air and a value of "0" corresponds to water. Using the intensity values ​​and the Hounsfield scale, tissue types may be determined from the three-dimensional digital image data (e.g., dense bone is +300, fat is -100 to -50). The correlation between tissue types and Hounsfield values ​​may be known, entered into the device 100, or available to the device 100 via a database, etc.

[0079] Having knowledge in 3D terms of where the unwanted features are (correctly) located in the three-dimensional digital image data may be taken into account, for example, in a ray casting step, when adding the contribution of each linear position to the final result (DRR or augmented two-dimensional digital image data), or when removing or ignoring the corresponding voxels in the calculation of ray attenuation. Alternatively, data components associated with the unwanted features may be essentially removed from the three-dimensional digital image data, and then a projection may be performed (e.g., calculating the simulated attenuation of X-rays or the like passing through the remaining three-dimensional digital image data). Depending on the implementation details, the final result will essentially be the same.

[0080] If linear attenuation is not the most clinically useful projection representation, other projection methods may be chosen (e.g., maximum intensity projection collects the maximum pixel intensity along each raycast, such that nothing below the maximum value contributes to the final result).

[0081] In calculating the DRR data, the selected viewing angle may also be taken into account so that the simulated light rays are directed accordingly.

[0082] In particular, one aspect of an advantageous embodiment of the present invention may use the generated data representing the three-dimensional model of the element to effectively determine which data components, voxels, etc. contained in or derivable from the three-dimensional image data associated with which physical features or elements. Data associated with the selected unwanted elements or physical features may then essentially be removed without removing data associated with the remaining physical features of interest.

[0083] A visualization of the augmented two-dimensional image data and / or other data (such as a three-dimensional model) may be provided to a user of the device via display element 108 .

[0084] 2-4 relate to one use case scenario of an exemplary embodiment of the present invention. FIG. 2 illustrates an exemplary visualization of three-dimensional image data that may be received or utilized by device 100 in one embodiment of the present invention. The illustrated slice of image data in FIG. 2 illustrates multiple physical features. In FIG. 2, the physical features may be several bones 202, 204, 206, 208 of the imaged patient's foot. Only some of the physical features in the illustration are numbered here. Of course, various additional image slices may be provided, more physical features may be identified, and / or physical features may be characterized at various depths and / or viewing angles.

[0085] As can be easily seen from FIG. 2, the visualization, in other words the two-dimensional image that can be obtained from the three-dimensional image data, may not be very informative in itself, especially separately, regarding physical features or any three-dimensional surface.

[0086] Figure 3 shows, at 3A, an exemplary visualization of data representing a three-dimensional model of physical features 202, 204, 206, 208, and 210 that may be identified utilizing at least the data of Figure 2. Of course, the slices visualized in Figure 2 are shown by way of example only and are insufficient to obtain data related to the three-dimensional model visualized in Figure 3 (bone 210 is not shown in Figure 2, and data related to this physical feature is not feasibly displayed; therefore, the feature was obtained using supplemental image data that cannot be shown in full here). Data representing the three-dimensional model may be obtained by utilizing image segmentation.

[0087] In some embodiments, the data representing the three-dimensional model may be visualized and provided to a user of the device. In one embodiment, the user can rotate such a rendered three-dimensional model (e.g., one corresponding to the model of FIG. 3A) and inspect its physical features.

[0088] One or more physical features identifiable from the three-dimensional model may be selected as unwanted physical features. Selection of unwanted physical features, as used herein, may also refer to implicit selection of unwanted physical features by selecting important physical features that should not be included in the group of unwanted physical features.

[0089] In one embodiment, a three-dimensional model may be visualized for inspection by a user of the device 100, and the user may then select unwanted physical features via the UI 106, such as by clicking on one or more visualized physical features.

[0090] 3A, physical features other than bones 202, 204, and 210 may be selected as unwanted. Thus, for example, bones 206 and 208 may be selected as unwanted physical features, along with other, or preferably all, other depicted physical features. This selection may be by, for example, a user of the device indicating selection of bones 206, 208, etc. as unwanted, or indicating an interest in bones 202, 204, and 210, thus implicitly selecting bones 206, 208, etc. as unwanted.

[0091] Next, data representing a three-dimensional model of at least one physical feature may be provided, from which the unwanted physical features have been removed. FIG. 3B shows a visualization of such data, in which only bones 202 and 202 are included. Thus, a three-dimensional model may be provided that includes the physical features identified via the image data and from which the unwanted physical features may be removed. Such a model or visualization (i.e., corresponding to FIG. 3B) does not necessarily need to be created or depicted, as the augmented two-dimensional image data may be obtained through the original three-dimensional model (corresponding to FIG. 3A) and information about the selected unwanted physical features.

[0092] In some embodiments, a visualization of the three-dimensional model may be provided to a user of the device, and the user may indicate the viewing angle based on which the augmented two-dimensional image data is provided / generated. For example, the user may rotate the three-dimensional model (perhaps with selected unwanted features removed) on the display 108 screen to arrive at a desired viewing angle and indicate what the augmented two-dimensional image data should correspond to the image as seen from that angle.

[0093] The selected viewing angle for the augmented two-dimensional digital image data (or augmented two-dimensional digital image) may be selected by a user of the device, or the selected viewing angle may be determined by the device as information characterizing the region of interest (e.g., data regarding a particular joint) based on, for example, such a use case scenario. The selected viewing angle may be an optimized viewing angle that is optimized for the region of interest.

[0094] Visualizations of an exemplary use case scenario are shown in Figures 4A and 4B. 4A shows a visualization of data representing a three-dimensional model of at least one physical feature with unwanted physical features removed (showing only bones 202, 204, and 210 with other bones removed). 4B shows a visualization of augmented two-dimensional image data (augmented projection image) that essentially includes only bones 202, 204, and 210.

[0095] In relation to the use case scenario above, the ankle AP talocrural arthrogram is part of a three-view series of the distal tibia, distal fibula, talus, and proximal metatarsal. This is the most appropriate projection to evaluate the articulation of the two malleoli with the tibial plafond and talar dome, also known as the ankle mortise joint.

[0096] In the scenario of Figure 4, the enhanced 2D image in 4B shows the ankle mortise and tenon joint from a selected / optimized viewing angle. The selected viewing angle allows for an optimal view of the joint, but also provides an unobstructed view as unnecessary elements have been removed from the enhanced 2D image.

[0097] 5-7 relate to another possible / example use case scenario of one embodiment of the present invention. FIG. 5 shows an example slice of three-dimensional image data that may be acquired by apparatus 100 showing multiple bones. FIG. 6 shows one example visualization of data representing a three-dimensional model of a physical feature, i.e., bones that may be identified from the data visualized in FIG. 5 (and other slice data associated with the same target three-dimensional image data).

[0098] Figure 6A shows an example visualization of data representing three-dimensional models of physical features that may be identified from at least the data visualized in Figure 5. Figure 6B shows a visualization of the physical features with selected unwanted physical features removed. As mentioned above, the selection of unwanted physical features may be done explicitly or implicitly, and it may be done by a user of the device, or it may be done automatically or at least semi-automatically based on information received from or available to the user of device 100, for example.

[0099] Providing visualizations such as the exemplary visualizations presented herein in connection with the exemplary use case scenarios is not necessary for operation of device 100, but in some cases may be advantageous to provide the user of the device with information related to identified physical information, allowing the user to more effectively determine which physical features to select as unwanted physical features.

[0100] Figure 7 shows, in Figures 7A and 7B, visualizations of conventional two-dimensional imaging data that may be acquired with prior art methods. Figure 7A shows a conventional native X-ray image that may be acquired from a target in the same or similar cases corresponding to the previous Figures 5-6. Figure 7B shows a two-dimensional projection image that may be obtained from three-dimensional image data in which data representing three-dimensional models of physical features is not used and unwanted physical features have not been removed.

[0101] From Figures 7A and 7B, it can be difficult to distinguish between different physical features, to see / distinguish a particular physical feature from the image, and / or to determine some relevant data related to a particular physical feature when other physical features interfere or even hinder interpretation.

[0102] 7C and 7D show example visualizations of augmented two-dimensional image data (augmented projection images) that may be acquired through an embodiment of the device. Figures 7C and 7D correspond to the use case scenarios shown in Figures 5-6 and may be acquired using three-dimensional image data and determined data and selected unwanted physical features representing a three-dimensional model (see visualizations of data representing a three-dimensional model in Figures 6A and 6B, with unwanted physical features removed in Figure 6B). Figures 7C and 7D show augmented projection images that may be acquired using different methods of acquiring two-dimensional image data from three-dimensional image data in which intensity data is mapped or projected in different ways.

[0103] The unwanted physical features are essentially removed in Figures 7C and 7D. In Figures 7C and 7D, the remaining visualized physical features may be more clearly visible than in Figures 7C and 7D. Using enhanced two-dimensional image data, such as that corresponding to the visualization in Figures 7C and 7D (i.e., those physical features shown in the resulting three-dimensional data are not among the selected unwanted physical features), a medical professional can more effectively arrive at a diagnosis or preferred treatment plan associated with a medical condition associated with the physical features, which is taken into account with the enhanced two-dimensional image data, compared to when the medical professional is only able to consider traditional two-dimensional image data such as that shown in Figures 7A or 7B.

[0104] 8 shows a flowchart of a method according to one embodiment of the present invention. At 802, three-dimensional image data indicative of at least one physical feature is obtained. At 804, data representing a three-dimensional model of at least one element that can be identified or segmented from the three-dimensional image data is determined, the at least one element including the at least one physical feature, and at 808, one or more unwanted elements are selected.

[0105] Data components of the three-dimensional image data that are associated with the unwanted elements may be determined, while data associated with the unwanted elements may be removed from the three-dimensional image data at 810 .

[0106] Finally, at 812, enhanced two-dimensional image data indicative of at least one physical feature is provided, and selected unwanted elements are essentially removed by utilizing the remaining portion of the three-dimensional image data in a three-dimensional projection.

[0107] In some embodiments, determining data components within the three-dimensional image data that are associated with unwanted elements and / or determining the remaining portions of the three-dimensional digital image data is performed through masking of the three-dimensional image data based on data representing a three-dimensional model.

[0108] The present invention has been described above with reference to the aforementioned embodiments, and some advantages of the present invention are demonstrated. It is clear that the present invention is not limited to these embodiments, but includes all possible embodiments within the inventive concept and the spirit and scope of the following claims. The features recited in the dependent claims may be freely combined with one another unless expressly stated otherwise.

Claims

1. 1. An apparatus for providing two-dimensional digital image data indicative of a physical characteristic, comprising: at least one processor; the processor is coupled to at least one display element; The processor: determining data representing a three-dimensional model using model-based segmentation to represent elements of one or more physical features represented by the three-dimensional digital image data using points in three-dimensional space; Accepting a selection of unnecessary or necessary elements for each element that is one or more physical characteristics and determining unnecessary elements; generating two-dimensional digital image data representing at least one of the physical features based on the portion of the three-dimensional digital image data from which the unnecessary elements have been removed; providing the two-dimensional digital image data Device.

2. The processor: The generating the two-dimensional digital image data includes generating the two-dimensional digital image data at a viewing angle determined based on a region of interest and a specified use case scenario.

10. The apparatus of claim 1.

3. providing the two-dimensional digital image data by obtaining digitally reconstructed projection image data using the remaining portion of the three-dimensional digital image data; 3. The apparatus of claim 2.

4. the digitally reconstructed projection image data is obtained by simulating the attenuation of a beam of electromagnetic radiation through a remaining element based on the remaining portion of the three-dimensional digital image data; the beam attenuation is local intensity data of the three-dimensional digital image data; 4. The apparatus of claim 3.

5. providing the two-dimensional digital image data using the remaining portion of the three-dimensional digital image data in a three-dimensional projection selected from the group of maximum intensity projection, minimum intensity projection, shaded surface display, volume rendering, and virtual endoscopy; 5. An apparatus according to any one of claims 1 to 4.

6. providing the remaining portion of the three-dimensional digital image data through masking the three-dimensional digital image data based on data representing a three-dimensional model; 6. An apparatus according to any one of claims 1 to 5.

7. the at least one physical feature comprises a tissue selected from the group consisting of bone, ligament, cartilage, and soft tissue; 7. Apparatus according to any one of claims 1 to 6.

8. The one or more elements include tissue or noise data.

8. Apparatus according to any one of claims 1 to 7.

9. selecting one or more unwanted elements based on predetermined criteria; the predetermined criteria indicate elements of no interest; The predetermined criteria are obtained via a user interface.

9. Apparatus according to any one of claims 1 to 8.

10. the three-dimensional digital image data is acquired by a scanning method selected from the group consisting of computed tomography, magnetic resonance imaging, and ultrasound scanning; 10. Apparatus according to any one of claims 1 to 9.

11. providing an output to a user of the device; The output includes a visualization of the determined data representing a three-dimensional model and / or a visualization of the two-dimensional digital image data.

11. Apparatus according to any one of claims 1 to 10.

12. 1. A computer-implemented method for providing two-dimensional digital image data indicative of a physical feature, comprising: determining data representing a three-dimensional model using model-based segmentation to represent elements of one or more physical features represented by the three-dimensional digital image data using points in three-dimensional space; Accepting a selection of unnecessary or necessary elements for each element that is one or more physical characteristics and determining unnecessary elements; generating two-dimensional digital image data representing at least one of the physical features based on the portion of the three-dimensional digital image data from which the unnecessary elements have been removed; providing the two-dimensional digital image data method.

13. A computer program comprising instructions for causing a computer to carry out the method of claim 12.

Images