Device for generating enhanced images of objects
By using a device to generate enhanced images and determining the degree of saliency to enhance the basic image, the problem of unintuitive information in spectral imaging is solved, and high-quality and high-information-content image display is achieved.
Patent Information
- Application Number
- CN202180027226.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2020-04-09
- Filing Date
- 2021-04-06
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2041-04-06
AI Technical Summary
In spectral imaging, users need to view multiple images to access the complete information content, which leads to problems such as unintuitive information and poor image quality.
The apparatus for generating enhanced images utilizes a saliency determination unit to enhance the voxels of a base image for each voxel of a contrast image and a predetermined template image, thereby generating an enhanced image containing improved information content and quality.
It provides a more intuitive display of image information, combining the high quality of the base image with the additional information from the contrast image, thus improving the information content and quality of the image.
Smart Images

Figure CN115428025B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to apparatus, methods and computer programs for generating enhanced images of objects. Background Technology
[0002] In recent years, spectral imaging (especially spectral X-ray computed tomography) has become more widely available in medical imaging facilities. When using spectral imaging, image data of an object is acquired using at least two different radiation spectra (e.g., using X-ray radiation including a first wavelength and X-ray radiation including a second wavelength). These image data acquired under different radiation spectra form spectral image data, from which multiple different images of the object can be derived. For example, the information provided in the spectral image data allows for the identification of images that only reflect the attenuation of the object due to the Compton effect or photoelectric effect. These so-called Compton and photoelectric images can then be further combined in various ways to derive images with multiple different informational contents from the spectral image data, such as deriving virtual single-energy images of different energy levels, material decomposition images, etc. All these derived images provide different informational contents to the user (e.g., a physician) and show different levels of noise and sensitivity for certain materials. Therefore, to access the complete informational content of the acquired spectral image data, the user often has to view multiple images derived from the spectral image data. For example, a user must first view a derived image showing soft tissue in the region of interest with low noise and high image quality, and then view a second derived image that provides lower image quality but shows some material of interest with higher contrast. Therefore, it would be helpful if the information provided by the spectral image data could be presented to the user more directly. Summary of the Invention
[0003] The object of the present invention is to provide apparatus, methods and computer programs that allow the provision of images derived from spectral image data, including improved information content and quality.
[0004] In a first aspect of the invention, an apparatus for generating an enhanced image of an object is provided, wherein the apparatus comprises: a) a base image providing unit configured to provide a base image of the object, wherein the base image is generated based on a combination of spectral image data of the object acquired using a spectral X-ray imaging apparatus; b) a contrast image providing unit configured to provide a contrast image of the object, wherein the contrast image is generated based on a combination of spectral image data of the object that is different from the combination used to generate the base image; c) a saliency degree determining unit configured to determine a saliency degree for each voxel of the contrast image, wherein the saliency degree indicates the difference between the image value of a voxel of the contrast image and the image value of a corresponding voxel of a predetermined template image; and d) an enhanced image generating unit configured to generate an enhanced base image of the object by enhancing the voxels of the base image based on the saliency degree determined for the corresponding voxels of the contrast image. The saliency determination unit is configured to determine an approximation function that allows the image values of voxels in the contrast image to be approximated based on the image values of corresponding voxels in the predetermined template image. The saliency determination unit is further configured to generate an approximate contrast image based on the approximation function and the predetermined template image, and to determine the saliency level based on the approximation function. The approximation function can be determined such that it approximates the image values of voxels in the contrast image as closely as possible to the image values of corresponding voxels in the predetermined template image. Preferably, the approximation function is a linear regression function between the image values of voxels in the contrast image and the image values of voxels in the predetermined template image. However, a non-linear approximation between the image values of voxels in the contrast image and the image values of voxels in the predetermined template image can also be provided. Therefore, based on the approximation function, the predetermined template image can be used to generate an approximate contrast image, whereby the saliency level can then be determined based on the approximate contrast image. Preferably, the saliency determination unit is configured to determine the saliency level of the voxels such that it indicates the deviation between the image values of voxels in the contrast image and the image values of corresponding voxels in the approximate contrast image. This allows for the identification of voxels in a contrast image that cannot be approximated using an approximation function and a predetermined template image. These voxels can be considered to contain information not yet included in the predetermined template image, and therefore information that may be of interest in enhancing the base image. Preferably, the enhanced image generation unit is configured to generate the enhanced base image by enhancing the following voxels of the base image: for the voxels, the saliency level indicates that the image value of the voxel shows an upward deviation from the approximate image value of the voxel.
[0005] Since a saliency level is determined for each voxel of the contrast image, and the base image is enhanced by enhancing voxels of the base image based on the saliency level determined for the corresponding voxels of the contrast image, the image providing the first information to the user can be enhanced by the additional information provided by the second image. Furthermore, the base image can be selected to provide good image quality, wherein even if the contrast image itself has only low image quality, the additional information provided by the contrast image can be extracted and provided to the base image. Therefore, improved image quality and information content can be provided to the enhanced base image.
[0006] The object for which enhanced images are generated can be any object capable of being imaged using a spectral X-ray imaging device. Preferably, the object refers to an anatomical structure within a region of interest of a living organism (particularly a patient), where the patient can be a human or an animal. However, the object can also be an inanimate object, such as a suitcase or bag in a security application.
[0007] The base image providing unit, which can also be considered a base image provider, is configured to provide a base image of the object, particularly to provide the base image to the enhancement image generation unit. The base image providing unit can be a storage unit where the base image of the object is stored and can be retrieved from the storage unit for providing the base image. The base image providing unit can also be a receiving unit for receiving the base image of the object and for providing the received image. Furthermore, the base image providing unit can also be a base image generation unit configured to generate a base image based on a combination of spectral image data of the object acquired using a spectral X-ray imaging device. Preferably, the base image providing unit is a base image selection unit configured to provide a user with multiple images generated from different combinations of spectral image data, wherein the user can then select one of the images as the base image, and the base image providing unit is then configured to provide the selected base image.
[0008] A base image refers to an image generated based on a combination of spectral image data of an object acquired using a spectral X-ray imaging device. Spectral X-ray imaging devices can be based, for example, dual-source technology, fast kV switching technology, photodetector technology, etc. The spectral image data acquired using the spectral X-ray imaging device provides image data of the object acquired using at least two different radiation spectra, and thus indicates the attenuation of the object relative to at least two different radiation spectra. If the object refers to an anatomical structure in a patient's region of interest, it is preferable that the spectral image data is acquired after a contrast agent (such as an iodine-based contrast agent) has been administered to the patient. However, in other embodiments, the spectral image data can also be acquired without applying a contrast agent to the object, particularly when the object is inanimate. Multiple different images can be derived from different combinations of the spectral image data, as explained in more detail, for example, in the article "Detector-based spectral CT with a novel dual-layer technology: principles and applications" (Insights into Imaging, Vol. 8, pp. 589–598 (2017)) by N. Rassouli et al.
[0009] The base image can refer to one of the images generated based on a combination of spectral image data, and can refer to a predetermined combination of spectral image data or a combination of spectral image data selected by the user, for example, via user input. Preferably, the base image includes high image quality, for example, providing virtually no streak artifacts. Furthermore, it is preferred that the base image provides information about the soft tissue material in the region of interest of the object with sufficient sensitivity. Preferably, the base image refers to a virtual single-energy image derived from the spectral image data, particularly a virtual single-energy image at 200 keV. However, in some applications, it may be preferred that the base image refers to a conventional CT image, as this type of image is associated with most of the radiologist's viewing expertise.
[0010] A contrast image providing unit, which can also be considered a contrast image provider, is configured to provide a contrast image of an object, particularly to provide the contrast image to a salience determination unit. The contrast image providing unit can be a storage unit where a contrast image of the object is already stored, and a contrast image can be retrieved from the storage unit for providing the contrast image. The contrast image providing unit can also be a receiving unit for receiving, for example, an input unit where a user can select a contrast image, and for providing the received contrast image. Furthermore, the contrast image providing unit can also be a contrast image generating unit configured to generate a contrast image based on a combination of spectral image data of the object.
[0011] A contrast image refers to an image generated based on a combination of spectral image data that differs from the combination used to generate the base image. Therefore, both the base image and the contrast image refer to images that include different informational content about the object. Preferably, the contrast image includes information about the distribution of a specific material (e.g., a contrast agent like iodine) in the region of interest of the object. Preferably, the contrast image refers to a virtual single-energy image (particularly a virtual single-energy image at 40 keV) or a photoelectric image derived from the spectral image data. Furthermore, if the contrast agent applied to the patient is iodine-based, the contrast image can also refer to an iodine density map that quantifies the iodine density in each image region (particularly voxels). This contrast image has the advantage that the iodine density map very accurately and clearly shows the distribution of the contrast agent in the region of interest, where structures affected by calcium, such as bone, are not shown.
[0012] Typically, a base image and a contrast image can refer to a two-dimensional, three-dimensional, or even four-dimensional image that considers the evolution of an object over time. In this context, the term "voxel," referring to image elements, will also be used for 2D or 4D images, and therefore, if addressing a 2D image, the term "voxel" should be interpreted as "pixel." Furthermore, a base image and a contrast image can also refer to a spectral image, i.e., an image comprising more than one image value for each voxel, where the image value of each voxel reflects the attenuation of the corresponding portion of the object at different radiation spectra. Therefore, a spectral image can also be considered as containing two distinct images generated from different combinations of spectral image data.
[0013] A saliency determination unit, which can also be viewed as a saliency determination unit, is configured to determine the saliency of each voxel in the contrast image. The saliency indicates the difference between the image value of a voxel in the contrast image and the image value of the corresponding voxel in a predetermined template image. The difference between the image value of a voxel in the contrast image and the image value of the corresponding voxel in the predetermined template image also indicates an information difference between the contrast image and the predetermined template image, such that the saliency also indicates this information difference. Preferably, the saliency determination unit is configured to determine the saliency as greater than zero only for image regions (specifically voxels) in the contrast image that include image values greater than those in the template image. The saliency can generally be considered as defining how much the image value of the contrast image stands out relative to a predetermined norm represented by the predetermined template image.
[0014] The predetermined template image can be determined in advance, for example, by a user selecting it from a plurality of provided possible template images. Furthermore, the predetermined template image can also be automatically determined in advance based on a selected base image and a selected contrast image and / or based on the target of spectral image acquisition. For example, the predetermined template image may refer to an image generated based on a combination of spectral image data similar to the combination used for the contrast image, wherein spectral image data for generating the template image has been acquired from a healthy person. Then, the spectral image data for generating the predetermined template image can be selected, for example, based on information about the patient (e.g., age, size, weight, etc.) and / or based on the reason for examining the patient. In this case, the saliency indicates the difference in the region of interest in the patient relative to a healthy person. Where the template image does not involve an image generated based on the patient's spectral image data, the selected template image can be registered to the contrast image based on generally known registration methods. However, the template image may also be provided in a manner where registration is not necessary, for example, where the template image involves a constant template image.
[0015] An enhanced image generation unit, which can also be considered an enhanced image generator, is configured to generate an enhanced base image of an object by enhancing voxels of a base image based on a saliency level determined for corresponding voxels of a contrast image. The enhanced image generation unit can be configured to enhance voxels of the base image, for example, by increasing voxel intensity, by providing a predetermined color to voxels, by changing the transparency of a color layer provided on the base image, etc. The enhancement of voxels of the base image is based on a saliency level determined for corresponding voxels of a contrast image. For example, if the saliency level of a voxel is higher than a predetermined threshold of saliency level, the voxels of the base image can be enhanced by adding a predetermined intensity value or providing a predetermined color to the voxel. Alternatively or additionally, a functional relationship describing the relationship between the saliency level and the intensity or color value that should be added to the corresponding voxel can be provided, wherein the functional relationship can be, for example, a linear function, a quadratic function, or any other kind of function based on the saliency level.
[0016] Therefore, the enhanced base image is provided with additional information from the contrast image, specifically, additional information about the saliency of voxels in the contrast image relative to a predetermined template image. For example, the base image might refer to a spectral image with good image quality (i.e., virtually no image artifacts) and good soft tissue contrast, but without much information about the distribution of the contrast agent. Furthermore, in this example, the contrast image might refer to an image providing high contrast between soft tissue and areas where the contrast agent can be found, and the template image might be selected such that the areas containing the contrast agent are provided with high saliency. In this case, high saliency is determined for the areas containing the contrast agent in the contrast image, such that the enhanced base image will not only contain the good quality and good soft tissue contrast of the base image, but also the voxels enhanced in the areas containing the contrast agent. Thus, information about the soft tissue in the region of interest can be provided to the user (e.g., a physician), and also information about the apparent distribution of the contrast agent within the region of interest.
[0017] In this embodiment, the predetermined template image refers to the base image. Choosing the base image as the template image has the advantage that voxels in a contrast image containing additional informational content (i.e., voxels that deviate from the base image) can be identified. By enhancing these voxels in the base image, their informational content is also provided to the base image. However, other choices of the template image may also be advantageous; for example, by selecting the template image as an average image of objects averaged across multiple patients, only voxels that deviate from the average can be identified and enhanced. Furthermore, images of regions of interest from healthy patients can be used as template images, allowing only regions in the base image showing high deviations from the images of healthy patients to be identified and enhanced.
[0018] In this embodiment, the predetermined template image refers to a constant image in which the image values of each voxel are the same, wherein the saliency determination unit is configured to determine the saliency based on the difference between the image values of the voxels in the contrast image and the constant image values of the predetermined template image. Since the constant image contains only one image value, namely the constant image value, in this embodiment, the template image can also be considered a constant. The constant image value can be selected, for example, as the average of all image values of the contrast image, an image value lower than the expected image value for a particular material (e.g., a contrast agent), a constant image value that has previously shown to be advantageous, etc. The saliency determination unit can be configured to determine the saliency in this case, for example, by subtracting the constant image value from the image values of each voxel in the contrast image, wherein the resulting difference image indicates the saliency of each voxel. This embodiment allows for the very quick and simple determination of potentially interesting information in a contrast image.
[0019] In a preferred embodiment, the enhanced image generation unit is configured to generate the enhanced base image by enhancing the following voxels of the base image: for each voxel, the saliency level indicates that the image value of the voxel in the contrast image is greater than a constant image value of the predetermined template image. For example, in this embodiment, the saliency level determination unit may be configured to set all image values of the difference image that indicate negative values to zero, wherein all remaining voxels in the difference image that include positive image values refer to voxels with image values greater than the constant image value of the predetermined template image. The enhanced image generation unit may then be configured, for example, to enhance the base image by simply adding the resulting difference image to the base image. In this embodiment, the saliency level refers to the difference between the image values of voxels in the contrast image and the constant image value, and if the difference is negative, it refers to zero, wherein the saliency level is directly used to enhance the base image by adding the saliency level to the image values of the corresponding voxels in the base image.
[0020] Preferably, the contrast image refers to a virtual single-energy image or a photoelectric image. Virtual single-energy images and photoelectric images determined from spectral image data are advantageous as contrast images because they provide better contrast between areas containing high-contrast materials (such as contrast agents) and soft tissue. Even more preferably, the contrast image refers to a virtual single-energy image simulating 40 keV of radiation energy. However, other images generated by combining spectral image data containing information about materials of particular interest in the region of interest can also be used as contrast images.
[0021] In a preferred embodiment, the base image and the contrast image are virtual single-energy images, and the contrast image is generated with reference to an energy lower than the energy used to generate the base image. Typically, virtual single-energy images generated for higher energies exhibit good contrast of soft tissue material and also provide high image quality with virtually no image artifacts (particularly fewer image artifacts than virtual single-energy images generated for lower energies). However, it is generally known that virtual single-energy images generated for lower energies provide higher contrast between the region containing the contrast agent and the soft tissue region, but are more sensitive to image artifacts. Therefore, using a virtual single-energy image as a contrast image involving an energy lower than the energy used to generate the base image allows for the utilization of information about the distribution of high-contrast materials (such as contrast agents) to enhance the base image, which includes good image quality and good soft tissue contrast.
[0022] In an embodiment, the contrast image and the predetermined template image refer to a spectral image, wherein the spectral image includes at least two intensity values for each voxel, the at least two intensity values indicating the attenuation of a portion of the object corresponding to the voxel at different radiation energy spectra, and the saliency determination unit is configured to determine the saliency based on a spectral metric indicating the difference between image values of the contrast image and the predetermined template image in a spectral space defined by different radiation energy spectra. A spectral image can be defined as an image in which each voxel includes at least two intensity values indicating the attenuation of a portion of the object corresponding to the voxel at different radiation energy spectra. For example, a spectral image may include two virtual single-energy images such that each voxel of the spectral image includes two intensity values indicating the attenuation of the object corresponding to that voxel at different radiation energies. In this case, the saliency determination unit can determine the saliency based on a spectral metric in the spectral space. The spectral space can be defined as being spanned by different energy spectra, providing intensity values in the spectral images involved at different energy spectra, such that at least two intensity values of a voxel in the spectral image define the position of the voxel in the spectral space. For example, if the spectral image includes a virtual single-energy image generated for 40 keV and a virtual single-energy image generated for 200 keV, the intensity of a voxel in each image defines its position in the spectral space. Spectral metrics (such as Euclidean vector distance, spectral vector angle, or Bach distance) can be used to determine the difference between the image values in the spectral contrast image and the predetermined spectral template image in the spectral space. Preferably, the spectral contrast image and the spectral template image include values for each voxel involving the same energy spectrum; that is, the image forming the spectral contrast image and the image forming the spectral template image are based on the same radiation energy spectrum, respectively.
[0023] In an embodiment, the saliency determination unit is configured to generate the saliency using a trained deep learning algorithm, wherein the trained deep learning algorithm is trained based on training contrast images, in which each voxel is associated with a saliency. For example, the training images can be generated by a user using multiple possible contrast images from a previous context and providing annotations to these contrast images regarding the saliency of each voxel in those contrast images relative to a predetermined template image (e.g., by labeling different voxels in the contrast images with different colors representing different saliency levels). These annotated images are then provided as training images to the deep learning algorithm in training. Alternatively, training images can be provided using any of the above embodiments for determining saliency, wherein the determined saliency, along with the contrast images and optionally the template images, is then provided as input to the deep learning algorithm for training. The deep learning algorithm can then learn which voxels in the contrast images should be provided with which saliency level. In this embodiment, the predetermined template image can be considered to be reflected in the variables of the trained deep learning algorithm. Preferably, the trained deep learning algorithm refers to a convolutional neural network.
[0024] In an embodiment, the enhanced image generation unit is configured to enhance the voxels by adding enhancement values to the voxels of the base image and / or by providing color to the voxels, wherein the enhancement values are based on the saliency level, and the color is based on the saliency level. For example, the enhancement value may be equal to the saliency level, but it may also be connected to the saliency level via a predetermined function (such as a linear or nonlinear function). Color may also be connected to the saliency level via a color function that determines which color or color intensity refers to which saliency level. The function connecting the enhancement value or color can be a continuous function or a discrete function.
[0025] In another aspect of the invention, a method for generating an enhanced image of an object is proposed, the method comprising: a) providing a base image of the object, wherein the base image is generated based on a combination of spectral image data of the object acquired using a spectral X-ray imaging device; b) providing a contrast image of the object, wherein the contrast image is generated based on a combination of spectral image data of the object that is different from the combination used to generate the base image; c) determining a saliency level for each voxel of the contrast image, wherein the saliency level indicates the difference between the image value of a voxel in the contrast image and the image value of a corresponding voxel in a predetermined template image; and d) generating an enhanced base image of the object by enhancing the voxels of the base image based on the saliency level determined for the corresponding voxels of the contrast image. Determining the saliency level for each voxel of the contrast image comprises: determining an approximation function that allows the image values of the voxels in the contrast image to be approximated based on the image values of the corresponding voxels in the predetermined template image; generating an approximate contrast image based on the approximation function and the predetermined template image; and determining the saliency level based on the approximation function.
[0026] In another aspect of the invention, a computer program for enhancing an image of an object is provided, wherein the computer program includes program code modules that, when the computer program is run by the apparatus described above, cause the apparatus to perform the steps of the method described above.
[0027] It should be understood that the apparatus according to claim 1, the method according to claim 13, and the computer program according to claim 14 have similar and / or identical preferred embodiments, particularly as defined in the dependent claims.
[0028] It should be understood that the preferred embodiments of the present invention can also be any combination of the independent claims or the above embodiments and the corresponding dependent claims.
[0029] These and other aspects of the invention will be apparent and explained with reference to the embodiments described below. Attached Figure Description
[0030] In the following figures:
[0031] Figure 1 An embodiment of a spectral imaging system including means for generating enhanced images of objects is illustrated schematically and exemplary.
[0032] Figure 2 A flowchart illustrating an exemplary embodiment of a method for generating an enhanced image of an object is shown.
[0033] Figure 3 An exemplary embodiment for determining the degree of salience is illustrated, and
[0034] Figure 4 and 5 The result of image enhancement is illustrated exemplarily. Detailed Implementation
[0035] Figure 1 An embodiment of a spectral imaging system including means for generating enhanced images of objects is illustrated schematically and exemplary. In this embodiment, the spectral imaging system 100 includes an imaging unit 120, such as a dual-energy X-ray CT imaging unit, for acquiring spectral image data of a region of interest of a patient 121 lying on a patient table 122. In a preferred embodiment, the patient 121 is injected with a contrast agent, such as an iodine contrast agent, before acquiring the spectral image data. The spectral image data of the patient 121 acquired by the spectral imaging unit 120 includes spectral image data of objects relating to anatomical structures in the region of interest.
[0036] To provide enhanced images of an object (in this example, an anatomical structure), the spectral imaging system 100 includes means 110 for generating enhanced images of the object. Means 110 includes a base image providing unit 111, a contrast image providing unit 112, a salience determination unit 113, and an enhanced image generation unit 114. Furthermore, means 110 may include an input unit 115 such as a mouse or keyboard and / or, for example, a display unit 116 for displaying the enhanced image.
[0037] In this example, the base image providing unit 111 is configured to receive spectral image data acquired by the spectral imaging unit 120. The base image providing unit 111 is then configured to generate a base image based on a combination of the received spectral image data. Specifically, the base image can be generated based on a pre-selected combination of spectral image data. However, the base image providing unit 111 can also provide a user with multiple possible base images, for example via a display 116, and can then receive user input, for example via an input unit 115, indicating the base image to be selected. The base image providing unit 111 is then configured to specifically provide the corresponding selected image as the base image to the enhancement image generation unit 114.
[0038] In this example, the contrast image providing unit 112 is also configured to receive spectral image data acquired by the spectral imaging unit 120 and generate a contrast image based on the received spectral image data. Furthermore, the contrast image can be generated based on a pre-selected combination of spectral image data, or it can be selected by the user, as described with respect to the base image providing unit 111. Typically, the contrast image is generated based on a combination of spectral image data that differs from the base image. The contrast image providing unit 112 is then specifically configured to provide the contrast image to the salience determination unit 113.
[0039] The saliency determination unit 113 is configured to determine the saliency of each voxel in the contrast image. Specifically, the saliency indicates the difference between the image value of a voxel in the contrast image and the image value of the corresponding voxel in a predetermined template image. The predetermined template image can be predetermined, for example, by providing a user with a selection of possible template images using the display 116, whereby the user can then select one of the presented template images as the predetermined template image using the input unit 115. However, the predetermined template image can also be a default template image for all objects, or it can be predetermined for specific situations (e.g., specific selections for the contrast image and the base image). Furthermore, the predetermined template image can be predetermined based on information provided about the patient and the imaging procedure. For example, the predetermined template image can be predetermined based on the contrast agent applied to the patient 121, the region of interest to be imaged, the age or size of the patient 121, the reason for examining the patient 121, etc. Therefore, the saliency determined by the saliency determination unit 113 can also be considered as indicating the information provided by the contrast image, which deviates from the information provided in the predetermined template image.
[0040] Then, the enhanced image generation unit 114 is configured to generate an enhanced base image of the patient 121's region of interest by enhancing the voxels of the base image based on a saliency level determined for the corresponding voxels of the contrast image. For example, the enhanced image generation unit 114 may add the saliency level determined for the voxels of the contrast image to the image values of the corresponding voxels of the base image. Alternatively or additionally, the enhanced image generation unit 114 may be configured to provide color to the voxels of the base image based on the saliency level for enhancing the voxels of the base image.
[0041] Figure 2An embodiment of a method for generating an enhanced image of an object is illustrated schematically and exemplary. Method 200 includes a first step 210 of providing a base image of the object (e.g., the anatomy of patient 121), wherein the base image is generated based on a combination of spectral image data of the object acquired using, for example, a spectral X-ray imaging device 120. Furthermore, method 200 includes a step 220 of providing a contrast image of the object, wherein the contrast image is generated based on a combination of spectral image data of the object that differs from the combination used to generate the base image. Typically, steps 210 of providing the base image and 220 of providing the contrast image can be performed in any order or even simultaneously. Based on the contrast image provided in step 230, a salience level is determined for each voxel of the contrast image. In a final step 240, an enhanced base image is then generated by enhancing the voxels of the base image based on the salience level determined for the corresponding voxels of the contrast image in step 230.
[0042] In the following, some preferred embodiments of the invention will be described in more detail with respect to the above configuration of device 110. In a preferred embodiment, the predetermined template image is a constant image, i.e., an image that includes only a constant image value for all voxels. In this case, the saliency determination unit 113 is configured to subtract the constant image (i.e., the constant image value) from the image value of each voxel in the contrast image to obtain a difference image. Furthermore, in this embodiment, it is preferred that the saliency determination unit 113 is configured to set all image values of the difference image to zero, where all image values refer to negative image values, i.e., indicating that the contrast image contains voxels with image values lower than the constant image value. Optionally, the saliency determination unit 113 may be configured to smooth the obtained difference image, for example, by using a Gaussian smoothing algorithm or a binomial smoothing algorithm, to reduce image noise and provide a difference image with a softer appearance (i.e., with a strong transition in reduction between image values of adjacent voxels). The image values of the obtained difference image are then determined as the saliency of each voxel in the contrast image. Thus, the obtained difference image can also be regarded as a saliency image, wherein each image value indicates the saliency of the contrast image for the corresponding voxel. In this embodiment, it is also preferred that the enhanced image generation unit 114 is configured to generate an enhanced base image by adding a saliency level image to a base image, i.e., enhancing each voxel of the base image using the corresponding saliency level in the saliency level image. Optionally, the enhanced image generation unit 114 may be configured to multiply each image value of the saliency level image by a weighting factor α, which may be predetermined by the user or interactively changed. The enhanced image can then be determined by the enhanced image generation unit 114 using the following equation:
[0043] I 提升 (x)=I基础 (x)+α·G*(max(0,I) 低E (x)-T)),
[0044] Among them, I 提升 (x) refers to the image value of the enhanced image for each voxel x, I 基础 (x) refers to the image value of the base image for each voxel x, G refers to the Gaussian convolution function, and I 低E (x) refers to the image value of the contrast image for each voxel x, and T refers to the constant image value of the predetermined template image.
[0045] In another preferred embodiment, a predetermined template image is selected as a reference base image. Furthermore, in this embodiment, the saliency level can be determined by subtracting the predetermined template image, which is the base image in this case, from the contrast image according to the above embodiment. However, in this embodiment, it is preferred that the saliency level determination unit 113 is configured to determine an approximation function that allows the image values of the voxels in the contrast image to be approximated based on the image values of the corresponding voxels of the base image (here, the predetermined template image). To determine the approximation function, a regression function, such as a linear regression function or a nonlinear regression function, is preferably used.
[0046] exist Figure 3 The diagram illustrates an example of how approximate functions can be determined. Figure 3 An example is shown in which the base image, and therefore the predetermined template image, is selected as a 200keV virtual single-energy image and the contrast image is selected as a 40keV virtual single-energy image. Figure 3The x-axis 310 of the spectral space 300 shown refers to the intensity (i.e., image value) corresponding to the 200 keV virtual single-energy spectrum, and the y-axis 320 refers to the intensity corresponding to the 40 keV virtual single-energy spectrum, where, in this example, both axes are provided in Henle units. Each voxel in the base image and the contrast image can then be considered as comprising two image values: one image value is the image value the voxel has in the base image, and the other image value refers to the image value the voxel includes in the contrast image. Based on these two image values, each voxel can be placed in the spectral space, i.e., the position of the voxel in the spectral space 300 is defined by its intensity value in the 40 keV virtual single-energy image and its intensity value in the 200 keV single-energy image. If all voxels are placed accordingly in the spectral space, the scattering region 340 is formed by these voxel references in the spectral space 300, representing the region covered by the voxel's image value. An exemplary voxel placement is point 331, which includes higher image values (particularly intensity values) relative to a 40 keV virtual single-energy spectrum than relative to a 200 keV virtual single-energy spectrum. Point 350 corresponds to the image value of air in the corresponding energy spectrum. Linear regression can then be performed relative to the scattering region 340 to determine the linear regression function 330 as an approximation function. Preferably, linear regression is performed only on the portion of the scattering region 340 containing image values below zero (as indicated by the portion of the linear regression function 330 provided as a dashed line), wherein linear regression can then be used to extrapolate also to the region above zero (as indicated by the portion of the linear regression function 330 provided as a solid line). It has been found that limiting the linear regression to the region below zero results in more reliable regression and also results in better enhancement results. The approximate image values of the linear regression or extrapolation and thus the contrast image can be represented by the following equation:
[0047] I 低E_Fit (x)=α+β·I 基础 (x),
[0048] Among them, I 低E_Fit (x) refers to the approximate image value of the contrast image for each voxel x, I 基础 (x) refers to the image value of the base image for each voxel x, and α and β are regression constants determined by linear regression. After the approximation function 330 has been determined based on regression, an approximate contrast image can be generated based on the approximation function and a predetermined template image (here, the base image). The image values of such an approximate contrast image then all lie on the approximation function 330, such as... Figure 3As shown. The image values of the voxels in the contrast image can then be compared with the image values of the approximate contrast image; specifically, the difference between the image values of the contrast image and the approximate image values of the approximate contrast image can be determined. Preferably, the difference is determined by subtracting the approximate image value from the image value of the contrast image for the corresponding voxel, wherein if the subtraction result is positive, the result refers to the significance of that voxel, and if the result is negative, the significance of that voxel is set to zero. This is in Figure 3 The following diagram illustrates the representation of an exemplary voxel. Point 333 refers to an approximate image value of the voxel in the contrast image, while point 331 refers to the original image value of the voxel. The difference 332 can then be set as the saliency level of the voxel. The enhanced image generation unit 114 is then configured to enhance the base image by increasing the image values of the voxels of the base image according to the determined saliency level, for example, as described above with respect to the first preferred embodiment.
[0049] Figure 4 and 5 The results of the enhanced image generated based on the above principles are shown. Figure 4 The left side shows two conventional CT slice images compared to the enhanced image on the right. In the upper right corner, a 200 keV virtual single-energy image is shown, which includes a lower noise level than the conventional CT image but also lower intensity for areas containing iodine contrast material. In the lower right corner, an enhanced image generated according to the above embodiment is shown, where it can be seen, in particular, that voxels involving areas containing iodine contrast material have been enhanced compared to the unenhanced image in the upper right corner (i.e., the base image), while in all other areas, the preferred soft tissue representation of the 200 keV base image is maintained.
[0050] Figure 5 On the left, conventional CT slice images for comparison are also shown. Typical streak artifacts are visible in these conventional CT slice images, as marked by wide white arrows. Furthermore, areas containing iodine contrast material are indicated by thin white arrows. Again, in the upper right, a 200keV virtual single-energy image used as the base image is shown, which does not contain any of the streak artifacts shown in the conventional CT slice images, but also shows only the very low intensity of areas containing iodine contrast material. In the lower right, an enhanced image generated according to the above embodiment is shown, in which it can be clearly seen, in particular, that areas containing iodine contrast material are enhanced, while simultaneously maintaining the absence of streak artifacts and the preferred soft tissue representation of the 200keV base image.
[0051] While it is preferred in the above embodiments that the predetermined template image refers to the base image, the determination of the saliency level as described above can also be applied to other predetermined template images, particularly all template images that do not refer to constant images. For example, the predetermined template image may be an image of a healthy patient derived from spectral image data or non-spectral image data. Moreover, in this case, the saliency level can be determined by determining an approximation function according to the above description. Furthermore, in the above embodiments, the contrast image and the predetermined template image may also refer to spectral images, as defined in the following embodiments.
[0052] In another preferred embodiment, the contrast image and the predetermined template image refer to spectral images, wherein a spectral image is defined as an image comprising more than one image value for a voxel, wherein each image value indicates the attenuation of the portion of the object corresponding to the voxel when subjected to a different energy spectrum. Thus, a spectral image can be considered as comprising at least two images generated from different combinations of spectral image data. In this case, the image values of the voxels in the spectral image can also be considered as more dimensional image values, i.e., referring to image value vectors. The contrast image and the predetermined template image can then be selected such that they represent multiple different images generated from different combinations of spectral image data, wherein all the different combinations of spectral image data are also different from the combinations of spectral image data from which the base image is generated. In this case, the degree of significance of the difference between the image values of the voxels in the contrast image and the corresponding voxel image values in the predetermined template image can be determined by representing the image value vectors in the spectral space and applying a spectral metric for comparing the image value vectors, wherein, in this case, the image value refers to the image value vector. The spectral space is spanned by different radiation spectra (i.e., different spectral channels) provided by the spectral images. For example, the Euclidean vector distance in the spectral space to, for example, a spectral reference point defined by the predetermined template image can be used to determine the degree of significance. Furthermore, the angle of the image value vector relative to the spectral origin and, for example, a reference vector defined by a predetermined template image in the spectral space can also be used to determine the degree of saliency. Additionally, the Bach distance, defined by the predetermined template image, i.e., the quadratic weighted distance to the spectral reference ellipsoid, can also be used. Typically, the representation of the image value vector of the spectral image in the spectral space can be determined in a manner similar to that described in the embodiments involving providing approximation functions. However, since the spectral space in the embodiments involving spectral images will in most cases include more than two dimensions, similar to... Figure 3 The attached diagram is unavailable.
[0053] The enhanced image generation unit 114 can then be configured to generate an enhanced base image based on a degree of salience determined by the proximity of the more-dimensional image values of the contrast image to the more-dimensional image values of a predetermined template image. Thus, in this case, the degree of salience indicates the distance between points in the more-dimensional space (particularly including a spectral space with more than two dimensions). In this way, information not provided by the spectral template image can be identified in the spectral contrast image (i.e., in multiple combinations of spectral image data). For example, if the spectral template image refers to different combinations of spectral image data acquired for a healthy patient, the method allows for the identification of voxels in the patient's spectral image data that provide additional information in the overall overview across multiple possible radiation spectra and are therefore of interest for the enhanced base image.
[0054] In another embodiment, the saliency determination unit 113 includes a trained deep learning algorithm, particularly a convolutional neural network, trained to determine the saliency of each voxel in a contrast image. Preferably, the deep learning algorithm is trained by providing it with contrast images (e.g., virtual single-energy images, photoelectric images, or spectral images), wherein the saliency of these training contrast images for each voxel has been determined, for example, by a user or through any of the aforementioned principles. The deep learning algorithm can then learn to determine the saliency of each voxel in a contrast image based on the contrast images without further input, wherein the predetermined template image in this case can be considered as part of the variable setting of the trained deep learning algorithm. Based on the saliency determined by the trained deep learning algorithm with the contrast images as input, the base image can then be enhanced.
[0055] Spectral CT imaging systems generate multiple (i.e., at least two) spectral channels ranging from low to high energy levels, governed to varying degrees by the Compton effect and photoelectric effect. These spectral channels (i.e., spectral image data) can also be converted into various representations, such as pure Compton images, pure photoelectric images, and pure virtual single-energy images. Generally, high-keV virtual single-energy images are known to have fewer streak artifacts and less image noise, and are more sensitive in depicting soft tissue material, and are therefore generally preferred for viewing. Moreover, conventional images (i.e., non-virtual single-energy images) are generally preferred for viewing due to existing reading habits and accumulated expertise. On the other hand, low-keV virtual single-energy images are known to show contrasting material more sensitively, while weak concentrations of contrasting material are not well visible in high-keV virtual single-energy images or conventional images. However, patches of weak contrasting material may be helpful in detecting and differentiating abnormalities such as polyps, feces, etc.
[0056] Therefore, in the embodiments of the present invention, a base image is arbitrarily selected by the user, for example, as a conventional or high keV virtual single-energy image, and the image is enhanced by adding intensity or color at image locations that show high intensity in a low keV virtual single-energy image or photoelectric image, i.e., "contrast enhancement".
[0057] In an embodiment, a threshold T (i.e., a constant image serving as a template image) is subtracted from a low-keV single-energy image or photoelectric image, which serves as a contrast image. The resulting subtracted image (i.e., the difference image) is then clamped at zero, leaving only high-contrast regions with positive values and otherwise mostly zero, thus producing a high-contrast image. Optionally, the difference image can be smoothed, for example, by Gaussian smoothing or binomial smoothing, to reduce noise and produce a softer appearance. The smoothed difference image can then be added to the base image on a voxel-by-voxel basis. Optionally, before addition, the difference image can be multiplied by a weighting factor α, which can be interactively changed by the user.
[0058] In this embodiment, the comparison on a voxel-by-voxel basis refers, in this case, to the intensity of a selected base image of the template image and a low-keV virtual single-energy image or photoelectric image, which serves as a contrast image. Regression is performed such that the low-keV virtual single-energy image or photoelectric image is fitted, for example, by linear regression to the prediction made from the base image; however, nonlinear fitting is also possible. Optionally, since the highest contrast point is expected for areas denser than water, the regression is limited to points with Heinz values below 0. After establishing the regression, a low-keV regression value is calculated for all voxels x as an approximate contrast image value. For voxels whose actual low-keV value is higher than the regression value predicted by regression from the base image value, the display intensity of the base image increases the intensity of that difference, which represents the degree of significance in this case. The result is a better image quality including the base image while also showing enhancement (i.e., “upgrading”) of the base image in areas with weak contrast material concentration.
[0059] In this embodiment, spectral salient voxels are determined by comparing a spectral contrast image with a spectral template image. The proximity to the spectral template can be measured using several metrics across the spectral space spanned by the spectral channels. These metrics can be, for example, the Euclidean vector distance to a spectral reference point in the spectral space, the spectral vector angle relative to the spectral origin and reference point, the Bachmann distance to the spectral reference ellipsoid (i.e., the second-weighted distance), etc. The image values of the base image can then be increased based on the degree of salientity determined by the proximity to the spectral template image.
[0060] In one embodiment, during the training phase, a user can annotate regions in, for example, low-keV virtual single-energy images or photoelectric images as images they deem spectrally significant in contrast. A convolutional neural network (CNN) can then be trained using deep learning algorithms to recognize these regions at the voxel level given a multispectral input image. During the deployment phase, the trained CNN generates voxel-level responses based on the multispectral input image, and these voxels are emphasized in intensity in the base image.
[0061] The present invention, as described above, offers the following advantages: users can evaluate only one image type, rather than having to read two or even an entire series of different image representations. Furthermore, for diagnostic viewing, i.e., reading, users are free to choose their preferred conventional or virtual single-energy images to best support their workflow without concern for impaired contrast agent visibility. If white is used for the enhanced salient voxels, the enhanced image looks very similar to the standard image, eliminating the need to develop new reading expertise. This also allows for deployment on a wide range of black and white monitors. Moreover, the intensity is only increased, i.e., “enhanced,” but never decreased, so that dark streaks, for example, arising from beam deficiency in low-energy images are not carried, but rather cues from the high contrast of iodine are. Furthermore, the enhancement embodiments described above are fast enough for interactive, real-time computation and do not require additional storage for the enhanced image volume. The enhanced images can also be used as input to existing image processing algorithms trained on conventional images but benefiting from enhancement of weak-contrast areas (e.g., so-called virtual cleaning of marked feces in CT colonoscopy).
[0062] Although the above embodiments use a specific combination of contrast and base images as examples, other combinations of base and contrast images may be used in other embodiments. For example, while it is generally preferred that the base image refers to a high keV virtual single-energy image (e.g., a 200 keV virtual single-energy image), the base image may also refer to any combination of spectral image data, such as a Compton image or a material decomposition image. While it is also generally preferred that the contrast image refers to a photoelectric image or a low keV virtual single-energy image (e.g., a 40 keV virtual single-energy image), the contrast image may also refer to any other type of image, such as a Compton image, a material decomposition image, or any other combination of spectral image data.
[0063] Although the above embodiments are described in a medical setting, where the object refers to anatomical structures within a patient's body, the principles described can be readily applied to other objects, particularly inanimate objects such as suitcases or bags. In particular, the invention can also be applied to security scenarios, such as scanning bags for dangerous or prohibited objects.
[0064] While it is preferred to inject an iodine-based contrast agent into the patient before acquiring spectral images, other contrast agents, such as barium-based ones, may also be used. Furthermore, contrast agents may not be used for clinical reasons related to spectral imaging. In such cases, baseline and contrast images containing additional information about the patient's anatomy that is visible even without contrast agents can be used.
[0065] Those skilled in the art, through studying the accompanying drawings, the disclosure, and the claims, can understand and implement other variations of the disclosed embodiments when practicing the claimed invention.
[0066] In the claims, the word "comprising" does not exclude other elements or steps, and the words "a" or "an" do not exclude a plurality.
[0067] A single unit or device can perform the functions of several items recited in the claims. Although some measures are recited in different dependent claims, this does not mean that combinations of these measures cannot be used advantageously.
[0068] Processes such as providing contrast images, providing base images, determining saliency levels, or enhancing base images, performed by one or more units or devices, can be performed by any other number of units or devices. These processes can be implemented as program code modules of computer programs and / or as dedicated hardware.
[0069] Computer programs can be stored / distributed on suitable media, such as optical storage media or solid-state media supplied together with or as part of other hardware, but computer programs can also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems.
[0070] No reference numerals in the claims should be construed as limiting the scope.
[0071] This invention relates to an apparatus for generating enhanced images, comprising: a) a base image providing unit, wherein the base image is generated based on a combination of spectral image data; b) a contrast image providing unit, wherein the contrast images are generated based on different combinations of the spectral image data; c) a saliency determination unit, wherein the saliency indicates the difference between the image values of voxels in the contrast image and the image values of corresponding voxels in a predetermined template image; and d) an enhanced image generation unit for generating an enhanced base image of an object by enhancing the voxels of the base image based on the saliency. This invention allows for providing enhanced base images with improved image quality and information content.
Claims
1. An apparatus for generating an enhanced image of an object, wherein, The device (110) includes: - A base image providing unit (111) is configured to provide a base image of an object, wherein the base image is generated based on a combination of spectral image data of the object acquired using a spectral X-ray imaging device (120). - A contrast image providing unit (112) is configured to provide a contrast image of the object, wherein the contrast image is generated based on a combination of spectral image data of the object that is different from the combination used to generate the base image. - A saliency determination unit (113) is configured to determine a saliency level for each voxel of the contrast image, wherein the saliency level indicates the difference between the image value of a voxel in the contrast image and the image value of a corresponding voxel in a predetermined template image, and - An enhanced image generation unit (114) is configured to generate an enhanced base image of the object by enhancing the voxels of the base image based on the saliency degree determined for the corresponding voxels of the contrast image, wherein the saliency degree determination unit (113) is configured to determine an approximation function that allows the image values of the voxels of the contrast image to be approximated based on the image values of the corresponding voxels of the predetermined template image, and wherein the saliency degree determination unit (113) is further configured to generate an approximate contrast image based on the approximation function and the predetermined template image and determine the saliency degree based on the approximation function.
2. The apparatus according to claim 1, wherein, The saliency determination unit (113) is configured to determine the saliency of a voxel such that the saliency indicates the deviation between the image value of the voxel in the contrast image and the image value of the corresponding voxel in the approximate contrast image.
3. The apparatus according to claim 2, wherein, The enhanced image generation unit (114) is configured to generate the enhanced base image by enhancing the following voxels of the base image: for the voxels, the saliency level indicates that the image value of the voxel shows an upward deviation from the approximate image value of the voxel.
4. The apparatus according to any one of claims 1 to 3, wherein, The approximation function refers to a linear regression function between the image value of the voxel in the contrast image and the image value of the voxel in the predetermined template image.
5. The apparatus according to any one of the preceding claims, wherein, The predetermined template image refers to the base image.
6. The apparatus according to claim 1, wherein, The predetermined template image refers to a constant image in which the image values of each voxel are the same, wherein the salience determination unit (113) is configured to determine the salience based on the difference between the image values of the voxels in the contrast image and the constant image values of the predetermined template image.
7. The apparatus according to claim 6, wherein, The enhanced image generation unit (114) is configured to generate the enhanced base image by enhancing the following voxels of the base image: for the voxels, the saliency level indicates that the image value of the voxel of the contrast image is greater than the constant image value of the predetermined template image.
8. The apparatus according to any one of the preceding claims, wherein, The contrast image refers to a virtual single-energy image or a photoelectric image.
9. The apparatus according to any one of claims 1 to 7, wherein, The base image and the contrast image are virtual single-energy images, wherein the contrast image is generated with reference to an energy lower than the energy used to generate the base image.
10. The apparatus according to claim 1, wherein, The contrast image and the predetermined template image refer to spectral images, wherein the spectral image includes at least two intensity values for each voxel, the at least two intensity values indicating the attenuation of a portion of the object corresponding to the voxel at different radiation energy spectra, and wherein the saliency determination unit (113) is configured to determine the saliency based on a spectral metric indicating the difference between the image value of the contrast image and the image value of the predetermined template image in the spectral space defined by the different radiation energy spectra.
11. The apparatus according to any one of the preceding claims, wherein, The saliency determination unit (113) is configured to generate the saliency using a trained deep learning algorithm, wherein the trained deep learning algorithm is trained based on a training contrast image in which each voxel is associated with a saliency.
12. The apparatus according to any one of the preceding claims, wherein, The enhanced image generation unit (114) is configured to enhance the voxels by adding enhancement values to the voxels of the base image and / or by providing color to the voxels, wherein the enhancement values are based on the saliency level and the color is based on the saliency level.
13. A method for generating an enhanced image of an object, wherein, The method includes: - Provide a base image of the object (210), wherein the base image is generated based on a combination of spectral image data of the object acquired using a spectral X-ray imaging device (120). - Provide (220) a contrast image of the object, wherein the contrast image is generated based on a combination of spectral image data of the object that is different from the combination used to generate the base image. - For each voxel of the contrast image, a saliency level is determined (230), wherein the saliency level indicates the difference between the image value of a voxel in the contrast image and the image value of a corresponding voxel in a predetermined template image, and - An enhanced base image of the object is generated by enhancing the voxels of the base image based on the degree of salience determined for the corresponding voxels of the contrast image. (240) The determination of saliency (230) for each voxel of the contrast image includes: determining an approximation function that allows the image values of the voxels of the contrast image to be approximated based on the image values of the corresponding voxels of the predetermined template image; generating an approximate contrast image based on the approximation function and the predetermined template image; and determining the saliency based on the approximation function.
14. A computer program for enhancing an image of an object, wherein, The computer program includes a program code module that, when the computer program is run by the apparatus (110) according to claim 1, the program code module is used to cause the apparatus (110) to perform the steps of the method according to claim 13.
Citation Information
Patent Citations
Robust image registration method based on association saliency image in global abnormal signal environment
CN101038669A
Hyperspectral remote sensing image small target detection method based on spectrum saliency
CN103729848A
