3-D Virtual Endoscopy Rendering System, Method, and Storage Medium

Abstract

The three-dimensional virtual endoscopy rendering of the lumen of a tubular structure is based on non-spectral and spectral volume imaging data. The three-dimensional virtual endoscopy rendering includes 2D images of the lumen of the tubular structure from the viewpoint of a virtual camera of a virtual endoscope passing through the lumen. In one case, the three-dimensional virtual endoscopy rendering is similar to the view provided by a physical endoscope camera inserted into and positioned at the actual tubular structure. The non-spectral volume image data is used to determine the opacity and shading of the three-dimensional virtual endoscopy rendering. The spectral volume image data is used to visually encode the three-dimensional virtual endoscopy rendering to visually distinguish the inner wall of the tubular structure and the structures of interest on the wall.

Description

Technical Field

[0001] The following generally relates to imaging, and more specifically to 3-D virtual endoscopy rendering, and is described with respect to a particular application of computed tomography. Background Art

[0002] Polyps in the colon may develop into colon cancer. Literature indicates that if such polyps are removed early, cancer can be very effectively prevented. Colonoscopy is a procedure available to asymptomatic subjects over a certain age to detect and evaluate possible polyps. For colonoscopy, gas is blown into the colon to inflate the colon, making it easier to examine the colon wall. An endoscope, which includes a video camera on a flexible tube, is inserted through the anus into the colon and through the lumen of the colon. As the video camera passes through the lumen, it records images of the inner wall of the colon. These images can be used for a visual inspection of the wall. During the procedure, biopsies and / or removals can be performed on suspicious polyps. Endoscopic colonoscopy is an invasive procedure.

[0003] Computed tomography (CT) virtual colonoscopy (VC) is a non-invasive imaging procedure. Using CT VC, volumetric image data of the colon is acquired and processed to generate a three-dimensional virtual endoscopy (3-D VE) rendering of the lumen of the colon from the viewpoint of a virtual camera passing through the lumen of the colon via 2D images of the colon lumen, with local shapes of shadows and isosurfaces derived according to the viewing direction to provide depth information. Generally, the 2-D images provide a 3-D impression similar to a real endoscopic view. However, there are several types of polyps (e.g., flat and / or serrated polyps) that are difficult to visually detect by shape in 3-D VE because they are inconspicuous on the colon wall. Thus, there is an unmet need for an improved 3-D VE. Summary of the Invention

[0004] Aspects described herein solve the above-mentioned problems and other problems.

[0005] A 3-D VE rendering of the lumen of a tubular structure is based on non-spectral volumetric imaging data and spectral volumetric imaging data. The non-spectral volumetric image data is used to determine the opacity and shadows of the 3-D VE rendering. The spectral volumetric image data is used to visually encode the 3-D VE rendering to visually distinguish the inner wall of the tubular structure and the structures of interest on the wall.

[0006] In one aspect, a system includes a processor and a memory storage device configured with a three-dimensional virtual endoscopy module and a rendering module. The processor is configured to process non-spectral volumetric image data from a scan of a tubular structure using the three-dimensional virtual endoscopy module to generate a three-dimensional endoscopic rendering of the lumen of the tubular structure having opacity and shading that provide a three-dimensional impression. The processor is further configured to process spectral volumetric image data from the same scan using the three-dimensional virtual endoscopy module to produce a visual encoding on the three-dimensional endoscopic rendering that visually differentiates the wall of the tubular structure from the structures of interest on the wall based on spectral characteristics. The processor is further configured to execute the rendering module to display the three-dimensional endoscopic rendering with the visual encoding via a display monitor.

[0007] In another aspect, a method includes generating a three-dimensional endoscopic rendering of the lumen of a tubular structure having opacity and shading that provide a three-dimensional impression based on non-spectral volumetric image data from a scan of the tubular structure. The method further includes generating a visual encoding for the three-dimensional endoscopic rendering that visually differentiates the wall of the tubular structure from the structures of interest on the wall based on spectral characteristics determined from the spectral volumetric image data from the scan. The method further includes displaying the three-dimensional endoscopic rendering with the visual encoding.

[0008] In another aspect, a computer-readable storage medium stores instructions that, when run on a computer, cause the computer to perform a method for generating a three-dimensional endoscopic rendering using a computer system. The method includes generating a three-dimensional endoscopic rendering of the lumen of a tubular structure having opacity and shading that provide a three-dimensional impression based on non-spectral volumetric image data from a scan of the tubular structure. The method further includes generating a visual encoding for the three-dimensional endoscopic rendering that visually differentiates the wall of the tubular structure from the structures of interest on the wall based on spectral characteristics determined from the spectral volumetric image data from the scan. The method further includes displaying the three-dimensional endoscopic rendering with the visual encoding.

[0009] Those skilled in the art will recognize other aspects of the present application upon reading and understanding the attached description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The invention may take form in various components and arrangements of components and in various steps and arrangements of steps. The drawings are for purposes of illustrating embodiments only and should not be regarded as limiting the invention.

[0011] Figure 1 The figures illustrate a system including a 3-D VE module in accordance with one or more embodiments herein.

[0012] Figure 2Shows an example 3-D VE rendering generated by the 3-D VE module of Figure 1 according to one or more embodiments herein. Figure 1 of

[0013] Figure 3 Shows a display with multiple windows that displays the 3-D VE rendering and other images.

[0014] Figure 4 Shows an example scatter plot of a dual-energy scan according to one or more embodiments herein.

[0015] Figure 5 Illustrates an exemplary method according to one or more embodiments herein. Detailed Description

[0016] A method for generating a 3-D VE rendering of a lumen based on non-spectral and spectral volume imaging data is described below. The spectral volume imaging data is at least used to visually encode the inner wall of a tubular structure and the structures / materials thereon based on the spectral properties of the wall and the structures / materials. 3-D VE is a 2-D image that provides a 3-D impression similar to a real endoscopic view. Examples of tubular structure / VE procedures include colon / VC, bronchus / virtual bronchoscopy (VB), etc. For 3-D VC, in one case, structures such as flat and / or serrated polyps are difficult to visually detect by shape due to their inconspicuousness with respect to the colon, and are video-encoded in the displayed 2-D image based on their spectral properties and are different from the colon wall, which may allow the polyps to be visually distinguished from the wall tissue, feces, etc. based on the spectral properties.

[0017] Figure 1 Illustratively shows a system 100 according to one or more embodiments herein. The system 100 includes a computed tomography (CT) scanner 102. The CT scanner 102 is configured for non-spectral imaging and spectral (multi-energy) imaging, such as dual-energy imaging. The CT scanner 102 includes a stationary gantry 104 and a rotating gantry 106, and the rotating gantry 106 is rotatably supported by the stationary gantry 104 and rotates around the examination area 108 (and the object or a part of the object therein) about a longitudinal or z-axis. An object support 110, such as a couch, supports the object or target in the examination area 108. The object support 110 can be moved in coordination with the execution of the imaging procedure so as to guide the object or target with respect to the examination area 108 to load, scan, and / or unload the object or target. For VC scans, the object ingests a radioactive contrast agent, such as iodine, barium, etc., before scanning.

[0018] The radiation source 112, such as an X-ray tube, is supported by a rotating gantry 106 and rotates around the examination region 108. The radiation source 112 emits X-ray radiation, which is collimated by, for example, a source collimator (not visible) to form an X-ray radiation beam having a generally fan-shaped, wedge-shaped, conical, or other shape passing through the examination region 108. In one case, the radiation source 112 is a single X-ray tube configured to emit broadband (polychromatic) radiation at a single selected peak emission voltage (kVp) of interest. In another example, the radiation source 112 is configured to switch between at least two different emission voltages (e.g., 70 keV, 100 keV, 120 keV, 140 keV, etc.) during a scan. In yet another example, the radiation source 112 includes two or more X-ray tubes angularly offset on the rotating gantry 104, each X-ray tube being configured to emit radiation having a different average energy spectrum. In yet another case, the CT scanner 102 includes a combination of two or more of the above. Examples of kVp switching and / or multiple X-ray tubes are described in US 8442184 B2, entitled "Spectral CT," filed on June 1, 2009, which is hereby incorporated by reference in its entirety.

[0019] The radiation-sensitive detector array 114 faces an angular arc and is opposite the radiation source 112 across the examination region 108. The detector array 114 includes one or more rows of detectors arranged relative to each other along the z-axis direction and detects the radiation passing through the examination region 108. In one case, the detector array 114 includes energy-resolving detectors, such as multi-layer scintillator / photosensor detectors. An example system is described in US 7968853B2, entitled "Doubledecker detector for spectral CT," filed on April 10, 2006, which is hereby incorporated by reference in its entirety. In another example, the detector array 114 includes photon-counting (direct conversion) detectors. An example system is described in US 7668289B2, entitled "Energy-resolved photon counting for CT," filed on April 25, 2006, which is hereby incorporated by reference in its entirety. In these cases, the radiation source 112 includes a broadband, kVp-switching, and / or multiple X-ray tube radiation source. In the case where the detector array 114 includes non-energy-resolving detectors, the radiation source 112 includes a kVp-switching and / or multiple X-ray tube radiation source. The radiation-sensitive detector array 114 at least produces spectral projection data (line integrals) indicative of the examination region 108. In one configuration, the radiation-sensitive detector array 114 also produces non-spectral projection data.

[0020] The reconstructor 116 processes projection data from the same scan and generates spectral volume image data and non-spectral volume image data. The reconstructor 116 generates spectral volume image data by reconstructing spectral projection data of different energy bands. Examples of spectral volume image data include low-energy volume image data and high-energy volume image data for dual-energy scans. Other spectral volume image data can be obtained by material decomposition in the projection domain followed by reconstruction or derived in the image domain. Examples include Compton scatter (Sc) and photoelectric effect (Pe) bases, effective atomic number (Z-value), contrast agent (such as iodine, barium, etc.) concentration, and / or other bases. In the case where the CT scanner 102 generates non-spectral projection data, the reconstructor 116 uses the non-spectral projection data to generate non-spectral volume image data. Otherwise, the reconstructor 116 generates non-spectral volume image data by combining spectral projection data to produce non-spectral projection data and reconstructing the non-spectral projection data and / or combining spectral volume image data to produce non-spectral volume image data. The reconstructor 116 can be implemented using a processor such as a central processing unit (CPU), microprocessor, etc.

[0021] The operator console 118 includes a human-readable output device 120, such as a display monitor, film printer, etc., and an input device 122, such as a keyboard, mouse, etc. The console 118 also includes a processor 124 (e.g., CPU, microprocessor, etc.) and a computer-readable storage medium 126 (excluding temporary media) such as physical memory, like a memory storage device, etc. In the illustrated embodiment, the computer-readable storage medium 126 includes a 3-D virtual endoscopy (VE) module 128, a rendering module 130, and an artificial intelligence (AI) module 132, and the processor 124 is configured to execute computer-readable instructions of the 3-D VE module 128, the rendering module 130, and / or the AI module 132, which causes the functions described below to be performed.

[0022] In a variant, the 3-D VE module 128, the rendering module 130, and / or the AI module 132 are executed by processors in different computing systems, such as a dedicated workstation located far from the CT scanner 102, cloud-based resources, etc. The different computing systems can receive data from the CT scanner 102, another scanner, a data repository (e.g., a radiology information system (RIS), a picture archiving and communication system (PACS), a hospital information system (HIS), etc.), and so on. The different computing systems can additionally or alternatively receive projection data from the CT scanner 102, another scanner, a data repository, etc. In this case, the different computing systems can include a reconstructor similarly configured to the reconstructor 116, as it can process projection data and generate spectral and non-spectral volume image data.

[0023] The 3-D VE module 128 generates a 3-D VE rendering (i.e., a 2-D image with a 3-D impression) of a tubular structure from the viewpoint of a virtual camera of a virtual endoscope passing through the lumen, based on non-spectral volume image data and spectral volume image data. In one case, the 3-D VE rendering resembles the view provided by a physical endoscope camera inserted into and positioned within an actual tubular structure. In such a case, for example, the 3-D VE rendering shows the inner wall of the tubular structure, including surfaces and structures / materials on the inner wall surface.

[0024] In one case, the 3-D VE module 128 uses non-spectral volume image data to determine local opacity and gradient shading. The 3-D VE module 128 may employ volume rendering, surface rendering, and / or other methods. Using volume rendering, virtual view rays are projected through the non-spectral volume image data, and regions around the tubular structure wall are probed by steep gradients at the air / wall interfaces along the rays, where the opacity of data points in the regions starts low and increases sharply with a gradient. The shading is determined based on the angle between the viewing angle and the local gradient and is achieved through intensity. Using surface rendering, a mesh (e.g., triangular or otherwise) is fitted to the wall, e.g., at the strongest gradient, which is at the air / wall interface along the ray. Example methods for generating 3-D VE renderings are described in US 7839402 B2, titled “Virtual endoscopy,” filed on June 2, 2005, which is hereby incorporated by reference in its entirety.

[0025] The 3-D VE module 128 generates a visual encoding for the pixels of the 3-D rendering, based on spectral characteristics of the spectral volume image data, representing the surface of the inner wall of the tubular structure and / or structures / materials thereon. As described in more detail below, in one case, the visual encoding includes hue encoding and corresponds to spectral angle, effective atomic number, contrast agent concentration, other spectral characteristics, and / or combinations thereof. The visual encoding can convey visual cues to the observer regarding the presence and / or type of structures / materials. These spectral characteristics cannot be determined based on non-spectral volume image data alone, and thus the visual encoding is not part of existing 3-D VE techniques. Thus, the methods described herein also utilize spectral volume image data to visually encode the 3-D VE rendering based on spectral characteristics, representing an improvement over existing 3-D VE techniques. Additionally, the visual encoding allows for effective visual assessment.

[0026] The rendering module 130 displays the 3-D VE rendering via the display monitor of the output device 120. As Figure 2Shows an example 3-D VE rendering 200 of the inner wall 202 of the colon, with visual encodings 204 and 206 for polyps 208 and 210 respectively. For purposes of explanation, visual encodings 204 and 206 are shown as the black perimeters of polyps 208 and 210. Other visual encodings may include hue, brightness, cross-hatching, annotations, and / or other visual encodings. In one example, the visual encodings are displayed based on a predetermined opacity level, such as a default static opacity level, a user-defined opacity level, an opacity level defined for a particular type of scan, etc. The predetermined opacity level may be fully or semi-transparent. In another example, the opacity level can be adjusted between full (or other level of) transparency and full (or other level of) opacity through one or more intermediate partially transparent / opaque levels. In this case, the console 118 includes controls such as physical controls like keyboard buttons or soft controls such as graphical user interface menu options, graphical sliders, dials, etc. In one example, the adjustable controls allow for comprehensive and effective visual evaluation.

[0027] The rendering module 130 displays the 3-D VE rendering alone or in combination with one or more other images, such as one or more slices (axial, coronal, sagittal, oblique, etc.) of non-spectral and / or spectral volume image data, presenting only the tubular structure such that the material outside the tubular structure is not visible, etc. In one case, the same visual encoding (e.g., hue) is displayed simultaneously in one or more other displayed images, and its visualization is controlled independently of or dependent on the control of the visual encoding in the displayed 3-D VE. Additionally or alternatively, the 3-D VE rendering is interactively linked (spatially coupled) to one or more other displayed images such that hovering the display pointer (e.g., mouse pointer) over the displayed 3-D VE rendering causes a position indicator to be displayed in one or more other displayed images indicating the position of the pointer in the displayed 3-D VE rendering.

[0028] Figure 3 Shows an example display with multiple windows 302, 304, 306, and 310. Window 302 shows the 3-D VE rendering 200 from Figure 2 Window 304 and 306 display polar (angular) representations derived from the photoelectric effect and Compton scattering spectral volume image datasets. Window 304 shows a first axial slice image representing amplitude. Window 306 shows a second axial slice image representing angle. Windows 304 and 306 respectively include indicators of a portion of the colon shown in window 302 and corresponding to Figure 2Visual coding markers 310 and 312 for polyps 208 and 210. For purposes of explanation, polyps 208 and 210 are color-coded in grayscale in windows 304 and 306, while polyps 208 and 210 are contour-coded in window 302. However, it should be understood that the same visual coding can be used in both window 302 and windows 304 and 306. Window 306 shows only an image of the colon, where the markers indicate positions in the colon corresponding to the 3-D rendering in window 302.

[0029] Return to Figure 1 , the AI module 132 processes at least the 3-D VE rendering and generates and displays, via the output device 120, information that can assist the computing system and / or clinician in differentiating the visual coding of the structure / material of interest from other structures / materials having similar shape, texture, and / or spectral characteristics. For example, for CT VC, the information can assist in differentiating the visual coding of contrast-enhanced flat polyps and / or serrated polyps from the similar visual coding of contrast-enhanced feces on the surface wall of the colon. In one example, the AI module 132 is trained with a training set including 3-D VE renderings having the structure / material of interest and no other structures / materials (e.g., flat and / or serrated polyps and no feces) and a training set of 3-D VE renderings having other structures / materials and no structure / material of interest (e.g., feces and no flat and / or serrated polyps). This information does not provide a diagnosis or treatment, but can assist the clinician in evaluating the 3-D VE rendering.

[0030] In one example, AI module 132 includes deep learning algorithms, such as feedforward artificial neural networks (e.g., convolutional neural networks) and / or other neural networks to learn patterns of spectral characteristics of different materials and / or structures, etc., to distinguish a structure / material of interest from other structures / materials. Examples of such algorithms are discussed in “Fast Sliding Window Classification with Convolutional Neural Networks” by Gouk et al., IVNVZ‘14 Proceedings of the 29th International Conference on Image and Vision Computing New Zealand, pages 114 - 118 (November 19 - 21, 2014), “Fully convolutional networks for semantic segmentation”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015), and “U-Net: Convolution Networks for Biomedical Image Segmentation” by Ronneberger et al., Medical Image Computing and Computer-Assisted Intervention (MICCAI), Springer, LNCS, volume 9351: 234 - 241 (2015). In one variant, AI module 132 is omitted.

[0031] For some scans, an auxiliary device is used with CT scanner 102. For example, for VC scans, an insufflator 122 can be used to insufflate the colon for the scan. In one case, the insufflator 122 is used to blow gas into the colon to insufflate the colon so that the colon wall can be more easily examined via the displayed 3-D VE rendering.

[0032] As described above, the VE module 128 generates a visual encoding for 3-D rendering based on spectral characteristics determined from spectral volume image data. Non-limiting examples of the visual encoding including hue are described below. In one case, this includes determining the hue for a pixel based on a spectral angle. In another example, this includes determining the hue for a pixel based on an effective atomic number. In yet another example, this includes determining the hue for a pixel based on a contrast agent concentration. In another example, this includes determining the hue for a pixel based on one or more other spectral characteristics. In yet another example, this includes determining the hue for a pixel based on a combination of two or more of the above.

[0033] A method for determining a spectral angle includes determining a scatter plot based on a spectral volume image data set, where different spectral volume image data sets are located on different axes of the scatter plot. Figure 4 An example scatter plot 400 of a dual-energy scan with Compton scatter (Sc) and photoelectric effect (Pe) substrates is shown. A first axis 402 represents voxel values of the Sc volume image data (e.g., in Hounsfield units (HU) / CT number scale) and a second axis 404 represents voxel values of the Pe volume image data (i.e., in the same scale). In this example, the first axis 402 is assigned a first hue and the second axis 404 is assigned a second different hue. For example, the first axis 402 is assigned red and the second axis 404 is assigned blue. In another example, the first axis 402 is assigned red and the second axis 404 is assigned white. The origin 406 represents the HU values of materials of interest (e.g., air (-1000), water (0), contrast agent, soft tissue (100 - 300), etc.).

[0034] In Figure 4 it, a first point 408 corresponds to voxel values from different spectral volume image data sets for the same first voxel position (x, y, z) in the spectral volume image data set. A second point 410 corresponds to voxel values in different spectral volume image data sets for the same second voxel position in the spectral volume image data set, where the first and second voxel positions are different. In this example, the voxels for the first and second points 408 and 410 represent different materials with different spectral characteristics. The voxel for point 410 has a greater Pe contribution and a lower Sc contribution relative to the voxel for point 408. Other voxels representing the material corresponding to point 408 will be located near point 408, while other voxels representing the material corresponding to point 410 will be located near point 410.

[0035] In this example, the spectral angle θ for the first point 408 is determined as the angle between the first axis 402 and the line 412 extending from the first point 408 to the origin 406. In one case, θ is determined by calculating the reciprocal of the tangent of the ratio of the Pe voxel value to the Sc voxel value (i.e., θ = arctan(HU Pe / HU Sc ))). The spectral angle φ for the first point 410 is determined as the angle between the first axis 402 and the line 414 extending from the second point 410 to the origin 406. In one case, φ is determined by calculating the reciprocal of the tangent of the ratio of the Pe voxel value to the Sc voxel value (i.e., φ = arctan(HU Pe / HU Sc ))). In this example, the spectral angle θ for the pixel corresponding to point 408 is less than the spectral angle φ for the pixel corresponding to point 410. As Figure 4 only two points are shown for illustrative purposes, it should be understood that FIG. 400 can be used to determine the hue for all pixels in the 3-D VE rendering or for a predetermined set of pixels.

[0036] In one instance, the hue of a pixel is determined as a linear blend between a first hue of the first axis 402 and a second hue of the second axis 404 according to the spectral angle. The hue of the pixel corresponding to point 408 will be a linear blend based on the angle θ and will include a greater contribution of the first color relative to the second color because the illustrated angle θ is less than ninety degrees. The hue of the pixel corresponding to point 410 will be a linear blend based on the angle φ and will include a greater contribution of the second color relative to the first color because the illustrated angle φ is greater than ninety degrees. In one variant, the spectral angle is non-linearly converted into a pseudocolor scale, such as a rainbow color scale, a temperature scale, and / or other hue chromaticities.

[0037] In another embodiment, a volume rendering algorithm is employed, in which the spectral angle is calculated according to a predetermined region of the wall of a tubular structure in the spectral volume image data, in the wall where the ray opacity saturates to a unit value, and the predetermined region includes the voxels before the wall and the voxels after the wall. In one case, the hue is determined according to the linear superposition from each position within the predetermined region, and each position is weighted by the local opacity. In one variant, alternatively the hue is determined as the maximum value within the predetermined region. In yet another variant, alternatively the hue is determined as the average or median value within the predetermined region. In yet another instance, the hue is determined in other ways or based on a combination of the foregoing.

[0038] In another embodiment, the hue is also determined based on the radial distance and the spectral angle between points on the drawing. For example, in Figure 4In this embodiment, the hue of the pixel corresponding to point 408 is based not only on the angle θ, but also on the distance between point 408 and the origin 406. Similarly, the hue of the pixel corresponding to point 410 is based not only on the angle φ, but also on the distance between point 410 and the origin 406. For this reason, voxels with the same spectral angle do not necessarily have the same hue, depending on their distance from the origin 406 at the points in plot 400. Alternatively, the hue is not determined based on the radial distance and spectral angle between points on the plot.

[0039] In another embodiment, Figure 4 each coordinate in the grid in scatter plot 400 in is assigned a hue. In this embodiment, the pixels are visually encoded with a hue corresponding to the hue of the coordinate at the intersection of the pixel values assigned to the spectral volume image dataset. For example, the pixel for point 408 is visually encoded with the hue assigned to the coordinate at point 408 in the grid, the pixel for point 410 is visually encoded with the hue assigned to the coordinate at point 410 in the grid, and so on. This embodiment may not employ the spectral angle.

[0040] In another example, the hue is determined in another way based on a combination of two or more of the above methods.

[0041] Figure 5 illustrates an exemplary method according to one or more embodiments herein.

[0042] It is to be understood that the order of the actions in the method is not restrictive. Thus, other orders are contemplated herein. Additionally, one or more actions may be omitted and / or one or more additional actions may be included.

[0043] At 502, non-spectral and spectral volume image data of the same scan from a tubular structure are obtained as described herein and / or otherwise.

[0044] At 504, a 3-D VE rendering is generated based on the non-spectral volume image data, which includes opacity and shading, as described herein and / or otherwise.

[0045] At 506, the visual encoding for the pixels of the 3-D VE rendering is determined based on the spectral volume image data, as described herein and / or otherwise.

[0046] At 508, the 3-D VE rendering is displayed using the visual encoding, as described herein and / or otherwise.

[0047] The above can be implemented by computer-readable instructions that are embedded in or encoded on a computer-readable storage medium and that, when executed by a computer processor, cause the processor to perform the actions. Additionally or alternatively, at least one of the computer-readable instructions is carried by a signal, carrier, or other transitory medium that is not a computer-readable storage medium.

[0048] Although the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive. The invention is not limited to the disclosed embodiments. Those skilled in the art will be able to understand and realize other variations of the disclosed embodiments when practicing the claimed invention by studying the drawings, the disclosure, and the claims.

[0049] The word "comprising" does not exclude other elements or steps, and the words "a" or "an" do not exclude a plurality. A single processor or other unit may implement the functions of several items recited in the claims. Although specific measures are recited in mutually different dependent claims, this does not indicate that a combination of these measures cannot be used to advantage.

[0050] A computer program may be stored / distributed on a suitable medium such as an optical storage medium or a solid-state medium provided together with or as part of other hardware, but may also be distributed in other forms such as via the Internet or other wired or wireless telecommunication systems. Any reference signs in the claims should not be construed as limiting the scope.

Claims

1. A 3-D virtual endoscopy rendering system (100) comprising: A processor (124); And A memory storage device (126) configured with a three-dimensional virtual endoscopy module (128) and a rendering module (130), Wherein the processor is configured to process non-spectral volume image data from a CT scan of a tubular structure using the three-dimensional virtual endoscopy module to generate a three-dimensional endoscopy rendering of the lumen of the tubular structure, the three-dimensional endoscopy rendering having an opacity and shading that provide a three-dimensional impression; Wherein the processor is further configured to process spectral volume image data from the same CT scan using the three-dimensional virtual endoscopy module to produce a visual encoding on the three-dimensional endoscopy rendering, the visual encoding visually differentiating the wall of the tubular structure from the structure of interest on the wall based on spectral characteristics; and Wherein the processor is further configured to run the rendering module to display the three-dimensional endoscopy rendering with the visual encoding via a display monitor; Wherein the processor is configured to determine the visual encoding based on a spectral angle determined from the spectral volume image data, wherein determining the visual encoding includes determining a corresponding hue for the pixel based on a spectral angle determined from voxel values for voxels corresponding to the pixel on the three-dimensional endoscopy rendering.

2. The 3-D virtual endoscope rendering system according to claim 1, wherein, The visual encoding includes different hues for the wall of the tubular structure and the structure of interest.

3. The 3-D virtual endoscope rendering system according to claim 1 or 2, wherein, The spectral volume image data includes a first set of spectral volume images corresponding to a first energy and a second set of spectral volume images corresponding to a second different energy, and the processor is configured to determine the spectral angle based on the arctangent of the ratio of a first voxel value of voxel coordinates from the first set of spectral volume image data to a second voxel value of the voxel coordinates from the second set of spectral volume image data.

4. The 3-D virtual endoscope rendering system according to claim 1 or 2, wherein, The processor is configured to determine the hue for the pixel based on a linear mixture of a first color representing a minimum spectral angle and a second color representing a maximum spectral angle.

5. The 3-D virtual endoscope rendering system according to claim 1 or 2, wherein, The processor is configured to determine the hue for the pixel based on non-linearly converting the spectral angle into a color scale.

6. The 3-D virtual endoscope rendering system according to claim 1 or 2, wherein, The processor is configured to determine the hue for the pixel based on a linear superposition of voxel values from different positions within a predetermined region of the wall, the voxel values from the different positions being weighted by a local opacity for each position.

7. The 3-D virtual endoscope rendering system according to claim 1 or 2, wherein, The processor is configured to determine the hue for the pixel based on a maximum voxel value, an average voxel value, or a median voxel value within a predetermined voxel of the wall.

8. The 3-D virtual endoscope rendering system according to claim 1 or 2, wherein, The processor is further configured to interactively spatially couple and simultaneously display the three-dimensional endoscopy rendering and a slice image such that a region in the slice image corresponding to a pixel selected in the three-dimensional endoscopy rendering is visually identified.

9. The 3-D virtual endoscope rendering system according to claim 1 or 2, wherein, The processor is further configured to interactively spatially couple and simultaneously display the three-dimensional endoscopy rendering and the slice image with the same visual encoding used in both the three-dimensional endoscopy rendering and the slice image.

10. The 3-D virtual endoscope rendering system according to claim 1 or 2, wherein, The memory storage device is further configured with an artificial intelligence module (132), the artificial intelligence module being trained to distinguish the wall of the tubular structure and the structure of interest on the wall based on the three-dimensional endoscope rendering training set, and the processor is further configured to employ the artificial intelligence module in the three-dimensional endoscope rendering to present information that differentiates the visual encoding of the wall from the visual encoding of the structure of interest.

11. A method for 3-D virtual endoscope rendering, comprising: generating a three-dimensional endoscope rendering of the lumen of the tubular structure with opacity and shading that provide a three-dimensional impression based on non-spectral volume image data from a CT scan of the tubular structure; generating a visual encoding for the three-dimensional endoscope rendering, the visual encoding visually differentiating the wall of the tubular structure and the structure of interest on the wall based on spectral characteristics determined from spectral volume image data from the CT scan, wherein the visual encoding is determined based on a spectral angle determined from the spectral volume image data, and determining the visual encoding includes determining a corresponding hue for the pixel based on a spectral angle determined from voxel values for voxels corresponding to the pixel on the three-dimensional endoscope rendering; and displaying the three-dimensional endoscope rendering with the visual encoding.

12. A computer-readable storage medium storing instructions that, when executed by a computer, cause the computer to perform a method for generating a three-dimensional endoscope rendering using a computer system, the method comprising: generating a three-dimensional endoscope rendering of the lumen of the tubular structure with opacity and shading that provide a three-dimensional impression based on non-spectral volume image data from a CT scan of the tubular structure; generating a visual encoding for the three-dimensional endoscope rendering, the visual encoding visually differentiating the wall of the tubular structure and the structure of interest on the wall based on spectral characteristics determined from spectral volume image data from the CT scan, wherein the visual encoding is determined based on a spectral angle determined from the spectral volume image data, and determining the visual encoding includes determining a corresponding hue for the pixel based on a spectral angle determined from voxel values for voxels corresponding to the pixel on the three-dimensional endoscope rendering; and displaying the three-dimensional endoscope rendering with the visual encoding.

Images