System for fast and improved denoising and contrast enhancement of tomographic imagery and a method thereof

Abstract

ABSTRACT Title: SYSTEM FOR FAST AND IMPROVED DENOISING AND CONTRAST ENHANCEMENT OF TOMOGRAPHIC IMAGERY AND A METHOD THEREOF.  The present invention discloses a system for denoising and enhancing contrast of tomograph generated images comprising of an input interfacing means adapted to communicate with output of any tomograph, a image processor having an operative communication with the tomograph output through said input interfacing means to receive the tomograph generated image, said image processor embodies parameter controlled image operators for processing of the tomograph generated image and providing an enhanced and denoised version of the same, and an output interfacing means for transferring the denoised and the enhanced tomograph image from the image processor to user’s display.

Description

BRIEF DESCRIPTION OF THE ACCOMPANYING DRAWINGS:Figure 1 shows denoising and contrast enhancement method workflow for tomographic imagery in accordance with the present invention.Figure 2 shows highly zoomed portion of (a) a normal image from a typical tomogram of a biological sample taken by a tomograph (FIBSEM device) and (b) the enhanced image after being processed with the image denoising and contrast enhancement method of the present invention.Figure 3 shows highly zoomed portion of (a) a normal image from a typical tomogram of a biological sample taken by tomograph (FIBSEM device) and (b) the enhanced image after being processed with the image denoising and contrast enhancement method of the present invention.Figure 4 shows highly zoomed portion of (a) a normal image from a typical tomogram of a biological sample taken by a tomograph (FIBSEM device) and (b) the enhanced image after being processed with the image denoising and contrast enhancement method of the present invention.Figure 5 shows a full frame of (a) a normal image from a typical tomogram of a biological sample taken by a tomograph (FIBSEM device) and (b) the enhanced image after being processed with the image denoising and contrast enhancement method of the present invention.Figure 6 shows a full frame of (a) a normal image from a typical tomogram of a metallic sample taken by a tomograph (FIBSEM device) and (b) the enhanced image after being processed with the image denoising and contrast enhancement method of the present invention.Figure 7 shows a full frame of (a) a normal image from a typical tomogram of the human pelvic region taken by a tomograph (Computed Tomography (CT) device) and (b) the enhanced image after being processed with the image denoising and contrast enhancement method of the present invention.Figure 8 shows a full frame of (a) a normal image from a typical tomogram of the human brain taken by a tomograph (Magnetic Resonance Imaging (MRI) device) and (b) the enhanced image after being processed with the image denoising and contrast enhancement method of the present invention.Figure 9 shows a sagittal view of (a) a normal image from a typical tomogram of the human brain taken by a tomograph (Magnetic Resonance Imaging (MRI) device) and (b) the enhanced image after being processed with the image denoising and contrast enhancement method of the present invention trained only on the images from a different view direction (axial view) of the MRI volume.Figure 10 shows a 150% magnified view of (a) a normal image from a typical tomogram of a biological sample taken by a tomograph (FIBSEM device) and (b) the enhanced image after being processed with the image denoising and contrast enhancement method of the present invention trained only on the unmagnified FIBSEM images. Figure 11 shows a 150% magnified view of (a) a normal image from a typical tomogram of the human brain taken by a tomograph (Magnetic Resonance Imaging (MRI) device) and (b) the enhanced image after being processed with the image denoising and contrast enhancement method of the present invention trained only on the images from the unmagnified view of the tomographic volume.Figure 12 shows a 45 degrees anti-clockwise rotated view of (a) a normal image from a typical tomogram of a biological sample taken by a tomograph (FIBSEM device) and (b) the enhanced image after being processed with the image denoising and contrast enhancement method of the present invention trained only on the images from the unrotated view of the tomographic volume. Figure 13 shows a 45 degrees anti-clockwise rotated view of (a) a normal image from a typical tomogram of the human brain taken by a tomograph (Magnetic Resonance Imaging (MRI) device) and (b) the enhanced image after being processed with the image denoising and contrast enhancement method of the present invention trained only on the images from the unrotated view of the tomographic volume.Figure 14 shows graphical representation of the contrast enhancement for present method. DESCRIPTION OF THE INVENTION WITH REFERENCE TO THE ACCOMPANYING DRAWINGS:As stated hereinbefore, the present invention discloses a tomographic image denoising and contrast enhancement system and method which would be adapted to cooperate with any tomograph for improving the contrast and denoising the tomograph generated images without physically modifying the device itself. The tomographic image contrast enhancement and denoising system of the present invention basically includes an input interfacing means, a standalone image processor and an output interfacing means. The input interfacing means is adapted to communicate with output of any tomograph and dispose an operative communication between the image processor and the tomograph output to receive the 3D tomograph generated image and provide an enhanced version of the same after performing the image processing steps in the image processor platform through the set of computer program based operators embodied in said image processor. The denoised and contrast enhanced tomographic image is then transferred to user’s display through the output interfacing means.The basic workflow of the tomograph generated image contrast enhancement and denoising is shown in the accompanying figure 1. The stages are described hereunder:Input: The input operator receives the tomographic images from the tomograph, which is then sectioned into (patch_size, patch_size, 64) sub-volumes of the pixels cuboids of the tomographic volume. These cuboids are fed to the image processor. The partitioning is done to standardize the input to the image processor. The invention can thus handle various resolutions and isn’t restricted to a particular resolution. The patch_size variable refers to the size of the patch considered when training the model. This value is obtained by experimentation. Image Processor:Image Processor embodies a parameterized tomographic image transformation which corresponds to a high level architecture of neural network model adapted for receiving the tomograph generated image in 3D format and provide the enhanced and denoised version of the same by sequentially analyzing non-overlapping cuboids of pixels of said 3D tomograph generated image.The 3D tomograph generated image is constructed by stacking several images in sequential order whereby multiple non-overlapping cuboids of pixels of size (patch_size, patch_size, 64) are iteratively fed to the image processor as training data of the model. The parameters of the network are tuned using Stochastic Gradient Descent. This is a generic process used in several non-linear machine learning problems. After a predefined number of iterations, which is sufficient to train the model, the feeding of the training data to the model is stopped. The trained model thus can be used for denoising and enhancing contrast of the output of the tomographic image. This training is performed only once and is important as the model studies the noise composition for a particular tomograph device. After the training is complete, the model can be directly applied to the output of the said tomograph without the requirement of further training.Output: The output operator is collates all the aforementioned sub-volumes and reconstructs the original volume which is of the same dimensions as the input and transfers it to the user's display.Testing:The model has been executed and tested on four different types of tomographic datasets, viz.Dataset 1: FIBSEM dataset released by the Computer Vision Laboratory at EPFL. The dataset represents a 5x5x5µm section taken from the CA1 hippocampus region of the brain, corresponding to a 1065x2048x1536 volume. The resolution of each voxel is approximately 5x5x5nm.Dataset 2: FIBSEM dataset procured from a TESCAN Lyra3 FIB + FESEM. The dataset consists of 8x5x20µm section of a 3D printed metallographic stainless steel sample prepared on a direct metal laser sintering machine. The resolution of each voxel is approximately 11x11x101nm.Dataset 3: Magnetic Resonance Imaging scan data obtained from a simulated anatomical model of the brain corrupted with an additional 10% noise. The resolution of each voxel is 1x1x1 mm.Dataset 4: Computed Tomography Scan data of a human pelvis obtained from a Siemens SOMATOM Sensation 16 device. The resolution of each voxel is approximately 1.36x2x1mm.A random sample of 10% of each of the dataset was used to train the invention and the results were validated on the remaining 90%. The invention has consistently given positive results in all the cases.Results:Image Denoising:There is no metric available to quantify the amount of noise reduction but visually one can observe that a significant amount has been achieved.As apparent from the figure 2, the image on the left (figure 2 (a)) contains a lot of noise in the form of grainy dots distributed throughout the image. Also the boundaries of the structures are hazy and blend into the background. The output on the right (figure 2 (b)) clearly reduces this noise by a significant factor. This grainy noise obfuscates the result and makes the identification of artifacts in the image tougher.The accompanying figure 3 (a) and 4(a) shows original images with following featuresNo Distinct Boundary between the Background and the artifactEasy to overlookCannot observe tiny artifacts due to noise.The accompanying figure 3(b) and 4(b) shows processed FIBSEM images with following features Well defined boundary between background and the artifact. Appears as a significant elementEnables the researcher to analyze the tiny structures of the sample which was previously not possible due to noise.In the image of figure 5, the textures and specific features are being brought about in the image on the left 5(a) to produce the image on the right 5(b). The image on the left is the tomographic output and the one on the right is its enhanced version produced by this invention. The details in the form of grain boundaries are clearly visible for the image on the right and they are occluded in the original image.In the image in figure 6, the image on the left 6(a) is a normal image from a tomogram of a biological sample taken by a FIBSEM device. The image on the right 6(b) is the enhanced image after being processed with the image denoising and contrast enhancement method of the present invention. The details in the form of structural boundaries are clearly visible in the enhanced image as and they are occluded in the original image.In the image in figure 7, the image on the left 7(a) is a normal image from a typical tomogram of the human pelvic region taken by a Computed Tomography (CT) device. The image on the right 7(b) is the enhanced image after being processed with the image denoising and contrast enhancement method of the present invention. The details in the form of organ boundaries and organ structure can be clearly understood in the enhanced image and they are not clearly visible due to noise in the original image.In the image in figure 8, the image on the left 8(a) is a normal image from a typical tomogram of the human brain taken by a Magnetic Resonance Imaging (MRI) device. The image on the right 8(b) is the enhanced image after being processed with the image denoising and contrast enhancement method of the present invention. The details in the form of brain membrane structure and skull boundary can be clearly seen in the enhanced image and they are not clearly visible due to noise in the original image.Contrast Enhancement:The contrast of an image can be quantified by using the following two rules:Michelson Contrast = RMS Contrast = Where M, N are the number of rows and columns of the image.The present invention is tested on an image set of 64 images and the contrast is found improving consistently. The results are shown in the following table and the graph of the accompanying figure 7. Original ProcessedMichelson Contrast 0.82 1.0RMS Contrast 30.86 46.78The image of figure 8 is a sample of the post processing which is done. The image on the left (a) is the raw data while the one on the right (b) is the invention’s processed output. Contrast enhancement as shown in the figure 8, gives the researcher the ability to quickly and accurately segment the areas of interest from the background.Plane Invariance:The invention once trained on a tomograph is agnostic to the plane of viewing of the image. The invention trained on images from one view of the volume can be directly applied to images in another plane without the need to retrain the invention.As an illustration of this property, figure 10 shows (a) the normal MRI scan image viewed in the sagittal plane and (b) the contrast enhanced and denoised image obtained by treating (a) with the present invention which was trained on MRI scan images viewed on the axial planeMagnification Invariance:The invention once trained on a tomograph is agnostic to the magnification of the tomograph. The invention trained on images with a particular magnification can be applied to images with different magnifications without the need to retrain the invention.As an illustration of this property, figure 11 shows (a) the normal FIBSEM scan image magnified by 150% the original magnification and (b) the contrast enhanced and denoised image obtained by treating (a) with the present invention which was trained on FIBSEM scan images with the original magnification.As an illustration of this property, figure 12 shows (a) the normal MRI scan image magnified by 150% the original magnification and (b) the contrast enhanced and denoised image obtained by treating (a) with the present invention which was trained on MRI scan images with the original magnification.Rotation Invariance:The invention once trained on a tomograph is robust to rotation of the specimen. It can account for rotations induced as a result of human operations without the need to train it again. As an illustration of this property, figure 13 shows (a) the normal FIBSEM scan image rotated by 45 degrees in the anticlockwise direction and (b) the contrast enhanced and denoised image of obtained by treating (a) with the present invention which was trained on FIBSEM scan images without rotation.As an illustration of this property, figure 14 shows (a) the normal MRI scan image rotated by 45 degrees in the anticlockwise direction and (b) the contrast enhanced and denoised image of obtained by treating (a) with the present invention which was trained on MRI scan images without rotation. Time Analysis:In a preferred embodiment, the present system can process 64 frames is 5.893 seconds on an NVIDIA Tesla K40 GPU which amounts to an average of 0.092 seconds per frame, hence this enhancement can happen at 11 fps. This is faster than any prevalent methods by a significantly large margin. This faster operation also enables the present suitable for application in real-time processing of the FIBSEM images facilitating the better understanding of different microscopic biological processes, transformations which happen through time and open up a new avenue for research.

Claims

Claims:WE CLAIM:

1. A system for denoising and enhancing contrast of a tomograph generated image comprisingan input interfacing means adapted to communicate with output of any tomograph;an image processor having an operative communication with the tomograph output through said input interfacing means to receive the tomograph generated image, said image processor embodies parameter controlled image operators for processing of the tomograph generated image and providing an enhanced and denoised version of the same; andan output interfacing means for transferring the denoised and the enhanced tomographic image from the image processor to user’s display.

2. The system as claimed in claim 1, wherein the parameter controlled image operators embodied in the image processor corresponds to high level architecture of neural network model adapted for receiving the tomographic volume in 3D format and provide the enhanced and denoised version of the same by sequentially analyzing non-overlapping cuboids of pixels of said 3D tomographic volume.

3. The system as claimed in claim 1 or 2, wherein the 3D tomograph generated image volume includes multiple tomographic images stacked in sequential order whereby selectively plurality of the non-overlapping cuboids of the pixels of the tomographic images are iteratively fed to the neural network model as training data to train;4. The system as claimed in anyone of claims 1 to 3, wherein the embodied parameter controlled image operators corresponding to the neural network model are tuned using Stochastic Gradient Descent during the training.

5. The system as claimed in anyone of claims 1 to 4, wherein the input operator is adapted to section the 3D tomograph generated image volume into the sub-volumes of non-overlapping cuboids of pixels corresponding to the tomographic image having size of (patch_size, patch_size, 64).

6. The system as claimed in anyone of the claims 1 to 5, is adapted to denoise and enhance contrast of the tomographic images in real time for facilitating better understanding of different microscopic biological processes, transformations which happen through time.

7. A method for denoising and enhancing contrast of tomograph generated image generated image by involving the system as claimed in anyone of the claims 1 to 6 comprising the steps ofinvolving the input interfacing means for communicating with output of any tomograph, receiving the tomograph generated images and segmenting them into sub-volumes to feed in the connected image processor;processing the tomograph generated images in the image processor by involving the parameter controlled image operators embodied in said image processor to denoise and enhance contrast of the tomographic images;reconstructing the original volume from the sub-volumes and transferring the denoised and the enhanced tomographic image from the image processor to user’s display by the output interfacing means.

8. The method as claimed in claim 7, wherein processing of the tomograph generated images is rotation invariant, wherein the method once trained can work on tomographic image samples which are rotated without need to retrain it again.

9. The method as claimed in claim 7 or 8, wherein processing of the tomograph generated images is magnification invariant, wherein the method once trained can be used to enhance images of the sample at different magnifications without the need to retrain it again.

10. The method as claimed in anyone of the claims 8 to 9, wherein processing of the tomograph generated images is plane invariant, wherein the method once trained can be used to enhance images of the sample viewed from a different perpendicular plane without the need to retrain it again.

11. The method as claimed in anyone of the claims 8 to 10, wherein processing of the tomograph generated images is valid for all devices which are used for tomography, wherein multiple images stacked in a sequential order which represent sections of a three dimensional object.Dated this the 14th day of November, 2018 Anjan Sen (Applicants Agent) IN / PA-199 , Description:FIELD OF THE INVENTION:The present invention relates to the improvement of contrast and denoising of tomographic imagery generated by destructive or non-destructive tomographic methods. More specifically, the present invention is directed to develop a method and system cooperative to any tomographic imaging device for improving the contrast or denoising of the output images without physically modifying the device.BACKGROUND OF THE INVENTION:Tomogram / Tomographic volume is basically the generated 3D volume of multiple stacked images in sequential order by the tomogram as a result of the tomography procedure. The tomographic image refers to one image from the multiple stacked images in sequential order. Tomograph refers to the device used to perform tomographyTomographic techniques are used to understand the three dimensional structural details of samples obtained from biotic or abiotic sources. In a tomogram, as the order of magnification increases, there is an increase in the noise in the generated image which makes it necessary to use denoising and contrast enhancement techniques on the tomograph generated image. Improving the contrast and denoising of the tomograph generated image will allow researchers to study intricate molecular and structural details at higher magnifications the likes of which haven't been possible due to the noise produced by the tomograph. Recently some techniques for processing and improvement of the microscopic images have been reported in the art, e.g,CN 104458683 discloses deep cell super-resolution imaging methods, deep cell super-resolution imaging optical system and prism sheet device.US 7173261 B2 discloses an image noise removing method for FIB / SEM complex apparatus which is physically altered by the addition of three units, viz. a unit that acquires plural SEM images and stores the SEM images; a unit that extracts an FIB blanking period noise area in one SEM image; a unit that slices image information of the area from other SEM images, and a unit that replaces image information in the FIB blanking period noise area with the sliced image information.US 8090183 B2 discloses a pattern noise correction for pseudo projections which is also physically altered by the installation of a rotating micro-capillary tube and a refraction matching gel in which the sample is placed and is rotated to obtain different pseudo-projectionsUS 10096109 B1 discloses a method to denoise medical images requiring to acquire plural images with different specifications for contrast in each of the images.US 20070280519 A1 discloses a method to enhance a Computed Tomography image by analysing a plurality of CT images and combining them on the basis of their intensity. This method is applicable only to CT images of the human body.US 6987831 B2 discloses a method to improve Computed Tomography imagery of breast by modifying the apparatus used to capture the image.CN 103186888 A discloses a domain specific mathematical model for removing the noise in Computed Tomography images.The majority of the research in this field is thus focussed on (i) physically modifying the tomograph itself to improve the image capturing ability; OR(ii) developing a mathematical model which is domain specific and cannot be applied across domains; OR(iii) use convolutional neural networks which aren't Plane / Rotation / Magnification Invariant.This hinders the applicability of the said research as it heavily depends on the specifications of the particular device or the imaging procedure. Thus, there has been a need for developing a technique for improving the contrast and denoising of the tomograph generated images without modifying the device itself of having specific domain knowledge of the imaging procedure.OBJECTS OF THE INVENTION:It is thus the basic object of the present invention is to develop an image denoising and contrast enhancement system and method which would be adapted to cooperate with any tomograph for improving the contrast and denoising the tomographic images.Another object of the present invention is to develop a tomogram denoising and contrast enhancement (processing) system and a method which would be adapted to enhance the contrast and to denoise the tomograph generated images without physically modifying the tomograph itself. Another object of the present invention is to develop a tomogram denoising and contrast enhancement system and method which would be adapted to enhance the contrast and denoise the tomograph generated images in real time with the generation of the images facilitating the users to study unstable biological samples and also the processes which happen to them with time. Another object of the present invention is to develop a tomographic image denoising and contrast enhancement system and method which would be adapted to cooperate with any tomograph and provide a live-stream of the enhanced imagery. Another object of the present invention is to develop a tomographic image denoising and contrast enhancement system and method which would be adapted to enhance the contrast and denoise the tomograph generated images independent of model and resolution of the device used enables its use by a wide pool of users. Another object of the present invention is to develop a tomographic image denoising and contrast enhancement system and method which would be adapted to denoise the tomograph generated images by involving just a single view of the sample taken from a standard tomograph in one pass using standard procedures without need of expensive equipment.Another object of the present invention is to develop a tomographic image denoising and contrast enhancement system and method which would be adapted to enhance the contrast and denoise the tomogram generated in different perpendicular view directions.Another object of the present invention is to develop a tomographic image denoising and contrast enhancement system and methods which would be adapted to enhance the contrast and denoise the tomogram generated at different magnifications.Another object of the present invention is to develop a tomographic image denoising and contrast enhancement system and methods which would be adapted to enhance the contrast and denoise the tomogram generated with different rotations.SUMMARY OF THE INVENTION:Thus according to the basic aspect of the present invention there is provided a system for denoising and enhancing contrast of a tomograph generated image comprisingIn an embodiment of the present invention, the parameter controlled image operators embodied in the image processor corresponds to high level architecture of neural network model adapted for receiving the tomographic volume in 3D format and provide the enhanced and denoised version of the same by sequentially analyzing non-overlapping cuboids of pixels of said 3D tomographic volume.In an embodiment of the present invention, the 3D tomograph generated image volume includes multiple tomographic images stacked in sequential order whereby selectively plurality of the non-overlapping cuboids of the pixels of the tomographic images are iteratively fed to the neural network model as training data to train;In an embodiment of the present invention, the embodied parameter controlled image operators corresponding to the neural network model are tuned using Stochastic Gradient Descent during the training.In an embodiment of the present invention, the input operator is adapted to section the 3D tomograph generated image volume into the sub-volumes of non-overlapping cuboids of pixels corresponding to the tomographic image having size of (patch_size, patch_size, 64).The present system is adapted to denoise and enhance contrast of the tomographic images in real time for facilitating better understanding of different microscopic biological processes, transformations which happen through time.According to another aspect in the present invention there is provided a method for denoising and enhancing contrast of tomograph generated image generated image by involving the above system comprising the steps ofIn the above method, processing of the tomograph generated images is rotation invariant, wherein the method once trained can work on tomographic image samples which are rotated without need to retrain it again.In the above method, processing of the tomograph generated images is magnification invariant, wherein the method once trained can be used to enhance images of the sample at different magnifications without the need to retrain it again.In the above method, processing of the tomograph generated images is plane invariant, wherein the method once trained can be used to enhance images of the sample viewed from a different perpendicular plane without the need to retrain it again.In the above method, processing of the tomograph generated images is valid for all devices which are used for tomography, wherein multiple images stacked in a sequential order which represent sections of a three dimensional object.