An enhanced method for denoising and restoring microscopic images

Abstract

The present invention discloses an enhanced method for denoising and restoring microscopic images, which includes: 1) performing noise superposition processing on microscopic images with high signal-to-noise ratio to obtain microscopic images with low signal-to-noise ratio; 2) taking paired high- and low-signal-to-noise ratio microscopic images as training data pairs and inputting them into an image denoising neural network in pairs for training to obtain a trained image denoising neural network model; 3) inputting the to-be-tested noise images, the standard Gaussian noise image, and the standard salt-and-pepper noise image into an image discrimination system respectively; 4) using an image algebraic operation method to process the to-be-tested noise images that have been evaluated for noise level and classified in step 3) to obtain preprocessed microscopic images; 5) taking the preprocessed microscopic images as inputs and calling the trained image denoising neural network model to obtain denoised and restored microscopic images, thereby realizing the restoration of microscopic images; 6) outputting and displaying the denoised and restored microscopic images to obtain corresponding denoised and restored microscopic images, so as to achieve the purpose of denoising and restoring microscopic images.

Description

Technical Field

[0001] The invention relates to the field of optical microscopic imaging and digital image processing, and in particular to an enhanced microscopic image noise reduction and restoration method. Background Art

[0002] Fluorescence microscopy is an important tool to promote cell research and is mostly used to observe and record the true situation of samples. During the imaging process of fluorescence microscopy, longer exposure time and stronger exposure intensity help to obtain images with higher signal-to-noise ratio. However, strong exposure may cause sample bleaching or reduced cell activity or even death. In the case of weak exposure, the image signal-to-noise ratio will drop significantly, and the obtained image contains a lot of noise. With the development of technology, it is more feasible to remove noise and obtain high-quality images through computer methods. Among the traditional methods, traditional noise reduction methods such as mean filtering, adaptive Wiener filtering, and statistical filtering are based on image noise reduction in spatial domain or transform domain, and have been widely used, but the loss of image details or edge information is often more serious. With the development of deep learning methods, the application scope of image noise reduction through deep learning methods is constantly increasing, which can not only cover the advantages of traditional methods, but also build on this to achieve clearer images, stronger self-expandability, efficient and unsupervised image noise reduction. In the invention patent "Grayscale image denoising method based on dilated convolution and automatic encoding and decoding neural network" with publication number 109671026A (authorization announcement number 109671026B), a grayscale image denoising method based on dilated convolution and automatic encoding and decoding neural network is disclosed. The grayscale image denoising method has a relatively simple structure. After training, the parameters occupy fewer bytes, which is suitable for some non-embedded small systems to enhance the visual effect of images and remove image noise. However, this method has a large difference in the processing effect of different types of noise, especially for microscopic imaging, the processing effect of more complex mixed noise is not ideal. Summary of the invention

[0003] In view of the fact that the processing effects of different types of noise in the current microscopic image denoising and restoration are quite different, especially the processing effect of more complex mixed noise is not ideal, the present invention proposes an enhanced microscopic image denoising and restoration method for complex noise conditions. The method proposes to classify images through an image discrimination system, so as to use image algebraic operations for preprocessing according to different situations. First, pairs of high and low signal-to-noise ratio microscopic images are used as data sets and input into the image denoising neural network for training. Secondly, the noise image to be tested is preprocessed through the image discrimination system and input into the trained image denoising neural network model. Finally, the result after denoising and restoration is output, so as to achieve the purpose of microscopic image denoising and restoration.

[0004] An enhanced microscopic image noise reduction and restoration method, characterized by comprising the following steps:

[0005] 1) Perform noise superposition processing on the high signal-to-noise ratio microscopic image to obtain a low signal-to-noise ratio microscopic image;

[0006] The high signal-to-noise ratio microscopic image refers to an image with a signal-to-noise ratio value of more than 25 dB. The low signal-to-noise ratio microscopic image refers to an image with a high signal-to-noise ratio microscopic image to which Gaussian noise, salt and pepper noise, or a mixture of Gaussian noise and salt and pepper noise is added.

[0007] 2) inputting the high and low signal-to-noise ratio microscopic images in step 1) into the image denoising neural network as training data pairs for training, thereby obtaining a trained image denoising neural network model;

[0008] Preferably, the structure of the image denoising neural network is: four convolutional layers, two pooling layers and three fully connected layers, and the size of all convolution kernels in the network is 5*5; the convolutional layers serve to extract paired high and low signal-to-noise ratio microscopic images as the input of the image denoising neural network, the pooling layers serve to reduce the dimension of the features extracted by the convolutional layers, compress the number of data and parameters, reduce overfitting, and improve the fault tolerance of the model, and the fully connected layers serve to construct a mapping relationship between paired high and low signal-to-noise ratio microscopic images.

[0009] 3) Input the noise image to be tested, the standard Gaussian noise image and the standard salt and pepper noise image into the image discrimination system respectively;

[0010] The standard Gaussian noise image is a picture obtained by adding Gaussian noise to a black background picture.

[0011] Preferably, Gaussian noise with a mean of 0 and a variance of 0.5 is added to the black background image to obtain a standard Gaussian noise image.

[0012] The standard salt and pepper noise image is an image obtained by adding salt and pepper noise to a black background image.

[0013] Preferably, salt and pepper noise with a noise density of 0.5 is added to the black background image to obtain a standard salt and pepper noise image.

[0014] The image discrimination system refers to a system that evaluates and classifies the noise level of the noise image to be tested after calculating the image similarity.

[0015] The image similarity calculation refers to the calculation using cosine similarity. In the cosine similarity calculation process, the noise image to be tested, the standard Gaussian noise image and the standard salt and pepper noise image are represented as vectors respectively, and the cosine distance between the noise image to be tested and the standard Gaussian noise image vector, and the cosine distance between the noise image to be tested and the standard salt and pepper noise image vector are calculated to characterize the similarity of the two images, that is, the closer the cosine similarity is to 1, the higher the similarity of the two images.

[0016] Preferably, when the image similarity calculation result between the noise image to be tested and the standard Gaussian noise image is greater than the image similarity calculation result between the noise image to be tested and the standard salt and pepper noise image, the noise image to be tested is considered to be similar to Gaussian noise; when the image similarity calculation result between the noise image to be tested and the standard salt and pepper noise image is greater than the image similarity calculation result between the noise image to be tested and the standard Gaussian noise image, the noise image to be tested is considered to be similar to salt and pepper noise; the image similarity calculation is performed on the noise image to be tested, the standard Gaussian noise image and the standard salt and pepper noise image respectively, and when the difference in the image similarities obtained by the two calculations is less than or equal to 0.01, the noise image to be tested is considered to be similar to both Gaussian noise and salt and pepper noise.

[0017] 4) using an image algebraic algorithm to process the noise image to be tested after the noise level evaluation and classification in step 3) to obtain a preprocessed microscopic image;

[0018] The image algebraic algorithm includes two parts: image subtraction algorithm and image addition algorithm. When the noise image to be tested is similar to Gaussian noise, the image subtraction algorithm is used to preprocess the noise image to be tested; when the noise image to be tested is similar to salt and pepper noise, the image addition algorithm is used to preprocess the noise image to be tested; when the noise image to be tested is similar to both Gaussian noise and salt and pepper noise, the image addition algorithm is first used to preprocess the noise image to be tested, and then the image subtraction algorithm is used to preprocess the noise image to be tested;

[0019] The image subtraction algorithm refers to a method of subtracting the pixel values ​​of corresponding points of the input noise image to be tested and the standard mixed noise image, and then using the image obtained by subtracting the pixel values ​​of the corresponding points as the preprocessed image.

[0020] The standard mixed noise image refers to an image obtained by adding Gaussian noise and salt and pepper noise to a black background image.

[0021] Preferably, Gaussian noise with a mean of 0 and a variance of 0.5 and salt and pepper noise with a noise density of 0.5 are added to the black background image to obtain a standard mixed noise image.

[0022] The image addition algorithm refers to a method of superimposing and averaging multiple groups of noise images to be tested taken under different laser intensity conditions, thereby reducing the noise level of the noise images to be tested, and then using the images with reduced noise levels as preprocessed microscopic images.

[0023] The number of the multiple groups of noise images to be tested is at least 4 groups, and the multiple groups of noise images to be tested need to be obtained by photographing the same sample under different laser intensity conditions.

[0024] 5) using the preprocessed microscopic image in step 4) as input, calling the image denoising neural network model trained in step 2), obtaining the microscopic image after denoising and restoration, and realizing restoration of the microscopic image;

[0025] 6) Outputting and displaying the microscopic image after noise reduction and restoration;

[0026] Compared with the prior art, the present invention has the following beneficial technical effects:

[0027] For current microscopic images, in traditional image restoration methods, the restoration effect is not ideal for images seriously polluted by Gaussian noise, and the loss of details or edge information is serious, and the processing effect is even worse for mixed noise. The present invention uses image algebraic operation methods to optimize data for preprocessing, which can not only achieve high-quality restoration of images seriously polluted by Gaussian noise, retain their details or edge information, but also optimize images polluted by mixed noise.

[0028] The invention has a simple structure, is easy to implement, has a wide application range and is highly universal. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 The flowchart of the enhanced microscopic image denoising and restoration method of the present invention is as follows: first, a high signal-to-noise ratio microscopic image is subjected to noise superposition using MATLAB software to obtain a low signal-to-noise ratio microscopic image, and pairs of high and low signal-to-noise ratio microscopic images are used as data sets. Secondly, the paired high and low signal-to-noise ratio microscopic images are input into an image denoising neural network for training to obtain a trained image denoising neural network model. Afterwards, the noise image to be tested is passed through an image discrimination system and preprocessed using an image algebraic operation algorithm. Finally, the obtained image is used as the input of the image denoising neural network, and the result after denoising and restoration is output, thereby realizing the denoising and restoration of the enhanced microscopic image of the present invention.

[0030] Figure 2The present invention is a flowchart of an image discrimination system and image preprocessing: the flowchart includes an image discrimination system and an image preprocessing part. For the input noise image to be tested, the image similarity is first calculated with the standard Gaussian noise image and the standard salt and pepper noise image using cosine similarity. When the image similarity calculation result between the noise image to be tested and the standard Gaussian noise image is greater than the image similarity calculation result between the noise image to be tested and the standard salt and pepper noise image, the noise image to be tested is preprocessed using an image subtraction algorithm; when the image similarity calculation result between the noise image to be tested and the standard salt and pepper noise image is greater than the image similarity calculation result between the noise image to be tested and the standard Gaussian noise image, the noise image to be tested is preprocessed using an image addition algorithm; the noise image to be tested is respectively calculated with the standard Gaussian noise image and the standard salt and pepper noise image for image similarity, and when the image similarity difference obtained by the two calculations is less than or equal to 0.01, the noise image to be tested is first preprocessed using an image addition algorithm, and then preprocessed using an image subtraction algorithm, and finally the processed image is output. In this embodiment, the image algebraic algorithm can be obtained by calculation and processing by software such as MATLAB.

[0031] FIG3 is an example of a training set image produced by the present invention: Figure 3.1 is a high signal-to-noise ratio microscopic image of the training set example, Figure 3.2 Low signal-to-noise ratio microscopic images of training set examples.

[0032] FIG4 is a comparison diagram of the noise reduction effect when the present invention does not use the image discrimination system and when the image is preprocessed: Figure 4.1 is the noise image 1 to be tested, Figure 4.2 The effect of denoising without using image discrimination system and image preprocessing Figure 1 ; Figure 4.3 is the noise image 2 to be tested, Figure 4.4 The effect of denoising without using image discrimination system and image preprocessing Figure 2

[0033] FIG5 is a diagram showing the noise reduction effect when the present invention uses an image recognition system and image preprocessing: Figure 5.1 The effect of noise reduction using image discrimination system and image preprocessing Figure 1 , Figure 5.2 The effect of noise reduction using image discrimination system and image preprocessing Figure 2 .

[0034] FIG6 is a diagram showing parameter changes of the image denoising neural network used for training in the present invention: Figure 6.1 6.2 is the change curve of the mean square error, and 6.3 is the change curve of the absolute average error. DETAILED DESCRIPTION

[0035] The present invention will be described in detail below in conjunction with the accompanying drawings, but the present invention is not limited thereto.

[0036] First, we obtain low signal-to-noise ratio microscopic images through high signal-to-noise ratio microscopic images to create a data set, and obtain 30 pairs of high and low signal-to-noise ratio microscopic images: Figure 3 shows an example of a training set image: Figure 3.1 is a high signal-to-noise ratio microscopic image of the training set example, with a signal-to-noise ratio of 25.27dB; Figure 3.2 is a low signal-to-noise ratio microscopic image of the training set example. Figure 3.1 By adding Gaussian noise with a mean of 0 and a variance of 0.5 and salt and pepper noise with a noise density of 0.5 in turn, the signal-to-noise ratio is 7.35dB.

[0037] Then, the noise image to be tested without using the image discrimination system and image preprocessing is denoised and the results are output: Figure 4.1 The noise image to be tested is 1, and the image signal-to-noise ratio result is 7.37dB. The noise image to be tested 1 is input into the image denoising neural network for prediction and output, and the following is obtained: Figure 4.2 The noise reduction effect diagram shown has a signal-to-noise ratio of 21.05dB. Figure 4.3 The noise image 2 to be tested has a signal-to-noise ratio of 7.87 dB. The noise image 2 to be tested is input into the image denoising neural network for prediction and output, and the result is as follows: Figure 4.4 The noise reduction effect diagram shown has a signal-to-noise ratio of 19.99dB.

[0038] Then, the noise image to be tested using the image discrimination system and image preprocessing is denoised and the results are output: Figure 5.1 The effect of noise reduction using image discrimination system and image preprocessing Figure 1 ,use Figure 4.1 The noise image to be tested in the image 1 is used as the noise image to be tested, and its signal-to-noise ratio is 7.37dB. After the image recognition system, it is compared with the standard Gaussian noise image, and the image similarity value calculated is 0.94; compared with the standard salt and pepper noise image, the image similarity value calculated is 0.83, and it is considered that the noise image to be tested is similar to Gaussian noise. After the image recognition system determines that the image subtraction algorithm is used for preprocessing, it is then input into the image denoising neural network for prediction and output, and the image is obtained. Figure 5.1 , its signal-to-noise ratio is 23.12dB. Figure 5.2 The effect of noise reduction using image discrimination system and image preprocessing Figure 2 ,use Figure 4.3The noise image to be tested in the image 1 is used as the noise image to be tested, and its signal-to-noise ratio is 7.87dB. After the image recognition system, it is compared with the standard salt and pepper noise image, and the image similarity value calculated is 0.99; compared with the standard Gaussian noise image, the image similarity value calculated is 0.84, and it is considered that the noise image to be tested is similar to the salt and pepper noise. The image recognition system determines that the image addition algorithm is used for preprocessing, and 4 groups of noise images to be tested are taken when the laser intensity is 30μW, 80μW, 120μW, and 160μW, respectively. Figure 4.3 The noise images to be tested are superimposed and averaged, and the obtained image is input into the denoising neural network for prediction and output. Figure 5.2 , its signal-to-noise ratio is 21.56dB.

[0039] In this embodiment, the Python version selected by the software is 3.7, the TensorFlow version is 1.14.0, and the Keras version is 2.2.5.

[0040] As shown in FIG6 , this is a diagram showing the parameter changes of the image denoising neural network used for training in the present invention: Figure 6.1 is the loss function change curve, Figure 6.2 is the curve of the change of mean square error, Figure 6.3 The graph is a curve of the change of the absolute average error. The number of training iterations is 350, the image segmentation size is 128, and the training step is 25. As the training progresses, the change curves of the loss function, mean square error, and absolute average error tend to converge, where the loss function value converges to -3.35, the mean square error value converges to 0.005, and the absolute average error converges to 0.025.

[0041] The denoising effect of the microscopic image after using the image discrimination system and image preprocessing is greatly improved compared with the microscopic image without using the image discrimination system and image preprocessing, achieving high-quality restoration while retaining details and edge information.

[0042] Finally, it should be noted that the above implementation methods are only used to illustrate the technical solution of the patent rather than to limit it. Ordinary technicians in this field can make several modifications and improvements without departing from the principle of this patent, which should also be regarded as the scope of protection of this patent.

Claims

1. An enhanced method for denoising and restoring microscopic images, characterized in that, the method comprises the following steps: 1) Perform noise superposition processing on a high signal-to-noise ratio microscopic image to obtain a low signal-to-noise ratio microscopic image; the high signal-to-noise ratio microscopic image refers to an image with a signal-to-noise ratio value above 25 dB; the low signal-to-noise ratio microscopic image refers to an image obtained by adding Gaussian noise or salt-and-pepper noise or a mixed noise of Gaussian noise and salt-and-pepper noise to the high signal-to-noise ratio microscopic image on this basis; 2) Use the paired high and low signal-to-noise ratio microscopic images in step 1) as training data pairs and input them into an image denoising neural network in pairs for training to obtain a trained image denoising neural network model; 3) Input the test noise image and the standard Gaussian noise image and the standard salt-and-pepper noise image into an image discrimination system respectively. After calculating the image similarity of the test noise image, perform noise level evaluation and classification: when the calculation result of the image similarity between the test noise image and the standard Gaussian noise image is greater than the calculation result of the image similarity between the test noise image and the standard salt-and-pepper noise image, it is considered that the test noise image is similar to Gaussian noise, and use the image subtraction algorithm to preprocess the test noise image; when the calculation result of the image similarity between the test noise image and the standard salt-and-pepper noise image is greater than the calculation result of the image similarity between the test noise image and the standard Gaussian noise image, it is considered that the test noise image is similar to salt-and-pepper noise, and use the image addition algorithm to preprocess the test noise image; Calculate the image similarity between the test noise image and the standard Gaussian noise image and the standard salt-and-pepper noise image respectively. When the difference between the two calculated image similarity values is less than or equal to 0.01, it is considered that the test noise image is similar to both Gaussian noise and salt-and-pepper noise. First, use the image addition algorithm and then use the image subtraction algorithm to preprocess the test noise image to obtain a preprocessed microscopic image; 4) Use the preprocessed microscopic image obtained in step 3) as the input, and call the trained image denoising neural network model in step 2) to obtain a denoised and restored microscopic image, realizing the restoration of the microscopic image.

2. The enhanced method for denoising and restoring microscopic images according to claim 1, characterized in that, the structure of the image denoising neural network is: four convolutional layers, two pooling layers and three fully connected layers, and the size of all convolutional kernels in the network is 5×5.

3. The enhanced method for denoising and restoring microscopic images according to claim 1, characterized in that, in step 3), the standard Gaussian noise image is obtained by adding Gaussian noise with a mean of 0 and a variance of 0.5 to a black background picture.

4. The enhanced method for denoising and restoring microscopic images according to claim 1, characterized in that, in step 3), the standard salt-and-pepper noise image is obtained by adding salt-and-pepper noise with a noise density of 0.5 to a black background picture.

5. The enhanced method for denoising and restoring microscopic images according to claim 1, characterized in that, the image subtraction algorithm refers to subtracting the pixel values of the corresponding points of the input test noise image and the standard mixed noise image.

6. The enhanced microscopic image denoising and restoration method according to claim 5, characterized in that, the standard mixed noise image refers to an image obtained by adding Gaussian noise with a mean of 0 and a variance of 0.5 and salt-and-pepper noise with a noise density of 0.5 to a black background picture.

7. The enhanced microscopic image denoising and restoration method according to claim 1, characterized in that, the image addition algorithm refers to taking the average after superimposing a group of test noise images captured under different laser intensity conditions.

Images