A method to improve the quality of digital holographic reconstruction images

Abstract

This invention discloses a method for improving the quality of digital holographic reconstruction images, comprising the following steps: acquiring digital holograms to be processed, including digital holograms containing speckle noise and digital holograms without speckle noise, and obtaining the dataset required for neural network training; denoising the digital holograms using a deep learning algorithm and a neural network method with image denoising capabilities; performing preset reconstruction processing on the denoised digital holograms to obtain a reconstructed image of the digital hologram; and denoising the reconstructed image using a preset image processing algorithm to obtain the final result. This invention, based on a combination of deep learning and multi-sub-hologram reconstruction, reduces the impact of speckle noise on the quality of holographic reconstruction images, overcoming the shortcomings of existing technologies and improving the quality of reconstructed images. Experiments demonstrate that this invention has a better effect in removing speckle noise from digital holograms, and its advantages are significant compared to traditional algorithms.

Description

Technical Field

[0001] This invention discloses a method for improving the quality of digital holographic reconstruction images, relating to the fields of deep learning and image processing technology. Background Technology

[0002] In digital holography, when a laser source is used to digitally record a hologram, the high coherence of the laser source and the rough surface of the object cause reflections that lead to coherent superposition, resulting in speckle noise in the obtained digital hologram and degrading the quality of the numerically reconstructed image. Therefore, a research focus in this field is on effectively removing noise while preserving as much information as possible from the digital holographic image, thereby improving the quality of the reconstructed image.

[0003] Currently, there are three common methods for eliminating speckle noise in digital holograms: The first method uses partially coherent or low-coherence light sources. Replacing laser sources with partially coherent ones can reduce the impact of speckle noise, but this requires high-brightness partially coherent light sources; otherwise, image quality will be reduced. The second method utilizes digital image processing techniques, such as traditional filters, resampling masks, and BM3D, which can effectively remove speckle noise, but this destroys image details, degrading image quality. The third method is based on deep learning denoising. In 2017, K. Zhang et al. proposed a denoising convolutional neural network using residual learning and batch normalization to reduce general noise. In 2018, WONSEOK JEON et al. used a multi-scale convolutional neural network to achieve speckle denoising in digital holographic images. In 2021, Z. J. Shen et al. used a dual-path parallel convolutional neural network to remove speckle noise in different types of OCT images. In 2022, Q. Fang et al. used a conditional generative adversarial network based on deep learning to remove speckle noise. Compared with the two traditional methods mentioned above, deep learning-based methods do not require complex and rigorous experiments. Instead, they train the neural network based on a pre-acquired dataset and directly map the denoised result. With the breakthrough in GPU computing speed, the learning speed and performance have been improved, enabling real-time applications. Summary of the Invention

[0004] Based on the advantages and disadvantages of the above methods, this invention proposes a method to improve the quality of digital holographic reconstruction images. By combining deep learning with the reconstruction of multiple sub-holograms, the method effectively solves the problem of speckle noise affecting the quality of holographic reconstruction images and improves the quality of the reconstructed images. Experiments have proven that this invention is effective.

[0005] The technical solution of the present invention is as follows:

[0006] A method for improving the quality of digital holographic reconstruction images includes the following steps:

[0007] (1). Obtain the digital holograms to be processed, which are respectively the digital hologram with speckle noise and the digital hologram without speckle noise, and obtain the data set required for neural network training;

[0008] (2). Use the deep learning algorithm to denoise the digital hologram according to the neural network method with image denoising function;

[0009] (3). Perform a preset reconstruction process on the denoised digital hologram to obtain the reconstructed image of the digital hologram;

[0010] (4). Denoise the reconstructed image according to the preset image processing algorithm to obtain the final result.

[0011] Preferably, the specific steps of the above step (1) are as follows:

[0012] (1-1). Use matlab to simulate the process of off-axis digital hologram to obtain an off-axis digital hologram of size L×W;

[0013] (1-2). Add Rayleigh noise to the digital hologram through the raylrnd function of matlab to simulate the real situation; the probability density function of Rayleigh noise is as shown in formula (2) when z≥a and 0 when z<a.

[0014]

[0015]

[0016] Where U[0,1] in formula (1) is the uniform distribution function, and both a and b are constants.

[0017] (1-3). Multiply the digital hologram and the noisy hologram with added Rayleigh noise by a moving binary mask window to construct N sub-holograms to obtain a series of different sub-holograms, as shown in formula (3), where W(x,y) represents the hologram, x and y represent the coordinates on the hologram, and I K (x,y) represents k sub-holograms.

[0018] W(x,y) = {I1(x,y), I2(x,y), I3(x,y), …, I k (x,y)}(3)

[0019] The speckle distributions of the sub-holograms are uncorrelated, and a data set of noisy holograms and noise-free holograms available for neural network training is obtained.

[0020] Preferably, the specific steps of the above step (2) are:

[0021] (2-1) Perform binary masking on the noisy hologram to be tested to obtain N noisy sub-holograms;

[0022] (2-2) Based on the trained neural network with image denoising function, the noisy sub-holograms are denoised to obtain N preliminary denoised sub-holograms.

[0023] Preferably, the structure of the neural network with image denoising function is an Encoder-Decoder structure. The Encoder part uses a Resblock module to replace the original convolution module, which improves the feature extraction capability of the network. The Decoder part uses bilinear interpolation to restore the resolution of the original image. When performing feature extraction and upsampling, a skip connection operation is used to fuse deep features with shallow features.

[0024] The neural network employs a residual structure during both upsampling and downsampling, which improves the network's feature extraction capability and denoising effect.

[0025] The neural network was trained using the Adam optimizer, with MSE loss function, a learning rate of 0.001, a batch size of 20, and an epoch of 35. All code was implemented using Python and PyTorch.

[0026] Preferably, the preset reproduction processing method in step (3) above is to perform wavefront reconstruction of simulated digital holography on MATLAB on the sub-holograms that have been denoised by the denoising neural network. In these sub-holograms, the distribution of target information is unchanged on the reconstructed image plane, while the distribution of speckle is different.

[0027] Preferably, the specific steps of step (4) above are as follows:

[0028] After obtaining N different reconstructed images, the pixels located at the same position in each reconstructed sub-image are extracted. When extracting the N pixel values, the gray values ​​of these N pixels are averaged to obtain the final pixel gray value, as shown in formula (4):

[0029]

[0030] Where x and y represent the coordinates on the hologram, I n (x,y) represents the gray value of the nth sub-hologram at position x,y. The final image is generated by repeating the above operation for each pixel position.

[0031] The beneficial effects of this invention are:

[0032] The advantage of this invention over traditional methods lies in its effective combination of the strengths of deep learning and image processing methods. Currently, methods for speckle denoising of digital holograms based on deep learning are mainly limited by the lack of datasets for neural network training and the absence of open-source training datasets. Therefore, creating a usable training dataset is crucial. Furthermore, using image processing methods to remove speckle noise from digital holograms can destroy image details, leading to a decrease in image quality. This invention effectively solves the problems of the above two methods. By segmenting the hologram into blocks, a large number of sub-holograms are obtained, effectively solving the problem of dataset acquisition. A denoising neural network is trained using this dataset, effectively removing most of the speckle noise. Subsequently, wavefront reconstruction of simulated digital holography is performed using the obtained sub-holograms, resulting in a large number of different reconstructed images. Pixels located at the same position in each reconstructed sub-image are extracted. The gray values ​​of these pixels are averaged to obtain the final pixel gray value. The final image is generated by repeating the above operation for each pixel position. Experimental analysis shows that this method can reduce the impact of speckle noise on the reconstructed image, improve the quality of the reconstructed image, and also improve the running speed. Attached Figure Description

[0033] Figure 1 This is a detailed flowchart of an embodiment;

[0034] Figure 2 This is a schematic diagram of a denoising neural network structure;

[0035] Figure 3 The original experimental image and its digital hologram are shown; among them Figure 3 (1) is the original experimental diagram. Figure 3 (2) It is the off-axis digital hologram recorded by MATLAB simulation. Figure 3 (3) A noisy digital hologram with added speckle noise;

[0036] Figure 4 This is a schematic diagram of multiplying a digital hologram with a moving binary mask window.

[0037] Figure 5 A schematic diagram illustrating the averaging of gray values ​​of pixels at the same location in the reconstructed sub-image;

[0038] Figure 6 A schematic diagram showing the comparison results of different binary mask window sizes;

[0039] Figure 6 (1) is the original image. Figure 6 (2) is the reconstructed image obtained by adding speckle noise and then reconstructing it using an off-axis digital hologram algorithm. Figure 6 (3) is the reconstructed image after denoising using the present invention. Figure 6(4) is the reconstructed image after denoising by the UET neural network. Figure 6 (5) is the reconstructed image after Fourier filtering and denoising. Detailed Implementation

[0040] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0041] Example:

[0042] like Figure 1 As shown, a method for improving the quality of digital holographic reconstruction images includes the following steps:

[0043] (1) Obtain the digital hologram to be processed.

[0044] The commonly used dataset set12 for digital image processing is selected. It contains twelve different grayscale images. This invention selects ten of them to create the training dataset and three other images as the test set. The image size is 512×512.

[0045] The process of recording off-axis digital holograms was simulated using MATLAB. The simulation parameters were: light source wavelength h = 0.532 * 10⁻⁶. -3 With a diffraction distance of z0 = 1500 mm and a CCD pixel width of p = 0.00465 mm, an off-axis digital hologram of size 1024 × 1024 was obtained using the Fresnel diffraction integral algorithm.

[0046] (2) Multiply the hologram and the noise hologram with the moving binary mask window to obtain the sub-hologram, and add speckle noise to the sub-hologram according to the preset speckle noise parameters to form the dataset of the network.

[0047] Since the digital holograms obtained from MATLAB simulations do not contain speckle noise, Rayleigh noise is added to the digital holograms using the raylrnd function in MATLAB to simulate the real situation in the experiment.

[0048] After obtaining the noise-free hologram and the noisy hologram respectively, the noise-free hologram and the noisy hologram are multiplied by a moving binary mask window. Each binary mask window is different, thus constructing 100 sub-holograms of size 1000×1000 for each hologram, and finally obtaining 1000 sub-holograms.

[0049] These 1000 noiseless sub-holograms and 1000 noisy sub-holograms serve as the dataset for training the denoising neural network.

[0050] (3) Train the neural network to learn the characteristics of speckle noise, and finally train a network to remove speckle noise.

[0051] The denoising neural network proposed in this invention is based on the U-net structure with some improvements made according to actual conditions. The overall structure of the denoising neural network proposed in this invention is an Encoder-Decoder structure, which is simple yet effective. Figure 2 As shown.

[0052] The Encoder section uses the Resblock module instead of the original convolution module, which improves the network's feature extraction capability. The Decoder section uses bilinear interpolation to restore the resolution of the original image. During feature extraction and upsampling, skip connection operations are used to fuse deep features with shallow features.

[0053] The entire network structure is as follows Figure 2 As shown, Figure 2 In this paper, N represents the number of channels. The network of this invention uses a residual structure in both upsampling and downsampling, which improves the feature extraction capability and denoising effect of the network.

[0054] First, the input image to the network is set to a grayscale image with a size of (1000, 1000).

[0055] Then, pairwise 3x3 convolutions are used to deepen the image, transforming it to dimensions (32, 1000, 1000). The image size remains unchanged, with kernel_size = 3, stride = 1, and padding = 1. Next is a downsampling layer, first passing through a max-pooling layer with stride = 2 and kernel_size = 2, halving the image size. Then, two 3x3 convolutions are applied, resulting in a 64-dimensional feature map. There are a total of four downsampling layers, calculated based on how each downsampling halves the image size and doubles the number of channels. The final downsampling layer results in an image dimension of 512. Upsampling halves the number of channels and doubles the size, exactly the reverse of downsampling. Then, just like the downsampling, we repeat the upsampling four times until the image is restored to (32, 1000, 1000). Finally, since it is a classification of pixels, a single image is generated. At this point, the image size is fixed at 1000. We change the number of channels of the image and use a convolution kernel with kernel_size=1.

[0056] Based on U-net, the number of channels in each layer is halved to reduce the network parameters and speed up the operation of the denoising neural network.

[0057] During training, this invention uses the Adam optimizer, the MSE loss function, a learning rate of 0.001, a batch size of 20, and 35 epochs. All code is implemented in Python and PyTorch and trained on a remote server with an Intel(R) Xeon(R) Platinum 8255C CPU and an Nvidia GeForce GTX 3070 GPU, which takes approximately 8 hours.

[0058] (4) Input the digital hologram with added speckle noise into the neural network and obtain the denoised digital hologram.

[0059] After the denoising network is trained, three test set images (not training set images) are selected and the above dataset creation steps are repeated. Each image yields 100 noisy sub-holograms, which are then input into the denoising neural network for denoising, ultimately resulting in 100 denoised sub-holograms.

[0060] (5) Cut the denoised hologram into 100 sub-holograms and perform holographic reconstruction to obtain 100 original images of the same size.

[0061] Based on these 100 sub-holograms, the sub-holograms are regarded as the initial light wave field illuminated by a plane wave of unit amplitude. The S-FFT algorithm of diffraction is used to calculate the Fresnel diffraction at a distance of z0 = 1500 mm, and 100 reconstructed images are obtained.

[0062] These 100 reconstructed images are the result of initial denoising using a denoising neural network. The trained network captures the noise components of the reconstructed images and effectively preserves the details of the target. Although denoising has been achieved, the reconstructed images can still be further optimized.

[0063] (6) Average the values ​​of pixels at the same location in 100 reconstructed images to reconstruct the final denoised image.

[0064] The speckle patterns of these 100 reconstructed images are different because these images at different locations were reconstructed from holograms. Each spatial frequency of the reconstructed image is distributed across the entire area of ​​the hologram, resulting in different speckle patterns.

[0065] An improved image is obtained by averaging the intensity of the reconstructed image. Pixels located at the same position in each of the 100 sub-holograms are extracted. The grayscale values ​​of these 100 extracted pixels are averaged to obtain the final pixel grayscale value. The final image is generated by repeating the above operation for each pixel position.

[0066] After obtaining the final denoised and reconstructed image using the method of this invention, the advantages of this method are verified using quality evaluation indicators including Edge Preservation Index (EPI), Equivalent Appearance Number (ENL), and Speckle Suppression Index (SSI).

[0067] The closer the Edge Preservation Index (EPI) is to 1, the stronger the edge preservation ability of the algorithm. The formula for calculating the Edge Preservation Index (EPI) is as follows:

[0068]

[0069] Where I o and I d The denoising represents the noisy image and the denoised image, while i and j represent the vertical and horizontal coordinates in the image.

[0070] The equivalent appearance number (ENL) is commonly used to measure the smoothness of uniform regions in an image, and is calculated using the following formula:

[0071]

[0072] μ and σ represent the mean and standard deviation of the image, respectively. The larger the equivalent number of views, the smoother the image.

[0073] The speckle suppression index (SSI) is commonly used to measure the effectiveness of speckle suppression. The calculation formula is shown below:

[0074]

[0075] Among them, S i and M i Let S represent the standard deviation and mean of the original reconstructed image, respectively. Similarly, S... O and M O These represent the standard deviation and mean of the filtered image, respectively. A smaller SSI (Standard Deviation and Mean) results in better speckle noise reduction performance.

[0076] Results analysis:

[0077] like Figure 3 As shown, Figure 3 (1) is the original image. Figure 3 (2) It is the off-axis digital hologram recorded by MATLAB simulation. Figure 3 (3) The noise digital hologram with added speckle noise can be seen to have a lot of speckle noise. Under the same conditions, the present invention recorded three binary mask windows of 600×600, 800×800 and 1000×1000. Table 1 is a comparison of the experimental data of the three binary mask windows.

[0078] like Figure 4 As shown, Figure 4This represents the result of multiplying a noise-free hologram and a noisy hologram by a moving binary mask window. Each binary mask window is different.

[0079] like Figure 5 As shown, Figure 5 This means that an improved image is obtained by averaging the intensity of the reconstructed image. Pixels located at the same position in each of the 100 sub-holograms are extracted. When extracting the 100 pixel values, the grayscale values ​​of these 100 pixels are averaged to obtain the final pixel grayscale value. The final image is generated by repeating the above operation for each pixel position.

[0080] like Figure 6 As shown, Figure 6 (1) is the original image. Figure 6 (2) is the reconstructed image obtained by adding speckle noise and then reconstructing it using an off-axis digital hologram algorithm. Figure 6 (3) is the reconstructed image after denoising using the present invention. Figure 6 (4) is the reconstructed image after denoising by the UET neural network. Figure 6 (5) is the reconstructed image after Fourier filtering and denoising.

[0081] This invention also compares the experimental data of Fourier filtering and the classic neural network U-net with the method proposed in this invention, as shown in Table 2 below.

[0082] The experimental data in Table 1 show that the denoising method proposed in this invention is closely related to the size of the binary mask window. When the binary mask window is larger, the edge preservation index (EPI), equivalent appearance number (ENL), and speckle suppression index (SSI) are all improved. When the binary mask window is 600×600 and 800×800, the speckle suppression is poor, but the edge preservation index and equivalent appearance number are also improved.

[0083] The experimental data in Table 2 show that the denoising method proposed in this invention has significant advantages over traditional filter denoising and denoising using only neural networks, and the denoising performance is significantly improved.

[0084] The advantage of this invention compared to traditional methods lies in its effective combination of the strengths of deep learning and image processing methods while mitigating their respective weaknesses. By segmenting the hologram into blocks, a large number of sub-holograms are obtained. Since there is currently no open-source dataset for holograms, this method effectively solves the problem of acquiring a dataset. A denoising neural network is trained using the obtained dataset, which effectively removes most of the speckle noise. Subsequently, wavefront reconstruction of analog digital holography is performed using the obtained sub-holograms, resulting in a large number of different reconstructed images. Pixels located at the same position in each reconstructed sub-image are extracted. The gray values ​​of these pixels are averaged to obtain the final pixel gray value. The final image is generated by repeating the above operation for each pixel position.

[0085] Experimental analysis shows that this method can reduce the impact of speckle noise on reconstructed images and improve the quality of reconstructed images.

[0086]

[0087] Table 1

[0088]

[0089] Table 2

[0090] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for improving the quality of digital holographic reconstruction images, characterized in that... It includes the following steps: (1). Obtain the digital holograms to be processed, which are respectively the digital holograms with speckle noise and those without speckle noise, and obtain the data set required for neural network training; (2). Use the deep learning algorithm to denoise the digital holograms according to the neural network method with the function of image denoising; (2-1) Perform binary mask processing on the noise hologram to be tested to obtain N sub-holograms with noise; (2-2) Denoise the noise sub-holograms according to the trained neural network with the function of image denoising to obtain N initially denoised sub-holograms; (3). Perform preset reconstruction processing on the denoised digital holograms to obtain the reconstructed image of the digital holograms; The method of the preset reconstruction processing in step (3) is to perform wavefront reconstruction of simulated digital holography on the sub-holograms denoised by the denoising neural network in matlab. On the reconstructed image plane, the distribution of the target information of these sub-holograms remains unchanged, while the distribution of the speckles is different; (4). Denoise the reconstructed image according to the preset image processing algorithm to obtain the final result; The specific steps of step (4) are as follows: After obtaining K different reconstructed sub-holograms, extract the pixels at the same position in each reconstructed sub-hologram image. When extracting the K pixel values, the gray value of the final pixel is obtained by averaging the gray values of these K pixels, as shown in formula (4): Where x and y represent the coordinates on the hologram, I n (x,y) represents the gray value of the nth sub-hologram at position x,y. The final image is generated by repeating the above operation for each pixel position.

2. The method for improving the quality of digital holographic reconstruction images according to claim 1, characterized in that... The specific steps of step (1) are as follows: (1-1). Use matlab to simulate the process of off-axis digital holography to obtain an off-axis digital hologram with a size of L×W; (1-2). Add Rayleigh noise to the digital hologram through the raylrnd function of matlab to simulate the real situation; the probability density function of the Rayleigh noise is as shown in formula (2) when z≥a, and is 0 when z<a, where U(0,1) in formula (1) is the uniform distribution function, and a and b are both constants, (1-3) Multiply the digital hologram and the noise hologram with added Rayleigh noise by a moving binary mask window to construct N sub-holograms, resulting in a series of different sub-holograms, as shown in formula (3), where W(x,y) represents the hologram, x,y represent the coordinates on the hologram, and H represents the coordinates on the hologram. K (x,y) represents the k-th sub-hologram; W(x,y)={H1(x,y),H2(x,y),H3(x,y),…,H k (x,y)}(3) The speckle distributions of the sub-holograms are uncorrelated, and a data set of noise holograms and noise-free holograms available for neural network training is obtained.

3. The method for improving the quality of digital holographic reconstruction images according to claim 1, characterized in that... The structure of the neural network with the function of image denoising is the Encoder-Decoder structure. The Encoder part uses the Resblock module, and the Decoder part uses the method of bilinear interpolation to restore the resolution of the original image. When performing feature extraction and upsampling, the operation of skip connection is adopted to fuse the deep features and the shallow features; The neural network adopts the residual structure both in upsampling and downsampling; When training the neural network, the Adam optimizer is used, the loss function uses MSEloss, the learning rate is 0.001, the batch_size is 20, and the epoch is 35. All the codes are implemented using Python and pytorch.

Images