Continuous phase noise reduction method for digital holography of microstructure measurement based on deep learning

By combining an end-to-end convolutional neural network with the subspace projection method and using the Brown and Perlin noise models to simulate digital holographic continuous phase noise, the problem of filtering out complex phase noise in digital holograms is solved, and efficient phase measurement accuracy is achieved.

CN116485680BActive Publication Date: 2025-09-05ZHEJIANG SCI-TECH UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202310482708.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-04-28
Publication Date
2025-09-05
Estimated Expiration
2043-04-28

AI Technical Summary

Technical Problem

Existing technologies have difficulty in effectively removing complex continuous phase noise in digital holograms, resulting in degraded phase reconstruction quality and insufficient measurement accuracy.

Method used

An end-to-end convolutional neural network combined with the subspace projection method is designed. The Brown and Perlin noise models are used to simulate digital holographic continuous phase noise. By generating a large number of simulation data sets to train the convolutional neural network, efficient filtering of complex phase noise is achieved.

Benefits of technology

The accuracy of digital holographic phase measurement has been improved, with good noise reduction performance, fewer network parameters, fast operation speed, small residual noise, and complete preservation of object detail information.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116485680B_ABST
    Figure CN116485680B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for denoising digital holographic continuous phase of microstructure measurement based on deep learning. A phase image of an object is generated by simulating a MEMS microstructure by generating a random matrix superposition method, and a noise grayscale image is generated by simulating the noise in the digital holographic continuous phase image. A simulation data set is then produced. An end-to-end convolutional neural network is designed and trained to obtain a trained convolutional neural network. A holographic interference pattern of the object to be measured is captured and collected, and a continuous phase pattern containing only the object phase and noise is obtained after spectrum extraction, angular spectrum diffraction, phase unwrapping and distortion compensation. The continuous phase pattern is input into the trained convolutional neural network to obtain the object phase pattern. The present invention accurately produces a simulation data set, avoids the difficulty of collecting a large amount of experimental data, and greatly improves the noise reduction performance and reduces the number of network parameters by adding a subspace projection module to the network structure. It has the advantages of fast computing speed and good noise reduction performance.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to an object measurement method in the field of digital holography technology, and in particular to a method for performing noise reduction on a continuous phase image of an object using deep learning in digital holography. Background Art

[0002] When digital holographic (DH) measurements of micro- and nanostructures are performed, a significant amount of noise, including photon noise, electronic noise, quantum noise, and coherent noise, is introduced into the digital hologram due to factors such as the interference characteristics of the coherent light source, the electronic properties of the image acquisition device (CCD), and the rough surface structure of the object being measured. This noise appears as phase noise of varying shapes in the phase image. This significant amount of phase noise is superimposed on the true phase of the object being measured, severely impacting the quality of phase reconstruction and reducing the accuracy of phase measurement. Most current phase filtering methods use a Gaussian noise model as an approximation of speckle noise. However, phase noise has many sources and cannot be simulated solely using the Gaussian noise model. This is one reason why residual noise still exists in the phase filtering results of these algorithms.

[0003] Furthermore, the wrapped phase of digital holographic microscopy often contains a significant amount of phase distortion. The characteristics of speckle noise are more pronounced in the wrapped phase (appearing as randomly distributed discontinuous values), while the characteristics of continuous phase noise with smaller fluctuations between adjacent pixels are often masked by the distorted phase. Most current traditional or deep learning-based phase filtering methods typically filter the wrapped phase image, where phase noise is often masked by significant phase distortion. Even when filtering the distortion-compensated phase image, the superimposed phase noise is too complex to be removed by a suitable filter. Consequently, a significant amount of unfiltered phase noise still remains in the final reconstructed object phase, limiting the performance of phase filtering. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention proposes a deep learning-based digital holographic continuous phase denoising method for microstructure measurement, designs an end-to-end filtering convolutional neural network combined with the subspace projection method, which can efficiently filter out the complex phase noise existing in the continuous phase image of the digital holographic experiment. The Brown and Perlin noise models are used to accurately simulate the digital holographic continuous phase noise, and a large number of mixed data sets are produced to train the convolutional neural network to filter out the digital holographic continuous phase noise. It has the advantages of good filtering performance, few network parameters and fast execution speed.

[0005] The present invention is achieved through the following technical solutions:

[0006] Step 1: Generate an object phase image by simulating the MEMS microstructure through random matrix superposition. Simultaneously, simulate the noise in the digital holographic continuous phase image to generate a noise grayscale image. Add the object phase image and the noise grayscale image as input data and use the pure object phase image as the label to produce a large number of simulation data sets. Design an end-to-end convolutional neural network combined with the subspace projection method. Input the simulation data set into the convolutional neural network to train the convolutional neural network to obtain a trained convolutional neural network to achieve the denoising task.

[0007] Step 2: Collect the holographic interference pattern of the object to be measured by shooting, obtain the complex amplitude U of the object light field containing the information of the object to be measured through image processing, extract the phase information in the complex amplitude U of the light field and wrap it between (-π, π] to obtain the wrapped phase map

[0008] Step 3: Wrapping the Phase Diagram The unwrapping operation is performed to obtain a continuous phase image containing phase distortion, and then the phase distortion is removed by Zernike polynomial fitting to obtain a continuous phase image containing only the object phase and the noise phase.

[0009] Step 4: After the continuous phase image is input into the trained convolutional neural network, the trained convolutional neural network model is equivalent to a function mapping relationship, which outputs the noise-reduced object phase image, and converts the object phase image into height data to achieve accurate measurement of micro-nano structures.

[0010] The present invention adopts a microstructure standard part as an object to be measured and collects a holographic interference pattern on the surface of the object to be measured.

[0011] The step 1 is specifically as follows:

[0012] 1.1) Generate a large number of step-like structure images as the object phase image by generating non-overlapping random rectangles, where the length and width of each rectangle are pre-limited according to the image size;

[0013] 1.2) For each object phase image generated in step 1.1), generate a noise grayscale image of the same size using the Brown and Perlin noise model algorithms, normalizing the noise standard deviation to a range of 0.05 to 0.26 rad.

[0014] The object phase image and the noise grayscale image are in one-to-one correspondence.

[0015] According to experimental data, the standard deviation of the digital holographic continuous phase noise to be simulated is obtained in the range of 0.1 to 0.15 rad. In the specific implementation, the standard deviation of the noise generated by the simulation is normalized to the range of 0.05 to 0.26 rad, which improves the generalization ability of the network while being in line with the actual experimental situation.

[0016] 1.3) Add the simulated object phase image and the noisy grayscale image to obtain a noisy continuous phase image, which serves as the input data for the convolutional neural network. Use the simulated object phase image without noise as the learning label for the convolutional neural network. Create a simulated dataset containing 40,000 data pairs, and then train the convolutional neural network to obtain a trained convolutional neural network.

[0017] The step 1.1) is specifically as follows:

[0018] First, use Matlab to generate a grayscale image of the MEMS. Then, use the following method to generate 8 to 64 rectangles in the grayscale image. In the grayscale image, set the overlapping parts between rectangles and the parts outside the rectangles to zero. This results in a grayscale image containing multiple non-overlapping figures, which serves as the phase grayscale image simulating the surface structure of the MEMS chip:

[0019] Randomly select a coordinate in the grayscale image as the vertex of the lower left corner of the rectangle, and then randomly generate two random integers within a preset range as the length and width, respectively, to create a filled rectangle;

[0020] Finally, the phase grayscale image is processed with a mean filter with a window size of 3×3 to obtain the object phase image.

[0021] The convolutional neural network specifically includes a first convolution module, a plurality of consecutive basic convolution layers, a subspace projection layer SSA, a second convolution module and an addition layer which are connected in sequence. The first convolution module receives a continuous phase map input into the convolutional neural network. The output of the first convolution module is input into a plurality of consecutive basic convolution layers. The output of the plurality of consecutive basic convolution layers and the output of the first convolution module are both input into the subspace projection layer SSA for processing. The output of the subspace projection layer SSA is input into the second convolution module. The output of the second convolution module and the continuous phase map input into the convolutional neural network are added through the addition layer as the output of the convolutional neural network.

[0022] Each basic convolutional layer is mainly composed of two consecutive first convolutional modules and an addition layer connected in sequence. The input of the basic convolutional layer is processed by two consecutive first convolutional modules and then added to the input of the basic convolutional layer itself through the addition layer as the output of the basic convolutional layer.

[0023] The first convolution module is mainly composed of a convolution operation and an activation function connected in sequence.

[0024] The second convolution module is mainly composed of a first convolution operation, an activation function and a second convolution operation connected in sequence.

[0025] The subspace projection layer SSA includes a convolution regularization module and a convolution operation, an addition layer, a basis vector processing operation Basic Vectors and a projection operation Projection. The outputs of multiple consecutive basic convolution layers and the output of the first convolution module are first spliced ​​and then input into the convolution regularization module and the convolution operation respectively. The output of the convolution regularization module and the output of the convolution operation are added through the addition layer, and then input into the basis vector processing operation Basic Vectors to obtain basis vectors. The output of the basis vector processing operation Basic Vectors and the output of the first convolution module are input into the projection operation Projection together. The projection operation Projection uses the output of the basis vector processing operation Basic Vectors to perform weighted optimization on the output of the first convolution module to obtain the final denoised object phase map;

[0026] The convolution regularization module is mainly composed of a first convolution operation, a first BatchNormal batch normalization operation, a first activation function, a second convolution operation, a second BatchNormal batch normalization operation, and a second activation function connected in sequence.

[0027] The first step of the present invention is to design an end-to-end convolutional neural network combined with the subspace projection method. The network uses dilated convolution to increase the receptive field of the convolution kernel instead of downsampling. At the same time, the Brown and Perlin noise models are used to simulate the noise in the digital holographic continuous phase image. The noise morphology is manifested as a continuously undulating water wave. The step-shaped MEMS microstructure is simulated by generating a random matrix superposition method. The simulated object phase and the simulated noise are added as input data, and the object phase without superimposed noise is used as a label. A large number of simulation data sets are produced to train the convolutional neural network to achieve the noise reduction task.

[0028] The step 2 is specifically as follows:

[0029] 2.1) Using a CCD photosensitive electronic imaging device to record the holographic interferogram of the object to be measured, a spectrum is obtained through Fourier transform, the positive first-order spectrum in the spectrum is extracted and the hologram is reconstructed using inverse Fourier transform. The reconstructed hologram is then diffracted using the angular spectrum diffraction method to obtain the complex amplitude of the object light field containing information about the object to be measured;

[0030] 2.2) Extract the exponential term in the complex amplitude U of the light field and wrap it between (-π,π] to obtain the wrapped phase map.

[0031] The step three is specifically as follows:

[0032] 3.1) After unwrapping the wrapped phase image, a continuous phase image is obtained, which usually contains the phase of the object under test, the distortion phase, and the phase noise;

[0033] 3.2) For the continuous phase diagram φ c Perform Zernike polynomial fitting to obtain the Zernike coefficients of the distortion phase, and calculate the distortion phase φ using the fitted Zernike coefficients a , and finally use the unwrapped phase φ c Subtract the distortion phase φ a A phase image containing the object under test and noise is obtained.

[0034] The fourth step is specifically as follows: for each specific continuous phase image to be measured, a noise-reduced object phase image is obtained for the trained convolutional neural network model:

[0035] Y=Γ(φ)

[0036] Here, Γ(·) represents the trained convolutional neural network, φ is the continuous phase map input to the convolutional neural network, and Y is the denoised object phase map output by the convolutional neural network.

[0037] The present invention establishes an end-to-end convolutional neural network, simulates and produces a large number of mixed data sets, and then trains a noise reduction model. The holographic interferogram of the object to be measured is then collected. After spectrum extraction, angular spectrum diffraction, phase unwrapping, and distortion compensation, a continuous phase map containing only the object phase and noise is obtained. This is then input into the trained convolutional neural network model, and the network outputs the noise-reduced object phase map, which is the object phase map.

[0038] Compared with the existing technology, the beneficial effects of the present invention are:

[0039] The present invention uses the Brown and Perlin noise models to produce simulation data sets, avoiding the difficulty of collecting a large amount of experimental data. By adding a subspace projection module to the network structure, the noise reduction performance is greatly improved, the number of network parameters is reduced, and it has the advantages of fast computing speed and good noise reduction performance.

[0040] The present invention uses the Brown and Perlin noise models to create a mixed data set instead of the commonly used Gaussian noise to simulate the complex phase noise in the continuous phase of digital holographic experiments. An end-to-end convolutional neural network using the subspace projection method is designed and trained to achieve the denoising task, which can efficiently remove the complex phase noise in the continuous phase images of digital holographic experiments.

[0041] The noise reduction process is fully automatic and requires no human intervention. There are no preset parameters, few network parameters, fast running speed, low residual noise, and complete preservation of object detail information. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 The convolutional neural network structure and processing flow chart used in the method of the present invention;

[0043] Table 1 shows the Zernike polynomials used in the examples;

[0044] Figure 2 It is a processing diagram in the embodiment;

[0045] Figure 3 This is a diagram of the final phase filtering result in the embodiment. DETAILED DESCRIPTION

[0046] The present invention will be further described below with reference to the accompanying drawings and examples.

[0047] The present invention is implemented as follows Figure 1 As shown in the flowchart of (a), the specific steps are as follows:

[0048] Step 1: Generate the object phase image by simulating the MEMS microstructure through the superposition of random matrices, and at the same time simulate the noise in the digital holographic continuous phase image to generate a noise grayscale image. Add the object phase image and the noise grayscale image and their labels to produce a large number of simulation data sets; design an end-to-end convolutional neural network combined with the subspace projection method, input the simulation data set into the convolutional neural network to train the convolutional neural network to obtain a trained convolutional neural network to achieve the denoising task.

[0049] Specifically:

[0050] 1.1) In the specific implementation, the size of the convolutional neural network input data is set to M×M. First, several grayscale images of the MEMS with a size of M×M pixels are generated using Matlab. 8 to 64 rectangles are generated in each grayscale image according to the following method. The portions within the rectangles are set to 1, and the overlapping portions between rectangles and the portions outside the rectangles are set to zero. This results in a grayscale image containing multiple non-overlapping shapes, which is equivalent to taking the difference between all rectangles. This serves as a phase grayscale image simulating the surface structure of the MEMS chip, resulting in a grayscale image containing only 0s and 1s. This image is then multiplied by a random number in the range [0, π] to give each image a different phase value:

[0051] First, randomly select a coordinate in the M×M pixel matrix and use this coordinate as the vertex of the lower left corner of the rectangle. Then randomly generate two random integers within a certain range as the length and width respectively, and a filled rectangle is obtained.

[0052] Finally, the phase grayscale image is processed with a mean filter with a window size of 3×3 to reduce the edge gradient and obtain the object phase image, so as to optimize the learning ability of the convolutional neural network for the data set.

[0053] 1.2) For each object phase image generated in step 1.1), generate a noise grayscale image of the same size M × M using Brown and Perlin noise model algorithms that simulate natural random noise morphology, with the standard deviation of the noise normalized to a range of 0.05 to 0.26 rad during generation;

[0054] 1.3) The simulated object phase image and the noisy grayscale image are added together to obtain a noisy continuous phase image, which is used as the input data for the convolutional neural network. The simulated object phase image without added noise is used as the learning label for the convolutional neural network. A simulated dataset containing 40,000 pairs of data (20,000 pairs each containing Brown and Perlin noise) is created to train the convolutional neural network and obtain a trained convolutional neural network.

[0055] In step 1.3, when training the convolutional neural network, the initial training parameters are set as follows: learning rate of 0.0001, Adam optimizer, root mean square error loss function, and cosine annealing learning rate decay function. The dataset is trained 20 times.

[0056] Figure 1 (b) in the figure is the designed convolutional neural network combined with the subspace projection method.

[0057] The convolutional neural network specifically includes an input layer, a first convolution module, multiple consecutive basic convolution layers, a subspace projection layer SSA, a second convolution module, an addition layer, and an output layer, which are connected in sequence. The first convolution module receives the continuous phase map input into the convolutional neural network, and the output of the first convolution module is input into multiple consecutive basic convolution layers. The output of multiple consecutive basic convolution layers and the output of the first convolution module are both input into the subspace projection layer SSA for processing. The output of the subspace projection layer SSA is input into the second convolution module. The output of the second convolution module and the continuous phase map input into the convolutional neural network are added through the addition layer as the output of the convolutional neural network.

[0058] Each basic convolutional layer is mainly composed of two consecutive first convolutional modules and an addition layer connected in sequence. The input of the basic convolutional layer is processed by two consecutive first convolutional modules and then added to the input of the basic convolutional layer itself through the addition layer as the output of the basic convolutional layer.

[0059] The first convolution module mainly consists of a convolution operation and an activation function connected in sequence.

[0060] The second convolution module mainly consists of the first convolution operation, an activation function and the second convolution operation connected in sequence.

[0061] The subspace projection layer SSA includes a convolution regularization module and convolution operation, an addition layer, a basis vector processing operation Basic Vectors and a projection operation Projection. The outputs of multiple consecutive basic convolution layers and the output of the first convolution module are first concatenated and then input into the convolution regularization module and the convolution operation respectively. The output of the convolution regularization module and the output of the convolution operation are added through the addition layer and then input into the basis vector processing operation Basic Vectors to obtain basis vectors. The output X2 of the basis vector processing operation Basic Vectors and the output X1 of the first convolution module are input into the projection operation Projection together. The projection operation Projection uses the output of the basis vector processing operation Basic Vectors to perform weighted optimization on the output X1 of the first convolution module to obtain the final denoised object phase map.

[0062] The convolution regularization module is mainly composed of the first convolution operation, the first BatchNormal batch normalization operation, the first activation function, the second convolution operation, the second BatchNormal batch normalization operation, and the second activation function connected in sequence.

[0063] For the input noisy phase image, features are first extracted using a 7×7 convolution kernel, which is then expanded to 32 channels. Features are then extracted sequentially through 19 convolution modules with residual structures. Features are then separated using a subspace projection module. Finally, two convolution layers are used to integrate the noise phase into a single-channel grayscale image. This image is then added to the original input image to output the filtered phase image. The BasicConvLayer residual module consists of two convolutional layers. The first uses a standard 3×3 convolution kernel, while the second uses a 3×3 dilated convolution kernel with a dilation factor of 2. Both layers use the LeakReLU activation function, with a negative semi-axis slope of 0.2.

[0064] In the subspace projection module, the input low-dimensional features X1 and high-dimensional features X2 are first merged and spliced ​​in their channel dimensions, and feature extraction is performed through two basic convolutional layers with 3×3 convolution kernels and residual connections. Then, the feature map is mapped to k channels, where k is the subspace dimension.

[0065] Expand the feature map on each channel into a one-dimensional vector to obtain k vectors of size M 2 Vector, M is the size of the feature map, and a set of basic vectors are recorded as Then the projection operation is performed, and the low-dimensional image feature map X1 is projected into the k-dimensional subspace using orthogonal linear projection to separate the signal, which is expressed as:

[0066] P=V(V T V) -1 V T

[0067] Among them, P is the orthogonal projection matrix of the signal subspace, and V represents the basic vectors;

[0068] Finally, the low-dimensional feature map X1 is reconstructed in the signal subspace as:

[0069] Y=PX1

[0070] Where Y is the final denoised object phase map, which is reconstructed and transformed into a feature map of the same dimension as X1 and is sent to the next layer of the convolutional neural network as the output of the subspace projection module.

[0071] Step 2: Collect the holographic interference pattern of the object to be measured by shooting, obtain the complex amplitude U(x,y) of the object light field with a size of M×M containing the information of the object to be measured through image processing, extract the phase information in the complex amplitude U(x,y) and wrap it between (-π,π] to obtain the wrapped phase Specifically:

[0072] 2.1) A CCD photosensitive electronic imaging device is used to record the holographic interference pattern of the object to be measured. A spectrum is obtained through Fourier transform. The positive first-order spectrum in the spectrum is extracted and reconstructed using inverse Fourier transform. The reconstructed hologram is then diffracted using the angular spectrum diffraction method to obtain the complex amplitude of the object light field containing the information of the object to be measured, which is specifically expressed as:

[0073] U(x,y)=A(x,y)exp[iψ(x,y)],x,y=1,2,…,M

[0074] ψ(x,y)=φ o (x,y)+φ a (x,y)+φ e(x,y),x,y=1,2,…,M

[0075] Where U is the complex amplitude of the object light field, (x, y) is the coordinate point on the two-dimensional plane, and i represents the imaginary unit. A is the amplitude of the light field, ψ is the phase information, including the phase φ of the object to be measured o , distortion phase φ a and phase noise φ e ;

[0076] 2.2) Extract the exponential term in the complex amplitude U of the light field and wrap it between (-π,π] to obtain the wrapped phase diagram, which is expressed as:

[0077]

[0078] in, is the wrapped phase diagram, arctan{·} is the inverse tangent function, Im[·] is the operation of taking the imaginary part, and Re[·] is the operation of taking the real part.

[0079] Figure 2 (a) is the microstructure holographic interference pattern collected in this embodiment, Figure 2 (b) in the Figure 2 The holographic interference pattern of (a) is Fourier transformed to obtain a spectrum diagram, the +1-level spectrum is extracted and inverse Fourier transformed, the complex amplitude distribution of the object light field is obtained using the angular spectrum diffraction method, and the phase information is extracted to obtain the wrapped phase diagram. Figure 2 (c) in the figure is a continuous phase diagram containing a large amount of phase distortion obtained after unwrapping the wrapped phase diagram.

[0080] Step 3: Wrapping the Phase Diagram The unwrapping operation is performed to obtain a continuous phase image containing phase distortion, and then the phase distortion is removed by Zernike polynomial fitting to obtain a continuous phase image containing only the object phase and the noise phase; specifically:

[0081] 3.1) After performing least squares unwrapping on the wrapped phase image, a continuous phase image is obtained, which usually contains the phase of the object under test, the distortion phase, and the phase noise, and is expressed as:

[0082]

[0083] Among them, φ c is the unwrapping phase, continuous surface, unwrap[·] is the unwrapping operation, φ o (x,y),φ a (x,y) and φ e (x,y) are the continuous object phase, distortion phase and noise phase respectively;

[0084] 3.2) For the continuous phase diagram φ c Perform Zernike polynomial fitting to obtain the Zernike coefficients of the distortion phase, and calculate the distortion phase φ using the fitted Zernike coefficients a , and finally use the unwrapped phase φ c Subtract the distortion phase φ a The phase including the DUT and noise is obtained, which is expressed as:

[0085] φ(x,y)=φ c (x,y)-φ a (x,y)

[0086] Among them, φ is the noisy continuous phase image that needs to be input into the convolutional neural network model for denoising.

[0087] Table 1

[0088]

[0089]

[0090] Table 1 above is the Zernike polynomial in the Cartesian coordinate system used in this implementation. Figure 2 (d) in the figure is a continuous phase image containing only the object phase and phase noise after compensating for phase distortion.

[0091] Step 4: After the continuous phase image is input into the trained convolutional neural network, the network outputs the noise-reduced object phase image. Specifically:

[0092] For the trained convolutional neural network model, it is regarded as a function mapping relationship. For each specific continuous phase image to be measured, a noise-reduced object phase image is obtained:

[0093] Y=Γ(φ)

[0094] Here, Γ(·) represents the trained convolutional neural network model with specific network parameters, φ is the continuous phase image input to the convolutional neural network, and Y is the denoised object phase image output by the convolutional neural network after processing the input data. Only the object phase information remains, which is converted into height data to obtain the topography measurement value.

[0095] Figure 3 For the general Figure 2 The noisy continuous phase image in (d) is input into the trained convolutional neural network, and the network outputs the denoised object phase image.

[0096] In order to solve the problem that existing phase filtering algorithms are difficult to solve the complex noise existing in the continuous phase of digital holography, the present invention designs a deep convolutional neural network combined with the subspace projection method, uses the Brown and Perlin noise models to simulate the noise in the continuous phase of digital holography, and produces a large number of data sets to train the designed convolutional neural network. This achieves the purpose of efficiently filtering out the phase noise in digital holography experiments, greatly improving the accuracy of digital holographic phase measurement.

Claims

1. A method for continuous phase noise reduction of digital holography for microstructure measurement based on deep learning, characterized by: Step 1: Generate an object phase image by simulating the MEMS microstructure through random matrix superposition. Simultaneously, simulate the noise in the digital holographic continuous phase image to generate a noise grayscale image. Add the object phase image and the noise grayscale image as input data and use the object phase image as a label to create a simulation data set. Design an end-to-end convolutional neural network combined with the subspace projection method. Input the simulation data set into the convolutional neural network to train the convolutional neural network and obtain a trained convolutional neural network. Step 2: Collect the holographic interference pattern of the object to be measured by shooting, obtain the complex amplitude U of the object light field containing the information of the object to be measured through image processing, extract the phase information in the complex amplitude U of the light field and wrap it between (-π, π] to obtain the wrapped phase map Step 3: Wrapping the Phase Diagram The unwrapping operation is performed to obtain a continuous phase image containing phase distortion, and then the phase distortion is removed by Zernike polynomial fitting to obtain a continuous phase image containing only the object phase and the noise phase. Step 4: After inputting the continuous phase image into the trained convolutional neural network, the denoised object phase image is output.

2. The method for continuous phase noise reduction of digital holography for microstructure measurement based on deep learning according to claim 1, characterized in that: The step 1 is specifically as follows: 1.1) Generate several step-like structure images as the object phase image by generating non-overlapping random rectangles; 1.2) For each object phase image generated in step 1.1), generate a noise grayscale image of the same size using the Brown and Perlin noise model algorithms, normalizing the noise standard deviation to a range of 0.05 to 0.26 rad. 1.3) Add the simulated object phase image and the noisy grayscale image to obtain a noisy continuous phase image, which is used as the input data of the convolutional neural network. Use the simulated object phase image without added noise as the learning label of the convolutional neural network to create a simulation dataset, and then train the convolutional neural network to obtain a trained convolutional neural network.

3. The method for continuous phase noise reduction of digital holography for microstructure measurement based on deep learning according to claim 2, characterized in that: The step 1.1) is specifically as follows: First, use Matlab to generate a grayscale image of the MEMS. Then, use the following method to generate 8 to 64 rectangles in the grayscale image. In the grayscale image, set the overlapping parts between rectangles and the parts outside the rectangles to zero. This results in a grayscale image containing multiple non-overlapping figures, which serves as the phase grayscale image simulating the surface structure of the MEMS chip: Randomly select a coordinate in the grayscale image as the vertex of the lower left corner of the rectangle, and then randomly generate two random integers within a preset range as the length and width, respectively, to create a filled rectangle; Finally, the phase grayscale image is processed with a mean filter with a window size of 3×3 to obtain the object phase image.

4. The method for continuous phase noise reduction of digital holography for microstructure measurement based on deep learning according to claim 1 or 2, characterized in that: The convolutional neural network specifically includes a first convolution module, a plurality of consecutive basic convolution layers, a subspace projection layer SSA, a second convolution module and an addition layer which are connected in sequence. The first convolution module receives a continuous phase map input into the convolutional neural network. The output of the first convolution module is input into a plurality of consecutive basic convolution layers. The output of the plurality of consecutive basic convolution layers and the output of the first convolution module are both input into the subspace projection layer SSA for processing. The output of the subspace projection layer SSA is input into the second convolution module. The output of the second convolution module and the continuous phase map input into the convolutional neural network are added through the addition layer as the output of the convolutional neural network.

5. The method for continuous phase noise reduction of digital holography for microstructure measurement based on deep learning according to claim 4, characterized in that: Each basic convolutional layer is mainly composed of two consecutive first convolutional modules and an addition layer connected in sequence. The input of the basic convolutional layer is processed by two consecutive first convolutional modules and then added to the input of the basic convolutional layer itself through the addition layer as the output of the basic convolutional layer.

6. The method for continuous phase noise reduction of digital holography for microstructure measurement based on deep learning according to claim 4, characterized in that: The subspace projection layer SSA includes a convolution regularization module and a convolution operation, an addition layer, a basis vector processing operation Basic Vectors and a projection operation Projection. The outputs of multiple consecutive basic convolution layers and the output of the first convolution module are first spliced ​​and then input into the convolution regularization module and the convolution operation respectively. The output of the convolution regularization module and the output of the convolution operation are added through the addition layer and then input into the basis vector processing operation Basic Vectors. The output of the basis vector processing operation Basic Vectors and the output of the first convolution module are input into the projection operation Projection together. The projection operation Projection uses the output of the basis vector processing operation Basic Vectors to perform weighted optimization on the output of the first convolution module to obtain the final denoised object phase map; The convolution regularization module is mainly composed of a first convolution operation, a first BatchNormal batch normalization operation, a first activation function, a second convolution operation, a second BatchNormal batch normalization operation, and a second activation function connected in sequence.

7. The method for continuous phase noise reduction of digital holography for microstructure measurement based on deep learning according to claim 1, characterized in that: The step 2 is specifically as follows: 2.1) Using a CCD photosensitive electronic imaging device to record the holographic interferogram of the object to be measured, a spectrum is obtained through Fourier transform, the positive first-order spectrum in the spectrum is extracted and the hologram is reconstructed using inverse Fourier transform. The reconstructed hologram is then diffracted using the angular spectrum diffraction method to obtain the complex amplitude of the object light field containing information about the object to be measured; 2.2) Extract the exponential term in the complex amplitude U of the light field and wrap it between (-π,π] to obtain the wrapped phase map.

8. The method for continuous phase noise reduction of digital holography for microstructure measurement based on deep learning according to claim 1, characterized in that: The step three is specifically as follows: 3.1) After unwrapping the wrapped phase image, a continuous phase image is obtained, which usually contains the phase of the object under test, the distortion phase, and the phase noise; 3.2) For the continuous phase diagram φ c Perform Zernike polynomial fitting to obtain the Zernike coefficients of the distortion phase, and calculate the distortion phase φ using the fitted Zernike coefficients a , and finally use the unwrapped phase φ c Subtract the distortion phase φ a A phase image containing the object under test and noise is obtained.

9. The method for continuous phase noise reduction of digital holography for microstructure measurement based on deep learning according to claim 1, characterized in that: The fourth step is specifically as follows: for each continuous phase image to be measured, a noise-reduced object phase image is obtained for the trained convolutional neural network model: Y=Γ(φ) Here, Γ(·) represents the trained convolutional neural network, φ is the continuous phase map input to the convolutional neural network, and Y is the denoised object phase map output by the convolutional neural network.

Citation Information

Patent Citations

  • Deep learning-based optical scanning holography defocus noise eliminating method

    CN108089425A

  • Phase denoising network and method based on convolutional neural network

    CN113327205A