A method for optimizing a bi-phase hologram reconstruction image

Abstract

The application provides a method for optimizing a two-phase hologram reconstruction image. The method comprises the following steps: obtaining a reverse diffraction field of a target image; generating a two-phase hologram of the target image by using a two-phase hologram encoding method according to the reverse diffraction field of the target image; inputting the two-phase hologram into a re-encoding model to obtain a hologram of the target image; the re-encoding model is obtained by training an initial full convolutional neural network by using a plurality of training target images and corresponding training holograms; and determining a reconstruction image of the target image based on the hologram. The two-phase hologram is re-encoded as a whole by constructing the re-encoding model, so that the quality and definition of the two-phase hologram reconstruction image are improved.

Description

Technical Field

[0001] This invention relates to the field of image reconstruction technology, and in particular to a method for optimizing the reconstruction of dual-phase holograms. Background Technology

[0002] Computer Generated Holography (CGH) is a technology that combines computer technology with optical holography. It can achieve some functions that are difficult or impossible to achieve with optical holography, such as controlling the size of the reproduced object or reproducing objects that do not exist in reality.

[0003] Computational holography primarily employs spatial light modulators (SLMs) to replace the photosensitive materials used in optical holography. Currently, SLMs are mainly phase-type, modulating the information of incident light by altering its phase distribution. Therefore, generating pure phase information based on diffraction field information has become a major research problem in computational holography. Traditional generation methods include the iterative GS (Gerchberg-Saxton) algorithm and WH (Wirtinger Holography) algorithm; the double-phase hologram (DPH) encoding method and the stochastic gradient descent (SGD) method, which directly encode the diffraction field. Traditional methods often present a trade-off between reconstruction quality and generation speed. While the double-phase hologram algorithm generates reconstructed holograms quickly, it introduces spatial shift noise during the conversion of complex amplitude information into pure phase information, resulting in lower reconstruction quality. The WH algorithm generates high-quality holograms but requires a long iteration time.

[0004] In recent years, researchers have combined emerging neural networks with computational holography, achieving promising results by fitting the relationship between diffraction fields and holograms using a massive number of nodes in the network. Incomplete Fresnel zone plates at the edges of the diffraction field cause numerous bright and dark fringes. The pixel-by-pixel encoding method of biphase holograms also encodes these fringes into the hologram. Therefore, many fringes also appear at the edges of the reconstructed biphase hologram image. Furthermore, spatial offset noise generated during the conversion from complex amplitude information to pure phase information also degrades the reconstruction quality and sharpness of the biphase hologram. Summary of the Invention

[0005] The purpose of this invention is to provide a method for optimizing the reconstruction of a two-phase hologram. By recoding the two-phase hologram as a whole, the quality and clarity of the reconstructed image can be improved.

[0006] To achieve the above objectives, the present invention provides the following solution:

[0007] A method for optimizing reconstructed images from dual-phase holograms includes:

[0008] Obtain the reverse diffraction field of the target image;

[0009] Based on the reverse diffraction field of the target image, a biphase hologram of the target image is generated using a biphase hologram encoding method.

[0010] The biphase hologram is input into the recoding model for recoding to obtain the hologram of the target image; the recoding model is obtained by training an initial fully convolutional neural network using multiple training target images and their corresponding training holograms.

[0011] Based on the hologram, a reconstructed image of the target image is determined.

[0012] Optionally, before acquiring the diffraction field of the backpropagation of the target image, the method further includes:

[0013] The initial fully convolutional neural network is constructed as the fully convolutional neural network at the 0th iteration.

[0014] Acquire multiple target images for training and their corresponding training holograms;

[0015] Let the number of iterations n = 1;

[0016] Multiple training holograms are input into the fully convolutional neural network in the (n-1)th iteration for recoding to obtain the training hologram for each target image;

[0017] Based on the multiple training holograms, a training reconstructed image is determined for each training target image;

[0018] Based on each training target image and the training reconstructed image of each training target image, the loss function value at the nth iteration is determined using the loss function formula.

[0019] Determine whether the loss function value is less than the loss function value threshold, and obtain the determination result;

[0020] If the judgment result is yes, then the fully convolutional neural network in the (n-1)th iteration is determined to be a recoding model;

[0021] If the judgment result is negative, the fully convolutional neural network at the (n-1)th iteration is updated using the backpropagation algorithm to obtain the fully convolutional neural network at the nth iteration.

[0022] Increment the iteration number n by 1 and return to the step "Input the multiple training holograms into the fully convolutional neural network at the (n-1)th iteration for recoding to obtain the training hologram for each target image".

[0023] Optionally, the loss function formula is L1=∑||U rec (x,y)|-|U0(x,y)||

[0024] Where L1 is the loss function; |U rec (x,y)| represents the amplitude information of the reconstructed image used for training; |U0(x,y)| represents the amplitude information of the target image used for training.

[0025] Optionally, the initial fully convolutional neural network includes an input layer, a first downsampling layer, a second downsampling layer, a first upsampling layer, a second upsampling layer, and an output layer connected in sequence.

[0026] The first downsampling layer and the second upsampling layer are connected;

[0027] The second downsampling layer is connected to the first upsampling layer.

[0028] Optionally, acquiring the reverse diffraction field of the target image includes:

[0029] Obtain the intensity distribution of the target image;

[0030] Based on the intensity distribution, using the formula Determine the complex amplitude of the light field containing the target image; where U0(x0,y0) is the complex amplitude of the light field containing the target image; and I0(x0,y0) is the intensity distribution of the target image.

[0031] Based on the complex amplitude of the light field where the target image is located, the formula is used. Determine the reverse diffraction field of the target image;

[0032] Among them, U dif (x,y) represents the reverse diffraction field of the target image; For Fourier transform, This represents the inverse Fourier transform, where λ is the wavelength of the reference light. denoted as , where is the optical transfer function in the reverse diffraction process; u is the spatial frequency of the diffraction field along the x-axis; and v is the spatial frequency of the diffraction field along the y-axis.

[0033] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0034] This invention provides a method for optimizing the reconstruction of a two-phase hologram. The method involves obtaining the reverse diffraction field of a target image; generating a two-phase hologram of the target image using a two-phase hologram encoding method based on the reverse diffraction field; inputting the two-phase hologram into a recoding model for recoding to obtain the hologram of the target image; the recoding model is obtained by training an initial fully convolutional neural network using multiple training target images and their corresponding training holograms; and determining the reconstructed image of the target image based on the hologram. By constructing a recoding model to recode the two-phase hologram as a whole, the quality and clarity of the reconstructed image of the two-phase hologram can be improved. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a flowchart of the dual-phase hologram reconstruction image optimization method in Embodiment 1 of the present invention;

[0037] Figure 2 This is a schematic diagram of the dual-phase hologram reconstruction image optimization method in Embodiment 1 of the present invention;

[0038] Figure 3 This is a flowchart of the angular spectrum method for calculating the diffraction field in Embodiment 1 of the present invention;

[0039] Figure 4 This is a flowchart of the process for generating a biphase hologram using checkerboard coding in Embodiment 1 of the present invention;

[0040] Figure 5 This is a schematic diagram of the initial fully convolutional neural network structure in Embodiment 1 of the present invention. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] The purpose of this invention is to provide a method, system, and electronic device for optimizing the reconstruction of a two-phase hologram. By recoding pixels from the entire two-phase hologram, the quality and clarity of the reconstructed two-phase hologram can be improved.

[0043] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] Example 1

[0045] like Figures 1-2 As shown, Figure 2 In this context, ① represents propagating; ② represents encoding; ③ represents input / output; ④ represents propagating; and ⑤ represents backpropagation. This embodiment provides a method for optimizing the reconstruction of a dual-phase hologram, including:

[0046] Step 101: Obtain the reverse diffraction field of the target image.

[0047] For example, such as Figure 3 ( Figure 3 In the diagram, (a) is the target image; (b) is the spectrum of the target image; (c) is the spectrum of the diffraction field; (d) is the diffraction field; FFT represents Fast Fourier Transform; IFFT represents Inverse Fast Fourier Transform. As shown, step 101 includes:

[0048] Step 101-1: Obtain the intensity distribution of the target image.

[0049] Step 101-2: Based on the intensity distribution, use the formula Determine the complex amplitude of the light field where the target image is located; where U0(x0,y0) is the complex amplitude of the light field where the target image is located; and I0(x0,y0) is the intensity distribution of the target image.

[0050] Step 101-3: Based on the complex amplitude of the light field containing the target image, use the formula... Determine the reverse diffraction field of the target image.

[0051] Among them, U dif (x,y) represents the reverse diffraction field of the target image; For Fourier transform, This represents the inverse Fourier transform, where λ is the wavelength of the reference light. denoted as , where is the optical transfer function in the reverse diffraction process; u is the spatial frequency of the diffraction field along the x-axis; and v is the spatial frequency of the diffraction field along the y-axis.

[0052] Let the intensity distribution of the target image be I0(x0,y0). When the target image is perpendicularly illuminated by a unit amplitude incident wave, the complex amplitude of the light field can be expressed as:

[0053] According to the ASM algorithm, the reverse diffraction field U at a distance z from the target image is... dif (x, y) can be represented as

[0054] According to the Nyquist sampling theorem, the spatial frequency should be limited to [-1 / 2Δp, 1 / 2Δp], where Δp is the sampling interval of the spatial light modulator. The optical transfer function (OTF) in the backward diffraction process can be expressed as:

[0055] In the formula, k is an intermediate variable, k = 2π / λ, and z is the diffraction distance.

[0056] Step 102: Based on the reverse diffraction field of the target image, generate a two-phase hologram of the target image using the two-phase hologram encoding method.

[0057] Using the theory of two-phase holograms, the complex amplitude information of a two-dimensional distributed backward diffraction field is represented as:

[0058] U dif ′(x,y)=|U dif ′(x,y)|exp[jφ(x,y)].

[0059] In the formula, |U dif ′(x,y)| is the modulus of the complex amplitude, which has been normalized to the interval [0,1]. U dif ′(x,y) can be decomposed into the sum of two complex amplitudes of unit length:

[0060]

[0061] In the formula, θ1(x,y) is the phase of the first complex amplitude, θ2(x,y) is the phase of the second complex amplitude, and its relationship with the phase information φ(x,y) of the reverse diffraction field can be expressed as:

[0062]

[0063] The phase distribution θ(x,y) of a two-phase hologram is obtained by means of... Figure 4 The chessboard-style encoding method shown is used to obtain the results. Figure 4The complex amplitude distribution of the U-shaped reverse diffraction field; the phase information of the θ-shaped biphase hologram. It can be seen that the biphase hologram encodes the diffraction field pixel-by-pixel, thus the fringes in the diffraction field are also encoded into the hologram. This also leads to fringes appearing in the reconstructed image. As the imaging distance increases, the fringes in the reconstructed image become more and more obvious, and the reconstruction quality deteriorates. Furthermore, due to the phase difference between the two different phases, the biphase hologram reconstructed image contains a large amount of spatial displacement noise, which will lead to a reduction in reconstruction quality. To solve this problem, this embodiment designs a fully convolutional neural network to recode the biphase hologram to remove fringes and spatial displacement noise from the reconstructed image.

[0064] Step 103: Input the biphase hologram into the recoding model for recoding to obtain the hologram of the target image (i.e., pure phase hologram, Recoded Hologram); the recoding model is obtained by training an initial fully convolutional neural network using multiple training target images and their corresponding training holograms.

[0065] Step 104: Based on the hologram, determine the reconstructed image of the target image. The dual-phase hologram reconstruction image optimization method provided in this embodiment further includes the following steps before step 101:

[0066] Step 105: Construct the initial fully convolutional neural network as shown in the 0th iteration. The initial fully convolutional neural network includes an input layer, a first downsampling layer, a second downsampling layer, a first upsampling layer, a second upsampling layer, and an output layer connected in sequence; the first downsampling layer and the second upsampling layer are connected; the second downsampling layer and the first upsampling layer are connected.

[0067] Step 106: Obtain multiple training target images and their corresponding training holograms.

[0068] Step 107: Let the number of iterations n = 1.

[0069] Step 108: Input the multiple training holograms into the fully convolutional neural network at the (n-1)th iteration for recoding to obtain the training hologram for each target image.

[0070] Step 109: Based on the multiple training holograms, determine the training reconstructed image for each training target image.

[0071] Step 1010: Based on each training target image and its corresponding reconstructed training image, determine the loss function value for the nth iteration using the loss function formula. The loss function formula is L1 = ∑||U rec (x,y)|-|U0(x,y)||;

[0072] Where L1 is the loss function; |U rec (x,y)| represents the amplitude information of the reconstructed image used for training; |U0(x,y)| represents the amplitude information of the target image used for training.

[0073] Step 1011: Determine whether the loss function value is less than the loss function value threshold, and obtain the determination result; if the determination result is yes, then proceed to step 1012; if the determination result is no, then proceed to step 1013.

[0074] Step 1012: Determine that the fully convolutional neural network in the (n-1)th iteration is a recoding model.

[0075] Step 1013: If the judgment result is negative, then use the backpropagation algorithm to update the fully convolutional neural network in the (n-1)th iteration to obtain the fully convolutional neural network in the nth iteration.

[0076] Step 1014: Increment the iteration number n by 1 and return to step 108.

[0077] like Figure 5 As shown, a fully convolutional neural network (FCN) includes an input layer, an output layer, a downsampling layer, and an upsampling layer. The DPH, as the input information to the FCN, is encoded by the downsampling layer for shallow feature extraction. Then, the upsampling layer decodes the encoded information to extract deep features. The downsampling and upsampling layers are connected via a "skip-connection," combining shallow and deep features to encode the DPH.

[0078] The FCN is configured with two downsampling layers and two upsampling layers. During downsampling, 3×3 convolutional kernels are used to extract image features, and 2×2 max pooling is used to encode these features. With one downsampling operation, the data size is reduced to one-quarter, and the number of channels is doubled. During upsampling, a 2×2 convolutional kernel is used for both upconvolution and decoding. With one upsampling operation, the data size is quadrupled, and the number of channels is halved. The highest level has 32 features, and the maximum number of channels is 512. The Adam optimizer, used to optimize weights and biases, has a learning rate of 0.0005. DIV2K-train-HR is used as the training dataset for FCN. The computing platform used to load the FCN model is an NVIDIA GeForce RTX 3090ti. All numerical experiments below are based on this computing platform.

[0079] The optical experimental platform connects an unpolarized semiconductor laser to a single-mode fiber with a core diameter of 4 μm and a wavelength of 638 (±8) nm, which serves as the reconstruction light source. The output end of the fiber is positioned at the focal point of a collimating lens with a focal length of 100 mm to obtain a plane wave. A filter is inserted as an attenuator to adjust the light intensity, and a polarizer is used to obtain linearly polarized light. A half-wave plate (HWP) is then used to change the polarization of the light to optimize the polarization direction of the LCoS. A rectangular aperture is placed after the half-wave plate to reduce the size of the plane wave and obtain a rectangular profile, and a 50% / 50% unpolarized beam splitter is used to split the laser beam. Only the phase hologram is loaded onto a nematic twisted liquid crystal LCoS with a resolution of 4094 × 2400 and a pixel pitch of 3.74 μm. The incident laser is modulated and reflected by the LCoS and propagates again through the beam splitter. A 150mm Fourier lens was used to magnify the reconstructed image and as a spatial filter to allow the desired diffraction order to pass through while blocking others. A Canon EOS 5D Mark III camera equipped with an EF 100-mm f / 2.8 macro lens was used to capture the magnified reconstructed image.

[0080] Using the ASM algorithm, the forward diffraction field of the recoded hologram at a distance z is represented as follows:

[0081]

[0082] θ rec (x,y) represents the phase distribution of the DPH after FCN recoding. Let be the optical transfer function in the forward diffraction process, and u and v be the spatial frequencies of the diffraction field in the x-axis and y-axis directions, respectively.

[0083]

[0084] Amplitude information of the reconstructed image | U rec The amplitude information of the target image, |U0(x,y)|, is compared with that of the target image. The error between the reconstructed image and the target image is calculated using the L1 loss function with a stable solution. The network parameters of the FCN are then updated using the backpropagation algorithm.

[0085] Using the DIV2K-train-HR dataset as the training set for FCN, the quality of the re-encoded biphase hologram reconstructed image can be continuously improved by repeatedly training and optimizing the network model.

[0086] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0087] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for optimizing the reconstructed image of a dual-phase hologram, characterized in that, include: The initial fully convolutional neural network is constructed as the fully convolutional neural network at the 0th iteration. Acquire multiple target images for training and their corresponding biphase holograms for training; Let the number of iterations n=1; Multiple training biphase holograms are input into the fully convolutional neural network at the (n-1)th iteration for recoding to obtain the training hologram for each target image; Based on the multiple training holograms, a training reconstructed image is determined for each training target image; Based on each training target image and the training reconstructed image of each training target image, the loss function value at the nth iteration is determined using the loss function formula. Determine whether the loss function value is less than the loss function value threshold, and obtain the determination result; If the judgment result is yes, then the fully convolutional neural network in the (n-1)th iteration is determined to be a recoding model; If the judgment result is negative, the fully convolutional neural network at the (n-1)th iteration is updated using the backpropagation algorithm to obtain the fully convolutional neural network at the nth iteration. Increment the iteration number n by 1 and return to the step "Input the multiple training biphase holograms into the fully convolutional neural network at the (n-1)th iteration for recoding to obtain the training hologram for each target image"; Obtain the reverse diffraction field of the target image; Based on the reverse diffraction field of the target image, a biphase hologram of the target image is generated using a biphase hologram encoding method. The biphase hologram is input into the recoding model for recoding to obtain the hologram of the target image; The recoding model is obtained by training an initial fully convolutional neural network using multiple training target images and their corresponding training biphase holograms. Based on the hologram, a reconstructed image of the target image is determined.

2. The method for optimizing a dual-phase holographic reconstruction image according to claim 1, characterized in that, The loss function formula is as follows: in, The loss function; The amplitude information of the reconstructed image is used for training; The amplitude information of the target image is used for training.

3. The method for optimizing a dual-phase holographic reconstruction image according to claim 1, characterized in that, The initial fully convolutional neural network includes an input layer, a first downsampling layer, a second downsampling layer, a first upsampling layer, a second upsampling layer, and an output layer connected in sequence. The first downsampling layer and the second upsampling layer are connected; The second downsampling layer is connected to the first upsampling layer.

4. The method for optimizing a dual-phase holographic reconstruction image according to claim 1, characterized in that, The acquisition of the reverse diffraction field of the target image includes: Obtain the intensity distribution of the target image; Based on the intensity distribution, using the formula Determine the complex amplitude of the light field containing the target image; where, The complex amplitude of the light field containing the target image; The intensity distribution of the target image; Based on the complex amplitude of the light field where the target image is located, the formula is used. Determine the reverse diffraction field of the target image; in, The reverse diffraction field of the target image; For Fourier transform, This is the inverse Fourier transform. For the wavelength of the reference light, The optical transfer function is used in the reverse diffraction process. is the spatial frequency of the diffraction field along the x-axis; denoted as y, which is the spatial frequency of the diffraction field along the y-axis.

Images