A method and system for generating pure phase holograms combined with DNN

Abstract

This invention relates to a method and system for generating pure phase holograms using a deep neural network (DNN). The method includes: acquiring a target image; backpropagating the target image to obtain a diffraction field on the holographic surface; processing the complex amplitude of the diffraction field using a deep neural network to obtain pure phase information; wherein the deep neural network is a generator structure; and reconstructing the image using the angular spectrum method based on the pure phase information to obtain a reconstructed image. This invention can improve the display quality of the reconstructed image.

Description

Technical Field

[0001] The present invention relates to the field of pure phase holography, and in particular to a method and system for generating a pure phase hologram in combination with a DNN. Background Art

[0002] Computational holography can flexibly reconstruct the object light wavefront of various three-dimensional objects including large volumes, large scenes, and even those that do not exist in the real world, and is widely used in the field of three-dimensional display. At present, the SLM used to reconstruct images is mostly pure phase spatial light modulators, so it is necessary to generate high-quality pure phase holograms. The classic GS algorithm requires multiple iterations to generate a hologram with good reproduction quality, which takes too much time. Traditional holographic calculation methods that speed up calculations often approximate the object light wavefront or interference fringes, resulting in low quality of the reproduced image.

[0003] When calculating pure phase holograms, traditional neural networks directly input the target image as input data into the network model, and then directly output pure phase information. The calculation process of the hidden layer of the neural network is mixed with complex operation relationships such as Fourier transform, inverse Fourier transform, and transformation between complex amplitude and pure phase. The network structure is very complex, and the neural network has a weak ability to fit Fourier transform, which leads to poor quality of reconstructed images. When the target image has too many pixels, the traditional method places high demands on the structural design and node parameters of the neural network, and overfitting is prone to occur. Summary of the invention

[0004] The purpose of the present invention is to provide a method and system for generating a pure phase hologram combined with a DNN to improve the display quality of the reconstructed image.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A method for generating a pure phase hologram combined with a DNN, comprising:

[0007] Get the target image;

[0008] Back-propagating the target image to obtain a diffraction field on the holographic surface;

[0009] Processing the complex amplitude of the diffraction field using a deep neural network to obtain pure phase information; the deep neural network is a generator structure;

[0010] Image reconstruction is performed using an angular spectrum method according to the pure phase information to obtain a reconstructed image.

[0011] Optionally, the training process of the deep neural network specifically includes:

[0012] Taking the training complex amplitude of the diffraction field as input, taking the pure phase training information as output, taking the loss between the target image and the reconstructed image as the loss function, the parameters in the deep neural network are optimized and trained to obtain a trained deep neural network; the loss includes pixel loss and perceptual loss.

[0013] Optionally, the expression of the diffraction field is:

[0014]

[0015] Among them, U(x,y;z) is the diffraction field, is the Fourier transform symbol, is the symbol of inverse Fourier transform, is the band-limited transfer function of the target image back propagation, u(x,y) is the real part of the complex amplitude, v(x,y) is the imaginary part of the complex amplitude, j is the imaginary unit, and I(x',y') is the target image.

[0016] Optionally, the expression of the reconstructed image is:

[0017]

[0018] in, To reconstruct the image, is the Fourier transform symbol, is the symbol of inverse Fourier transform, is pure phase information, is the band-limited transfer function for pure phase forward propagation.

[0019] A pure phase hologram generation system combined with DNN, comprising:

[0020] An acquisition module, used for acquiring a target image;

[0021] A reverse propagation module, used for reversely propagating the target image to obtain a diffraction field on the holographic surface;

[0022] A pure phase information determination module, used for processing the complex amplitude of the diffraction field using a deep neural network to obtain pure phase information; the deep neural network is a generator structure;

[0023] The reconstruction module is used to reconstruct the image using the angular spectrum method according to the pure phase information to obtain a reconstructed image.

[0024] Optionally, the training process of the deep neural network specifically includes:

[0025] Taking the training complex amplitude of the diffraction field as input, taking the pure phase training information as output, taking the loss between the target image and the reconstructed image as the loss function, the parameters in the deep neural network are optimized and trained to obtain a trained deep neural network; the loss includes pixel loss and perceptual loss.

[0026] Optionally, the expression of the diffraction field is:

[0027]

[0028] Among them, U(x,y;z) is the diffraction field, is the Fourier transform symbol, is the symbol of inverse Fourier transform, is the band-limited transfer function of the target image back propagation, u(x,y) is the real part of the complex amplitude, v(x,y) is the imaginary part of the complex amplitude, j is the imaginary unit, and I(x',y') is the target image.

[0029] Optionally, the expression of the reconstructed image is:

[0030]

[0031] in, To reconstruct the image, is the Fourier transform symbol, is the symbol of inverse Fourier transform, is pure phase information, is the band-limited transfer function for pure phase forward propagation.

[0032] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0033] The present invention provides a method and system for generating a pure phase hologram in combination with a DNN, which performs reverse propagation of a target image to obtain a diffraction field on a holographic surface; processes the complex amplitude of the diffraction field using a deep neural network to obtain pure phase information; the deep neural network is a generator structure; and reconstructs the image using the angular spectrum method based on the pure phase information to obtain a reconstructed image. The present invention divides the calculation process of the pure phase hologram generation method into two steps. First, the angular spectrum method is used to realize Fourier transform and inverse Fourier transform, and then a deep neural network is used to realize the transformation between complex amplitude and pure phase, which reduces the workload of the neural network, reduces the structural complexity of the network model and the number of node parameters, effectively improves the accuracy of the calculation, and significantly enhances the display quality of the reconstructed image. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0035] Figure 1 A flow chart of the method for generating a pure phase hologram in combination with DNN provided by the present invention;

[0036] Figure 2 A schematic diagram of a method for generating a pure phase hologram in combination with DNN provided by the present invention;

[0037] Figure 3 This is a block diagram of the pure phase hologram generation system combined with DNN provided by the present invention. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] The purpose of the present invention is to provide a method and system for generating a pure phase hologram combined with a DNN to improve the display quality of the reconstructed image.

[0040] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0041] like Figure 1 As shown, the present invention provides a method for generating a pure phase hologram combined with DNN, comprising:

[0042] Step 101: Acquire a target image.

[0043] Step 102: Back-propagate the target image to obtain the diffraction field on the holographic surface.

[0044] Step 103: Process the complex amplitude of the diffraction field using a deep neural network to obtain pure phase information; the deep neural network is a generator structure.

[0045] Step 104: Reconstruct an image using an angular spectrum method according to the pure phase information to obtain a reconstructed image.

[0046] In practical applications, the training process of the deep neural network specifically includes:

[0047] The training complex amplitude of the diffraction field is used as input, the pure phase training information is used as output, and the loss between the target image and the reconstructed image is used as the loss function to optimize the parameters in the deep neural network to obtain a trained deep neural network; the loss includes pixel loss and perceptual loss. The pixel loss is represented by the mean absolute error or square difference between the reconstructed image and the target image; the perceptual loss is determined by the features obtained by convolution of the reconstructed image and the target image respectively, and is the difference between high-level image features including content and global structure.

[0048] In practical applications, the expression of the diffraction field is:

[0049]

[0050] Among them, U(x,y;z) is the diffraction field, is the Fourier transform symbol, is the symbol of inverse Fourier transform, is the band-limited transfer function of the target image back propagation, u(x,y) is the real part of the complex amplitude, v(x,y) is the imaginary part of the complex amplitude, j is the imaginary unit, and I(x',y') is the target image.

[0051] In practical applications, the expression of the reconstructed image is:

[0052]

[0053] in, To reconstruct the image, is the Fourier transform symbol, is the symbol of inverse Fourier transform, is pure phase information, is the band-limited transfer function for pure phase forward propagation.

[0054] like Figure 2 As shown, the present invention also provides a specific workflow of a method for generating a pure phase hologram in combination with DNN in practical applications, and the specific steps are as follows:

[0055] Step 1: Backward propagation of the target image. First, the input target image is back-propagated a certain distance -d to reach the holographic surface. The diffraction field on the holographic surface is described by the following formula:

[0056]

[0057] in is the Fourier transform symbol, is the symbol of inverse Fourier transform, is the band-limited transfer function of the target image back propagation, u(x,y) is the real part of the complex amplitude, v(x,y) is the imaginary part of the complex amplitude, and j is the imaginary unit.

[0058] Step 2: Output pure phase information. The pixel loss is represented by the mean absolute error or squared difference between the reconstructed image and the target image, and the perceptual loss is represented by the difference between the high-level image features including content and global structure obtained by convolution between the reconstructed image and the target image respectively.

[0059] A deep neural network with multiple hidden layers is established, where the neural network structure is a generator structure. The specific structure of the generator can be set by yourself, and some specific convolutional layers are selected to calculate the loss of feature reconstruction. The specific convolutional layers are selected from the VGG network. Pixel loss and perceptual loss are used to represent the loss of the overall network model, so that the neural network can efficiently learn effective information to achieve the purpose of rapid convergence.

[0060] Then the complex amplitude part of the diffraction field on the holographic surface in step 1 is input into the deep neural network as input data. After being processed by multiple hidden layers and a large number of nodes, the corresponding pure phase information is output. During the training process, the neural network will gradually learn the transformation relationship between complex amplitude and pure phase.

[0061] Step 3: Forward propagation of phase information. The phase information calculated by the deep neural network in step 2 is propagated forward a certain distance d, and the reconstructed image formed on the plane where the target image is located is calculated using the ASM (angular spectrum method), which is described by the following formula:

[0062]

[0063] Step 4: Update the network model parameters. Find the loss function between the reconstructed image and the target image in step 3 (i.e. the loss of the overall network model in step 2) This loss function is then used to optimize and update the parameters in the deep neural network using the back-propagation algorithm.

[0064] Step 5: Go to step 2 to generate new phase information again, and repeat the process from step 2 to step 4. In the continuous cycle of training, the deep neural network will gradually learn the transformation relationship between complex amplitude and pure phase until the quality of the reconstructed image can meet the requirements of the initial setting.

[0065] like Figure 3 As shown, the present invention provides a pure phase hologram generation system combined with DNN, comprising:

[0066] The acquisition module 301 is used to acquire a target image.

[0067] The reverse propagation module 302 is used to reversely propagate the target image to obtain the diffraction field on the holographic surface.

[0068] The pure phase information determination module 303 is used to process the complex amplitude of the diffraction field using a deep neural network to obtain pure phase information; the deep neural network is a generator structure.

[0069] The reconstruction module 304 is used to perform image reconstruction using an angular spectrum method according to the pure phase information to obtain a reconstructed image.

[0070] The training process of the deep neural network specifically includes:

[0071] Taking the training complex amplitude of the diffraction field as input, taking the pure phase training information as output, taking the loss between the target image and the reconstructed image as the loss function, the parameters in the deep neural network are optimized and trained to obtain a trained deep neural network; the loss includes pixel loss and perceptual loss.

[0072] Wherein, the expression of the diffraction field is:

[0073]

[0074] Among them, U(x,y;z) is the diffraction field, is the Fourier transform symbol, is the symbol of inverse Fourier transform, is the band-limited transfer function of the target image back propagation, u(x,y) is the real part of the complex amplitude, v(x,y) is the imaginary part of the complex amplitude, j is the imaginary unit, and I(x',y') is the target image.

[0075] Wherein, the expression of the reconstructed image is:

[0076]

[0077] in, To reconstruct the image, is the Fourier transform symbol, is the symbol of inverse Fourier transform, is pure phase information, is the band-limited transfer function for pure phase forward propagation.

[0078] Compared with amplitude holograms, pure phase holograms theoretically have higher diffraction efficiency, so they are the mainstream encoding method for computer-generated holograms. In view of this, the method provided by the present invention converts the hologram generation process into a process of finding the optimal solution based on the optimization theory in the field of computational science; by establishing a neural network model between the bandwidth-limited diffraction complex amplitude wave field and the pure phase wave field on the plane of the spatial light modulator (Spatial Light Modulator, SLM), a large number of high-dimensional parameters are used to find the optimal phase map generation method. This method eliminates the serious artifacts and speckle problems existing in the existing methods, and performs well in the image quality evaluation indicators PSNR and SSIM.

[0079] Compared with the traditional single-step method, the pure phase hologram generation method proposed in the present invention establishes a deep relationship model between the target image and the hologram through machine learning, optimizes the hologram generation algorithm combined with it, and divides the calculation process into two steps. First, the angular spectrum method is used to realize Fourier transform and inverse Fourier transform, and then the deep neural network is used to realize the transformation between complex amplitude and pure phase. This reduces the workload of the neural network, reduces the structural complexity of the network model and the number of node parameters, effectively improves the calculation accuracy, and significantly enhances the display quality of the reconstructed image.

[0080] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0081] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for generating pure phase holograms in combination with DNN, characterized in that: include: Get the target image; Back-propagating the target image to obtain a diffraction field on the holographic surface; The complex amplitude of the diffraction field is processed by a deep neural network to obtain pure phase information; the deep neural network is a generator structure, and the training process of the deep neural network specifically includes: taking the training complex amplitude of the diffraction field as input, taking the pure phase training information as output, taking the loss between the target image and the reconstructed image as a loss function, optimizing and training the parameters in the deep neural network to obtain a trained deep neural network; the loss includes pixel loss and perceptual loss; the pixel loss is represented by the mean absolute error or square difference between the reconstructed image and the target image; the perceptual loss is determined by the features obtained by convolution of the reconstructed image and the target image respectively, and is the difference between high-level image features including content and global structure; The image is reconstructed using the angular spectrum method according to the pure phase information to obtain a reconstructed image, specifically comprising: propagating the phase information calculated by the deep neural network forward a certain distance d, and using the angular spectrum method to calculate a reconstructed image formed on the plane where the target image is located. The expression of the reconstructed image is: in, To reconstruct the image, is the Fourier transform symbol, is the symbol of inverse Fourier transform, is pure phase information, is the band-limited transfer function for pure phase forward propagation.

2. The method for generating a pure phase hologram in combination with a DNN according to claim 1, characterized in that: The expression of the diffraction field is: Among them, U(x,y;z) is the diffraction field, is the Fourier transform symbol, is the symbol of inverse Fourier transform, is the band-limited transfer function of the target image back propagation, u(x,y) is the real part of the complex amplitude, v(x,y) is the imaginary part of the complex amplitude, j is the imaginary unit, and I(x',y') is the target image.

3. A pure phase hologram generation system combined with DNN, characterized in that: include: An acquisition module, used for acquiring a target image; A reverse propagation module, used for reversely propagating the target image to obtain a diffraction field on the holographic surface; A pure phase information determination module is used to process the complex amplitude of the diffraction field using a deep neural network to obtain pure phase information; the deep neural network is a generator structure, and the training process of the deep neural network specifically includes: taking the training complex amplitude of the diffraction field as input, taking the pure phase training information as output, taking the loss between the target image and the reconstructed image as a loss function, optimizing and training the parameters in the deep neural network to obtain a trained deep neural network; the loss includes pixel loss and perceptual loss; the pixel loss is represented by the mean absolute error or square difference between the reconstructed image and the target image; the perceptual loss is determined by the features obtained by convolution of the reconstructed image and the target image respectively, and is the difference between high-level image features including content and global structure; The reconstruction module is used to reconstruct the image using the angular spectrum method according to the pure phase information to obtain a reconstructed image. Specifically, the phase information calculated by the deep neural network is propagated forward a certain distance d, and the angular spectrum method is used to calculate the reconstructed image formed on the plane where the target image is located. The expression of the reconstructed image is: in, To reconstruct the image, is the Fourier transform symbol, is the symbol of inverse Fourier transform, is pure phase information, is the band-limited transfer function for pure phase forward propagation.

4. The pure phase hologram generation system combined with DNN according to claim 3 is characterized in that: The expression of the diffraction field is: Among them, U(x,y;z) is the diffraction field, is the Fourier transform symbol, is the symbol of inverse Fourier transform, is the band-limited transfer function of the target image back propagation, u(x,y) is the real part of the complex amplitude, v(x,y) is the imaginary part of the complex amplitude, j is the imaginary unit, and I(x',y') is the target image.

Images