Multilayer hologram generation method based on learning layered phase coding and complex valued adversarial network

Abstract

The invention discloses a multilayer hologram generation method based on learning layered phase coding and a complex valued adversarial network, and belongs to the field of deep learning and holographic projection imaging. The problems that an existing phase hologram generation method is difficult to achieve consideration of generation quality and generation speed, and a multi-layer hologram is difficult to generate at a time are solved. The generator network predicts the hologram by learning the complex amplitude in the hologram plane as input, the generator network can process the angular spectrum diffraction value of a single-layer image and can also process the angular spectrum diffraction value of a multi-layer image, the situation that the images are directly processed by using a neural network is avoided, and the image processing efficiency is improved. Therefore, the problem that only a single-layer image can be generated in subsequent network model calling is solved, and one-time generation of a multi-layer hologram can be realized. The data set is utilized to train the complex value adversarial network, the generalization of the model can be improved, and the method can simply and efficiently generate a high-quality hologram. The method can be applied to multi-layer hologram generation.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning and holographic projection imaging, and specifically relates to a multi-layer hologram generation method based on learning hierarchical phase coding and complex-valued adversarial networks. Background Art

[0002] Holographic display technology can completely record and reproduce the wavefront of three-dimensional objects and is a cutting-edge technology for realizing three-dimensional visualization. Computer-generated phase holograms (POH) can generally be divided into two categories: iterative and non-iterative. Typical iterative algorithms include the Gerchberg-Saxton (GS) algorithm and the stochastic gradient descent (SGD) algorithm; typical non-iterative algorithms include the dual phase method (DPM), the error diffusion method, and the random phase method. Iterative algorithms can generate high-quality POH, but the calculation is large and the calculation speed is slow. Non-iterative algorithms do not require a large number of iterative optimizations and have a fast calculation speed, but the quality of the generated POH is not high; generating real-time, high-quality POH is the current key issue. In recent studies, learning-based computational holography algorithms have been used to reconstruct point-based and layer-based scenes.

[0003] JiachenWu et al. proposed a neural network holographic encoder (Holoencoder) based on an autoencoder, and added physical diffraction propagation to the decoding part of the autoencoder. The holographic encoder can automatically learn the encoding of pure phase holograms in an unsupervised manner. The designed holographic encoder can generate high-fidelity 4K resolution holograms within 0.15s. Subsequently, the team proposed a diffraction model-driven network (4K-DMDNet) for 4K computational holography. 4K-DMDNet is a model-driven neural network that can achieve high-quality full-color optical reconstruction of 4K holograms at wavelengths of 450nm, 520nm and 638nm. Chongli Zhong proposed a complex-valued convolutional neural network (CCNN-CGH). The CCNN-CGH algorithm has richer network representation capabilities and is also effective when the network size is smaller. In recent years, the Generative Adversarial Network (GAN) has also been widely used. It introduces an adversarial learning mechanism to improve the quality of network training through mutual competition between the generator and the discriminator. The application of deep learning can quickly generate a single-layer hologram of an image, but when generating multi-layer holograms, it is necessary to call this model at the same time to generate multiple holograms and further synthesize a hologram. This process is time-consuming.

[0004] In summary, it is difficult for existing phase hologram generation methods to achieve a balance between generation quality and generation speed, and it is difficult to generate multi-layer holograms at one time. Therefore, in order to further improve the performance of the model, it is necessary to propose a deep learning model that can quickly and one-time generate high-quality multi-layer planar holograms. Summary of the invention

[0005] The purpose of the present invention is to solve the problem that the existing phase hologram generation method is difficult to achieve a balance between generation quality and generation speed, and it is difficult to generate multiple layers of holograms at one time, and proposes a multi-layer hologram generation method based on learning hierarchical phase encoding and complex-valued adversarial networks.

[0006] The technical solution adopted by the present invention to solve the above technical problems is: a multi-layer hologram generation method based on learning hierarchical phase coding and complex-valued adversarial network, the method specifically comprising the following steps:

[0007] Step 1: Select the distance hologram positions as z 0 ,z 1 ,…,z N-1 The plane with the distance from the hologram position as z n The plane is denoted as a n , n=0,1,…,N-1;

[0008] Construct a complex-valued adversarial network consisting of a generator network and a discriminator network, and obtain the original image dataset used for complex-valued adversarial network training;

[0009] Step 2: Initialize the number of training times l = 1;

[0010] Step 3: Select a plane from the N planes, perform angular spectrum diffraction on the original image according to the selected plane, and obtain the amplitude and phase distribution of the image after angular spectrum diffraction;

[0011] The amplitude and phase distribution of the image angular spectrum after diffraction is used as the input of the generator network in the complex-valued adversarial network, and the hologram is output through the generator network;

[0012] Step 4: The hologram of step 3 is propagated through angular spectrum diffraction to obtain a restored image;

[0013] Step 5: Use the restored image and the original image as the input of the discriminator network in the complex-valued adversarial network, calculate the loss function value according to the restored image, the original image and the output of the discriminator network, and adjust the parameters of the complex-valued adversarial network according to the loss function value;

[0014] Determine whether the training is finished. If the training is finished, execute step 6. If the training is not finished, set l=l+1 and return to execute step 3.

[0015] Step six: select several planes from the N planes in step one, perform angular spectrum diffraction on the image of the hologram to be generated according to each selected plane, add the amplitude and phase distribution after angular spectrum diffraction corresponding to each plane, use the addition result as the input of the trained generator network, and output the hologram through the generator network.

[0016] The beneficial effects of the present invention are:

[0017] The generator network of the present invention predicts the hologram by learning the complex amplitude in the hologram plane as input. The generator network can process the angular spectrum diffraction values ​​of a single-layer image, and can also process the angular spectrum diffraction values ​​of a multi-layer image, avoiding the use of a neural network to directly process the image, thereby solving the problem of only being able to generate a single-layer image in subsequent network model calls. The use of a data set to train a complex-valued adversarial network can improve the generalization of the model. Numerical and optical reconstruction is performed based on the generated hologram, and the PSNR and SSIM of the restored image are calculated. The results show that compared with traditional methods, the method of the present invention can simply and efficiently generate high-quality holograms, thereby improving the quality of the reproduced image, and can also achieve one-time generation of multi-layer holograms. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A flowchart of a multi-layer hologram generation method based on learning hierarchical phase coding and complex-valued adversarial networks;

[0019] Figure 2 It is a schematic diagram of the structure of the generator network;

[0020] Figure 3 Schematic diagram of the structure of the discriminator network;

[0021] Figure 4(a) shows a single-layer hologram generated at Z = 150 mm;

[0022] Figure 4(b) shows a single-layer hologram generated at Z = 160 mm;

[0023] Figure 4(c) shows a single-layer hologram generated at Z = 170 mm;

[0024] Figure 4(d) shows a single-layer hologram generated at Z = 180 mm;

[0025] Figure 4(e) is a single-layer hologram generated at Z = 190 mm;

[0026] Figure 5(a) is the restored image of the single-layer hologram generated at Z = 150 mm;

[0027] Figure 5(b) is the restored image of the single-layer hologram generated at Z = 160 mm;

[0028] Figure 5(c) is the restored image of the single-layer hologram generated at Z = 170 mm;

[0029] Figure 5(d) is the restored image of the single-layer hologram generated at Z = 180 mm;

[0030] Figure 5(e) is the restored image of the single-layer hologram generated at Z = 190 mm;

[0031] Figure 6 Holograms for two planes Z = 150 mm and Z = 190 mm;

[0032] Figure 7(a) shows Figure 6 Reconstructed image of the hologram at Z = 150 mm;

[0033] Figure 7(b) shows Figure 6 Reconstructed image of the hologram at Z = 190 mm;

[0034] Figure 8 Holograms for Z=150mm, Z=170mm and Z=190mm;

[0035] Figure 9(a) shows Figure 8 Reconstructed image of the hologram at Z = 150 mm;

[0036] Figure 9(b) shows Figure 8 Reconstructed image of the hologram at Z = 170 mm;

[0037] Figure 9(c) shows Figure 8 Reconstruction of the hologram at Z=190mm. DETAILED DESCRIPTION

[0038] Specific implementation method 1: Combination Figure 1 The present embodiment describes a method for generating a multi-layer hologram based on learning hierarchical phase coding and a complex-valued adversarial network, and the method specifically comprises the following steps:

[0039] Step 1: Select the distance hologram positions as z 0 ,z 1 ,…,z N-1 The distances between the plane where the hologram is located and the object are z 0 ,z 1 ,…,z N-1 ), the distance hologram position is z n The plane is denoted as a n , n=0,1,…,N-1;

[0040] Construct a complex-valued adversarial network consisting of a generator network and a discriminator network, and obtain the original image dataset used for complex-valued adversarial network training;

[0041] Step 2: Initialize the number of training times l = 1;

[0042] Step 3: Select a plane from the N planes, perform angular spectrum diffraction on the original image (the original image is trained in batches) according to the selected plane, and obtain the amplitude and phase distribution of the image after angular spectrum diffraction;

[0043] The amplitude and phase distribution of the image angular spectrum after diffraction is used as the input of the generator network in the complex-valued adversarial network, and the hologram is output through the generator network;

[0044] Step 4: The hologram of step 3 is propagated through angular spectrum diffraction to obtain a restored image;

[0045] Step 5: Use the restored image and the original image as the input of the discriminator network in the complex-valued adversarial network, calculate the loss function value according to the restored image, the original image and the output of the discriminator network (a probability map can be generated according to each pair of restored image and original image), and adjust the parameters of the complex-valued adversarial network according to the loss function value;

[0046] Determine whether the training is finished. If the training is finished, execute step 6. If the training is not finished, set l=l+1 and return to execute step 3.

[0047] Step six: select several planes from the N planes in step one, perform angular spectrum diffraction on the image of the hologram to be generated according to each selected plane, add the amplitude and phase distribution after angular spectrum diffraction corresponding to each plane, use the addition result as the input of the trained generator network, and output the hologram through the generator network.

[0048] The present invention selects a number of planes from the N planes in step one. The specific number can be 1, 2, 3, ..., N. The number of planes selected is the number of layers of holograms generated in step six.

[0049] Specific implementation method 2: Combination Figure 2 This embodiment is different from the first embodiment in that the structure of the generator network is a unet structure including an encoder and a decoder; the encoder includes a first downsampling module and a second downsampling module, and the decoder includes a first upsampling module and a second upsampling module; wherein:

[0050] Using the input of the generator network as the input of the first downsampling module, and using the output of the first downsampling module as the input of the second downsampling module;

[0051] Using the output of the second down-sampling module as the input of the first up-sampling module;

[0052] Using the output of the first upsampling module and the output of the first downsampling module as inputs of the second upsampling module;

[0053] The inverse tangent is calculated on the output of the second upsampling module and the result is used as the output of the generator network.

[0054] The other steps and parameters are the same as those in the first embodiment.

[0055] Specific implementation method three: Combination Figure 2 This embodiment is different from the first or second embodiment in that the first downsampling module includes a complex-valued convolution layer and a complex-valued ReLU layer, that is, in the first downsampling module, the input passes through the complex-valued convolution layer and the complex-valued ReLU layer in sequence, and the output of the complex-valued ReLU layer is used as the output of the first downsampling module;

[0056] The second down-sampling module has the same structure and working process as the first down-sampling module.

[0057] The other steps and parameters are the same as those in the first or second embodiment.

[0058] Specific implementation method four: Combination Figure 2 This embodiment is different from any one of the first to third embodiments in that the first upsampling module includes a complex-valued transposed convolution layer and a complex-valued ReLU layer, that is, in the first upsampling module, the input is sequentially passed through the complex-valued transposed convolution layer and the complex-valued ReLU layer, and the output of the complex-valued ReLU layer is used as the output of the first upsampling module;

[0059] The second upsampling module only includes a complex-valued transposed convolution layer.

[0060] The other steps and parameters are the same as those in Specific Embodiments 1 to 3.

[0061] Specific implementation method five: Combination Figure 3 This embodiment is different from any one of the specific embodiments 1 to 4 in that the discriminator network includes a first convolution module, a second convolution module, a third convolution module, a fourth convolution module and a fifth convolution module; wherein:

[0062] The first convolution module includes a convolution layer and a LeakyReLU activation layer in sequence;

[0063] The second convolution module includes a convolution layer, a BN layer, and a LeakyReLU layer in sequence;

[0064] The third convolution module includes a convolution layer, a BN layer, and a LeakyReLU layer in sequence;

[0065] The fourth convolution module includes a convolution layer, a BN layer, and a LeakyReLU layer in sequence;

[0066] The fifth convolution module is a convolution layer.

[0067] The other steps and parameters are the same as those in Specific Embodiments 1 to 4.

[0068] Specific implementation method six: This implementation method is different from any one of specific implementation methods one to five in that the optimizer used in training the generator network and the discriminator network is Adam.

[0069] The other steps and parameters are the same as those in Specific Implementation Methods 1 to 5.

[0070] Specific implementation method 7: This implementation method is different from any one of the specific implementation methods 1 to 6 in that the loss function is:

[0071] L G =MSE+0.01*L adv

[0072] Among them, MSE is the mean square error loss between the restored image and the original image, L adv represents the adversarial loss function, L G Represents the total loss.

[0073] The other steps and parameters are the same as those in Specific Embodiments 1 to 6.

[0074] Specific implementation eight: This implementation differs from any one of specific implementations one to seven in that the mean square error loss MSE is:

[0075]

[0076] Among them, A i represents the value of the i-th pixel in the restored image, Represents the value of the i-th pixel in the original image, and M represents the number of pixels in the image.

[0077] The other steps and parameters are the same as those in Specific Embodiments 1 to 7.

[0078] Specific implementation method 9: This implementation method is different from any one of the specific implementation methods 1 to 8 in that the adversarial loss function L adv for:

[0079]

[0080] Among them, D m [G(A)] represents the probability value of the mth element in the probability map D[G(A)] output by the discriminator network (indicating the probability that each local area is true), and M′ represents the total number of elements in the probability map D[G(A)].

[0081] The other steps and parameters are the same as those in Specific Embodiments 1 to 8.

[0082] Specific implementation method ten: This implementation method is different from any one of specific implementation methods one to nine in that the learning rate during training of the generator network and the discriminator network is set to 0.001, and the epoch is set to 30.

[0083] The other steps and parameters are the same as those in Specific Embodiments 1 to 9.

[0084] Example

[0085] This embodiment proposes a multi-layer hologram generation method based on learning hierarchical phase coding and complex-valued adversarial network, and the method specifically includes the following steps:

[0086] Step 1: Select the distance hologram positions as z 0 ,z 1 ,…,z 4 The distances between the plane where the hologram is located and the object are z 0 ,z 1 ,…,z 4 , where z 0 =150mm, z 1 =160mm, z 2 =170mm, z 3 =180mm, z 4 =190mm), and the distance from the hologram position is z n The plane is denoted as a n , n=0,1,…,4;

[0087] Construct a complex-valued adversarial network consisting of a generator network and a discriminator network, and obtain the original image dataset used for complex-valued adversarial network training;

[0088] The structure of the generator network is a unet structure including an encoder and a decoder; the encoder includes a first downsampling module and a second downsampling module, and the decoder includes a first upsampling module and a second upsampling module; the first downsampling module includes a complex-valued convolution layer and a complex-valued ReLU layer, that is, in the first downsampling module, the input passes through the complex-valued convolution layer and the complex-valued ReLU layer in sequence, and the output of the complex-valued ReLU layer is used as the output of the first downsampling module; the first upsampling module includes a complex-valued transposed convolution layer and a complex-valued ReLU layer, that is, in the first upsampling module, the input passes through the complex-valued transposed convolution layer and the complex-valued ReLU layer in sequence, and the output of the complex-valued ReLU layer is used as the output of the first upsampling module; the second upsampling module only includes a complex-valued transposed convolution layer; wherein:

[0089] Using the output of the first downsampling module as the input of the second downsampling module;

[0090] Using the output of the second down-sampling module as the input of the first up-sampling module;

[0091] Using the output of the first upsampling module and the output of the first downsampling module as inputs of the second upsampling module;

[0092] The inverse tangent is calculated on the output of the second upsampling module and the result is used as the output of the generator network.

[0093] The discriminator network includes a first convolution module, a second convolution module, a third convolution module, a fourth convolution module and a fifth convolution module; wherein:

[0094] The first convolution module includes a convolution layer and a LeakyReLU activation layer in sequence;

[0095] The second convolution module includes a convolution layer, a BN layer, and a LeakyReLU layer in sequence;

[0096] The third convolution module includes a convolution layer, a BN layer, and a LeakyReLU layer in sequence;

[0097] The fourth convolution module includes a convolution layer, a BN layer, and a LeakyReLU layer in sequence;

[0098] The fifth convolution module is a convolution layer.

[0099] Step 2: Initialize the number of training times l = 1; the optimizer used for training the generator network and the discriminator network is Adam, the learning rate is set to 0.001, and the epoch is set to 30;

[0100] Step 3: Select a plane from the N planes, perform angular spectrum diffraction on the original image according to the selected plane, and obtain the amplitude and phase distribution of the image after angular spectrum diffraction;

[0101] The amplitude and phase distribution of the image angular spectrum after diffraction is used as the input of the generator network in the complex-valued adversarial network, and the hologram is output through the generator network;

[0102] Step 4: The hologram of step 3 is propagated through angular spectrum diffraction to obtain a restored image;

[0103] Step 5: Use the restored image and the original image as the input of the discriminator network in the complex-valued adversarial network, calculate the loss function value according to the restored image, the original image and the output of the discriminator network, and adjust the parameters of the complex-valued adversarial network according to the loss function value;

[0104] The loss function is:

[0105] L G =MSE+0.01*L adv

[0106] Among them, MSE is the mean square error loss between the restored image and the original image, Ladv represents the adversarial loss function, L G Represents the total loss.

[0107]

[0108] Among them, A i represents the value of the i-th pixel in the restored image, Represents the value of the i-th pixel in the original image, and M represents the number of pixels in the image.

[0109]

[0110] Among them, D m [G(A)] represents the probability value of the mth element in the probability map D[G(A)] output by the discriminator network, and M′ represents the total number of elements in the probability map D[G(A)].

[0111] Determine whether the training is finished (the training data set selected in the present invention is the DIV2K data set, which contains 1000 high-definition images with 2K resolution, 800 of which are used as training sets, 100 as verification sets, and 100 as test sets. The model is trained using the training set. When the training process converges, the training is stopped and verification is started. If the verification passes, the test continues. If the verification fails, the training process is returned to until both the test and verification processes pass or the maximum epoch of the training is reached to obtain the final trained model). If the training is finished, execute step 6. If the training is not finished, set l=l+1 and return to execute step 3.

[0112] Step 6: Select several planes from the N planes in step 1, perform angular spectrum diffraction on the image of the hologram to be generated according to each selected plane, add the amplitude and phase distribution after angular spectrum diffraction corresponding to each plane, use the addition result as the input of the trained generator network, and output the hologram through the generator network;

[0113] For example, calling the trained model, selecting one of the five planes as the input layer of the image to be processed, and entering the generator network after angular spectrum diffraction to obtain a single-layer hologram;

[0114] Call the trained model and select two layers (z 0 =150mm, z 2 =170mm), the two layers of images to be processed are used as the input of the first layer and the second layer respectively, and then the two layers are added after the angular spectrum diffraction to obtain the phase amplitude distribution after diffraction, and then the added phase amplitude distribution is output as a hologram through the generator;

[0115] Call the trained model and select three layers (z 0 =150mm, z2 =170mm, z 4 =190mm), the three layers of images to be processed are used as the input of the first layer, the second layer, and the third layer respectively, and then the three layers are angularly spectra diffracted respectively and then added together to obtain the phase amplitude distribution after diffraction, and then the added phase amplitude distribution is output as a hologram through a generator.

[0116] The method of the present invention only needs to add a discriminator network during the training process, introduce the discriminator network and the generator network, that is, introduce the adversarial mechanism of the complex-valued convolutional neural network. In the model calling process, only the trained generator network needs to be used. During the training process, one of the five planes is randomly trained. Compared with training five planes, it is easier to generate high-quality single-layer holograms and multi-layer holograms in five different positions without increasing the complexity of the network, compared with traditional algorithms.

[0117] All codes of the present invention are implemented using Python and pytorch, and training is performed on a computer with an NVIDIA GeForce RTX 3060 GPU, which takes about 1 hour.

[0118] Experimental Section

[0119] Generation of single-layer holograms: Figure 4(a) is a single-layer hologram generated at Z=150mm, Figure 4(b) is a single-layer hologram generated at Z=160mm, Figure 4(c) is a single-layer hologram generated at Z=170mm, Figure 4(d) is a single-layer hologram generated at Z=180mm, and Figure 4(e) is a single-layer hologram generated at Z=190mm. Their restoration effect diagrams are shown in Figures 5(a), 5(b), 5(c), 5(d), and 5(e), respectively. The corresponding indicators of each layer, Peak Signal-to-noise Ratio (PSNR), Speckle Suppression Index (SSI), and Structural Similarity (SSIM), are calculated to verify the advantages of the method of the present invention.

[0120] The specific calculation formula of PSNR is as follows:

[0121]

[0122] Among them, MAX i It represents the maximum value of the pixels in the restored image, and MSE is the mean square error between the original image and the restored image;

[0123]

[0124] The unit of PSNR is dB. The larger the PSNR value is, the less distortion the restored image has and the higher the similarity between the restored image and the original image is.

[0125] The specific calculation formula of SSIM is as follows:

[0126]

[0127] Among them, μ x , μ y Represent the average values ​​of the pixels in the restored image and the original image, respectively. and Represent the variance of pixel values ​​in the restored image and the original image, σ xy Represents the covariance of the pixel values ​​in the restored image and the original image. SSIM is measured from three aspects: contrast, brightness and structure. Its value range is [0,1]. The larger the SSIM value, the smaller the distortion of the restored image. As shown in Table 1, the numerical evaluation results of the restored single-layer hologram at five distances are:

[0128] Table 1

[0129] <![CDATA[a 0 (z=150mm)]]> <![CDATA[a 1 (z=160mm)]]> <![CDATA[a 2 (z=170mm)]]> <![CDATA[a 3 (z=180mm)]]> <![CDATA[a 4 (z=190mm)]]> PSNR 25.34337799dB 24.4606088dB 23.71771341dB 24.08560371dB 23.93364862dB SSIM 0.881407914 0.874446411 0.860833294 0.866476025 0.878076531

[0130] Table 2

[0131] <![CDATA[a 0 (z=150mm)]]> <![CDATA[a 1 (z=160mm)]]> <![CDATA[a 2 (z=170mm)]]> <![CDATA[a 3 (z=180mm)]]> <![CDATA[a 4 (z=190mm) <!-- 7 -->]]> PSNR 23.14607161 22.84291759 22.73911003 23.16661812 22.94810443 SSIM 0.858684993 0.859491308 0.846118112 0.864926798 0.854927592

[0132] Table 2 lists the PSNR and SSIM calculation results of the generator network without adding an adversarial mechanism, using the MSE of the original image and the restored image to calculate the loss for training under the same circumstances. It can be seen that the best single-layer call effect is to call the first layer, and a restored image with a PSNR of 25.343dB and an SSIM of 0.881 can be obtained. It can be seen from Tables 1 and 2 that the present invention can obtain a restored image with a higher PSNR than the case without an adversarial network, and the PSNR is about 2dB higher than that of an ordinary complex-valued network. It can be seen that the network of the method of the present invention can obtain a better single-layer effect.

[0133] The results of PSNR and SSIM calculations for the two-layer hologram in the embodiment are shown in Table 3. Figure 6 As shown:

[0134] Table 3

[0135] <![CDATA[a 0 (Z=150mm)]]> <![CDATA[a 4 (Z=190mm)]]> PSNR 13.66432738 18.41595919 SSIM 0.11096322 0.175510248

[0136] Similarly, Table 4 lists the results of calculating PSNR and SSIM under the same conditions, using the MSE of the original image and the restored image to calculate the loss for training, and without adding the adversarial mechanism to the generator network;

[0137] Table 4

[0138] <![CDATA[a 0 (z=150mm)]]> <![CDATA[a 4 (z=190mm)]]> PSNR 12.86772028 12.48080064 SSIM 0.070529465 0.076103654

[0139] It can be seen from Tables 3 and 4 that the adversarial mechanism used in the present invention has better performance in generating a two-layer CGH. Figure 6 The hologram is in a 0 (Z = 150 mm), Figure 7(b) is Figure 6 The hologram is in a 4 According to the original image at the initial plane Z=150mm and the restored image of the hologram at Z=150mm, the PSNR and SSIM are calculated respectively, and the numerical results in the first column of Table 3 and Table 4 are obtained, and the second column is obtained in the same way.

[0140] The results of PSNR and SSIM calculations for the three-layer hologram in the embodiment are shown in Table 5:

[0141] Table 5

[0142] <![CDATA[a 0 (z=150mm)]]> <![CDATA[a 2 (z=170mm)]]> <![CDATA[a 4 (z=190mm)]]> PSNR 12.4062152 12.86593616 16.35679608 SSIM 0.069650455 0.075541764 0.108047088

[0143] Similarly, Table 6 lists the results of calculating PSNR and SSIM under the same conditions, using the MSE of the original image and the restored image to calculate the loss for training, and without adding the adversarial mechanism to the generator network;

[0144] Table 6

[0145] <![CDATA[a 0 (z=150mm)]]> <![CDATA[a 2 (z=170mm)]]> <![CDATA[a 4 (z=190mm)]]> PSNR 12.15351394 12.64372872 16.3488307 SSIM 0.055798743 0.060933811 0.098555099

[0146] From the comparison between Table 5 and Table 6, it can be seen that the present invention can also have good performance when generating a three-layer hologram.

[0147] Figure 8 The three-layer planar hologram generated by the present invention, FIG9(a) is Figure 8 The hologram is in a 0 (Z = 150 mm), Figure 9(b) is Figure 8 The hologram is in a 2 (Z = 170 mm), Figure 9(c) is Figure 8 The hologram is in a 4 The restored image of (Z=190mm), the PSNR and SSIM are calculated according to the initial image at the initial plane Z=150mm and the restored image of the hologram at Z=150mm to obtain the numerical results of the first column, and the second and third columns are obtained in the same way.

[0148] The above calculation examples of the present invention are only used to explain the calculation model and calculation process of the present invention in detail, and are not intended to limit the implementation methods of the present invention. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the implementation methods here. All obvious changes or modifications derived from the technical solution of the present invention are still within the scope of protection of the present invention.

Claims

1. A multi-layer hologram generation method based on learning hierarchical phase coding and complex-valued adversarial network, characterized in that: The method specifically comprises the following steps: Step 1: Select the distance hologram positions as z0, z1, …, z N-1 The plane with a distance of z from the hologram position n The plane is denoted as a n , n=0,1,…,N-1; Construct a complex-valued adversarial network consisting of a generator network and a discriminator network, and obtain the original image dataset used for complex-valued adversarial network training; Step 2: Initialize the number of training times l = 1; Step 3: Select a plane from the N planes, perform angular spectrum diffraction on the original image according to the selected plane, and obtain the amplitude and phase distribution of the image after angular spectrum diffraction; The amplitude and phase distribution of the image angular spectrum after diffraction is used as the input of the generator network in the complex-valued adversarial network, and the hologram is output through the generator network; Step 4: The hologram of step 3 is propagated through angular spectrum diffraction to obtain a restored image; Step 5: Use the restored image and the original image as the input of the discriminator network in the complex-valued adversarial network, calculate the loss function value according to the restored image, the original image and the output of the discriminator network, and adjust the parameters of the complex-valued adversarial network according to the loss function value; Determine whether the training is finished. If the training is finished, execute step 6. If the training is not finished, set l=l+1 and return to execute step 3. Step six: select several planes from the N planes in step one, perform angular spectrum diffraction on the image of the hologram to be generated according to each selected plane, add the amplitude and phase distribution after angular spectrum diffraction corresponding to each plane, use the addition result as the input of the trained generator network, and output the hologram through the generator network.

2. According to claim 1, a multi-layer hologram generation method based on learning hierarchical phase coding and complex-valued adversarial network is characterized in that: The structure of the generator network is a unet structure including an encoder and a decoder; the encoder includes a first downsampling module and a second downsampling module, and the decoder includes a first upsampling module and a second upsampling module; wherein: Using the input of the generator network as the input of the first downsampling module, and using the output of the first downsampling module as the input of the second downsampling module; Using the output of the second down-sampling module as the input of the first up-sampling module; Using the output of the first upsampling module and the output of the first downsampling module as inputs of the second upsampling module; The inverse tangent is calculated on the output of the second upsampling module and the result is used as the output of the generator network.

3. The method for generating a multi-layer hologram based on learning hierarchical phase coding and complex-valued adversarial network according to claim 2, characterized in that: The first downsampling module includes a complex-valued convolution layer and a complex-valued ReLU layer, that is, in the first downsampling module, the input passes through the complex-valued convolution layer and the complex-valued ReLU layer in sequence, and the output of the complex-valued ReLU layer is used as the output of the first downsampling module; The second down-sampling module has the same structure and working process as the first down-sampling module.

4. The method for generating a multi-layer hologram based on learning hierarchical phase coding and complex-valued adversarial network according to claim 3, characterized in that: The first upsampling module includes a complex-valued transposed convolution layer and a complex-valued ReLU layer, that is, in the first upsampling module, the input passes through the complex-valued transposed convolution layer and the complex-valued ReLU layer in sequence, and the output of the complex-valued ReLU layer is used as the output of the first upsampling module; The second upsampling module only includes a complex-valued transposed convolution layer.

5. The method for generating a multi-layer hologram based on learning hierarchical phase coding and complex-valued adversarial network according to claim 4, characterized in that: The discriminator network includes a first convolution module, a second convolution module, a third convolution module, a fourth convolution module and a fifth convolution module; wherein: The first convolution module includes a convolution layer and a LeakyReLU activation layer in sequence; The second convolution module includes a convolution layer, a BN layer, and a LeakyReLU layer in sequence; The third convolution module includes a convolution layer, a BN layer, and a LeakyReLU layer in sequence; The fourth convolution module includes a convolution layer, a BN layer, and a LeakyReLU layer in sequence; The fifth convolution module is a convolution layer.

6. The method for generating a multi-layer hologram based on learning hierarchical phase coding and complex-valued adversarial network according to claim 5, characterized in that: The optimizer used in training the generator network and the discriminator network is Adam.

7. The method for generating a multi-layer hologram based on learning hierarchical phase coding and complex-valued adversarial network according to claim 6, characterized in that: The loss function is: L G =MSE+0.01*L adv Among them, MSE is the mean square error loss between the restored image and the original image, L adv represents the adversarial loss function, L G Represents the total loss.

8. The method for generating a multi-layer hologram based on learning hierarchical phase coding and complex-valued adversarial network according to claim 7, characterized in that: The mean square error loss MSE is: Among them, A i represents the value of the i-th pixel in the restored image, Represents the value of the i-th pixel in the original image, and M represents the number of pixels in the image.

9. The method for generating a multi-layer hologram based on learning hierarchical phase coding and complex-valued adversarial network according to claim 8, characterized in that: The adversarial loss function L adv for: Among them, D m [G(A)] represents the probability value of the mth element in the probability map D[G(A)] output by the discriminator network, and M′ represents the total number of elements in the probability map D[G(A)].

10. The method for generating a multi-layer hologram based on learning hierarchical phase coding and complex-valued adversarial network according to claim 9, characterized in that: The learning rate during training of the generator network and the discriminator network is set to 0.001, and the epoch is set to 30.

Images