A fourier slice based light field holographic display method

Abstract

The application provides a Fourier slice-based light field holographic near-eye display method, which comprises the following steps: (1) processing a light field by using a Fourier slice, generating a corresponding spatial refocusing picture stack according to the light field, (2) taking the light field refocusing picture stack as an optimization target, generating a corresponding calculated hologram by using an iterative or non-iterative algorithm, (3) loading the hologram on a spatial light modulator, and performing optical diffraction after modulation. The near-eye holographic display using the method has a real spatial scene defocus blur display, and realizes real three-dimensional near-eye display.

Description

Technical Field

[0001] This invention relates to the field of light field holographic display technology, and in particular to a light field holographic display method based on Fourier slices. Background Technology

[0002] With the rapid development of digital media and interactive technologies, from professional virtual reality to everyday multimedia consumption, the demand for higher levels of visual performance and user experience is growing. While traditional 3D display technologies can provide a certain degree of spatial perception, they still have many limitations in creating truly immersive and multi-dimensional interactive experiences. These limitations are mainly reflected in the depth information of the image and the natural representation of the viewing angle. Light field holographic display technology, due to its ability to record and reproduce the directional information of light, is considered one of the key technologies for solving these problems. This technology not only presents realistic 3D scenes but also allows users to view 3D images from different angles without any auxiliary equipment, greatly enhancing the realism and interactivity of the visual experience.

[0003] Nevertheless, the development of light field holographic display technology still faces some technical challenges, especially in the generation and reconstruction of holographic images. Traditional holographic image generation methods, such as phase retrieval algorithms and iterative optimization algorithms, are often computationally complex, inefficient, and prone to information loss when processing large-sized images.

[0004] To address these challenges, a novel solution has been proposed for light field holographic display based on Fourier slices. This method utilizes the principle of Fourier transform to achieve highly efficient holographic computation by decomposing the light field information of a 3D scene into multiple two-dimensional Fourier slices. This approach not only optimizes the computational process but also improves the image reconstruction quality, enabling holographic images to be displayed with higher resolution and less optical distortion, providing technical support for large-size and high-resolution light field displays. With continuous advancements in computer science, algorithm optimization, and optical materials, the light field holographic display method based on Fourier slices foreshadows the future development direction of 3D display technology and will find wide application in various fields such as medical imaging, industrial design, entertainment, and education. Summary of the Invention

[0005] The purpose of this invention is to provide a computational holographic display method that uses the Fourier slicing principle to process light field information for calculating light field holograms.

[0006] The present invention adopts the following technical solution:

[0007] A light field holographic display method based on Fourier slices, the method comprising the following steps:

[0008] Step 1: Process the light field using Fourier slices, and generate a corresponding spatial refocusing image stack based on the light field.

[0009] Step 2: Using the light field refocusing image stack as the target, generate the corresponding computational hologram using iterative or non-iterative algorithms.

[0010] Step 3: Load the hologram onto the spatial light modulator, and then perform optical diffraction after modulation. The specific calculation process is as follows:

[0011] Acquire the light field image, and calculate the two-dimensional imaging of the light field at different distance planes according to the Fourier slice theorem to obtain the required spatial refocusing image stack A1, A2…A n ;

[0012] When using the iterative algorithm: set the initial random phase of the hologram to... Optical diffraction propagation is performed to obtain the reconstructed amplitudes rA1, rA2…rA in multiple planes in space. n To refocus the image stack A1, A2…A n To optimize the objective, the phase Update until the required number of iterations or reconstruction accuracy is met.

[0013] When using a non-iterative algorithm: The algorithm takes the light field image or its Fourier transform as input and generates a hologram corresponding to the light field. After optical diffraction propagation, the amplitude rA1, rA2…rA is reconstructed in space across multiple planes. n For the light field refocusing image stack A1, A2…A n The approximation makes the hologram The required light field is reconstructed in space.

[0014] Furthermore, the specific formula for calculating the Fourier slices is as follows: Where L F Let be the light source, and α be the ratio of the virtual focal length F′ to the original focal length F of the refocusing image plane. For refocusing images, For Fourier transform, For inverse Fourier transform, B -T Basis transformation The inversion of .

[0015] Furthermore, the refocusing distance ratio α satisfies the following condition: |αΔx|≥|(1-α)Δu|, where Δx and Δu are the spatial and directional sampling rates of the light field, respectively.

[0016] Furthermore, holograms Adjacent planes rA after optical diffraction propagation i and rAi+1 The closest distance Δz between them needs to satisfy This Δz also includes the distance to the nearest propagation plane of the SLM, where u is the maximum angular resolution of the optical field and θ is the maximum diffraction angle of the spatial light modulator.

[0017] The iterative algorithm used for further calculation of phase holograms can be the GS iterative algorithm; it can be the stochastic gradient descent method; it can be a heuristic algorithm; when using a non-iterative algorithm, it can be a deep learning algorithm or a machine learning algorithm.

[0018] Furthermore, in computing holograms In free-space diffraction, one of the following methods can be used: angular spectrum diffraction, Fresnel diffraction, or Fraunhofer diffraction. In optical diffraction propagation, low-pass filtering or Gaussian filtering can be used in the frequency domain to accelerate convergence.

[0019] Furthermore, after the hologram is loaded onto a spatial light modulator, the complex amplitude modulated by the spatial light modulator undergoes optical diffraction, enters the optical system, and then enters the human eye. By focusing the eye, the image at the corresponding focused position can be seen. The optical system includes systems with optical filters and systems without optical filters, and can be integrated into a waveguide system to expand the eye box.

[0020] Beneficial effects: 1. This scheme obtains a true 2D refocusing slice of the 4D light field within optical constraints using a Fourier slicing algorithm, and calculates a light field hologram based on this slice. The generated hologram is loaded onto a spatial light modulator, diffracted and propagated through an optical imaging system, generating reconstructed images at different distances that conform to the true focusing effect. After passing through the optical system, it enters the human eye's retina for imaging, achieving a true multi-plane 3D viewing effect with realistic focusing visual effects; 2. This scheme encodes different light field image information into a single hologram, completing the generation of a high-quality hologram. It utilizes Fourier slicing to quickly calculate the light field refocusing image, resulting in a faster calculation speed compared to conventional light field holography calculations; 3. This scheme only needs to consider the amplitude value in the calculation of the target plane, replacing the need for complex amplitude calculations by limiting the amplitude of continuous planes; 4. This scheme is suitable for holographic near-eye displays. After passing through the optical system, it enters the human eye to obtain a realistic and natural focusing effect, achieving a 3D display experience. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the overall process of the calculation method for light field holographic display based on Fourier slices;

[0022] Figure 2 This is a schematic diagram illustrating the relationship between the virtual plane for light field refocusing and the sensor plane.

[0023] Figure 3 A schematic diagram illustrating the nearest neighbor plane constraint requirements for refocusing;

[0024] Figure 4 A near-eye waveguide optical system used to implement Cases 1 and 2. Detailed Implementation

[0025] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0026] Example 1:

[0027] In this example, optical diffraction uses angular spectral diffraction, and the holographic computation algorithm uses the iterative algorithm stochastic gradient descent method.

[0028] Acquire light field images. Based on the parameters of the device used to acquire the light field images, calculate the maximum refocusing depth range that can be calculated under the condition |αΔx|≥|(1-α)Δu| using the Fourier slicing algorithm, and obtain the refocused image stack A1, A2…A n .

[0029] A hologram generation system is constructed using the stochastic gradient descent method, and the refocusing image stack A1, A2…A n As constraint information for the spatial domain, specifically:

[0030]

[0031] ASM zi For a propagation distance of z i The angular spectrum transfer function at time t must also satisfy Δz.

[0032]

[0033] The calculated hologram Loaded onto a spatial light modulator, it undergoes optical diffraction after modulation, and then passes through a... Figure 4 The near-eye waveguide system shown enters the human eye for retinal imaging, comprising: a) a reflective phase-type spatial light modulator, b) a linear polarizer, c) a quarter-wave plate, d) a lens, e) a polarizing holographic in-feed coupling optical element, f) a waveguide, g) a polarizing holographic out-feed coupling optical element, and h) a point light source. The polarizing holographic optical element can be either right-handed or left-handed circularly polarized, with the corresponding rotation direction required by adjusting the quarter-wave plate and the linear polarizer.

[0034] Example 2:

[0035] In this example, optical diffraction uses angular spectral diffraction, and the computational holography algorithm uses a non-iterative algorithm complex UNet neural network for calculation.

[0036] Obtain the light field image dataset. Based on the parameters of the device used to acquire the light field image dataset, use the Fourier slicing algorithm to calculate the maximum refocusing depth range that can be calculated under the condition |αΔx|≥|(1-α)Δu|, and obtain the refocusing image stack dataset, where each light field image has a corresponding refocusing image stack A1, A2…A n .

[0037] A hologram generation system is constructed using a complex UNet network. The network comprises four downsampled encoder modules and corresponding upsampled decoder modules. The encoder and decoder modules form a bottleneck structure. An additional resblock at the input encodes the input light field image, and three consecutive resblocks at the output output the decoded result as hologram information. The light field image dataset and the corresponding refocusing image stack dataset are used as the network training dataset. During training, the light field image is used as the network input, and the corresponding refocusing image stack A1, A2…A… is used as the input. n As constraint information for the spatial domain, specifically:

[0038]

[0039] ASM zi For a propagation distance of z i The angular spectrum transfer function at time t must also satisfy Δz. After the network is trained, a new light field image can be used as input to the network to obtain the desired light field hologram. The reconstructed images rA1, rA2…rA obtained after optical diffraction in space n The desired light field refocusing images A1, A2…A n Approximate to .

[0040] The calculated hologram Loaded onto a spatial light modulator, it undergoes optical diffraction after modulation, and then passes through a... Figure 4 The near-eye waveguide system shown enters the human eye for retinal imaging, comprising: a) a reflective phase-type spatial light modulator, b) a linear polarizer, c) a quarter-wave plate, d) a lens, e) a polarizing holographic in-feed coupling optical element, f) a waveguide, g) a polarizing holographic out-feed coupling optical element, and h) a point light source. The polarizing holographic optical element can be either right-handed or left-handed circularly polarized, with the corresponding rotation direction required by adjusting the quarter-wave plate and the linear polarizer.

[0041] It should be noted that the above embodiments are not intended to limit the scope of protection of the present invention. Equivalent transformations or substitutions made based on the above technical solutions all fall within the scope of protection of the claims of the present invention.

Claims

1. A light field holographic display method based on Fourier slices, characterized in that, The method includes the following steps: Step 1: Process the light field using Fourier slices to generate a corresponding spatial refocusing image stack. Step 2: Using the light field refocusing image stack as the target, generate the corresponding computational hologram using iterative or non-iterative algorithms. Step 3: Load the hologram onto a spatial light modulator, modulate it, and then perform optical diffraction. The calculation process for the light field hologram based on Fourier slices is as follows: Acquire the light field image, and calculate the two-dimensional imaging of the light field at different distance planes according to the Fourier slice theorem to obtain the required spatial refocusing image stack A1, A2…A n ; When using the iterative algorithm: set the initial random phase of the hologram to... Optical diffraction propagation is performed to obtain the reconstructed amplitudes rA1, rA2…rA in multiple planes in space. n To refocus the image stack A1, A2…A n To optimize the target, the phase of the hologram is adjusted. Update until the required number of iterations or reconstruction accuracy is met. When using a non-iterative algorithm: The algorithm takes the light field image or its Fourier transform as input and generates a hologram corresponding to the light field. After optical diffraction propagation, the amplitude rA1, rA2…rA is reconstructed in space across multiple planes. n For the light field refocusing image stack A1, A2…A n Approximate to .

2. The light field holographic display method based on Fourier slices according to claim 1, characterized in that, The Fourier slicing theorem used is based on the following calculation principle: In the spatial domain, the basis of an N-dimensional function is changed using basis transformation B, and then projected onto an M-dimensional plane. Performing a Fourier transform on the resulting function is equivalent to performing a Fourier transform on the N-dimensional function, then performing a basis transformation using the normalized inverse of basis transformation B, and finally slicing it into M dimensions. Specifically, the calculation of the two-dimensional imaging of the light field on the virtual sensor plane in the spatial domain is as follows: Its basis transformation B is: Correspondingly, its normalized inverse is: Where α=F ′ / F, where F is the distance between the sensor and the lens during light field imaging. ′ The distance between the virtual sensor surface and the lens, therefore The corresponding Fourier slices are calculated as follows: in For Fourier transform, L F For light field, This is the operator that projects the light field onto αF.

3. The light field holographic display method based on Fourier slices according to claim 1, characterized in that, The refocusing distance ratio α calculated by the Fourier slice method satisfies the following condition: |αΔx|≥|(1-α)Δu|, where Δx and Δu are the spatial and directional sampling rates of the light field, respectively.

4. The light field holographic near-eye display method based on Fourier slices according to claim 1, characterized in that, Holographic phase Adjacent planes rA after optical diffraction propagation i and rA i+1 The closest distance Δz between them needs to satisfy Where u is the maximum angular resolution of the optical field, and θ is the maximum diffraction angle of the spatial light modulator.

5. The light field holographic display method based on Fourier slices according to claim 1, characterized in that, The iterative or non-iterative algorithm used to compute phase holograms can be one of the following: (1) GS algorithm, (2) stochastic gradient descent, (3) deep learning, (4) heuristic algorithm.

6. The light field holographic display method based on Fourier slices according to claim 1, characterized in that, In calculating holograms The method used in free-space diffraction can be one of angular spectrum diffraction, Fresnel diffraction, or Fraunhofer diffraction.

7. The light field holographic display method based on Fourier slices according to claim 1, characterized in that, In optical diffraction propagation, low-pass filtering or Gaussian filtering is used in the frequency domain to accelerate convergence.

8. The light field holographic display method based on Fourier slices according to claim 1, characterized in that, After the hologram is loaded onto the spatial light modulator, the complex amplitude modulated by the spatial light modulator is diffracted by optical diffraction, and then enters the human eye through the optical system. The human eye can then focus to see the image at the corresponding focus position.

Images