Wide-viewing-angle light field holographic display method based on time division multiplexing

Abstract

The invention provides a wide-view-angle light field holographic display method based on time division multiplexing. The space and angle resolution information of a light field is dispersed into different time frames, and the information amount of the space and the angle is increased by increasing the frame rate, so that a wider visual angle and a higher resolution are realized. The method comprises the following steps: firstly, acquiring a light field image, and acquiring target light field information by using a light field camera or a computer modeling method as an image source of holographic coding; secondly, information in different angle directions in the light field is coded through the holographic coding technology; dimensionality reduction sparsification is carried out on the light field to avoid frequency spectrum aliasing of the light field holograms at different angles, so that the corresponding light field holograms are generated; and finally, loading the generated light field holograms of the corresponding viewpoints on a spatial light modulator in sequence, switching the holograms in different angle directions through a high frame rate, and realizing time division multiplexing wide-view-angle light field reconstruction by utilizing a persistence of vision effect of human eyes.

Description

Technical Field

[0001] The present invention relates to the technical field of light field holographic display, and in particular to a time-division multiplexed wide-viewing angle light field holographic display method. Background Art

[0002] With the rapid development of modern display technology, light field holographic display has gradually become an important direction for improving the three-dimensional visual experience. Light field holographic display is different from traditional flat display. It reconstructs the complete light field information of the scene, allowing the observer to naturally feel the parallax, depth and focus changes when watching. Compared with traditional display, light field holographic display can more realistically present the stereoscopic effect of three-dimensional scenes, and therefore has broad application potential in virtual reality (VR), augmented reality (AR), medical imaging, remote communication and other fields. This immersive display can not only bring a more natural visual experience, but also is expected to lead the development of display technology to a higher level and meet people's ever-increasing demand for display fidelity and immersion.

[0003] In order to achieve a more realistic light field holographic display, the current main technologies are spatial multiplexing, time division multiplexing and frequency multiplexing. These multiplexing methods encode and decode light field information in the spatial, temporal and frequency domains respectively to improve the viewing angle range, image resolution and display stability under limited hardware conditions. For example, the spatial multiplexing method transmits information of different viewing angles by dividing the display area in space, thereby expanding the viewing angle range and improving the clarity of the image. However, this method is limited by the physical resolution of the display, and it is difficult to achieve high-precision holographic display under existing hardware conditions. The frequency multiplexing method is often used to modulate the color of the light source, and by adjusting the frequency and color output of the light source, high fidelity and resolution optimization of the full-color display can be achieved. However, this method usually has high hardware performance requirements, complex design and high cost, and is not suitable for most application scenarios.

[0004] Among these technologies, the time-division multiplexing method has become a promising method in light field holographic display because it can be modulated in the time dimension and can effectively reduce the hardware burden and cost requirements. The general time-division multiplexing method synthesizes color holograms by displaying RGB channels in time to improve color reproduction. We combine light field information with time frequency and propose a wide-angle light field holographic display method based on time-division multiplexing. The light field information is distributed into different time frames, and the light field viewing angle is widened by increasing the frame rate to improve the spatial bandwidth product utilization. By introducing the multiplexing method, the display system is optimized to achieve a larger viewing angle range and higher resolution under limited hardware resources. This method has significant advantages in improving the viewing angle range and clarity of the display system, and can provide a stable three-dimensional visual experience. The introduction of this method provides a new idea for realizing a more immersive and realistic light field holographic display, and also lays the foundation for promoting light field holographic technology in a wider range of application fields in the future. Summary of the invention

[0005] The object of the present invention is to provide a method for completing light field reconstruction by representing light field information through different time frames in a time division multiplexing manner.

[0006] The present invention adopts the following technical solutions:

[0007] A wide-viewing angle light field holographic display method based on time division multiplexing is characterized in that the spatial and angular resolution information of the light field is dispersed into different time frames, and the amount of spatial and angular information is increased by increasing the frame rate, thereby achieving a wider viewing angle and higher resolution. It includes the following steps:

[0008] ① Light field image acquisition: Use a light field camera or computer modeling method to obtain the target light field information as the image source of the holographic encoding;

[0009] ② Conversion from light field to hologram: Holographic coding technology is used to encode information at different angles in the light field. In order to avoid spectral aliasing of light field holograms at different angles, the light field is dimensionality reduced and sparsely generated to generate the corresponding light field hologram;

[0010] ③ Time-division multiplexed light field reconstruction: The generated light field holograms of the corresponding viewpoints are loaded sequentially on the spatial light modulator, and the holograms of different angles are switched at a high frame rate, and the visual persistence effect of the human eye is utilized to finally achieve wide-viewing angle light field reproduction.

[0011] Furthermore, the light field reconstruction of the time-division multiplexing step ③ can be completed using a spatial light modulator with a higher refresh rate, wherein the spatial light modulator can be a liquid crystal on silicon spatial light modulator (LCOS) or a digital micromirror device (DMD).

[0012] Furthermore, the ② light field to hologram conversion can be encoded using different optimization methods, including:

[0013] (1) Stochastic gradient descent optimization method: The loss function is calculated based on the objective function and the reconstruction result, the weights are updated in the opposite direction of the gradient, and the holographic encoding is optimized by minimizing the loss function;

[0014] (2) Neural network optimization method: Combined with the physical diffraction model, the neural network is trained through the data set to optimize the light field hologram;

[0015] (3) Optimization method of compressed sensing: construct a measurement matrix through sparse sampling to achieve light field reconstruction.

[0016] Furthermore, the light field image acquisition can be achieved in a variety of ways, including: photographing the real scene with a two-dimensional camera array; photographing with a light field camera with a microlens array; using computer modeling software such as Blinder and 3DMAX to construct the scene and generate the light field image.

[0017] Furthermore, the refresh frame rate of the spatial light modulator should meet the continuity requirement of human visual persistence to ensure that users can watch continuous and smooth images, and the refresh frequency should not be less than 24 Hz per second.

[0018] Furthermore, the light field hologram can be in various forms, including: a pure phase hologram, a pure amplitude hologram, or a complex amplitude hologram.

[0019] Furthermore, wide viewing angle display is achieved by controlling the parallax information of the light field in different frames. The specific method includes:

[0020] (1) Parallax adjustment: Calculate the corresponding parallax for different viewpoints to ensure that a true 3D visual effect can be obtained at all angles;

[0021] (2) Angle information encoding: Specific angle information is added during the encoding process of the light field hologram to achieve fine control of viewing angle and parallax.

[0022] Furthermore, the system configuration includes: a light field image acquisition device, a holographic encoding processing module, and a high refresh rate spatial light modulator.

[0023] Beneficial effects:

[0024] 1. This solution effectively disperses the information density by allocating the spatial and angular resolution information of the light field to different time frames, thereby increasing the total amount of spatial and angular information by increasing the time frame. This method increases the display frame rate, allowing the system to display more viewing angles and resolution information per unit time, thereby achieving wider viewing angle light field reconstruction with limited hardware resources and obtaining higher display accuracy.

[0025] 2. The hardware design of this solution is relatively simple. Since different contents are displayed sequentially on the same display panel, the number of required displays is reduced, thereby reducing system complexity and cost.

[0026] 3. It helps to achieve efficient full-color display. By switching different color channels in a fast-changing timing, full-color images can be effectively controlled and displayed, reducing the interference caused by color overlap.

[0027] 4. The solution reduces parallax or artifacts between colors through high-speed switching, ensuring that observers can obtain smooth and stable image display. This is especially beneficial for dynamic holographic displays, which can avoid flickering and delay while ensuring high-quality display.

[0028] This time-reuse-based optimization method has broad application prospects in virtual reality, augmented reality, and holographic displays. It can not only greatly expand the observer's perspective, but also maintain the smoothness and realism of the picture in dynamic scenes, thereby providing users with an immersive three-dimensional experience.

[0029] In summary, time-division multiplexing technology has demonstrated significant advantages in multi-view display, full-color imaging, hardware economy, and artifact reduction, providing an effective means to enhance the immersion and realism of holographic display. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic flow chart of a wide viewing angle light field display method based on time division multiplexing of the present invention;

[0031] Figure 2 is a schematic diagram of a light field image with two-dimensional parallax, (r x ,r y ),(v x ,v y ) represent the spatial resolution and angular resolution of the light field respectively;

[0032] Figure 3 The algorithm flow charts of the holographic encoding processing module of the present invention are (a) random gradient descent algorithm flow chart, (b) neural network algorithm flow chart, and (c) compressed sensing algorithm flow chart;

[0033] Figure 4 The figure is a schematic diagram of the principle of retinal imaging of the human eye according to the wide viewing angle light field display method based on time division multiplexing of the present invention. DETAILED DESCRIPTION

[0034] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0035] Embodiment: A wide-viewing angle light field holographic display method based on time division multiplexing disperses the spatial and angular resolution information of the light field into different time frames, and increases the amount of spatial and angular information by increasing the frame rate, thereby achieving a wider viewing angle and higher resolution. The following steps are included:

[0036] ① Light field image acquisition: Use a light field camera or computer modeling method to obtain the target light field information as the image source of the holographic encoding;

[0037] ② Conversion from light field to hologram: Holographic coding technology is used to encode information at different angles in the light field. In order to avoid spectral aliasing of light field holograms at different angles, the light field is dimensionality reduced and sparsely generated to generate the corresponding light field hologram;

[0038] ③ Time-division multiplexed light field reconstruction: The generated light field holograms of the corresponding viewpoints are loaded sequentially on the spatial light modulator, and the holograms of different angles are switched at a high frame rate, and the visual persistence effect of the human eye is utilized to finally achieve wide-viewing angle light field reproduction.

[0039] The light field reconstruction of the time-division multiplexing method ③ can be completed by using a spatial light modulator with a higher refresh rate, wherein the spatial light modulator can be a liquid crystal on silicon spatial light modulator (LCOS) or a digital micromirror device (DMD).

[0040] Specifically, in this embodiment, a liquid crystal on silicon spatial light modulator (LCOS) is used to load the hologram.

[0041] Furthermore, the ② light field to hologram conversion can be encoded using different optimization methods, including:

[0042] (1) Stochastic gradient descent optimization method: The loss function is calculated based on the objective function and the reconstruction result, the weights are updated in the opposite direction of the gradient, and the holographic encoding is optimized by minimizing the loss function;

[0043] Specifically, the generated hologram H is expressed as:

[0044] H=f SGD {L SGD {U,U target}}

[0045] U=P{H}

[0046] L SGD =αL amp+βL pha

[0047] Where P{·} represents the propagation operator, L SGD To optimize the system loss function, it is expressed as the amplitude loss function L amp And the phase loss function L pha The weighted sum of , α, β represent the weighting coefficients. SGD represents the stochastic gradient descent method.

[0048] (2) Neural network optimization method: Combined with the physical diffraction model, the neural network is trained through the data set to optimize the light field hologram;

[0049] Specifically, the generated hologram H is expressed as:

[0050] H=f Net {L Net {U,U target}}

[0051] Among them, f Net Represents the neural network calculation process, the loss function L Net and other parameters are defined the same as in the stochastic gradient descent method.

[0052] (3) Optimization method of compressed sensing: The measurement matrix is ​​constructed through sparse sampling, the sparse hologram is obtained, and it is inversely transformed into the image domain to achieve light field reconstruction.

[0053]

[0054] y=ΦΨx

[0055] U target =Ψx

[0056] Among them, f optimized represents the optimization system, L CS represents the loss function, x, They respectively represent the sparse hologram and its estimated value, Ψ represents the sparse matrix, Φ represents the measurement matrix (compression matrix), and y represents the observation matrix, which is the compressed encoded hologram data.

[0057] Furthermore, adaptive optimization strategies can be integrated to make the light field holographic encoding process more efficient. These include:

[0058] (1) Adaptive optimization based on loss function: The encoding parameters are adjusted in real time according to the quality of light field reconstruction, so as to reduce the consumption of computing resources while ensuring visual quality.

[0059] (2) Dynamically updated compressed sensing: Dynamically adjust the measurement matrix to better adapt to complex light field scenes and improve reconstruction accuracy.

[0060] (3) Adaptive method based on deep learning: Utilize online training of neural networks and update weights based on real-time feedback to generate high-quality holograms more quickly.

[0061] Specifically, the present embodiment uses a stochastic gradient descent optimization method.

[0062] Furthermore, the light field image acquisition can be achieved in a variety of ways, including: photographing the real scene with a two-dimensional camera array; photographing with a light field camera with a microlens array; using computer modeling software such as Blinder and 3DMAX to construct the scene and generate the light field image.

[0063] Specifically, in this embodiment, Blinder modeling software is used to construct a scene to obtain a light field.

[0064] The refresh frame rate of the spatial light modulator should meet the continuity requirements of human visual persistence to ensure that users can watch continuous and smooth images, and the refresh frequency should not be less than 24Hz per second.

[0065] Furthermore, the refresh rate of the spatial light modulator can be adjusted according to actual needs to achieve visual effects under different application requirements. For example, in low-precision applications such as entertainment and education, the frame rate can be appropriately reduced, while in precision display scenarios, it can be increased to a higher frame rate to enhance the continuity and clarity of the picture.

[0066] Light field holograms can be in many forms, including: pure phase holograms, pure amplitude holograms, or complex amplitude holograms.

[0067] Specifically, a pure phase hologram is generated in this embodiment.

[0068] By controlling the parallax information of the light field in different frames, wide viewing angle display is achieved. Specifically:

[0069] (1) Parallax adjustment: Calculate the corresponding parallax for different viewpoints to ensure that a true 3D visual effect can be obtained at all angles;

[0070] (2) Angle information encoding: Specific angle information is added during the encoding process of the light field hologram to achieve fine control of viewing angle and parallax.

[0071] The system configuration includes: a light field image acquisition device, a holographic encoding processing module, a high refresh rate spatial light modulator, and a frame rate control module.

[0072] Specifically, the holographic encoding module is used to process light field data and generate holograms, supporting multiple encoding methods and algorithm switching. The spatial light modulation module is used to load and display holograms, supporting different types of modulators such as LCOS and DMD. The frame rate control module is used to adjust the system frame rate according to application requirements, supporting multiple adjustable frame rates from 24Hz to 120Hz.

[0073] The implementation process of the present invention is:

[0074] First, the Blinder modeling software is used to establish a light field model to generate a light field image containing multi-angle information, forming a multi-dimensional data set as the system target image. Next, the holographic encoding method is used to encode the information of different angles and directions in the light field image. In order to avoid spectral aliasing between light field holograms of different viewing angles, the light field data is reduced in dimension and sparse. The light field information is mapped to a lower-dimensional space, thereby reducing data redundancy and improving coding efficiency. This process generates light field holograms of multiple corresponding viewpoints, each of which contains information of a specific viewing angle and can effectively express the spatial and depth details of the scene. Then, the generated hologram is loaded onto the spatial light modulator, and the light field holograms of different angles and directions are quickly switched through the frame rate control module to complete the dynamic reproduction of the scene and the perspective switching. The light waves will converge at different positions to form the depth of the light field, and at the same time have a coherent motion parallax, thereby reconstructing a three-dimensional scene on a two-dimensional plane. This solution makes full use of the persistence of vision effect of the human eye. By frequently switching holograms from different viewpoints, it eliminates the feeling of perspective jump and achieves smooth three-dimensional image transition, thus providing a natural and immersive visual experience and meeting the needs of high-quality light field display.

[0075] It should be noted that the above embodiments are not intended to limit the protection scope of the present invention, and equivalent changes or substitutions made on the basis of the above technical solutions all fall within the protection scope of the claims of the present invention.

Claims

1. A wide-viewing angle light field holographic display method based on time division multiplexing, characterized in that: Disperse the spatial and angular resolution information of the light field into different time frames, and increase the amount of spatial and angular information by increasing the frame rate, thereby achieving a wider viewing angle and higher resolution. The following steps are included: ① Light field image acquisition: Use a light field camera or computer modeling method to obtain the target light field information as the image source of the holographic encoding; ② Conversion from light field to hologram: Using holographic coding technology, the information of different angles in the light field is encoded, the light field is dimensionalized and sparsed, and the corresponding light field hologram is generated; ③ Time-division multiplexed light field reconstruction: The generated light field holograms of the corresponding viewpoints are loaded sequentially on the spatial light modulator, and the holograms of different angles are switched at a high frame rate, and the visual persistence effect of the human eye is utilized to finally achieve wide-viewing angle light field reproduction.

2. The wide viewing angle light field holographic display method based on time division multiplexing according to claim 1, characterized in that: The light field reconstruction of ③ time division multiplexing is completed using a spatial light modulator with a higher refresh rate, wherein the spatial light modulator is a liquid crystal on silicon spatial light modulator (LCOS); or a digital micromirror device (DMD).

3. The wide viewing angle light field holographic display method based on time division multiplexing according to claim 1, characterized in that: The ② light field to hologram conversion is encoded using different optimization methods, including: (1) Stochastic gradient descent optimization method: The loss function is calculated based on the objective function and the reconstruction result, the weights are updated in the opposite direction of the gradient, and the holographic encoding is optimized by minimizing the loss function; (2) Neural network optimization method: Combined with the physical diffraction model, the neural network is trained through the data set to optimize the light field hologram; (3) Optimization method of compressed sensing: construct the measurement matrix through sparse sampling, and solve the sampled hologram to achieve light field reconstruction.

4. The wide viewing angle light field holographic display method based on time division multiplexing according to claim 1, characterized in that: The light field image acquisition includes: photographing the real scene through a two-dimensional camera array or photographing through a light field camera with a microlens array or using Blinder, 3DMAX computer modeling software to build a scene and generate a light field image.

5. The wide viewing angle light field holographic display method based on time division multiplexing according to claim 2, characterized in that: The refresh frame rate of the spatial light modulator is not less than 24 Hz per second.

6. The wide viewing angle light field holographic display method based on time division multiplexing according to claim 3, characterized in that: Light field holograms come in many forms, including pure phase holograms, pure amplitude holograms, or complex amplitude holograms.

7. The wide viewing angle light field holographic display method based on time division multiplexing according to claim 1, characterized in that: By controlling the parallax information of the light field in different frames, wide viewing angle display is achieved. The specific method includes: (1) Parallax adjustment: Calculate the corresponding parallax for different viewpoints to ensure that a true 3D visual effect can be obtained at all angles; (2) Angle information encoding: Specific angle information is added during the encoding process of the light field hologram to achieve fine control of viewing angle and parallax.

8. The wide viewing angle light field holographic display method based on time division multiplexing according to claim 1, characterized in that: The system configuration includes: a light field image acquisition device, a holographic encoding processing module, a high refresh rate spatial light modulator, and a frame rate control module.

9. The wide viewing angle light field holographic display method based on time division multiplexing according to claim 3, characterized in that: (1) Stochastic gradient descent optimization method: The loss function is calculated based on the objective function and the reconstruction result, the weights are updated in the opposite direction of the gradient, and the holographic encoding is optimized by minimizing the loss function; Specifically, the generated hologram H is expressed as: H=f SGD {L SGD {U,U target }} U=P{H} L SGD =αL amp +βL pha Where P{·} represents the propagation operator, L SGD To optimize the system loss function, it is expressed as the amplitude loss function L amp And the phase loss function L pha The weighted sum of , α, β represent weighting coefficients, f SGD represents the stochastic gradient descent method, (2) Neural network optimization method: Combined with the physical diffraction model, the neural network is trained through the data set to optimize the light field hologram; Specifically, the generated hologram H is expressed as: H=f Net {L Net {U,U target }} Among them, f Net Represents the neural network calculation process, the loss function L Net and other parameters are defined the same as in the stochastic gradient descent method, (3) Optimization method of compressed sensing: construct the measurement matrix through sparse sampling, obtain the sparse hologram, and inversely transform it to the image domain to achieve light field reconstruction. y=ΦΨx U target =Ψx Among them, f optimized represents the optimization system, L CS represents the loss function, x, They respectively represent the sparse hologram and its estimated value, Ψ represents the sparse matrix, Φ represents the measurement matrix (compression matrix), and y represents the observation matrix, which is the compressed encoded hologram data.

Images