High-image-quality light field near-to-eye display system and aberration pre-correction method

Abstract

The invention provides a high-image-quality light field near-eye display system and an aberration pre-correction method, which cooperatively solve the problem of poor display imaging quality caused by boundary and system aberration of element views in light field near-eye display through the controllable diffusion effect of a holographic function screen and the correction effect of an aberration pre-correction model. Boundaries between element views are eliminated through the holographic function screen, continuous motion parallax and natural visual experience are provided, an eye box and a field angle are enlarged to a certain extent, and immersion is enhanced; the real three-dimensional characteristic of light field display is kept while high image quality is guaranteed, accurate focus prompt can be provided, visual fatigue is effectively relieved, and the VAC problem is solved.

Description

Technical Field

[0001] This invention belongs to the field of near-eye display technology, specifically relating to a high-quality light field near-eye display system and an aberration pre-correction method. Background Technology

[0002] In recent years, near-eye displays have become a research hotspot. However, mainstream commercial near-eye displays typically construct only a single depth plane, leading to a mismatch between the focal depth of the human eye's lens and the convergence depth of the binoculars, causing Vergence-Accommodation Conflict (VAC) and resulting in visual fatigue. Light field display schemes have attracted considerable attention due to their ability to provide realistic light field information. In traditional light field near-eye display systems, microlens arrays are typically used as light field modulation elements. However, existing systems suffer from the following significant bottlenecks: Low spatial resolution and perceptual discontinuity: In traditional microlens array schemes, there are obvious boundaries between element views, resulting in low spatial resolution and perceptual discontinuity in 3D images, as well as image distortion. Optical aberrations: The inherent single-lens structure of the microlens unit introduces wavefront aberrations, causing blurred reconstructed images and affecting the accuracy of focus cues. While existing research has optimized distortion to some extent, it has not systematically addressed the aforementioned resolution, continuity, and aberration issues simultaneously at the physical optical path and algorithmic levels. Summary of the Invention

[0003] In view of this, the purpose of this invention is to provide a high-quality light field near-eye display system and aberration pre-correction method, which aims to solve the problems of discontinuous element view perception, low fill factor and poor imaging quality caused by aberrations introduced by microlens arrays in the prior art, and proposes a hybrid solution combining optics and algorithms.

[0004] A light field near-eye display system includes: a microdisplay, a microlens array, a holographic functional screen, an eyepiece, and an aberration pre-correction model;

[0005] The microdisplay is used to load and display a primitive image array; The microlens array is disposed on the light-emitting side of the microdisplay and is used to shape the light wavefront emitted by the primitive image array through refraction, converting it from pixel position information into angle information to reconstruct the light field. The holographic functional screen is located on the imaging surface behind the microlens array and is used to controllably diffuse the light field. The eyepiece is located on the light-emitting side of the holographic screen and is used to amplify the reconstructed three-dimensional light field and project it onto the human eye. The aberration pre-correction model serves as the data processing front end of the system. It receives the initial primitive image array, compensates for the wavefront aberrations introduced by the microlens array, and then outputs the pre-corrected primitive image array to the microdisplay.

[0006] Preferably, the holographic functional screen distributes the light beam at a defined diffusion angle θ; the diffusion angle θ satisfies the following relationship: ;in, The spacing between the microlens units in the microlens array is [missing information]. The distance between the microlens array and the holographic functional screen is denoted as .

[0007] Preferably, the aberration pre-correction model employs an autoencoder architecture based on a convolutional neural network, comprising an encoder for feature extraction and a decoder for image reconstruction. The architecture also integrates skip connections and batch normalization: the skip connections pass feature maps from the encoder to the corresponding layers of the decoder, thereby constructing a residual learning structure that allows the network to focus on learning compensation residuals for aberrations, preserving image edge and texture details; the batch normalization layer, placed after the convolutional neural network, normalizes the feature distribution to alleviate the gradient vanishing problem when fitting complex nonlinear aberrations, enabling the network to support larger learning rates and accelerate convergence.

[0008] Preferably, the aberration pre-correction model specifically adopts an autoencoder architecture based on the U-Net convolutional neural network structure: its encoder consists of 5 cascaded downsampling convolutional modules, with the number of feature channels set sequentially to 32, 64, 128, 256 and 512, extracting high-dimensional features and compressing spatial resolution through stride convolution; its decoder consists of 5 corresponding upsampling transposed convolutional modules, with the number of feature channels decreasing in reverse order from 512 to 32, used to restore image size; the convolutional neural network constructs skip connections between the same-resolution layers of the encoder and decoder, concatenating the shallow features of the encoder with the deep features of the decoder in the channel dimension to preserve edge details; in addition, each convolutional layer is configured with a batch normalization layer and a non-linear activation function Leaky ReLU, and finally the output layer generates a pre-correction primitive image array with limited pixel values.

[0009] An aberration pre-correction method based on the above-mentioned light field near-eye display system includes: Step S1: Obtain the initial primitive image array of the target 3D scene; Step S2: Input the initial primitive image array into the trained aberration pre-correction model and output the pre-corrected primitive image array; Step S3: Load the pre-calibrated primitive image array onto the microdisplay; Step S4: The light emitted by the microdisplay is modulated and converged to the holographic functional screen by the microlens array. The light is reconstructed by the holographic functional screen located on the plane and finally magnified by the eyepiece to form a 3D image. The human eye observes the target 3D scene through the eyepiece.

[0010] Preferred methods for training aberration precorrection models include: Step S1: Obtain the initial primitive image of the target 3D scene as a dataset; Step S2: Analyze the optical aberrations of the microlens based on Zernike polynomials and calculate the point spread function array; Step S3: Convolve the pre-corrected primitive image output by the network with the point spread function array to simulate the effect of aberration on imaging; Step S4: Introduce the nearest neighbor interpolation algorithm to simulate the magnification effect of the microlens and obtain a simulated display primitive image; Step S5: Calculate the structural similarity index between the simulated display primitive image and the target image, and use it as a loss function to optimize the model.

[0011] The present invention has the following beneficial effects: 1. This invention provides a high-quality light field near-eye display system, which, through the controllable diffusion effect of the holographic functional screen and the correction effect of the aberration pre-correction model, synergistically solves the problem of poor display imaging quality caused by the existence of boundary and system aberration in the element view of light field near-eye display.

[0012] 2. This invention provides a high-resolution light field near-eye display system that eliminates the boundaries between element views through the holographic functional screen, providing continuous motion parallax and a natural visual experience, while also expanding the eye box and field of view to a certain extent, enhancing the sense of immersion.

[0013] 3. This invention provides a high-quality light field near-eye display system that, while ensuring high image quality, retains the true three-dimensional characteristics of light field display, provides accurate focus cues, effectively alleviates visual fatigue, and solves the VAC problem. Attached Figure Description

[0014] Figure 1 A simplified structural diagram of a high-resolution light field near-eye display system; Figure 2 A schematic diagram of the fill factor for the view on the exit pupil plane; Figure 3 This is a schematic diagram illustrating the usage process of a high-resolution light field near-eye display system. Figure 4 This is a schematic diagram of the display principle of a high-resolution light field near-eye display system. Figure 5 This is a schematic diagram of the training process for the aberration pre-correction model. Detailed Implementation

[0015] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0016] like Figure 1 As shown, the present invention provides a high-quality light field near-eye display system, comprising: a microdisplay, a microlens array, a holographic functional screen, an eyepiece, and an aberration pre-correction model.

[0017] The microdisplay is used to load and display a primitive image array; The microlens array is disposed on the light-emitting side of the microdisplay and is used to shape the light wavefront emitted by the primitive image array through refraction, converting it from pixel position information into angle information to reconstruct the light field. The holographic functional screen is located on the imaging surface behind the microlens array and is used to controllably diffuse the light field. The eyepiece is located on the light-emitting side of the holographic screen and is used to amplify the reconstructed three-dimensional light field and project it onto the human eye. The aberration pre-correction model serves as the data processing front end of the system. It receives the initial primitive image array, and after processing by the model, it outputs the pre-corrected primitive image array.

[0018] In this embodiment, the microdisplay employs a 3.59-inch liquid crystal display screen with a pixel pitch of 16.8 micrometers and a total of 3840×3744 color pixels. The primitive image array displayed on the microdisplay is a pre-corrected primitive image array processed by an aberration pre-correction model; the pre-corrected primitive image array is compensated for wavefront aberrations introduced by the microlens array.

[0019] The distance between the microdisplay and the microlens array is greater than the focal length of the microlens unit, causing light rays from the primitive image array to converge at the location of the holographic functional screen after being refracted by the microlens array, forming a real image. In this embodiment, the focal length of the microlens unit is 3.3 mm, the distance between the microdisplay and the microlens array is 4 mm, and the corresponding lateral magnification is 5.

[0020] The holographic functional screen is an optical element made by exposing a speckle pattern on a photosensitive material using holographic technology. It is used to controllably diffuse the incident light field, so that the light beam is distributed at a certain diffusion angle θ; the diffusion angle θ satisfies the following relationship: .in, The spacing between the microlens units in the microlens array is [missing information]. The distance between the microlens array and the holographic screen is 1 mm. In this embodiment, the spacing between the microlens units is 1 mm, the distance between the microlens array and the holographic screen is 20 mm, and the diffusion angle of the holographic screen is 3.5°.

[0021] like Figure 1 , 2As shown, the introduction of the holographic functional screen makes the fill factor Pf of the system's view on the exit pupil plane greater than 1. Figure 2 As shown, the fill factor Pf is defined as the ratio of the diameter D of the light spot projected onto the exit pupil plane by a single pixel's light beam to the distance d between adjacent views: Pf = D / d. In this embodiment, the fill factor of the views on the exit pupil plane is increased to 1.21. By making the fill factor greater than 1, the boundaries between element views are weakened, the image distortion caused by the microlens array structure is corrected, and the eyebox and field of view are improved to some extent.

[0022] The system should receive at least two views on the exit pupil plane to enable stereoscopic vision and resolve the VAC problem. In this embodiment, the system receives four views on the exit pupil plane.

[0023] like Figure 3 As shown, the high-resolution light field near-eye display system should include the following steps when in use: Step S1: Obtain an initial primitive image array of the target 3D scene; Step S2: Input the initial primitive image array into a trained aberration pre-correction model and output a pre-corrected primitive image array; Step S3: Load the pre-corrected primitive image array onto the microdisplay; Step S4: As shown Figure 4 As shown, the light emitted from the microdisplay is modulated and converged onto the holographic functional screen by a microlens array. The light is then reconstructed in the wavefront by the holographic functional screen located on this plane, and finally magnified by the eyepiece to form a 3D image. The human eye observes the target 3D scene through the eyepiece. The initial primitive image array and the pre-correction primitive image array are respectively composed of an initial primitive image and a pre-correction primitive image.

[0024] The aberration pre-correction model employs an autoencoder architecture based on a convolutional neural network, primarily consisting of an encoder for feature extraction and a decoder for image reconstruction. To improve model performance, the network integrates skip connections and batch normalization: the skip connections pass feature maps from the encoder to the corresponding layers of the decoder, thereby constructing a residual learning structure that allows the network to focus on learning compensation residuals for aberrations rather than reconstructing the entire image, effectively preserving image edge and texture details; the batch normalization layer, placed after the convolutional layers, normalizes the feature distribution to alleviate the gradient vanishing problem when fitting complex nonlinear aberrations, enabling the network to support larger learning rates and significantly accelerate convergence.

[0025] In this embodiment, the aberration pre-correction model specifically adopts an autoencoder architecture based on the U-Net convolutional neural network structure: its encoder consists of 5 cascaded downsampling convolutional modules, with the number of feature channels set sequentially to 32, 64, 128, 256, and 512, extracting high-dimensional features and compressing spatial resolution through stride convolution; its decoder consists of 5 corresponding upsampling transposed convolutional modules, with the number of feature channels decreasing in reverse order from 512 to 32, used to restore image size. The convolutional neural network constructs skip connections between the same-resolution layers of the encoder and decoder, concatenating the shallow features of the encoder with the deep features of the decoder in the channel dimension to preserve edge details; in addition, each convolutional layer is followed by a batch normalization layer and a non-linear activation function Leaky ReLU, and finally, a pre-correction primitive image array with pixel-limited values ​​is generated through the output layer.

[0026] like Figure 5 As shown, the training steps of the aberration pre-correction model include: Step S1: Obtain the initial primitive image of the target 3D scene as a dataset; Step S2: Analyze the optical aberrations of the microlens based on Zernike polynomials and calculate the point spread function array; Step S3: Convolve the pre-corrected primitive image output by the network with the point spread function array to simulate the effect of aberration on imaging; Step S4: Introduce the nearest neighbor interpolation algorithm to simulate the magnification effect of the microlens and obtain a simulated display primitive image; Step S5: Calculate the structural similarity index between the simulated display primitive image and the target image, and use it as a loss function to optimize the network. The loss function optimizes the network parameters by minimizing the structural similarity index error between the simulated display primitive image and the target primitive image, thereby enabling the generated pre-corrected primitive image array to inversely compensate for the wavefront aberrations generated by the microlens array.

[0027] In this embodiment, the dataset consists of approximately 4,000 initial primitive images. The training loss stabilizes after approximately 68,000 iterations, indicating that the aberration pre-correction model has fully converged. The entire training takes approximately 20,000 seconds, the model parameter file size is 97.8 MB, and the average structural similarity index of the dataset is increased from 0.9215 to a high-fidelity 0.9801.

[0028] like Figure 4 As shown, by using the aforementioned high-resolution light field near-eye display system, users can accurately reconstruct scene depth and provide precise focus cues. In this embodiment, the system has a monocular diagonal field of view of approximately 70° and an exit pupil diameter of 7mm, enabling clear reconstruction of complex 3D scenes within a depth range exceeding 3 diopters.

[0029] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A light field near-eye display system, characterized in that, include: Microdisplays, microlens arrays, holographic functional screens, eyepieces, and aberration pre-correction models; The microdisplay is used to load and display a primitive image array; The microlens array is disposed on the light-emitting side of the microdisplay and is used to shape the light wavefront emitted by the primitive image array through refraction, converting it from pixel position information into angle information to reconstruct the light field. The holographic functional screen is located on the imaging surface behind the microlens array and is used to controllably diffuse the light field. The eyepiece is located on the light-emitting side of the holographic screen and is used to amplify the reconstructed three-dimensional light field and project it onto the human eye. The aberration pre-correction model serves as the data processing front end of the system. It receives the initial primitive image array, compensates for the wavefront aberrations introduced by the microlens array, and then outputs the pre-corrected primitive image array to the microdisplay.

2. The light field near-eye display system as described in claim 1, characterized in that, The holographic screen distributes the light beam at a defined diffusion angle θ; the diffusion angle θ satisfies the following relationship: ;in, The spacing between the microlens units in the microlens array is [missing information]. The distance between the microlens array and the holographic functional screen is denoted as .

3. The light field near-eye display system as described in claim 1, characterized in that, The aberration pre-correction model employs an autoencoder architecture based on a convolutional neural network, comprising an encoder for feature extraction and a decoder for image reconstruction. The architecture also integrates skip connections and batch normalization: the skip connections pass feature maps from the encoder to the corresponding layers of the decoder, thereby constructing a residual learning structure that allows the network to focus on learning compensation residuals for aberrations, preserving image edge and texture details; the batch normalization layer, placed after the convolutional neural network, normalizes the feature distribution to alleviate the gradient vanishing problem when fitting complex nonlinear aberrations, enabling the network to support larger learning rates and accelerate convergence.

4. The light field near-eye display system as described in claim 3, characterized in that, The aberration precorrection model specifically adopts an autoencoder architecture based on the U-Net convolutional neural network structure: its encoder consists of 5 cascaded downsampling convolutional modules, with the number of feature channels set to 32, 64, 128, 256 and 512 respectively. High-dimensional features are extracted step by step through stride convolution and spatial resolution is compressed. Its decoder consists of 5 corresponding upsampled transposed convolutional modules, with the number of feature channels decreasing in reverse order from 512 to 32, used to restore the image size; the convolutional neural network constructs skip connections between the encoder and decoder at the same resolution, concatenating the shallow features of the encoder with the deep features of the decoder in the channel dimension to preserve edge details; in addition, each convolutional layer is followed by a batch normalization layer and a non-linear activation function Leaky ReLU, and finally the output layer generates a pre-corrected primitive image array with limited pixel values.

5. An aberration pre-correction method for a near-eye display system based on claim 1, characterized in that, include: Step S1: Obtain the initial primitive image array of the target 3D scene; Step S2: Input the initial primitive image array into the trained aberration pre-correction model and output the pre-corrected primitive image array; Step S3: Load the pre-calibrated primitive image array onto the microdisplay; Step S4: The light emitted by the microdisplay is modulated and converged to the holographic functional screen by the microlens array. The light is reconstructed by the holographic functional screen located on the plane and finally magnified by the eyepiece to form a 3D image. The human eye observes the target 3D scene through the eyepiece.

6. The aberration pre-correction method as described in claim 5, characterized in that, Methods for training aberration precorrection models include: Step S1: Obtain the initial primitive image of the target 3D scene as a dataset; Step S2: Analyze the optical aberrations of the microlens based on Zernike polynomials and calculate the point spread function array; Step S3: Convolve the pre-corrected primitive image output by the network with the point spread function array to simulate the effect of aberration on imaging; Step S4: Introduce the nearest neighbor interpolation algorithm to simulate the magnification effect of the microlens and obtain a simulated display primitive image; Step S5: Calculate the structural similarity index between the simulated display primitive image and the target image, and use it as a loss function to optimize the model.

Images