Joint Multi-Lens Array and Neural Network Optimization for 3D Light Field Displays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 3D integral-imaging light field display technologies face challenges in holistically optimizing image quality and viewing experience due to limitations in spatial resolution, angular resolution, field of view, and depth of field, as well as issues like faceting effect, facet braiding, and deterioration of lateral resolution of out-of-focus objects, which prior art methods have not adequately addressed.
Innovation Solution
An end-to-end design framework that jointly optimizes the parameters of the multi-lens array and the image-generation model using a deconvolution neural network, minimizing perceptual loss by comparing output images from different viewing angles to the ideal images, thereby enhancing display quality and viewing experience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional stereoscopic displays are used, then 3D visual information is provided, but vergence-accommodation conflict causes visual fatigue and eyestrain
Solution Approach 1:
The patent changes the fundamental parameters of light field display by using a multi-lens array with optimized focal lengths and lens pitch to generate multiple elemental images that provide both horizontal and vertical parallax. This enables the display to present authentic 3D light fields that naturally accommodate human visual system parameters, resolving the vergence-accommodation conflict inherent in conventional stereoscopic displays.
Solution Approach 2:
The patent transitions from conventional 2D or limited 3D stereoscopic displays to a full 4D light field display by adding the dimension of angular resolution through the multi-lens array. This enables viewers to experience 3D images from multiple viewing angles simultaneously, providing complete depth cues including both horizontal and vertical parallax, thereby eliminating the visual conflicts of traditional stereoscopic methods.
2Ease of manufacture
If multi-lens array parameters are optimized independently from image generation, then manufacturing is simplified, but holistic image quality optimization is limited
Solution Approach 1:
The patent merges the optimization of multi-lens array physical parameters with the image generation algorithm into a unified joint optimization framework. The neural network model simultaneously learns optimal lens parameters (focal length, pitch, curvature) and image rendering strategies, ensuring that both optical hardware and software work together harmoniously to maximize overall image quality rather than being constrained by independent optimization limitations.
3Measurement precision
If spatial resolution is increased, then image detail is improved, but angular resolution and field of view are reduced
Solution Approach 1:
The patent segments the display into multiple elemental images, each viewed from different angular positions through the multi-lens array. By dividing the overall image into discrete angular components, the system can maintain high spatial resolution within each elemental image while simultaneously providing wide angular coverage and large field of view through the collective arrangement of multiple lenslets, effectively resolving the trade-off between resolution and viewing flexibility.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach holistically optimizes performance metrics, improving image quality and minimizing various image quality issues, resulting in superior 3D image display specifications without relying on commercial lens design software simulations.
Implementation Method 1
a projection optics (such as a parallax barrier, a pinhole array, or a multi-lens array (MLA)) for projecting the different directional views
Implementation Method 2
a model for converting a plurality of perspective views to the plurality of elemental images and sending the plurality of elemental images to the pixel array; the model and one or more characteristics of the plurality of lenslets are jointly optimized
Data Source
AI summary
An apparatus has a pixel array, a multi-lens array (MLA) coupled to the pixel array, and circuitry functionally coupled to the pixel array. The pixel array has a plurality of pixels for receiving-and-displaying or sensing-and-outputting a plurality of elemental images. The MLA has a plurality of lenslets. The circuitry has a model for processing the plurality of elemental images. The model and one or more characteristics of the plurality of lenslets are jointly optimized.