Constrained Rendering for Artificial Reality Displays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial reality systems face artifacts due to differences in subpixel sizes, shapes, and layouts across color channels, leading to poor image quality and user experience, especially when eye movement causes temporal misalignment during sequential display of color channels.
Innovation Solution
A constrained rendering framework that uses a 2D matrix filter to optimize pixel values and a constraint matrix to align subpixels, minimizing mean-squared error in the opponent color space by dynamically adjusting filter values based on eye tracking data and subpixel geometries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If subpixels of different color channels have different sizes, shapes, and layouts, then the display can be manufactured with current technology, but artifacts appear in displayed images due to misalignment
Solution Approach 1:
The patent changes the parameter space by transforming from RGB color space to opponent color space (LMS or XYZ), allowing independent optimization of luminance and chrominance components. This enables separate handling of subpixel alignment issues for different color channels through mathematical transformation rather than physical realignment
Solution Approach 2:
The patent introduces an intermediary processing stage that includes: (1) measuring actual subpixel positions, (2) calculating misalignment offsets, (3) applying offset compensation in the opponent color space, and (4) transforming back to RGB for display. This intermediary process acts as a mediator between the physical display hardware and the final perceived image
2Device complexity
If color channels are displayed sequentially, then power consumption and device complexity are reduced, but temporal misalignment artifacts occur during eye movement
Solution Approach 1:
The patent performs preliminary measurement of subpixel positions and calculation of misalignment offsets before actual image display. The offset compensation parameters are pre-computed and stored, then applied during sequential color channel display without requiring real-time recalculation, enabling compensation for temporal misalignment while maintaining sequential display benefits
Solution Approach 2:
The patent implements a feedback mechanism where eye tracking data is used to detect actual eye movement, and this information feeds back to adjust the offset compensation parameters dynamically. This closed-loop feedback system maintains temporal alignment precision even during eye movement while preserving the simplified sequential display approach
3Manufacturing precision
If filter values are optimized for minimum error, then image quality improves, but computational complexity and processing time increase
Solution Approach 1:
The patent performs filter optimization in advance during system initialization or calibration phase, computing the optimal filter values that minimize mean-squared error in the opponent color space. These pre-optimized filter values are then stored and applied directly during normal operation without requiring real-time optimization, reducing computational complexity while maintaining image quality precision
Solution Approach 2:
The patent changes the optimization objective from minimizing error in RGB space to minimizing mean-squared error in opponent color space (LMS or XYZ). This parameter space transformation simplifies the optimization problem by decoupling luminance and chrominance errors, allowing more efficient computation of optimal filter values while achieving better perceived image quality
Data Source
AI summary
In one embodiment, a computing system may determine a group of subpixels, that are associated with different color channels, within a display region of a display. The system may determine a micro-pixel corresponding to a basic unit shape configured to evenly divide the display region and each subpixel. The system may represent the display region as a group of micro-pixels and each subpixel as a combination of one or more micro-pixels in the group of micro-pixels. The system may determine a constraint for each color channel of the display region based on the group of micro-pixels. The constraint may constrain the micro-pixels associated with a same subpixel to have a same color value. The system may generate, based on an optimization process using the constraint, a filter for the display region. The filter may be configured to adjust image pixel values to be displayed by the group of subpixels.