Subsampled Image Merge Masks for Distorted Display Optics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing imaging systems face challenges in merging multiple image sources in a distorted image space, requiring complex per-pixel masks that consume significant processor and FPGA resources, and existing solutions compromise image quality or bandwidth.
Innovation Solution
A method using sub-sampled merge masks defined by sub-sampling nodes with adjustable granularity, allowing for efficient resource utilization by employing look-up tables to apply masks with varying densities based on distortion levels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a per pixel mask is used to accurately represent the merge region in distorted image space, then the precision of region definition is improved, but the processor and FPGA resource consumption increases significantly
Solution Approach 1:
The mask is segmented into multiple tiles, each covering a specific region of the distorted image space. Each tile contains only the subset of distorted coordinates needed for that region, reducing the overall mask size while maintaining precision where needed. The processor divides the full mask into manageable tile segments that can be processed independently.
Solution Approach 2:
Different regions of the mask are processed with different levels of detail and resource allocation. High-priority regions with complex distortion patterns receive more processing attention and finer granularity, while lower-priority regions use coarser sampling. This allows precise region definition in critical areas while reducing overall computational load.
2Adaptability or versatility
If a per pixel mask is used to accommodate any shape region in distorted image space, then the adaptability of region shaping is improved, but the bandwidth required to access the mask increases
Solution Approach 1:
The mask data is segmented into multiple tiles that can be loaded and processed independently. Each tile contains a subset of the full mask data corresponding to a specific spatial region, reducing the bandwidth required to access the complete mask at any given time while maintaining the ability to define arbitrary region shapes.
Solution Approach 2:
The mask is organized in a tiled spatial structure that adds a hierarchical dimension to data access. Instead of accessing a single large flat mask structure, the system accesses multiple smaller tile structures arranged in a grid, reducing memory bandwidth requirements through spatial locality and parallel access patterns.
3Device complexity
If image subsampling is applied to reduce computational load, then the resource consumption is reduced, but the fusion quality may be compromised
Solution Approach 1:
Subsampling is applied selectively based on local distortion characteristics. Regions with high distortion gradients and complex boundaries maintain higher sampling density to preserve fusion quality, while regions with smooth distortion patterns use coarser sampling to reduce computational load. The system dynamically adjusts local subsampling rates to maintain quality where needed.
Solution Approach 2:
The subsampling rate is made dynamic rather than fixed, allowing the system to adapt the level of detail based on local image characteristics, distortion severity, and computational resource availability. This enables the system to maintain high fusion quality in critical regions while reducing overall computational requirements through adaptive rate adjustment.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
A system for merging a plurality of image sources into a single image stream defines a predistortion image mask for each source. Each mask defines a distorted image space to account for distortion in the display optics, and indicates a region or regions in the desired output image where one merge method should be used versus another (e.g., overlay vs. replace). Each source mask may have a separate level of granularity according to the requirements of the included image and corresponding distortion.