Multi-resolution 3D Video Rendering via Proxy Region Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for 3D view synthesis using projective mapping techniques require high memory bandwidth, computational complexity, and data rate due to processing entire reference pixels, leading to increased memory and computational requirements, especially in multi-view video rendering applications.
Innovation Solution
The solution involves determining a proxy image region larger than the target image region by a margin, selecting reference pixels that fit within this proxy region from multiple reference viewpoints, projecting these pixels to the target image region, and compositing rendered regions to create video rendering, thereby reducing data rate and computational complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If entire reference pixels are processed in 3D warping, then rendering quality is improved, but memory bandwidth and computational complexity increase significantly
Solution Approach 1:
The patent segments the reference image into multiple regions (first region and second region) and processes them differently. The first region uses full-resolution processing for high quality, while the second region uses downsampled processing to reduce computational load. This segmentation resolves the contradiction by applying high computational complexity only where necessary for quality while using lower complexity elsewhere.
Solution Approach 2:
The patent implements local quality by applying different processing resolutions to different spatial regions of the reference image. High resolution is applied locally to the first region where it is most needed, while lower resolution is applied to the second region, optimizing the balance between rendering quality and computational complexity.
2Measurement precision
If entire reference pixels are processed in 3D warping, then rendering quality is improved, but memory bandwidth requirements increase significantly
Solution Approach 1:
The patent divides the reference image into multiple regions and applies different processing strategies. By segmenting the image, the system can load and process only the necessary portions at full resolution, reducing overall memory bandwidth requirements while maintaining quality where needed.
Solution Approach 2:
The patent applies local quality by providing high-resolution processing only to the first region and downsampled processing to the second region. This reduces the total memory bandwidth required compared to processing the entire reference image at full resolution, while still maintaining high quality in the critical first region.
3Measurement precision
If entire reference pixels are processed in 3D warping, then rendering quality is improved, but data rate requirements increase significantly
Solution Approach 1:
The patent segments the reference image into multiple regions with different processing resolutions. This segmentation allows the system to transmit and process fewer data points for the second region while maintaining acceptable quality, thereby reducing overall data rate requirements compared to full-resolution processing of the entire image.
Solution Approach 2:
The patent implements local quality by providing high-resolution data only for the first region and downsampled data for the second region. This selective approach reduces the total quantity of data that must be transmitted and processed, lowering data rate requirements while maintaining rendering quality where most important.
Data Source
AI summary
A device and method for video rendering. The device includes a memory and an electronic processor. The electronic processor is configured to receive, from a source device, video data including multiple reference viewpoints, determine a target image plane corresponding to a target viewpoint, determine, within the target image plane, one or more target image regions, and determine, for each target image region, a proxy image region larger than the corresponding target image region. The electronic processor is configured to determine, for each target image region, a plurality of reference pixels that fit within the corresponding proxy image region, project, for each target image region, the plurality of reference pixels that fit within the corresponding proxy image region to the target image region, producing a rendered target region from each target image region, and composite one or more of the rendered target regions to create video rendering.