Stereoscopic Playback Distortion Correction Using Camera-Specific UV Meshes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing stereoscopic systems face distortions in left and right eye images due to manufacturing variations in fisheye lenses, leading to improper depth cues and image distortions, especially when streaming 360-degree content with data transmission constraints.
Innovation Solution
Generate camera-specific distortion correction information, known as a correction mesh, which is communicated to a playback device to correct lens distortions during rendering, rather than pre-processing images for encoding and transmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If pre-processing images to correct lens distortions during encoding, then image quality is improved, but processing time and computational complexity increase
Solution Approach 1:
The system performs preliminary calibration of camera distortion parameters during the encoding phase, storing the corrected images or correction parameters for later use during playback. This preliminary action ensures that when the content is viewed on different devices, the distortion correction is already prepared and can be applied quickly without requiring complex real-time processing.
Solution Approach 2:
Instead of performing computationally intensive distortion correction on every frame during playback, the system creates a copy of the corrected image data or correction parameters during encoding. These pre-computed corrections are then transmitted and applied during playback, significantly reducing the computational burden on the playback device while maintaining image quality.
2Area of moving object
If using fisheye lenses to capture wide viewing area, then field of view is improved, but lens manufacturing variations cause distortions
Solution Approach 1:
The system applies different distortion correction parameters to different regions of the captured image based on the specific lens characteristics and the local area being captured. By calibrating each camera's distortion characteristics individually and applying region-specific corrections, the system maintains the wide field of view benefit while compensating for lens manufacturing variations in each specific area.
Solution Approach 2:
The system changes the distortion correction parameters dynamically based on the specific camera and lens configuration used during capture. By storing calibration data that describes the actual optical characteristics of each fisheye lens and adjusting the correction parameters accordingly during encoding and playback, the system eliminates distortions caused by manufacturing variations while preserving the extended field of view.
3Manufacturing precision
If streaming full 360 degree content at high definition, then image quality is improved, but data transmission requirements increase
Solution Approach 1:
The system segments the 360-degree environment into multiple discrete camera views or zones, each captured by a specific camera. By organizing the content into these segments and transmitting them as separate data streams with associated distortion correction parameters, the system enables selective transmission of only the portions needed for the user's current viewing direction, reducing overall data requirements while maintaining high image quality where transmitted.
Solution Approach 2:
The system dynamically adjusts what content is transmitted based on the user's viewing direction and preferences. By using head tracking or viewing angle detection to determine which camera segments are currently needed, the system can prioritize transmission of those specific segments over others, reducing the total data transmitted while maintaining high quality for the active viewing area.
4Manufacturing precision
If applying distortion correction at playback device, then image quality is maintained, but device complexity increases
Solution Approach 1:
The system copies the distortion correction parameters or pre-corrected image data during the encoding phase and transmits these copies to the playback device. The playback device then applies these pre-computed corrections, which significantly simplifies the processing requirements compared to performing complex real-time distortion correction algorithms, while still maintaining high image quality.
Solution Approach 2:
The system introduces an intermediary layer of correction parameters or lookup tables that mediate between the captured distorted images and the final corrected output. Instead of requiring the playback device to perform complex distortion correction algorithms, these intermediary correction data structures enable simple parameter-based adjustments, reducing processing complexity while maintaining image quality.
Data Source
AI summary
Methods and apparatus for streaming or playing back stereoscopic content are described. Camera dependent correction information is communicated to a playback device and applied in the playback device to compensate for distortions introduced by the lenses of individual cameras. By performing lens dependent distortion compensation in the playback device edges which might be lost if correction were performed prior to encoding are preserved. Distortion correction information maybe in the form of UV map correction information. The correction information may indicate changes to be made to information in a UV map, e.g., at rendering time, to compensate for distortions specific to an individual camera. Different sets of correction information maybe communicated and used for different cameras of a stereoscopic pair which provide images that are rendered using the same UV map. The communicated correction information is sometimes called a correction mesh since it is used to correct mesh related information.