Pre-stitched Panoramic Video Compression Using Projection Seam Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video compression techniques are inefficient for 360-degree panoramic videos captured by multiple cameras, leading to high bandwidth and storage requirements due to the computational intensity of stitching processes and distortion in spherical images, which reduces coding efficiency.
Innovation Solution
The use of projection-based and seam-based Inter prediction methods that incorporate stitching information, such as calibration data and seam detection, to encode and decode pre-stitched pictures, projecting reference blocks in spherical coordinates and adjusting pixel intensities to match seams, respectively, to improve prediction accuracy and reduce residuals.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional video compression techniques are used for 360-degree panoramic videos, then the encoding process is simple, but coding efficiency is low and bandwidth/storage requirements are high
Solution Approach 1:
The patent performs stitching operations before video compression encoding. By pre-stitching the panoramic images from multiple cameras into a unified spherical projection, the encoding process receives pre-processed data that maintains spatial coherence, thereby improving compression efficiency without requiring complex real-time stitching during encoding
Solution Approach 2:
The patent transforms the panoramic images from equirectangular projection to spherical coordinate system representation. This parameter change in the projection method allows for more efficient prediction and compression by aligning the data structure with the natural geometry of 360-degree views, improving coding efficiency while managing complexity through mathematical transformation
2Manufacturing precision
If stitching operations are performed on high-resolution panoramic videos, then a unified 360-degree view is achieved, but computational intensity increases and processing time is extended
Solution Approach 1:
The stitching operations are performed in advance before video compression, allowing computationally intensive alignment and blending operations to be completed offline. This preliminary stitching ensures high accuracy for the entire video sequence without requiring repeated processing during real-time compression or playback
Solution Approach 2:
The patent divides the stitching process into separate stages: calibration of individual camera views, projection to spherical coordinates, and blending of overlapping regions. This segmentation allows each stage to be optimized independently and processed efficiently, reducing overall processing time while maintaining stitching accuracy
3Adaptability or versatility
If spherical projection is used for 360-degree videos, then immersive viewing experience is achieved, but image distortion occurs and coding efficiency decreases
Solution Approach 1:
The patent explicitly adopts spherical projection for representing 360-degree panoramic content, accepting the inherent curvature-induced distortion as necessary for achieving immersive viewing. The spherical coordinate system naturally represents the full 360-degree field of view, providing adaptability for VR applications while the distortion is managed through specialized compression techniques designed for spherical data
4Area of stationary object
If multiple cameras are used to capture panoramic views, then coverage of 360-degree field of view is achieved, but data volume increases and bandwidth requirements become formidable
Solution Approach 1:
The patent merges multiple camera views into a single unified spherical projection representation. By combining the data from multiple cameras into one coherent 360-degree panorama rather than storing separate video streams, the total data volume is reduced while maintaining complete field of view coverage, thereby reducing bandwidth and storage requirements
Data Source
AI summary
Methods and apparatus of compression for pre-stitched pictures captured by multiple cameras of a panoramic video capture device are disclosed. At the encoder side, stitching information associated with a stitching process to form the pre-stitched pictures is used to encode a current block according to embodiments of the present invention, where the stitching information comprises calibration data, matching results, seam position, blending level, sensor data, or a combination thereof. In one embodiment, the stitching information corresponds to matching results associated with a projection process, and projection-based Inter prediction is used to encode the current block by projecting a reference block in a reference pre-stitched picture to coordinates of the current block. In another embodiment, the stitching information corresponds to seam information associated with seam detection, and seam-based Inter prediction is used to encode the current block by utilizing the seam information.