Free Viewpoint Video Encoding via Planar Splicing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video encoding technologies face delays and high computational pressures when handling multiple client viewpoints, limiting scalability and viewer experience.
Innovation Solution
The method involves generating a planar splicing image and splice information from multiple single-viewpoint videos at a server, encoding this information along with camera side information to create a planar splicing video bit stream, which is then decoded at the client side to synthesize virtual viewpoints, reducing delay and computational load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If viewpoint information is transmitted from client to server for real-time rendering, then viewing experience is improved, but transmission delay increases
Solution Approach 1:
Instead of transmitting viewpoint information from client to server and rendering at server, the patent inverts the approach by pre-rendering multiple viewpoints at the server and transmitting only the necessary view synthesis data to the client for local rendering. This reverses the traditional client-server rendering responsibility and eliminates transmission delay for viewpoint information.
Solution Approach 2:
The patent performs preliminary rendering of multiple viewpoints (left-eye and right-eye views) at the server before transmission. By pre-processing the video content into multiple viewpoints and encoding them in advance, the system avoids real-time rendering delays at the server and enables faster client-side playback.
2Adaptability or versatility
If multiple client viewpoints are processed at server, then viewing flexibility is improved, but computational pressure increases
Solution Approach 1:
The patent segments the rendering task by dividing the field of view into left-eye and right-eye view components. Each viewpoint is processed independently and encoded separately, allowing the server to handle multiple clients simultaneously with reduced computational overhead per client while maintaining viewing flexibility.
Solution Approach 2:
The patent creates copies of video content for different viewpoints (left and right views) at the server side. By pre-generating these viewpoint copies and transmitting them to clients, the server avoids real-time computational pressure when multiple clients request different viewpoints, as the rendering work is already completed during encoding.
3Adaptability or versatility
If all possible viewpoints are generated for storage, then viewing freedom is improved, but storage requirements increase
Solution Approach 1:
The patent applies partial action by generating only the essential viewpoints needed for free viewpoint video (typically left-eye and right-eye views) rather than all possible viewpoints. This selective generation approach provides sufficient viewing freedom for stereoscopic and free viewpoint applications while keeping storage requirements manageable.
Data Source
AI summary
The present application provides a method and a device of encoding and decoding based on free viewpoint, and relates to the technical field of video encoding. The method includes: generating a planar splicing image and splice information based on multiple single-viewpoint videos at a server side; generating a planar splicing video based on the planar splicing image; generating camera side information of the planar splicing video based on camera side information existing in the multiple single-viewpoint videos; and encoding the planar splicing video, the splice information and the camera side information of the planar splicing video to generate a planar splicing video bit stream, and decoding a planar splicing video bit stream to acquire a virtual viewpoint according to viewpoint information of a viewer at client side.