3D Scene Image Synthesis Using Adaptive Anchor Poses
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing virtual reality technologies are limited in providing flexible and immersive experiences due to high computational requirements, restricted freedom of movement, and suboptimal user interaction, often requiring high data rates and complex implementations.
Innovation Solution
An apparatus and method that generates images based on viewer poses, allowing natural parallax for small movements and teleportation to different viewpoints, using a selector to switch anchor poses based on criteria and mapping viewer poses to render poses, reducing complexity and data requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple video streams are loaded for switching viewpoints, then viewpoint switching capability is improved, but data rate requirement increases and user experience is interrupted
Solution Approach 1:
The patent extracts only the essential viewpoint information needed for rendering by using a single video stream with depth map encoding. Instead of transmitting multiple complete video streams for different viewpoints, the system extracts viewpoint-dependent information through depth maps and synthesizes other viewpoints computationally, significantly reducing data transmission requirements while maintaining viewpoint switching capability
Solution Approach 2:
The patent creates computational copies of the viewpoint information by synthesizing virtual viewpoints from a single captured video stream using depth maps. Rather than transmitting multiple physical video streams, the system generates synthetic video streams through computational rendering, reducing bandwidth requirements while providing multiple viewpoint access
2Ease of operation
If panoramic depth map is encoded with panoramic stereo video, then small translational motion compensation is improved, but freedom of movement remains limited
Solution Approach 1:
The patent implements dynamic viewpoint synthesis by continuously generating virtual viewpoints based on the user's current position and orientation. Instead of being limited to pre-encoded fixed viewpoints, the system dynamically renders viewpoints that match the user's real-time movements, providing continuous freedom of movement within the captured scene boundaries
Solution Approach 2:
The patent introduces depth maps as an intermediary data structure that enables flexible viewpoint synthesis. The depth maps serve as a mediator between the captured video stream and the synthesized virtual viewpoints, allowing the system to generate arbitrary viewpoints by combining the video data with depth information through computational rendering
3Adaptability or versatility
If free-viewpoint video with multiple viewpoints and depth maps is encoded, then viewpoint flexibility is improved, but bitrate requirement increases and image generation freedom is restricted
Solution Approach 1:
The patent extracts only the essential viewpoint information needed for rendering by using a single video stream with depth map encoding. Instead of transmitting multiple complete video streams for different viewpoints, the system extracts viewpoint-dependent information through depth maps and synthesizes other viewpoints computationally, significantly reducing data transmission requirements while maintaining viewpoint switching capability
Solution Approach 2:
The patent changes the representation parameters of the video data by encoding a single video stream with associated depth maps rather than multiple video streams. This parameter change allows the system to generate multiple viewpoints from a single encoded stream, reducing bitrate requirements while maintaining viewpoint flexibility through computational rendering
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An apparatus comprises a store (209) storing a set of anchor poses for a scene, as well as typically 3D image data for the scene. A receiver (201) receives viewer poses for a viewer and a render pose processor (203) determines a render pose in the scene for a current viewer pose of the viewer pose where the render pose is determined relative to a reference anchor pose. A retriever (207) retrieves 3D image data for the reference anchor pose and a synthesizer (205) synthesizes images for the render pose in response to the 3D image data. A selector selects the reference anchor pose from the set of anchor poses and is arranged to switch the reference anchor pose from a first anchor pose of the set of anchor poses to a second anchor pose of the set of anchor poses in response to the viewer poses.