Free Viewpoint Video Rendering via Depth Hull and Distributed Computing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video production methods do not allow viewers to control the viewpoint of a video stream effectively, especially during playback, as they are limited to fixed camera angles or require complex computational resources for dynamic viewpoint changes.
Innovation Solution
A system using at least one depth video camera and one standard video camera to record a subject, creating a 3D depth hull that allows viewers to select any viewpoint within a continuous segment around the subject, with the central computer performing calculations to render the view and optionally distributing the workload to remote viewers for scalability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple cameras are used to provide multiple viewpoints, then viewer control over viewpoint is improved, but device complexity increases
Solution Approach 1:
The system segments the viewpoint control functionality by separating the camera array from the rendering computation. Multiple cameras capture footage from different angles, but the complex task of synthesizing arbitrary viewpoints is segmented out to be performed separately by distributed computing resources rather than requiring a single complex centralized system.
Solution Approach 2:
The camera array is designed to capture footage that can be used for multiple different viewpoints simultaneously. The same set of cameras serves all potential viewing angles, making the camera system universal rather than requiring dedicated cameras for each specific viewpoint.
2Adaptability or versatility
If complex graphical calculations are performed to create requested views, then viewpoint flexibility is improved, but computational burden increases
Solution Approach 1:
The computational workload is segmented and distributed across multiple independent computing devices. Instead of one powerful computer performing all the complex graphical calculations for every requested viewpoint, the rendering tasks are divided and assigned to different devices in a distributed network, reducing the burden on any single system.
Solution Approach 2:
The system creates simplified representations or pre-computed data structures from the multi-camera footage that can be efficiently rendered on client devices. Rather than transmitting raw footage and requiring full re-rendering, copied or derived data structures enable viewpoint synthesis with reduced computational requirements.
3Adaptability or versatility
If all recording data is transmitted to each viewer, then viewpoint calculation flexibility is improved, but data transmission requirements increase
Solution Approach 1:
The system extracts only the essential data needed for viewpoint synthesis and transmits that selectively to viewers. Rather than sending all raw recording data from every camera to every client, the system extracts and transmits only the specific portions of data required for the requested viewpoint calculations, reducing overall data transmission requirements.
4Device complexity
If a single computer performs all calculations, then system simplicity is improved, but scalability worsens
Solution Approach 1:
The centralized computing task is segmented into distributed computing nodes. The system architecture evolves from a single computer handling all calculations to multiple independent computing devices that can each handle portions of the rendering workload, enabling the system to scale to serve more viewers simultaneously.
Solution Approach 2:
Each computing device in the distributed system can independently perform viewpoint rendering calculations for its assigned tasks. Rather than relying on a central computer to service all rendering requests, each node in the distributed network serves itself by autonomously performing the graphical calculations needed for its designated viewpoint synthesis tasks.
Data Source
AI summary
A method of remotely viewing a video from a selected viewpoint selected by the viewer from a continuous segment, including, recording a video of a subject using at least one depth video camera that records a video comprising a sequence of picture frames and additionally records a depth value for each pixel of the picture frames, recording a video of the subject using at least one standard video camera positioned to record a video at a viewpoint that differs from the viewpoint of the depth video camera, rendering a depth hull that defines a three dimensional outline of the subject being recorded using the depth values recorded by the depth video cameras, providing the recorded video from one or more cameras positioned on either side of the selected viewpoint, incorporating the recorded video from the one or more cameras onto the rendered depth hull to render a viewable video from the selected viewpoint; and displaying the rendered viewable video to the viewer.