Virtual Viewpoint Display for 3D Performance Capture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Subjects performing actions like singing and dancing are not aware of the position of the virtual camera, leading to challenges in coordinating their performance effectively.
Innovation Solution
An information processing apparatus that acquires real images from multiple cameras, generates a 3D model of the subject, and presents virtual camera information to the subject using display panels, allowing them to be aware of the virtual camera's position and direction during rendering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a 3D model is generated from multiple real images for rendering, then the visual quality and realism of the distributed content is improved, but the performer cannot be aware of the virtual camera position, making it difficult to perform tasks like singing and dancing with proper awareness of camera perspective
Solution Approach 1:
The patent introduces a display device as an intermediary between the virtual camera system and the performer. This display device presents information about the virtual camera's position and viewpoint to the performer, allowing them to understand where they are being viewed from without interfering with the 3D rendering process. The intermediary translates technical camera data into performer-friendly visual information.
Solution Approach 2:
The system implements feedback by continuously providing the performer with information about the virtual camera's position, viewpoint, and movement. This feedback loop allows the performer to adjust their performance in real-time based on camera positioning, similar to how performers work with real cameras, thereby maintaining awareness and control during the performance.
2Manufacturing precision
If multiple imaging devices are arranged around the subject to capture real images, then the coverage and quality of 3D reconstruction is improved, but the system complexity increases
Solution Approach 1:
The patent makes the imaging devices multi-functional by having them serve dual purposes: (1) capturing real images for 3D model generation, and (2) providing positional information for the virtual camera system. This eliminates the need for separate sensor systems and reduces overall system complexity while maintaining high 3D reconstruction quality.
Solution Approach 2:
The system merges the imaging function with the camera position sensing function. By combining these functions into a unified system where the same imaging devices provide both visual data and positional data, the patent reduces the number of separate components needed and simplifies the overall system architecture.
Data Source
AI summary
An image processing apparatus (information processing apparatus) includes a volumetric image shooting unit (first acquisition unit) that acquires a plurality of real images respectively captured by a plurality of cameras (first imaging devices) arranged around a subject, a volumetric image generation unit (generation unit) that generates a 3D model of the subject from the plurality of real images, and an in-studio image display unit (presentation unit) that presents, to the subject, virtual viewpoint information used in rendering, the 3D model, into an image in a form corresponding to a viewing device.