3D Skeleton Retargeting for Editable Volumetric Video
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for generating and utilizing 3D shapes of subjects in volumetric video are cumbersome and costly, requiring manual adjustments and significant effort, especially when errors or partial reshooting is needed.
Innovation Solution
A video processing device that estimates a 3D skeleton of a subject from multi-viewpoint images, applies this skeleton to separate the subject from the background, and generates 3D data using skeleton data for flexible editing and retargeting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If volumetric video is generated using multi-viewpoint images to create accurate 3D shapes, then manufacturing precision is improved, but device complexity and ease of operation deteriorate due to requiring manual modification of multiple images when errors occur
Solution Approach 1:
The patent extracts the essential 3D structural information by generating a skeleton model from multi-viewpoint images, separating the core geometric data from the complete image set. This skeleton can then be applied to multiple views without needing to manually adjust each original image, thus improving ease of operation while preserving 3D accuracy.
Solution Approach 2:
The patent creates a reusable skeleton model that can be copied and applied across different viewpoints and scenes. Instead of manually modifying multiple original images when corrections are needed, the skeleton serves as a master template that can be replicated, significantly reducing the operational burden while maintaining consistent 3D shape accuracy.
2Manufacturing precision
If complete volumetric video production is performed with multiple multi-viewpoint cameras, then manufacturing precision is improved, but loss of time and productivity worsen when only partial scenes need reshooting
Solution Approach 1:
The patent segments the volumetric video production into independent skeleton extraction and application phases. The skeleton can be generated once from multi-viewpoint images and then independently applied to different scenes or time periods, allowing partial scenes to be updated without requiring complete reshooting, thus reducing time loss while maintaining 3D accuracy.
Solution Approach 2:
The patent performs preliminary skeleton extraction from multi-viewpoint images in advance, creating a reusable 3D structural template. This preliminary action allows subsequent scene modifications or reshoots to focus only on specific portions, rather than requiring complete reprocessing of all multi-viewpoint data, thereby reducing time loss while preserving manufacturing precision.
3Manufacturing precision
If manual modification of multi-viewpoint images is performed to correct 3D shape errors, then manufacturing precision is improved, but productivity deteriorates due to the large workload
Solution Approach 1:
The patent extracts the essential 3D structural information into a skeleton model, separating the core geometric corrections from the complex multi-viewpoint image data. This allows precision improvements to be made by modifying the skeleton once, rather than manually adjusting multiple images, dramatically improving productivity while maintaining 3D shape accuracy.
Solution Approach 2:
The patent uses the skeleton model as a master template that can be copied and applied across multiple viewpoints and scenes. Corrections made to the skeleton are automatically replicated, eliminating the need for repetitive manual modification of each image, thus improving productivity while ensuring consistent manufacturing precision across all views.
Data Source
AI summary
A video processing device according to an aspect of the present disclosure includes: an estimation unit that estimates a 3D skeleton of a subject on the basis of multi-viewpoint images obtained by shooting the subject from a plurality of viewpoints; an application unit that applies a 3D skeleton of the subject estimated by the estimation unit to the subject included in other image different from the multi-viewpoint images and separated from a background in the image; and a generation unit that generates 3D data of a subject to which a 3D skeleton is applied by the application unit.