Virtual Viewpoint Image Generation with Subject Position Deviation Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for generating virtual viewpoint images struggle to accurately align the position of subjects in captured images with desired positions, especially in performances like dancing, where deviations can occur, making it difficult for users to notice and correct these deviations.
Innovation Solution
A system that includes an acquisition unit for shape data from multiple cameras, a first determination unit for subject position, a second determination unit for a reference position, and a generation unit that adjusts the virtual viewpoint image based on the deviation between the subject and reference positions, using methods like Shape-from-Silhouette and Multi-View-Stereo for three-dimensional shape reconstruction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If virtual viewpoint images are generated using captured images from multiple imaging apparatuses, then users can view performances from various angles with high presence, but the position of subjects in the generated images may deviate from desired positions when subjects move during capturing
Solution Approach 1:
The system performs preliminary actions by acquiring shape data representing the subject's shape and determining both the actual subject position and the desired reference position before generating the virtual viewpoint image. This allows the system to proactively calculate and compensate for position deviations, ensuring accurate subject positioning in the final image even when subjects move during performance capture.
Solution Approach 2:
The system implements feedback by calculating the deviation between the determined subject position and the reference position, then using this deviation information to adjust and correct the subject's position in the generated virtual viewpoint image. This feedback mechanism ensures that position accuracy is maintained despite subject movement during capture.
2Reliability
If the position of the subject deviates from the desired position during image capturing, then the captured images reflect the actual subject position, but the user may not notice the difference depending on the virtual viewpoint position
Solution Approach 1:
The system calculates the deviation between actual subject position and reference position, then uses this feedback to adjust the subject's position in the virtual viewpoint image. This ensures that the generated images consistently display subjects at their desired positions, making position deviations detectable and correctable regardless of the virtual viewpoint angle.
Solution Approach 2:
The system replaces physical/mechanical positioning with computational processing. Instead of physically repositioning imaging apparatuses or subjects, the system uses image processing and coordinate transformations to virtually adjust subject positions, achieving precise positioning control through software rather than mechanical means.
Data Source
AI summary
A generation apparatus includes an acquisition unit configured to acquire shape data representing a shape of a subject based on a plurality of captured images obtained by a plurality of imaging apparatuses capturing the subject in an imaging region, a first determination unit configured to determine a subject position which is a position of the subject in the imaging region, a second determination unit configured to determine a reference position serving as a reference of the position of the subject, and a generation unit configured to generate, based on the shape data acquired by the acquisition unit, a virtual viewpoint image in accordance with a deviation between the subject position determined by the first determination unit and the reference position determined by the second determination unit.