3D View Synthesis with Dynamic Object Pose Adjustment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image synthesis systems face challenges in generating high-quality views of virtual or augmented objects from limited capture data, particularly for positions and orientations outside the captured range, leading to degraded image quality and restricted user movement in VR, AR, and MR applications.
Innovation Solution
An image synthesis system that determines an object pose and generates views by considering a viewing region defined by a viewing zone, adjusting the object pose when the distance from the viewer's pose exceeds a threshold, ensuring high-quality image synthesis across various viewer positions and orientations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If view images are generated from limited capture data for positions outside the captured range, then user movement freedom is improved, but image quality deteriorates
Solution Approach 1:
The system dynamically adjusts the object pose based on the viewer's current position and the available capture data. Instead of using a fixed object pose, the system continuously determines optimal poses that keep the viewer within the viewing zone, adapting to real-time viewer movement while maintaining image quality through dynamic recalibration.
Solution Approach 2:
The system changes the object pose parameters (position and orientation) based on the viewer's position relative to the viewing zone. When the viewer moves outside the acceptable range, the system modifies the object pose parameters to realign the viewing region with the viewer, thereby maintaining image quality across different viewer positions.
2Adaptability or versatility
If the viewing region is expanded to cover all possible viewer positions, then user movement freedom is improved, but data completeness deteriorates
Solution Approach 1:
The system resolves the contradiction by operating in the object pose parameter space rather than expanding the viewing region in physical space. By adjusting the object's position and orientation parameters, the system effectively provides unlimited viewing coverage while maintaining data completeness, as the same capture data is reused with different object poses.
3Manufacturing precision
If the object pose is frequently adjusted to maintain viewing quality, then image quality is improved, but computational complexity increases
Solution Approach 1:
The system implements a feedback mechanism where the viewer's position is continuously monitored, and the object pose is adjusted only when necessary (when the viewer moves outside the viewing zone). This feedback-based approach maintains image quality while minimizing unnecessary computational operations, as adjustments are made reactively rather than continuously.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An image synthesis system comprises receivers (201, 203, 205) receiving scene data describing at least part of a scene; object data describing a 3D object from a viewing zone having a relative pose with respect to the object, and a view pose in the scene. A pose determiner circuit (207) determines an object pose for the object in the scene in response to the scene data and the view pose; and a view synthesis circuit (209) generates a view image of the object from the object data, the object pose, and the view pose. A circuit (211) determines a viewing region in the scene which corresponds to the viewing zone for the object being at the object pose. The pose determiner circuit (207) determines a distance measure for the view pose relative to the viewing region and changes the object pose if the distance measure meets a criterion including a requirement that a distance between the view pose and a pose of the viewing region exceeds a threshold.