Hand Surface Normal Estimation for Real-Time 3D Augmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems require large memory storage and computational resources to generate realistic 3D augmentations for 2D images, making it difficult to create such effects in real-time on mobile devices.
Innovation Solution
Estimate normals for portions of a human body, such as hands, based on 2D image data to generate 3D textures, reducing computational load and enabling quick creation of image augmentations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If conventional systems use large 3D models and full computational processing to generate realistic 3D augmentations, then the realism and quality of the augmentations is improved, but the memory storage requirements and computational resources increase significantly, making real-time processing on mobile devices difficult
Solution Approach 1:
The patent extracts only the essential geometric information (surface normals) needed for realistic rendering, rather than processing complete 3D models. By estimating normals directly from 2D image data, the system obtains sufficient information for lighting and shading calculations without requiring full 3D model reconstruction, thereby reducing memory and computational requirements while maintaining visual realism
Solution Approach 2:
Instead of the conventional approach of generating 3D models first and then extracting normals, the patent inverts the process by directly estimating surface normals from 2D image data. This reverse approach bypasses the need for complex 3D model construction and storage, enabling real-time processing on mobile devices while still achieving realistic rendering effects
2Productivity
If conventional systems process complete 3D models in real-time, then the quality of image augmentation is maintained, but the processing time and computational load become prohibitive for mobile devices
Solution Approach 1:
The system extracts only the critical surface normal information required for realistic rendering, eliminating the need to process entire 3D models. This selective extraction of essential geometric data dramatically reduces computational load and processing time, enabling real-time operation on mobile devices while maintaining visual quality
Solution Approach 2:
The patent applies partial action by computing only the specific geometric property (surface normals) needed for rendering, rather than performing complete 3D model processing. This partial computation approach provides sufficient information for realistic augmentation while keeping computational requirements within the capabilities of mobile devices
Data Source
AI summary
An system for augmenting images using hand surface normal estimation is provided. In a model training phase, 3D models of hands are generated using 3D data of hands in a variety of positions. Target normal training data is generated that includes normals of surfaces of the 3D models and synthetic 2D image training data corresponding to the 3D models and the normals. The target normal training data and the synthetic image training data are used to train a normal estimation model. The normal estimation is used by an interactive application to generate augmentations that are applied to hand image data.