Hand Surface Normal Estimation for Mobile 3D Augmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems require large memory storage and computational resources to generate realistic 3D augmentations for 2D images, making it difficult to achieve real-time rendering on mobile devices.
Innovation Solution
Estimate normals for 3D surfaces based on 2D image data to generate 3D textures, reducing computational load and enabling quick creation of image augmentations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If conventional systems use large 3D models for rendering, then realistic 3D augmentations are achieved, but memory storage and computational resources increase significantly
Solution Approach 1:
The patent extracts only the essential normal vector information from complete 3D models. Instead of storing and processing entire 3D model datasets, the system extracts normal maps that contain only surface orientation data. This extraction principle allows the system to achieve realistic lighting and shading effects without the computational burden of full 3D models, directly resolving the contradiction between augmentation quality and resource consumption.
Solution Approach 2:
The patent creates simplified copies of 3D surface information in the form of normal maps. These normal maps are 2D representations that copy the essential geometric properties (surface normals) of 3D objects without replicating the full 3D structure. This copying approach enables realistic rendering by providing sufficient geometric data for lighting calculations while occupying minimal storage space and requiring minimal computational resources.
2Productivity
If conventional systems process 3D models in real-time, then interactive augmentations are achieved, but computational load becomes too heavy for mobile devices
Solution Approach 1:
The patent replaces expensive, complex 3D model processing with cheap, lightweight normal map processing. Normal maps are computationally inexpensive to process because they are essentially 2D texture maps that encode surface orientation. This substitution allows mobile devices to perform real-time rendering operations that would be too computationally intensive with full 3D models, directly addressing the contradiction between rendering speed and computational load.
3Measurement precision
If complete 3D models are stored, then accurate surface information is available, but memory storage requirements increase
Solution Approach 1:
The patent creates compact 2D copies of 3D surface normal information. Instead of storing volumetric 3D model data, the system stores normal maps that are essentially 2D images encoding surface orientation. This copying method preserves the essential geometric information needed for realistic rendering while reducing storage requirements by an order of magnitude or more, directly resolving the contradiction between surface information accuracy and memory storage requirements.
Data Source
AI summary
An system for augmenting images using hand surface normal estimation is provided. In a model training phase, 3D models of hands are generated using 3D data of hands in a variety of positions. Target normal training data is generated that includes normals of surfaces of the 3D models and synthetic 2D image training data corresponding to the 3D models and the normals. The target normal training data and the synthetic image training data are used to train a normal estimation model. The normal estimation is used by an interactive application to generate augmentations that are applied to hand image data.