Mixed Reality Object Depth Modeling via Normal Projection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current mixed reality technologies face challenges in accurately determining contact between a real object and a computer-generated model, particularly in representing the front and back surfaces of objects, leading to inconsistencies and a sense of strangeness for users due to differences in shape and appearance.
Innovation Solution
An image processing apparatus that detects the area of a target object and generates both front and back surface models by adding thickness to the front surface model based on provided thickness information, allowing for accurate representation of object depth and preventing inconsistencies in polygon shapes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If only front surface depth is measured using a distance measurement device, then measurement speed is improved, but measurement precision of the complete object (including back surface) deteriorates
Solution Approach 1:
The patent segments the object modeling process into two distinct phases: front surface detection using rapid depth measurement device, and back surface generation through polygon projection and movement. This segmentation allows each phase to use the most appropriate method for its specific requirement, achieving both speed and precision.
Solution Approach 2:
The patent introduces an intermediary computational process that uses the front surface polygon data as a basis to generate back surface polygons. This intermediary step bridges the gap between fast front surface measurement and complete object modeling, allowing indirect acquisition of back surface information.
2Ease of manufacture
If a CG model is used to add thickness to a joint acquired by motion capture, then the model reconstruction is simplified, but the positions, thicknesses, and lengths of the CG model and real object become difficult to match
Solution Approach 1:
The patent changes the approach from using pre-prepared CG models with fixed dimensions to dynamically generating back surface polygons by moving front surface vertices by a calculated distance (thickness) in normal directions. This parameter-based approach allows the model to adapt to the actual dimensions of the captured object, ensuring accurate matching of positions, thicknesses, and lengths.
3Measurement precision
If fine voxels are used to estimate depth of a model with good accuracy, then measurement precision is improved, but memory volume and processing time increase
Solution Approach 1:
The patent uses polygon-based representation to copy and project geometric information from the front surface to generate the back surface, avoiding the need to store fine voxel data for the entire volume. This copying approach maintains depth estimation accuracy while significantly reducing memory requirements compared to voxel-based methods.
4Shape
If back surface polygon vertices are moved in line-of-sight direction, then the back surface position is adjusted, but vertexes may intersect with adjacent vertexes causing inconsistency in polygon shape
Solution Approach 1:
The patent moves back surface vertices in the normal direction of the front surface polygon rather than uniformly in the line-of-sight direction. This asymmetric approach, where each vertex is moved perpendicular to its local surface orientation, prevents intersections with adjacent vertices and maintains polygon shape consistency while achieving correct back surface positioning.
Data Source
AI summary
An image processing apparatus includes a detection unit that detects an area of a target object from an image captured by an image capturing apparatus, a first model generation unit that generates a front surface model representing a front surface of the target object based on the area of the target object, and a second model generation unit that generates a back surface model representing a back surface of the target object based on points provided by movement of a plurality of points on the front surface model by a distance corresponding to a thickness of the target object in respective normal directions.