AR Spatial Structure Recognition via Point Cloud Normal Vectors

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current augmented reality technologies face challenges in accurately recognizing and implementing 3D spatial structures without pre-learning, leading to inaccuracies in virtual object placement due to reliance on pre-learned image features and difficulties in obtaining 3D posture information.

Innovation Solution

An augmented reality device and method that utilize a point cloud normal vector extracting unit, plane object segmenting unit, representative plane selecting unit, spatial structure extracting unit, and virtual object matching unit to recognize indoor spatial structures using contextual knowledge and 3D posture information, allowing for accurate virtual object placement without pre-learning the space.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Device complexity

If pre-learned image features are used for spatial recognition, then the recognition process is simplified, but the accuracy of 3D posture information is reduced when features are not present

Engineering Contradiction:
Improverecognition process complexityVSAvoid3D posture information accuracy
Core Design Contradiction:
Device complexityVSMeasurement precision

Solution Approach 1:

The patent segments the spatial recognition process into multiple independent modules: point cloud extraction, normal vector calculation, plane segmentation, representative plane selection, and spatial structure extraction. Each module processes specific aspects of the spatial data independently, allowing the system to achieve high accuracy without relying on pre-learned features by breaking down the complex recognition task into manageable segments that can be processed systematically.

Inventive Principle:
Principle #1Segmentation

2Reliability

If marker-based methods are used for 3D spatial recognition, then 3D posture information can be obtained, but the system cannot adapt to spaces without pre-placed markers

Engineering Contradiction:
Improve3D posture information availabilityVSAvoidspace compatibility
Core Design Contradiction:
ReliabilityVSAdaptability or versatility

Solution Approach 1:

The patent enables the system to perform self-service spatial recognition by automatically extracting geometric features directly from the captured image data without requiring external markers or pre-placed reference objects. The point cloud extraction and normal vector calculation modules process the spatial environment autonomously, allowing the system to adapt to any space regardless of whether markers are present, thereby achieving both reliability and versatility.

Inventive Principle:
Principle #25Self-service

3Adaptability or versatility

If markerless methods are used for spatial recognition, then adaptability to different spaces is improved, but accuracy of 3D posture information is reduced

Engineering Contradiction:
Improvespace compatibilityVSAvoid3D posture information accuracy
Core Design Contradiction:
Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent transitions from 2D image plane analysis to 3D spatial structure recognition by extracting point clouds and calculating normal vectors in three-dimensional space. This dimensional transformation allows the system to capture the full geometric complexity of the environment, achieving high accuracy in 3D posture information while maintaining adaptability to different spaces without markers. The normal vector calculations provide additional dimensional information that enhances precision.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

4Measurement precision

If pre-learning of indoor space is performed, then augmented reality accuracy is improved, but the system cannot handle new or unvisited spaces

Engineering Contradiction:
Improveaugmented reality accuracyVSAvoidnew space capability
Core Design Contradiction:
Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent performs preliminary extraction of geometric features (point clouds and normal vectors) from the spatial environment that can be stored and reused for augmented reality applications. By pre-processing and extracting fundamental geometric structures without requiring full pre-learning of the space, the system creates a reusable spatial framework that maintains accuracy while enabling rapid adaptation to new spaces through the same extraction process.

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentUS9892323B2Augmented reality device based on recognition of spatial structure and method thereof
Publication Date: 2018.02.13 ELECTRONICS & TELECOMM RES INST
  • US9892323B2 patent drawing
  • US9892323B2 patent drawing
  • US9892323B2 patent drawing

AI summary

An augmented reality device based on recognition of a spatial structure includes: a point cloud normal vector extracting unit extracting a normal vector for a point cloud from image data input from a camera; a plane object segmenting unit segmenting a plane in the image data by using the extracted normal vector; a representative plane selecting unit selecting a representative plane among the segmented planes; a spatial structure extracting unit recognizing a spatial structure by using the representative plane; and a virtual object matching unit matching a virtual object in the recognized spatial structure.