3D Map Loop Closing via Visual Position Guidance

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing methods for generating high precision three-dimensional maps in augmented and mixed reality environments, such as those using SfM and SLAM, face challenges in aligning the position and orientation of image capturing devices over long distances, leading to misalignment issues that are not effectively addressed by current loop closing techniques without user guidance.

Innovation Solution

An information processing apparatus that guides users to capture images at initial positions and orientations, using guidance instructions and determination units to ensure accurate loop closing, thereby optimizing the alignment of start and end points for high precision three-dimensional map generation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If loop closing is performed without user guidance, then the system can automatically optimize position and orientation alignment, but the user cannot reliably return to the start point resulting in imprecise three-dimensional map generation

Engineering Contradiction:
Improveposition and orientation alignment precisionVSAvoiduser operation complexity
Core Design Contradiction:
Measurement precisionVSEase of operation

Solution Approach 1:

The system provides real-time feedback to the user by displaying the captured image at the start point and guiding the user to capture images that form a closed path. The determination unit continuously monitors whether the captured images satisfy loop closing conditions, providing feedback on the user's progress toward returning to the start point, thereby enabling precise position and orientation alignment through guided operation.

Inventive Principle:
Principle #23Feedback

Solution Approach 2:

The system introduces an intermediary guidance mechanism that mediates between the user's movement and the loop closing requirement. By displaying reference images and providing visual feedback, the system acts as an intermediary that helps the user understand and achieve the correct positioning without requiring complex manual operations or precise manual judgment.

Inventive Principle:
Principle #24Intermediary (Mediator)

2Reliability

If the user is guided to capture images at specific positions and orientations, then loop closing accuracy is improved, but the operation process becomes more complex

Engineering Contradiction:
Improveloop closing reliabilityVSAvoidguidance system complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The system enables self-service loop closing by allowing the user to independently capture images at the required positions and orientations using visual guidance from the display unit. The determination unit automatically evaluates whether the captured images satisfy loop closing conditions, eliminating the need for complex external guidance systems or manual intervention while ensuring reliable loop closing.

Inventive Principle:
Principle #25Self-service

3Manufacturing precision

If multiple captured images are required to form a closed path, then three-dimensional map precision is improved, but the time required for image capture increases

Engineering Contradiction:
Improvethree-dimensional map precisionVSAvoidimage capture time
Core Design Contradiction:
Manufacturing precisionVSLoss of time

Solution Approach 1:

The system requires the user to capture images that partially or excessively satisfy the loop closing condition. The determination unit evaluates whether the captured images meet the necessary conditions for loop closing, allowing flexibility in the capture process while ensuring that the essential requirement of forming a closed path is met, thereby balancing map precision with capture efficiency.

Inventive Principle:
Principle #16Partial or excessive action

Data Source

PatentUS10171730B2Information processing apparatus, method of controlling information processing apparatus, and storage medium
Publication Date: 2019.01.01 CANON KK
  • US10171730B2 patent drawing
  • US10171730B2 patent drawing
  • US10171730B2 patent drawing

AI summary

An apparatus comprises an unit configured to obtain captured images captured at a plurality of positions/orientations; an unit configured to generate a guidance instruction for causing a captured image corresponding to a position/orientation substantially the same as a captured image of a stipulated state to be obtained; an unit configured to output the guidance instruction; an unit configured to determine whether or not one captured image obtained after output of the guidance instruction was captured at a position/orientation substantially the same as the captured image of the stipulated state; and an unit configured to, if the one captured image is determined to be captured at the position/orientation, generate or update a three-dimensional map from three-dimensional coordinates of a feature included in a captured image based on the plurality of captured images including the one captured image.