Object Tracking Device Using 3D Scene Feature Points

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current SLAM methods face challenges in accurately tracking the 3D pose of objects when the 3D model of the scene or object is unknown, leading to reduced tracking accuracy.

Innovation Solution

A method that involves acquiring images from different camera positions, deriving 3D scene feature points using triangulation or bundle adjustment, and establishing a 3D-2D relationship to update the object's pose, utilizing a 3D model to improve tracking accuracy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If SLAM method is used with unknown 3D model, then the method can be applied to general scenes, but the tracking accuracy of object pose deteriorates

Engineering Contradiction:

Improveapplicability to general scenesVSAvoidtracking accuracy of object pose

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent segments the scene into two distinct components: the target object with known 3D model and the surrounding scene with unknown geometry. By separating object tracking from scene mapping, the system can apply different methods to each - using model-based tracking for the object and SLAM for the scene, thereby resolving the contradiction between general applicability and tracking accuracy.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces 3D scene feature points as an intermediary element that bridges the known object model and the unknown scene. These feature points are extracted from the scene and used to establish correspondences between 3D model points and 2D image points, enabling accurate pose estimation even when the overall scene model is unknown.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Device complexity

If only 3D model points are used for pose derivation, then the process is simpler, but the pose estimation accuracy deteriorates

Engineering Contradiction:

Improveprocess simplicityVSAvoidpose estimation accuracy

Core Design Contradiction:

Device complexityVSMeasurement precision

Solution Approach 1:

The patent merges two types of feature points - 3D model points from the object's known model and 3D scene feature points extracted from the surrounding scene - into a unified set of correspondences. This combination enriches the available information for pose estimation, improving accuracy while maintaining a relatively simple overall process by reusing existing SLAM feature extraction techniques.

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS10373334B2Computer program, object tracking method, and object tracking device

Publication Date: 2019.08.06 SEIKO EPSON CORP

AI summary

A computer program causes an object tracking device to realize functions of: acquiring a first image of a scene including an object captured with a camera positioned at a first position; deriving a 3D pose of the object in a second image captured with the camera positioned at a second position using a 3D model corresponding to the object; deriving 3D scene feature points of the scene based at least on the first image and the second image; obtaining a 3D-2D relationship between 3D points represented in a 3D coordinate system of the 3D model and image feature points on the second image; and updating the derived pose using the 3D-2D relationship, wherein the 3D points include the 3D scene feature points and 3D model points on the 3D model.