Egocentric Visual Servoing With Self-Supervised 3D Pose Learning

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing visual servoing methods struggle with accuracy in large movements due to occlusions and require 3D supervision, making them ineffective for general object manipulation in unstructured environments.

Innovation Solution

A self-supervised learning method using a Siamese neural network architecture with 3D equivariance to learn visual representations without 3D ground truth, enabling accurate relative pose estimation and manipulation of objects through a robot arm.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If local image feature mapping is used for visual servoing, then local accuracy is improved, but accuracy deteriorates with large movements or multiple movements

Engineering Contradiction:

Improvelocal image feature mapping accuracyVSAvoidaccuracy with large movements

Core Design Contradiction:

Measurement precisionVSReliability

Solution Approach 1:

The patent transitions from local 2D image feature matching to global 3D pose estimation, adding a dimensional perspective that enables accurate tracking across large movements by considering spatial relationships in three-dimensional space rather than relying on local image features alone

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Measurement precision

If 3D supervision with ground truth data is used, then absolute pose regression accuracy is improved, but data acquisition complexity increases

Engineering Contradiction:

Improveabsolute pose regression accuracyVSAvoidground truth data labeling complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The system performs self-supervised learning by automatically generating supervision signals from image sequences and estimated poses without requiring manual 3D annotation, enabling the model to train itself using readily available unstructured image data

Inventive Principle:

Principle #25Self-service

3Measurement precision

If CNN-based supervised regression is used, then relative pose estimation is improved, but training data requirements increase

Engineering Contradiction:

Improverelative pose estimation accuracyVSAvoidtraining data requirements

Core Design Contradiction:

Measurement precisionVSQuantity of substance

Solution Approach 1:

The system generates its own training data and supervision signals automatically from image sequences and motion estimates, eliminating the need for large annotated datasets and enabling training on unstructured image data without manual labeling

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS12629829B2Apparatus and method for self-supervised learning for visual feature representation of egocentric images

Publication Date: 2026.05.19 SAMSUNG ELECTRONICS CO LTD

AI summary

An electronic device for manipulating a target object, including: a camera; a memory; and at least one processor configured to: obtain a first image of the target object, wherein the first image is captured by the camera, provide the first image and a target image to an artificial intelligence (AI) model to obtain relative pose information, based on the obtained relative pose information, generate a similarity value, and generate a control command based on the similarity value, wherein based on the similarity value being greater than a threshold value, the control command includes a movement command for moving a manipulator associated with the electronic device from a first position to a second position, and wherein based on the similarity value being less than or equal to the threshold value, the control command includes a manipulation command for manipulating the target object using the manipulator.