3D Keypoint Grouping for Multi-Object Pose Recognition

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing methods for recognizing the position and attitude of objects in varying conditions, such as changes in shape and environment, are prone to errors due to insufficient learning data and manual label variations, and struggle with multiple object recognition.

Innovation Solution

A method involving generating learning object models with varied shapes and surface properties, simulating scenes, and creating correct feature maps to train a machine learning model for robust estimation of keypoints, allowing for the determination of positions and attitudes of multiple objects.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If vast amounts of learning data corresponding to combinations of various conditions including shape and surrounding environment are prepared, then robust recognition of object position and attitude is achieved, but data preparation complexity and time consumption increase significantly

Engineering Contradiction:
Improverobust recognition accuracyVSAvoiddata preparation time
Core Design Contradiction:
ReliabilityVSLoss of time

Solution Approach 1:

The patent applies preliminary action by pre-generating multiple learning object models with varied shapes and surface properties, and pre-simulating multiple scenes with different environments before actual recognition tasks. This preparation of diverse training data in advance allows the neural network to be trained on comprehensive variations without requiring extensive data collection during deployment, thus achieving robust recognition while reducing real-time data preparation time.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent uses copying by creating multiple virtual copies of object models with different shapes and surface properties through digital manipulation rather than physical object creation. These copied and modified models are then used in simulated scenes to generate training data, eliminating the need to collect vast amounts of real-world data for every possible variation, thereby reducing data preparation time while maintaining recognition robustness.

Inventive Principle:
Principle #26Copying

2Adaptability or versatility

If manual attachments of correct labels are performed by multiple people to create learning data, then comprehensive training data is obtained, but annotation errors are introduced and recognition accuracy degrades

Engineering Contradiction:
Improvetraining data coverageVSAvoidlabel annotation accuracy
Core Design Contradiction:
Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent replaces manual human annotation with automated synthetic label generation by copying and transforming object models programmatically. The system automatically generates correct labels for multiple learning object models and simulation scenes through computational processes, eliminating human annotation errors while maintaining comprehensive training data coverage across various shapes and environments.

Inventive Principle:
Principle #26Copying

Solution Approach 2:

The patent substitutes the mechanical process of manual human annotation with an automated computational system. Instead of relying on human operators to attach labels to images, the system uses algorithms to automatically generate ground truth labels from the known parameters of simulated objects, thereby eliminating human error while maintaining versatility in training data generation.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

3Device complexity

If a single object is assumed in the recognition model, then the model is simpler to implement, but it becomes impossible to recognize positions and attitudes of multiple objects simultaneously

Engineering Contradiction:
Improvemodel implementation complexityVSAvoidmulti-object recognition capability
Core Design Contradiction:
Device complexityVSAdaptability or versatility

Solution Approach 1:

The patent applies universality by designing a recognition model that can handle both single and multiple objects through the same keypoint detection mechanism. The neural network detects keypoints on any number of objects in the scene and the system automatically associates these keypoints with their respective objects, allowing the model to scale from single-object to multi-object recognition without fundamental changes to the implementation, thus achieving multi-functionality.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent uses segmentation by dividing the recognition task into independent keypoint detection for each object, followed by clustering and association of detected keypoints into groups corresponding to individual objects. This segmentation approach allows the system to process multiple objects simultaneously by treating each object's keypoints as a separate subset, maintaining implementation simplicity while enabling multi-object capability.

Inventive Principle:
Principle #1Segmentation

Data Source

PatentUS12620121B2Method of recognizing position and attitude of object, and non-transitory computer-readable storage medium
Publication Date: 2026.05.05 SEIKO EPSON CORP
  • US12620121B2 patent drawing
  • US12620121B2 patent drawing
  • US12620121B2 patent drawing

AI summary

A method of the present disclosure includes (a) generating an input image by imaging a scene containing the M objects by a camera, (b) obtaining a feature map showing feature amounts relating to the N keypoints from the input image using a learned machine learning model with the input image as input and the feature map as output, (c) obtaining three-dimensional coordinates of the N keypoints belonging to each of the M objects using the feature map, and (d) determining positions and attitudes of one or more objects of the M objects using the three-dimensional coordinates of the N keypoints belonging to each of the M objects, wherein (c) includes (c1) obtaining M×N keypoints having undetermined correspondence relationships with the M objects and determining the three-dimensional coordinates of the M×N keypoints, and (c2) grouping the M×N keypoints to the N keypoints belonging to each of the M objects.