3D Keypoint Grouping for Multi-Object Pose Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for recognizing the position and attitude of objects in varying conditions, such as changes in shape and environment, are prone to errors due to insufficient learning data and manual label variations, and struggle with multiple object recognition.
Innovation Solution
A method involving generating learning object models with varied shapes and surface properties, simulating scenes, and creating correct feature maps to train a machine learning model for robust estimation of keypoints, allowing for the determination of positions and attitudes of multiple objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If vast amounts of learning data corresponding to combinations of various conditions including shape and surrounding environment are prepared, then robust recognition of object position and attitude is achieved, but data preparation complexity and time consumption increase significantly
Solution Approach 1:
The patent applies preliminary action by pre-generating multiple learning object models with varied shapes and surface properties, and pre-simulating multiple scenes with different environments before actual recognition tasks. This preparation of diverse training data in advance allows the neural network to be trained on comprehensive variations without requiring extensive data collection during deployment, thus achieving robust recognition while reducing real-time data preparation time.
Solution Approach 2:
The patent uses copying by creating multiple virtual copies of object models with different shapes and surface properties through digital manipulation rather than physical object creation. These copied and modified models are then used in simulated scenes to generate training data, eliminating the need to collect vast amounts of real-world data for every possible variation, thereby reducing data preparation time while maintaining recognition robustness.
2Adaptability or versatility
If manual attachments of correct labels are performed by multiple people to create learning data, then comprehensive training data is obtained, but annotation errors are introduced and recognition accuracy degrades
Solution Approach 1:
The patent replaces manual human annotation with automated synthetic label generation by copying and transforming object models programmatically. The system automatically generates correct labels for multiple learning object models and simulation scenes through computational processes, eliminating human annotation errors while maintaining comprehensive training data coverage across various shapes and environments.
Solution Approach 2:
The patent substitutes the mechanical process of manual human annotation with an automated computational system. Instead of relying on human operators to attach labels to images, the system uses algorithms to automatically generate ground truth labels from the known parameters of simulated objects, thereby eliminating human error while maintaining versatility in training data generation.
3Device complexity
If a single object is assumed in the recognition model, then the model is simpler to implement, but it becomes impossible to recognize positions and attitudes of multiple objects simultaneously
Solution Approach 1:
The patent applies universality by designing a recognition model that can handle both single and multiple objects through the same keypoint detection mechanism. The neural network detects keypoints on any number of objects in the scene and the system automatically associates these keypoints with their respective objects, allowing the model to scale from single-object to multi-object recognition without fundamental changes to the implementation, thus achieving multi-functionality.
Solution Approach 2:
The patent uses segmentation by dividing the recognition task into independent keypoint detection for each object, followed by clustering and association of detected keypoints into groups corresponding to individual objects. This segmentation approach allows the system to process multiple objects simultaneously by treating each object's keypoints as a separate subset, maintaining implementation simplicity while enabling multi-object capability.
Data Source
AI summary
A method of the present disclosure includes (a) generating an input image by imaging a scene containing the M objects by a camera, (b) obtaining a feature map showing feature amounts relating to the N keypoints from the input image using a learned machine learning model with the input image as input and the feature map as output, (c) obtaining three-dimensional coordinates of the N keypoints belonging to each of the M objects using the feature map, and (d) determining positions and attitudes of one or more objects of the M objects using the three-dimensional coordinates of the N keypoints belonging to each of the M objects, wherein (c) includes (c1) obtaining M×N keypoints having undetermined correspondence relationships with the M objects and determining the three-dimensional coordinates of the M×N keypoints, and (c2) grouping the M×N keypoints to the N keypoints belonging to each of the M objects.


