Joint Position Detection Using Missing-Joint Relationship Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing joint detection systems inaccurately position joints that are not visible in images, leading to incorrect positioning of visible joints due to reliance on neural networks that refine positions based on incorrect data.
Innovation Solution
A joint detection apparatus and method that utilizes an all-feature-amount-outputting unit and a partial-feature-amount-outputting unit, employing machine learning to generate and refine feature amounts for visible and missing joints, using a convolutional neural network to learn positional relationships between joints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If neural networks are used to refine joint positions from heat maps, then joint position estimation is performed, but estimation accuracy decreases when joints are not visible in the image
Solution Approach 1:
The system performs preliminary action by generating training data that includes synthetic images with missing joint features before actual detection. The machine learning model learns positional relationships between joints during training using this pre-generated data, enabling accurate inference even when joints are not visible in the actual image. This preliminary preparation of training scenarios resolves the contradiction by pre-learning how to handle missing joint information.
Solution Approach 2:
The invention introduces an intermediary mechanism - a machine learning model that mediates between the heat map output and the final joint position estimation. This intermediary learns to compensate for missing joint information by inferring positional relationships from visible joints and their spatial relationships, thereby improving accuracy reliability without requiring direct visibility of all joints.
2Device complexity
If all joints are processed equally in the heat map, then processing is simplified, but incorrect positions of invisible joints drag visible joints to incorrect positions
Solution Approach 1:
The invention applies local quality by treating different joints differently based on their visibility status. The machine learning model assigns different weights and processing priorities to visible versus invisible joints, allowing the system to maintain simple heat map processing while correcting the impact of invisible joints on visible joint positions through learned positional relationships.
Solution Approach 2:
The system implements feedback by using the machine learning model to continuously refine joint position estimates based on the relationships between visible and invisible joints. The model learns from the spatial relationships between joints and provides feedback corrections to the initial heat map-based positions, preventing incorrect positioning while maintaining processing simplicity.
Data Source
AI summary
A learning-model generation apparatus 10 includes: an all-feature-amount-outputting unit that output, from image data of an object and for each joint of the object, a feature amount representing the joint; a feature-amount-generating unit that generates, from the feature amounts of the individual joints of the object and as training feature amounts, feature amounts in a case in which the feature amount of a certain joint is missing; and a learning-model-generating unit that, by using training data including the generated training feature amounts, generates a machine learning model by machine-learning positional relationships between the other joints in the case in which the feature amount of the certain joint is missing.