Skeleton Estimation from Occluded Joints Using Blank-Region Expansion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional skeleton estimation techniques using deep learning, such as OpenPose, struggle to accurately estimate joint positions when certain joints like the neck or shoulder are not visible in the image, posing challenges for privacy protection and work analysis.
Innovation Solution
An estimation device that includes an image acquisition part, a blank region expansion part, and an estimation part to generate and process images with expanded blank regions, using a learned estimation model to estimate joint positions in these regions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional skeleton estimation techniques (e.g., OpenPose) are used, then joint positions can be estimated from images, but the estimation becomes unstable when certain joints (e.g., neck, shoulder) are not visible in the image
Solution Approach 1:
The system performs preliminary action by generating synthetic training data that includes cases where certain joints are not visible. The data generation unit creates images with occluded joints and corresponding skeleton information, allowing the estimation model to learn how to predict missing joint positions based on visible joints and spatial relationships, thereby improving reliability when joints are not visible during actual estimation
Solution Approach 2:
The system changes parameters by introducing occlusion parameters during data generation. The data generation unit varies the degree and location of joint occlusion in training images, enabling the estimation model to adapt to different visibility conditions and maintain accurate estimation even when certain joints are not visible in the input image
2Object-affected harmful factors
If images are captured from angles that protect privacy (e.g., from above without showing face), then privacy protection is improved, but conventional techniques cannot stably estimate certain joint positions
Solution Approach 1:
The system prepares in advance by generating training data that specifically includes images captured from privacy-protecting angles where certain joints are not visible. The data generation unit creates synthetic training pairs with top-down views and corresponding skeleton information, enabling the estimation model to learn accurate joint position prediction from such angles without requiring face or sensitive area visibility
Solution Approach 2:
The system transitions to another dimension by considering spatial relationships and body geometry in 3D space. The estimation model uses the relative positions of visible joints and inferred spatial relationships to predict the positions of occluded joints, effectively using dimensional reasoning to recover missing information from 2D images taken from privacy-protecting angles
Data Source
AI summary
In an image having a blank region, the present invention estimates skeleton information having a part of a joint in the blank region. An estimation device (1) is provided with: an input part (130) for acquiring a first image that includes a first joint and does not include a second joint of the subject; a blank region expansion part (101) for generating a second image from the first image by expanding the blank region; and an estimation part (12) for estimating, using the second image and a prelearned estimation model, skeleton information that includes the joint position of the second joint located in the blank region.