Bone-Guided 3D Model Generation Under Multi-Player Occlusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Occlusion in 3D model generation due to limitations in camera arrangement and number of players in volumetric image capture environments, such as sports games, leads to inaccurate 3D model generation.
Innovation Solution
An image processing device and method that estimates bone information from captured images and generates a 3D model based on this information, even when occlusion occurs, using a bone information estimation unit and a 3D model estimation unit to handle occluded areas by deforming rigged 3D models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If volumetric image capture is performed with limited camera arrangement and number of cameras, then device complexity is reduced, but measurement precision of 3D model generation deteriorates due to occlusion
Solution Approach 1:
The system performs preliminary actions by capturing images at multiple time points before the occlusion occurs, storing these images in a buffer. When occlusion is detected at a predetermined time, the system retrieves previously captured non-occluded images from the buffer to generate the 3D model, ensuring accurate measurement despite current occlusion conditions.
Solution Approach 2:
The system creates a copy of the 3D model generation process by using previously captured images as substitutes for the current occluded images. This copying approach allows the system to maintain measurement precision by referring to accurate historical data rather than relying on compromised current data.
2Adaptability or versatility
If the number of players to be generated is increased, then adaptability of the system improves, but measurement precision deteriorates due to occlusion between players
Solution Approach 1:
The system captures images of all players at multiple time points before occlusion occurs, storing these images in association with each player's identification information. When occlusion is detected during processing, the system retrieves previously captured non-occluded images from the buffer to maintain accurate 3D model generation for each player, regardless of the number of players being processed simultaneously.
3Productivity
If 3D model generation is performed in real-time, then productivity is improved, but measurement precision deteriorates when occlusion occurs
Solution Approach 1:
The system performs preliminary image capture at multiple time points and stores these images in a buffer before occlusion occurs. During real-time processing, when occlusion is detected, the system quickly retrieves pre-captured non-occluded images from the buffer, maintaining both high productivity through real-time processing and high measurement precision by using accurate historical image data.
Data Source
AI summary
The present disclosure relates to an image processing device, an image processing method, and a program that are capable of generating a 3D model with high accuracy even when occlusion has occurred.The image processing device includes: a bone information estimation unit that estimates, based on captured images at a predetermined time captured by a plurality of imaging devices, bone information of a person appearing in the captured images; and a 3D model estimation unit that, when occlusion has occurred on the person appearing in the captured images at the predetermined time, estimates a 3D model of the person based on the bone information at the predetermined time. The technology of the present disclosure can be applied to, for example, an image processing device that distributes free viewpoint video.