3D Model Interpolation for Rolling Shutter Camera Artifacts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In volumetric capture technologies, using captured images from rolling shutter type cameras can lead to inappropriate generation of 3D models due to exposure timing shifts for each scan line, resulting in missing sections in the 3D shape data.
Innovation Solution
An information processing device and method that includes a 3D shape data generation unit to generate 3D shape data from captured images and an interpolation unit to interpolate missing sections in the 3D shape data, ensuring complete and accurate 3D model generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If captured images from rolling shutter type cameras are used for 3D model generation, then the imaging speed and frame rate are improved, but the exposure timing shifts for each scan line cause missing sections in the 3D shape data
Solution Approach 1:
The patent introduces an intermediary process (image synthesis processing) that mediates between the rolling shutter captured images and the final 3D model generation. This intermediary step synthesizes multiple captured images to compensate for the exposure timing shifts, preventing information loss in the 3D shape data while maintaining the high imaging speed benefit of rolling shutter cameras
2Device complexity
If rolling shutter type image sensor is used in multi-view imaging device, then the device complexity is reduced and cost is lowered, but the 3D model generation accuracy deteriorates due to exposure timing shifts
Solution Approach 1:
The patent applies preliminary action by performing image synthesis processing before final 3D model generation. This preliminary step pre-compensates for the exposure timing shifts inherent in rolling shutter cameras, ensuring that when the 3D model is generated from the synthesized images, the accuracy is maintained without requiring complex hardware modifications
3Productivity
If captured images with exposure timing shifts are used directly for 3D model generation, then the processing time is reduced and productivity is improved, but the completeness of the 3D model deteriorates due to missing sections
Solution Approach 1:
The patent implements continuity of useful action by continuously synthesizing images from multiple viewpoints to fill in missing sections. Rather than interrupting the 3D model generation process, the image synthesis operates continuously on the captured images, ensuring that no useful information is lost while maintaining efficient processing throughput
Data Source
AI summary
The present technology relates to an information processing device, an information processing method, and a program that makes it possible to appropriately generate a 3D model of a modeling target on the basis of captured images obtained by capturing the modeling target from multiple viewpoints. 3D shape data representing a 3D shape of a modeling target, which is a target for generating a 3D model, is generated on the basis of a plurality of captured images obtained by capturing the modeling target from different viewpoint positions, and 3D shape data of a missing section of the modeling target is interpolated, the missing section being missing in the 3D shape data.