Virtual Viewpoint Image Shake Correction via Positional Change Calculation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing technologies fail to effectively correct image shake caused by hand shake and positional variations of the imaging unit, often resulting in excess correction and unnecessary image shake when combining image compositing with conventional shake correction methods.
Innovation Solution
An image pickup apparatus with a processor and memory that acquires data from multiple viewpoint images, calculates positional changes of the imaging unit, and generates a virtual viewpoint image using a mixture ratio to correct shake, combining electronic and optical shake correction methods.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional shake correction methods are combined with image compositing, then shake correction capability is improved, but excess correction occurs causing unnecessary image shake
Solution Approach 1:
The patent performs preliminary calculation of the correction quantity by determining the amount of change in position of the imaging unit before executing the shake correction. This preliminary action allows the system to precisely determine the necessary correction amount and avoid excess correction that would cause unnecessary image shake.
Solution Approach 2:
The patent uses feedback by calculating the correction quantity based on the actual amount of change in position of the imaging unit, which is determined through ranging operations. This feedback mechanism ensures that the shake correction is precisely matched to the actual shake occurrence, preventing both insufficient and excessive correction.
2Device complexity
If shake correction is performed without considering positional variation of imaging unit, then processing complexity is reduced, but correction precision deteriorates
Solution Approach 1:
The patent integrates multiple functions into a unified processing flow: the image compositing unit performs both image blending and positional variation calculation, while the shake correction unit uses this positional information for precise correction. This multi-functionality approach maintains reasonable processing complexity while achieving high correction precision through comprehensive use of available data.
Data Source
AI summary
In an image pickup apparatus, a parallax image acquisition unit acquires data of a plurality of viewpoint images having different viewpoints from an imaging unit. A detection unit detects the amount of change in a position or an attitude of the imaging unit which has generated the data of the plurality of viewpoint images. A calculation unit calculates a correction quantity of a shake correction unit and a mixture ratio of images when an image compositing unit generates a composite image from the plurality of viewpoint images on the basis of the amount of change in a position or an attitude of the image pickup apparatus. The image compositing unit generates a composite image from the plurality of viewpoint images using the calculated mixture ratio. The shake correction unit corrects image shake of the viewpoint images or the composite image according to the calculated correction quantity.