Stereo Image Shake Estimation via 3D Point Projection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image stabilization techniques face challenges in accurately separating camera work and shake from optical flow, leading to reduced accuracy in shake estimation, especially when camerawork such as forward movement is involved.
Innovation Solution
An image processing apparatus and method that obtain three-dimensional points corresponding to motion vector start and end points in stereo images, remove camerawork motion components, and calculate the shake amount by projecting these points into a three-dimensional space, allowing for precise shake estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If digital geometric correction is performed using optical flow from temporally successive image frames, then image stabilization can be achieved, but the accuracy of shake estimation is reduced when camerawork such as forward movement is involved because it becomes difficult to separate camerawork and shake from the optical flow
Solution Approach 1:
The patent transforms the problem from two-dimensional image plane analysis to three-dimensional space analysis. By converting corresponding points between left and right images into 3D coordinates using disparity information, the system analyzes motion in 3D space where camerawork and shake can be separated based on their different geometric characteristics. This dimensional transformation enables accurate shake estimation even during forward movement.
2Measurement precision
If optical flow is obtained from temporally successive image frames, then motion vectors can be detected, but the separation of camerawork and shake becomes difficult leading to reduced measurement precision
Solution Approach 1:
The patent segments the motion analysis into distinct components by processing left and right images separately. Corresponding points are identified in both images, converted to 3D coordinates independently, and then their relative motion is analyzed. This segmentation allows the system to distinguish between camerawork (affecting both images similarly) and shake (affecting the stereo pair differently), thereby improving measurement precision.
Solution Approach 2:
The patent introduces 3D coordinate conversion as an intermediary step between 2D image coordinates and motion analysis. By converting corresponding points from 2D image planes to 3D space using disparity information, the system creates an intermediate representation that preserves depth information. This intermediary transformation enables the separation of camerawork and shake components that cannot be distinguished in 2D optical flow alone.
3Measurement precision
If conventional shake detection methods are used that rely on optical flow from single images, then processing is simpler, but accuracy is reduced when forward movement or other camerawork is present
Solution Approach 1:
The patent merges information from both left and right images by identifying corresponding points in the stereo pair and converting them to 3D coordinates. This combination of stereo information creates a more robust representation of scene geometry that enables accurate shake detection even during camerawork. The merged 3D point cloud from both images provides sufficient information to separate camerawork from shake.
Data Source
AI summary
An apparatus includes first, second, and third units. The first unit is configured to obtain a first three-dimensional point based on an initial start point of a motion vector in one image of a stereo image and a point corresponding to the initial start point in the other image of the stereo image. The second unit is configured to obtain a second three-dimensional point based on an initial end point of the motion vector in the one image and a point corresponding to the initial end point in the other image. The third unit is configured to obtain, based on the first and second three-dimensional points, a third three-dimensional point for which a camerawork motion component has been removed from the second three-dimensional point and obtain, based on the third and first three-dimensional points, an amount of shake in the stereo image.