Parallax Image Combining for Smooth Viewpoint Movement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
There is no existing technique for generating a moving image with a viewpoint movement effect using parallax images, making it difficult for general camera users to achieve smooth viewpoint movement in video shooting and editing.
Innovation Solution
An image processing apparatus and method that obtain and combine parallax images from an image sensor with subpixels sharing a microlens, controlling the combining ratio to generate moving image data with a temporal change in the viewpoint movement effect, allowing for the creation of moving images with a smooth viewpoint movement effect without the need for complex equipment like dollies or rails.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional tracking or dollying techniques are used to achieve smooth viewpoint movement in moving images, then the viewpoint movement effect is improved, but the device complexity and cost increase due to requiring dollies, rails, and specialized equipment
Solution Approach 1:
The patent creates a virtual copy of the camera viewpoint by combining multiple parallax images captured from different angles. Instead of physically moving the camera on expensive dollies and rails, the system synthesizes a virtual camera position by computationally merging images from fixed multiple lenses, achieving the same viewpoint movement effect without specialized equipment
Solution Approach 2:
The patent replaces the mechanical system of physical camera movement (dollies, rails, tracking equipment) with a computational system that achieves viewpoint movement through image processing. By combining parallax images from multiple fixed lenses with different combining ratios, the system substitutes mechanical viewpoint change with optical-computational viewpoint synthesis
2Reliability
If multiple parallax images are combined with temporal changes in combining ratio to achieve smooth viewpoint movement, then the viewpoint movement effect is improved, but the computational load increases
Solution Approach 1:
The patent applies partial action by selectively combining parallax images from specific lenses based on the desired viewpoint position, rather than processing all possible image combinations. The combining ratio is adjusted to include only the necessary subset of parallax images for achieving the target viewpoint, reducing unnecessary computational operations while maintaining the smooth viewpoint movement effect
3Stability of the object's composition
If parallax images are combined in units of frames with temporal changing combining ratio, then the smoothness of viewpoint movement is improved, but the processing complexity increases
Solution Approach 1:
The patent implements dynamics by making the combining ratio variable across time and viewpoint position rather than static. The combining ratio changes dynamically as the viewpoint moves between different virtual positions, allowing smooth transitions between parallax images. This dynamic adjustment creates the perception of continuous smooth movement while the actual processing remains discrete frame-by-frame combination
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The solution enables the generation of moving images with a smooth viewpoint movement effect while reducing computational load and eliminating the need for specialized equipment, making it cost-effective and accessible for general camera users.
Implementation Method 1
an image sensor in which unit pixels each having a plurality of subpixels that share a microlens are two-dimensionally arranged
Data Source
AI summary
Disclosed are an image processing apparatus that can generate a moving image having a movement shooting effect with a simple configuration and an image processing method. The image processing apparatus combines a plurality of parallax images in units of frames in accordance with a combining ratio, and generates moving image data in which an image obtained by the combining is included in frames. The image processing apparatus controls the combining ratio so as to temporally change in the moving image data.