Stereo 360-Degree Image Capture Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Capturing full stereoscopic 360-degree video remains impractical due to high hardware costs and processing time, making it difficult for many applications, despite advancements in capturing full-spherical 360-degree content.
Innovation Solution
A system comprising two image capture devices with wide-angle or fisheye lenses, coupled via a connecting member, and a controller that geometrically and photometrically aligns images, corrects misalignment, and adjusts disparity to generate equirectangular images, enabling efficient stereo image capture and processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple image sensors are used to capture full stereo 360-degree video, then stereoscopic quality is improved, but hardware cost and device complexity increase significantly
Solution Approach 1:
The patent divides the 360-degree capture task into two separate 180-degree captures using two image capture devices positioned at different locations. Each device captures a hemisphere, and these are subsequently combined through geometric and photometric alignment to create the full stereoscopic 360-degree video, reducing the need for complex multi-sensor arrangements
Solution Approach 2:
The system uses two identical image capture devices positioned at standard interpupillary distance to capture left and right eye views simultaneously. This copying approach simplifies the hardware configuration compared to using multiple different specialized sensors, while maintaining stereoscopic quality
2Measurement precision
If multiple image sensors are used to capture full stereo 360-degree video, then stereoscopic quality is improved, but processing time increases
Solution Approach 1:
The patent performs geometric alignment and photometric alignment during the capture phase rather than as separate post-processing steps. The controller aligns images from the two devices in real-time, reducing overall processing time while maintaining stereoscopic quality
Solution Approach 2:
The system transforms images to equirectangular projection format and adjusts disparity parameters during alignment. By changing the projection parameters and performing alignment operations in this standardized format, the processing becomes more efficient compared to working with raw spherical images from multiple sensors
3Adaptability or versatility
If images from multiple devices are combined, then full stereoscopic 360-degree coverage is achieved, but image misalignment and disparity issues arise
Solution Approach 1:
The controller acts as an intermediary that performs geometric alignment and photometric alignment between images from the two devices. It uses feature point detection and matching to identify corresponding points, then applies transformation operations to align the images, resolving misalignment issues automatically
Solution Approach 2:
The system uses disparity measurement as feedback to iteratively adjust the alignment of images from the two devices. By measuring horizontal and vertical disparities and using these measurements to refine the alignment transformation, the system achieves precise registration while maintaining full 360-degree coverage
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The system reduces hardware costs and processing time, enabling practical capture and presentation of full stereoscopic 360-degree video by aligning and correcting image disparities, improving user experience with reduced eyestrain and enhanced depth cues.
Implementation Method 1
two image capture devices 110, 120 with wide-angle or fisheye lenses
Implementation Method 2
wide-angle or fisheye lenses, coupled via a connecting member
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An assembly includes a pair of image capture devices that capture 360-degree, stereo cubemap representation images of a scene. A controller generates a representation of the scene by correcting errors caused by placement of the image capture devices relative to each other in the assembly. The controller rotates an image from the image capture device to align objects in the image with objects in an image from the additional image capture device. Additionally, the controller replaces portions of an image from the image capture device including the additional image capture device with portions of an image from the additional image capture device. Additionally, the controller uses optical flow to cancel horizontal disparity and vertical disparity between images captured by the image capture device and by the additional image capture device.