Omnidirectional Image Stitching with Angular Range Adjustment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In environments where cameras are limited in number and location, such as an airplane, it is challenging to acquire a contiguous multi-directional image, resulting in areas that fall outside the imaging range and fail to cover a natural omnidirectional view, especially when capturing views from a flying airplane for passenger display.
Innovation Solution
An image processing system that includes cameras with overlapping and non-overlapping fields of view, a server that stitches and processes images to generate omnidirectional image data by adjusting angular ranges, and a portable terminal that displays the processed image data, filling gaps with predetermined image data to create a seamless and natural all-round view.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a limited number of cameras with fixed fields of view are used to capture images, then the device complexity is reduced, but the imaging coverage becomes discontinuous and cannot achieve a natural omnidirectional view
Solution Approach 1:
The patent divides the omnidirectional imaging task into multiple segments by using multiple cameras with different pointing directions. Each camera captures a specific portion of the environment, and these segments are then stitched together through image processing to form a complete omnidirectional view, resolving the contradiction between limited camera numbers and continuous coverage
Solution Approach 2:
The patent introduces an intermediary image processing system that acts as a mediator between the limited camera array and the requirement for continuous omnidirectional coverage. This intermediary processes the discrete camera images by adjusting angular ranges and filling gaps to create a seamless virtual omnidirectional view, bridging the gap between hardware limitations and imaging requirements
2Reliability
If images from multiple cameras with different pointing directions are combined, then omnidirectional coverage is improved, but the imaging ranges may overlap or leave gaps resulting in unnatural displays
Solution Approach 1:
The patent applies parameter changes by adjusting the angular ranges of the captured images. Instead of using fixed fields of view, the system dynamically modifies the angular parameters of each camera's image data to optimize the stitching process. This allows the system to compensate for overlaps and gaps by redistributing angular coverage, thereby achieving more accurate and natural omnidirectional reconstruction
Solution Approach 2:
The patent employs partial action by selectively processing only the portions of images that are needed for optimal stitching. Rather than treating all image data equally, the system identifies and processes specific regions based on their spatial relationships and coverage requirements, filling gaps where necessary while avoiding redundant processing in overlapping regions
3Reliability
If the angular ranges of image data are expanded beyond the original fields of view, then the coverage continuity is improved, but the original image data becomes insufficient and gaps appear
Solution Approach 1:
The patent uses copying by generating synthetic image data that replicates the appearance of uncovered areas. When gaps are detected after expanding angular ranges, the system creates virtual image content that copies the characteristics of adjacent regions, maintaining visual continuity. This synthetic copying fills in the missing information without requiring additional physical cameras
Solution Approach 2:
The patent applies preliminary action by pre-processing the camera images to adjust their angular ranges before final stitching. The system anticipates potential gaps and overlaps by expanding angular coverage in advance, allowing the image processing algorithm to better align and transition between different camera views, thereby reducing the need for gap-filling operations
Data Source
AI summary
An image processing device for generating a single image based on a plurality of images captured by a plurality of imaging devices having different pointing directions from each other includes a memory and a controller. The memory is configured to store information containing at least the pointing directions and fields of view of the plurality of imaging devices. The controller is configured to operate as follows: to acquire or generate, from the plurality of images, first image data and second image data whose imaging ranges do not overlap; to apply image processing to the first image data and the second image data in such a manner that the first image data and the second image data have angular ranges larger than the fields of view in at least one direction; and to generate a single piece of multi-directional image data containing the image-processed first image data and second image data.