Multi-Camera Image Registration Using Temporal Coherence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Achieving precise image registration among multiple cameras, especially when there are geometrical differences due to changes in distance and elevation angles, is challenging, particularly in remote sensing applications using infrared cameras where optical flow assumptions are not valid.
Innovation Solution
The system creates extended mini-frames by stitching together sequential images taken over time, utilizing temporal coherence to increase overlap and ease registration, and employs a rigid model to determine relative positions and rotations of cameras, allowing for accurate alignment even with minimal initial overlap.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional image registration methods are used with multiple cameras, then processing power requirements increase, but registration precision deteriorates
Solution Approach 1:
The patent divides the image registration process into multiple segments: first registering images from individual cameras to a common coordinate system, then registering sequences of images from each camera. This segmentation allows complex multi-camera registration to be broken down into simpler single-camera registration tasks, reducing overall computational complexity while maintaining precision.
Solution Approach 2:
The patent performs preliminary registration of individual cameras to establish their relative positions and orientations before processing the actual image sequences. This preliminary action creates a foundation that simplifies subsequent registration tasks, as the system already knows the spatial relationships between cameras without needing to compute everything from scratch for each image pair.
2Measurement precision
If geometrical differences due to distance and elevation angle changes are considered, then registration accuracy improves, but computational complexity increases
Solution Approach 1:
The patent dynamically adjusts the registration process based on changes in distance and elevation angles between frames. Instead of using a fixed registration model, the system adapts to temporal variations in camera geometry, applying different transformation models as needed. This dynamic approach maintains accuracy under varying conditions without requiring a completely complex static solution.
Solution Approach 2:
The patent changes registration parameters based on the specific geometric conditions of each image pair. When distance and elevation angle changes are significant, the system switches to transformation models that account for these variations. This parameter adaptation allows the system to maintain high accuracy only where needed, reducing overall computational complexity compared to always using the most complex model.
3Ease of operation
If extended mini-frames are created by stitching sequential images, then overlap increases and registration becomes easier, but processing time increases
Solution Approach 1:
The patent creates extended mini-frames by stitching together sequential images from each camera before performing the final registration. This preliminary stitching action increases the amount of overlapping information available, making the subsequent registration process easier and more accurate. By preparing this extended data structure in advance, the system can perform simpler, faster registration operations on the enriched data.
Solution Approach 2:
The patent maintains continuous processing of image sequences, creating extended mini-frames as it processes each frame. Rather than batch processing all images at once or stopping between operations, the system continuously stitches and registers images in a flowing manner. This continuous action reduces idle time and keeps the processing pipeline efficient, minimizing total processing time while building up the useful overlap information.
Data Source
AI summary
A video imaging system for use with or in a mobile video capturing system (e.g., an airplane or UAV). A multi-camera rig containing a number of cameras (e.g., 4) receives a series of mini-frames (e.g., from respective field steerable mirrors (FSMs)). The mini-frames received by the cameras are supplied to (1) an image registration system that calibrates the system by registering relationships corresponding to the cameras and/or (2) an image processor that processes the mini-frames in real-time to produce a video signal. The cameras can be infra-red (IR) cameras or other electro-optical cameras. By creating a rigid model of the relationships between the mini-frames of the plural cameras, real-time video stitching can be accelerated by reusing the movement relationship of a first mini-frame of a first camera on corresponding mini-frames of the other cameras in the system.