Motion Compensated Image Registration for Fused Video
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Real-time image fusion systems face challenges in combining video imagery from multiple sensors due to time lag disparities, leading to image mismatch and misregistration, especially when sensors or objects are in motion.
Innovation Solution
A system and method that compensates for image misregistration by calculating and adjusting the spatial offset between images from different sensors using motion sensors and processing delays, allowing for re-centering and fusion of images to produce a composite image.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If real-time image fusion is performed using multiple sensors, then the system provides comprehensive multi-spectral imagery, but image mismatch and misregistration occur due to time lag disparities between sensors
Solution Approach 1:
The system calculates spatial offsets based on sensor lag times and scene motion parameters before fusing the images. This preliminary compensation action ensures that images from multiple sensors with different time lags are pre-aligned to a common reference time, preventing misregistration in the final fused image
Solution Approach 2:
A reference sensor is selected as an intermediary time reference for the fusion process. All other sensors are temporally aligned to this reference sensor's timing, acting as a mediator that coordinates the timing of multiple sensors with different lag characteristics to achieve proper image registration
2Ease of operation
If sensors operate in dynamic environments with motion, then the system captures real-world scenarios, but image mismatch increases due to relative velocity between sensors and scene objects
Solution Approach 1:
The system dynamically adjusts the spatial offset compensation based on real-time scene motion parameters and sensor velocities. Rather than using fixed compensation values, the system continuously updates the alignment parameters to match the current dynamic state of the sensors and scene, maintaining registration accuracy during motion
Solution Approach 2:
The system uses motion sensors and scene analysis to provide feedback about the current velocity and motion state. This feedback is used to continuously update the temporal and spatial compensation parameters, creating a closed-loop system that adapts to changing motion conditions and maintains image registration accuracy
3Adaptability or versatility
If time lag disparity between sensors is large compared to video frame rate, then sensor diversity is utilized, but noticeable separation between video images becomes apparent to the viewer
Solution Approach 1:
The system performs preliminary temporal alignment by calculating and applying spatial offsets that compensate for large time lag disparities between sensors. This pre-alignment ensures that even when sensor lag exceeds the video frame rate, the images are properly registered before fusion, preventing noticeable separation to the viewer
Solution Approach 2:
The system creates a temporally referenced copy of the scene state at a common reference time by compensating for sensor lag. Instead of directly fusing images at different times, it reconstructs what each sensor would have captured at the reference time, enabling proper alignment even with large time disparities
Data Source
Figure 1~2
Figure 3
Figure 4~5
AI summary
A system (100) for compensating image misregistration between at least two image sensors (120,130) includes a first image sensor and a second image sensor which are disposed on a platform (170) and are configured to provide first and second output images, respectively. The system also includes a motion sensor (110) and a processor (140). The motion sensor senses movement of the platform. The processor calculates a lag time between the first and second image sensors based on first and second processing delay times of the first and second image sensors, respectively. The processor also calculates an image offset based on the lag time and the movement of the platform sensed by the motion sensor and offsets one of the first or second output image with respect to the other one of the first or second output image based on the image offset. A fuser combines the offset image with the non-offset image.