Multi-Sensor Data Re-Projection and Fusion for Spatial Visualization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In applications using multiple cameras for environmental visualization, such as security monitoring and aerial imaging, the differing viewpoints from separate cameras can be difficult for operators to integrate, leading to challenges in understanding spatial relationships and tracking objects across views. Additionally, blind spots in camera coverage can result in missed events, and wide-angle cameras often produce distorted images that are hard to interpret.
Innovation Solution
A computer-implemented method and system for re-projecting and combining sensor data from multiple sensors into a new viewpoint, involving calibration of sensors to determine calibration values, generation of point clouds, and application of a matrix representing the new viewpoint. This process includes simultaneous localization and mapping (SLAM) to position and orient the re-projected data relative to each other, and the addition of supporting elements to the combined image.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If multiple separate camera viewpoints are displayed to operators, then coverage of the environment is improved, but understanding spatial relationships and tracking objects becomes difficult
Solution Approach 1:
The patent combines multiple separate camera viewpoints into a single synthesized viewpoint that presents a unified view of the environment. This merging process integrates data from multiple cameras to create one coherent image, allowing operators to understand spatial relationships and track objects across the entire coverage area without mentally integrating multiple separate views.
Solution Approach 2:
The system introduces an intermediary processing layer that transforms raw camera data from multiple viewpoints into a synthesized viewpoint. This intermediary process includes sensor calibration, coordinate transformation, and image fusion techniques that reconcile the different perspectives into a single coherent representation, making spatial understanding effortless for operators.
2Area of stationary object
If wide angle cameras are used to increase coverage scope, then the scope of coverage is improved, but image distortion increases making interpretation difficult
Solution Approach 1:
The system changes the parameters of the captured images through computational processing. Specifically, it applies distortion correction algorithms that modify the geometric parameters of wide-angle images, transforming them into undistorted representations. This allows the system to maintain the broad coverage scope of wide-angle cameras while eliminating the interpretation difficulties caused by distortion.
3Area of stationary object
If discrete video feeds are displayed in a bank of displays, then complete environmental coverage is achieved, but cognitive load on operators increases
Solution Approach 1:
The patent merges multiple discrete video feeds into a single synthesized viewpoint, eliminating the need for operators to mentally integrate information across multiple displays. This consolidation maintains complete environmental coverage while significantly reducing cognitive load by presenting all relevant information in one unified view.
Data Source
AI summary
There is provided a system and method of re-projecting and combining sensor data of a scene from a plurality of sensors for visualization. The method including: receiving the sensor data from the plurality of sensors; re-projecting the sensor data from each of the sensors into a new viewpoint; localizing each of the re-projected sensor data; combining the localized re-projected sensor data into a combined image; and outputting the combined image. In a particular case, the receiving and re-projecting can be performed locally at each of the sensors.