Multi-Capture Device Synthesis for NUI Scene Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current natural user interface (NUI) systems using a single capture device often face limitations in resolution, illumination, and occlusion, making it inadequate for larger play spaces or scenes, necessitating the use of multiple capture devices to synthesize data from multiple sources effectively.
Innovation Solution
A system that calibrates and synchronizes multiple audio and visual capture devices to reconcile their data, determining their positions and orientations relative to each other and the scene, allowing for improved visual and audio data synthesis, noise reduction, and acoustic source localization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a single capture device is used, then the system is simple and easy to operate, but the resolution, illumination coverage, and ability to capture large scenes are inadequate
Solution Approach 1:
The system divides the large scene into multiple smaller fields of view, each captured by a separate capture device. Multiple capture devices are positioned at different locations to capture different portions of the scene, and their data is synthesized to create a complete high-resolution representation of the entire scene.
2Area of stationary object
If multiple capture devices are used, then the scene coverage and resolution are improved, but the complexity of calibrating and synchronizing the devices increases
Solution Approach 1:
A common reference scene serves as an intermediary to establish spatial relationships between multiple capture devices. By identifying key-point discontinuities and common cues within the image data of various capture devices relative to this reference, the system automatically determines relative positions and orientations without requiring complex manual calibration procedures.
3Loss of information
If multiple capture devices are used, then the ability to capture large scenes is improved, but the difficulty of detecting and measuring and synchronizing data from multiple sources increases
Solution Approach 1:
The system performs initial calibration by identifying key-point discontinuities and common cues in the image data from multiple capture devices. This calibration process establishes relative positions and orientations, which are then used to guide the synchronization and synthesis of audio and visual data from all devices, ensuring accurate temporal and spatial alignment.
4Volume of moving object
If multiple capture devices are used, then the play space size is increased, but the complexity of reconciling and stitching data from multiple sources increases
Solution Approach 1:
The system transitions from two-dimensional image data to three-dimensional spatial representation by determining the positions and orientations of multiple capture devices in 3D space. This dimensional transformation enables accurate stitching and synthesis of audio and visual data from multiple sources, creating a comprehensive three-dimensional representation of the expanded play space.
Data Source
AI summary
A system and method are disclosed for synthesizing information received from multiple audio and visual sources focused on a single scene. The system may determine the positions of capture devices based on a common set of cues identified in the image data of the capture devices. As a scene may often have users and objects moving into and out of the scene, data from the multiple capture devices may be time synchronized to ensure that data from the audio and visual sources are providing data of the same scene at the same time. Audio and/or visual data from the multiple sources may be reconciled and assimilated together to improve an ability of the system to interpret audio and/or visual aspects from the scene.