Immersive 3D Capture Stage With Event-Camera Trajectory Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional machine vision systems lack inherent support for continuous surface representation of objects, making it difficult to efficiently determine if scanning information from multiple sensors represents the same points or objects, especially in immersive 3D video capture applications.
Innovation Solution
A stage studio system that uses a combination of frame cameras and event cameras with scanning signal generators to capture 2D video and generate trajectories, allowing for the creation of a 3D scene representation by aligning and modifying video streams based on overlapping fields of view and sensor outputs, enabling precise positioning and orientation of objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If multiple scanning sensors are used to concurrently scan the same scene, then the coverage and information richness are improved, but the difficulty of determining if scanning information from different sensors represents the same points or objects increases
Solution Approach 1:
The patent introduces a common reference frame as an intermediary that mediates between multiple scanning sensors. Each sensor's data is transformed into this common reference frame, enabling consistent identification and matching of points across different sensors without direct complex inter-sensor coordination.
Solution Approach 2:
The patent transforms scanning information from different sensor coordinate systems into a common reference frame coordinate system. This parameter transformation enables points from different sensors to be compared and matched by their coordinates in the common frame, resolving the matching difficulty.
2Device complexity
If conventional 2D or 3D representations are used, then the data processing is simplified, but the inherent support for continuous surface representation is lost
Solution Approach 1:
The patent transitions from discrete 2D image planes or sparse 3D point clouds to continuous 3D surface representations by introducing a common reference frame that accommodates continuous coordinate transformations. This enables inherent support for continuous surface representation while maintaining manageable data processing through systematic coordinate transformations.
Data Source
AI summary
Embodiments are directed providing a stage studio for immersive 3-D video capture. A two-dimensional video of a scene may be captured with frame cameras oriented towards a center of the scene. Paths may be scanned across objects in the scene with signal beams oriented towards the center of the scene. Events may be generated based on signal beams that are reflected by the objects and detected by event cameras oriented towards the center of the scene. Trajectories may be generated based on the paths and the events. A scene that includes a representation of the objects based on the captured two-dimensional video and the trajectories may be generated such that a position and an orientation of the represented objects in the scene are based on the trajectories.