NUI Camera 6DoF Pose Calibration via 3D Environment Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current natural user interface (NUI) systems face challenges in accurately calibrating the six-degrees-of-freedom (6DoF) pose of a camera relative to a display, which is essential for precise gesture recognition and interaction, as existing methods lack robustness in mapping the camera's pose within the environment and relative to the user.
Innovation Solution
A method involving calibration video imaging from a calibration perspective to create a three-dimensional map of the environment, including the display and features, followed by primary video imaging from an operation perspective to determine the 6DoF pose of the NUI camera within this map, allowing for accurate tracking of user interactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional calibration methods are used for NUI camera pose estimation, then the calibration process is simpler, but the measurement precision of the 6DoF pose relative to the display and user is insufficient
Solution Approach 1:
The patent transitions from traditional 2D image plane calibration to 3D spatial calibration by constructing a three-dimensional map of the environment. The calibration video is processed to create a 3D representation that captures depth information, allowing the system to determine the camera's six-degree-of-freedom pose (three translational + three rotational degrees) relative to the display and user, thereby achieving higher measurement precision through dimensional expansion.
Solution Approach 2:
The patent introduces a three-dimensional map as an intermediary data structure between the calibration video and the final pose estimation. This 3D map serves as a mediator that transforms raw calibration images into structured spatial information, enabling accurate determination of the camera's 6DoF pose while systematically managing the complexity of the calibration process through intermediate representation.
2Reliability
If 6DoF pose calibration is implemented for accurate gesture recognition, then the reliability of gesture interaction improves, but the device complexity increases
Solution Approach 1:
The patent performs calibration actions in advance by capturing calibration video and constructing the three-dimensional map before actual gesture recognition operations. This preliminary calibration establishes the camera's 6DoF pose relative to the display and user upfront, ensuring reliable gesture recognition during subsequent operations without adding complexity to the real-time interaction system.
Solution Approach 2:
The calibration system uses the NUI camera itself to capture calibration video of the environment, display, and user. The same camera that will be used for gesture recognition performs the calibration, making the system self-calibrating and eliminating the need for separate calibration hardware, thereby improving reliability while managing complexity through self-service.
Data Source
AI summary
A method of calibrating a six-degree-of-freedom (6DoF) pose of a natural user interface (NUI) camera relative to a display is provided. Calibration video imaging an environment from a calibration perspective, which sites the display and one or more features, is received from the NUI camera or a calibration camera. A three-dimensional map of the environment, which defines a 6DoF pose of the display and a three-dimensional location of each of the one or more features, is modeled from the calibration video. Primary video imaging the environment from an operation perspective, which sites the one or more features, is received from the NUI camera. A 6DoF pose of the NUI camera is found within the three-dimensional map of the environment based on the operation perspective view of the one or more features.