Mixed Reality Coordinate-Frame Registration Without Fiducial Markers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing mixed-reality technologies face challenges in efficiently and accurately registering coordinate frames between head-mounted displays (HMDs) and other entities, such as 3D models or robots, due to the reliance on fiducial markers or keypoints that are time-consuming, prone to damage, and pose security risks, especially in environments lacking discernable features.
Innovation Solution
A method that uses minimal human interaction to register HMD coordinate frames by having users point at, gaze at, or touch real-world features with known 3D positions, capturing sensor data to compute correspondences, and apply geometric calculations like the Kabsch algorithm to determine registration information, eliminating the need for fiducial markers and keypoints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If fiducial markers or keypoints are used for coordinate frame registration, then registration accuracy can be achieved, but the process becomes time-consuming and the system becomes vulnerable to damage and security risks
Solution Approach 1:
The patent extracts and eliminates the dependency on fiducial markers and keypoints from the registration process. Instead of using these artificial features, the system directly utilizes sensor data from the HMD to compute correspondences between real-world features and the second coordinate frame, thereby removing the time-consuming steps of marker placement and detection while maintaining registration accuracy
Solution Approach 2:
The patent replaces the mechanical/physical fiducial markers with a sensor-based computational approach. The HMD's sensors (cameras, depth sensors, IMUs) capture data and compute 3D positions of real-world features directly, substituting the physical marker system with an optical and computational system that is faster and more secure
2Reliability
If fiducial markers are used for registration, then coordinate frame mapping can be established, but the system becomes vulnerable to damage and security risks
Solution Approach 1:
The patent removes the physical fiducial markers from the system, eliminating their vulnerability to damage and theft. The registration process now relies on detecting natural features in the environment through sensors, which cannot be damaged or stolen, thereby improving reliability and eliminating security risks
Solution Approach 2:
The system uses the HMD's own sensors and the environment's natural features to perform registration, without requiring external fiducial markers. The real-world features serve themselves as registration targets, eliminating the need for separate marker objects that could be compromised
3Adaptability or versatility
If traditional registration methods are used, then coordinate frames can be registered, but the process is complex and requires fiducial markers that may not be available in all environments
Solution Approach 1:
The patent creates a universal registration method that works across different environments without requiring specific fiducial markers. The HMD sensors can detect and use any distinguishable real-world features (edges, corners, surfaces) as registration targets, making the system adaptable to diverse environments including those lacking traditional markers
Solution Approach 2:
The patent replaces the complex mechanical system of fiducial marker placement and detection with a simpler sensor-based system. The HMD's existing sensors (cameras, depth sensors) are used to directly compute feature positions and establish correspondences, reducing overall system complexity while improving environmental adaptability
Data Source
AI summary
A method of registering a coordinate frame of an HMD with a second coordinate frame comprises receiving sensor data depicting the wearer of the HMD pointing at, gazing at, or touching, a real world feature, where the 3D position of the real world feature in the second coordinate frame is known. The method computes a 3D position of the real world feature, in the coordinate frame of the HMD, from the sensor data. A correspondence is stored comprising: the 3D position of the real world feature in the coordinate frame of the HMD, and a 3D position of the real world feature in the second coordinate frame. The method repeats so that a second correspondence is stored. The method registers the coordinate frame of the HMD and the second coordinate frame by computing registration from the correspondences.