Mixed Reality Coordinate Transformation for Accurate Pass-Through Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing mixed reality systems face issues with visual discomfort and alignment inaccuracies due to direct application of depth warping algorithms, which cause warping effects and misalignment of mixed reality content with the pass-through view.
Innovation Solution
A method involving coordinate transformation and reference reprojection parameters is employed to convert rendering positions and combine virtual objects with pass-through views, using camera information to improve alignment and reduce distortion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a precise depth warping algorithm is directly applied to generate pass-through view, then alignment accuracy is improved, but visual comfort deteriorates due to warping effect
Solution Approach 1:
The patent introduces an intermediary coordinate transformation process that mediates between the camera's reference frame and the user's eye frame. By determining coordinate transformation information based on camera information and using it to convert rendering positions, the system avoids direct warping of the pass-through view while still achieving accurate alignment of virtual objects. This intermediary transformation resolves the contradiction by decoupling the alignment function from the visual rendering function.
Solution Approach 2:
The patent changes the rendering position parameter from the camera's reference position to the user's eye position by applying coordinate transformation information. This parameter change allows the system to maintain the original pass-through view without warping while achieving accurate alignment through the transformed virtual object positions. The reprojection parameter is also adjusted accordingly to maintain visual consistency.
2Measurement precision
If coordinate transformation is applied to convert rendering position, then alignment accuracy is improved, but device complexity increases
Solution Approach 1:
The patent performs preliminary coordinate transformation by determining coordinate transformation information based on camera information before the actual rendering process. This preliminary action includes calculating the transformation matrix and storing it for use during rendering. By preparing the transformation parameters in advance, the system reduces the computational burden during real-time rendering while maintaining accurate alignment.
Solution Approach 2:
The patent uses camera information (intrinsic and extrinsic parameters) as a template to generate the coordinate transformation information. Instead of performing complex real-time calculations, the system copies and adapts the camera's reference frame parameters to create the transformation matrix. This approach simplifies the processing by reusing well-defined camera calibration data rather than performing ad-hoc calculations.
3Measurement precision
If depth warping processing is applied to combine virtual objects with pass-through view, then alignment accuracy is improved, but visual experience deteriorates
Solution Approach 1:
The patent segments the rendering process into distinct independent steps: (1) obtaining reference image from color camera, (2) determining coordinate transformation information, (3) converting rendering position, (4) rendering virtual object at transformed position, and (5) combining with reference image. By separating the alignment function (coordinate transformation) from the visual rendering function (image combination), the system achieves accurate alignment without introducing warping distortion to the pass-through view.
Solution Approach 2:
Instead of warping the pass-through view to align virtual objects (conventional approach), the patent inverts the approach by transforming the virtual object positions to match the camera's reference frame. The virtual objects are rendered at positions that correspond to the camera's viewpoint, and then simply combined with the reference image. This inversion eliminates the need for depth warping and its associated visual distortions.
Data Source
AI summary
The embodiments of the disclosure provide a method for improving a mixed reality service and a host. The method includes the following. A reference image for rendering a pass-through view is obtained from a color camera of the host. Coordinate transformation information is determined based on camera information of the host. The coordinate transformation information is used to convert a rendering position from a preset rendering position to a reference rendering position. A reference reprojection parameter is determined based on a preset reprojection parameter, the camera information of the host, and the coordinate transformation information. A virtual object corresponding to a target object to be rendered is rendered based on object information of the target object to be rendered and the coordinate transformation information. The virtual object is combined with the reference image into the pass-through view. The pass-through view is reprojected into visual content of the mixed reality service based on the reference reprojection parameter, and the visual content is displayed.