XR Book Page Projection With 6DoF Tracking and Deformation Modeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing XR systems struggle to seamlessly integrate digital media content with physical books, lacking the ability to track and render content on arbitrary orientations and surfaces while maintaining user interaction and immersion.
Innovation Solution
An XR system that uses 6DoF tracking and SLAM techniques to detect and project digital media content onto physical book pages, allowing for dynamic rendering and interaction, including page deformation modeling and user gestures for control.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If digital media content is projected onto physical book pages using XR systems, then user immersion and interactivity are improved, but tracking precision and content alignment on arbitrary orientations and surfaces deteriorate
Solution Approach 1:
The patent introduces physical markers or fiducial markers placed on book pages as intermediary elements. These markers serve as reference points that the XR system's camera can reliably detect and track, enabling precise content alignment on arbitrary surfaces and orientations without requiring complex direct tracking of the book itself.
Solution Approach 2:
The system dynamically adjusts tracking parameters and camera field-of-view based on detected book orientation, distance, and angle. By changing these parameters in real-time, the system maintains measurement precision across varying viewing conditions and arbitrary surface orientations.
2Adaptability or versatility
If 6DoF tracking and SLAM techniques are implemented to enable dynamic content projection, then adaptability to different orientations and surfaces is improved, but device complexity increases
Solution Approach 1:
The patent divides the tracking task into separate components: marker detection, book orientation estimation, and content projection. This segmentation allows each component to be optimized independently, reducing overall system complexity while maintaining adaptability to different orientations and surfaces.
Solution Approach 2:
The system creates a virtual copy or digital twin of the book's physical state, including its orientation, position, and page curvature. This virtual model is then used to guide content projection, simplifying the complexity by working with a standardized digital representation rather than directly manipulating complex physical tracking data.
3Adaptability or versatility
If dynamic content projection and page deformation modeling are implemented, then user interaction and immersion are improved, but processing requirements and system complexity increase
Solution Approach 1:
The patent applies page deformation modeling selectively only to regions of the book pages that are currently visible or interacted with, rather than modeling the entire book structure. This partial action approach reduces processing requirements while maintaining immersion and interaction quality in the relevant viewing area.
Solution Approach 2:
The system pre-calculates and stores deformation models for common book types and page configurations. When a book is detected, the system selects the matching pre-computed model rather than performing full deformation analysis in real-time, significantly reducing processing requirements while maintaining accurate content alignment.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems and techniques are described herein for displaying digital media content (e.g., electronic books) on physical surfaces or objects. The systems and techniques can be implemented by various types of systems, such as by an extended reality (XR) system or device. For example, a process can include receiving, by an extended reality device, a request to display media content on a display surface. The process can include determining a pose of the display surface and a pose of the extended reality device. The process can include, based on the pose of the display surface and the pose of the extended reality device, displaying the media content by the extended reality device relative to the display surface.