Dynamic Camera Frame Arbiter for Mixed Reality Scheduling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional mixed-reality systems, such as VR and AR, face inflexibility and scalability issues due to static scheduling mechanisms in head-mounted devices (HMDs) when allocating camera resources among competing image consuming components like head trackers, hand trackers, and depth detectors, leading to interference and reduced flexibility.
Innovation Solution
The introduction of an arbiter that dynamically allocates camera frame processing, allowing image consuming components to submit requests at their own frequencies, and dynamically determines when to process these requests, enabling asynchronous and flexible allocation of camera resources, which can prioritize, phase shift, or drop requests to manage interference.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If static scheduling mechanisms are used to allocate camera resources among image consuming components, then scheduling conflicts are avoided through predetermined timing, but the system loses flexibility and scalability when new components are introduced
Solution Approach 1:
The patent implements a dynamic scheduling mechanism where the arbiter continuously monitors and adjusts camera frame allocation based on real-time system conditions and component requirements, replacing static predetermined schedules with adaptive timing decisions that respond to changing system needs
Solution Approach 2:
The arbiter acts as an intermediary component between the camera system and multiple image consuming components, managing resource allocation conflicts and coordinating access timing to prevent scheduling conflicts while maintaining system flexibility
2Ease of operation
If multiple image consuming components submit camera frame requests at their own predetermined frequencies, then each component operates independently, but requests may interfere with one another when received at substantially the same time
Solution Approach 1:
The arbiter serves as a mediator that receives requests from multiple independent components operating at their own frequencies, buffers and manages these requests, and coordinates their processing to prevent interference while maintaining independent component operation
Solution Approach 2:
The arbiter performs preliminary assessment and buffering of incoming camera frame requests before processing, evaluating multiple simultaneous requests and determining optimal processing timing to prevent conflicts before they occur
3Stability of the object's composition
If rigid static scheduling is implemented in HMDs, then resource allocation is predictable, but the system effectively limits flexibility and scalability
Solution Approach 1:
The system transitions from rigid static scheduling to dynamic scheduling where the arbiter continuously adapts camera frame allocation based on real-time system conditions, maintaining predictability through systematic decision-making while enabling flexibility and scalability
Solution Approach 2:
The arbiter dynamically adjusts scheduling parameters such as frame timing and allocation priorities based on system conditions and component requirements, allowing the system to adapt to changing needs while maintaining controlled and predictable resource allocation
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Optimizations are provided for asynchronously performing camera frame allocations in which camera frame requests are received from multiple image consuming components that each compete for use of one or more shared cameras to obtain one or more digital images of a surrounding environment. After receiving a batch of multiple camera frame requests, an arbiter pre-processes those requests by determining an execution order for each request in the batch. Once this execution order is established, then the arbiter sends those requests to a camera system for processing. Processing a request includes obtaining one or more digital images of the surrounding environment. In this manner, significant advantages are realized because the arbiter dynamically allocates the camera system's processing.