XR Sensory Event Labeling via Edge Cloud Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current extended reality (XR) technologies lack a method to efficiently encode and localize sensory data from the environment, generate real-time alerts, and provide situational awareness, particularly for sensory-impaired users, due to limitations in existing network connectivity and processing capabilities.
Innovation Solution
The proposed Locating, Labeling, and Rendering (LLR) mechanism leverages 5G network connectivity and edgecloud computing to encode sensory data, localize it in three-dimensional space, and convert it into real-time alerts and overlays, allowing for adaptive feedback across multiple sensory dimensions, including audio, visual, and haptic feedback.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If XR headsets use cables to connect to computers or tether to local WiFi networks, then speed and latency issues are addressed, but users cannot interact with environments and objects outside the tethered environment
Solution Approach 1:
The patent extracts the processing capability from the headset to external servers, allowing the headset to function wirelessly while maintaining high processing speeds. The headset only handles sensory input collection and basic display, while complex sensory data processing and localization are performed remotely on servers, eliminating the need for local high-speed processing hardware.
Solution Approach 2:
The patent introduces servers as intermediaries between the headset and the environment. The servers receive sensory data from the headset, process it using machine learning models, and generate appropriate XR feedback. This intermediary architecture allows wireless operation while maintaining high processing capabilities.
2Adaptability or versatility
If XR experiences include rich three-dimensional audio and tactile sensations, then immersive sensory environment is enhanced, but processing complexity and data requirements increase
Solution Approach 1:
The patent extracts complex sensory processing from the headset to external servers. The headset only collects raw sensory data and displays basic XR content, while multi-dimensional sensory processing, localization, and feedback generation are performed remotely, reducing device complexity while enhancing sensory capabilities.
Solution Approach 2:
The patent creates a universal processing platform that handles multiple sensory modalities (visual, auditory, tactile, olfactory) through a single server-based architecture. This multi-functional approach allows the system to process diverse sensory inputs and generate corresponding feedback without requiring separate specialized hardware for each sensory type.
3Loss of information
If sensory data is continuously gathered and processed in real-time, then situational awareness is improved, but data processing time and computational resources increase
Solution Approach 1:
The patent implements preliminary action by pre-training machine learning models offline before deployment. The models are trained on extensive sensory datasets beforehand, enabling them to rapidly process real-time sensory inputs without requiring extensive computational resources during actual operation. This pre-processing approach minimizes real-time processing time while maintaining high situational awareness.
Solution Approach 2:
The patent replaces traditional mechanical signal processing methods with machine learning-based computational approaches. Instead of using rule-based or threshold-based processing, the system employs trained neural networks that can rapidly classify and interpret sensory data, significantly reducing processing time while improving accuracy of situational awareness.
Data Source
AI summary
The present disclosure relates to labeling, prioritization, and processing of event data. In some embodiments, a-A system receives event data from one or more sensors, wherein the event data represents an event detected in a sensory environment, and processes the event data, including: performing a localization operation using the event and performing a labeling operation using the event data. The system determines, based on a set of criteria, whether to perform a prioritization action related to the event data and, in response to determining to perform a prioritization action, performs one or more prioritization action including causing prioritized output, by one or more sensory feedback devices of a user device, of sensory feedback in at least one sensory dimension based on the event data, the location data, and the one or more label.