User-Centric Mixed Reality Detection with Eye-Tracked ROI Transmission
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current MR technologies face high latency and bandwidth usage due to transmitting full or substantial image frames for remote processing, which is resource-intensive and leads to network congestion and delays, degrading the user experience.
Innovation Solution
A system that uses user-centric adaptive object detection by monitoring eye and head movements to identify areas of interest and crops frames to transmit only relevant areas to an edge server for processing, reducing latency and bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If full or substantial image frames are transmitted for remote processing, then processing accuracy is maintained, but bandwidth usage increases and latency increases
Solution Approach 1:
The patent segments the image frame into multiple regions of interest (ROIs) based on eye-tracking data and driving context. Only these segmented ROIs are transmitted to the edge server for processing, rather than transmitting the entire frame. This segmentation approach maintains processing accuracy for critical areas while significantly reducing overall bandwidth consumption.
Solution Approach 2:
The patent extracts and transmits only the essential portions of the image frame (the ROIs containing objects of interest to the driver) to the edge server. By taking out and transmitting only these critical regions rather than the complete frame, the system reduces bandwidth usage while preserving the accuracy needed for safety-critical object detection.
2Reliability
If full or substantial image frames are transmitted for remote processing, then processing completeness is maintained, but latency increases
Solution Approach 1:
The patent divides the image transmission task into segmented ROIs that are processed independently and in parallel. This segmentation enables the edge server to process multiple small regions simultaneously rather than waiting for complete large frames, significantly reducing processing latency while maintaining detection reliability for each segmented region.
Solution Approach 2:
The system performs preliminary actions by pre-identifying ROIs using eye-tracking data and driving statistics before transmission. This preliminary segmentation and prioritization allows the edge server to immediately begin processing the most critical regions as they arrive, rather than waiting for complete frames, thereby reducing overall latency while maintaining processing completeness.
3Loss of energy
If eye-tracking-based ROI identification is implemented, then bandwidth usage is reduced, but device complexity increases
Solution Approach 1:
The reality device performs self-service by using its own eye-tracking sensors and processing capabilities to automatically identify and segment ROIs based on driver gaze and driving context. This self-service approach eliminates the need for external systems to determine what regions to transmit, reducing overall system complexity despite adding local processing functions.
Solution Approach 2:
The reality device integrates multiple functions including eye-tracking, ROI identification, frame segmentation, and selective transmission into a single multi-functional system. By combining these functions into one device rather than requiring separate systems, the patent reduces overall device complexity while achieving bandwidth reduction through intelligent ROI-based transmission.
Data Source
AI summary
System and method for reducing latency and bandwidth usage in reality devices include a reality device and one or more processors. The reality device includes a camera to operably capture a set of consequent frames of views external to a vehicle. The one or more processors are operable to monitor movements of one or more eyes of a user to obtain eye-tracking data using the reality device, identify an area of interest (AoI) in the frame based on the eye-tracking data, AoI history, or driving statistics of the user, crop the frame to obtain a reduced-size frame including the AoI in the frame, and transmit the reduced-size frame to an edge server for performing a task on behalf of the vehicle.