Selective Frame Transmission for Mixed Reality Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for remote XR-based object detection in mixed reality applications for autonomous vehicles are resource-intensive, leading to high bandwidth consumption and latency due to the transmission of full or substantial image frames, which do not adapt to network conditions.
Innovation Solution
A system and method that selectively transmit frames based on network quality metrics, user focus areas, and a pending frame queue, using reality devices with cameras to capture frames and processors to skip unnecessary frames and transmit only the necessary ones to an edge server for processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If full or substantial image frames are transmitted for remote XR-based object detection, then processing accuracy is improved, but bandwidth consumption increases and latency increases
Solution Approach 1:
The patent extracts and transmits only the most essential frame elements (key frames and difference maps) rather than full image frames. The system identifies and transmits only the critical visual information needed for object detection, separating essential data from redundant data to reduce bandwidth consumption while maintaining detection accuracy.
Solution Approach 2:
The patent segments the frame transmission into two distinct components: key frames that contain complete visual information and difference maps that contain only the changes from previous frames. This segmentation allows the system to transmit minimal data while preserving all necessary information for accurate object detection.
2Measurement precision
If full or substantial image frames are transmitted for remote XR-based object detection, then processing accuracy is improved, but latency increases
Solution Approach 1:
The patent extracts and transmits only the most essential frame elements (key frames and difference maps) rather than full image frames. The system identifies and transmits only the critical visual information needed for object detection, separating essential data from redundant data to reduce bandwidth consumption while maintaining detection accuracy.
Solution Approach 2:
The patent implements periodic transmission of key frames interspersed with more frequent difference maps. This periodic action pattern allows the system to maintain continuous object detection capability while reducing overall data transmission volume and latency, as difference maps are smaller and can be transmitted more quickly.
3Quantity of substance
If selective frame transmission is implemented based on network quality metrics, then bandwidth usage is reduced, but system complexity increases
Solution Approach 1:
The patent incorporates feedback mechanisms where the system continuously monitors network quality metrics and adjusts frame transmission decisions in real-time. The processors analyze network conditions and feedback from previous transmissions to dynamically determine which frames to send, optimizing bandwidth usage while managing system complexity through adaptive control.
Solution Approach 2:
The patent implements dynamic frame transmission where the transmission strategy adapts based on current network conditions. The system transitions between different transmission modes (key frames only, difference maps only, or combinations) depending on real-time network quality, making the system flexible and efficient while managing complexity through conditional logic.
Data Source
AI summary
System and method for reducing latency and bandwidth usage include a reality device and one or more processors. The reality device includes a camera to operably capture a set of consequent frames of views external to a vehicle. The one or more processors are operable to select one or more frames of the set of consequent frames and skip rest of the set of consequent frames based on network quality metrics, user focus areas of a user, and a pending frame queue, and transmit the selected one or more frames to an edge server for performing a task on behalf of the vehicle.