Latency Reduction in Drone-to-VR Image Transmission
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems experience high communication latency in transmitting image data from remote-controlled devices, such as drones, to receiver-controller devices like VR headsets, leading to lag and potential motion sickness due to the roundtrip latency in orientation transmission, gimbal processing, and video transmission.
Innovation Solution
The system reduces latency by determining a predicted field-of-view on the drone based on the user's head motion, cropping and encoding only the necessary image data, and transmitting it to the VR headset, which then synthesizes the updated view using motion tracking data, allowing for minimal latency in responding to head rotations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If stereo cameras are mounted to a mechanical gimbal to maintain constant intraocular distance, then depth perception is improved, but device complexity and bulk increase significantly
Solution Approach 1:
The patent replaces the mechanical gimbal system with a digital image processing approach. Instead of using stereo cameras mounted on a mechanical gimbal to maintain constant intraocular distance, the system uses a single camera capturing a wide-angle image and applies digital processing to synthesize depth information and generate appropriate views for the head-mounted device, thereby eliminating the complex mechanical components while achieving the same depth perception function
Solution Approach 2:
The patent extracts only the necessary portion of the captured image data for transmission. By determining the predicted field-of-view based on current and predicted head orientations, the system crops and transmits only the relevant image region rather than the entire captured image, reducing data transmission requirements while maintaining depth perception quality
2Ease of operation
If the field of view is increased to accommodate maximum head movement during roundtrip latency, then responsiveness to head motion is improved, but data transmission volume increases
Solution Approach 1:
The patent applies local quality by differentiating the treatment of different regions of the captured image. Instead of uniformly processing or transmitting the entire image, the system identifies and prioritizes the specific region corresponding to the predicted field-of-view, allocating transmission resources selectively to the most relevant areas while reducing or eliminating data from less relevant regions
3Stability of the object's composition
If a mechanical gimbal is used to rotate stereo cameras as a unit, then intraocular distance is maintained, but the system becomes bulky and functionality is limited
Solution Approach 1:
The patent substitutes the mechanical gimbal system with a computational approach using monocular images and depth data. Instead of physically rotating stereo cameras to maintain intraocular distance, the system processes a single wide-angle image with depth information to synthesize appropriate views for different head orientations, eliminating mechanical constraints and enabling greater system versatility
Solution Approach 2:
The patent changes the fundamental parameters of the imaging system from multiple physical cameras with fixed geometric relationships to a single camera with variable virtual viewpoints achieved through image processing. This parameter change allows the system to maintain the functional equivalent of constant intraocular distance without the physical constraints of a mechanical gimbal
Data Source
AI summary
Methods, apparatus, and computer-readable media are provided for processing image data captured by a first device for display on a second device. For example, a range of predicted orientations of the second device can be determined. A predicted field-of-view of the second device can then be determined. The predicted field-of-view corresponds to the range of predicted orientations of the second device. The predicted field-of-view can be transmitted to the first device. Cropped image data may then be received from the first device, which includes image data cropped to include the predicted field-of-view. An updated orientation of the second device can be determined, and an updated field-of-view within the cropped image data can be determined that corresponds to the updated orientation of the second device.