Electronic Image Stabilization Warp Table for Drone FPV Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Drone racing experiences significant latency and video quality issues due to the lack of electronic image stabilization in live video feeds, resulting in jerky and unstable video streams, which can hinder pilots' reaction times and immersion during first-person view racing.
Innovation Solution
An apparatus comprising an image sensor and a processor that performs electronic image stabilization prior to video encoding, generating both a low-latency and high-quality video stream, with the ability to correct rolling shutter artifacts, and downscale video for wireless transmission, reducing latency and improving video stability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If high-quality video encoding is performed for live feed, then video quality is improved, but latency increases making it unsuitable for real-time control
Solution Approach 1:
The video processing system is segmented into two separate paths: a low-latency path that processes video quickly for real-time control without full encoding, and a high-quality path that performs full encoding for storage. This segmentation allows each path to be optimized for its specific purpose without compromise.
Solution Approach 2:
The system changes processing parameters dynamically based on the desired output. For the live feed, encoding is disabled and processing is minimized to achieve low latency. For high-quality video, full encoding parameters are applied. This parameter switching resolves the contradiction between quality and speed.
2Loss of time
If no video encoding is performed for live feed to reduce latency, then latency is reduced, but image stabilization is not performed resulting in jerky and unstable video
Solution Approach 1:
Image stabilization processing is extracted from the full encoding pipeline and applied independently in the low-latency path. This allows stabilization to be performed without undergoing the complete encoding process, maintaining low latency while improving video stability.
Solution Approach 2:
Image stabilization is performed as a preliminary action before video output in the low-latency path. By pre-processing the stabilization without waiting for full encoding, the system achieves both low latency and stable video output.
3Stability of the object's composition
If electronic image stabilization is performed on video signal, then video stability is improved, but processing time increases
Solution Approach 1:
The system dynamically adjusts the level of processing applied to video based on the output path. In the low-latency path, stabilization is performed with minimal processing overhead. In the high-quality path, full processing is applied. This dynamic approach optimizes the balance between stability and processing time for each use case.
Data Source
AI summary
An apparatus comprising an image sensor and a processor. The image sensor may be configured to generate a video signal based on a targeted view of an environment. The processor may be configured to (A) perform an image stabilization on the video signal and (B) generate (i) a first output video signal and (ii) a second output video signal. The first output video signal may be generated on a low-latency path. The second output video signal may be generated on an encoding path. The image stabilization may be performed prior to the generation of the first output video signal and the second output video signal.