Multi-Stage Optical Flow Splicing for Low-Latency VR Video
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current VR video processing methods fail to provide a high-quality, immersive experience with low latency due to inefficiencies in video splicing and optical flow calculation, resulting in distortion and slower processing speeds.
Innovation Solution
A video processing method involving multi-stage optical flow calculation and splicing of overlapping regions from raw videos captured by multiple dynamic image recorders, using downsampling and gradient descent methods to improve accuracy and efficiency, while preserving image quality and reducing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional optical flow calculation is used for splicing raw videos, then the splicing process can be completed, but the processing speed is slow and latency is high
Solution Approach 1:
The patent divides the optical flow calculation into multiple stages with different sampling rates. The first stage uses a lower sampling rate to process a broader time range, while subsequent stages use higher sampling rates for more precise temporal analysis. This segmentation allows the system to achieve high processing speed for coarse temporal relationships while maintaining accuracy for fine-grained motion detection, thereby reducing overall latency without sacrificing precision.
Solution Approach 2:
The patent applies partial action by selectively calculating optical flow at different sampling rates based on the specific requirements of each video segment. For regions with minimal motion, lower sampling rates are sufficient, while regions with significant motion receive higher sampling rate processing. This selective approach reduces unnecessary computational overhead and accelerates processing speed while maintaining accuracy where needed.
2Measurement precision
If multi-stage optical flow calculation is performed on overlapping regions, then splicing accuracy is improved, but computational complexity increases
Solution Approach 1:
The patent segments the optical flow calculation into multiple stages, where each stage processes the overlapping region at a different sampling rate. The first stage uses a lower sampling rate to establish coarse correspondences, and subsequent stages refine these correspondences with higher sampling rates. This segmentation improves splicing accuracy by capturing both gross and fine motion details while managing computational complexity through hierarchical processing.
Solution Approach 2:
The patent introduces a temporal dimension to the optical flow calculation by using multi-stage sampling rates. Instead of uniformly processing all frames at the maximum sampling rate, the system varies the sampling rate across different time stages, adding a temporal dimension to the computational strategy. This approach improves accuracy by capturing motion at multiple temporal resolutions while reducing overall computational complexity.
3Manufacturing precision
If high resolution videos are processed, then image quality is maintained, but processing time increases
Solution Approach 1:
The patent segments the video processing into multiple stages with different sampling rates. In earlier stages, lower sampling rates are used to process a broader time range at reduced resolution, establishing basic motion patterns. In later stages, higher sampling rates are applied to specific regions or frames to maintain high image quality where needed. This segmentation allows the system to process high resolution videos while reducing overall processing time by avoiding full high-resolution processing across all frames.
Solution Approach 2:
The patent applies local quality by using higher sampling rates and processing power only for specific regions or frames where high image quality is critical, while using lower sampling rates for other regions. This localized approach maintains image quality in important areas while reducing processing time for the overall video, effectively balancing quality and speed requirements.
Data Source
AI summary
A video processing method, including: acquiring a plurality of raw videos, the plurality of raw videos being videos acquired by a plurality of dynamic image recorders arranged according to preset locations; determining an overlapping region between every two adjacent raw videos according to preset rules corresponding to the preset locations, the adjacent raw videos being the raw videos acquired by the dynamic image recorders arranged adjacent to each other; performing multi-stage optical flow calculation on the raw videos in each overlapping region to obtain a plurality of pieces of target optical flow information; and splicing the overlapping region of every two adjacent raw videos based on the target optical flow information to obtain target videos.