Optical Flow Vector Shifting for Latency Compensation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Unmanned vehicles experience deviations and potential collisions due to the latency in network transmission of video streams, leading to non-real-time image reception during remote control operations.
Innovation Solution
A method and system that generate a virtual image by calculating optical flow vectors, modifying them based on network latency and vehicle parameters, and shifting frames to compensate for latency, thereby reducing transmission delays and improving real-time control.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If video stream data is transmitted through a mobile broadband network, then the images can be transmitted to a remote display, but the network latency causes non-real-time transmission resulting in deviation or collision of the unmanned vehicle
Solution Approach 1:
The system performs preliminary actions by predicting the future position of the unmanned vehicle based on its current motion state and network latency characteristics. The virtual image generation proactively compensates for the expected delay, showing where the vehicle will be rather than where it currently is, thus maintaining real-time control accuracy despite network latency
Solution Approach 2:
The system applies preliminary anti-action by pre-compensating for the harmful effect of network latency. Through optical flow analysis and motion prediction, the system generates counteracting adjustments to the video feed that neutralize the delay effect, preventing the deviation and collision issues before they occur
2Reliability
If optical flow vectors are calculated and frames are shifted to generate a virtual image, then the deviation caused by network latency is reduced, but the computational complexity and processing requirements increase
Solution Approach 1:
The system segments the complex image processing task into distinct modules: optical flow vector calculation, motion prediction, and frame shifting. By dividing the processing into these manageable segments, the system reduces overall computational complexity while maintaining control accuracy through specialized handling of each processing stage
Solution Approach 2:
The system applies partial action by focusing computational resources on key areas that most impact control accuracy, such as calculating optical flow vectors only for relevant regions of interest in the video feed, rather than processing every pixel uniformly, thus reducing overall computational burden while maintaining essential control accuracy
Data Source
AI summary
The present disclosure provides a method for generating a video frame and a system thereof, including: receiving at least two frames of a video captured by an image capture unit through a network; calculating a first set of optical flow vectors of the at least two frames by a first algorithm; generating a set of modified vectors according to at least one parameter; combining the set of modified vectors and the first set of optical flow vectors to obtain a second set of optical flow vectors; and shifting one of the at least two frames according to the second set of optical flow vectors to generate a virtual image. Therefore, the present disclosure can reduce deviation caused by the latency of the network and improve user experience.