Remote Desktop Video Stream Synchronization via Key Frame Prioritization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for remote access to desktop applications from mobile devices face delays and frame loss due to unsynchronized video streams, leading to a non-real-time user experience.
Innovation Solution
A method and system where a mobile device controls remote desktop applications by sending commands to a remote server, with image rendering and processing handled on the local computer, using a codec to optimize video stream quality and reduce traffic, allowing for real-time synchronization and improved user interaction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If video stream is transmitted over WAN network, then remote access is enabled, but delays and frame loss occur
Solution Approach 1:
The system segments video transmission by identifying and prioritizing key frames over incremental frames. Key frames contain complete image data while incremental frames contain only differences, allowing the system to recover from packet loss more effectively and reduce perceived delay by displaying key frames independently.
Solution Approach 2:
The system performs preliminary actions by pre-buffering and prioritizing key frame transmission before incremental frames. This ensures that even if some frames are lost during transmission, the receiver can reconstruct the image using key frames and previously received data, minimizing delays and frame loss impact.
2Manufacturing precision
If video stream quality is maintained, then image fidelity is preserved, but network traffic increases
Solution Approach 1:
The system dynamically changes transmission parameters by alternating between key frames (high data volume, complete image) and incremental frames (low data volume, image differences). This parameter variation maintains overall image fidelity while significantly reducing average network traffic compared to continuous full-frame transmission.
Solution Approach 2:
Instead of transmitting complete high-fidelity images continuously, the system transmits copies of only the changed portions (incremental frames) after an initial key frame. This copying approach preserves image fidelity for updated areas while minimizing network traffic by avoiding redundant transmission of unchanged regions.
3Loss of information
If all video frames are captured, then complete visual information is transmitted, but some frames are lost during transmission
Solution Approach 1:
The system provides beforehand cushioning by designing the video stream with redundant key frames that can serve as fallbacks. If incremental frames are lost during transmission, the receiver can use the most recent key frame as a cushion to reconstruct the image, preventing complete information loss and improving transmission reliability.
Solution Approach 2:
The system implements feedback mechanisms where the receiver acknowledges received frames and requests retransmission of lost critical frames. This feedback loop ensures that key frames and important incremental frames are reliably delivered, maintaining visual information completeness while managing transmission reliability through selective retransmission.
Data Source
AI summary
A computer-implemented method for transmitting and controlling images of a remotely executed application, the method including connecting a mobile computing device to a remote server; connecting a desktop computing device to the remote server and to the mobile computing device; starting an application on the remote server, wherein the application generates graphical objects controllable by commands from the mobile computing device; encoding each graphical object into a video stream; transferring the video stream to the mobile computing device and to the desktop computing device; generating the commands on mobile computing device; sending the commands to the remote server; detecting if the video stream contains results of execution of the commands; displaying the graphical objects on the local computing device; and generating an image of a substitute graphic object image if the video stream does not contain the results.