Real-Time Screen Sharing With Latency-Based Encoding Pauses
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing screen sharing technologies struggle to accurately reduce latency and ensure image quality due to inadequate consideration of network conditions and receiver decoding capabilities, leading to poor user experience.
Innovation Solution
Pause image processing on the transmitting device when certain conditions are met, adjust quantization parameters based on pause time and latency, and use multiple codecs for partial image regions to optimize data transmission, without estimating network bandwidth or decoding speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If video encoding and transmission is performed continuously at high frame rates, then image quality and smoothness are improved, but transmission latency increases and network bandwidth is consumed
Solution Approach 1:
The patent implements periodic pausing of image processing operations based on latency thresholds. When latency exceeds a predetermined threshold, the system pauses capturing, encoding, or transmitting operations for a pause time. This periodic interruption allows the system to control transmission latency while maintaining image quality during active processing periods.
Solution Approach 2:
The patent dynamically adjusts the pause time duration based on current latency conditions and network status. The pause time is determined by comparing current latency with historical latency data, allowing the system to adaptively control processing intervals to optimize both latency and image quality in real-time.
2Loss of time
If image processing and transmission is paused to reduce latency, then transmission latency is reduced, but image processing efficiency decreases
Solution Approach 1:
The system alternates between active processing periods and paused periods based on latency conditions. During active periods, full image processing efficiency is maintained; during paused periods, processing is suspended to allow latency reduction. This periodic alternation balances latency control with processing efficiency.
Solution Approach 2:
The system continuously monitors latency metrics and uses this feedback to determine when to pause and resume processing. The pause time is adjusted based on feedback from latency measurements, ensuring that pausing occurs only when necessary to reduce latency, thereby minimizing impact on overall processing efficiency.
3Loss of time
If quantization parameters are adjusted to control transmission code rate, then transmission latency is reduced, but image compression ratio changes affecting quality
Solution Approach 1:
The patent adjusts quantization parameters dynamically based on pause time and latency conditions. When pause time exceeds a threshold, the system modifies quantization parameters to control the transmission code rate, thereby reducing latency. The parameter adjustments are carefully managed to maintain acceptable image quality while achieving latency reduction goals.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
In implementations of the subject matter described herein, there are provided a method and apparatus for real time screen sharing. During the screen sharing of two devices, if a certain condition is satisfied for an image encoded by a first device and an image decoded by a second device, the first device pauses image processing images. If the pause time of pausing satisfies a certain length, a parameter associated with an image compression ratio is adjusted. After the first device resumes image processing, the adjusted parameter is used to encode a new image captured on the first device. According to implementations of the subject matter described herein, a transmission code rate during the screen sharing can be controlled according to the pause time of pausing image processing. The implementations of the subject matter described herein can reduce transmission latency of screen sharing, thereby effectively ensuring the user experience.