Video and UI Frame Encoding for Low-Latency Display Merging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing electronic devices experience delays and inconveniences due to processing time requirements in real-time video processing operations, leading to screen stutters and low response times when merging video frames and user interface frames.
Innovation Solution
An electronic apparatus and method that identifies non-overlap and overlap regions between video and UI frames, encodes these regions separately, and applies appropriate color space conversions to generate final encoding data, optimizing processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If video processing operation is performed to merge video frame and UI frame, then screen output is generated, but processing time is required causing screen delay and stutter
Solution Approach 1:
The video frame is divided into multiple blocks, and the UI frame is divided into multiple blocks. By segmenting the frames into smaller units, the processing can be performed more efficiently on each block separately, reducing the overall processing time while maintaining screen output quality.
Solution Approach 2:
Different processing operations are applied to different regions of the frame based on local characteristics. The video processing operation is selectively applied only to blocks that require it, while other blocks may use different processing methods or no processing at all, thereby reducing unnecessary processing time while maintaining quality where needed.
2Manufacturing precision
If color space conversion is applied to UI frame, then color accuracy is improved, but processing complexity increases
Solution Approach 1:
Color space conversion is applied selectively only to specific blocks of the UI frame where it is necessary, rather than converting the entire frame. This localized approach maintains color accuracy in regions that require it while reducing the overall processing complexity and computational burden.
Solution Approach 2:
Instead of applying color space conversion to the entire UI frame (excessive action), the method applies conversion only to the necessary blocks (partial action). This partial application of the conversion operation achieves the required color accuracy while significantly reducing processing complexity.
Data Source
AI summary
An electronic apparatus may include memory storing instructions, and at least one processor, wherein the instructions that, when executed by the at least one processor, cause the electronic apparatus to: obtain a video frame of a first color space and a user interface (UI) frame of a second color space; identify a video region corresponding to the video frame and a UI region corresponding to the UI frame from a display region in which the video frame and the UI frame are displayed; identify a non-overlap video region from the video region that does not overlap the UI region; identify an overlap video region from the video region that overlaps the UI region; obtain first encoding data by encoding the non-overlap video region; obtain combining data by combining the overlap video region and the UI frame; and obtain second encoding data by encoding the combining data.