Video Mosaic Display Buffer Scaling Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video mosaic display techniques require significant scalar processing power and memory bandwidth due to unnecessary scaling and copying of video streams, especially when handling multiple video streams with different resolutions and frame rates.
Innovation Solution
The system employs a dual-display buffer approach where only available input frames are scaled, and video frame windows for unavailable streams are copied or retained from previous buffer states, reducing computational complexity and memory requirements by avoiding intermediate buffers and minimizing unnecessary scaling and copying.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If scaling of each video stream is performed a number of times equal to the frame rate of display of the video mosaic layout irrespective of frame rates of individual video streams, then the video mosaic layout can be generated at the required frame rate, but the scalar processing power and memory bandwidth requirements increase significantly
Solution Approach 1:
The patent applies partial action by scaling video streams only when necessary - specifically, when a new frame is available from the source. Instead of scaling every frame at the display frame rate, the system scales only the subset of frames that contain new content, reducing scalar processing power consumption while maintaining the required display frame rate through intelligent frame selection and copying
2Speed
If scaling of each video stream is performed a number of times equal to the frame rate of display of the video mosaic layout irrespective of frame rates of individual video streams, then the video mosaic layout can be generated at the required frame rate, but the memory bandwidth requirements increase significantly
Solution Approach 1:
The patent extracts and eliminates unnecessary scaling operations from the processing pipeline. By identifying and removing redundant scaling steps (where no new frame data is available), the system reduces memory bandwidth consumption while maintaining display performance through direct frame copying from the input buffer to the display buffer
3Power
If offline scaling of video streams is performed and scaled frame copies are made in an intermediate video buffer, then the scalar processing power is decreased, but additional copying increases the memory requirement and memory bandwidth
Solution Approach 1:
The patent merges the scaling operation with the frame buffer management by performing scaling directly in the display buffer when new frames arrive, rather than pre-scaling to an intermediate buffer. This integration eliminates the need for separate intermediate buffering, reducing memory requirements while distributing the scaling workload efficiently across the processing timeline
4Power
If offline scaling of video streams is performed and scaled frame copies are made in an intermediate video buffer, then the scalar processing power is decreased, but additional copying increases the memory bandwidth
Solution Approach 1:
The patent applies partial action by performing copying operations only when necessary - specifically, when new frame data arrives from the input stream. Instead of continuously copying frames from an intermediate buffer to the display buffer, the system copies only when new content is available, reducing memory bandwidth consumption while maintaining scalar processing efficiency
Data Source
AI summary
Systems and methods for generating a video mosaic display are disclosed. In an embodiment, an electronic system for generating a video mosaic display of a plurality of video streams includes display buffers, a processor and a display. The processor manages storage of the display buffers for storing video frame windows corresponding to the video streams by scaling resolution of input frames of a first set of video streams that have input frames available and storing the scaled input frames as corresponding video frame windows for the first set of video streams in a first display buffer, and copying, in the first display buffer from a second display buffer, video frame windows corresponding to a second set of video streams for which input frames are not available. The display is configured to display the video mosaic layout based on the video frame windows stored in the display buffers.