Media Synchronization via Image Queue Slot ID
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As the number of applications running on an electronic device increases, coordinating the display requests and managing media processing becomes cumbersome, leading to difficulties in ensuring that only one application decodes media and provides graphics to the render server at a given time, and that user playback operations do not overburden the device's hardware.
Innovation Solution
A system that synchronizes audio and video processing using an image queue, where the media server processes media files and provides decoded frames to the render server, with each application referencing a slot ID instead of the image queue directly, allowing for efficient manipulation and coordination of media display across multiple applications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple applications run simultaneously and each coordinates with the render server directly, then each application can control its own media processing, but the coordination complexity increases significantly making it cumbersome to manage
Solution Approach 1:
The patent introduces a media server as an intermediary component between multiple applications and the render server. The media server centralizes media processing functions including decoding, rendering, and frame buffering, while applications only need to communicate with the media server through standardized interfaces. This mediator architecture reduces coordination complexity by consolidating management functions in a single component while preserving application independence.
2Ease of operation
If applications directly manage their own media decoding and frame buffers, then they have full control over playback operations, but hardware resources become overburdened with multiple decoding operations
Solution Approach 1:
The patent merges multiple media processing functions (decoding, rendering, frame buffering, and synchronization) into a single media server component. Instead of each application having its own decoding pipeline and frame buffers, the media server consolidates these resources. Multiple applications share the same decoding hardware and frame buffer memory through the media server's centralized management, reducing overall hardware resource consumption while maintaining playback control capabilities.
3Device complexity
If the system uses host time stamps for video frame synchronization, then time coordination is straightforward, but pausing and resuming playback requires re-decoding video which wastes processing resources
Solution Approach 1:
The patent changes the time reference parameter from host time (system clock) to media time (playback-specific time). The media server maintains a mapping between media time and host time, allowing the system to use media time for synchronization and playback control. When pausing or resuming playback, the media server can simply adjust the media time counter without re-decoding video frames, as the decoded frames are already buffered with their media time stamps. This parameter change eliminates unnecessary re-decoding operations while maintaining accurate time synchronization.
Data Source
AI summary
This invention is directed to a system for processing media in an electronic device. An application operative to request the display of one or more graphics elements may be implemented on the electronic device. The application may identify graphics elements for display and organize the graphics elements in a layer tree for processing and display by a render server (e.g., a core animation render server). To coordinate the processing of graphics elements of the application or of several applications, the electronic device may include a media server operative to receive media from the application and generate an image queue that includes video frames of the media. The render server may associate a slot ID with each image queue, while the media server may provide the slot ID to the application for inclusion in the layer tree. When the render server reaches a slot ID in the layer tree, the render server may identify and display the associated image queue.