Multi-threaded Media Processing Architecture for Tile-based Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multi-threaded systems face challenges in efficiently processing multiple streams of bulk non-real-time data sources, particularly in managing data storage and processing thread allocation while minimizing latency and resource conflicts.
Innovation Solution
A three-phase scheduling system that differentiates data storage management from processing thread allocation, using a programmable data sequencer to load data and manage resource dependencies, ensuring active threads are selected based on available resources and data dependencies, and executing tasks across multiple processing threads to maximize resource utilization and minimize latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple data streams are processed simultaneously in a multi-threaded system, then productivity increases, but resource conflicts and latency increase
Solution Approach 1:
The patent segments the processing system into multiple independent threads that can execute simultaneously. Each thread handles specific data streams or processing tasks, allowing the system to process multiple data streams in parallel without overwhelming single-threaded resources. This segmentation enables high productivity while managing latency through distributed execution.
Solution Approach 2:
The patent implements preliminary actions by pre-allocating buffers, pre-compiling shaders, and pre-loading data before actual processing begins. Data is staged in intermediate buffers and processing resources are prepared in advance, reducing wait times and latency during actual execution while maintaining high throughput.
2Productivity
If more processing threads are allocated, then resource utilization improves, but complexity of thread management increases
Solution Approach 1:
The patent implements self-service mechanisms where threads automatically manage their own execution states, buffer allocations, and synchronization. The system includes automatic thread scheduling, dynamic buffer management, and self-synchronization protocols that reduce the need for complex external coordination, thereby improving resource utilization while managing complexity.
Solution Approach 2:
The patent employs dynamic thread management where the number and configuration of active threads adapt based on workload demands, resource availability, and system state. Threads can be dynamically created, suspended, or terminated, and resource allocation is adjusted in real-time, optimizing utilization without requiring static complex management structures.
3Productivity
If data is stored in unified storage area, then access efficiency improves, but data dependency conflicts increase
Solution Approach 1:
The patent introduces intermediary structures between unified storage and processing threads, including intermediate buffers, staging areas, and descriptor tables. These intermediaries decouple direct data dependencies, allowing threads to access data efficiently from unified storage while reducing blocking by enabling asynchronous data transfer and independent thread execution.
Data Source
Figure 1
Figure 2~3
Figure 4~6
AI summary
A method and apparatus are provided for processing multiple streams of data on a plurality of execution threads. Data is selected from a plurality of data sources (1001). An address in the data storage means (1036) is allocated for the data thus selected. The selected data is then loaded into the allocated address. Following this an execution task comprising the selected data source, the data address and an execution address is constructed and the data task is queued with previously constructed tasks. A determination is made as to which processing resources are required for each task and tasks are selected for execution in dependence on this. Tasks selected for execution are distributed across a plurality of processing threads (170). The allocation of data storage in the data storage means includes the steps of selecting data from one of the data sources and supplying a code execution address to a programmable data sequencer (1004). The code from the code execution address is executed to cause data to be written to the data storage means. Furthermore, a tile based computer graphic rendering system may comprise a plurality of mulit-threaded processor cores. Each processor core is allocated to different sectors of the tile.