Configurable Thread Ordering for GPU Cache Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing apparatuses, such as GPUs, face inefficiencies in processing large volumes of data due to fixed thread execution orders, which do not adapt well to the specific requirements of the data processing operations, leading to suboptimal cache performance and increased processing burden on each kernel.
Innovation Solution
A data processing apparatus with configurable thread allocation circuitry that specifies coordinate values for each thread to access input data within a reference domain, allowing for flexible ordering of data processing points to optimize cache usage and reduce processing burden, utilizing execution circuitry to execute multiple threads across multiple execution units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the order in which data values are processed is adapted to benefit the configuration of the data processing apparatus, then cache performance is improved, but the processing burden on each kernel increases
Solution Approach 1:
The patent segments the data processing task by dividing the reference domain into multiple blocks, each assigned to a specific thread. This segmentation allows complex processing to be distributed across simpler, dedicated thread tasks, improving cache performance without increasing individual kernel complexity
Solution Approach 2:
The patent implements dynamic thread allocation where the order in which points in the reference domain are allocated to threads is configurable. This dynamic allocation adapts the processing order to benefit the specific configuration and requirements of the data processing apparatus, optimizing cache performance
2Reliability
If block-wise approach is used to process pixels, then cache performance is improved, but the complexity of address calculation increases
Solution Approach 1:
The patent enables threads to automatically determine their own reference points in the reference domain based on their thread identifier. Each thread self-calculates its assigned block without requiring complex external address calculation, simplifying the system while maintaining block-wise processing benefits
Solution Approach 2:
The patent changes the parameter of thread allocation order to be configurable rather than fixed. By allowing the allocation order to adapt to the specific data processing requirements, the system achieves block-wise processing benefits without permanently increasing address calculation complexity
3Device complexity
If simple linear increment address calculation is used, then processing is simplified, but cache locality is reduced
Solution Approach 1:
The patent makes the address calculation dynamic by allowing configurable allocation orders. Instead of a fixed simple linear increment, the system can adaptively assign points to threads in orders that optimize cache locality for specific data processing operations, while keeping the underlying mechanism relatively simple
Data Source
AI summary
A data processing apparatus and method processing data are disclosed. Execution circuitry is configured to execute multiple threads to perform data processing on input data by reference to at least one coordinate value of points in a reference domain. Thread allocation circuitry is configured to specify a selected point in the reference domain for each thread of the multiple threads respectively in order to allocate the data processing by specifying for each thread the at least one coordinate value of the specified point for that thread. Each thread accesses the input data with reference to its selected point in the reference domain and an order in which points in the reference domain are allocated to threads for data processing is configurable in the thread allocation circuitry.