Data Processor Cache Thrashing Control via Thread Throttling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data processors that use caches to store data face challenges with cache thrashing, where multiple main memory locations compete for the same cache lines, leading to excessive cache misses and impacting performance and energy usage.
Innovation Solution
The technology involves monitoring the cache operation during program execution and controlling the issuing of instructions for execution threads based on this monitoring. This includes restricting the number of threads when a risk of cache thrashing is identified to prevent cache thrashing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple threads are issued for execution to improve throughput, then productivity increases, but cache thrashing occurs leading to excessive cache misses and performance degradation
Solution Approach 1:
The patent implements a feedback mechanism where cache performance metrics (such as cache miss rates or thrashing detection) are monitored and fed back to the thread issuance controller. When cache thrashing is detected, the system reduces the number of issued threads, and when cache performance is good, it increases thread issuance. This dynamic feedback loop resolves the contradiction by adapting throughput to actual cache conditions.
Solution Approach 2:
The system dynamically adjusts the number of threads issued for execution based on real-time cache performance conditions. Rather than using a fixed thread issuance rate, the patent employs dynamic control that responds to changing cache states, allowing the system to optimize between throughput and performance reliability adaptively.
2Productivity
If more threads are executed to utilize cache capacity, then productivity improves, but energy consumption increases due to excessive cache misses and memory access overhead
Solution Approach 1:
The feedback mechanism monitors not only cache performance but also energy-related metrics such as memory access frequency and cache miss rates. When high energy consumption is detected due to excessive cache misses, the system reduces thread issuance to lower energy usage. This resolves the contradiction by linking energy efficiency to adaptive thread management.
Solution Approach 2:
The system changes operational parameters (number of issued threads) based on cache performance conditions. By adjusting the thread count parameter in response to cache thrashing or high energy consumption, the system optimizes the balance between cache utilization and energy efficiency.
3Reliability
If the number of threads is restricted to prevent cache thrashing, then performance improves, but productivity decreases due to reduced throughput
Solution Approach 1:
The patent employs dynamic thread issuance control where the number of threads is adjusted in real-time based on cache conditions. When cache thrashing is detected, thread issuance is restricted to maintain performance; when cache performance is good, thread issuance is increased to maximize throughput. This dynamic approach resolves the contradiction by making throughput flexible rather than fixed.
Solution Approach 2:
The feedback loop continuously monitors cache performance and adjusts thread issuance accordingly. When performance metrics indicate cache thrashing, the system restricts thread issuance to protect performance; when metrics show healthy cache operation, the system increases throughput. This closed-loop control resolves the contradiction between performance reliability and productivity.
4Reliability
If cache size is increased to accommodate more data, then cache hit rate improves, but device complexity and cost increase
Solution Approach 1:
Instead of increasing cache size to accommodate all possible data (excessive action), the patent uses partial action by dynamically controlling thread issuance to match the actual data access patterns and cache capacity. This approach achieves high cache hit rates without requiring larger cache structures, thus avoiding increased device complexity.
Solution Approach 2:
The system changes the operational parameter of thread count rather than changing the physical parameter of cache size. By adjusting the number of issued threads based on cache performance, the system achieves high cache hit rates without modifying the cache structure, thereby avoiding increased device complexity and cost.
Data Source
AI summary
A data processor comprising an execution engine 51 for executing programs for execution threads and one or more caches 48, 49 operable to store data values for use when executing program instructions to perform processing operations for execution threads. The data processor further comprises a thread throttling control unit 54 configured to monitor the operation of the caches 48, 49 during execution of programs for execution threads, and to control the issuing of instructions for execution threads to the execution engine for executing a program based on the monitoring of the operation of the caches during execution of the program.