Asynchronous Cache Maintenance via Fixup Queue
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cache maintenance operations, such as cache flush and invalidate, introduce performance degradation in small workload offloads to hardware accelerators due to their negative impact on processing performance, particularly affecting the efficiency of CPU cache data transfer.
Innovation Solution
Implementing asynchronous cache maintenance operations that allow for delayed execution of cache maintenance tasks during idle periods or when the cache is not actively in use, using a fixup queue to manage these operations and execute them in parallel with workload processing, thereby reducing latency and overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If synchronous cache maintenance operations are performed before and after accelerator workloads, then cache coherence is maintained, but processing performance deteriorates due to latency and overhead
Solution Approach 1:
The system performs cache maintenance operations asynchronously in the background before they are strictly needed, allowing the accelerator to execute workload instructions without waiting for cache maintenance completion. The cache maintenance operation is initiated and tracked, but does not block the main execution flow, thus maintaining cache coherence while avoiding performance degradation from synchronous waiting.
Solution Approach 2:
The patent introduces an intermediary mechanism (cache maintenance operation tracker and fixup queue) that decouples the cache maintenance operation from the workload execution flow. This intermediary allows the system to track and manage cache maintenance operations without forcing them to complete synchronously, enabling the accelerator to continue processing while cache maintenance proceeds in the background.
2Stability of the object's composition
If cache maintenance operations are performed synchronously, then cache data consistency is ensured, but small workload offloads suffer performance degradation
Solution Approach 1:
Cache maintenance operations are initiated as preliminary background tasks that do not block workload execution. The system starts cache maintenance operations and continues with workload processing, ensuring data consistency is worked on without delaying the critical path of small workload offloads.
Solution Approach 2:
The system dynamically adjusts the execution model of cache maintenance operations based on workload characteristics. For small workload offloads, cache maintenance is executed asynchronously in the background rather than synchronously, allowing the system to maintain data consistency while minimizing the time loss associated with blocking operations.
3Productivity
If asynchronous cache maintenance operations are implemented, then processing performance improves, but system complexity increases due to fixup queue management
Solution Approach 1:
The cache maintenance system is segmented into distinct functional components: operation tracking structures, fixup queues for deferred operations, and execution mechanisms. This segmentation allows each component to handle specific aspects of asynchronous cache maintenance independently, managing complexity through modular organization while enabling performance improvements from asynchronous execution.
Solution Approach 2:
The system changes the execution parameter of cache maintenance operations from synchronous to asynchronous mode. This parameter change enables performance improvement by allowing overlapping of cache maintenance and workload execution, while the complexity increase is managed through systematic changes to operation tracking and queue management parameters.
4Reliability
If cache flush and invalidate operations are performed before accelerator execution, then data coherence is maintained, but latency increases for small workload offloads
Solution Approach 1:
Cache flush and invalidate operations are performed as preliminary background tasks that are initiated before accelerator execution but do not block it. The operations are tracked and executed asynchronously, allowing data coherence to be maintained while the accelerator executes small workload offloads at full speed without waiting for cache maintenance completion.
Data Source
AI summary
Aspects include computing devices, apparatus, and methods implemented by the apparatus for implementing asynchronous cache maintenance operations on a computing device, including activating a first asynchronous cache maintenance operation, determining whether an active address of a memory access request to a cache is in a first range of addresses of the first active asynchronous cache maintenance operation, and queuing the first active asynchronous cache maintenance operation as the first asynchronous cache maintenance operation in a fixup queue in response to determining that the active address is in the first range of addresses.