Cache-Embedded Configurable Engine for Irregular Data Flows
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern CPUs face inefficiencies due to mis-predictions and data movement inefficiencies in the cache hierarchy, particularly for applications with irregular workloads, which current solutions like on-core integration complicate core performance and limit operation flexibility.
Innovation Solution
A configurable engine is embedded in the cache hierarchy, allowing software to customize data flow management between main memory, caches, and cores, enabling efficient data sharing and reconfiguration for various applications without requiring ISA extensions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If an execution engine is integrated inside the core, then fine-grain integration and operation execution capability are improved, but core performance optimization becomes more complex and core design is disrupted
Solution Approach 1:
The execution engine is segmented from the core and placed in the shared cache, creating a separate functional module. This allows the core to maintain its simple design while the engine provides specialized operation execution capabilities through the cache hierarchy.
Solution Approach 2:
The shared cache acts as an intermediary between the cores and main memory, housing the execution engine. This intermediary structure enables data flow management and operation execution without directly modifying core architecture, thus avoiding design disruption.
2Adaptability or versatility
If an execution engine is integrated inside the core, then data flow management capability is improved, but the area available for core functionality is reduced and performance optimization becomes more complex
Solution Approach 1:
The data flow management functionality is segmented from the core and relocated to the shared cache. This segmentation allows the core to retain its full area for execution functionality while the shared cache handles data flow management tasks.
Solution Approach 2:
The execution engine is moved to a different spatial dimension in the architecture - from inside the core to the shared cache hierarchy. This dimensional relocation provides the needed functionality without consuming core area.
3Adaptability or versatility
If ISA extensions are provided for SIMD operations, then operation capability is improved, but core design is complicated and backward compatibility is degraded
Solution Approach 1:
The SIMD operation capability is extracted from the ISA and implemented as a separate execution engine in the shared cache. This extraction allows the engine to handle vector operations without modifying the core ISA, thus maintaining backward compatibility while providing enhanced capability.
Solution Approach 2:
The execution engine provides universal operation support including SIMD operations through a single integrated architecture. This multi-functional engine can handle various operation types without requiring separate ISA extensions for each operation class.
4Productivity
If a specialized execution engine is placed in the main data path, then operation execution efficiency is improved, but the dispatcher becomes more complicated and instruction scheduling complexity increases
Solution Approach 1:
The execution engine is extracted from the main data path and placed in the shared cache. This extraction reduces the dispatcher's workload by removing the complexity of coordinating engine operations within the core, while the engine maintains efficient operation execution through the cache hierarchy.
Solution Approach 2:
The shared cache serves as an intermediary layer between the dispatcher and the execution engine. This intermediary structure simplifies the dispatcher by providing a standardized interface for operation issuance, while the engine executes operations efficiently through the cache's data flow management capabilities.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
Embodiments herein describe a configurable engine that is embedded into the cache hierarchy of a processor. The configurable engine can enable efficient data sharing between the main memory, cache memories, and the core. The configurable engine can perform operations that are more efficient to be done in the cache hierarchy. In one embodiment the configurable engine is controlled (or configured) by software (e.g., the operating system (OS)), adapting to each application domain. That is, the OS can configure the engine according to a data flow profile of a particular application being executed by the processor.