Active Memory Processing Element Bypassing Cache for Low Latency Data Exchange
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In computer systems with distributed task execution, data transfer between a remote processing unit and a main processor increases latency and power consumption due to data copying through multiple cache levels.
Innovation Solution
A method and system where a processing element in an active memory device receives instructions and requests from the main processor, storing or loading data directly into a queue, bypassing system caches, and communicating results directly to the main processor through an interconnect network, reducing latency and power consumption by minimizing cache interactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is copied to each cache level when transferred to or from remote location, then data consistency across cache levels is maintained, but latency and power consumption increase
Solution Approach 1:
The patent extracts the data copying operation from the traditional cache hierarchy path. Instead of copying data through multiple cache levels (L1, L2, L3), the system directly transfers data between the processing element and main memory, eliminating unnecessary intermediate copying steps while maintaining data consistency through direct address translation and data movement.
Solution Approach 2:
The patent introduces an intermediary mechanism in the form of address translation and direct data path routing. The system uses address translation to map logical addresses to physical memory locations, enabling direct data access between the processing element and main memory without requiring traditional cache copying, thus reducing latency while maintaining consistency.
2Ease of operation
If data is copied to each cache level when transferred to or from remote location, then data is available at multiple cache levels, but power consumption increases
Solution Approach 1:
The patent removes the energy-consuming data copying operations from the cache hierarchy. By directly accessing main memory from the processing element through optimized data paths, the system eliminates the power consumption associated with repeated data copying to multiple cache levels, while data remains available through direct memory access.
Solution Approach 2:
The patent applies partial action by selectively bypassing the cache copying mechanism only for remote memory accesses. The system maintains the ability to use caches when beneficial but eliminates unnecessary copying for remote locations, reducing power consumption without completely sacrificing the ease of data availability that caches provide for local accesses.
3Adaptability or versatility
If traditional cache hierarchy is used for data transfer, then data access flexibility is maintained, but communication overhead increases
Solution Approach 1:
The patent segments the data access path into distinct functional components: address translation, data movement, and processing. By separating these functions and optimizing each independently, the system reduces communication overhead while maintaining flexibility. The address translation unit handles mapping, while direct data paths handle transfer, eliminating the need for complex cache coherence protocols across multiple levels.
Solution Approach 2:
The patent inverts the traditional approach by making main memory the primary data source rather than having all accesses go through the cache hierarchy. Instead of caches serving as intermediaries for all accesses, the system directly accesses main memory when needed, inverting the traditional data flow and reducing the complexity of cache management and communication overhead.
Data Source
AI summary
According to one embodiment, a method for exchanging data in a system that includes a main processor in communication with an active memory device is provided. The method includes a processing element in the active memory device receiving an instruction from the main processor and receiving a store request from a thread running on the main processor, the store request specifying a memory address associated with the processing element. The method also includes storing a value provided in the store request in a queue in the processing element and the processing element performing the instruction using the value from the queue.