Near-Memory Reduction Unit for Store-Time Data Aggregation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computing processes, such as machine learning, face high data movement costs and cache pollution when reducing large datasets that exceed on-chip or on-die memory limits, as data reduction operations are often inefficiently performed after data retrieval from main memory.
Innovation Solution
Implementing a Near-Memory Reduction (NMR) unit that performs data reduction during store operations by accumulating data values within a specified address range, reducing the need for costly data retrieval and maintaining the reduction result in accessible registers, thereby allowing concurrent data reduction with computation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If data reduction operations are performed after data retrieval from main memory, then data reduction can be completed, but data movement costs are prohibitively high and cache pollution occurs
Solution Approach 1:
The patent performs data reduction operations preliminarily during the store operation phase, before data retrieval from main memory is needed. The NMR unit accumulates data values during writing to memory, so when data needs to be retrieved, the reduction is already complete or in progress, eliminating the need to move all raw data back to on-chip memory for reduction.
Solution Approach 2:
The patent introduces a Near-Memory Reduction (NMR) unit as an intermediary component between main memory and on-chip memory. This NMR unit performs reduction operations in the near-memory space, acting as a mediator that prevents the need to move large datasets back to on-chip memory while still enabling efficient reduction.
2Productivity
If data reduction operations are included with data generation operations, then data reduction is performed concurrently, but it adversely affects performance and hinders reusability of generic computations
Solution Approach 1:
The patent segments the data generation and data reduction operations into separate functional units. The NMR unit is a distinct component that operates independently from the generic computation units, allowing data generation operations to remain reusable and optimized while data reduction occurs concurrently in the segmented NMR unit.
Solution Approach 2:
The NMR unit acts as an intermediary that handles data reduction separately from generic computation operations. This mediation allows generic computations to maintain their reusability and optimization while still achieving concurrent data reduction through the intermediary NMR unit.
3Quantity of substance
If datasets larger than on-chip or on-die memory are stored in main memory, then large datasets can be handled, but retrieval costs are expensive and data movement is required
Solution Approach 1:
The patent extracts the data reduction function from the main computation flow and places it in a dedicated NMR unit near memory. This extraction allows large datasets to remain in main memory while the reduction operation is performed in the near-memory space, eliminating the need to retrieve all data to on-chip memory.
Solution Approach 2:
The patent changes the spatial dimension of where data reduction occurs, moving it from on-chip memory space to near-memory space. This dimensional shift allows large datasets to be processed in the near-memory dimension without requiring expensive retrieval to on-chip memory.
Data Source
AI summary
An approach is provided for implementing near-memory data reduction during store operations to off-chip or off-die memory. A Near-Memory Reduction (NMR) unit provides near-memory data reduction during write operations to a specified address range. The NMR unit is configured with a range of addresses to be reduced and when a store operation specifies an address within the range of addresses, the NRM unit performs data reduction by adding the data value specified by the store operation to an accumulated reduction result. According to an embodiment, the NRM unit maintains a count of the number of updates to the accumulated reduction result that are used to determine when data reduction has been completed.