Near-Memory Reduction During Store Operations to Cut Data Movement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data reduction methods incur high costs and cache pollution due to the need to read large datasets from off-chip or off-die memory, especially in computing processes like machine learning, and are hindered by the inefficiencies of integrating data reduction operations with reusable computations and parallel methodologies.
Innovation Solution
Implementing a Near-Memory Reduction (NMR) unit that performs data reduction during store operations by adding data values to an accumulated result, maintaining the reduction in a register accessible by the microprocessor, and using tree-based pair-wise data reduction to minimize data movement and cache pollution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored in main memory for later reduction, then data capacity is improved, but data movement cost increases
Solution Approach 1:
The patent performs data reduction operations during the store operation itself, before the data would otherwise need to be read back from main memory. The NMR unit accumulates reduction results in near-memory location during the write operation, eliminating the need for subsequent read-back and separate reduction operations.
Solution Approach 2:
The patent combines the data reduction function with the memory store operation. The NMR unit is integrated into the memory system and performs reduction operations as data is being stored, merging two separate operations (store and reduce) into a single coordinated process that occurs during the store operation.
2Speed
If data is read from main memory to on-chip memory for reduction, then computation speed is improved, but cache pollution increases
Solution Approach 1:
The patent extracts the data reduction operation from the traditional compute path that would otherwise require reading data into on-chip caches. By performing reduction in a near-memory location, the harmful effect of cache pollution is removed while maintaining computational efficiency.
3Loss of energy
If data reduction is performed during store operations, then data movement cost is reduced, but generic computation reusability deteriorates
Solution Approach 1:
The patent introduces an intermediary NMR unit that handles data reduction operations. This intermediary component allows the store operation to maintain its generic, reusable nature while the NMR unit independently performs reduction on the data being stored, without requiring modifications to the original computation kernels.
4Productivity
If multiple writers use the same output location, then parallel processing capability is improved, but data reduction integration becomes infeasible
Solution Approach 1:
The patent implements a self-service mechanism where the NMR unit automatically accumulates reduction results from multiple writers using atomic operations. Each writer independently contributes to the reduction without coordination overhead, and the NMR unit self-manages the accumulation process, making the system both parallel and reducible.
Data Source
Figure 1
Figure 2
Figure 3A~3C
AI summary
An approach is provided for implementing near-memory data reduction during store operations to off-chip or off-die memory. A Near-Memory Reduction (NMR) unit (100) provides near-memory data reduction during write operations to a specified address range (110). The NMR unit is configured with a range of addresses to be reduced and when a store operation specifies an address within the range of addresses, the NRM unit performs data reduction by adding the data value specified by the store operation to an accumulated reduction result (120). According to an embodiment, the NRM unit maintains a count (130) of the number of updates to the accumulated reduction result that are used to determine when data reduction has been completed.