Near-Memory Processing Unit for Minimizing Off-Chip Data Access Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current memory management techniques, such as swapping, suffer from performance degradation due to high latency associated with moving data through the system bus, especially when data is moved between main memory and auxiliary memory, leading to inefficient memory usage and application program performance issues in systems with limited memory resources.
Innovation Solution
A near-memory processing unit is introduced, which compresses data upon receiving a swap-out command and stores it in a compressed memory space within the main memory, and decompresses data upon receiving a swap-in command, minimizing off-chip communication and latency by managing memory operations independently of the host, using a compressor, decompressor, and memory controller to efficiently allocate and manage memory areas.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is moved between main memory and auxiliary memory using traditional swapping, then memory capacity is improved, but data access latency increases due to system bus communication
Solution Approach 1:
The patent divides memory into normal memory space and compressed memory space, allowing different types of data to be stored in different segments. Frequently accessed data remains in normal memory space while less frequently accessed data is moved to compressed memory space, reducing the need for slow auxiliary memory access while maintaining overall memory capacity.
Solution Approach 2:
The patent introduces compression as an additional dimension for memory management. Instead of only expanding memory capacity through auxiliary memory, the system uses data compression to effectively double the usable memory space within the same physical memory boundaries, thereby avoiding slow off-chip access while increasing available memory.
2Loss of time
If a near-memory processing unit is introduced to compress and decompress data, then data access latency is reduced by minimizing off-chip communication, but device complexity increases
Solution Approach 1:
The near-memory processing unit acts as an intermediary between the host and main memory. It handles compression, decompression, and memory management tasks locally, preventing the host from being directly involved in these time-consuming operations. This intermediary approach reduces host memory access latency while distributing the computational complexity to a dedicated component.
Solution Approach 2:
The near-memory processing unit provides self-service capabilities by autonomously managing compression, decompression, and swapping operations without requiring constant host intervention. The unit can independently determine when to compress or decompress data based on access patterns, reducing the need for complex host-based control logic.
3Quantity of substance
If compressed memory space is used to store more data, then memory capacity utilization is improved, but memory area allocation complexity increases
Solution Approach 1:
The system pre-allocates a dedicated compressed memory space area in advance, rather than dynamically calculating and allocating space during each compression operation. This preliminary allocation simplifies the memory management process by establishing fixed boundaries between normal and compressed memory spaces before data operations begin.
Solution Approach 2:
The near-memory processing unit maintains an entry tree that copies and tracks metadata about compressed pages. This entry tree serves as a simplified representation of the actual compressed data locations, allowing the system to manage memory allocation through metadata operations rather than direct physical memory manipulation, thereby reducing allocation complexity.
Data Source
AI summary
A near-memory processing unit is configured to compress a page present in a normal memory space of a memory when receiving a swap-out command from a host, allocate a memory area in which the compressed page is to be stored in a compressed memory space which is a memory area previously allocated by the host, copy the compressed page into the allocated memory area, generate an entry corresponding to the compressed page, and insert the generated entry into an entry tree.