Near-Memory Computing for Fine-Grained Data Access Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing applications, such as k-mer counting, face inefficiencies in accessing and processing small units of data scattered randomly throughout a dataset stored in a working memory space, particularly when data is remotely located, leading to time-consuming and inefficient CPU operations.
Innovation Solution
Implementing a near-data processing (NDP) system with memory modules containing near-memory computing (NMC) modules and hierarchical bus architectures to facilitate fine-grained data access and processing, including independent chip select signals and workload balancing, enabling parallel processing and efficient data transfer within memory modules.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored remotely in working memory space and accessed by CPU, then data storage capacity is improved, but data access time and processing efficiency deteriorate
Solution Approach 1:
The patent introduces near-memory computing modules as intermediary processing units positioned between the CPU and remote memory. These NMC modules perform data processing operations locally at the memory location, acting as mediators that eliminate the need for continuous data transfer to the CPU, thereby reducing access time while maintaining storage capacity.
Solution Approach 2:
The patent adds a spatial dimension to the memory hierarchy by placing computing resources directly at the memory location rather than relying on sequential CPU access. This dimensional change from centralized CPU processing to distributed in-memory processing reduces the temporal distance for data access while preserving storage capacity.
2Adaptability or versatility
If CPU performs all data processing operations, then processing flexibility is improved, but energy consumption and processing time deteriorate
Solution Approach 1:
The patent segments the processing function by dividing it between the CPU and near-memory computing modules. The CPU retains high-level control and complex decision-making capabilities, while NMC modules handle specific data processing operations locally. This segmentation reduces energy consumption by avoiding unnecessary data movement while preserving processing flexibility through coordinated operation between the two processing levels.
3Quantity of substance
If data is scattered randomly throughout memory space, then data storage efficiency is improved, but data access efficiency deteriorates
Solution Approach 1:
The patent applies local quality by enabling each near-memory computing module to independently process data locally at its specific memory location. Rather than requiring centralized CPU access to scattered data, each NMC module performs processing operations on its local data portion, maintaining storage efficiency while dramatically improving access efficiency through parallel local processing.
4Adaptability or versatility
If hierarchical bus structure is implemented for data transfer, then system scalability is improved, but system complexity deteriorates
Solution Approach 1:
The patent implements a dynamic hierarchical bus structure that adapts its complexity based on system needs. The bus architecture provides scalable connectivity between multiple NMC modules and the CPU, with the complexity managed through standardized interfaces and protocols. This dynamic structure allows the system to scale from simple to complex configurations without requiring complete redesign, balancing scalability with manageable complexity.
Data Source
AI summary
A memory module may include one or more memory devices, and a near-memory computing module coupled to the one or more memory devices, the near-memory computing module including one or more processing elements configured to process data from the one or more memory devices, and a memory controller configured to coordinate access of the one or more memory devices from a host and the one or more processing elements. A method of processing a dataset may include distributing a first portion of the dataset to a first memory module, distributing a second portion of the dataset to a second memory module, constructing a first local data structure at the first memory module based on the first portion of the dataset, constructing a second local data structure at the second memory module based on the second portion of the dataset, and merging the first and second local data structures.