PIM Memory Chip Modes for Vector Accumulation Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Random access memory devices face inefficiencies in performing calculations due to increased data communication when handling vector inputs, as they cannot perform accumulation operations between elements, leading to bottlenecks in processing-in-memory (PIM) operations.
Innovation Solution
A memory device with multiple memory chips and logic circuits that can perform PIM operations on both vector and partial scalar data, utilizing different modes to optimize data processing and reduce communication overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If the random access memory receives only vector input from host, then the communication between central processing unit and random access memory decreases, but the random access memory is incapable of performing accumulation operation between elements, leading to increased data communication when host performs accumulation operation
Solution Approach 1:
The patent merges the accumulation operation capability into the memory device itself by introducing a logic circuit that can perform accumulation operations on vector data stored in memory banks. This combines data storage and data processing functions in one device, eliminating the need for separate host processing and reducing data communication overhead.
Solution Approach 2:
The memory device is designed with multi-functionality to handle both vector inputs and scalar inputs, and to perform both storage and accumulation operations. The logic circuit can operate in different modes depending on the input type, making the device versatile and adaptable to different processing requirements without increasing communication overhead.
2Productivity
If the logic circuit performs PIM operation on vector data, then processing efficiency improves, but device complexity increases due to need for multiple operation modes
Solution Approach 1:
The logic circuit is designed with dynamic operation modes that can switch between vector mode and scalar mode based on the input data type. This dynamic adaptability allows the same hardware to efficiently handle different processing tasks without requiring separate dedicated circuits for each mode, thus improving processing efficiency while controlling complexity through unified design.
Solution Approach 2:
The system manages complexity by changing operational parameters rather than structural complexity. The logic circuit uses the same basic architecture but adjusts its operation parameters (such as data processing mode) based on the input type. This parameter-based adaptation achieves high processing efficiency across different workloads while maintaining relatively simple hardware structure.
3Adaptability or versatility
If the memory device supports both vector and scalar input modes, then adaptability improves, but data communication overhead increases when handling scalar data with non-zero values
Solution Approach 1:
The patent extracts and processes only the necessary data for scalar operations by using a zero-check mechanism that identifies non-zero scalar values and processes only those. This extraction approach prevents unnecessary data communication by filtering out redundant zero values before they need to be processed, thus maintaining adaptability to scalar inputs while reducing communication overhead.
Solution Approach 2:
The system applies partial action by processing only the subset of scalar data that is actually needed (non-zero values) rather than processing all input data. The zero-check mechanism enables the system to selectively process only the necessary portions of scalar input, reducing communication overhead while maintaining full adaptability to scalar data types.
Data Source
AI summary
Disclosed is a memory device which includes a plurality of memory chips. Each of the plurality of memory chips includes a plurality of memory banks and a logic circuit. In a first operation mode, the logic circuit writes first data in the plurality of memory banks based on a first command and a first address received from the host, and performs a first processing-in-memory (PIM) operation based on third data received from the host and the first data. In a second operation mode, the logic circuit writes second data in the plurality of memory banks based on the first command and the first address received from the host, and performs a second PIM operation based on fourth data different from the third data received from the host and the second data.