PIM HBM DRAM Sub-Core Circuits for Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current semiconductor devices face challenges in achieving improved processing performance and low power consumption, particularly in processing-in-memory (PIM) high bandwidth memory (HBM) devices used for data-intensive applications like machine learning and artificial intelligence.
Innovation Solution
The proposed solution involves a memory device architecture that includes multiple banks, each comprising sub-cell blocks, word lines, bit lines, and a bank core circuit with a row decoder, sense amplifier, and sub-core circuits. These sub-core circuits integrate a processing element, sub-word line driver segments, and sense amplifier segments, enabling logical operations on data loaded into the sense amplifier segments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If computational operations are performed externally by processors, then processing capability is sufficient, but memory access latency increases and power consumption increases
Solution Approach 1:
The patent combines memory storage functions with computational processing functions within the same memory device. Processing elements are integrated directly into the memory device, allowing data to be processed in-place without being transferred to external processors, thereby reducing memory access latency and enabling compute-in-memory operations.
Solution Approach 2:
The patent employs a three-dimensional stacked architecture where memory cell arrays are stacked vertically over processing elements. This 3D integration allows simultaneous access to multiple memory layers while maintaining close proximity to processing units, reducing latency through vertical stacking rather than horizontal expansion.
2Use of energy by moving object
If data is transferred between memory and processor, then computation can be performed, but power consumption increases
Solution Approach 1:
The patent extracts the computational processing function from external processors and relocates it directly into the memory device. By integrating processing elements within the memory architecture, data can be processed where it is stored, eliminating the energy-consuming data transfer between memory and processor while maintaining high processing performance.
Solution Approach 2:
The memory device performs computational operations autonomously using integrated processing elements without requiring external processor intervention. The processing elements can execute operations directly on data stored in the memory cells, enabling the memory system to serve its own processing needs and reducing overall system power consumption.
3Productivity
If processing elements are integrated into memory device, then processing performance improves, but device complexity increases
Solution Approach 1:
The patent divides the memory device into multiple independently operable banks, each containing its own processing elements and memory cell arrays. This segmentation allows parallel processing across multiple banks, improving overall processing performance while distributing the complexity across modular units that can be managed independently.
Solution Approach 2:
The processing elements integrated into the memory device are designed to perform multiple computational functions including arithmetic operations, logical operations, and data movement. This multi-functionality allows a single integrated structure to handle diverse processing tasks, improving processing performance without proportionally increasing device complexity through specialized components.
Data Source
AI summary
There is provided a memory device including a processing-in-memory (PIM) dynamic random access memory (DRAM) die(s) and PIM-high bandwidth memory (HBM) devices. The memory device includes a core peripheral circuit structure including a bank core circuit including a row decoder and a sense amplifier of each of a plurality of banks, and a cell array structure disposed on the core peripheral circuit structure. The bank core circuit includes a plurality of sub core circuits correspondingly connected to the plurality of sub cell blocks, respectively. Each of the plurality of sub core circuits includes a sense amplifier segmented to be connected to a corresponding sub cell block, and a processing element connected to the segmented sense amplifier and configured to perform a logical operation on an operand that is data loaded into the segmented sense amplifier.