Memory Block Bit Line Layout for In-Memory AI Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The traditional von Neumann architecture faces challenges with 'memory wall' and 'power consumption wall', and the bandwidth between CPU and memory becomes a bottleneck, limiting the performance of AI chips, especially with the rise of big data and artificial intelligence applications.
Innovation Solution
The semiconductor device incorporates a memory array with memory blocks arranged in specific configurations, where bit lines are coupled to all memory strings in a column, increasing the number of memory strings per bit line and reducing coupling capacitance, allowing for higher computational power and accuracy through improved data access and in-memory operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If bit lines are coupled to all memory strings in a column, then computational power and accuracy are improved, but bit line resistance increases
Solution Approach 1:
The patent divides the memory array into multiple memory blocks, each with its own bit line group. This segmentation allows parallel data access paths, reducing the effective resistance seen by each bit line while maintaining high computational power through the combined capacity of multiple blocks.
Solution Approach 2:
The patent introduces a third dimension by stacking memory blocks vertically with bit line groups positioned at different heights. This three-dimensional arrangement allows multiple bit lines to operate in parallel without interfering with each other, effectively reducing resistance while maintaining computational power.
2Measurement precision
If the number of memory strings per bit line is increased, then computational accuracy is improved, but coupling capacitance between bit lines increases
Solution Approach 1:
By dividing the memory array into separate blocks with isolated bit line groups, the patent reduces coupling capacitance between bit lines. Each bit line group operates independently, allowing high computational accuracy without the harmful capacitive coupling that would occur in a single large block.
Solution Approach 2:
The vertical stacking of memory blocks with bit line groups at different heights creates spatial separation in the third dimension. This dimensional separation significantly reduces coupling capacitance between adjacent bit lines while maintaining the high number of memory strings per bit line needed for computational accuracy.
3Ease of operation
If traditional von Neumann architecture is used, then data transfer between CPU and memory is simple, but bandwidth becomes a bottleneck
Solution Approach 1:
The patent merges computing functions directly with memory by implementing computational operations within the memory array itself. This combining of computation and storage eliminates the need for continuous data transfer between CPU and memory, maintaining operational simplicity while dramatically increasing bandwidth through parallel in-memory computations.
Solution Approach 2:
The patent enables continuous computational operations within memory without interruption for data transfer. By performing computations directly in the memory array, the system maintains continuous useful action, eliminating the bandwidth bottleneck that would occur with traditional von Neumann architecture where data must be constantly transferred between CPU and memory.
Data Source
AI summary
According to one aspect of the present disclosure, a semiconductor device is provided. The semiconductor device may include a memory array comprising at least one memory block. The memory block may include a plurality of memory string columns disposed in a first direction. Each of the memory string columns may include a plurality of memory strings disposed in a second direction perpendicular to the first direction. Each of the memory strings may include a plurality of memory cells disposed along a third direction and connected in series. The semiconductor device may include a bit line layer including at least one bit line group. The bit line group may include a plurality of bit lines disposed in the first direction. Each of the bit lines may extend along the second direction and is coupled to all memory strings of one of the memory string columns within the memory block.