Processing-In-Memory Device With In-Memory Arithmetic for Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The separation of memory and processor in traditional hardware systems limits data communication, degrading the performance of artificial intelligence systems, particularly in deep learning applications, due to the increased computational demands of neural networks with multiple layers.
Innovation Solution
A processing-in-memory (PIM) system integrates a processor and memory on a semiconductor chip, enabling the PIM device to perform both memory access and arithmetic operations in response to commands from a PIM controller and a high-speed interface.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If memory and processor are separated in traditional hardware systems, then device complexity is reduced and manufacturing is easier, but data communication between memory and processor is limited, degrading AI system performance
Solution Approach 1:
The patent merges the processor and memory into a single integrated PIM device, where computing units are directly coupled to memory cells within the same semiconductor chip. This eliminates the need for separate memory and processor components, resolving the contradiction by combining previously separated functions into one unified device that delivers both high AI performance and manageable complexity.
Solution Approach 2:
The PIM device performs multiple functions within a single integrated structure: it stores data in memory cells and executes arithmetic operations through computing units that directly access the memory. This multi-functionality allows the device to serve as both memory and processor, improving AI system performance without requiring separate specialized components for each function.
2Measurement precision
If the number of layers in neural networks is increased to improve AI performance, then computational accuracy improves, but the amount of computation required increases exponentially
Solution Approach 1:
The patent transitions computation from a separate processor dimension to an integrated memory dimension, where computing units are embedded within the memory array. This dimensional integration allows data to be processed in-place without being moved to a separate computing unit, reducing the power required for data transfer and enabling deeper neural networks to be executed with lower overall power consumption.
3Productivity
If a general hardware system with separate memory and processor is used, then device complexity is reduced, but data communication limitation degrades AI performance
Solution Approach 1:
The patent combines memory storage and arithmetic processing functions into a single PIM device, eliminating the data communication bottleneck between separate memory and processor components. The computing units are directly coupled to memory cells, allowing data to be processed immediately after retrieval without external data transfer, thus achieving high data processing speed while maintaining manageable system architecture complexity.
Data Source
AI summary
A processing-in-memory (PIM) system includes a host configured to generate a first request for a memory access operation and a second request for an arithmetic operation, a PIM controller configured to generate a first command based on the first request or the second request, a high speed interface configured to generate a second command based on the second request, and a PIM device configured to perform the memory access operation in response to the first command from the PIM controller and to perform the arithmetic operation in response to the second command from the high speed interface.