PIM Device Global Buffer Integration for AI Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Processing-in-memory (PIM) devices face limitations in performing deterministic arithmetic operations efficiently due to the separation of processors and memory in traditional hardware systems, leading to degraded performance in artificial intelligence applications like deep learning, where increased computational demands and data communication limitations hinder speed and predictability.
Innovation Solution
A PIM device is designed with a data storage region, a global buffer, and an arithmetic circuit, allowing for the integration of memory and processing functions within a semiconductor chip, enabling deterministic arithmetic operations by performing calculations directly on vector data and weight data stored within the device.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a general hardware system with separated memory and processor is used, then the system structure is simple and easy to manufacture, but the data communication limitations between memory and processor degrade the performance of artificial intelligence applications
Solution Approach 1:
The patent merges the processor and memory into a single integrated device, where the processing circuit is directly coupled to the memory circuit within the same semiconductor device. This integration eliminates the need for external data communication between separate components, thereby improving AI application performance while managing device complexity through unified architecture design.
2Productivity
If the number of layers in neural network is increased to improve AI performance, then the computational requirements increase exponentially, but the separated memory-processor architecture limits data processing speed
Solution Approach 1:
By integrating the processing circuit and memory circuit into a single semiconductor device, the patent eliminates data communication time between external components. The processing circuit can directly access and operate on data stored in the memory circuit without external transmission delays, enabling exponential growth in computational throughput as neural network layers are added.
Solution Approach 2:
The memory circuit within the integrated device can pre-load and prepare data in advance for processing operations. This preliminary action allows data to be readily available when needed by the processing circuit, reducing waiting time and enabling faster execution of computationally intensive neural network operations.
3Speed
If processing operations are performed externally with data transfer to and from memory, then the system is easier to control, but the data processing speed is limited by communication latency
Solution Approach 1:
The patent combines the processing circuit and memory circuit into a single integrated semiconductor device, enabling high-speed data processing by eliminating external communication latency. The processing circuit can directly access the memory circuit without data transfer delays, achieving superior processing speed while managing architectural complexity through integrated design.
Data Source
AI summary
A processing-in-memory (PIM) device including a data storage region, a global buffer and an arithmetic circuit. The data storage region configured to store vector data and weight data. The global buffer configured to store vector data read from the data storage region. The arithmetic circuit configured to generate a calculation result by performing a calculation on vector data read from the global buffer and weight data read from the data storage region.