Processing-in-Memory Architecture for AI Data Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The performance of artificial intelligence hardware is hindered by limitations in data communication between memory and processor in existing systems where they are separated, leading to increased arithmetic requirements for deep learning operations.
Innovation Solution
A processing-in-memory (PIM) device integrates arithmetic circuits and memory circuits in a semiconductor chip, allowing for memory cells to be designated by memory addresses and accessed through a register for arithmetic operations, using input data to generate arithmetic results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If memory and processor are separated in existing hardware systems, then device complexity is reduced and manufacturing is easier, but data communication between memory and processor becomes a bottleneck that hinders AI performance
Solution Approach 1:
The patent merges the processor and memory into a single integrated device. The processing-in-memory device includes both an arithmetic circuit for performing computations and a memory circuit for storing data, eliminating the need for separate memory and processor components. This integration allows arithmetic operations to be performed directly within the memory device, removing data communication bottlenecks and significantly improving AI computing performance.
2Productivity
If arithmetic circuits and memory circuits are integrated in a processing-in-memory device, then data communication bottlenecks are reduced and computational efficiency is improved, but device complexity increases
Solution Approach 1:
The processing-in-memory device performs multiple functions within a single integrated structure. The arithmetic circuit can perform various arithmetic operations (multiplication, accumulation, etc.) while the memory circuit can store different types of data (input data, weight data, result data). This multi-functionality allows the device to handle complete neural network computing tasks internally, improving computational efficiency for AI workloads.
Solution Approach 2:
The integrated device is segmented into distinct functional units: an arithmetic circuit with multiple operation units for different computing tasks, and a memory circuit with multiple banks for different data types. This segmentation allows each unit to be optimized for its specific function while working together as an integrated system, managing the complexity through modular organization.
3Ease of operation
If multiple temporary memory addresses are allocated for arithmetic operations, then access to input data and weight data is improved, but memory address space management becomes more complex
Solution Approach 1:
The patent introduces a temporary memory address as an intermediary between the external address space and the internal memory structure. When the processing-in-memory device receives a memory address in arithmetic mode, it allocates a temporary memory address that maps to the appropriate location in the memory circuit. This intermediary address simplifies the access process for input data and weight data while the control circuit handles the complex mapping and management internally.
Data Source
AI summary
A processing-in-memory device includes a memory circuit including a memory circuit including a plurality of memory cells, the memory circuit configured to designate the plurality of memory cells by a plurality of memory addresses, wherein the plurality of memory addresses are allocated to a memory address space in a memory mode, a register configured to be accessed based on a temporary memory address allocated among the plurality of memory addresses in an arithmetic mode, and an arithmetic circuit configured to perform an arithmetic operation using first and second input data to generate arithmetic result data.