In-Memory MAC Operator for Neural Network Data Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning neural networks require increased computational resources due to the exponential growth in data processing needs, and traditional hardware systems with separate processors and memory lead to performance degradation due to limited data communication between these components.
Innovation Solution
A PIM (Processing-In-Memory) system with integrated processors and memory, utilizing a MAC operator that includes data type converters and multipliers to perform arithmetic operations directly within the semiconductor chip, enhancing data processing speed for neural networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional hardware systems with separate processors and memory are used, then system architecture is simple and易于制造, but data communication between memory and processor is limited causing performance degradation
Solution Approach 1:
The patent merges the processor and memory into a single integrated device, allowing arithmetic operations to be performed directly within the memory structure. This integration eliminates the need for separate data communication channels between processor and memory, thereby resolving the performance degradation caused by limited data communication while maintaining a unified system architecture.
Solution Approach 2:
The integrated device performs multiple functions: it stores data in memory cells and simultaneously performs arithmetic operations using the stored data. The memory structure serves both as storage medium and as the basis for computational operations, enabling the system to handle both data storage and processing tasks within a single unified structure.
2Productivity
If the number of layers in neural network is increased to improve AI performance, then AI performance is improved, but the amount of computation required increases exponentially
Solution Approach 1:
The system performs computation directly where data is stored, eliminating the need to transfer data between separate processor and memory units. Each memory cell participates in arithmetic operations using its stored data, allowing the system to handle increased computational requirements of deeper neural networks without proportionally increasing power consumption or computational resource demands.
3Speed
If PIM device performs arithmetic operations internally, then data processing speed is improved, but device structure becomes more complex
Solution Approach 1:
The patent replaces traditional mechanical/electrical data transfer mechanisms with direct in-memory computation. Instead of physically moving data between separate processor and memory components, the system performs arithmetic operations directly on the data stored in memory cells, eliminating the need for complex data transfer infrastructure and reducing overall device structural complexity.
Data Source
AI summary
A MAC operator includes a plurality of data type converters and a plurality of multipliers. Each of the plurality of data type converters may receive 16-bit input data of one of first to fourth data types of a floating-point format to convert into L-bit output data of the floating-point format. Each of the plurality of multipliers may perform a multiplication on the “L”-bit output data of the floating-point format outputted from two of the plurality of data type converters to output multiplication result data of the floating-point format.