Memory Chip with Integrated AI Engine and Dedicated Bus
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computer devices fail to provide effective and quick performance for neural network operations due to the limitations of existing memory architectures, such as FPGA, ASIC, and GPU, which are constrained by the memory wall, making it difficult to efficiently process complex AI operations.
Innovation Solution
A memory chip with an integrated artificial intelligence engine and a memory controller that directly accesses memory areas via dedicated buses, allowing for quick retrieval of digitized input data and weight data, and utilizing cache units for pre-reading and pipelining to perform neural network operations efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If traditional memory architectures (FPGA, ASIC, GPU) are used to perform neural network operations, then computational capability is improved, but operation speed deteriorates due to memory wall limitations
Solution Approach 1:
The patent merges the memory function and processing function into a single integrated memory device. The memory device includes memory cells for storage and an artificial intelligence engine for processing, eliminating the separation between memory and processor that causes the memory wall effect. This allows data to be processed directly within the memory device without being transferred to external processors, thereby improving operation speed while maintaining computational capability.
Solution Approach 2:
The patent introduces a dedicated bus as an intermediary channel between the memory controller and the artificial intelligence engine. This dedicated bus provides high-speed data transmission specifically for AI operations, bypassing the limitations of general-purpose memory interfaces. The dedicated bus acts as a mediator that enables fast data exchange between storage and processing components within the memory device.
2Quantity of substance
If memory bandwidth is increased to feed more data to processors, then data availability is improved, but memory latency increases due to the memory wall
Solution Approach 1:
The patent implements a data buffer within the memory device that can pre-load and store data before it is needed for processing. The artificial intelligence engine can access this pre-loaded data directly from the buffer, eliminating the need to wait for data to be fetched from external memory or storage. This preliminary action of pre-loading data into the internal buffer reduces memory latency while maintaining high data availability for AI operations.
3Power
If external processors access memory areas, then processing capability is improved, but access efficiency deteriorates due to shared bus contention
Solution Approach 1:
The patent segments the data access paths by providing a dedicated bus specifically for the artificial intelligence engine within the memory device. This dedicated bus is separate from the general-purpose memory interfaces used by external processors. By segmenting the access paths, the AI engine can access data without contending with external processors for bus resources, thereby improving access efficiency while maintaining high processing capability.
Data Source
AI summary
A memory chip capable of performing artificial intelligence operation and an operation method thereof are provided. The memory chip includes a memory array, a memory controller, and an artificial intelligence engine. The memory array includes a plurality of memory areas. The memory areas are configured to store digitized input data and weight data. The memory controller is coupled to the memory array via a bus dedicated to the artificial intelligence engine. The artificial intelligence engine accesses the memory array via the memory controller and the bus to obtain the digitized input data and the weight data. The artificial intelligence engine performs a neural network operation based on the digitized input data and the weight data.