AI-Accelerated Memory Processing to Reduce Data Transfer Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing memory devices face challenges in performing artificial intelligence operations efficiently due to high latency and power consumption when data is transferred between the memory device and the host.
Innovation Solution
Incorporating an artificial intelligence (AI) accelerator within the memory device to perform AI operations using data stored in memory arrays, reducing the need for data transfer and enhancing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is transferred between the memory device and the host for AI operations, then the AI operations can be performed, but the latency increases and power consumption increases
Solution Approach 1:
The patent combines the AI accelerator with the memory device into a single integrated system. The AI accelerator is physically coupled to the memory arrays, allowing data to be processed directly within the memory device without external data transfer. This merging eliminates the time loss associated with data transmission between separate memory and processing units.
Solution Approach 2:
The patent introduces an intermediary mechanism (the coupled AI accelerator) that enables direct processing of data within the memory device. This intermediary allows the system to perform AI operations on data while it resides in the memory arrays, acting as a bridge between data storage and data processing functions.
2Productivity
If data is transferred between the memory device and the host for AI operations, then the AI operations can be performed, but the power consumption increases
Solution Approach 1:
The integration of the AI accelerator with the memory device consolidates data storage and data processing functions into a single system. This merging eliminates the energy-consuming data transfer process between separate memory and processing units, significantly reducing overall power consumption while maintaining AI operation capability.
Solution Approach 2:
The memory device becomes self-sufficient by incorporating the AI accelerator, allowing it to perform AI operations internally without requiring external processing resources. This self-service capability eliminates the need for energy-intensive data transmission to and from external hosts, reducing power consumption.
3Productivity
If an AI accelerator is integrated within the memory device, then latency is reduced and power consumption is reduced, but the device complexity increases
Solution Approach 1:
The integrated system performs multiple functions: it serves as both a memory device for data storage and an AI accelerator for data processing. This multi-functionality justifies the increased device complexity by providing dual capabilities within a single integrated unit, eliminating the need for separate memory and processing components.
Solution Approach 2:
The patent merges two previously separate functions (memory storage and AI processing) into a single integrated device. While this increases internal complexity, it simplifies the overall system architecture by eliminating the need for external AI processing units and their associated data transfer interfaces.
Data Source
AI summary
Apparatuses and methods are related to an artificial intelligence accelerator in memory. Apparatuses can include a number of registers configured to enable the apparatus to operate in an artificial intelligence mode to perform artificial intelligence operations and an artificial intelligence (AI) accelerator configured to perform the artificial intelligence operations using the data stored in the number of memory arrays. The AI accelerator can include hardware, software, and or firmware that is configured to perform operations associated with AI operations. The hardware can include circuitry configured as an adder and/or multiplier to perform operations, such as logic operations, associated with AI operations.