Memory Device AI Accelerator Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current memory devices face challenges in efficiently performing artificial intelligence (AI) operations due to high latency and power consumption when these operations are executed on a host, as they require data transfer between the memory device and the host, leading to increased latency and power usage.
Innovation Solution
Incorporating an AI accelerator within the memory device that performs AI operations using hardware, software, or firmware, including circuitry for logic operations, and allows for data copying between memory devices to execute AI operations without external processing resources, thereby reducing latency and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If AI operations are executed on the host with data transfer between memory device and host, then general-purpose processing is maintained, but latency increases and power consumption increases
Solution Approach 1:
The system segments AI processing functionality by providing dedicated AI accelerator circuits within the memory device for AI-specific operations, while the host processor handles general-purpose tasks. This segmentation allows AI operations to be executed locally in the memory device, reducing data transfer requirements and latency, while the host maintains its general-purpose processing capabilities.
2Adaptability or versatility
If AI operations are executed on the host with data transfer between memory device and host, then general-purpose processing is maintained, but power consumption increases
Solution Approach 1:
The system segments AI processing functionality by providing dedicated AI accelerator circuits within the memory device for AI-specific operations, while the host processor handles general-purpose tasks. This segmentation allows AI operations to be executed locally in the memory device, reducing data transfer requirements and latency, while the host maintains its general-purpose processing capabilities.
3Loss of time
If AI accelerator is integrated within memory device, then latency is reduced and power consumption is reduced, but device complexity increases
Solution Approach 1:
The patent merges AI accelerator circuits directly into the memory device structure, combining storage and AI processing functions in a single integrated component. This integration eliminates the need for separate AI processing units and reduces data transfer distances, thereby reducing latency and power consumption despite the increased internal complexity of the memory device.
4Use of energy by moving object
If AI accelerator is integrated within memory device, then power consumption is reduced, but device complexity increases
Solution Approach 1:
The patent merges AI accelerator circuits directly into the memory device structure, combining storage and AI processing functions in a single integrated component. This integration eliminates the need for separate AI processing units and reduces data transfer distances, thereby reducing latency and power consumption despite the increased internal complexity of the memory device.
Data Source
AI summary
The present disclosure includes apparatuses and methods related to copying data in a memory system with an artificial intelligence (AI) mode. An apparatus can receive a command indicating that the apparatus operate in an artificial intelligence (AI) mode, a command to perform AI operations using an AI accelerator based on a status of a number of registers, and a command to copy data between memory devices that are performing AI operations. The memory system can copy neural network data, activation function data, bias data, input data, and/or output data from a first memory device to a second memory device, such that that the first memory device can use the neural network data, activation function data, bias data, input data, and/or output data in a first AI operation and the second memory device can use the neural network data, activation function data, bias data, input data, and/or output data in a second AI operation.