Memory-Operator Request Scheduling for AI Data Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The integration of processors and memories in semiconductor chips is hindered by limitations in data communication, which impedes the performance improvement of artificial intelligence hardware due to increased operational requirements.
Innovation Solution
A memory device with a memory-operator unit and a request processing circuit that schedules memory and operation requests based on status, allowing efficient transmission and processing of commands and addresses, and includes queues and arbiter circuits to manage request prioritization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If processors and memories are integrated in a semiconductor chip, then the performance of artificial intelligence hardware is improved, but the limitations in data communication between memory and processor worsen
Solution Approach 1:
The patent merges the memory device and processor into a single semiconductor chip, creating an integrated neural network computing device. This integration eliminates the need for external data communication between separate memory and processor components, directly resolving the bottleneck of data communication limitations while maintaining improved AI performance
Solution Approach 2:
The integrated device performs multiple functions within a single chip, combining both memory storage and processor computing operations. This multi-functionality allows the device to handle both data storage and neural network computations internally, overcoming the communication limitations between separate components
2Productivity
If the number of neural network layers is increased, then the performance of artificial intelligence is improved, but the amount of data communication required worsens
Solution Approach 1:
By combining memory and processing functions in a single chip, the patent enables neural network layers to access data internally without external communication overhead. This allows increasing the number of neural network layers to improve AI performance while avoiding the proportional increase in data communication requirements that would occur with separate memory and processor components
3Device complexity
If memory and processor are separated, then device simplicity is maintained, but the performance improvement of artificial intelligence hardware is hindered
Solution Approach 1:
The patent implements integration of memory and processor in a single semiconductor chip, creating a unified neural network computing device. This merger enables improved AI hardware performance by eliminating external data communication bottlenecks while maintaining internal structural efficiency through shared resources and optimized data flow paths
Data Source
AI summary
A memory device includes a memory-operator unit including memory circuits and operating circuits, and a request processing circuit configured to process a memory request and an operation request transmitted from a software domain to transmit a memory command and an address corresponding to the memory request and an operation command and an address corresponding to the operation request to the memory-operator unit. The request processing circuit is configured to schedule processing of the memory request and the operation request, based on a status of the memory request remaining in the request processing circuit, when the operation request is transmitted to the request processing circuit from the software domain.