PIM Controller Queue Scheduling for In-Memory Neural Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Processing-in-memory (PIM) systems face performance degradation due to the separation of memory and processor in traditional hardware systems, leading to inefficiencies in data communication and computation for deep learning applications, particularly in deep neural networks.
Innovation Solution
A PIM controller is configured with read/arithmetic queue logic, write queue logic, and scheduling logic to manage data processing and arithmetic operations within a PIM device, ensuring efficient scheduling and data flow between memory and arithmetic operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If memory and processor are separated in traditional hardware systems, then device complexity is reduced and ease of manufacture is improved, but data communication latency increases and processing speed deteriorates
Solution Approach 1:
The patent merges memory and processor functions into a single PIM device, where memory cells directly perform arithmetic operations on stored data. This integration eliminates data communication latency between separate memory and processor components while maintaining manageable device complexity through unified architecture design.
Solution Approach 2:
The patent introduces a controller as an intermediary component that manages the PIM device, handling queue operations and scheduling logic. This mediator coordinates between the host system and the integrated PIM device, abstracting the complexity of in-memory computing operations while enabling high-speed processing.
2Productivity
If the number of layers in neural networks is increased to improve AI performance, then AI performance is improved, but the amount of computation required increases exponentially
Solution Approach 1:
The patent extracts arithmetic operation capabilities directly from the processor and places them within the memory cells themselves. By performing computations where data is stored, the system eliminates the need to move data between memory and processor, significantly reducing the computational power and energy required for deep neural network operations with increased layers.
3Speed
If data is frequently communicated between memory and processor, then data accessibility is improved, but data communication latency increases and performance degrades
Solution Approach 1:
The patent combines memory storage and arithmetic processing functions within the same physical location (memory cells). This merging eliminates the physical data communication path between separate memory and processor components, thereby eliminating data communication latency and enabling immediate processing of stored data.
Data Source
AI summary
A processing-in-memory (PIM) controller includes a read/arithmetic queue logic circuit, a write queue logic circuit, and a scheduling logic circuit. The read/arithmetic queue logic circuit is configured to store a read queue and an arithmetic queue. The write queue logic circuit is configured to store a write queue for requesting to write data in the PIM device. The scheduling logic circuit is configured to perform the scheduling operation such that the write queue logic circuit outputs the arithmetic write queue before the read/arithmetic queue logic circuit outputs the arithmetic queue when the arithmetic queue exists in the read/arithmetic queue logic circuit and the arithmetic write queue exists in the write queue logic circuit.