DMA Data Access with FIFO Prefetch for Low-Latency Channel Switching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing Direct Memory Access (DMA) technology in computer systems experiences latency and reduced efficiency during channel switching due to pipeline processing failures, particularly in neural-network processing units (NPUs) like Eyeriss V2, which affects data transmission between Global Buffer clusters.
Innovation Solution
Implementing a preset First-In-First-Out (FIFO) queue and a double Round Robin scheduling mechanism to pre-fetch instruction data, separating the fetching operation from the post-stage circuit, thereby reducing latency and improving DMA efficiency by allowing pipeline processing during channel switching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If DMA channel switching is implemented without pipeline processing, then channel switching can be completed, but latency increases and efficiency decreases
Solution Approach 1:
The patent applies preliminary action by pre-fetching instruction data into a FIFO queue before channel switching occurs. The double Round Robin scheduling mechanism prepares the next channel's instruction data in advance, so when switching is needed, the data is already ready in the FIFO queue, eliminating the pipeline stall that would otherwise occur during channel transitions.
Solution Approach 2:
The patent segments the DMA operation into distinct phases: instruction fetching, pipeline processing, and channel switching. By separating the fetching operation from the post-stage circuit and using a dedicated FIFO queue for instruction data, the system can independently manage each phase, allowing pipeline processing to continue uninterrupted during channel switching.
2Productivity
If pipeline processing is used during channel switching, then DMA efficiency improves, but system complexity increases
Solution Approach 1:
The patent introduces a FIFO queue as an intermediary buffer between the instruction fetch unit and the pipeline processing unit. This intermediary structure decouples the fetching operation from the post-stage circuit, allowing independent optimization of each component while maintaining smooth data flow during channel switching, thus improving efficiency without proportionally increasing complexity.
Solution Approach 2:
The double Round Robin scheduling mechanism dynamically selects which channel's instruction data to fetch based on current system state and channel priorities. This dynamic adaptation allows the system to optimize performance for different workload patterns without requiring a completely different architectural approach for each scenario.
Data Source
AI summary
An apparatus and a method for accessing data, a processing apparatus, and a computer system are disclosed. The apparatus for accessing data includes a storing circuit, a reading circuit, a first-in-first-out queue, and a fetching circuit; the storing circuit is configured to store the data received by the apparatus; the reading circuit is configured to read the data out of the storing circuit according to a read request signal and write the data into the first-in-first-out queue; the first-in-first-out queue is configured to execute writing of the data according to a control signal; the fetching circuit is configured to fetch the data out of the first-in-first-out queue; and the read request signal is under control of a first state signal of the first-in-first-out queue and a second state signal of the storing circuit, and the first state signal indicates a full state or a non-full state of the first-in-first-out queue.