Multi-Buffer Data Transfer for Indeterminate Accelerator Outputs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data management techniques in digital systems, particularly for accelerators, face inefficiencies due to buffer overflows and latency issues when handling workloads with indeterminate output sizes, leading to increased resource usage and processing delays.
Innovation Solution
Implementing a mechanism where accelerators can continue processing commands without waiting for previous commands to complete, using ordered output storage across multiple buffers with automatic overflow management, allowing parallel execution and reducing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If data is stored in a buffer after processing, then data management is simplified, but buffer overflow and capacity limitations reduce system efficiency
Solution Approach 1:
The patent divides the storage system into multiple buffers (first buffer, second buffer, third buffer) that can be used in sequence. When one buffer becomes full, the system automatically transitions to the next buffer, allowing continuous operation without waiting for data retrieval. This segmentation resolves the contradiction by maintaining ease of operation through automated buffer management while improving productivity by eliminating idle time caused by buffer overflow.
Solution Approach 2:
The patent pre-allocates multiple buffers before processing begins and establishes the buffer chain structure in advance. The system prepares the first buffer for immediate use, and when it fills up, the pre-configured second buffer is automatically activated. This preliminary action eliminates the need for dynamic buffer allocation during processing, maintaining operational simplicity while ensuring continuous data flow and high system efficiency.
2Productivity
If commands are executed in parallel for workloads with indeterminate output size, then processing speed increases, but additional memory management processes are required
Solution Approach 1:
The patent implements self-service buffer management where the system automatically tracks which buffer is currently in use and when to transition to the next buffer. The buffer management logic is embedded in the processing system itself, eliminating the need for external memory management processes. This allows parallel command execution with indeterminate output sizes while keeping memory management simple and integrated, resolving the contradiction between processing speed and device complexity.
3Device complexity
If the system waits for previous commands to complete before issuing next commands, then buffer management is simplified, but processing latency increases
Solution Approach 1:
The patent enables continuous useful action by allowing the issuer to send the next command immediately after issuing a previous command, without waiting for completion. The multiple buffers absorb the output data from ongoing processing while new commands are already being executed. This continuity maintains simplified buffer management through automatic buffer tracking while dramatically reducing processing latency by eliminating idle waiting time.
4Productivity
If side buffers are used for parallel command execution, then processing throughput increases, but a second process is needed to copy outputs
Solution Approach 1:
The patent merges the buffer management function into the processing system itself rather than requiring a separate copying process. The same system that executes commands also manages the buffer chain, tracking data locations and automating the transition between buffers. This merging eliminates the need for a second process to copy outputs from side buffers, maintaining high processing throughput while reducing process overhead and system complexity.
Data Source
AI summary
Embodiments of the present disclosure include techniques for moving data between electronic system components using buffers. A digital processor stores data generated in response to a series of commands in a first buffer. Commands are received with a reference to a second buffer. The digital processor tracks the last location that a data result was stored in the first buffer. When a data result fills the buffer, the remaining data is automatically stored in the second buffer. Downstream devices may empty full buffers. A client may receive an indication that a buffer is empty and subsequently send commands with a reference to empty buffer.