Gateway Credit Control for Work Accelerator Data Synchronization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In computer systems, particularly in machine intelligence applications, there is a challenge in ensuring that data is available to a work accelerator subsystem at the right time during compute and exchange phases, preventing the accelerator from running ahead of data availability, especially when scaling multiple subsystems together.
Innovation Solution
A gateway is introduced to manage data transfer between external storage and the work accelerator, using a credit control mechanism to ensure data is available at pre-compiled synchronization points, allowing the accelerator to receive data in the exchange phase and process it in the compute phase, thereby preventing data unavailability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the work accelerator subsystem is scaled by connecting multiple subsystems together, then processing capacity is improved, but data availability becomes problematic as different elements may run ahead of each other causing data unavailability
Solution Approach 1:
A gateway is introduced as an intermediary component between the host and work accelerator subsystems. The gateway manages data transfer and synchronization, using credit control mechanisms to ensure data is available at pre-compiled synchronization points. This mediator prevents elements from running ahead of data availability while maintaining scaled processing capacity.
Solution Approach 2:
The system uses pre-compiled data exchange synchronization points and prepares data in advance before transfer to the work accelerator. The gateway pre-processes and stages data so that when synchronization points are reached, data is already available, preventing the accelerator from running ahead of data availability.
2Reliability
If data transfer is synchronized at pre-compiled synchronization points, then data availability is ensured, but system complexity increases due to the need for coordination mechanisms
Solution Approach 1:
The gateway autonomously manages data transfer operations using credit control mechanisms. It automatically tracks data availability, initiates transfers at appropriate synchronization points, and coordinates with the work accelerator without requiring complex external control logic, thereby reducing overall system complexity.
Solution Approach 2:
The system uses credit control parameters to manage data transfer states. By changing the state of credit counters (available vs. unavailable), the gateway simplifies the coordination mechanism into a manageable parameter-based control system that ensures data availability without excessive complexity.
3Reliability
If the gateway manages data transfer autonomously, then data synchronization is improved, but the gateway becomes a bottleneck for data transfer
Solution Approach 1:
The gateway performs preliminary data preparation and staging before synchronization points are reached. By pre-processing and positioning data in advance, the gateway reduces the amount of work needed during actual transfer operations, minimizing bottleneck effects while maintaining synchronization reliability.
Solution Approach 2:
The gateway maintains continuous data preparation and staging operations parallel to accelerator computation. While the accelerator processes data, the gateway continuously prepares the next batch, ensuring uninterrupted data flow and reducing idle time that would otherwise create bottlenecks.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A gateway for interfacing a host with a subsystem for acting as a work accelerator to the host, the gateway enabling the transfer of batches of data to and from the subsystem at precompiled data exchange synchronisation points attained by the subsystem. The gateway is configured to: receive from a storage system data determined by the host to be processed by the subsystem; store a number of credits indicating the availability of data for transfer to the subsystem at each pre-compiled data exchange synchronisation point; receive a synchronisation request from the subsystem when it attains a data exchange synchronisation point; and in response to determining that the number of credits comprises a non-zero number of credits: transmit a synchronisation acknowledgment to the subsystem; and cause the received data to be transferred to the subsystem.