AIoF Protocol With Credit-Based Flow Control for AI Compute
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional communication protocols in datacenters are not optimized for AI computing tasks, leading to latency, low performance, and inefficient resource management, which hampers the optimization of AI compute resources in disaggregated environments.
Innovation Solution
A communication protocol, AI over Fabric (AIoF), which encapsulates AI task requests in data frames and uses a credit-based flow control mechanism over a transport protocol to avoid compute resource congestion, ensuring high-performance, low-latency connectivity and end-to-end quality of service.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional communication protocols (HTTP/TCP/GRPC) are used for AI task communication, then compatibility and ease of operation are improved, but latency and processing overhead increase significantly
Solution Approach 1:
The patent changes the fundamental parameters of the communication protocol by introducing a custom binary frame format instead of text-based protocols, and by implementing a credit-based flow control mechanism that operates at the transport layer rather than application layer, thereby reducing processing overhead and latency while maintaining operational simplicity through standardized frame structures
Solution Approach 2:
The patent introduces an intermediary communication layer (AIoF protocol) that sits between the AI compute resources and the application layer, translating high-level communication needs into optimized low-level data transmissions with explicit flow control credits, thus mediating between compatibility requirements and performance optimization
2Adaptability or versatility
If CPU-based software processes manage AI compute resources, then flexibility and adaptability are improved, but processing speed and productivity decrease due to software overhead
Solution Approach 1:
The patent replaces the mechanical software-based resource management system with a hardware-accelerated communication interface that uses dedicated network interface cards (NICs) with embedded flow control logic, eliminating CPU intervention in the communication path and thereby increasing processing speed while maintaining management flexibility through programmable credit allocation
Solution Approach 2:
The patent implements self-service by enabling AI compute resources to autonomously manage their own communication flow through credit-based flow control, where each resource can independently request and receive credits without requiring centralized software scheduling, thus improving productivity while maintaining adaptability through configurable credit policies
3Adaptability or versatility
If compute resources are disaggregated into separate AI servers, then adaptability and resource utilization are improved, but communication overhead and system complexity increase
Solution Approach 1:
The patent applies universality by designing a single standardized AIoF communication protocol and frame format that can handle all types of AI workloads (training, inference, preprocessing, postprocessing) across disaggregated resources, eliminating the need for multiple specialized communication interfaces and thereby reducing system complexity while maintaining disaggregation flexibility
Solution Approach 2:
The patent segments the communication protocol into distinct, well-defined layers (frame header, payload, flow control credits) that can be independently implemented and optimized, allowing complex disaggregated systems to be built from simple, standardized building blocks, thus managing complexity while enabling flexible resource allocation
4Reliability
If credit-based flow control is implemented to avoid compute resource congestion, then reliability and performance assurance are improved, but protocol complexity and overhead increase
Solution Approach 1:
The patent implements preliminary action by pre-allocating flow control credits to AI compute resources before data transmission begins, allowing resources to immediately send data frames without waiting for real-time flow control decisions, thus ensuring reliable performance while reducing the complexity of runtime flow control management
Solution Approach 2:
The patent ensures continuity of useful action by maintaining a continuous stream of credit allocations and data transmissions without idle waiting periods, where credits are replenished and consumed in a continuous cycle, thereby ensuring reliable performance assurance while keeping the flow control mechanism simple and predictable
Data Source
AI summary
A method and system for communicating artificial intelligence (AI) tasks between AI resources are provided. The method includes establishing a connection between a first AI resource and a second AI resource; encapsulating a request to process an AI task in at least one request data frame compliant with a communication protocol, wherein the at least one request data frame is encapsulated at the first AI resource; transporting the at least one request data frame over a network using a transport protocol to the second AI resource, wherein the transport protocol is different than the communication protocol; and using a credit-based flow control mechanism to transfer messages between the first AI resource and the second AI resource over the transport protocol, thereby avoiding congestion on compute resources.