AI Task Communication Protocol Using Direct Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing communication protocols in datacenters are not optimized for AI computing tasks, leading to latency, low performance, and high CPU resource utilization, which limits the efficiency of AI compute resources in disaggregated systems.
Innovation Solution
The AI over Fabric (AIoF) protocol enables direct memory access and direct data transfer between AI clients and servers, using a transport abstraction layer to support low-latency, high-performance connectivity and quality of service for AI tasks, including machine learning and neural network processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If traditional communication protocols (HTTP over TCP or gRPC) are used for AI task communication, then compatibility and ease of operation are maintained, but latency increases and processing performance deteriorates
Solution Approach 1:
The patent changes the fundamental parameters of the communication protocol by introducing AIoF with specialized frame structures, direct memory access capabilities, and optimized data transfer mechanisms that bypass traditional TCP/IP stack processing, thereby reducing latency while maintaining ease of use through abstraction
Solution Approach 2:
The patent introduces an AIoF protocol as an intermediary layer between AI clients and servers, providing a specialized communication mechanism that optimizes for AI workloads while presenting a simplified interface to applications, thus reducing latency without compromising ease of operation
2Productivity
If disaggregated AI compute resources are deployed, then resource utilization efficiency improves, but communication overhead and CPU resource consumption increase
Solution Approach 1:
The patent extracts the communication overhead from the CPU by implementing direct memory access and specialized AIoF protocol handling that operates independently of the main CPU processing pipeline, thereby reducing CPU resource consumption while maintaining efficient resource utilization
Solution Approach 2:
The patent replaces the traditional software-based TCP/IP stack with a specialized AIoF protocol mechanism that uses direct memory access and hardware-assisted data transfer, substituting CPU-intensive software processing with more efficient hardware-level operations
3Productivity
If optimized communication protocol is implemented, then processing performance and latency improve, but device complexity increases
Solution Approach 1:
The patent implements a universal AIoF protocol that handles multiple AI workloads and communication patterns through a unified framework, reducing the apparent complexity by providing a single optimized interface for diverse AI processing tasks
Data Source
AI summary
A system and method for communicating artificial intelligence (AI) tasks between AI resources are provided. The method comprises establishing a connection between a first AI resource and a second AI resource; encapsulating a request to process an AI task in at least one request data frame compliant with a communication protocol, wherein the at least one request data frame is encapsulated at the first AI resource; and transporting the at least one request data frame over a network using a transport protocol to the second AI resource, wherein the transport protocol provisions the transport characteristics of the AI task, and wherein the transport protocol is different than the communication protocol.