Reconfigurable Processor Chip with Distributed Storage and Data Flow Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI chip architectures face communication bottlenecks and low computing power utilization due to shared storage architectures, leading to inefficiencies in data transmission and high costs in large-scale GPU clusters, especially when handling super-large AI models like GPT-3.
Innovation Solution
A data-flow-driven reconfigurable processor chip and cluster with distributed storage, programmable data routing elements, and a data flow controller that allows real-time configuration of data packet routing and communication, enabling flexible and efficient data transmission across reconfigurable processing elements without shared storage, and supporting various parallel computing modes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If shared storage architecture is used in GPU, then data access is simplified, but communication bottleneck and access bottleneck occur during distributed parallel computation
Solution Approach 1:
The patent divides the storage system into distributed storage across multiple nodes rather than a single shared storage pool. Each processing element has its own local storage, and data is replicated across multiple nodes. This segmentation eliminates the central bottleneck while maintaining data accessibility through distributed queries and transactions.
Solution Approach 2:
The patent introduces a coordination service as an intermediary layer between processing elements and storage. This mediator handles data routing, consistency management, and communication coordination, simplifying the interface for processing elements while enabling efficient distributed data access without shared storage constraints.
2Power
If data parallelism is used in GPU, then computing power is increased, but computing power utilization remains low due to inability to mine parallelism from other algorithm dimensions
Solution Approach 1:
The patent implements dynamic task scheduling and flexible data flow routing that adapts to different algorithm requirements. The system can dynamically reconfigure computation graphs and data paths to exploit parallelism across multiple algorithm dimensions, not just fixed data parallelism patterns, thereby improving computing power utilization.
Solution Approach 2:
The patent changes the fundamental parameters of computation by supporting both data parallelism and algorithmic parallelism simultaneously. Through configurable execution models and dynamic graph rewriting, the system can adjust parallelization strategies based on algorithm characteristics, enabling efficient utilization of computing power across diverse parallelism types.
3Adaptability or versatility
If distributed parallelism is increased to handle large models, then algorithm capability is improved, but communication demand increases significantly affecting system performance
Solution Approach 1:
The patent segments data and computation across distributed nodes, with each node handling a subset of data and computations. This segmentation reduces the communication footprint by enabling local processing and reducing the need for global synchronization, thereby managing communication demand while maintaining algorithm capability.
Solution Approach 2:
The patent implements asynchronous communication and continuous data flow mechanisms that maintain productive computation while managing communication. By allowing overlapping computation and communication operations, the system maintains useful action continuity and reduces idle waiting time associated with communication overhead.
4Power
If GPU cluster is scaled up for large model training, then computing capability is improved, but cost increases due to high-speed network switch requirements
Solution Approach 1:
The patent employs standard network infrastructure components rather than expensive specialized high-speed network switches. By using commodity hardware with appropriate software optimization, the system achieves scalable computing capability at lower cost, replacing expensive dedicated networking equipment with more affordable standard components.
Solution Approach 2:
The patent designs a universal communication framework that works with standard network hardware across different platforms. This multi-functional approach allows the same system to operate with various network configurations and hardware types, reducing dependency on expensive specialized equipment and lowering overall system cost.
Data Source
AI summary
A reconfigurable processor chip includes: a plurality of reconfigurable processing elements based on distributed storage, components of the reconfigurable processing elements being logically interconnected. The components include: a reconfigurable computing component configured to calculate data; a data flow controller using a data flow driving mode, the data flow driving mode is configured to control start and end of a computing task and a data transmission task based on data flow information about the computing task and message transferring of upstream and downstream reconfigurable processing elements; a distributed memory configured to implement data storage of a corresponding reconfigurable processing element; and a programmable data routing element configured to implement communication between the plurality of reconfigurable processing elements to control a direction of a data packet, and implement flexible transmission of the data packet.