Modular AI Training Architecture with Scalable Accelerator Baseboards
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computational systems for training AI large models face challenges with high system complexity, high costs, and scalability issues due to the massive number of parameters and computational requirements, necessitating substantial computational resources and data storage.
Innovation Solution
A computing system architecture that includes a mainboard with a CPU connected to a baseboard equipped with multiple accelerator cards, allowing for parallel model training tasks distributed across these cards, with direct interconnections between accelerator cards and expandable computing nodes for scalable computational power and bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing computational systems are used to train AI large models, then model training can be performed, but system complexity and costs increase significantly
Solution Approach 1:
The system segments the model training task into multiple parallel sub-tasks distributed across multiple accelerator cards. Each accelerator card processes a portion of the model parameters independently, and the mainboard coordinates the parallel computation and aggregates results, thereby reducing the computational burden on any single component and lowering overall system complexity.
Solution Approach 2:
The patent introduces a hierarchical dimensional structure with mainboard at one level and multiple accelerator cards at another level. This multi-dimensional architecture allows parallel processing across cards while maintaining centralized coordination, effectively distributing complexity across dimensions rather than concentrating it in a single system layer.
2Productivity
If existing computational systems are used to train AI large models, then model training can be performed, but costs increase significantly
Solution Approach 1:
The patent merges multiple accelerator cards into a coordinated parallel computing system under unified mainboard management. By combining the computational power of multiple cards and optimizing their collaborative operation, the system achieves high-throughput model training that is more cost-effective than using a single high-end system or multiple independent systems.
Solution Approach 2:
The system dynamically adjusts computational parameters such as batch size, learning rate, and parallel processing degree based on the model scale and hardware configuration. This parameter optimization ensures efficient utilization of computational resources, reducing waste and lowering the overall cost of model training while maintaining high productivity.
3Productivity
If accelerator cards are increased to handle larger models, then model training capability improves, but communication bandwidth requirements increase
Solution Approach 1:
The mainboard serves as an intermediary between accelerator cards, managing data distribution and result aggregation. It implements optimized communication protocols and buffering mechanisms that reduce the direct communication burden between accelerator cards, thereby scaling the system with manageable bandwidth requirements even as the number of cards increases.
Data Source
AI summary
The present application relates to a computing system, a model training method and apparatus, and a product. The computing system relates to a computing unit, and the computing unit comprises: a main board, which is configured with a central processing unit (CPU); and a base board, which is connected to the main board by means of a first communication link, wherein the base board is configured with a plurality of accelerator cards, and the plurality of accelerator cards are connected to each other by means of a second communication link. The main board is used for splitting a training task of a target model into a plurality of concurrent model training tasks and releasing same to the plurality of accelerator cards, and processing training results of the plurality of accelerator cards, so as to obtain a trained target model. The plurality of accelerator cards are used for concurrently executing the respective model training tasks thereof, so as to obtain the training results. The computing system forms an elastically scalable computing system architecture by means of modular base-board design and interconnection, such that the computing power and bandwidth of the computing system can match model training tasks at different parameter scales.