Multi-Layer Switch Parallel Processing for GPU In-Network Computing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Implementing In-Network Computing (INC) in GPU cluster scenarios is challenging due to the need for efficient data processing and transmission across network devices like switches.
Innovation Solution
A method for data processing that utilizes multi-layer switches to execute stage operations in parallel for GPUs, dividing GPU groups into subgroups and utilizing switches to perform in-network computing, enhancing computing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional computing nodes (GPUs) perform all computing tasks, then computing capabilities are concentrated, but data transmission and processing efficiency are limited by network bandwidth
Solution Approach 1:
The patent extracts computing capabilities from traditional computing nodes (GPUs) and relocates them to network devices (switches). Specifically, the switch performs partial computing tasks such as data preprocessing, filtering, and aggregation directly in the network, reducing the burden on GPUs and accelerating data transmission by processing data during transit rather than after complete transmission
Solution Approach 2:
The patent introduces network switches as intermediary devices that perform computing functions between data sources and computing nodes. The switch acts as a mediator that not only forwards data but also executes computing operations, thereby improving overall system efficiency by utilizing network infrastructure for computation
2Productivity
If multi-layer switch architecture is used to execute stage operations in parallel, then computing efficiency is improved, but system complexity increases
Solution Approach 1:
The patent segments the target group of GPUs into multiple subgroups, with each subgroup handled by a dedicated bottom-layer switch. This segmentation allows parallel processing of stage operations across different subgroups while maintaining manageable complexity at each switch level. The multi-layer architecture divides the computing task into stages that can be executed concurrently by different switch layers
Solution Approach 2:
The patent introduces a multi-layer dimensional architecture where switches are organized in hierarchical layers (bottom-layer and top-layer). This dimensional change from flat to hierarchical structure enables parallel execution of stage operations across layers, improving computing efficiency while distributing system complexity across multiple manageable layers rather than concentrating it in a single complex system
3Loss of time
If all stage operations are executed sequentially on single switch, then system complexity is low, but total time consumption increases
Solution Approach 1:
The patent merges multiple stage operations into a single multi-layer switch system that can execute them in parallel. The bottom-layer switch and top-layer switch work together as an integrated system, with the bottom-layer switch handling initial stage operations and the top-layer switch handling subsequent stages, thereby reducing total execution time while maintaining coordinated simplicity through standardized inter-layer communication protocols
Data Source
AI summary
A method for data processing, an electronic device, and a storage medium are described, which relates to the field of artificial intelligence technology, specifically to the fields of intelligent cloud, network communication, large language models and other technologies. A method for data processing is applied to a bottom-layer switch in a multi-layer switch, wherein the multi-layer switch is configured to complete a target operation, the target operation includes a plurality of stage operations. The method includes: receiving a plurality of in-network computing requests sent by a top-layer switch in the multi-layer switch; wherein the plurality of in-network computing requests correspond to the plurality of stage operations one-to-one, and the plurality of in-network computing requests are sent by a current GPU to the top-layer switch; executing the plurality of stage operations in parallel for a plurality of GPUs in a current subgroup based on the plurality of in-network computing requests; wherein a target group to which the current GPU belongs is divided into a plurality of subgroups, and the current subgroup is a subgroup corresponding to the bottom-layer switch among the plurality of subgroups.