Multi-GPU Sparse Embedding Updates for Faster AI Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The training speed of embedding operations in AI systems is limited due to the large scale of embedding operations being performed on central processing units (CPUs), and the storage capability of graphics processing units (GPUs) is inadequate for large-scale embeddings.
Innovation Solution
Implementing a data processing method and apparatus that utilizes multiple GPUs for efficient processing and storage of sparse features, including acquiring, forming, and updating embeddings across GPUs, with distributed storage based on content or data format types to optimize storage and reduce overheads.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If embedding operations are performed on CPU, then storage capability is sufficient, but training speed is slow
Solution Approach 1:
The patent segments the embedding storage and processing across multiple GPUs, dividing the large-scale embedding data into partitions that can be distributed and managed individually. This allows the system to leverage GPU computational power while maintaining sufficient storage capacity through distributed architecture.
Solution Approach 2:
The patent transitions from single-CPU or single-GPU architecture to a multi-GPU distributed system, adding the dimension of spatial distribution across multiple processing units. This dimensional change enables simultaneous achievement of high training speed through GPU parallelism and sufficient storage through distributed embedding partitions across multiple devices.
2Productivity
If large-scale embeddings are stored in CPU, then storage is adequate, but embedding training speed is slow
Solution Approach 1:
The patent introduces a distributed embedding storage mechanism that acts as an intermediary between multiple GPUs and the embedding data. This mediator manages the distribution, retrieval, and updating of embedding partitions across GPUs, enabling high training speed through GPU acceleration while managing system complexity through centralized coordination of the distributed storage system.
Solution Approach 2:
The patent combines multiple GPUs into a unified embedding processing system, merging their computational resources to handle large-scale embedding training. By combining the computing power of multiple GPUs with a coordinated storage management system, the patent achieves high productivity while managing complexity through integrated architecture design.
3Speed
If embeddings are distributed across multiple GPUs, then training speed improves, but storage management complexity increases
Solution Approach 1:
The patent implements a feedback mechanism where the distributed embedding storage system continuously monitors and coordinates updates across multiple GPUs. When embeddings are updated on one GPU, the system provides feedback to other GPUs to synchronize their local partitions, ensuring data consistency while maintaining high update processing speed through parallel operations.
Solution Approach 2:
The patent employs preliminary action by pre-distributing embedding partitions to multiple GPUs before training begins and pre-coordinating the update mechanisms. This preliminary setup reduces the complexity of real-time storage management during training, as the framework for handling updates and synchronizations is already established, enabling fast update processing without ad-hoc coordination overhead.
Data Source
AI summary
A method, apparatus, and computer-readable storage medium providing data processing on at least one graphics processing unit (GPU). The method includes: acquiring sparse features of a target sample comprising first and second sparse features of different types; acquiring embeddings corresponding to first sparse features from stored embeddings of full sparse features of the first type; acquiring embeddings corresponding to second sparse features from a second GPU based on querying embeddings of full sparse features of the second type; forming embeddings corresponding to the sparse features of the target sample; performing probability mapping on the formed embeddings; generating an update instruction based on the probability mapping; and updating the embeddings of the full sparse features of the second type.