Computational Storage Embedding Vector Calculation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network training operations are computationally burdensome, particularly due to the processing demands of embedding operations for categorical inputs, which require large amounts of storage.
Innovation Solution
A system comprising a GPU cluster and a computational storage system, where the GPU system handles continuous input processing and the computational storage system handles categorical input processing using embedding tables, with speculative recovery to reduce processing time by calculating embedded vectors without waiting for table updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If embedding tables are updated based on gradient calculations during neural network training, then the accuracy of the model is improved, but the processing time increases due to waiting for table updates
Solution Approach 1:
The computational storage device performs preliminary actions by calculating embedded vectors in advance using existing embedding table data before the gradient-based updates are complete. This allows the GPU to proceed with processing using pre-calculated embeddings without waiting for the sequential update operations to finish, thereby reducing processing time while maintaining model accuracy through subsequent update applications.
2Quantity of substance
If embedding tables are stored in computational storage system, then the storage capacity is improved, but the device complexity increases
Solution Approach 1:
The patent merges the storage function and computational function into a single computational storage device. The embedding tables are stored in the computational storage system's memory, and the same device performs the embedding calculations by retrieving data from its own storage and computing embedded vectors. This integration eliminates the need for separate storage and computation components, reducing overall system complexity while providing sufficient storage capacity for large embedding tables.
Data Source
AI summary
A system and method for training a neural network. In some embodiments, the system includes a computational storage device including a backing store. The computational storage device may be configured to: store, in the backing store, an embedding table for a neural network embedding operation; receive a first index vector including a first index and a second index; retrieve, from the backing store: a first row of the embedding table, corresponding to the first index, and a second row of the embedding table, corresponding to the second index; and calculate a first embedded vector based on the first row and the second row.