Storage Device Vector Generation for Embedding Bandwidth Relief
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage devices face a bandwidth bottleneck during memory-intensive embedding operations, particularly in recommendation systems using deep learning, due to the large size of one-hot vectors required for embedding operations.
Innovation Solution
The storage device generates vector data based on non-zero data and its index, performing a multiplication operation with matrix data to calculate embedding vectors, thereby reducing the bandwidth bottleneck.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If one-hot vectors are used for embedding operations in recommendation systems, then the embedding operation can be performed, but a bandwidth bottleneck occurs due to the large size of one-hot vectors
Solution Approach 1:
The patent extracts only the non-zero elements and their indices from the one-hot vector, transmitting only this compressed representation to the storage device. This extraction principle reduces the data quantity from transmitting the entire one-hot vector to only the essential non-zero information, thereby resolving the bandwidth bottleneck while maintaining embedding operation capability
Solution Approach 2:
Instead of transmitting the one-hot vector directly from host to storage device, the patent inverts the approach by having the host generate vector data from compressed non-zero data and indices, then perform the embedding operation. This inversion shifts the computational burden and changes the data flow direction, eliminating the bandwidth bottleneck associated with direct one-hot vector transmission
Data Source
AI summary
Methods, systems, and devices for alleviating a bandwidth bottleneck during an embedding operation are described. An example storage device, based on the disclosed technology, includes a memory device configured to store matrix data, a memory controller, coupled to the memory device, configured to receive, from a host, non-zero data and the index of the non-zero data, and generate vector data based on the non-zero data and the index, and an operating component, coupled to the memory device and the memory controller, configured to perform a multiplication operation between the matrix data and the vector data.