一种神经网络模型的嵌入层访问加速系统及方法
By working together with the processing unit, control layer and routing layer, the word embedding vectors of candidate nouns are predicted and read in advance, which solves the problem of low efficiency of high-frequency token access and cache prefetching in the traditional Embedding layer access architecture and improves the access speed of neural network models.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SUNMMIO SCIENCE & TECHNOLOGY (BEIJING) CO LTD
- Filing Date
- 2025-12-29
- Publication Date
- 2026-07-17
AI Technical Summary
Traditional embedding layer access architectures in neural network models suffer from PE queuing problems caused by high-frequency token access and low cache prefetching efficiency, and cannot adapt to dynamic loads and the co-occurrence patterns of semantically discontinuous natural language token sequences.
The system adopts an architecture in which the processing unit layer, control layer and routing layer work together. It predicts the candidate word for the next word by querying the word statistics table, and pre-reads the word embedding vector of the candidate word in the storage and saves it in the prefetch buffer. After the processing unit infers the next word, it directly reads it from the buffer, reducing direct access to the storage.
It improves the access speed of the embedding layer in the neural network model, reduces the waiting time of the processing unit, and improves memory access efficiency.
Smart Images
Figure CN121833779B_ABST