一种多级并行TopK选取方法、设备、介质及产品
By converting the raw data into equal-width unsigned integers and using hardware-accelerated histogram instructions for multi-level parallel histogram statistics and recall, the performance bottleneck of the SIMD architecture for TopK selection in large language models is solved, achieving efficient data filtering and improved computational efficiency.
CN122414293APending Publication Date: 2026-07-17SHANGHAI SUIYUAN TECH CO LTD
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANGHAI SUIYUAN TECH CO LTD
- Filing Date
- 2026-06-18
- Publication Date
- 2026-07-17
AI Technical Summary
Technical Problem
Existing technologies suffer from performance bottlenecks in SIMD architecture when selecting TopK in large language models, making it difficult to achieve efficient and stable parallel processing.
Method used
The original data is converted into standard unsigned integer data of equal bit width. Histogram statistics are performed using hardware-accelerated histogram instructions. The top K data are selected through multi-level parallel recall. By combining multi-stage iteration and parallel recall optimization, efficient TopK selection is achieved.
Benefits of technology
It significantly improves the computational efficiency of TopK selection for long sequences, solves the performance bottleneck in traditional methods, and achieves more efficient data filtering.
✦ Generated by Eureka AI based on patent content.
Smart Images
Figure CN122414293A_ABST
Abstract
本发明公开了一种多级并行TopK选取方法、设备、介质及产品,该方法包括：将原始数据统一转换为等位宽无符号整型标准数据；算法从最高位向最低位迭代，逐位段进行硬件加速直方图统计，并通过前缀和筛定第K大元素所在的目标分桶；若非桶内边界值，则通过多阶段并行召回方式，快速获取所有匹配该桶值的标准数据，形成新的待处理集合；若数据量仍大于阈值且未处理至最低位，则更新目标位置继续迭代；否则，从当前集合中精确确定第K大值，并据此从全部原始数据中筛选出不小于该值的候选数据集；对候选数据执行完整排序，输出有序的TopK结果，本发明实施例的技术方案能够显著提升海量长序列数据中TopK选取的计算效率。
Need to check novelty before this filing date? Find Prior Art