A sparse attention calculation method, device and medium for a GPU

By mapping the sparse attention method to a unit hypersphere and generating binary hash codes using a sphere hash function, combined with multi-probe bucket retrieval and a load-adaptive kernel, the problems of low parallelism and low hardware utilization of existing sparse attention methods on GPUs are solved, achieving efficient sparse attention computation.

CN121683877BActive Publication Date: 2026-06-02CENT SOUTH UNIV
2 Cites 0 Cited by

Patent Information

Application Number
CN202610187608.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-02-10
Publication Date
2026-06-02
Estimated Expiration
2046-02-10

AI Technical Summary

Technical Problem

Existing sparse attention methods on GPUs suffer from problems such as strong sequential dependence in the hash function learning process, low parallelism, long preprocessing time, linearly increasing computational complexity with sequence length in the online retrieval stage, and low hardware utilization and throughput loss due to unbalanced load in sparse attention computation.

Method used

A sparse attention computation method based on ball hash retrieval and load adaptive computation is adopted. By mapping the query vector and key vector to the unit hypersphere, generating binary hash codes using the ball hash function, and combining a multi-probe bucket retrieval mechanism and a load-adaptive sampling dense-dense matrix multiplication kernel, sparse attention computation is achieved.

Benefits of technology

It reduces preprocessing time by 1-2 orders of magnitude, reduces online retrieval complexity to O(1), significantly improves GPU throughput, solves thread dispersion and uneven load problems, and achieves efficient utilization of hardware computing resources.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The present application relates to the technical field of GPU computing optimization, and in particular to a sparse attention computing method, device and medium for GPU, wherein the method realizes the high efficiency of long context reasoning through the geometric perception sparse attention framework of ball hashing, and combines a large-scale parallel hashing optimization algorithm and a load adaptive computing kernel. Compared with the existing sparse attention methods based on heuristics or gradient learning, the present application realizes higher retrieval recall rate, lower preprocessing overhead and efficient hardware adaptation to irregular sparse patterns.
Need to check novelty before this filing date? Find Prior Art