Method, device and electronic device for calling kernel function by Triton operator

By building a new Triton runtime structure, using LibEntry to reduce parameter processing time and optimize multi-core operator parameters through global tuning, the problem of high CPU time-consuming for Triton operators is solved, and significant time overhead reduction and computational efficiency improvement is achieved.

CN119311281BActive Publication Date: 2025-03-21BEIJING ACAD OF ARTIFICIAL INTELLLIGENCE
0 Cites 0 Cited by

Patent Information

Application Number
CN202411833387.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-12-13
Publication Date
2025-03-21
Estimated Expiration
2044-12-13

AI Technical Summary

Technical Problem

The Triton operator takes a lot of time on the CPU, especially when the tensor size is small. The main reason is that the time overhead of Triton runtime is high, and the non-essential behavior of JITFunction at runtime increases CPU time.

Method used

Build a new runtime structure, including the decorator JITFunction and LibEntry. LibEntry only captures some of the parameters passed by the user at runtime, reduces parameter processing time, and adjusts the parameters of multi-core operators through global tuning to optimize the compilation and calling process of kernel functions.

Benefits of technology

The end-to-end time overhead is reduced when kernel function calls, especially in small-scale function calls, the time overhead can be reduced by about 70%. At the same time, the parameter tuning space of multi-core operators is expanded, the freedom of configuration parameters is improved, and the computing efficiency is improved.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The present invention discloses a method, device and electronic device for calling a kernel function of a Triton operator, and relates to the field of artificial intelligence technology. The method comprises: constructing a runtime structure of a Triton operator, if the kernel function is called for the first time and is a single core, compiling the kernel function through JITFunction, if the kernel function is called for the first time and is multi-core, adjusting the overall compilation parameters to obtain the best global performance, if the kernel function is not called for the first time, obtaining the kernel function through a mapping table in LibEntry, wherein the mapping table contains compilation parameters and corresponding kernel functions or optimal compilation parameters and corresponding kernel functions. Practice shows that the solution provided by the present invention can reduce the end-to-end time overhead by about 70% in small-scale function calls. Moreover, the parameter tuning space of the multi-core operator is expanded by global tuning, the degree of freedom of configuration parameters is improved, and the computational efficiency of the reduction class operator is improved.
Need to check novelty before this filing date? Find Prior Art