Instruction execution method, related device and medium

By decoupling loading instructions from matrix multiplication and addition instructions in the graphics processor, asynchronous execution and data preloading are achieved, solving the problem of data loading latency in ultra-large tensor kernels and improving computational efficiency and throughput.

CN121879847APending Publication Date: 2026-04-17SUZHOU YIZHU INTELLIGENT TECH CO LTD
0 Cites 0 Cited by

Patent Information

Application Number
CN202610113456.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-01-27
Publication Date
2026-04-17

AI Technical Summary

Technical Problem

In existing technologies, data loading of ultra-large tensor kernels is bundled with matrix multiplication and addition operations, which causes data loading delays to be exposed on critical computation paths, limiting the effective utilization of high computing power and restricting the overall instruction throughput.

Method used

By decoupling the loading instruction from the matrix multiplication and addition instruction and supporting asynchronous execution, the matrix data is preloaded into a dedicated matrix data buffer by issuing independent loading instructions through the scheduling unit, and the matrix multiplication and addition operations are executed asynchronously, thus hiding the data loading delay.

Benefits of technology

It improves the computational efficiency and hardware resource utilization of the tensor processing unit, increases the overall instruction throughput, and effectively solves the data supply bottleneck problem.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention provides an instruction execution method, a related device and a medium, and the method comprises the steps: issuing an independent loading instruction through a scheduling unit according to the required quantity of shared memory resources, and loading matrix data required by multiplication and addition operation from a storage medium to a special matrix data cache region; the tensor processing unit is controlled to obtain matrix data from the special matrix data cache region and execute multiplication and addition operation according to a matrix multiplication and addition operation instruction issued by the scheduling unit; wherein the loading instruction and the matrix multiply-add operation instruction are controlled by the scheduling unit to be executed asynchronously and are mutually independent. According to the method, the loading instruction and the matrix multiply-add operation instruction are decoupled and asynchronous execution is supported, so that data loading delay is hidden, the calculation efficiency of the tensor processing unit and the hardware resource utilization rate are improved, and the overall instruction throughput rate is improved.
Need to check novelty before this filing date? Find Prior Art