Accelerating inference performance on artificial intelligence accelerators
By categorizing and transforming operations in the computation graph to minimize preprocessing, the method optimizes the use of accelerators and CPUs, addressing inefficiencies in existing compilers and improving inference performance in AI models.
JP2026520238APending Publication Date: 2026-06-23INTERNATIONAL BUSINESS MACHINE CORPORATION
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- INTERNATIONAL BUSINESS MACHINE CORPORATION
- Filing Date
- 2024-04-24
- Publication Date
- 2026-06-23
Smart Images
Figure 2026520238000001_ABST
Abstract
A method for improving inference performance in an artificial intelligence model results in a reduction of preprocessing overhead. The method comprises a step of receiving a plurality of operations associated with the artificial intelligence model. A computation graph is generated for the artificial intelligence model. Each of the operations is classified into one of three categories, including: accelerator-specified operations; central processing unit (CPU)-specified operations; and undetermined operation-specified operations. An estimated processing time is determined for each operation. The operations are inserted into the computation graph. The computation graph is divided into subgraphs. Edges of the subgraphs where preprocessing steps will be performed are determined. Based on the condition of minimizing the number of preprocessing steps in the subgraphs, a transformation is applied to the subgraphs that converts the undetermined operation-specified operations into either accelerator-specified operations or CPU-specified operations.
Need to check novelty before this filing date? Find Prior Art