Automatic vector optimization method for width inconsistency of deep learning framework compiler

Abstract

The invention discloses an automatic vector optimization method for width inconsistency of a deep learning framework compiler, which is based on a heterogeneous platform, and comprises the following steps that: S1: a front end of the framework compiler identifies a sub-graph capable of carrying out vector optimization in a calculation graph, S2: a middle end of the framework compiler fuses operators in the sub-graph capable of carrying out vector optimization marked in the step S15, S3, the rear end of the framework compiler carries out vector optimization with inconsistent widths on the bottom-layer IR obtained in the step S2 according to vector widths of a control core and a calculation core of the heterogeneous many-core processor; and S4, a code generation module of the frame compilerconverts the underlying IR after vector optimization obtained in the step S32 into a high-level language code specified by a user, and generates a platform target code after vector optimization through a basic compiler. The instruction set parallel performance of the deep learning load is further mined, and the vectorization degree of the deep learning load is improved, so that the reasoning performance of the deep learning load on the heterogeneous many-core platform is improved.

Description

technical field [0001] The invention relates to an automatic vector optimization method for non-uniform width of a deep learning framework compiler, which belongs to the technical field of deep learning. Background technique [0002] Deep learning workloads need to be deployed on specific hardware to be fully effective. At present, developers in the field of deep learning have designed a variety of frameworks, such as Tensorflow, Caffe, etc., to complete the training and inference tasks of deep learning models. At the same time, hardware manufacturers have also launched a variety of hardware backends, such as GPU, FPGA, etc., to accelerate deep Learning model training and inference speed. A bridge is needed between the large number of different deep learning frameworks and the increasing number of hardware architectures. As a complete optimization toolchain, the Deep Learning Framework Compiler provides an end-to-end solution for deploying deep learning workloads of differ...

AI Technical Summary

Problems solved by technology

However, during the deployment of deep learning workloads, the framework compiler loses part of the calculation graph information while generating high-level language codes such as C++ and LLVM IR. Many code segments with optimization potential cannot be recognized by the basic compiler, causing deep learning problems. The load cannot give full play to the full performance of domestic many-core processors

Images