A standard system for decoupling a training framework from a just-in-time compiler and a working method thereof

Abstract

The application relates to a standard system for decoupling a training framework and an instant compiler and a working method thereof, the system comprising a standard interface layer for connecting the training framework and the instant compiling component, wherein a compiling standard interface and an executor standard interface are arranged in the standard interface layer; the method comprises the following steps: based on the compiling standard interface, developing a neural network executor, a neural network compiler, a compiling task data structure, a compiling error output data format and a compiling correct output data format respectively; registering a unique identifier of the neural network compiler to the training framework, and accessing the neural network compiler by the training framework through calling the compiling standard interface; and returning an execution result to the training framework by the neural network executor. Compared with the prior art, the application can enable the training framework to access the neural network compiler of different chip manufacturers through one-time adaptation, thereby effectively reducing the adaptation cost between the training framework and the neural network compiler and realizing the rapid access of the training framework and the compiling component.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of artificial intelligence algorithm application, in particular to a standard system for decoupling a training framework and an instant compiler and a working method thereof. BACKGROUND

[0002] In the digital economy era, there is an additional standard for measuring productivity, namely computing power. As a new productivity in the digital economy era, computing power is a solid foundation for the development of digital economy and plays an important role in promoting technological progress, promoting digital transformation of industry and supporting economic and social development. Computing power has become a new focus of global strategic competition and an important engine for national economic development. Computing power level is positively correlated with economic development level.

[0003] Computing power tasks are mainly divided into basic general computing and HPC high-performance computing (High-performance computing). HPC computing is further divided into three categories: scientific computing, engineering computing and intelligent computing (i.e. artificial intelligence AI computing). AI is the focus of current social development in all fields, and the application and landing of AI are being studied.

[0004] AI training framework is a standard interface, feature library and tool package for AI algorithm model design, training and verification, which integrates algorithm encapsulation, data calling and underlying computing resource usage. At present, various AI frameworks coexist at home and abroad. The development of training frameworks at home and abroad shows a trend of diversification and fierce competition. Internationally, TensorFlow and Pytorch occupy a dominant position, and their combination with Nvidia AI acceleration chips provides powerful computing performance and tool ecology for AI technology development and application. In China, domestic training frameworks are also developing rapidly, and domestic training frameworks such as MindSpore, PaddlePaddle, OneFlow and SenseParrots have emerged. Each domestic framework is constantly improving and innovating, in addition to fully adapting to Nvidia AI acceleration chips, it also actively adapts to various domestic AI acceleration chips to create a domestic AI training ecology; and provides more user-friendly, efficient and flexible functions to meet the growing needs of deep learning applications.

[0005] In addition to the basic single-operator execution capability, the training framework is also actively connected to the neural network compiler. The entire neural network is compiled and optimized by the neural network compiler, and the overall neural network execution after the compilation optimization is more optimized than the single-operator execution. In practical applications, the current training framework needs to be customized and adapted to the neural network compiler of each chip, or the neural network compiler of each chip manufacturer needs to be customized and adapted to different training frameworks. That is, the training framework and the neural network compiler are seriously coupled, the process of connecting the training framework to the neural network compiler of multiple chips is cumbersome, the adaptation cost between the training framework and the neural network compiler is high, and it is difficult to quickly connect the training framework and the compilation component. SUMMARY

[0006] The purpose of the present application is to overcome the defects of the prior art and provide a standard system for decoupling the training framework and the just-in-time compiler and a working method thereof, which can enable the training framework to access the neural network compiler of different chip manufacturers through one-time adaptation, thereby effectively reducing the adaptation cost between the training framework and the neural network compiler and realizing the rapid connection of the training framework and the compilation component.

[0007] The purpose of the present application can be achieved by the following technical solutions: a standard system for decoupling the training framework and the just-in-time compiler, comprising a standard interface layer for connecting the training framework and the just-in-time compilation component, wherein the standard interface layer is provided with a compilation standard interface and an executor standard interface, the compilation standard interface is used to provide neural network compilation functions for different training frameworks, and the executor standard interface is used to load the compilation results.

[0008] Further, the compilation standard interface comprises a compilation task standard interface, a compiler standard interface and a compilation output standard interface, and the compiler standard interface is connected with the compilation task standard interface and the compilation output standard interface respectively.

[0009] Further, the compiler standard interface is connected with the neural network compiler.

[0010] Further, the compilation output standard interface is connected with the executor standard interface.

[0011] Further, the executor standard interface is connected with the buffer through the neural network executor.

[0012] Further, the compilation standard interface and the executor standard interface are defined in the form of function declaration.

[0013] A working method of a standard system for decoupling the training framework and the just-in-time compiler, comprising the following steps:

[0014] S1, based on the compiled standard interface, a neural network executor, a neural network compiler, a compilation task data structure, a compilation error output data format, and a compilation correct output data format are respectively developed;

[0015] S2, the unique identifier of the neural network compiler is registered to the training framework, and the training framework accesses the neural network compiler by calling the compiled standard interface;

[0016] The neural network executor returns the execution result to the training framework.

[0017] Further, the neural network executor in step S1 is used to load the neural network and provide execution capability.

[0018] Further, the neural network compiler in step S1 is used to perform a compilation task, configure default compilation options, display compilation error information, and display compiler version information.

[0019] Further, the compilation task data structure in step S1 includes setting and obtaining compilation options and managing compilation input.

[0020] Compared with the prior art, the present application has the following advantages:

[0021] The present application designs a standard interface layer for connecting a training framework and a just-in-time compilation component, the standard interface layer is provided with a compiled standard interface and an executor standard interface, the compiled standard interface is used to provide neural network compilation functions for different training frameworks, and the executor standard interface is used to load compilation results, so that the corresponding relationship between different training frameworks and different neural network compilers is shielded through the standard compilation interface, the adaptation process of a traditional single training framework corresponding to a single neural network compiler can be converted into unified implementation independent of specific frameworks and chip architectures, and therefore the rapid access of the training framework and the compilation component is realized.

[0022] The present application defines a plurality of standard interfaces, including a compilation task standard interface, a compilation output standard interface, a compiler standard interface, and an executor standard interface, can effectively decouple the neural network compiler and the training framework, can provide neural network compilation capabilities for multiple training frameworks by developing a compiler, and can effectively reduce the adaptation cost between the training framework and the neural network compiler by adapting a training framework once to access neural network compilers of multiple chips. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 The working method flowchart of the standard system in the present application is shown in the figure;

[0024] Figure 2 The overall design architecture schematic diagram of the embodiment is shown in the figure;

[0025] Figure 3A standard interface schematic diagram in the embodiment;

[0026] Figure 4 A compiler and training framework adaptation flow schematic diagram in the embodiment. DETAILED DESCRIPTION

[0027] The application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] EMBODIMENT

[0029] A standard system for decoupling a training framework and a just-in-time compiler, comprising a standard interface layer for connecting the training framework and a just-in-time compilation component, a compilation standard interface and an executor standard interface are arranged in the standard interface layer, the compilation standard interface is used to provide a neural network compilation function for different training frameworks, and the executor standard interface is used to load a compilation result.

[0030] The working process of the above standard system is as shown in Figure 1 , which comprises:

[0031] S1, based on the compilation standard interface, a neural network executor, a neural network compiler, a compilation task data structure, a compilation error output data format, and a compilation correct output data format are respectively developed;

[0032] S2, a unique identifier of the neural network compiler is registered to the training framework, and the training framework accesses the neural network compiler by calling the compilation standard interface;

[0033] The neural network executor returns an execution result to the training framework.

[0034] The embodiment applies the above technical solution, and designs and defines a set of standard interface system for connecting the training framework and the just-in-time compilation component. A variety of training frameworks and just-in-time compilation components for various AI acceleration chips can be quickly accessed by a unified access standard, the training framework and the compilation component, which can shield the one-to-one correspondence relationship of different training frameworks and just-in-time compilation components for different types of AI acceleration chips, and convert the traditional single training framework embedded single just-in-time compilation component adaptation flow into a standardized implementation independent of specific frameworks and specific just-in-time compilation components.

[0035] The overall architecture is as shown in Figure 2 , the compilation standard interface and the execution standard interface are arranged in the standard interface layer, and for the standard interface, data structure definition is performed first, including a compilation task data structure, an error information data structure, a compilation result data structure, an abstract compiler data structure, an abstract executor, and a key-value cache;

[0036] Then, each standard interface is defined, as shown in Figure 3 , specifically:

[0037] Compile task standard interface: including ① setting compile options; ② getting compile options; ③ inputting source neural network data;

[0038] Compile output standard interface: according to the signal indication, if the compilation is successful, the normal output object containing the compilation result is output; if the compilation fails, the abnormal output object containing error information is output;

[0039] Executor standard interface: ① the executor loads the compilation result; ② the execution entry;

[0040] Cache: the cache component is a hash structure key-value cache, which uses a digest algorithm to digest the input information that needs to be compiled, and the digest is used as the key of the cache. The output generated by the just-in-time compiler is used as the value of the cache. The subsequent execution is obtained from the cache without repeated compilation;

[0041] Compiler standard interface: ① device information; ② unique identifier; ③ set default compile options; ④ execute compile task; ⑤ show compile error information; ⑥ show compiler version information.

[0042] Finally, as shown in Figure 4 , the adaptation of the compiler and the training framework is realized based on the compilation standard interface, and the following function development is completed:

[0043] Develop a neural network executor that is responsible for loading a neural network and providing execution capabilities;

[0044] Develop a compile task data structure, including setting and getting compile options, and managing compile input;

[0045] Develop the data format of compile error output and compile correct output;

[0046] Develop a compiler, including implementing functions such as executing a compile task, configuring default compile options, displaying compile error information, and displaying compiler version information;

[0047] The training framework developer registers the unique identifier of the neural network compiler in the training framework, so that the training framework can access the neural network compiler by calling the standard interface and use its compile, execute, and other functions.

[0048] As can be seen from the above, the present scheme defines the compilation standard interface in the form of function declaration: including the compile task standard interface, the compile output standard interface, the executor standard interface, and the compiler standard interface, which can effectively decouple the neural network compiler and the training framework. A compiler can provide neural network compilation capabilities for multiple training frameworks, and a training framework can access multiple chip neural network compilers with one adaptation.

[0049] The scheme can decouple the one-to-one correspondence between the training framework and the compiler based on the compiler and the training framework adaptation process of the standard interface system, and the neural network compiler of the chip manufacturer and multiple training framework manufacturers can access and adapt based on the unified standard interface, realizing one development supporting multiple ends.

[0050] Through the scheme, the correspondence between different training frameworks and neural network compilers can be shielded, the traditional single training framework adaptation process is converted into a unified implementation independent of the framework and the neural network compiler, the training framework can access the neural network compiler of different chip manufacturers through one adaptation, and at the same time, the chip manufacturer can develop a neural network compiler to support multiple training frameworks, thereby effectively reducing the adaptation cost between the training framework and the neural network compiler. It is helpful for compiler developers to focus more on the compilation optimization capability of the compiler, and for training framework manufacturers to focus more on the upper-layer training framework function development, which is helpful for building an artificial intelligence domestic software and hardware ecology.

Claims

1. A working method for a standard system that decouples a training framework from a just-in-time compiler, characterized in that, The standard system for decoupling the training framework and the just-in-time compiler includes a standard interface layer for connecting the training framework and the just-in-time compilation component. The standard interface layer is provided with a compilation standard interface and an executor standard interface. The compilation standard interface is used to provide neural network compilation functions for different training frameworks, and the executor standard interface is used to load the compilation results. The compilation standard interface includes a compilation task standard interface, a compiler standard interface, and a compilation output standard interface. The compiler standard interface is connected to the compilation task standard interface and the compilation output standard interface, respectively. The compiler standard interface is connected to the neural network compiler. The compilation output standard interface is connected to the executor standard interface. The executor standard interface is connected to the buffer through the neural network executor. The standard interface for compilation tasks includes functions for setting compilation options, obtaining compilation options, and inputting neural network data. The standard output interface of the compiler outputs a normal output object containing the compilation result if the compilation is successful, and an exception output object containing error information if the compilation fails, based on the signal indication. The executor standard interface is used to load compilation results and as the execution entry point; The cache uses a digest algorithm to digest the input information that needs to be compiled. The digest is used as the key of the cache, and the output generated by the just-in-time compiler is used as the value of the cache. Subsequent executions retrieve the output from the cache without recompiling. The compiler standard interface includes: device information; a unique identifier; and is used to set default compilation options, execute compilation tasks, display compilation error messages, and display compiler version information. The working method includes the following steps: S1. Based on the standard compilation interface, develop the neural network executor, neural network compiler, compilation task data structure, compilation error output data format, and compilation correct output data format respectively; S2. Register the unique identifier of the neural network compiler to the training framework. The training framework accesses the neural network compiler by calling the standard compilation interface. The neural network executor returns the execution results to the training framework.

2. The working method of a standard system for decoupling a training framework and a just-in-time compiler according to claim 1, characterized in that, Both the compiler standard interface and the executor standard interface are defined through function declarations.

3. The working method of the standard system of decoupling training framework and just-in-time compiler according to claim 1, characterized in that, In step S1, the neural network executor is responsible for loading the neural network and providing execution capabilities.

4. The working method of a standard system for decoupling a training framework and a just-in-time compiler according to claim 1, characterized in that, In step S1, the neural network compiler is used to perform compilation tasks, configure default compilation options, display compilation error information, and display compiler version information.

5. The working method of a standard system for decoupling a training framework and a just-in-time compiler according to claim 1, characterized in that, The compilation task data structure in step S1 includes setting and obtaining compilation options and managing compilation input.

Images