A high-level intermediate representation conversion method and related device for a deep learning compiler

Abstract

The present application discloses a high-level intermediate representation conversion method for a deep learning compiler, comprising: classifying the operator operations of multiple deep learning frameworks based on the functional information of the operator operations to obtain multiple operator operation categories; adding reverse operator operation categories to the multiple operator operation categories to obtain an initial operator library; performing attribute definition processing on each operator operation in the initial operator library to obtain a target operator library; performing intermediate representation conversion on the deep learning algorithm model based on the target operator library to obtain a high-level intermediate representation. This method unifies the operator operations of multiple deep learning frameworks into a unified operator library, improves the adaptability of supporting deep learning frameworks, and improves R&D efficiency. The present application also discloses a high-level intermediate representation conversion device, server, and computer-readable storage medium for a deep learning compiler, which have the above beneficial effects.

Description

Technical Field

[0001] The present application relates to the field of deep learning technology, and in particular to a high-level intermediate representation conversion method, a high-level intermediate representation conversion device, a server, and a computer-readable storage medium for a deep learning compiler. Background Art

[0002] The general design architecture of the current deep learning compiler mainly includes a compilation front-end and a compilation back-end. The deep learning AI algorithm model is converted into a multi-level IR (Intermediate Representation) in the deep learning compiler, where the high-level IR is in the compiler front-end and the low-level IR is in the compiler back-end. The compilation front-end is mainly responsible for hardware-independent processing. The front-end conversion layer will first convert the input of different frameworks into a unified format using a high-level intermediate representation (High-Level IR), perform hardware-independent optimization on it, and then send it to the back-end. The compilation back-end also has its own intermediate representation, called a low-level intermediate representation. Its main task is to convert the optimized high-level intermediate representation into a low-level intermediate representation, and then perform some specific optimizations for the output hardware, compile it to the corresponding hardware instructions, and finally call each hardware device to perform calculations.

[0003] In related technologies, each deep learning framework currently develops its own independent operator library. The classification of operators within each framework varies, including the category name and number of categories. Furthermore, an operator's category varies across frameworks, and the same operator name represents a different calculation method. For example, matmul is a math_OP in TensorFlow, which performs matrix multiplication. However, in other frameworks, it's a fusion OP (Operator), which can be parsed into operations such as dot and matrix multiplication depending on the matrix dimensions. However, when developing with different deep learning frameworks, it's necessary to re-examine the new framework's operator definitions and their meanings, reducing R&D efficiency.

[0004] Therefore, how to improve the support of deep learning compilers for different deep learning frameworks and improve the compilation effect is a key issue that technical personnel in this field are concerned about. Summary of the Invention

[0005] The purpose of this application is to provide a high-level intermediate representation conversion method, a high-level intermediate representation conversion device, a server and a computer-readable storage medium for a deep learning compiler to improve the efficiency and effect of high-level intermediate representation conversion.

[0006] To solve the above technical problems, this application provides a high-level intermediate representation conversion method for a deep learning compiler, including:

[0007] Classify the operator operations of multiple deep learning frameworks based on the functional information of the operator operations to obtain multiple operator operation categories;

[0008] Adding a reverse operator operation category to the multiple operator operation categories to obtain an initial operator library;

[0009] Performing attribute definition processing on each operator operation in the initial operator library to obtain a target operator library;

[0010] Based on the target operator library, the intermediate representation of the deep learning algorithm model is converted to obtain a high-level intermediate representation.

[0011] Optionally, operator operations of multiple deep learning frameworks are classified based on the functional information of the operator operations to obtain multiple operator operation categories, including:

[0012] Obtain the multiple deep learning frameworks from a preset path;

[0013] Performing operator operation analysis on the multiple deep learning frameworks to obtain operator operations corresponding to each deep learning framework;

[0014] All the operator operations are classified based on the function information of each operator operation to obtain the multiple operator operation categories.

[0015] Optionally, all the operator operations are classified based on the function information of each operator operation to obtain the multiple operator operation categories, including:

[0016] All the operator operations are classified based on element-by-element arithmetic tensor operations, general tensor comparison operations, tensor logical operations, conventional tensor arithmetic operations, tensor data movement operations, specific type fusion operations, traditional deep learning layers, distributed communication operations, quantization operations, data flow control operations, and user-defined operations to obtain the multiple operator operation categories.

[0017] Optionally, a reverse operator operation category is added to the multiple operator operation categories to obtain an initial operator library, including:

[0018] Analyze the deep learning training task and obtain multiple reverse operator operations;

[0019] using the plurality of reverse operator operations as the reverse operator operation category;

[0020] The reverse operator operation category is added to the multiple operator operation categories to obtain an initial operator library.

[0021] Optionally, attribute definition processing is performed on each operator operation in the initial operator library to obtain a target operator library, including:

[0022] The interface definition, parameter list setting and function implementation configuration are performed on the same operator operation in the initial operator library to obtain the target operator library.

[0023] Optionally, also include:

[0024] Graph optimization processing is performed based on the high-level intermediate representation to obtain a graph optimization result.

[0025] Optionally, also include:

[0026] Compilation backend processing is performed based on the graph optimization result to obtain a compilation result.

[0027] This application also provides a high-level intermediate representation conversion device for a deep learning compiler, comprising:

[0028] An operator operation classification module is used to classify the operator operations of multiple deep learning frameworks based on the functional information of the operator operations to obtain multiple operator operation categories;

[0029] A reverse operator adding module, configured to add a reverse operator operation category to the plurality of operator operation categories to obtain an initial operator library;

[0030] An operator attribute definition module, configured to perform attribute definition processing on each operator operation in the initial operator library to obtain a target operator library;

[0031] The model conversion module is used to convert the intermediate representation of the deep learning algorithm model based on the target operator library to obtain a high-level intermediate representation.

[0032] The present application also provides a server, comprising:

[0033] memory for storing computer programs;

[0034] The processor is configured to implement the steps of the high-level intermediate representation conversion method described in the above embodiment when executing the computer program.

[0035] The present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the high-level intermediate representation conversion method described in the above embodiment are implemented.

[0036] The present application provides a high-level intermediate representation conversion method for a deep learning compiler, comprising: classifying operator operations of multiple deep learning frameworks based on functional information of the operator operations to obtain multiple operator operation categories; adding reverse operator operation categories to the multiple operator operation categories to obtain an initial operator library; performing attribute definition processing on each operator operation in the initial operator library to obtain a target operator library; and performing intermediate representation conversion on a deep learning algorithm model based on the target operator library to obtain a high-level intermediate representation.

[0037] By classifying the operator operations of multiple deep learning frameworks, we obtain multiple operator operation categories, then add the operator operation categories corresponding to the reverse operator operations, and finally perform attribute definition processing to obtain the final target operator library. This achieves the unification of the operator operations of multiple deep learning frameworks into a unified operator library, improves the adaptability of support for deep learning frameworks, and improves R&D efficiency.

[0038] The present application also provides a high-level intermediate representation conversion device, server, and computer-readable storage medium for a deep learning compiler, which have the above beneficial effects and are not specifically limited here. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0040] Figure 1 A flowchart of a high-level intermediate representation conversion method for a deep learning compiler provided in an embodiment of the present application;

[0041] Figure 2 A schematic diagram of the structure of a high-level intermediate representation conversion device of a deep learning compiler provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] The core of this application is to provide a high-level intermediate representation conversion method, a high-level intermediate representation conversion device, a server and a computer-readable storage medium for a deep learning compiler to improve the efficiency and effect of high-level intermediate representation conversion.

[0043] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0044] In related technologies, each deep learning framework currently develops its own independent operator library. The classification of operators within each framework varies, including the category name and number of categories. Furthermore, an operator's category varies across frameworks, and the same operator name represents a different calculation method. For example, matmul is a math_OP in TensorFlow, which performs matrix multiplication. However, in other frameworks, it's a fusion OP that can be parsed into operations such as dot and matrix multiplication, depending on the matrix dimensions. However, when developing with different deep learning frameworks, it's necessary to re-examine the new framework's operator definitions and their meanings, reducing R&D efficiency.

[0045] Therefore, the present application provides a high-level intermediate representation conversion method for a deep learning compiler. By classifying the operator operations of multiple deep learning frameworks, multiple operator operation categories are obtained, and then the operator operation categories corresponding to the reverse operator operations are added. Finally, attribute definition processing is performed to obtain the final target operator library, thereby unifying the operator operations of multiple deep learning frameworks into a unified operator library, improving the adaptability of supporting deep learning frameworks, and improving R&D efficiency.

[0046] The following describes an embodiment of a high-level intermediate representation conversion method for a deep learning compiler provided by this application.

[0047] Please refer to Figure 1 , Figure 1 This is a flowchart of a high-level intermediate representation conversion method for a deep learning compiler provided in an embodiment of the present application.

[0048] In this embodiment, the method may include:

[0049] S101, classifying operator operations of multiple deep learning frameworks based on functional information of the operator operations to obtain multiple operator operation categories;

[0050] As can be seen, this step aims to classify the operator operations of multiple deep learning frameworks based on their functional information, thereby obtaining multiple operator operation categories. In other words, all operator operations corresponding to all existing deep learning frameworks are analyzed and classified to obtain the operator operations in all deep learning frameworks. Furthermore, all operator operations are classified to obtain multiple operator operation categories. Based on these operator operation categories, a unified operator operation can be abstracted.

[0051] Furthermore, this step may include:

[0052] Step 1: Obtain multiple deep learning frameworks from the preset path;

[0053] Step 2: Analyze the operator operations of multiple deep learning frameworks to obtain the operator operations corresponding to each deep learning framework;

[0054] Step 3: Classify all operator operations based on the functional information of each operator operation to obtain multiple operator operation categories.

[0055] As can be seen, this alternative solution primarily illustrates how to categorize operator operations. In this alternative solution, multiple deep learning frameworks are obtained from a preset path, and operator operations are analyzed for each of these frameworks to obtain the operator operations corresponding to each deep learning framework. Based on the functional information of each operator operation, all operator operations are categorized to obtain multiple operator operation categories.

[0056] Furthermore, step 3 in the previous optional solution may include:

[0057] All operator operations are classified based on element-wise arithmetic tensor operations, general tensor comparison operations, tensor logical operations, conventional tensor arithmetic operations, tensor data movement operations, specific type fusion operations, traditional deep learning layers, distributed communication operations, quantization operations, data flow control operations, and user-defined operations to obtain multiple operator operation categories.

[0058] As can be seen, this option mainly explains how to classify. In this option, all operator operations are classified based on element-wise arithmetic tensor operations, general tensor comparison operations, tensor logical operations, regular tensor arithmetic operations, tensor data movement operations, specific types of fusion operations, traditional deep learning layers, distributed communication operations, quantization operations, data flow control operations, and user-defined operations, resulting in multiple operator operation categories.

[0059] S102, adding a reverse operator operation category to multiple operator operation categories to obtain an initial operator library;

[0060] Building on S101, this step aims to add reverse operator operation categories to multiple operator operation categories to obtain an initial operator library. Specifically, reverse operator operations are added to the obtained multiple operator operation categories to form a reverse operator operation category. This supports the deep learning training process and expands the applicability of the deep learning compiler.

[0061] Furthermore, this step may include:

[0062] Step 1: Analyze the deep learning training task to obtain multiple reverse operator operations;

[0063] Step 2, multiple reverse operator operations are used as reverse operator operation categories;

[0064] Step 3: Add reverse operator operation categories to multiple operator operation categories to obtain an initial operator library.

[0065] As can be seen, this alternative solution mainly explains how to obtain an initial operator library. In this alternative solution, the deep learning training task is analyzed to obtain multiple reverse operator operations. These multiple reverse operator operations are used as reverse operator operation categories. The reverse operator operation category is added to the multiple operator operation categories to obtain the initial operator library.

[0066] S103, performing attribute definition processing on each operator operation in the initial operator library to obtain a target operator library;

[0067] Based on S102, this step aims to perform attribute definition processing on each operator operation in the initial operator library to obtain the target operator library. In other words, the attributes of each operator operation in the initial operator library are configured so that the operator operations in the initial operator library can be used.

[0068] The attribute definition operation may be any attribute definition operation provided by the prior art.

[0069] Furthermore, this step may include:

[0070] Perform interface definition, parameter list setting, and function implementation configuration for the same operator operation in the initial operator library to obtain the target operator library.

[0071] As can be seen, this option mainly explains how to configure. In this option, the interface definition, parameter list setting, and function implementation configuration are performed on the same operator operation in the initial operator library to obtain the target operator library.

[0072] S104: Perform intermediate representation conversion on the deep learning algorithm model based on the target operator library to obtain a high-level intermediate representation.

[0073] Based on S103, this step aims to convert the intermediate representation of the deep learning algorithm model based on the target operator library to obtain a high-level intermediate representation.

[0074] Based on obtaining the target operator library, this step can adopt any intermediate representation conversion operation provided by the existing technology.

[0075] In addition, this embodiment may further include:

[0076] Graph optimization is performed based on the high-level intermediate representation to obtain the graph optimization result.

[0077] It can be seen that this optional solution describes that graph optimization processing can also be performed based on the high-level intermediate representation to obtain a graph optimization result.

[0078] In addition, this embodiment may further include:

[0079] Perform compilation backend processing based on the graph optimization results to obtain the compilation results.

[0080] It can be seen that in this optional solution, compilation backend processing is performed based on the graph optimization result to obtain the compilation result.

[0081] In summary, this embodiment classifies the operator operations of multiple deep learning frameworks to obtain multiple operator operation categories, then adds the operator operation categories corresponding to the reverse operator operations, and finally performs attribute definition processing to obtain the final target operator library, thereby unifying the operator operations of multiple deep learning frameworks into a unified operator library, improving the adaptability of supporting deep learning frameworks, and improving R&D efficiency.

[0082] The following is a specific example to further illustrate the high-level intermediate representation conversion method of a deep learning compiler provided by this application.

[0083] In this embodiment, the method may include:

[0084] Step 1: First, develop a classification of high-level intermediate representation operators for the compiler. By analyzing and counting the support for OP operators in existing mainstream deep learning frameworks, we design operator categories for the high-level intermediate representation of deep learning compilers. Common OP operations in deep learning AI training tasks are classified according to their functions to achieve a unified mapping of operator categories in each deep learning framework to the deep learning compiler. These mainly include general element-wise arithmetic tensor operations, general tensor comparison operations, tensor logical operations, conventional tensor arithmetic operations, tensor data movement operations, specific type fusion operations, traditional deep learning layers, distributed communication operations, quantization operations, data flow control operations, and user-defined operations.

[0085] Step 2: Next, new OP definitions need to be added. By comparing the high-level intermediate representation operator library of mainstream deep learning compilers, such as the OP list supported in existing deep learning frameworks, the definitions of reverse OPs involved in deep learning training tasks and data loading-related OPs that need to be added to the high-level intermediate representation are proposed.

[0086] Step 3, finally, OP attribute definition. To address the problem of inconsistent interfaces, parameter lists, and functional implementations for the same OP operation in various deep learning frameworks, a unified operator interface design and operator classification are abstracted for the same OP, including operator categories, interface definitions, parameter lists, and functional implementations. This enables the compiler to meet the mapping requirements of different general deep learning frameworks to high-level intermediate representation operator libraries, thereby facilitating the unified scheduling of operator standard libraries to underlying operator libraries for AI devices.

[0087] Completing the above development steps means completing the high-level intermediate representation design of the deep learning compiler for AI training, standardizing the inconsistencies in operators across frameworks, lowering the user threshold, and meeting the mapping requirements of deep learning training applications.

[0088] It can be seen that this embodiment classifies the operator operations of multiple deep learning frameworks to obtain multiple operator operation categories, then adds the operator operation categories corresponding to the reverse operator operations, and finally performs attribute definition processing to obtain the final target operator library, thereby unifying the operator operations of multiple deep learning frameworks into a unified operator library, improving the adaptability of supporting deep learning frameworks, and improving R&D efficiency.

[0089] The following introduces the high-level intermediate representation conversion device of the deep learning compiler provided in an embodiment of the present application. The high-level intermediate representation conversion device of the deep learning compiler described below and the high-level intermediate representation conversion method of the deep learning compiler described above can be referenced to each other.

[0090] Please refer to Figure 2 , Figure 2 A schematic diagram of the structure of a high-level intermediate representation conversion device of a deep learning compiler provided in an embodiment of the present application.

[0091] In this embodiment, the device may include:

[0092] An operator operation classification module 100 is used to classify operator operations of multiple deep learning frameworks based on functional information of the operator operations to obtain multiple operator operation categories;

[0093] A reverse operator adding module 200 is used to add a reverse operator operation category to multiple operator operation categories to obtain an initial operator library;

[0094] The operator attribute definition module 300 is used to perform attribute definition processing on each operator operation in the initial operator library to obtain the target operator library;

[0095] The model conversion module 400 is used to convert the intermediate representation of the deep learning algorithm model based on the target operator library to obtain a high-level intermediate representation.

[0096] Optionally, the operator operation classification module 100 is specifically used to obtain multiple deep learning frameworks from a preset path; perform operator operation analysis on the multiple deep learning frameworks to obtain operator operations corresponding to each deep learning framework; and classify all operator operations based on the functional information of each operator operation to obtain multiple operator operation categories.

[0097] Optionally, the reverse operator adding module 200 is specifically used to analyze the deep learning training task to obtain multiple reverse operator operations; use the multiple reverse operator operations as reverse operator operation categories; add reverse operator operation categories to multiple operator operation categories to obtain an initial operator library.

[0098] Optionally, the operator attribute definition module 300 is specifically used to perform interface definition, parameter list setting, and function implementation configuration for the same operator operation in the initial operator library to obtain a target operator library.

[0099] The present application also provides a server, including:

[0100] memory for storing computer programs;

[0101] The processor is configured to implement the steps of the high-level intermediate representation conversion method described in the above embodiment when executing the computer program.

[0102] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the high-level intermediate representation conversion method as described in the above embodiment are implemented.

[0103] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0104] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0105] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0106] The above is a detailed introduction to the high-level intermediate representation conversion method, high-level intermediate representation conversion device, server and computer-readable storage medium of a deep learning compiler provided by this application. This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method of this application and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to this application, and these improvements and modifications also fall within the scope of protection of the claims of this application.

Claims

1. A high-level intermediate representation conversion method for a deep learning compiler, characterized in that: include: Classify the operator operations of multiple deep learning frameworks based on the functional information of the operator operations to obtain multiple operator operation categories; Adding a reverse operator operation category to the multiple operator operation categories to obtain an initial operator library; Performing attribute definition processing on each operator operation in the initial operator library to obtain a target operator library; Performing intermediate representation conversion on the deep learning algorithm model based on the target operator library to obtain a high-level intermediate representation; Adding a reverse operator operation category to the multiple operator operation categories to obtain an initial operator library, including: Analyze the deep learning training task and obtain multiple reverse operator operations; using the plurality of reverse operator operations as the reverse operator operation category; Adding the reverse operator operation category to the multiple operator operation categories to obtain an initial operator library; Perform attribute definition processing on each operator operation in the initial operator library to obtain a target operator library, including: A unified interface definition, parameter list setting, and function implementation configuration are performed on the same operator operation in the initial operator library to obtain the target operator library.

2. The high-level intermediate representation conversion method according to claim 1, characterized in that: Based on the functional information of operator operations, the operator operations of multiple deep learning frameworks are classified to obtain multiple operator operation categories, including: Obtain the multiple deep learning frameworks from a preset path; Performing operator operation analysis on the multiple deep learning frameworks to obtain operator operations corresponding to each deep learning framework; All the operator operations are classified based on the function information of each operator operation to obtain the multiple operator operation categories.

3. The high-level intermediate representation conversion method according to claim 2, characterized in that: All the operator operations are classified based on the functional information of each operator operation to obtain the multiple operator operation categories, including: All the operator operations are classified based on element-by-element arithmetic tensor operations, general tensor comparison operations, tensor logical operations, conventional tensor arithmetic operations, tensor data movement operations, specific type fusion operations, traditional deep learning layers, distributed communication operations, quantization operations, data flow control operations, and user-defined operations to obtain the multiple operator operation categories.

4. The high-level intermediate representation conversion method according to claim 1, characterized in that: Also includes: Graph optimization processing is performed based on the high-level intermediate representation to obtain a graph optimization result.

5. The high-level intermediate representation conversion method according to claim 4, characterized in that: Also includes: Compilation backend processing is performed based on the graph optimization result to obtain a compilation result.

6. A high-level intermediate representation conversion device for a deep learning compiler, characterized in that: include: An operator operation classification module is used to classify the operator operations of multiple deep learning frameworks based on the functional information of the operator operations to obtain multiple operator operation categories; A reverse operator adding module, configured to add a reverse operator operation category to the plurality of operator operation categories to obtain an initial operator library; An operator attribute definition module is used to perform attribute definition processing on each operator operation in the initial operator library to obtain a target operator library; A model conversion module, configured to convert the intermediate representation of the deep learning algorithm model based on the target operator library to obtain a high-level intermediate representation; The reverse operator adds a module, specifically used for: Analyze the deep learning training task and obtain multiple reverse operator operations; using the plurality of reverse operator operations as the reverse operator operation category; Adding the reverse operator operation category to the multiple operator operation categories to obtain an initial operator library; Operator attribute definition module, specifically used for: A unified interface definition, parameter list setting, and function implementation configuration are performed on the same operator operation in the initial operator library to obtain the target operator library.

7. A server, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the high-level intermediate representation conversion method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the high-level intermediate representation conversion method according to any one of claims 1 to 5.

Images