Dedicated backend code generation method and device supporting machine learning training

Abstract

The present invention provides a method and device for generating a special backend code supporting machine learning training, the method comprising: parsing and optimizing the first calculation graph to obtain a second calculation graph; distributing the at least one operator to an operator mapping module based on the operator type of the at least one operator, and outputting an operator call code corresponding to each operator; inputting the memory configuration information of the at least one operator into a memory management module, and outputting a memory management code; generating a target backend code based on the operator call code corresponding to each operator and the memory management code, and compiling the target backend code based on an operator library to obtain a deployment file. The present invention implements an end-to-end compiler that supports neural network training, improves the computing efficiency of GPUs used in neural network training and reasoning, and expands the application scope of GPUs.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and device for generating dedicated back-end code to support machine learning training. Background Art

[0002] Graphics Processing Unit (GPU) is a traditional neural network training backend device widely used in the field of machine learning. It integrates a large number of Single Instruction Multiple Data (SIMD) microprocessors, provides huge parallel computing capabilities, and achieves good results in neural network training. However, due to the hardware architecture of the GPU itself, there is still a lot of room for improvement in complex logic control and input-output (IO) intensive computing.

[0003] Currently, GPUs, as chips dedicated to neural network training and reasoning, lack an end-to-end compiler that supports training and reasoning, such as the Kunlun chip, which means that the computing efficiency of GPUs used in neural network training and reasoning needs to be optimized urgently. Summary of the invention

[0004] The present invention provides a method and device for generating a dedicated back-end code supporting machine learning training, so as to solve the problem that the computing efficiency of GPU applied in neural network training and reasoning needs to be optimized.

[0005] In a first aspect, the present invention provides a method for generating a dedicated backend code supporting machine learning training, comprising:

[0006] Obtaining a first computation graph, parsing and optimizing the first computation graph to obtain a second computation graph, wherein the first computation graph is a neural network model composed of at least one operator in a first order;

[0007] Obtain at least one operator of the second computation graph, distribute the at least one operator to an operator mapping module based on the operator type of the at least one operator, and output an operator call code corresponding to each operator;

[0008] Obtaining memory configuration information of the at least one operator, inputting the memory configuration information of the at least one operator into a memory management module, and outputting a memory management code;

[0009] A target backend code is generated based on the operator calling code corresponding to each operator and the memory management code, and the target backend code is compiled based on the operator library to obtain a deployment file, where the deployment file is used to execute the computing task of the first computing graph.

[0010] Optionally, the operator mapping module includes at least one operator code generator and an operator mapper;

[0011] The distributing the at least one operator to an operator mapping module based on the operator type of the at least one operator and outputting an operator call code corresponding to each operator specifically includes:

[0012] Acquire an operator type of at least one operator based on the operator mapper;

[0013] Distributing the at least one operator to an operator code generator corresponding to each operator based on an operator type of the at least one operator;

[0014] Based on the at least one operator and the code generation template class in the operator code generator, an operator calling code corresponding to each operator is output.

[0015] Optionally, before distributing the at least one operator to the operator mapping module based on the operator type of the at least one operator, the method further includes:

[0016] Obtain a first operator, where the first operator is an operator in the second computation graph;

[0017] Generate a code generation template class corresponding to the first operator based on the operator type of the first operator and a preset code generation template base class;

[0018] An operator code generator corresponding to the first operator is generated and registered based on the code generation template class corresponding to the first operator.

[0019] Optionally, the operator calling code includes at least one of an operator function, an input parameter, and a function calling code;

[0020] The input parameters include at least one of the following:

[0021] The first tensor data passed into the function as a pointer;

[0022] A second tensor data obtained from the at least one operator based on an AOT compilation method;

[0023] An integer vector, wherein the integer vector is obtained based on a vector group mechanism.

[0024] Optionally, the integer vector is obtained based on a vector group mechanism, specifically including:

[0025] Inputting a second operator into an operator code generator corresponding to the second operator, and outputting an integer vector, wherein the second operator is an operator in the second computation graph whose input parameter includes the integer vector;

[0026] Sending the integer vector to a vector manager, generating a variable name of the integer vector based on the vector manager, and sending the variable name of the integer vector to an operator code generator corresponding to the second operator;

[0027] An operator code generator corresponding to the second operator outputs the variable name to an operator calling code corresponding to the second operator.

[0028] Optionally, the memory management code is used to:

[0029] Acquire a memory pool based on memory configuration information of the at least one operator;

[0030] An input / output memory request of the at least one operator is obtained, and the memory pool is allocated based on the input / output memory request of the at least one operator.

[0031] Optionally, the memory management code is further used to:

[0032] Obtaining memory information corresponding to an input and output memory request of the at least one operator;

[0033] A weight parameter corresponding to the at least one operator is acquired based on the memory information, and the weight parameter is updated based on a weight parameter update mechanism.

[0034] Optionally, the compiling the target backend code based on the operator library specifically includes:

[0035] Obtaining a preset subtask division method corresponding to an operator type of a third operator in an operator library, wherein the third operator is a supplementary operator;

[0036] Performing subtask division processing on the operator call code corresponding to the third operator based on the preset subtask division method to obtain at least one subtask;

[0037] Calculating a thread number of each subtask based on the at least one subtask, and executing the at least one subtask in parallel based on the thread number of each subtask;

[0038] The target backend code includes an operator calling code corresponding to the third operator.

[0039] In a second aspect, the present invention provides a dedicated backend code generation device supporting machine learning training, comprising:

[0040] A processing unit, configured to obtain a first computation graph, parse and optimize the first computation graph to obtain a second computation graph, wherein the first computation graph is a neural network model composed of at least one operator in a first order;

[0041] An operator call code generation unit, used to obtain at least one operator of the second computation graph, distribute the at least one operator to an operator mapping module based on the operator type of the at least one operator, and output an operator call code corresponding to each operator;

[0042] A memory management code generation unit, configured to obtain memory configuration information of the at least one operator, input the memory configuration information of the at least one operator into a memory management module, and output a memory management code;

[0043] A compilation unit is used to generate a target backend code based on the operator call code corresponding to each operator and the memory management code, and compile the target backend code based on the operator library to obtain a deployment file, and the deployment file is used to execute the computing task of the first computing graph.

[0044] In a third aspect, the present invention further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the dedicated back-end code generation method for supporting machine learning training as described in the first aspect are implemented.

[0045] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a method for generating a dedicated back-end code to support machine learning training as described in the first aspect.

[0046] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the dedicated back-end code generation method for supporting machine learning training as described in the first aspect.

[0047] The dedicated back-end code generation method and device for supporting machine learning training provided by the present invention obtain a second calculation graph by parsing and optimizing the first calculation graph, thereby improving the calculation efficiency of the calculation graph, generating a target back-end code based on the operator call code corresponding to each operator generated by the operator mapping module and the memory management code generated by the memory management module, and then compiling the target back-end code based on the operator library to obtain a deployment file, which is used to execute the calculation task of the first calculation graph, realizes an end-to-end compiler that supports neural network training, improves the calculation efficiency of GPUs applied to neural network training and reasoning, and expands the application scope of GPUs. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 It is one of the flowcharts of the dedicated backend code generation method supporting machine learning training provided by the present invention;

[0050] Figure 2 It is a schematic diagram of the structure of the target backend code provided by the present invention;

[0051] Figure 3 This is the second flow chart of the dedicated backend code generation method supporting machine learning training provided by the present invention;

[0052] Figure 4 It is a schematic diagram of a flow chart of compiling the operator call code corresponding to the supplementary operator provided by the present invention;

[0053] Figure 5 It is a structural schematic diagram of a dedicated backend code generation device supporting machine learning training provided by the present invention;

[0054] Figure 6 This is a schematic structural diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] In order to solve the problem that the computing efficiency of GPU in neural network training and reasoning needs to be optimized, an embodiment of the present invention provides a dedicated backend code generation method that supports machine learning training. Figure 1 This is one of the flowcharts of the dedicated backend code generation method for supporting machine learning training provided by an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0057] Step 100: Obtain a first computation graph, analyze and optimize the first computation graph, and obtain a second computation graph.

[0058] It should be noted that in order to solve the problem that the computing efficiency of GPUs used in neural network training and reasoning needs to be optimized, an embodiment of the present invention provides a dedicated back-end code generation method that supports machine learning training, implements an end-to-end compiler that supports neural network training, and is applied to the GPU, thereby improving the computing efficiency of the GPUs used in neural network training and reasoning, and expanding the application scope of the GPU.

[0059] Among them, the operator represents an operation and calculation, such as convolution calculation or matrix product.

[0060] Optionally, the first computational graph is a neural network model composed of at least one operator in a first order.

[0061] It should be noted that the first computation graph is a graph-level structure that describes the entire computation task, and the computation task includes executing each operator in the first order of the first computation graph. The first order is an arbitrarily set topological order, and at least one operator can obtain a different computation graph by using a different topological order.

[0062] It can be understood that the operator provided by the embodiment of the present invention can support various general mathematical operations and prediction and training of multiple neural network models.

[0063] Optionally, the parsing and optimization process includes constant folding or operator fusion.

[0064] In one implementation, constant folding or operator fusion processing is performed on the first computation graph to obtain a second computation graph.

[0065] Step 101: Obtain at least one operator of the second computation graph, distribute the at least one operator to an operator mapping module based on the operator type of the at least one operator, and output an operator call code corresponding to each operator.

[0066] It should be noted that the dedicated back-end code generation method supporting machine learning training provided in an embodiment of the present invention is applied to a compilation system based on the TVM framework. The front end of the system is the open source neural network compilation framework TVM in the prior art. The back end of the system includes a code generation module for generating target back-end code. The entire system runs on an electronic device corresponding to the GPU.

[0067] Optionally, the code generation module includes a computation graph traversal module, an operator mapping module, a memory management module and a code output module.

[0068] Among them, the computation graph traversal module is used to traverse the second computation graph, obtain at least one operator of the second computation graph, and obtain memory configuration information of at least one operator.

[0069] The operator mapping module is used to generate an operator calling code corresponding to each operator based on at least one operator of the second computation graph, wherein the operator mapping module includes an operator mapper and a code generator.

[0070] The memory management module is used to generate a memory management code based on the memory configuration information of at least one operator of the second computation graph.

[0071] The code output module is used to generate and output the target backend code based on the operator call code and memory management code corresponding to each operator.

[0072] It should be noted that the operator mapping module provided in the embodiment of the present invention classifies at least one operator based on the operator type, simplifies the design and maintenance of the code generator, and adopts an Ahead Of Time (AOT) compilation method to traverse the calculation graph before generating the operator call code, and counts the input and output shapes and memory information of at least one operator in the calculation graph, thereby optimizing the computational overhead of the reshape operator. At the same time, for some fixed array parameters, vectors are directly composed as the input data of the operator.

[0073] Optionally, the operator type includes an Element-wise operator, a Reducing operator, and a Broadcasting operator.

[0074] Element-wise operators are used to perform the same operation on each position in the input tensor data, such as a unary operator (not or negative), a binary operator (addition, subtraction, multiplication, or division), or a ternary operator (if a; then b; else c).

[0075] The Reducing operator is used to solve the elements at specific positions of the input tensor data using a preset function and output the result.

[0076] The Broadcasting operator is used to rearrange the input tensor data and store it in the appropriate location of the output tensor data, such as broadcast or slice.

[0077] The operator calling code is used to execute the operator's computing tasks.

[0078] In one implementation, the second computational graph is traversed based on a computational graph traversal module to obtain at least one operator of the second computational graph, and at least one operator is distributed to an operator mapping module based on the operator type of the at least one operator, and an operator call code corresponding to each operator is output, wherein the operator call code is used to execute the computational task of the operator.

[0079] Step 102: Obtain memory configuration information of the at least one operator, input the memory configuration information of the at least one operator into a memory management module, and output a memory management code.

[0080] It can be understood that since the input data or output results of each operator are stored in the memory of the chip, there is a large amount of data transmission overhead between the host and the chip during the execution of the computing tasks of the calculation graph. Therefore, the embodiment of the present invention also provides a memory management module for generating memory management code based on the memory configuration information of at least one operator, thereby realizing a complete and efficient memory management mechanism, which greatly reduces the overhead of memory allocation, release and data transmission based on the memory pool and the newly extended operators.

[0081] Optionally, the memory configuration information is used to record input data memory information and output data memory information of at least one operator.

[0082] The input data of the operator is the variable node, and the output result of the operator is the intermediate result. For example, the weight parameter can be both the input data of the operator and the output result of the operator.

[0083] The memory management code is used to implement memory allocation, memory copying or memory release of input data memory information and output result memory information of at least one operator.

[0084] In one implementation, the second computational graph is input into a computational graph traversal module, and the second computational graph is traversed based on the computational graph traversal module to obtain input data memory information and output data memory information of at least one operator, and the input data memory information and the output data memory information are recorded into memory configuration information, and the memory configuration information is input into a memory management module, and a memory management code is output.

[0085] Step 103: Generate a target backend code based on the operator calling code corresponding to each operator and the memory management code, compile the target backend code based on the operator library, and obtain a deployment file, which is used to execute the computing task of the first computing graph.

[0086] Optionally, the target backend code includes a computation graph main function, at least one operator function, at least one global variable, a header file declaration, and a wrapper function. For example, the target backend code can be implemented based on the C++ language.

[0087] Among them, the calculation graph main function is used to instruct the second calculation graph to execute the calculation task according to the first order. The calculation graph main function includes the input and output memory request of at least one operator and the execution process of the second calculation graph, specifically including memory allocation, memory copy, input parameters, function call code and memory release.

[0088] It can be understood that the memory management code includes memory allocation, memory copy and memory release, all of which are generated by the memory management module; the operator call code includes input parameters, function call code and operator function, all of which are generated by the operator mapping module.

[0089] The memory copy includes executing a memory copy from the host to the chip and executing a memory copy from the chip to the host.

[0090] Global variables are mainly used in the main function of the computational graph to control the operating logic of the weight parameter update mechanism and to assist the memory management module in memory allocation, memory copying, and memory release.

[0091] The operator function is used to be called in sequence by the main function of the calculation graph based on the function interface of each type of operator, and executes the calculation task of the second calculation graph through the corresponding operation logic.

[0092] The header file and the wrapper function provide underlying support, where the wrapper function can call the main function of the computation graph to provide underlying support.

[0093] Optionally, the operator library includes a compilation method for at least one operator of the second computation graph.

[0094] It can be understood that the embodiment of the present invention obtains the target backend code by combining the operator calling code corresponding to each operator with the memory management code. After the operator library compiles and links the target backend code, a deployment file is obtained. The deployment file can be run directly on the chip to execute the computing task of the first computing graph.

[0095] The dedicated back-end code generation method supporting machine learning training provided by the embodiment of the present invention obtains a second calculation graph by parsing and optimizing the first calculation graph, thereby improving the calculation efficiency of the calculation graph, generating a target back-end code based on the operator call code corresponding to each operator generated by the operator mapping module and the memory management code generated by the memory management module, and then compiling the target back-end code based on the operator library to obtain a deployment file, which is used to execute the calculation task of the first calculation graph, realizes an end-to-end compiler that supports neural network training, improves the calculation efficiency of GPUs applied to neural network training and reasoning, and expands the application scope of GPUs.

[0096] Figure 2 Schematic diagram of the structure of the target backend code provided by the embodiment of the present invention. Figure 2As shown, the target backend code includes header file declaration, global variables, operator functions, calculation graph main function and wrapper function, among which the calculation graph main function includes memory allocation, memory copy, input parameters, function call code and memory release. The memory copy is mainly divided into the host executing memory copy to the electronic device with GPU and the electronic device with GPU executing memory copy to the host.

[0097] Based on the content of the above embodiment, the operator mapping module includes at least one operator code generator and an operator mapper;

[0098] The distributing the at least one operator to an operator mapping module based on the operator type of the at least one operator and outputting an operator call code corresponding to each operator specifically includes:

[0099] Acquire an operator type of at least one operator based on the operator mapper;

[0100] Distributing the at least one operator to an operator code generator corresponding to each operator based on an operator type of the at least one operator;

[0101] Based on the at least one operator and the code generation template class in the operator code generator, an operator calling code corresponding to each operator is output.

[0102] It should be noted that the operator mapping module provided in the embodiment of the present invention includes at least one operator code generator and one operator mapper, wherein each operator code generator forms a corresponding relationship with each operator.

[0103] It can be understood that in order to solve the problem that the number of operator types in the calculation graph is huge, resulting in reduced design efficiency and high maintenance cost of the code generator corresponding to each operator, an embodiment of the present invention provides an operator mapper, which classifies operators according to the computing characteristics of the operators in the calculation graph, such as Element-wise operators, Reducing operators or Broadcasting operators. The operator mapper provided by the embodiment of the present invention can improve the design efficiency of the code generator and reduce its maintenance workload, and can also improve the scalability of the operator.

[0104] Optionally, the operator mapper is used to obtain the operator type of at least one operator of the second computation graph, and distribute the at least one operator to an operator code generator corresponding to each operator based on the operator type.

[0105] It can be understood that the operator code generator provided in the embodiment of the present invention includes a code generation template class corresponding to an operator, and the code generation template class generates an operator call code corresponding to the operator according to specific parameters of the operator.

[0106] The dedicated back-end code generation method supporting machine learning training provided in an embodiment of the present invention distributes at least one operator to an operator code generator corresponding to each operator based on an operator mapper, and generates an operator call code corresponding to each operator according to a code generation template class in the operator code generator, which can improve the design efficiency of the code generator and reduce its maintenance workload, and can also improve the scalability of the operator.

[0107] Based on the content of the above embodiment, before distributing the at least one operator to the operator mapping module based on the operator type of the at least one operator, the method further includes:

[0108] Obtain a first operator, where the first operator is an operator in the second computation graph;

[0109] Generate a code generation template class corresponding to the first operator based on the operator type of the first operator and a preset code generation template base class;

[0110] An operator code generator corresponding to the first operator is generated and registered based on the code generation template class corresponding to the first operator.

[0111] It should be noted that, when an operator is extended, if the operator mapping module does not include the code generator corresponding to the newly extended operator, the operator calling code corresponding to the newly extended operator cannot be generated through the existing operator mapping module. Therefore, an embodiment of the present invention also provides a method for defining a new operator code generator for the newly extended operator.

[0112] The first operator belongs to the newly extended operator in the second computation graph.

[0113] Among them, the operator mapping module includes a preset code generation template base class.

[0114] In one implementation, a first operator is obtained, a preset code generation template base class is expanded based on the operator type of the first operator, a code generation template class corresponding to the first operator is generated, an operator code generator is defined based on the code generation template class corresponding to the first operator through a general tool function or macro definition method, and the operator code generator is registered in an operator mapping module, wherein the operator code generator corresponds to the first operator.

[0115] The dedicated back-end code generation method supporting machine learning training provided by the embodiment of the present invention generates a code generation template class corresponding to the first operator according to the operator type of the first operator and a preset code generation template base class, and then registers the operator code generator based on the code generation template class, thereby expanding the code generator, improving the scalability of the operator, and further expanding the application scope of the GPU.

[0116] Figure 3This is the second flow chart of the dedicated backend code generation method supporting machine learning training provided by the embodiment of the present invention. Figure 3 As shown, the specific process of the method includes:

[0117] The computation graph is input into the computation graph traversal module, and at least one operator node of the computation graph is output. The operator mapper maps at least one operator to the operator code generator corresponding to each operator, and generates the operator call code corresponding to each operator based on the operator code generator, wherein the operator call code includes input parameters, operator functions and function call codes.

[0118] The computation graph is input into the computation graph traversal module, the memory information of variables and intermediate results is output, and then the memory management code is generated through the memory management module.

[0119] The operator calling code and memory management code corresponding to each operator are input into the code output module, and the target backend code is output.

[0120] Based on the content of the above embodiment, the operator calling code includes at least one of an operator function, an input parameter and a function calling code;

[0121] The input parameters include at least one of the following:

[0122] The first tensor data passed into the function as a pointer;

[0123] A second tensor data obtained from the at least one operator based on an AOT compilation method;

[0124] An integer vector, wherein the integer vector is obtained based on a vector group mechanism.

[0125] It can be understood that the operator mapping module generates an operator function corresponding to each operator type based on the operator type of at least one operator; for at least one operator of the second computational graph, the operator mapping module generates a function call code corresponding to each operator in the main function; for fixed array parameters that require additional definition, the operator mapping module generates input parameters in the main function.

[0126] Optionally, the second tensor data is a tensor size, typically represented by a single number.

[0127] It should be noted that the embodiment of the present invention adopts the AOT compilation method, obtains the second tensor data from at least one operator in the compilation stage, and inputs the second tensor into the function call code.

[0128] Based on the content of the above embodiment, the integer vector is obtained based on the vector group mechanism, specifically including:

[0129] Inputting a second operator into an operator code generator corresponding to the second operator, and outputting an integer vector, wherein the second operator is an operator in the second computation graph whose input parameter includes the integer vector;

[0130] Sending the integer vector to a vector manager, generating a variable name of the integer vector based on the vector manager, and sending the variable name of the integer vector to an operator code generator corresponding to the second operator;

[0131] An operator code generator corresponding to the second operator outputs the variable name to an operator calling code corresponding to the second operator.

[0132] It should be noted that the input parameters include an integer vector, which is obtained in the compilation stage and output to the function call code in the running stage. Therefore, the embodiment of the present invention provides a method of a vector group mechanism.

[0133] In one implementation, the vector group mechanism method provided by an embodiment of the present invention maintains a vector manager during the compilation phase. When the operator code generator corresponding to the second operator processes the second operator, if the input parameter of the second operator is an integer vector, the operator code generator generates an integer vector for the second operator, and the second operator sends the integer vector to the vector manager.

[0134] Furthermore, the vector manager stores information of the integer vector of the second operator, assigns a variable name of the integer vector to the integer vector, and returns the variable name to the operator code generator, which outputs an operator call code corresponding to the second operator based on the variable name of the integer vector.

[0135] In the process of the code output module outputting the operator calling code corresponding to the second operator, the information of the integer vector recorded in the vector manager is read, and the integer vector is output.

[0136] The dedicated back-end code generation method supporting machine learning training provided by an embodiment of the present invention stores the information of integer vectors based on a vector manager and generates variable names of integer vectors. The operator code generator outputs operator call codes based on the variable names of integer vectors, thereby realizing the functions of obtaining integer vectors in the compilation stage and using integer vectors in the running stage.

[0137] Based on the content of the above embodiment, the memory management code is used to:

[0138] Acquire a memory pool based on memory configuration information of the at least one operator;

[0139] An input / output memory request of the at least one operator is obtained, and the memory pool is allocated based on the input / output memory request of the at least one operator.

[0140] It should be noted that executing the computational tasks of the computational graph during the running phase will generate overhead within the computational graph, that is, the memory required to execute the computational graph. When the TVM framework and the chip operator library in the prior art are used to call the computational graph to execute the computational tasks, redundant data operations will be generated, thereby reducing computing efficiency. Therefore, the memory management code generated by the memory management module in the embodiment of the present invention can reduce redundant operations, improve the computing efficiency of executing the computational graph, and effectively reduce the overhead within the computational graph.

[0141] Optionally, the input / output memory request is a request for a memory size required for input data and output results.

[0142] Among them, during the running stage, the memory management code is used to obtain the total memory required for the input data and output results of at least one operator based on the memory configuration information of at least one operator, and apply to the chip for a memory pool corresponding to the total memory. When the input and output memory requests of at least one operator are obtained, the memory corresponding to the input and output memory requests of each operator is allocated from the memory pool.

[0143] The dedicated back-end code generation method supporting machine learning training provided in an embodiment of the present invention generates memory management code through a memory management module to implement memory allocation, which can reduce redundant operations, improve the computational efficiency of executing calculation graphs, and effectively reduce the overhead within the calculation graphs.

[0144] Based on the content of the above embodiment, the memory management code is also used for:

[0145] Obtaining memory information corresponding to an input and output memory request of the at least one operator;

[0146] A weight parameter corresponding to the at least one operator is acquired based on the memory information, and the weight parameter is updated based on a weight parameter update mechanism.

[0147] It should be noted that executing the computational tasks of the computational graph during the running phase will also generate inter-computational graph overhead, that is, the data overhead between iterative executions of the computational graphs. Therefore, the memory management code generated based on the memory management module in the embodiment of the present invention can update the weight parameters in the chip memory based on the weight update mechanism, thereby reducing the data communication overhead of copying from the chip to the host, and the data communication overhead of copying the updated weight parameters from the host to the chip, thereby improving the running efficiency of the computational graph.

[0148] It can be understood that in the process of executing neural network training (the second calculation graph), in order to improve the computational efficiency of training, it is necessary to determine the number of training iterations and the iteration interval of the output results. Therefore, a command operator is provided in the memory management code implemented in the embodiment of the present invention to control the execution logic of the neural network training and execute the weight parameter update mechanism to realize the update of the weight parameters, thereby effectively reducing the overhead of the entire training process.

[0149] Among them, the weight parameter update mechanism runs on the chip. The specific process is that the update operator obtains the memory information corresponding to the weight matrix to be updated, and generates a tensor program corresponding to the weight matrix. The weight parameters are updated based on the tensor program, and the updated weight parameters are stored in the memory corresponding to the weight matrix.

[0150] The dedicated back-end code generation method supporting machine learning training provided by the embodiment of the present invention generates a memory management code through a memory management module, which is used to update weight parameters and realize memory copying, thereby reducing the overhead between calculation graphs and improving the operation efficiency of the calculation graphs.

[0151] Based on the content of the above embodiment, the compiling process of the target backend code based on the operator library specifically includes:

[0152] Obtaining a preset subtask division method corresponding to an operator type of a third operator in an operator library, wherein the third operator is a supplementary operator;

[0153] Performing subtask division processing on the operator call code corresponding to the third operator based on the preset subtask division method to obtain at least one subtask;

[0154] Calculating a thread number of each subtask based on the at least one subtask, and executing the at least one subtask in parallel based on the thread number of each subtask;

[0155] The target backend code includes an operator calling code corresponding to the third operator.

[0156] It should be noted that the operator library on the chip includes the compilation method of the existing operators, and is unable to compile the newly extended operators (i.e., the supplementary operators). Therefore, the embodiment of the present invention provides a compilation method for the supplementary operators, thereby realizing the expansion of the operator library and improving the efficiency of the operator compilation process.

[0157] In one embodiment, a subtask division method is preset for each operator type. For example, if the operator type is an Element-wise type, the preset subtask division method is for each thread to process a continuous input data region; if the operator is a Rotate function, each thread processes at least one continuous matrix; if the operator is a group convolution function, each thread processes at least one continuous group.

[0158] It can be understood that the compilation method for the third operator provided in the embodiment of the present invention respectively implements the entry KERNEL function and the external interface function, divides the operator call code of the third operator into subtasks in the external interface function, obtains at least one subtask corresponding to the third operator, and sends the at least one subtask to the entry KERNEL function, calculates the thread number of each subtask in the entry KERNEL function, and executes at least one subtask corresponding to the third operator in parallel according to the thread number of each subtask, obtains and integrates at least one running result, and sends the integrated running result to the external interface function.

[0159] The dedicated back-end code generation method supporting machine learning training provided by an embodiment of the present invention performs subtask division processing on the operator call code corresponding to the third operator based on the preset subtask division method corresponding to the operator type to obtain at least one subtask, and executes at least one subtask in parallel based on the thread number of each subtask, thereby realizing the compilation processing of the third operator and the expansion of the operator library, thereby improving the efficiency of the operator compilation process.

[0160] Figure 4 FIG. 1 is a flow chart of compiling the operator call code corresponding to the supplementary operator provided by an embodiment of the present invention. Figure 4 As shown, the method comprises the following steps:

[0161] Step 400: The host obtains at least one supplementary operator and determines the operator type of the at least one supplementary operator;

[0162] Step 401: The host performs subtask division processing on the operator call code corresponding to each supplementary operator based on the preset subtask division method corresponding to the operator type of at least one supplementary operator, and obtains at least one subtask corresponding to each supplementary operator;

[0163] Step 402: The electronic device obtains at least one subtask corresponding to each supplementary operator, and calculates the thread number of each subtask;

[0164] Step 403: The electronic device executes at least one subtask corresponding to each supplementary operator in parallel based on the thread number of each subtask, and obtains and integrates calculation results;

[0165] Step 404: The host obtains the integrated calculation result sent by the electronic device.

[0166] The dedicated back-end code generation device supporting machine learning training provided by the present invention is described below. The dedicated back-end code generation device supporting machine learning training described below and the dedicated back-end code generation method supporting machine learning training described above can be referenced to each other.

[0167] Figure 5 Schematic diagram of the structure of a dedicated backend code generation device supporting machine learning training provided by an embodiment of the present invention. Figure 5 As shown, the dedicated backend code generation device supporting machine learning training includes: a processing unit 500, an operator call code generation unit 510, a memory management code generation unit 520 and a compilation unit 530, wherein:

[0168] The processing unit 500 is used to obtain a first calculation graph, analyze and optimize the first calculation graph to obtain a second calculation graph, where the first calculation graph is a neural network model composed of at least one operator in a first order;

[0169] An operator call code generation unit 510 is used to obtain at least one operator of the second computation graph, distribute the at least one operator to an operator mapping module based on the operator type of the at least one operator, and output an operator call code corresponding to each operator;

[0170] A memory management code generating unit 520 is used to obtain the memory configuration information of the at least one operator, input the memory configuration information of the at least one operator into a memory management module, and output a memory management code;

[0171] The compilation unit 530 is used to generate a target backend code based on the operator calling code corresponding to each operator and the memory management code, and compile the target backend code based on the operator library to obtain a deployment file, which is used to execute the computing task of the first computing graph.

[0172] The dedicated back-end code generation device supporting machine learning training provided by the embodiment of the present invention obtains a second calculation graph by parsing and optimizing the first calculation graph, thereby improving the calculation efficiency of the calculation graph, generating a target back-end code based on the operator call code corresponding to each operator generated by the operator mapping module and the memory management code generated by the memory management module, and then compiling the target back-end code based on the operator library to obtain a deployment file, which is used to execute the calculation task of the first calculation graph, realizes an end-to-end compiler that supports neural network training, improves the calculation efficiency of GPUs applied to neural network training and reasoning, and expands the application scope of GPUs.

[0173] Optionally, the operator mapping module includes at least one operator code generator and an operator mapper;

[0174] The distributing the at least one operator to an operator mapping module based on the operator type of the at least one operator and outputting an operator call code corresponding to each operator specifically includes:

[0175] Acquire an operator type of at least one operator based on the operator mapper;

[0176] Distributing the at least one operator to an operator code generator corresponding to each operator based on an operator type of the at least one operator;

[0177] Based on the at least one operator and the code generation template class in the operator code generator, an operator calling code corresponding to each operator is output.

[0178] Optionally, before distributing the at least one operator to the operator mapping module based on the operator type of the at least one operator, the method further includes:

[0179] Obtain a first operator, where the first operator is an operator in the second computation graph;

[0180] Generate a code generation template class corresponding to the first operator based on the operator type of the first operator and a preset code generation template base class;

[0181] An operator code generator corresponding to the first operator is generated and registered based on the code generation template class corresponding to the first operator.

[0182] The operator calling code includes at least one of an operator function, an input parameter, and a function calling code;

[0183] The input parameters include at least one of the following:

[0184] The first tensor data passed into the function as a pointer;

[0185] A second tensor data obtained from the at least one operator based on an AOT compilation method;

[0186] An integer vector, wherein the integer vector is obtained based on a vector group mechanism.

[0187] The integer vector is obtained based on a vector group mechanism, specifically including:

[0188] Inputting a second operator into an operator code generator corresponding to the second operator, and outputting an integer vector, wherein the second operator is an operator in the second computation graph whose input parameter includes the integer vector;

[0189] Sending the integer vector to a vector manager, generating a variable name of the integer vector based on the vector manager, and sending the variable name of the integer vector to an operator code generator corresponding to the second operator;

[0190] An operator code generator corresponding to the second operator outputs the variable name to an operator calling code corresponding to the second operator.

[0191] The memory management code is used to:

[0192] Acquire a memory pool based on memory configuration information of the at least one operator;

[0193] An input / output memory request of the at least one operator is obtained, and the memory pool is allocated based on the input / output memory request of the at least one operator.

[0194] The memory management code is also used to:

[0195] Obtaining memory information corresponding to an input and output memory request of the at least one operator;

[0196] A weight parameter corresponding to the at least one operator is acquired based on the memory information, and the weight parameter is updated based on a weight parameter update mechanism.

[0197] The compiling process of the target backend code based on the operator library specifically includes:

[0198] Obtaining a preset subtask division method corresponding to an operator type of a third operator in an operator library, wherein the third operator is a supplementary operator;

[0199] Performing subtask division processing on the operator call code corresponding to the third operator based on the preset subtask division method to obtain at least one subtask;

[0200] Calculating a thread number of each subtask based on the at least one subtask, and executing the at least one subtask in parallel based on the thread number of each subtask;

[0201] The target backend code includes an operator calling code corresponding to the third operator.

[0202] The dedicated backend code generation device supporting machine learning training provided by the present invention can achieve Figures 1 to 4 The various processes implemented by the method embodiment and achieving the same technical effect are not described here to avoid repetition.

[0203] Figure 6 A schematic diagram of the structure of an electronic device provided by the present invention, such as Figure 6As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute a dedicated backend code generation method supporting machine learning training, the method comprising:

[0204] Obtaining a first computation graph, parsing and optimizing the first computation graph to obtain a second computation graph, wherein the first computation graph is a neural network model composed of at least one operator in a first order;

[0205] Obtain at least one operator of the second computation graph, distribute the at least one operator to an operator mapping module based on the operator type of the at least one operator, and output an operator call code corresponding to each operator;

[0206] Obtaining memory configuration information of the at least one operator, inputting the memory configuration information of the at least one operator into a memory management module, and outputting a memory management code;

[0207] A target backend code is generated based on the operator calling code corresponding to each operator and the memory management code, and the target backend code is compiled based on the operator library to obtain a deployment file, where the deployment file is used to execute the computing task of the first computing graph.

[0208] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0209] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, when the program instructions are executed by a computer, the computer can execute the dedicated backend code generation method for supporting machine learning training provided by the above methods, the method comprising:

[0210] Obtaining a first computation graph, parsing and optimizing the first computation graph to obtain a second computation graph, wherein the first computation graph is a neural network model composed of at least one operator in a first order;

[0211] Obtain at least one operator of the second computation graph, distribute the at least one operator to an operator mapping module based on the operator type of the at least one operator, and output an operator call code corresponding to each operator;

[0212] Obtaining memory configuration information of the at least one operator, inputting the memory configuration information of the at least one operator into a memory management module, and outputting a memory management code;

[0213] A target backend code is generated based on the operator calling code corresponding to each operator and the memory management code, and the target backend code is compiled based on the operator library to obtain a deployment file, where the deployment file is used to execute the computing task of the first computing graph.

[0214] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the dedicated backend code generation method for supporting machine learning training provided in the above embodiments, the method comprising:

[0215] Obtaining a first computation graph, parsing and optimizing the first computation graph to obtain a second computation graph, wherein the first computation graph is a neural network model composed of at least one operator in a first order;

[0216] Obtain at least one operator of the second computation graph, distribute the at least one operator to an operator mapping module based on the operator type of the at least one operator, and output an operator call code corresponding to each operator;

[0217] Obtaining memory configuration information of the at least one operator, inputting the memory configuration information of the at least one operator into a memory management module, and outputting a memory management code;

[0218] A target backend code is generated based on the operator calling code corresponding to each operator and the memory management code, and the target backend code is compiled based on the operator library to obtain a deployment file, where the deployment file is used to execute the computing task of the first computing graph.

[0219] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0220] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0221] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating dedicated backend code to support machine learning training, characterized in that: include: Obtaining a first computation graph, parsing and optimizing the first computation graph to obtain a second computation graph, wherein the first computation graph is a neural network model composed of at least one operator in a first order; Obtain at least one operator of the second computation graph, distribute the at least one operator to an operator mapping module based on the operator type of the at least one operator, and output an operator call code corresponding to each operator; Obtaining memory configuration information of the at least one operator, inputting the memory configuration information of the at least one operator into a memory management module, and outputting a memory management code; Generate a target backend code based on the operator call code corresponding to each operator and the memory management code, compile the target backend code based on the operator library to obtain a deployment file, and the deployment file is used to execute the computing task of the first computing graph; The memory management code is used to: Acquire a memory pool based on memory configuration information of the at least one operator; Obtaining an input and output memory request of the at least one operator, and allocating the memory pool based on the input and output memory request of the at least one operator; The memory management code is also used to: Obtaining memory information corresponding to an input and output memory request of the at least one operator; A weight parameter corresponding to the at least one operator is acquired based on the memory information, and the weight parameter is updated based on a weight parameter update mechanism.

2. The method for generating a dedicated backend code supporting machine learning training according to claim 1, characterized in that: The operator mapping module includes at least one operator code generator and an operator mapper; The distributing the at least one operator to an operator mapping module based on the operator type of the at least one operator and outputting an operator call code corresponding to each operator specifically includes: Acquire an operator type of at least one operator based on the operator mapper; Distributing the at least one operator to an operator code generator corresponding to each operator based on an operator type of the at least one operator; Based on the at least one operator and the code generation template class in the operator code generator, an operator calling code corresponding to each operator is output.

3. The method for generating a dedicated backend code supporting machine learning training according to claim 2, characterized in that: Before distributing the at least one operator to the operator mapping module based on the operator type of the at least one operator, the method further includes: Obtain a first operator, where the first operator is an operator in the second computation graph; Generate a code generation template class corresponding to the first operator based on the operator type of the first operator and a preset code generation template base class; An operator code generator corresponding to the first operator is generated and registered based on the code generation template class corresponding to the first operator.

4. The method for generating a dedicated backend code supporting machine learning training according to claim 2 or 3, characterized in that: The operator calling code includes at least one of an operator function, an input parameter, and a function calling code; The input parameters include at least one of the following: The first tensor data passed into the function as a pointer; A second tensor data obtained from the at least one operator based on an AOT compilation method; An integer vector, wherein the integer vector is obtained based on a vector group mechanism.

5. The method for generating a dedicated backend code supporting machine learning training according to claim 4, characterized in that: The integer vector is obtained based on a vector group mechanism, specifically including: Inputting a second operator into an operator code generator corresponding to the second operator, and outputting an integer vector, wherein the second operator is an operator in the second computation graph whose input parameter includes the integer vector; Sending the integer vector to a vector manager, generating a variable name of the integer vector based on the vector manager, and sending the variable name of the integer vector to an operator code generator corresponding to the second operator; An operator code generator corresponding to the second operator outputs the variable name to an operator calling code corresponding to the second operator.

6. The method for generating a dedicated backend code supporting machine learning training according to claim 1, characterized in that: The compiling process of the target backend code based on the operator library specifically includes: Obtaining a preset subtask division method corresponding to an operator type of a third operator in an operator library, wherein the third operator is a supplementary operator; Performing subtask division processing on the operator call code corresponding to the third operator based on the preset subtask division method to obtain at least one subtask; Calculating a thread number of each subtask based on the at least one subtask, and executing the at least one subtask in parallel based on the thread number of each subtask; The target backend code includes an operator calling code corresponding to the third operator.

7. A dedicated backend code generation device supporting machine learning training, characterized in that: include: A processing unit, configured to obtain a first computation graph, parse and optimize the first computation graph to obtain a second computation graph, wherein the first computation graph is a neural network model composed of at least one operator in a first order; An operator call code generation unit, used to obtain at least one operator of the second computation graph, distribute the at least one operator to an operator mapping module based on the operator type of the at least one operator, and output an operator call code corresponding to each operator; A memory management code generation unit, configured to obtain memory configuration information of the at least one operator, input the memory configuration information of the at least one operator into a memory management module, and output a memory management code; A compiling unit, configured to generate a target backend code based on the operator calling code corresponding to each operator and the memory management code, and to compile the target backend code based on the operator library to obtain a deployment file, wherein the deployment file is used to execute the computing task of the first computing graph; The memory management code is used to: Acquire a memory pool based on memory configuration information of the at least one operator; Obtaining an input and output memory request of the at least one operator, and allocating the memory pool based on the input and output memory request of the at least one operator; The memory management code is also used to: Obtaining memory information corresponding to an input and output memory request of the at least one operator; A weight parameter corresponding to the at least one operator is acquired based on the memory information, and the weight parameter is updated based on a weight parameter update mechanism.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the dedicated back-end code generation method supporting machine learning training as described in any one of claims 1 to 6 are implemented.

Images