Deep learning model optimization method, device and medium based on polyhedron compilation

Abstract

The present application discloses a method, device and medium for optimizing a deep learning model based on polyhedron compilation, which relates to the field of electrical data processing. The method includes: obtaining a deep learning model, parsing the deep learning model, and extracting a computational graph corresponding to the deep learning model; based on the computational graph, converting the computational problem in the deep learning model into a polyhedron model; wherein the operations in the deep learning model correspond to the vertices in the polyhedron model, and the data flows in the deep learning model correspond to the edges in the polyhedron model; and optimizing the deep learning model by compiling the polyhedron model. By converting the calculation of the deep learning model into a polyhedron problem and optimizing it using polyhedron compilation technology, an efficient execution plan can be generated on different hardware platforms, significantly improving the running speed and resource utilization of the model.

Description

Technical Field

[0001] The present application relates to the field of electrical data processing, and specifically to a method, device and medium for optimizing a deep learning model based on polyhedron compilation. Background Art

[0002] As technology develops, deep learning models are becoming increasingly important in various applications and scenarios. Deep learning models refer to machine learning models based on neural networks, especially those composed of multiple layers (i.e., deep networks). They process data in a hierarchical manner to automatically extract features and perform complex pattern recognition.

[0003] However, deep learning models often have high computational complexity, and execution efficiency has become a key factor limiting their application. For deep learning models, traditional compilers often find it difficult to fully utilize hardware resources when processing deep learning models, resulting in low computational efficiency. Summary of the invention

[0004] In order to solve the above problems, this application proposes a deep learning model optimization method based on polyhedron compilation, including:

[0005] Obtain a deep learning model, parse the deep learning model, and extract a computational graph corresponding to the deep learning model;

[0006] Based on the computational graph, converting the computational problem in the deep learning model into a polyhedron model; wherein the operations in the deep learning model correspond to the vertices in the polyhedron model, and the data flows in the deep learning model correspond to the edges in the polyhedron model;

[0007] The deep learning model is optimized by compiling the polyhedron model.

[0008] On the other hand, the present application also proposes a deep learning model optimization device based on polyhedron compilation, comprising:

[0009] at least one processor; and,

[0010] a memory communicatively connected to the at least one processor; wherein,

[0011] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the deep learning model optimization method based on polyhedron compilation as described in the above example.

[0012] On the other hand, the present application also proposes a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured as: the deep learning model optimization method based on polyhedron compilation described in the above example.

[0013] The deep learning model optimization method based on polyhedron compilation proposed in this application can bring the following beneficial effects:

[0014] By converting the calculation of deep learning models into polyhedral problems and optimizing them using polyhedral compilation technology, efficient execution plans can be generated on different hardware platforms, significantly improving the model's running speed and resource utilization.

[0015] The polyhedron model provides a formal method to describe the iteration space and data dependencies in deep learning computational tasks, which can help optimizers understand and transform computations to improve data locality and program parallelism, thereby improving efficiency.

[0016] Polyhedron compilation technology uses the optimization method of polyhedron theory to transform and reorganize the computational graph to generate an efficient execution plan. It is independent of specific hardware platforms and can be applied to various types of processors and accelerators, such as CPU, GPU, TPU, etc., to achieve better execution efficiency and resource utilization on various platforms. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 A schematic diagram of a process for optimizing a deep learning model based on polyhedron compilation in an embodiment of the present application;

[0019] Figure 2 A schematic diagram of the optimization process of polyhedron compilation in one case of an embodiment of the present application;

[0020] Figure 3 A schematic diagram of a genetic algorithm flow chart in one case in an embodiment of the present application;

[0021] Figure 4 Schematic diagram of a deep learning model optimization device based on polyhedron compilation in an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0023] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0024] like Figure 1 As shown, the embodiment of the present application provides a deep learning model optimization method based on polyhedron compilation, including:

[0025] S101: Obtain a deep learning model, parse the deep learning model, and extract a computational graph corresponding to the deep learning model.

[0026] You can choose the corresponding deep learning framework (for example, TensorFlow, PyTorch, etc.) to parse the deep learning model, convert the deep learning model into the corresponding intermediate representation (IR) and extract the computational graph corresponding to the deep learning model.

[0027] Among them, the computational graph is a graphical representation used to describe the computational process in a deep learning model. It decomposes the computational task into nodes (representing operations, such as addition, matrix multiplication, activation functions, etc.) and edges (representing data flow, representing the transmission process of operations).

[0028] S102: Based on the computational graph, convert the computational problem in the deep learning model into a polyhedron model; wherein the operations in the deep learning model correspond to the vertices in the polyhedron model, and the data flows in the deep learning model correspond to the edges in the polyhedron model.

[0029] Although the computational graph can describe the dependencies between deep learning operators, the computation of operators in deep learning is mostly performed in multiple ways. Therefore, the computational problems in the deep learning model are converted into polyhedron models here. The polyhedron problem is expressed in multi-dimensional space, which is more in line with the computational method of the deep learning model. The polyhedron model is used for problems such as the actual computational loop within the operator.

[0030] Specifically, based on the computational graph, deep learning operators in the deep learning model are determined, and computationally intensive operators in the deep learning operators, as well as matrix multiplication operations and sum-accumulation operations in the deep learning operators are determined.

[0031] Among them, deep learning operators can also be called deep learning tasks, deep learning calculations, etc., which usually include compute-intensive operators (also called compute-intensive tasks) and I / O-intensive operators (also called I / O-intensive tasks).

[0032] Compute-intensive operators refer to operations in deep learning models that require a lot of computing resources, such as convolution operations, matrix multiplication, etc. I / O-intensive operators refer to operators whose computing processes are mainly determined by the speed of input / output operations, such as reading data from a storage system and writing data to a disk.

[0033] Based on this, since the speed of I / O-intensive operators is limited by hardware performance and computation-intensive operators require a large amount of computing resources, compilation optimization is mainly performed on computation-intensive operators. Here, the computation-intensive operators are expressed in their form through Formula 1.

[0034] Formula 1; where A, B, and C represent different matrices required by the computationally intensive operator during the computation process. represents the element in the i-th row and j-th column of the matrix C, represents the element in the i-th row and k-th column of matrix A, Represents the element in the kth row and jth column of matrix B.

[0035] At this time, for computationally intensive operators, each matrix multiplication operation included therein is taken as an operation instance, and each operation instance is taken as a vertex in the polyhedral model. Based on sum-accumulation operations, edges are added between corresponding vertices.

[0036] Still taking Formula 1 as an example, each [i, j, k] instance can represent a vertex in the polyhedron model. Based on the sum-accumulation operation, the matrix multiplication operations that have dependencies on each other are determined, and edges are added between the vertices corresponding to the matrix multiplication operations that have dependencies.

[0037] Still taking Formula 1 as an example, since [i,j,k+1] depends on [i,j,k], there is an edge between the [i,j,k] instance and the [i,j,k+1] instance (due to the nature of the sum-accumulation operation), and since the addition depends on the last row, there is an edge between [n-1,j,k] and j.

[0038] S103: Optimizing the deep learning model by compiling the polyhedron model.

[0039] like Figure 2As shown in the figure, for the computationally intensive operators in the model, we first perform dependency analysis on the polyhedral model represented by the computationally intensive operators, that is, we analyze the data dependency and control dependency. The data dependency can be used to mine the locality of the data and carry out related performance optimization based on the locality of the data. The control dependency can be used to analyze the parallelism of the program.

[0040] Specifically, first, dependency analysis is performed based on the computational graph to determine the deep learning operators in the deep learning model, and the computationally intensive operators and loop structures in the deep learning operators are determined.

[0041] Among them, the deep learning operators and computationally intensive operators have been described above and will not be repeated here. The loop structure refers to the loop method of the computationally intensive operator during the calculation process.

[0042] At this time, based on the connection relationship in the polyhedron model, the data dependency relationship between the computationally intensive operators is obtained. The computation graph can usually reflect the data dependency relationship, and the polyhedron model obtained based on the computation graph can also obtain the data dependency relationship based on the connection relationship therein.

[0043] For control dependencies, the iteration domain of the corresponding dimension of the loop structure is established based on the parameter dimension included in the matrix multiplication operation. Taking Formula 1 as an example, the iteration domain [i, j, k] of the loop can be established, and based on the difference between each iteration domain and the access set, the corresponding dependency vector is obtained as the control dependency.

[0044] Furthermore, at this time, loop transformation can be performed on the loop structure based on the dependency analysis results. The dependency analysis results include data dependencies and control dependencies, and loop transformation includes one or a combination of multiple transformation methods among loop inversion, loop tilting, loop extension, loop merging, loop distribution, and loop offset. Loop reversal executes all iterations in a loop in reverse order to resolve backward iteration dependencies in the program. In loop transformation, loop reversal and loop tilting both adopt the method of unimodular transformation. Generally speaking, loop extension and loop merging have higher priority in loop transformation.

[0045] At this time, based on loop transformation, the order of the inner loop and the outer loop in the loop structure is exchanged to optimize performance for data locality, laying the foundation for the subsequent full exploitation of the locality of the program.

[0046] Specifically, locality includes temporal locality and spatial locality. Spatial locality means that the program tends to access adjacent memory addresses. By adjusting the spatial position of the loop, the outer loop can be mostly loops with data dependencies, which can ensure that related data is stored continuously in the memory, so as to utilize the cache line prefetch mechanism to provide spatial locality.

[0047] As for the inner loop, it is first determined that there is no designated loop structure with dependencies in the loop structure, and based on the loop transformation, the designated loop structure is exchanged to the inner loop.

[0048] That is, if there are multiple nested loops, and some of them have no data dependencies (that is, they can execute independently), these non-dependent loops are adjusted to the inner loop.

[0049] The loop structure in the inner loop is executed in a vectorized manner to optimize performance for program parallelism. This can improve cache locality and make the optimization of the inner loop more efficient.

[0050] Specifically, for the loops that have been placed in the inner layer, vectorization technology is used to convert them into a form that can utilize the Single Instruction Multiple Data (SIMD) instruction set. Vectorization allows multiple data elements to be processed simultaneously, thereby significantly improving parallelism and computing speed.

[0051] By converting the calculation of deep learning models into polyhedral problems and optimizing them using polyhedral compilation technology, efficient execution plans can be generated on different hardware platforms, significantly improving the model's running speed and resource utilization.

[0052] The polyhedron model provides a formal method to describe the iteration space and data dependencies in deep learning computational tasks, which can help optimizers understand and transform computations to improve data locality and program parallelism, thereby improving efficiency.

[0053] Polyhedron compilation technology uses the optimization method of polyhedron theory to transform and reorganize the computational graph to generate an efficient execution plan. It is independent of specific hardware platforms and can be applied to various types of processors and accelerators, such as CPU, GPU, TPU, etc., to achieve better execution efficiency and resource utilization on various platforms.

[0054] In one embodiment, considering that there may be optimization conflicts between program parallelism and data locality in some cases, it is necessary to weigh the benefits and losses brought by the two at certain times.

[0055] Therefore, under the premise of considering program parallelism and data locality, the cost model is used to establish evaluation criteria for various strategies of polyhedral compilation, so as to determine the sequential relationship that is conducive to achieving loop blocking. By establishing a mapping between the sequential relationship of the original program loop and the new program sequential relationship, the blocking of the loop program is achieved.

[0056] Specifically, a cost model is established; wherein the expression of the cost model is shown in Formula 2.

[0057] Formula 2: s represents the source statement instance, t represents the sink statement instance, The function represents an affine transformation along one dimension The corresponding normal vector direction depends on the number of hyperplanes passed through. , Respectively represent the source point statement instance and the sink point statement instance after affine transformation The normal vector of .

[0058] Source statements refer to statements in a program that are responsible for generating or providing data. They are usually used to initialize data or obtain data from external input. In data flow analysis, source statements are usually the starting point of the data flow. Sink statements refer to statements in a program that are responsible for consuming or using data. They usually output data to the outside or use it for further calculations. In data flow analysis, sink statements are usually the end point of the data flow.

[0059] At this point, the loop transformation is searched and optimized until it meets the evaluation criteria established by the cost model, or the preset upper limit of optimization times is reached, completing the search and optimization of the loop transformation, thereby finding the optimal loop transformation method based on locality and parallelism.

[0060] Furthermore, if Figure 3 As shown, when searching and optimizing the loop transformation, a genetic algorithm can be used to search and optimize.

[0061] By defining the transformation modes included in the cyclic transformation, a set composed of various cyclic transformations is obtained, and a corresponding search space is formed according to the set.

[0062] At this time, the search space contains a variety of cyclic transformations. Each cyclic transformation is an individual in the search space. Different cyclic transformations contain different transformation methods, or even if the transformation methods are different, the specific values ​​contained are also different. For example, the parameters used in loop skewing or loop tiling are different. Therefore, at this time, for cyclic transformations that need to specify specific values, cyclic transformations corresponding to different values ​​are added to the search space to deal with the situation where the transformation methods are the same but the specific values ​​are different.

[0063] An initial population is created through initialization; wherein the initial population includes a plurality of individuals, each individual corresponds to a cyclic transformation, and the creation of the initial population can be obtained by selecting in the search space.

[0064] The cost model is used as a fitness function to evaluate the fitness of each individual in the initial population, that is, to evaluate the fitness score of each individual.

[0065] Select the corresponding individuals based on the fitness. If the fitness meets the expectation, output it. Otherwise, continue to reproduce through crossover and mutation to update the initial population to increase the diversity of the population, until the fitness reaches a preset ratio (for example, 0.7 times) of the preset upper limit of the cost model (also called the upper bound, which can be deduced theoretically or obtained through multiple experiments), or the number of iterations reaches a preset upper limit (for example, 10,000 times).

[0066] like Figure 4 As shown, the embodiment of the present application also provides a deep learning model optimization device based on polyhedron compilation, including:

[0067] at least one processor; and,

[0068] a memory communicatively connected to the at least one processor; wherein,

[0069] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the deep learning model optimization method based on polyhedron compilation as described in any of the above embodiments.

[0070] An embodiment of the present application also provides a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured to be: a deep learning model optimization method based on polyhedron compilation as described in any of the above embodiments.

[0071] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0072] The devices and media provided in the embodiments of the present application correspond one-to-one to the methods. Therefore, the devices and media also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0073] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0074] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0075] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0076] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0077] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0078] Memory may include non-permanent storage in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0079] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0080] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0081] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A deep learning model optimization method based on polyhedron compilation, characterized in that: include: Obtain a deep learning model, parse the deep learning model, and extract a computational graph corresponding to the deep learning model; Based on the computational graph, converting the computational problem in the deep learning model into a polyhedron model; wherein the operations in the deep learning model correspond to the vertices in the polyhedron model, and the data flows in the deep learning model correspond to the edges in the polyhedron model; Optimizing the deep learning model by compiling the polyhedron model; By compiling the polyhedron model, the deep learning model is optimized, specifically including: Based on the computation graph, determine the deep learning operators in the deep learning model, and determine the computationally intensive operators and loop structures in the deep learning operators, and the matrix multiplication operations in the computationally intensive operators; Based on the connection relationship in the polyhedron model, obtaining the data dependency relationship between the computationally intensive operators; Based on the parameter dimensions included in the matrix multiplication operation, establishing an iteration domain of the corresponding dimension of the loop structure; Based on the difference between each iteration domain and the access set, the corresponding dependency vector is obtained as the control dependency; The method further comprises: Based on the dependency analysis result, loop transformation is performed on the loop structure; wherein the dependency analysis result includes data dependency and control dependency, and the loop transformation includes one or a combination of multiple transformation modes among loop inversion, loop tilting, loop extension, loop merging, loop distribution, and loop shifting; Based on the loop transformation, the order of the inner loop and the outer loop in the loop structure is exchanged to optimize the performance for data locality; Based on the loop transformation, the order of the inner loop and the outer loop in the loop structure is exchanged, specifically including: Determining that there is no dependent specified loop structure in the loop structure; Based on the loop transformation, exchanging the specified loop structure to an inner loop; The loop structure in the inner loop is executed in a vectorized manner to optimize performance for program parallelism; The method further comprises: Establish a cost model; wherein the expression of the cost model is: , s represents a source statement instance, t represents a sink statement instance, The function represents an affine transformation along one dimension The corresponding normal vector direction depends on the number of hyperplanes passed through. , Respectively represent the source point statement instance and the sink point statement instance after affine transformation The normal vector of Searching and optimizing the loop transformation until it meets the evaluation criteria established by the cost model, or reaches a preset upper limit of the number of optimizations; Searching and optimizing the loop transformation until it meets the evaluation criteria established by the cost model, or reaches a preset upper limit of the number of optimizations, specifically including: By defining the transformation modes included in the cyclic transformation, a set composed of the cyclic transformations is obtained, and a corresponding search space is formed according to the set; wherein, for cyclic transformations that need to specify specific values, cyclic transformations corresponding to different values ​​are added to the search space; By initialization, an initial population is created; wherein the initial population includes a plurality of individuals, each individual corresponding to a cyclic transformation; Using the cost model as a fitness function to evaluate the fitness of each individual in the initial population; Based on the fitness, a corresponding part of individuals is selected, and reproduction is performed through crossover and mutation, so as to update the initial population, until the fitness reaches a preset proportion of the preset upper limit of the cost model, or the number of iterations reaches a preset upper limit.

2. The method according to claim 1, characterized in that Based on the computational graph, the computational problem in the deep learning model is converted into a polyhedron model, specifically including: Based on the computation graph, determine the deep learning operators in the deep learning model, and determine the computationally intensive operators in the deep learning operators, as well as the matrix multiplication operations and sum-accumulation operations in the computationally intensive operators; For the computationally intensive operator, each matrix multiplication operation included therein is taken as an operation instance, and each operation instance is taken as a vertex in a polyhedron model; Based on the sum-accumulation operation, edges are added between corresponding vertices.

3. The method according to claim 2, characterized in that Based on the sum-accumulation operation, edges are added between corresponding vertices, specifically including: Based on the sum-accumulation operation, matrix multiplication operations that have dependencies on each other are determined, and edges are added between vertices corresponding to the matrix multiplication operations that have dependencies.

4. A deep learning model optimization device based on polyhedron compilation, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the deep learning model optimization method based on polyhedron compilation as described in any one of claims 1 to 3.

5. A non-volatile computer storage medium storing computer executable instructions, characterized in that: The computer executable instructions are set to: the deep learning model optimization method based on polyhedron compilation as described in any one of claims 1 to 3.

Images