Adjustment and optimization method and device based on sparse neural network, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an adjustment and optimization method and device based on a sparse neural network, electronic equipment and a storage medium. The adjustment and optimization method comprises the steps of obtaining the sparse neural network; determining the number of kernel functions corresponding to the sparse neural network according to the sparse neural network and a preset metadata division model; traversing each operator in the sparse neural network in sequence to obtain basic data of each operator, and dividing the sparse neural network according to the basic data to obtain metadata of each kernel function corresponding to the number of the kernel functions; according to the metadata of the kernel function and the preset scheduling strategy, the code file corresponding to the kernel function is determined, the problem that an existing operator fusion technology aiming at the sparse neural network is limited by manpower is solved, and meanwhile, the efficiency is improved. The problem that an unconventional sparse kernel function corresponding to a fusion operator in a sparse neural network model lacks in the aspect of automatic tuning is solved.

Description

Technical Field

[0001] This application relates to the technical field of graph neural networks. Specifically, it relates to a tuning method, device, electronic device, and storage medium based on a sparse neural network. Background Art

[0002] Dense operations such as matrix multiplication (GeMM) and convolution (Conv) commonly included in deep neural networks are typical computationally intensive tasks. However, with the development of deep learning technology in recent years, the role of memory access intensive sparse computing in deep learning has become increasingly prominent, giving rise to various sparse neural network models (sparse models). On the one hand, new progress in the field of deep learning shows that by sparsifying dense data (such as model weights, attention, etc.), a large reduction in floating-point computational volume can be achieved with a relatively small loss of model accuracy. On the other hand, graph neural networks that use sparse graph data as the learning object have emerged and demonstrated prominent advantages compared to classical graph computing methods, thus attracting extensive attention in the industry.

[0003] Specifically, although using kernel function auto-tuning technology alone can improve the computational efficiency of sparse operators in a sparse neural network model, sparse operators are only part of the computational tasks in the entire model, and the performance improvement of sparse kernel functions can only bring limited improvement to the performance of the entire model. Using operator fusion technology alone can effectively avoid redundant memory access and alleviate the bottleneck of memory access limitation in sparse models. However, the unconventional sparse kernel functions corresponding to fused operators rely on manual implementation, and it is difficult for developers to take into account auto-tuning while supporting a variety of operator fusion methods. Therefore, the sparse kernel functions corresponding to existing fused operators generally have poor performance. Summary of the Invention

[0004] Some embodiments of the present application aim to provide a tuning method, device, electronic device, and storage medium based on a sparse neural network. Through the technical solutions of the embodiments of the present application, a sparse neural network is obtained; according to the sparse neural network and a pre-set metadata partitioning model, the number of kernel functions corresponding to the sparse neural network is determined; each operator in the sparse neural network is traversed in sequence to obtain the basic data of each operator, and the basic data at least includes the operator type, input tensor, output tensor, and access mode; the sparse neural network is partitioned according to the basic data to obtain the metadata of each kernel function corresponding to the number of kernel functions; according to the metadata of the kernel function and a pre-set scheduling strategy, the code file corresponding to the kernel function is determined. In the embodiments of the present application, the number of kernel functions of the sparse neural network is determined through the metadata partitioning model, and the sparse neural network is partitioned to obtain the metadata corresponding to multiple kernel functions, and then according to the metadata and the scheduling strategy, the code file of the kernel function is automatically generated, solving the problem that the existing operator fusion technology for sparse neural networks is limited by manual work, and at the same time, solving the problem of the lack of automatic tuning for the unconventional sparse kernel functions corresponding to the fused operators in the sparse neural network model.

[0005] In a first aspect, some embodiments of the present application provide a tuning method based on a sparse neural network, including:

[0006] Obtain a sparse neural network;

[0007] According to the sparse neural network and a pre-set metadata partitioning model, determine the number of kernel functions corresponding to the sparse neural network;

[0008] Traverse each operator in the sparse neural network in sequence to obtain the basic data of each operator, and the basic data at least includes the operator type, input tensor, output tensor, and access mode;

[0009] Partition the sparse neural network according to the basic data to obtain the metadata of each kernel function corresponding to the number of kernel functions;

[0010] According to the metadata of the kernel function and a pre-set scheduling strategy, determine the code file corresponding to the kernel function.

[0011] Some embodiments of the present application determine the number of kernel functions of the sparse neural network through the metadata partitioning model, and partition the sparse neural network to obtain the metadata corresponding to multiple kernel functions, and then according to the metadata and the scheduling strategy, automatically generate the code file of the kernel function, solving the problem that the existing operator fusion technology for sparse neural networks is limited by manual work, and at the same time, solving the problem of the lack of automatic tuning for the unconventional sparse kernel functions corresponding to the fused operators in the sparse neural network model.

[0012] Optionally, traversing each operator in the sparse neural network in sequence to obtain the basic data of each operator includes:

[0013] Traversing from the input end of the sparse neural network, and determining the operator type of the operator according to the operator name;

[0014] Obtaining the input tensor of the operator and the access mode of the input tensor;

[0015] Determining the access mode of the output tensor according to the input tensor and the access mode of the input tensor;

[0016] Determining the traversal status of the operator according to the operator type, input tensor, output tensor, access mode of the input tensor, and access mode of the output tensor of each operator.

[0017] Optionally, partitioning the sparse neural network according to the basic data to obtain the metadata of each kernel function corresponding to the number of kernel functions, includes:

[0018] Judging each operator type according to the data dependence direction and the operator name, and the operator type at least includes a sparse operator and a dense operator;

[0019] When traversing to the first sparse operator, judging the type of the first sparse operator;

[0020] Determining the output tensor access mode corresponding to the sparse operator according to the type of the sparse operator;

[0021] If the traversed operator is a dense operator, the access modes of the input tensor and the output tensor of the dense operator are the same;

[0022] Partitioning the sparse neural network according to the access modes of the input tensor and the output tensor of the sparse operator, and the access modes of the input tensor and the output tensor of the dense operator, to obtain the metadata of each kernel function corresponding to the number of kernel functions.

[0023] In some embodiments of the present application, in order to ensure that the generated code of the embodiments of the present application has high performance and reduce the complexity of the decisions made during code generation, the first step in code generation in the embodiments of the present application is to partition the kernel functions to be generated from the computation graph provided by the user and collect relevant metadata for subsequent code generation and automatic tuning.

[0024] Optionally, determining the output tensor access mode corresponding to the sparse operator according to the type of the sparse operator, includes:

[0025] If the first sparse operator belongs to the g-SDDMM type, the output tensor access pattern is edge features;

[0026] If the first sparse operator belongs to the g-SpMM type, the output tensor access pattern is destination features.

[0027] Optionally, partitioning the sparse neural network according to the input tensor access pattern and output tensor access pattern of the sparse operator, and the input tensor access pattern and output tensor access pattern of the dense operator, to obtain metadata of each kernel function corresponding to the number of kernel functions, including:

[0028] When the operator is a sparse operator and the input tensor access pattern and output tensor access pattern of the sparse operator are different, partitioning the sparse neural network to obtain metadata of each kernel function corresponding to the number of kernel functions.

[0029] The purpose of partitioning the computational graph in some embodiments of the present application is to split the computations that cannot be independently parallelized into different kernel functions, ensuring to find a high-performance implementation method for the computational graph provided to the user.

[0030] Optionally, the metadata of the kernel function at least includes the mathematical meaning of the kernel function, and the mathematical meaning of the kernel function at least includes the input tensor, the output tensor, and the mathematical operation between the input tensor and the output tensor; the mathematical meaning of the kernel function is the names of the operators in the sub-computational graph corresponding to each kernel function and the data dependency relationship between them; the input tensor refers to the input tensor of the operator with the earliest topological order in the sub-computational graph corresponding to each kernel function, and the output tensor is the output tensor of the operator with the latest topological order.

[0031] Some embodiments of the present application split the sparse neural network to obtain metadata of each kernel function, and then generate corresponding code files according to the metadata, trying to represent the sparse neural network with the fewest kernel functions.

[0032] Optionally, the preset scheduling strategy at least includes one or more of: data mapping, thread mapping, caching method, reduction method, vectorization, and loop unrolling method.

[0033] Some embodiments of the present application generate a scheduling strategy by describing the parallelism and access pattern of the sparse kernel functions in the sparse neural network, and then generate code files for each kernel function of the sparse neural network.

[0034] In a second aspect, some embodiments of the present application provide a tuning device based on a sparse neural network, including:

[0035] An acquisition module, configured to acquire a sparse neural network;

[0036] A calculation module, configured to determine the number of kernel functions corresponding to the sparse neural network according to the sparse neural network and a preset metadata partitioning model;

[0037] A traversal module, configured to sequentially traverse each operator in the sparse neural network to obtain basic data of each operator, where the basic data at least includes operator type, input tensor, output tensor, and access mode;

[0038] A partitioning module, configured to partition the sparse neural network according to the basic data to obtain metadata of each kernel function corresponding to the number of kernel functions;

[0039] A generation module, configured to determine a code file corresponding to the kernel function according to the metadata of the kernel function and a preset scheduling strategy.

[0040] Some embodiments of the present application determine the number of kernel functions of a sparse neural network through a metadata partitioning model, partition the sparse neural network to obtain metadata corresponding to multiple kernel functions, and then automatically generate a code file of the kernel function according to the metadata and the scheduling strategy, which solves the problem that the existing operator fusion technology for sparse neural networks is limited by manual work. At the same time, it solves the problem of the lack of automatic tuning for unconventional sparse kernel functions corresponding to fused operators in the sparse neural network model.

[0041] In a third aspect, some embodiments of the present application provide an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where when the processor executes the program, the tuning method based on a sparse neural network described in any embodiment of the first aspect can be implemented.

[0042] In a fourth aspect, some embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the tuning method based on a sparse neural network described in any embodiment of the first aspect can be implemented.

[0043] In a fifth aspect, some embodiments of the present application provide a computer program product, where the computer program product includes a computer program, and when the computer program is executed by a processor, the tuning method based on a sparse neural network described in any embodiment of the first aspect can be implemented. Description of the Drawings

[0044] To more clearly illustrate the technical solutions of some embodiments of the present application, the following will briefly introduce the accompanying drawings required for use in some embodiments of the present application. It should be understood that the following drawings only show certain embodiments of the present application and should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0045] Figure 1 Schematic flowchart of a tuning method based on a sparse neural network provided by an embodiment of the present application;

[0046] Figure 2 Schematic diagram of kernel function splitting of a sparse neural network provided by an embodiment of the present application;

[0047] Figure 3 Schematic diagram of a metadata partitioning model provided by an embodiment of the present application;

[0048] Figure 4 Schematic diagram of a scheduling strategy provided by an embodiment of the present application;

[0049] Figure 5 Schematic structural diagram of a tuning device based on a sparse neural network provided by an embodiment of the present application;

[0050] Figure 6 Schematic diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0051] The following will describe the technical solutions in some embodiments of the present application in conjunction with the accompanying drawings in some embodiments of the present application.

[0052] It should be noted that: Similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of the present application, terms such as "first", "second", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0053] The inventors of the present application believe that only by most effectively combining the operator fusion technology for sparse neural networks and the automatic tuning of kernel functions can the sparse model be most comprehensively optimized. The biggest difficulty in combining these two types of technologies is the lack of self-tuning technology for the kernel functions of fused operators. Therefore, the inventors of the present application deeply analyzed the common computational patterns in sparse neural networks and found that the optimization technologies that can be adopted therein are basically the same as the computational patterns of classical sparse kernel functions. The difficulty in automatically tuning the kernel functions of fused operators lies in how to generate code for composite kernel functions that fuse multiple sparse operators and dense operators and how to effectively and automatically tune composite kernel functions with a significant change in computational complexity (compared with classical sparse kernel functions).

[0054] In view of this, some embodiments of the present application provide a tuning method based on a sparse neural network. The method includes obtaining a sparse neural network; determining the number of kernel functions corresponding to the sparse neural network according to the sparse neural network and a pre-set metadata partitioning model; traversing each operator in the sparse neural network in turn to obtain the basic data of each operator, where the basic data includes at least the operator type, input tensor, output tensor, and access pattern; partitioning the sparse neural network according to the basic data to obtain the metadata of each kernel function corresponding to the number of kernel functions; and determining the code file corresponding to the kernel function according to the metadata of the kernel function and a pre-set scheduling strategy. In the embodiments of the present application, the number of kernel functions of the sparse neural network is determined by the metadata partitioning model, and the sparse neural network is partitioned to obtain the metadata corresponding to multiple kernel functions. Then, according to the metadata and the scheduling strategy, the code file of the kernel function is automatically generated, which solves the problem that the existing operator fusion technology for sparse neural networks is limited by manual work. At the same time, it solves the problem of the lack of automatic tuning for unconventional sparse kernel functions corresponding to fused operators in the sparse neural network model.

[0055] As Figure 1 shown, the embodiments of the present application provide a tuning method based on a sparse neural network. The method includes:

[0056] S101. Obtain a sparse neural network;

[0057] Among them, the input neural network of the present application is a computational graph representing a sparse neural network model or a part of the computational graph. The sparse neural network computational graph is a computational graph in a deep learning framework, composed of nodes connected by directed edges. Among them, the nodes correspond to operators, representing an operation in the model; the directed edges represent tensors, and the direction of the edges represents the data dependency relationship between the operators, that is, the output data corresponding to the source node of the edge is used as the input data of the operator corresponding to the destination node. The output of the embodiments of the present application is the code of one or more sparse kernel functions, and the specific number is determined by the computational graph provided by the user.

[0058] In order to retain as much flexibility in code generation as possible while ensuring performance, in the embodiments of this application, as few restrictions as possible are imposed on the input sparse neural network. The basic requirements are as follows: the sub-computation graph should contain at least one sparse operator (including two types: g-SDDMM and g-SpMM), and among other operators, only sparse operators and lightweight dense operators (dense operators with a computational complexity of O(n), such as reduction, element-wise operations, etc.) can be included.

[0059] The operators of the sparse neural network all include multiple consecutive operator nodes, and each operator node belongs to a low-overhead dense operator or a sparse operator. Among them, the low-overhead dense operator refers to an operator with a complexity of O(n), such as element-wise operations; the sparse operator refers to an operator in the GNN framework that contains message-passing semantics. Such operators rely on the graph data structure (sparse matrix), so they are called sparse operators.

[0060] S102. Determine the number of kernel functions corresponding to the sparse neural network according to the sparse neural network and the pre-set metadata partitioning model;

[0061] Such as Figure 3As shown in the figure, the metadata partitioning model adopted in the embodiments of the present application is specifically a unified abstraction model, which is used to describe the parallelism and access patterns of sparse kernel functions in a sparse neural network model. In this abstraction model, both X and Y are input tensors with unlimited quantities, corresponding to the features of vertices and edges respectively. Among them, according to the access pattern of vertex feature X, the features can be divided into source features, target features, and edge features. Such a distinction method adopts relevant concepts in graph neural networks and is designed to facilitate determining the insertion positions of relevant codes when generating GPU sparse kernel functions with different parallel methods. If the row-wise parallel method in the sparse matrix is adopted, then each execution unit of the GPU (at different parallel granularities, referring to a thread, a warp, or a thread block respectively, and the specific situation is determined by the parallel strategy of this sparse kernel function) needs to first access the indices of each non-zero column in a row of the sparse matrix A, that is, the index task in the graph. Each index of a non-zero column is called an index unit, and each index unit corresponds to complex data reading and writing according to its row and column indices, that is, the data task in the graph. During the process of processing a row in the sparse matrix, the tensor corresponding to the feature of this row that each execution unit only needs to access once is defined as the target feature (dst), the tensor corresponding to the feature of each non-zero column in this row that needs to be accessed multiple times for the feature tensor is defined as the source feature (src), and the tensor that can be directly indexed by the serial number of the non-zero element in the sparse matrix without paying attention to the row and column where the non-zero element is located is defined as the edge feature (edge). During the generation process of the sparse kernel function, the source feature and the target feature have different access patterns, and the statements for accessing their data need to be placed in different positions in the code, so they need to be clearly distinguished; while for the edge feature, due to its indexing method, it is less affected by the parallel strategy. In the kernel function code using row-wise parallelism, the position of the code for accessing data is the same as that of the source feature.

[0062] In addition, according to the unified abstraction model, it also has the function of determining whether a fusion kernel function or multiple kernel functions need to be generated for the computation graph provided by the user.

[0063] That is to say, the terminal device matches according to the sparse neural network and the pre-set metadata partitioning model to determine the number of kernel functions corresponding to the sparse neural network. That is, the requirements of the finally obtained kernel functions correspond to each parameter in the metadata partitioning model. Therefore, this metadata partitioning model is used to determine the number of kernel functions.

[0064] S103. Traverse each operator in the sparse neural network in turn to obtain the basic data of each operator. The basic data includes at least the operator type, input tensor, output tensor, and access pattern.

[0065] S104. Partition the sparse neural network according to the basic data to obtain the metadata of each kernel function corresponding to the number of kernel functions.

[0066] Specifically, the terminal starts traversing from the input end of the sparse neural network, records the sparse operators that have been traversed in the traversal state, and collects the metadata of the current kernel function operator by operator; at the same time, infers the access pattern of the output tensor through the type of each operator and the access pattern of the input tensor. When the traversal state and the newly traversed sparse operator meet certain conditions, the sub-computation graph is split at the input vector of the input operator, and a new traversal state is created and recorded again starting from the new sparse operator until the traversal of the computation graph provided by the user is completed. Among them, the traversal state and the sparse kernel function metadata have the same data structure, and the traversal state obtained during the traversal process is the metadata of the sparse kernel function to be generated for the computation graph provided by the user.

[0067] S105. Determine the code file corresponding to the kernel function according to the metadata of the kernel function and the preset scheduling policy.

[0068] The scheduling policy of the sparse kernel function in this application includes optimization techniques such as thread mapping, memory access method, and reduction method of the kernel function, which have little to do with specific operations but directly affect the performance optimization of the kernel function. The scheduling policy used in the embodiments of this application is a summary of the existing sparse kernel function optimization techniques, and an intermediate representation - Design Process Graph (DPG) is used to effectively represent and manage the optimization policy used by a kernel function.

[0069] In the embodiments of this application, the DPG includes three parts:

[0070] Load distribution: The operations in this stage focus on transforming matrix A in the unified abstraction. The sparse matrix is divided according to the characteristics of different sparse matrix rows, a sparse format suitable for the sub-matrix situation is selected, and different strategies are allowed to complete the calculations of different sub-matrices.

[0071] Data mapping: The operations in this stage focus on mapping A and f to the hardware unit. Determine which data calculations are respectively responsible for the execution units (threads / thread warps / thread blocks) with different granularities of each GPU.

[0072] Computation task implementation: The operations in this stage determine the memory read / write methods, cache behavior, and reduction strategy (if any) during the computation process, without involving the specific computation content. For example, the memory read / write method only focuses on what kind of memory access instructions each thread uses to read / write data from / to the memory with what bit width, rather than on which tensor to read / write; the cache behavior only focuses on the cache strategies for different types of input tensors (source, destination), including whether to cache the results and in what way, rather than on how many source input tensors and destination tensors there are and which tensors they are respectively; the reduction strategy only focuses on what method to use for data exchange among multiple execution units during the reduction process, rather than on what kind of operations to use to aggregate partial results from different execution units to obtain the complete result.

[0073] It should be noted that the reason for using DPG as this intermediate representation in this application is that expressing the optimization strategy of the kernel function with an intermediate representation that is simpler and easier to operate than the kernel function code can facilitate local modification of the scheduling strategy of the existing kernel function, thus greatly simplifying the automatic tuning process.

[0074] For a kernel function divided from the computation graph provided by the user, use its corresponding metadata and the code snippets corresponding to the operations in the computation task implementation stage in DPG to insert into the corresponding functional areas of the kernel function skeleton, and then perform re-indexing - determine and assemble the index expressions in each array / pointer access statement in the code template through the scheduling strategies in the load distribution and data mapping stages of DPG to complete code generation.

[0075] In the embodiments of this application, corresponding code snippets are prepared for the common operators in the sparse neural network and the scheduling strategies in the computation implementation stage of DPG computations and are managed in the code snippet library. Specifically, among all the scheduling strategies, only the scheduling strategy in the computation task implementation stage can determine the code snippets for specific functional areas, while the scheduling strategies in the load distribution and data mapping stages will affect all data read / write statements in the code of the sparse kernel function. Therefore, vacancies are reserved in all array / pointer access instructions in the code snippets, and the vacancies in the index expressions of the memory access statements are filled in the re-indexing stage after the code assembly is completed.

[0076] In the embodiments of the present application, the kernel function skeleton is an induction of sparse kernel functions. The sparse kernel functions are divided into different functional regions. The code segments in each functional region have different contents in different sparse kernel functions, but perform the same functions. The kernel function skeleton proposed in the present application is composed of these functional regions that can cover all sparse kernel functions. First, the sparse kernel function instances generated in the embodiments of the present application do not need to include all functional regions. Second, in order to completely cover all semantics of complex sparse kernel functions, the embodiments of the present application allow some regions in the kernel function skeleton to repeat periodically. That is to say, functional regions with the same name may appear multiple times, but are located at different positions in the kernel function instance. In code generation, the code segments in the functional regions with the same name in the kernel function instance are used to implement the operations corresponding to different operators in the computation graph provided by the user. Therefore, the corresponding code segments are naturally different. The correspondence between the repeated region and the operator is as follows: in the operators to be fused corresponding to a kernel function, one g-SpMM corresponds to the end of a repeatable region. In other words, if there are multiple g-SpMM sparse operators in a kernel function, the code corresponding to the lightweight dense operator located between the i-th g-SpMM operator and the (i + 1)-th g-SpMM operator should be inserted into the corresponding functional region in the i-th repeatable region.

[0077] The process of generating the code file of the sparse kernel function in the embodiments of the present application includes three steps: 1. Determine the number of kernel functions to be generated and the metadata of each kernel function according to the computation graph provided by the user; 2. Select a scheduling strategy for each sparse kernel function; 3. Automatically generate the code of the sparse kernel function for each sparse kernel function according to the metadata and the selected scheduling strategy.

[0078] To ensure that the code generated in the embodiments of the present application has high performance and reduce the complexity of the decisions made during code generation, the first step of code generation in the embodiments of the present application is to divide the kernel functions to be generated from the computation graph provided by the user and collect relevant metadata for subsequent code generation and automatic tuning. Starting from graph neural networks, a mathematical model that can express various operator fusion methods is provided, and a code generation strategy for converting this mathematical model into kernel function code is proposed. In addition, the embodiments of the present application propose an intermediate representation of sparse kernel functions, which combines the optimization strategies of classical sparse kernel functions with the code generation mechanism. As a result, multiple kernel functions with different combinations of optimization strategies can be generated for the mathematical model corresponding to the same fused operator, making automatic tuning possible. To achieve high-efficiency automatic tuning, the embodiments of the present application propose a performance-driven automatic tuning technique, which ensures the tuning effect while shortening the tuning time.

[0079] The purpose of this application is to generate code for a more flexible operator fusion strategy in sparse neural networks. The main approach is to use the kernel function of sparse operators as a framework to accommodate more lightweight dense computations, thereby effectively reducing redundant memory access and improving the running performance of the code of memory-access-limited sparse neural networks on GPU devices. Through the code generation and automatic tuning technology for sparse neural networks provided by the embodiments of this application, sparse kernel functions can be generated for the vast majority of fused operators (obtained by fusing sparse operators and lightweight dense operators) in a sparse neural network model, and the kernel functions can be optimized through performance-driven automatic tuning technology to achieve higher performance on a specific input sparse matrix compared to existing sparse model optimization technologies. On the CUDA (Compute Unified Device Architecture) platform, which is a general computing platform and programming model launched for GPU products, some embodiments of this application can achieve a performance improvement of up to more than three times that of existing optimization technologies on mainstream sparse models.

[0080] Some embodiments of this application determine the number of kernel functions of a sparse neural network through a metadata partitioning model, partition the sparse neural network to obtain metadata corresponding to multiple kernel functions, and then automatically generate code files for the kernel functions according to the metadata and scheduling policies, solving the problem that the existing operator fusion technology for sparse neural networks is limited by manual work. At the same time, it solves the problem of the lack of automatic tuning for unconventional sparse kernel functions corresponding to fused operators in a sparse neural network model.

[0081] Another embodiment of this application further supplements and explains the tuning method based on a sparse neural network provided by the above embodiment.

[0082] Optionally, traverse each operator in the sparse neural network in sequence to obtain the basic data of each operator, including:

[0083] Traverse from the input end of the sparse neural network and determine the operator type of the operator according to the operator name;

[0084] Obtain the input tensor of the operator and the access pattern of the input tensor;

[0085] Determine the access pattern of the output tensor according to the input tensor and the access pattern of the input tensor;

[0086] Determine the traversal state of the operator according to the operator type, input tensor, output tensor, access pattern of the input tensor, and access pattern of the output tensor of each operator.

[0087] Optionally, partition the sparse neural network according to the basic data to obtain metadata for each kernel function corresponding to the number of kernel functions, including:

[0088] Judge each operator type according to the data dependence direction and operator name. The operator types include at least sparse operators and dense operators;

[0089] When the first sparse operator is traversed, judge the type of the first sparse operator;

[0090] Determine the output tensor access mode corresponding to the sparse operator according to the type of the sparse operator;

[0091] If the traversed operator is a dense operator, the access modes of the input tensor and the output tensor of the dense operator are the same;

[0092] Divide the sparse neural network according to the input tensor access mode and output tensor access mode of the sparse operator, and the input tensor access mode and output tensor access mode of the dense operator, to obtain the metadata of each kernel function corresponding to the number of kernel functions.

[0093] On this basis, the embodiments of the present application divide the computation graph according to the computation graph provided by the user to determine how many sparse kernel functions need to be generated for the computation graph given by the user. The purpose of dividing the computation graph in the embodiments of the present application is to split the computations that cannot be independently parallelized into different kernel functions to ensure a high-performance implementation method for the computation graph provided for the user. Unable to be independently parallelized means that after selecting a parallel strategy for the current sparse kernel function, global synchronization of all execution units is required between the computations corresponding to two operators to be fused to ensure data consistency. The most important thing in the scheduling strategy of sparse kernel functions is the parallel strategy because its selection affects the instruction selection of different operations in the entire kernel function and has a relatively direct correlation with the performance of the kernel function on different sparse matrices. Specifically, the parallel strategies of sparse kernel functions can be divided into two categories - row parallelism and non-zero element parallelism. Row parallelism means that each execution unit processes one row of the sparse matrix. The advantage of this strategy is that the operations between can be independently parallelized, and the disadvantage is the problem of uneven load; non-zero element parallelism means that each execution unit is responsible for the computational tasks corresponding to the same number of non-zero elements without caring which row this non-zero element belongs to. The advantage of this strategy is that the computational load is absolutely balanced, and the disadvantage is that a relatively high reduction overhead will be introduced when writing back the results.

[0094] In the unified abstract model of this application, only one access mode is allowed for a tensor. If there are two access modes for a tensor in a sparse kernel function, global synchronization is required between the two accesses to ensure data consistency. For example, in a part of the code, the tensor is written to the cache in the dst access mode, and in the subsequent code, the tensor is read from the cache in the src access mode. Then, a global synchronization statement must be added before the read operation. The calculation graph partitioning proposed in this application is precisely because global synchronization has a very high overhead, and in current GPU devices, this overhead is not as high as using two kernel functions to complete the same calculation.

[0095] The embodiments of this application provide a method for simultaneously completing calculation graph partitioning and metadata extraction by traversing the calculation graph once. Starting from the input end of the calculation graph provided by the user, the sparse operators that have been traversed are recorded in the traversal state, and the metadata of the current kernel function is collected operator by operator. At the same time, the access mode of the output tensor is inferred through the type of each operator and the access mode of the input tensor. When the traversal state and the newly traversed sparse operator meet certain conditions, the sub-calculation graph is split at the input vector of the input operator, and a new traversal state is created, starting from the new sparse operator and recording again until the traversal of the calculation graph provided by the user is completed. Among them, the traversal state and the sparse kernel function metadata have the same data structure, and the traversal state obtained during the traversal process is the metadata of the sparse kernel function that needs to be generated for the calculation graph provided by the user.

[0096] In some embodiments of this application, to ensure that the generated code has high performance and reduce the complexity of the decisions made during code generation, the first step in code generation in the embodiments of this application is to partition the kernel functions to be generated from the calculation graph provided by the user and collect relevant metadata for subsequent code generation and automatic tuning.

[0097] Optionally, determining the access mode of the output tensor corresponding to the sparse operator according to the type of the sparse operator includes:

[0098] If the first sparse operator belongs to the g-SDDMM type, the access mode of the output tensor is edge feature;

[0099] If the first sparse operator belongs to the g-SpMM type, the access mode of the output tensor is destination feature.

[0100] The difference between the calculation graph of the sparse model and the calculation graph of the conventional dense neural network model lies in the existence of sparse operators. In the sparse model, the sparse operators correspond to the operators involving sparse weights and attention; in the graph neural network, the sparse operators correspond to the calculation process of message passing on the learning graph. This application uses the descriptions of g-SpMM and g-SDDMM to classify the operators in the existing sparse model as follows:

[0101] g-SDDMM class: The access pattern of this type of operator is the same as that of sampled dense matrix multiplication (SDDMM). In a sparse sampled matrix A ∈ R m×n , if A[i, j] is a non-zero element, a vector of length f obtained through a certain access pattern is used to perform a certain operation with A[i, j] and then output to the result tensor Z[i, j], where Z ∈ R m×n×f′ , and the specific value of f' is related to the operation. The access pattern includes three types: First, read a vector Y[j] of length f from the dense matrix Y ∈ R n×f ; Second, read a vector X[i] of length f from the dense matrix X ∈ R m×f ; Third, read a vector E[i, j] of length f from the dense tensor E ∈ R m×n×f . For example, in the common conventional SDDMM calculation in the sparsification model, the operation is A[i, j]·X[i]·Y[j] T , f' = 1; while in the calculation of Scatter message passing in graph neural networks, the operation is X[i] + Y[j], and f' = f

[0102] g-SpMM class: The access pattern of this type of operator is the same as that of sparse-dense matrix multiplication (SpMM). In a sparse sampled matrix A ∈ R m×n , if A[i, j] is a non-zero element, a vector of length f obtained through a certain access pattern is used to perform a first operation with A[i, j], and the result is used to perform a second operation with Z[i] in the result matrix, where Z ∈ R m×f . For example, in the common SpMM calculation in the sparsification model, the first operation is Z' = A[i, j]*X[j], and the second operation is Z[i] = Z[i] + Z'; while in the calculation of the Gather message passing mechanism in graph neural networks, the access pattern is the third type, the first operation is Z' = E[i, j] (assignment operation, not using A[i, j]), and the second operation is Z[i] = Z[i] + Z'

[0103] For convenience, the embodiments of the present application respectively refer to the above three access modes as src, dst, and edge. In addition, the input and output regulations of the sparse kernel function in the present application are the same as those in the prior art. The input parameters of g-SDDMM and g-SpMM both include a sparse matrix, and the output parameters are both a dense matrix. For the input dense matrix, each input parameter has an access mode. g-SDDMM allows at most two input dense matrices, while g-SpMM has only one input dense matrix. It should be noted that the two types of sparse kernel functions of g-SDDMM and g-SpMM require template parameters to accurately describe the computational semantics of this kernel function. These parameters include the access modes of the two input parameters and the operation descriptors. Among them, g-SDDMM has only one operation descriptor, while g-SpMM includes two operation descriptors. The descriptor of g-SDDMM and the first operation descriptor option of g-SpMM are basically the same, including add, sub, mul, div, copy_lhs / rhs, etc., which respectively correspond to the element-wise four arithmetic operations between two vectors and only taking the first / second parameter as the result without performing operations. The second operation option of g-SpMM includes sum, min, and max, which respectively correspond to sum reduction, maximum reduction, and minimum reduction.

[0104] Therefore, the present application represents a sparse kernel function instance using the kernel function type and the template parameter list. For example, the conventional SDDMM is represented as g-SDDMM(src, dst, dot) in the embodiments of the present application, while the conventional SpMM is represented as g-SpMM(src, mul, sum) in the embodiments of the present application.

[0105] Optionally, according to the access modes of the input tensor and output tensor of the sparse operator, as well as the access modes of the input tensor and output tensor of the dense operator, the sparse neural network is partitioned to obtain the metadata of each kernel function corresponding to the number of kernel functions, including:

[0106] In the case where the operator is a sparse operator and the access modes of the input tensor and output tensor of the sparse operator are different, the sparse neural network is partitioned to obtain the metadata of each kernel function corresponding to the number of kernel functions.

[0107] At the beginning of the traversal, the embodiments of the present application first create the first traversal state and initialize it. The traversal state of the initial state is empty.

[0108] Subsequently, the embodiments of the present application traverse the computational graph provided by the user according to the topological order, and at the same time complete the collection of kernel function metadata and the partitioning of the computational graph.

[0109] The partitioning method of the computation graph is achieved by propagating the access patterns of tensors along the data dependency direction (the direction of the directed edges in the computation graph). When traversing to the first sparse operator, if this sparse operator belongs to the g-SDDMM type, the access pattern of the output tensor is edge; if it belongs to g-SpMM, the access pattern of the output tensor is dst. For subsequent dense operators traversed, the access pattern of the output tensor is the same as that of the input tensor. At the same time, the access pattern of the tensor can also help determine the label of the dense operator. When there are other sparse kernel functions in the traversal state, when traversing to a sparse operator again, it is necessary to check whether the access pattern of the input tensor matches the requirements of the kernel function itself for the access pattern of the input tensor. If they do not match, stop updating the current traversal state, create a new traversal state, and add this sparse operator to the new traversal state to achieve the partitioning of the computation graph.

[0110] The method for collecting kernel function metadata is to record each operator traversed and the data dependencies between them in the traversal state to obtain the mathematical meaning of the sparse kernel function, and other information is collected when specific conditions are met:

[0111] If the operator traversed cannot find a predecessor operator in the sub-computation graph of the current traversal state, its input tensor is added to the input tensor list of the kernel function.

[0112] There are two ways to collect the labels of dense operators according to the traversal state: 1. If there are no sparse operators in the traversal state, the dense operators are not labeled temporarily. When traversing to a sparse operator, the labels of the dense operators that have direct data dependencies with it are deduced according to the access pattern of the input tensor of the library sparse operator: if the access pattern of the output tensor in the sparse operator is src or dst, label this operator as ApplyVertex; if the access pattern of the output tensor in the sparse operator is edge, label this operator as ApplyEdge. According to the principle that the dense operator does not change the access pattern, the access pattern can be propagated backward along the data dependency chain formed by the dense operators, that is, the labels of the dense operators that have edges with the ApplyVertex dense operator are all ApplyVertex, and the labels of the dense operators that have edges with the ApplyEdge dense operator are all ApplyEdge. 2. If there are sparse operators in the traversal state, only use the access pattern of the input tensor obtained by access pattern propagation to determine the label of this operator: the labels of the dense operators with the input tensor access patterns of src and dst are ApplyVertex, and the labels of the dense operators with the input tensor access pattern of edge are ApplyEdge.

[0113] When finishing traversing the computation graph provided by the user or meeting the computation graph partitioning conditions, add the tensors corresponding to the out-edges of the operator with the last topological order in the sub-computation graph maintained in the traversal state to the output tensor table of the kernel function;

[0114] The purpose of partitioning the computation graph in some embodiments of this application is to split the computations that cannot be independently parallelized into different kernel functions, ensuring to find a high-performance implementation method for the computation graph provided to the user.

[0115] Optionally, the metadata of the kernel function includes at least the mathematical meaning of the kernel function. The mathematical meaning of the kernel function includes at least the input tensors, output tensors, and the mathematical operations between the input tensors and output tensors; the mathematical meaning of the kernel function is the names of the operators in the sub-computation graph corresponding to each kernel function and the data dependencies between them; the input tensors refer to the input tensors of the operator with the earliest topological order in the sub-computation graph corresponding to each kernel function, and the output tensors are the output tensors of the operator with the latest topological order.

[0116] The metadata of the sparse kernel function in the embodiments of this application is mainly the mathematical meaning of the function, including the input and output tensors and the mathematical operations performed by it. Additionally, the metadata of the kernel function in this application also contains some additional information to assist the subsequent code generation process.

[0117] Specifically, the input and output tensors refer to the input tensors of the operator with the earliest topological order in the sub-computation graph corresponding to each sparse kernel function, which are the input tensors of the kernel function, and the output tensors of the operator with the latest topological order are the output tensors of the kernel function. In the embodiments of this application, these tensors will be recorded in the input and output tensor table in the form of symbols so that corresponding memory read and write statements can be inserted during code generation; the mathematical meaning refers to the names of the operators in the sub-computation graph corresponding to each kernel function and the data dependencies between them; the additional information refers to the labels attached to the dense operators in the sub-computation graph corresponding to the sparse kernel function in the embodiments of this application.

[0118] Particularly, the reason for attaching labels to the dense operators in the sub-computation graph corresponding to the sparse kernel function in the embodiments of this application is to simplify the complexity of the decision-making during the code generation process while ensuring high-performance code generation. In the framework of the sparse kernel function, there are multiple options for the positions where the code corresponding to the lightweight dense operator fused with the sparse operator can be inserted, which is mainly determined by the data dependencies between the dense operator and the sparse operator. However, inserting the code corresponding to the operator at different positions will result in different memory access efficiencies. For example, in a sparse kernel function adopting a row-wise parallel strategy, inserting the read of the target feature into the loop for traversing the sparse rows will cause the execution unit to repeatedly read the same data, which does not affect the correctness of the computation, but the introduced redundant memory access will occupy additional memory bandwidth.

[0119] For the above reasons, the embodiments of the present application distinguish and label the above insertion positions, and use the same labels to label the lightweight dense operators in the sub-computation graph. During the code generation process, the code corresponding to this operator will be inserted into the functional area with the same label in the kernel function skeleton according to the label of the operator.

[0120] The lightweight dense operators to be fused in the embodiments of the present application are labeled into two categories: ApplyEdge and ApplyVertex. In the unified abstract model, the dense operator does not change the access pattern of the tensor. Therefore, the labels of the dense operators that have a data dependence relationship with it are judged by the access patterns of the input and output tensors of the sparse operator. Specifically, the dense operator that can act on the input tensor with the edge access pattern in the sparse operator should be labeled as ApplyEdge, and the dense operator that can act on the input tensor with the src and dst access patterns in the sparse operator should be labeled as ApplyVertex.

[0121] Some embodiments of the present application split the sparse neural network to obtain the metadata of each kernel function, and then generate the corresponding code files according to the metadata, and try to represent the sparse neural network with the fewest kernel functions.

[0122] Optionally, the preset scheduling strategy includes at least one or more of data mapping, thread mapping, caching method, reduction method, vectorization, and loop unrolling method.

[0123] Some embodiments of the present application generate a scheduling strategy by describing the parallelism and access patterns of the sparse kernel functions in the sparse neural network, and then generate the code files of each kernel function of the sparse neural network.

[0124] Exemplarily, Figure 2 This is an example of determining the metadata of the sparse kernel function to be generated by traversing the computation graph provided by the user in the embodiments of the present application.

[0125] Figure 2 In the first state of the traversal process shown in

[0125] , the first traversal state has successively traversed 3 operators with the earliest topological order: g-SDDMM(Add,src,dst), leakyrelu, and exp. Among them, since the earliest traversed sparse operator belongs to the g-SDDMM category, it can be inferred that the access pattern of its output tensor is edge. Therefore, it can be inferred that the labels of the dense operators leakyrelu and exp that have a data dependence with it are ApplyEdge.

[0126] Figure 2In the second state of the traversal process shown in the figure, in the first traversal state, the third sparse operator g-SDDMM(div,src,edge) has been traversed. However, the input parameter with the access pattern of src comes from the output of g-SpMM, and the access pattern should be dst, which does not match the access pattern of g-SDDMM(div,src,edge). Therefore, the computational graph needs to be partitioned here. The first traversal state stops updating, and a new second traversal state is created, and the operator g-SDDMM(div,src,edge) is added to it.

[0127] Finally, when the traversal of the computational graph provided by the user is completed, the embodiments of the present application obtain the metadata of two sparse kernel functions, and the corresponding sub-computational graphs are respectively shown in the figure.

[0128] It should be noted that each implementable manner in this embodiment can be implemented separately, or can be combined in any combination manner without conflict. The present application does not make any limitations.

[0129] Another embodiment of the present application provides a tuning device based on a sparse neural network, which is used to execute the tuning method based on a sparse neural network provided in the above embodiment.

[0130] As Figure 5 shown, it is a schematic structural diagram of a tuning device based on a sparse neural network provided by an embodiment of the present application. The tuning device based on a sparse neural network includes an acquisition module 501, a calculation module 502, a traversal module 503, a partitioning module 504, and a generation module 505, where:

[0131] The acquisition module 501 is used to acquire a sparse neural network;

[0132] The calculation module 502 is used to determine the number of kernel functions corresponding to the sparse neural network according to the sparse neural network and a pre-set metadata partitioning model;

[0133] The traversal module 503 is used to sequentially traverse each operator in the sparse neural network to obtain the basic data of each operator. The basic data at least includes the operator type, input tensor, output tensor, and access pattern;

[0134] The partitioning module 504 is used to partition the sparse neural network according to the basic data to obtain the metadata of each kernel function corresponding to the number of kernel functions;

[0135] The generation module 505 is used to determine the code file corresponding to the kernel function according to the metadata of the kernel function and a pre-set scheduling strategy.

[0136] Some embodiments of the present application determine the number of kernel functions of a sparse neural network through a metadata partitioning model, partition the sparse neural network to obtain metadata corresponding to multiple kernel functions, and then automatically generate code files for the kernel functions according to the metadata and a scheduling strategy, solving the problem that the existing operator fusion technology for sparse neural networks is limited by manual work. At the same time, it solves the problem of the lack of automatic tuning for unconventional sparse kernel functions corresponding to fused operators in the sparse neural network model.

[0137] Regarding the device in this embodiment, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0138] Another embodiment of the present application further supplements and explains the tuning device based on a sparse neural network provided in the above embodiment.

[0139] Optionally, the traversal module is used for:

[0140] Traverse from the input end of the sparse neural network, and determine the operator type of the operator according to the operator name;

[0141] Obtain the input tensor of the operator and the access pattern of the input tensor;

[0142] Determine the access pattern of the output tensor according to the input tensor and the access pattern of the input tensor;

[0143] Determine the traversal state of the operator according to the operator type, input tensor, output tensor, access pattern of the input tensor, and access pattern of the output tensor of each operator.

[0144] Optionally, the partitioning module is used for:

[0145] Judge each operator type according to the data dependence direction and the operator name, and the operator type includes at least a sparse operator and a dense operator;

[0146] When the first sparse operator is traversed, judge the type of the first sparse operator;

[0147] Determine the access pattern of the output tensor corresponding to the sparse operator according to the type of the sparse operator;

[0148] If the traversed operator is a dense operator, the access pattern of the input tensor and the access pattern of the output tensor of the dense operator are the same;

[0149] Partition the sparse neural network according to the access pattern of the input tensor and the output tensor of the sparse operator, and the access pattern of the input tensor and the output tensor of the dense operator, to obtain metadata of each kernel function corresponding to the number of kernel functions.

[0150] In some embodiments of the present application, in order to ensure that the generated code has high performance and reduce the complexity of the decisions made during code generation, the first step in code generation in the embodiments of the present application is to divide the kernel functions to be generated from the computation graph provided by the user and collect relevant metadata for subsequent code generation and auto-tuning.

[0151] Optionally, the partitioning module is configured to:

[0152] If the first sparse operator belongs to the g-SDDMM type, the output tensor access pattern is edge feature;

[0153] If the first sparse operator belongs to the g-SpMM type, the output tensor access pattern is destination feature.

[0154] Optionally, the partitioning module is configured to:

[0155] In the case where the operator is a sparse operator and the input tensor access pattern and the output tensor access pattern of the sparse operator are different, partition the sparse neural network to obtain the metadata of each kernel function corresponding to the number of kernel functions.

[0156] The purpose of partitioning the computation graph in some embodiments of the present application is to split the non-independent-parallel computations into different kernel functions to ensure finding a high-performance implementation for the computation graph provided to the user.

[0157] Optionally, the metadata of the kernel function at least includes the mathematical meaning of the kernel function, and the mathematical meaning of the kernel function at least includes the input tensor, the output tensor, and the mathematical operation between the input tensor and the output tensor; the mathematical meaning of the kernel function is the names of the operators in the sub-computation graph corresponding to each kernel function and the data dependency between them; the input tensor refers to the input tensor of the operator with the earliest topological order in the sub-computation graph corresponding to each kernel function, and the output tensor is the output tensor of the operator with the latest topological order.

[0158] Some embodiments of the present application split the sparse neural network to obtain the metadata of each kernel function, and then generate the corresponding code files according to the metadata, trying to represent the sparse neural network with the fewest kernel functions.

[0159] Optionally, the preset scheduling strategy at least includes one or more of: data mapping, thread mapping, caching method, reduction method, vectorization, and loop unrolling method.

[0160] Some embodiments of the present application generate a scheduling strategy by describing the parallelism and access patterns of the sparse kernel functions in the sparse neural network, and then generate the code files of each kernel function of the sparse neural network.

[0161] Regarding the device in this embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment related to the method, and will not be elaborated here.

[0162] It should be noted that each feasible manner in this embodiment can be implemented independently, or can be combined and implemented in any combination manner without conflict, and the present application does not make any limitations.

[0163] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it can implement the operations corresponding to any of the methods in the tuning method based on a sparse neural network provided in the above embodiment.

[0164] The embodiment of the present application also provides a computer program product. The computer program product includes a computer program. When the computer program is executed by a processor, it can implement the operations corresponding to any of the methods in the tuning method based on a sparse neural network provided in the above embodiment.

[0165] As Figure 6 shown, some embodiments of the present application provide an electronic device 600, which includes: a memory 610, a processor 620, and a computer program stored on the memory 610 and executable on the processor 620. When the processor 620 reads the program from the memory 610 through the bus 630 and executes the program, it can implement the methods of any of the embodiments included in the above-mentioned tuning method based on a sparse neural network.

[0166] The processor 620 can process digital signals and can include various computing architectures. For example, a complex instruction set computer architecture, a reduced instruction set computer architecture, or an architecture that implements a combination of multiple instruction sets. In some examples, the processor 620 can be a microprocessor.

[0167] The memory 610 can be used to store instructions executed by the processor 620 or data related to the execution of the instructions. These instructions and / or data can include code for implementing some or all of the functions of one or more modules described in the embodiments of the present application. The processor 620 of the present disclosure embodiment can be used to execute the instructions in the memory 610 to implement the method shown above. The memory 610 includes dynamic random access memory, static random access memory, flash memory, optical memory, or other memories well known to those skilled in the art.

[0168] The above are only the embodiments of the present application and are not used to limit the protection scope of the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application. It should be noted that similar reference numerals and letters indicate similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0169] As described above, the above are only the specific implementation manners of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, and all should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0170] It should be noted that in this text, relative terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element.

Claims

1. A tuning method based on sparse neural network, characterized in that: The method comprises: Get a sparse neural network; Determining the number of kernel functions corresponding to the sparse neural network according to the sparse neural network and a preset metadata partitioning model; Traversing each operator in the sparse neural network in turn to obtain basic data of each operator, wherein the basic data at least includes an operator type, an input tensor, an output tensor, and an access mode; Dividing the sparse neural network according to the basic data to obtain metadata of each kernel function corresponding to the number of kernel functions; According to the metadata of the kernel function and a preset scheduling policy, a code file corresponding to the kernel function is determined.

2. The tuning method based on sparse neural network according to claim 1, characterized in that: The step of sequentially traversing each operator in the sparse neural network to obtain basic data of each operator includes: Traversing from the input end of the sparse neural network, and determining the operator type of the operator according to the operator name; Get the input tensor of the operator and the access mode of the input tensor; Determining an access mode of an output tensor based on the input tensor and the access mode of the input tensor; The traversal state of the operator is determined according to the operator type, input tensor, output tensor, access mode of the input tensor and access mode of the output tensor of each of the operators.

3. The tuning method based on sparse neural network according to claim 1, characterized in that: The step of dividing the sparse neural network according to the basic data to obtain metadata of each kernel function corresponding to the number of kernel functions includes: According to the data dependency direction and the operator name, the type of each operator is judged, and the operator type includes at least a sparse operator and a dense operator; When traversing to the first sparse operator, judging the type of the first sparse operator; Determining, according to the type of the sparse operator, an output tensor access mode corresponding to the sparse operator; If the traversed operator is a dense operator, the access mode of the input tensor of the dense operator is the same as the access mode of the output tensor; According to the input tensor access mode and the output tensor access mode of the sparse operator and the input tensor access mode and the output tensor access mode of the dense operator, the sparse neural network is divided to obtain metadata of each kernel function corresponding to the number of kernel functions.

4. The sparse neural network-based tuning method according to claim 3, characterized in that: The step of determining, according to the type of the sparse operator, an output tensor access mode corresponding to the sparse operator includes: If the first sparse operator is of the g-SDDMM type, the output tensor access mode is edge feature; If the first sparse operator is of the g-SpMM type, the output tensor access mode is the destination feature.

5. The tuning method based on sparse neural network according to claim 3, characterized in that: The sparse neural network is divided according to the input tensor access mode and the output tensor access mode of the sparse operator and the input tensor access mode and the output tensor access mode of the dense operator to obtain metadata of each kernel function corresponding to the number of kernel functions, including: When the operator is a sparse operator and the input tensor access mode and the output tensor access mode of the sparse operator are different, the sparse neural network is divided to obtain metadata of each kernel function corresponding to the number of kernel functions.

6. The tuning method based on sparse neural network according to claim 1, characterized in that: The metadata of the kernel function includes at least the mathematical meaning of the kernel function, which includes at least the input tensor, the output tensor, and the mathematical operation between the input tensor and the output tensor; the mathematical meaning of the kernel function is the name of the operator in the sub-computation graph corresponding to each kernel function and the data dependency relationship between them; The input tensor refers to the input tensor of the operator with the earliest topological order in the subcomputation graph corresponding to each kernel function, and the output tensor refers to the output tensor of the operator with the latest topological order.

7. The tuning method based on sparse neural network according to claim 1, characterized in that: The preset scheduling strategy includes at least one or more of data mapping, thread mapping, cache mode, reduction mode, vectorization and loop unrolling mode.

8. A tuning device based on a sparse neural network, characterized in that: The device comprises: An acquisition module, used to acquire a sparse neural network; A calculation module, used for determining the number of kernel functions corresponding to the sparse neural network according to the sparse neural network and a preset metadata partitioning model; A traversal module, used to traverse each operator in the sparse neural network in turn to obtain basic data of each operator, wherein the basic data at least includes an operator type, an input tensor, an output tensor, and an access mode; A partitioning module, used for partitioning the sparse neural network according to the basic data to obtain metadata of each kernel function corresponding to the number of kernel functions; A generation module is used to determine a code file corresponding to the kernel function according to the metadata of the kernel function and a preset scheduling strategy.

9. An electronic device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor can implement the sparse neural network-based tuning method as described in any one of claims 1 to 7 when executing the program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the program is executed by a processor, the sparse neural network-based tuning method described in any one of claims 1 to 7 can be implemented.

Images