A high-performance sparse computing programming framework implementation method and system

Abstract

The present invention provides a method and system for implementing a high-performance sparse computing programming framework, and belongs to the field of high-performance computing technology. The method includes: providing a description, storage description, and operation description of the input sparse tensor dimension information through an interface; saving the dimension information and storage description using a sparse information storage structure, and storing the operation description using an intermediate representation; translating the calculation description using the saved sparse information to generate a decoding memory access statement in the sparse calculation; using sparse-aware scheduling primitives to optimize the memory access of the calculation description containing the decoding memory access statement, and generating an optimized intermediate representation; converting the intermediate representation into an intermediate representation recognizable by the target platform compiler, and obtaining a sparse computing kernel running on a general computing platform. The present invention improves the development efficiency and parallel performance of sparse computing by implementing an abstract data structure for sparse information storage, providing a sparse information programming interface to the upper layer, and automatic optimization scheduling based on sparse information.

Description

Technical Field

[0001] The invention relates to a method and system for realizing a high-performance sparse computing programming framework, and belongs to the technical field of high-performance computing. Background Art

[0002] As an important paradigm in the computing field, sparse computing plays a vital role in both scientific computing and artificial intelligence. In the field of scientific computing, such as social relationship networks, protein structures, and point cloud data, data is naturally sparse, that is, the data contains a large number of zero values ​​or invalid values. This sparsity not only affects the storage efficiency of the data, but also significantly increases the unnecessary overhead in the computing process.

[0003] In recent years, with the development of deep neural network (DNN) model pruning technology, sparse computing has ushered in new development opportunities and challenges. Deep neural networks usually obtain sparse model weights or activation values ​​through pruning methods. Compared with the sparse data in the field of scientific computing, these sparse data have lower sparsity, but the sparse pattern can be freely selected within a certain range, providing more flexible data features for sparse computing.

[0004] The development of high-performance sparse operators is more dependent on data characteristics. It is necessary to design sparse formats and parallel scheduling strategies based on the scale of the operation matrix, the sparsity of the sparse matrix, and the distribution characteristics of non-zero elements. Since there are a large number of zero elements inside the sparse matrix, if they are not specially processed, these zero elements will not only occupy storage space, but also introduce unnecessary computational overhead. Therefore, the acceleration of sparse computing is inseparable from efficient sparse storage formats. Sparse storage formats play a role in refining and summarizing sparse data. Designing efficient compression formats can minimize the storage and computational overhead of sparse computing.

[0005] In addition to designing sparse formats, it is also necessary to design parallel scheduling algorithms for specific data in combination with hardware characteristics to fully exploit the potential parallelism and locality in sparse operators. For example, when sparse data matrices are distributed in local blocks or along diagonals, there are specific sparse formats and algorithms that are applicable.

[0006] Existing sparse programming frameworks, such as SparseTIR and TACO, provide programming interfaces for sparse computing, but they are insufficient in processing the inter-dimensional relationship and cache scheduling of sparse data. The programming interfaces of these frameworks do not provide input support for the inter-dimensional relationship of sparse data, and also lack cache scheduling methods based on sparse information in scheduling optimization. Therefore, there is insufficient support for the development of high-performance sparse operators in specific scenarios, especially operators in sparse deep neural networks. Summary of the invention

[0007] The purpose of the present invention is to provide a method and system for implementing a high-performance sparse computing programming framework to meet the needs of high-performance sparse operator development in the context of the artificial intelligence era. The framework implements an abstract data structure for sparse information storage, on the one hand, provides a sparse information programming interface to the upper layer, and on the other hand, performs automatic optimization based on the stored sparse information data structure, thereby improving the efficiency and parallelism of sparse computing.

[0008] In order to achieve the above object, the present invention is implemented through the following technical solutions:

[0009] A method for implementing a high-performance sparse computing programming framework, comprising:

[0010] The description of the input sparse tensor dimension information is given through the dimension information interface, and the storage description and operation description are given through the storage description interface and the operation description interface;

[0011] The dimension information and storage description are saved using a sparse information storage structure, and the operation description is stored using an intermediate representation;

[0012] The saved sparse information is used to translate the computation description and generate the decoding memory access statement in the sparse computation;

[0013] Use sparse-aware scheduling primitives to cache the computational descriptions containing decoded memory access statements, use the hardware architecture's cache to improve memory access efficiency, and generate optimized intermediate representations.

[0014] The optimized intermediate representation is converted into an intermediate representation recognizable by the target platform compiler to obtain a sparse computing kernel running on a general computing platform.

[0015] Preferably, a description of the input sparse tensor dimension information is given through a dimension information interface, and a storage description and an operation description are given through a storage description interface and an operation description interface. The specific steps are as follows:

[0016] Traverse the dimensions of sparse tensor data and give feature descriptions, including the length, sparse or dense attributes, fixed-length or variable-length attributes of each dimension; if the current dimension does not contain non-zero elements, set it to a dense attribute; if the current dimension contains non-zero elements, set it to a sparse attribute, and index the name and bit width of the array; if the number of elements in the current dimension is fixed, set it to a fixed-length attribute, and if it is sparse at the same time, the number of non-zero elements in a fiber in the current dimension must be given; if the number of elements is not fixed, set it to a variable-length attribute, and give the name and bit width of the pointer array; if it is sparse at the same time, the number of all non-zero elements in the current dimension must be given;

[0017] Check the connection features between dimensions and set relationships, including sharing, fusion, and inheritance. If the number and value of subtree nodes in the same layer of the sparse tensor index tree containing non-zero elements are the same, then the dimension is set to a sharing relationship shared by the upper dimension. If the number of nodes in a layer containing non-zero elements in the sparse tensor index tree is the same as that in the upper layer, set a fusion relationship for the layer and add the index array of the upper layer. All other situations except sharing and fusion relationships are set to inheritance relationships.

[0018] Declare sparse storage space according to the data distribution characteristics of the sparse tensor, bind the determined dimension information to the sparse storage space, and provide the storage data name and data bit width in the sparse storage space;

[0019] Declare the iterators used in the operation, bind the iterators to the sparse dimensions, use the iterators to access the sparse storage space, and perform the calculation operation.

[0020] Preferably, the dimension information and storage description are stored using a sparse information storage structure, and the operation description is stored using an intermediate representation. The specific steps are as follows:

[0021] Create storage space for each dimension of the dimension information interface to store the sparse and variable-length attribute pairs of the current dimension, the length of the current dimension, the number of non-zero elements of the current dimension, the relationship between the current dimension and the upper dimension, the pointer array of the current dimension, and the index array of the current dimension;

[0022] Create a storage space for the sparse storage interface, store the specific dimensions in the dimension information interface bound to the storage space; store the non-zero element array;

[0023] Create storage space for the sparse iterator and declare the sparse dimensions bound to the sparse iterator;

[0024] The calculation code is stored and TensorIR is used as the intermediate representation of the calculation code storage.

[0025] Preferably, the stored sparse information is used to translate the computation description to generate a decoding memory access statement in the sparse computation, and the specific steps are as follows:

[0026] Traverse the dimension information and generate auxiliary sparse storage space;

[0027] Traverse the sparse iterator and generate nested loops around the intermediate expressions in the calculation code according to the order of the sparse iterator;

[0028] Traverse the intermediate representation code and replace the iterators of the sparse attribute dimensions bound by the intermediate representation code with the index array. For the iterators bound to the dense attribute dimensions but the dimensions are fused with the lower layer, replace them with the upper layer index.

[0029] Traverse the intermediate representation code. For BufferLoad nodes or BufferStore nodes, match the transformed memory access index with the iteration space on the sparse storage according to the attributes of the current sparse storage dimension, and calculate the flattened index on the sparse storage space in sequence.

[0030] Preferably, traversing the dimension information to generate the auxiliary sparse storage space is specifically as follows:

[0031] If the current dimension has a sparse attribute and is in a fusion relationship with the upper dimension, the current dimension index storage space and the upper dimension index storage space are generated, and the size of the index array space is the number of non-zero elements stored in the current dimension;

[0032] If the current dimension has a sparse attribute and is in a shared relationship with the upper dimension, generate the current dimension index storage space. The size of the index array space is the number of non-zero elements in a fiber of the current dimension stored;

[0033] If the current dimension has sparse attributes and it is not the case mentioned above, generate the current dimension index storage space, and the size of the index array space is the number of non-zero elements stored in the current dimension;

[0034] If the current dimension is a variable-length attribute and the upper dimension is dense, the upper dimension of the current dimension is searched in the sparse information storage structure to form an inheritance relationship, and the length of the generated pointer array is the product of the lengths of all dimensions plus 1;

[0035] If the current dimension is a variable-length attribute and the upper dimension is sparse, the pointer array size is the number of non-zero elements in the upper dimension plus 1.

[0036] Preferably, according to the order of the sparse iterator, a nested loop is generated outside the calculation code intermediate expression, and the specific steps are as follows:

[0037] Bind the iterator of a dense fixed-length dimension, generate a loop step of 1, and the loop length is the length of the dimension;

[0038] If it is a variable-length dimension, the generated loop step is 1, and the current variable-length loop length as follows:

[0039] ,

[0040] in, Represents the current dimension pointer array, and searches for the upper layer of the current dimension in the sparse information storage structure to form an inheritance relationship. represents the lengths of these dimensions, Represents the iterators corresponding to these dimensions;

[0041] If it is a fixed-length and sparse dimension, the generated cycle step is 1. When it is in a shared relationship with the upper layer, the generated cycle length is the number of non-zero elements in a single fiber of the current dimension. In other scenarios, the generated cycle length is the number of non-zero elements in the current dimension.

[0042] Preferably, the transformed memory access index is matched with the iteration space on the sparse storage according to the attribute of the current sparse storage dimension, as follows:

[0043] If it is a dense attribute, the index on the storage space does not change;

[0044] If it is a sparse attribute, the sparse dimension bound in the index is the same as the dimension bound to the sparse storage. Use the iterator bound to the sparse dimension to replace the index array to access the sparse storage.

[0045] If the index contains a dimension with sparse attributes, and the sparse dimension bound in the index is different from the dimension bound by the sparse storage, the non-zero element positions of the two sparse dimensions are unioned, and the union result array is used for replacement access.

[0046] Preferably, the flattened index on the sparse storage space is calculated in sequence, and the flattened index is initially 0; specifically as follows:

[0047] If a fixed-length dimension is bound and has an inheritance relationship with the next layer, it will not be processed;

[0048] Bind the fixed-length dimension and make it a non-inherited relationship with the next layer. Flatten the index and add the index of the dimension multiplied by the number of non-zero elements of the next layer of dimensions. If it is the first dimension, multiply it by 1.

[0049] Bind the dimension of the variable-length attribute, find the dimension of the upper layer of the current dimension in the sparse information storage structure to form an inheritance relationship, and flatten the index accumulation , the calculation formula is as follows:

[0050] ,

[0051] in, Index for this dimension.

[0052] Preferably, memory access optimization is performed using memory access statements generated by sparse-aware scheduling primitives, and the specific steps are as follows:

[0053] Enter the code block to be cached and the storage area to be cached;

[0054] Check the storage level of the storage area to be cached in the code to determine whether the current cache level is at the top level. If it is not the top level, then determine its corresponding upper-level high-speed storage unit. For storage areas without special declarations, consider their storage level as memory.

[0055] Traverse the intermediate representation stored in the computation code, find the input code block to be cached, and determine the position of the left sibling node in the abstract syntax tree as the position to insert the cache node;

[0056] Record the iterators that appear inside the code block area to be cached and add them to the active iterator variable set;

[0057] Traverse the statements to be cached, analyze the BufferLoad nodes or BufferStore nodes, and calculate the cache operation area; specifically as follows: if the traversed index is an iterator variable, if it is in the iterator set, the range of the iterator is used as the memory access range, if it is not in the iterator set, the value of the iterator in the code block does not change and is not processed; if the traversed index is a sparse index array, if the iterator on the sparse index array is in the iterator set, the length of the sparse dimension is used as the range; if the traversed index is a pointer array, search the sparse information storage structure for the upper layer of the current dimension to form an inheritance relationship, and the length of the generated pointer array is the product of the length of all dimensions and the length of the previous dimension; if the traversed index is other array types, the memory access range is determined according to the maximum value range of the data type in the array;

[0058] Generate a data transfer code from the storage level to be cached to the upper-level cache according to the calculated memory access range, and insert the operation statement block into the determined insertion cache node position;

[0059] Traverse the statements to be cached and replace the storage space access in the BufferLoad node or BufferStore node with the access to the cache area.

[0060] A system of high-performance sparse computing programming framework, comprising:

[0061] An interface module, used to receive and process the dimension information description, storage description and operation description of the sparse tensor;

[0062] The storage module saves the dimension information and storage description using a sparse information storage structure, and stores the operation description using an intermediate representation;

[0063] A decoding memory access statement generation module uses the stored sparse information to translate the calculation description and generate a decoding memory access statement in the sparse calculation;

[0064] The sparse-aware scheduling optimization module is used to optimize memory access statements and generate optimized intermediate representations.

[0065] The compiler interface module is used to convert the optimized intermediate representation into an intermediate representation recognizable by the target platform compiler.

[0066] The advantages of the present invention are:

[0067] Improve the efficiency of sparse computing. By designing an efficient sparse storage format, the present invention can minimize the storage and computing overhead of sparse computing. The sparse storage format refines and summarizes sparse data, effectively avoiding the occupation of storage space and unnecessary calculations by zero elements, thereby significantly improving the efficiency of sparse computing.

[0068] Enhance programming flexibility. The sparse programming framework provided by the present invention provides a rich sparse information programming interface to the upper layer by implementing an abstract data structure for sparse information storage. This allows programmers to more flexibly describe the dimensional information, storage structure, and computing requirements of sparse tensors, thereby developing high-performance sparse operators.

[0069] Optimize memory access performance, and use the saved sparse information. The present invention uses sparse-aware scheduling primitives to optimize memory access. By making full use of the cache on the hardware architecture, reducing memory access latency and maximizing parallelism, an optimized intermediate representation is finally generated. This optimization process significantly improves the memory access performance of sparse computing, allowing sparse operators to access data more efficiently during runtime.

[0070] Supporting cross-platform compilation optimization, by converting the optimized intermediate representation into an intermediate representation recognizable by compilers of other target platforms, the cache scheduling of the present invention can be applied to different general computing platforms to improve the performance of sparse computing kernel code. This feature makes the sparse programming framework of the present invention widely applicable and portable, and can meet the development needs of high-performance sparse operators in different scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0072] Figure 1 It is a schematic diagram of the process of the present invention.

[0073] Figure 2 Schematic diagram of the overall architecture of the sparse programming framework of the present invention.

[0074] Figure 3 Schematic diagram of sparse tensor representation of the present invention.

[0075] Figure 4 This is a schematic diagram of the sparse information data structure of the present invention.

[0076] Figure 5 A schematic flow chart of decoding memory access statements in sparse computing is generated for the present invention.

[0077] Figure 6This is a schematic diagram of the sparse-aware cache scheduling process of the present invention. DETAILED DESCRIPTION

[0078] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0079] like Figure 1-Figure 2 As shown, a method for implementing a high-performance sparse computing programming framework includes:

[0080] S1: Give a description of the input sparse tensor dimension information through the dimension information interface, and give a storage description and operation description through the storage description interface and operation description interface.

[0081] S2: The dimension information and storage description are saved using a sparse information storage structure, and the operation description is stored using an intermediate representation.

[0082] S3: Use the saved sparse information to translate the computation description and generate decoding memory access statements in the sparse computation.

[0083] S4: Use sparse-aware scheduling primitives to cache the computational descriptions containing decoded memory access statements, use the hardware architecture’s cache to improve memory access efficiency, and generate optimized intermediate representations.

[0084] S5: Convert the optimized intermediate representation into an intermediate representation recognizable by the target platform compiler to obtain a sparse computing kernel running on a general computing platform.

[0085] As a refinement of the above embodiment, the sparse tensor is described in step S1. There are related methods in SparseTIR and TACO to convert the sparse tensor into different sparse index tree representations corresponding to different sparse formats. Figure 3 As shown in the figure, each dimension of the 4-dimensional sparse tensor corresponds to a layer of the sparse tensor index tree. The most common description method is to give a description for each dimension of the sparse tensor. There are four attributes: sparse, dense, fixed-length, and variable-length. Sparse and dense are used to indicate whether the corresponding dimension contains non-zero elements, while fixed-length and variable-length attributes are used to indicate whether the number of elements in a specific dimension is fixed. On this basis, the connection relationship between different layers of the sparse tensor index tree can be described to make the distribution characteristics of non-zero elements in the sparse tensor clearer.

[0086] The specific implementation steps are as follows:

[0087] The description of the input sparse tensor dimension information is given through the dimension information interface, and the storage description and operation description are given through the storage description interface and operation description interface. The specific steps are as follows:

[0088] S101: Traverse the dimensions of sparse tensor data and give feature descriptions, including the length, sparse or dense attributes, fixed length or variable length attributes of each dimension; if the current dimension does not contain non-zero elements, set it to dense attributes; if the current dimension contains non-zero elements, set it to sparse attributes, and index the name and bit width of the array; if the number of elements in the current dimension is fixed, set it to fixed length attributes, if it is sparse at the same time, the number of non-zero elements in a fiber in the current dimension needs to be given; if the number of elements is not fixed, set it to variable length attributes, and give the name and bit width of the pointer array. If it is sparse at the same time, the number of all non-zero elements in the current dimension needs to be given.

[0089] S102: Check the connection features between dimensions and set relationships, including sharing relationships, fusion relationships, and inheritance relationships; if the number and value of subtree nodes in the same layer of the sparse tensor index tree containing non-zero elements are the same, then the dimension is set to a sharing relationship shared by the upper dimension; if the number of nodes in a layer containing non-zero elements in the sparse tensor index tree is the same as that of the upper layer, set a fusion relationship for the layer and add the index array of the upper layer; except for sharing relationships and fusion relationships, other situations are set to inheritance relationships.

[0090] S103: Declare a sparse storage space according to the data distribution characteristics of the sparse tensor, bind the determined dimension information to the sparse storage space, and provide a name of the stored data and a data bit width in the sparse storage space.

[0091] S104: Declare the iterator used in the operation, bind the iterator to the sparse dimension, use the iterator to access the sparse storage space, and perform the calculation operation.

[0092] The following code shows examples of the dimension description interface and sparse storage interface. The first four lines give four sparse dimension definitions. The first and second dimension attributes are dense fixed, with lengths of m and n. The third dimension attribute is sparse fixed, with its upper layer being J, length of 4, number of non-zero elements in the fiber being 2, data bit width being 2 bits, and a fusion relationship with the upper layer. The index arrays of this layer and the upper layer are index0 and index1. The fourth dimension attribute is sparse variable length, with its upper layer being I, length of n, number of non-zero elements in the current dimension being nnz, index array being index, pointer array being indptr, data bit width being 8 bits, and an inheritance relationship with the upper layer. The fifth and sixth lines define two storage spaces, A and B, respectively, to store the non-zero element arrays vals0 and vals1, with data bit widths of 16-bit floating point numbers. They are bound to the sparse dimensions (I, L) and (J, K), respectively, representing the distribution of sparse elements in the space.

[0093] Sample code:

[0094] I = T.dense fixed(m)

[0095] J = T.dense fixed(n)

[0096] K=T.sparse_fixed(J,(4,2),"fused",index0,index1,"int2")

[0097] L=T.sparse_variable(I,(n,nnz),"integrated", index, indptr, "int8")

[0098] A=T.match_sparse_buffer(vals0,(I,L),"float16")

[0099] B=T.match_sparse_buffer(vals1,(J,K),"float16").

[0100] The following code is an example of operation description. The first line of sparse iterator is an iterator declaration. Three iterators i, j and k are declared. These three iterators are bound to the three sparse dimensions I, J and K respectively. Lines 2-4 are the calculation definitions given for dense space. Because the distribution characteristics of sparse data are stored in the dimension information, users can only focus on the operation characteristics of dense space without worrying about indexing and other issues. The framework will automatically translate and optimize the statements with sparse decoding later.

[0101] Sample code:

[0102] with T.iter([I,J,K,"SSR")as [i,j,k]:

[0103] with T.init():

[0104] C[I,j]= 0.0

[0105] C[i,j]=c[i,j]+ A[i,k] * B[k,j].

[0106] As a refinement of the above embodiment, in step S2, the dimension information and storage description are stored using a sparse information storage structure, and the operation description is stored using an intermediate representation. The sparse information data structure is as follows: Figure 4 As shown, the specific steps are as follows:

[0107] S201: Create storage space for the four dimensions I, J, K, and L of the dimension information interface respectively, to store the sparse and variable-length attribute pairs of the current dimension, the length of the current dimension, the number of non-zero elements of the current dimension, the relationship between the current dimension and the upper dimension, the pointer array of the current dimension, and the index array of the current dimension respectively.

[0108] S202: Create storage spaces A and B for the sparse storage interface, store specific dimensions of the four dimensions I, J, K, and L of the dimension information interface bound to the storage space; and store non-zero element arrays values0 and values1.

[0109] S203: Create storage space for the sparse iterator, and declare the sparse dimension and sparse storage bound to the sparse iterator.

[0110] S204: Store the calculation code and use TensorIR as the intermediate representation of the calculation code storage. TensorIR uses a variety of different statement and expression nodes to form an overall abstract syntax tree. Taking the ForStmt node as an example, it means that there is a loop statement at the current position in the calculation code. The AddExpr node represents an addition operation expression, which needs to be followed by two expressions as its left and right operands. The BufferLoad and BufferStore nodes represent the read and write operations for a storage space, respectively. The operands in the nodes contain the storage area and memory access index for memory access.

[0111] As a refinement of the above embodiment, in step S3, the stored sparse information is used to translate the calculation description to generate a decoding memory access statement in the sparse calculation, such as Figure 5 The specific steps are as follows:

[0112] S301: traverse the dimension information to generate auxiliary sparse storage space; the details are as follows:

[0113] If the current dimension has a sparse attribute and is in a fusion relationship with the upper dimension, the current dimension index storage space and the upper dimension index storage space are generated, and the size of the index array space is the number of non-zero elements stored in the current dimension.

[0114] If the current dimension has a sparse attribute and is in a shared relationship with the upper dimension, the current dimension index storage space is generated, and the size of the index array space is the number of non-zero elements in a fiber stored in the current dimension.

[0115] If the current dimension has a sparse attribute and it is not the case as mentioned above, the current dimension index storage space is generated, and the size of the index array space is the number of non-zero elements stored in the current dimension.

[0116] If the current dimension is a variable-length attribute and the upper dimension is dense, the upper dimension of the current dimension is searched in the sparse information storage structure to form an inheritance relationship, and the length of the generated pointer array is the product of the lengths of all dimensions plus 1; the specific formula is as follows:

[0117] ,

[0118] in, Indicates the length of the dimension found.

[0119] If the current dimension is a variable-length attribute and the upper dimension is sparse, the pointer array size is the number of non-zero elements in the upper dimension plus 1.

[0120] S302: traverse the sparse iterators and generate a nested loop around the calculation code intermediate expression according to the order of the sparse iterators; the specific steps are as follows:

[0121] Binds an iterator of a dense fixed-length dimension, generating a loop with a step of 1 and a loop length equal to the length of the dimension.

[0122] If it is a variable-length dimension, the generated loop step is 1, and the current variable-length loop length as follows:

[0123] ,

[0124] in, Represents the current dimension pointer array, and searches for the upper layer of the current dimension in the sparse information storage structure to form an inheritance relationship. represents the lengths of these dimensions, Represents an iterator corresponding to these dimensions.

[0125] If it is a fixed-length and sparse dimension, the generated cycle step is 1. When it is in a shared relationship with the upper layer, the generated cycle length is the number of non-zero elements in a single fiber of the current dimension. In other scenarios, the generated cycle length is the number of non-zero elements in the current dimension.

[0126] S303: traverse and access the intermediate representation code, and replace the iterators of the sparse attribute dimensions bound to the intermediate representation code with the index array in step S301. For the iterators bound to the dense attribute dimensions but the dimensions are fused with the lower layer, replace them with the upper layer index in step S301.

[0127] S304: traverse the intermediate representation code, for a BufferLoad node or a BufferStore node, match the transformed memory access index with the iteration space on the sparse storage according to the attribute of the current sparse storage dimension, and calculate the flattened index on the sparse storage space in sequence.

[0128] The transformed access index is matched with the iteration space on the sparse storage according to the attributes of the current sparse storage dimension, as follows:

[0129] If it is a dense attribute, the index on the storage space does not change.

[0130] If it is a sparse attribute, the sparse dimension bound in the index is the same as the dimension bound to the sparse storage. Use the iterator bound to the sparse dimension to replace the index array to access the sparse storage.

[0131] If the index contains a dimension with sparse attributes, and the sparse dimension bound in the index is different from the dimension bound by the sparse storage, the non-zero element positions of the two sparse dimensions are unioned, and the union result array is used for replacement access.

[0132] The flattened index on the sparse storage space is calculated in order as follows:

[0133] The fixed-length dimension is bound and has an inheritance relationship with the next layer, so it is not processed.

[0134] Bind the fixed-length dimension and have a non-inheritance relationship with the next layer. Flatten the index by adding the index of the dimension multiplied by the number of non-zero elements of the next layer of dimensions. If it is the first dimension, multiply it by 1.

[0135] Bind the dimension of the variable-length attribute, find the dimension of the upper layer of the current dimension in the sparse information storage structure to form an inheritance relationship, and flatten the index accumulation , the calculation formula is as follows:

[0136] ,

[0137] in, Index for this dimension.

[0138] As a refinement of the above embodiment, in step S4, the sparse-aware scheduling primitive is used to perform cache scheduling on the computation description containing the decoded memory access statement, and the high-speed cache of the hardware architecture is used to improve the memory access efficiency, such as Figure 6 The specific steps shown are as follows;

[0139] S401: Input a code block to be cached and a storage area to be cached.

[0140] S402: Check the storage level of the storage area to be cached in the code to determine whether the current level to be cached is at the topmost layer. If it is not the topmost layer, its corresponding upper-level high-speed storage unit can be determined. For storage areas without special declarations, its storage level is considered to be memory.

[0141] S403: traverse the intermediate representation stored in the computation code, search for the input code block to be cached, and determine the position of the left sibling node in the abstract syntax tree as the position to insert the cache node.

[0142] S404: Record the iterators that appear in the code block area to be cached, and add them to the active iterator variable set.

[0143] S405: traverse the statements to be cached, analyze the BufferLoad nodes or BufferStore nodes, and calculate the cache operation area; the details are as follows:

[0144] The traversed index is the iterator variable. If it is in the iterator set, the range of the iterator is used as the memory access range. If it is not in the iterator set, the value of the iterator in the code block is not changed and is not processed;

[0145] The traversed index is a sparse index array. If the iterator on the sparse index array is in the iterator set, the length of the sparse dimension is used as the range;

[0146] The traversed index is a pointer array. If the iterator on the pointer array is in the iterator set, the dimension that forms an inheritance relationship with the upper layer of the current dimension is searched in the sparse information storage structure. The calculation range is the length of all dimensions found. With the previous dimension length The specific formula is as follows:

[0147] ;

[0148] If the traversed index is of other array types, the memory access range is determined according to the maximum value range of the data type in the array.

[0149] S406: Generate data transfer code from the to-be-cached storage level to the upper-level cache according to the calculated memory access range, and insert the operation statement block into the determined insertion cache node position.

[0150] S407: Traverse the statements to be cached, and replace the storage space access in the BufferLoad node or the BufferStore node with the access to the cache area.

[0151] A system of high-performance sparse computing programming framework, comprising:

[0152] The interface module is used to receive and process the dimension information description, storage description and operation description of the sparse tensor.

[0153] The storage module saves the dimension information and storage description using a sparse information storage structure, and stores the operation description using an intermediate representation.

[0154] The decoding memory access statement generation module uses the stored sparse information to translate the calculation description and generate the decoding memory access statement in the sparse calculation.

[0155] The sparse-aware scheduling optimization module is used to optimize memory access statements and generate optimized intermediate representations.

[0156] The compiler interface module is used to convert the optimized intermediate representation into an intermediate representation recognizable by the target platform compiler.

[0157] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for implementing a high-performance sparse computing programming framework, characterized in that: include: Receive a description of the input sparse tensor dimension information through a dimension information interface, and receive a storage description and an operation description through a storage description interface and an operation description interface; The dimension information and storage description are saved using a sparse information storage structure, and the operation description is stored using an intermediate representation; The saved sparse information is used to translate the computation description and generate the decoding memory access statement in the sparse computation; Use sparse-aware scheduling primitives to cache the computational descriptions containing decoded memory access statements, use the hardware architecture's cache to improve memory access efficiency, and generate optimized intermediate representations. Convert the optimized intermediate representation into an intermediate representation recognizable by the target platform compiler to obtain a sparse computing kernel running on a general computing platform; The description of the input sparse tensor dimension information is received through the dimension information interface, and the storage description and operation description are received through the storage description interface and the operation description interface. The specific steps are as follows: Traverse the dimensions of the sparse tensor data and determine the feature description, which includes the length, sparse or dense attribute, fixed length or variable length attribute of each dimension; if the current dimension does not contain non-zero elements, set it to dense attribute; if the current dimension contains non-zero elements, set it to sparse attribute, and determine the name and bit width of the index array; if the number of elements in the current dimension is fixed, set it to fixed length attribute, if it is sparse at the same time, determine the number of non-zero elements in a fiber in the current dimension; if the number of elements is not fixed, set it to variable length attribute, and determine the name and bit width of the pointer array; If both are sparse, determine the number of all non-zero elements in the current dimension; Check the connection characteristics between dimensions and set relationships, including sharing relationships, fusion relationships, and inheritance relationships; If the number and value of subtree nodes in the same layer of the sparse tensor index tree containing non-zero elements are the same, the dimension is set to be shared by the upper layer dimension; If the number of nodes in a layer containing non-zero elements in the sparse tensor index tree is the same as that in the upper layer, a fusion relationship is set for the layer and the index array of the upper layer is added; other situations except for sharing relationship and fusion relationship are set to inheritance relationship; Declare sparse storage space according to the data distribution characteristics of the sparse tensor, bind the determined dimension information to the sparse storage space, and provide the storage data name and data bit width in the sparse storage space; Declare the iterators used in the operation, bind the iterators to the sparse dimensions, use the iterators to access the sparse storage space, and perform the calculation operation; The dimension information and storage description are saved using a sparse information storage structure, and the operation description is stored using an intermediate representation. The specific steps are as follows: Create storage space for each dimension of the dimension information interface to store the sparse and variable-length attribute pairs of the current dimension, the length of the current dimension, the number of non-zero elements of the current dimension, the relationship between the current dimension and the upper dimension, the pointer array of the current dimension, and the index array of the current dimension; Create a storage space for the sparse storage interface and store the specific dimensions in the dimension information interface bound to the storage space; Store non-zero element array; Create storage space for the sparse iterator and declare the sparse dimensions bound to the sparse iterator; Store the computation code and use TensorIR as the intermediate representation of the computation code storage; The saved sparse information is used to translate the computation description and generate the decoding memory access statement in the sparse computation. The specific steps are as follows: Traverse the dimension information and generate auxiliary sparse storage space; Traverse the sparse iterator and generate nested loops around the intermediate expressions in the calculation code according to the order of the sparse iterator; Traverse the intermediate representation code and replace the iterators of the sparse attribute dimensions bound by the intermediate representation code with the index array. For the iterators bound to the dense attribute dimensions but the dimensions are fused with the lower layer, replace them with the upper layer index. Traverse the intermediate representation code. For BufferLoad nodes or BufferStore nodes, match the transformed memory access index with the iteration space on the sparse storage according to the attributes of the current sparse storage dimension, and calculate the flattened index on the sparse storage space in sequence.

2. The method for implementing a high-performance sparse computing programming framework according to claim 1, characterized in that: Traversing the dimension information and generating the auxiliary sparse storage space are as follows: If the current dimension has a sparse attribute and is in a fusion relationship with the upper dimension, the current dimension index storage space and the upper dimension index storage space are generated, and the size of the index array space is the number of non-zero elements stored in the current dimension; If the current dimension has a sparse attribute and is in a shared relationship with the upper dimension, generate the current dimension index storage space. The size of the index array space is the number of non-zero elements in a fiber of the current dimension stored; If the current dimension has sparse attributes and it is not the case mentioned above, generate the current dimension index storage space, and the size of the index array space is the number of non-zero elements stored in the current dimension; If the current dimension is a variable-length attribute and the upper dimension is dense, the upper dimension of the current dimension is searched in the sparse information storage structure to form an inheritance relationship, and the length of the generated pointer array is the product of the lengths of all dimensions plus one; If the current dimension is a variable-length attribute and the upper dimension is sparse, the pointer array size is the number of non-zero elements in the upper dimension plus 1.

3. The method for implementing a high-performance sparse computing programming framework according to claim 2, characterized in that: According to the order of the sparse iterator, a nested loop is generated around the calculation code's intermediate expression. The specific steps are as follows: Bind the iterator of a dense fixed-length dimension, generate a loop step of 1, and the loop length is the length of the dimension; If it is a variable-length dimension, the generated loop step is 1, and the dimension that forms an inheritance relationship with the upper layer of the current dimension is searched in the sparse information storage structure. The current variable-length loop length as follows: ， in, Indicates that the current dimension has sparse attributes and is in a shared relationship with the upper dimension. represents the dimension length, Represents the iterator corresponding to the dimension; If it is a sparse dimension, the generated loop step is 1. When it is in a shared relationship with the upper layer, the generated loop length is the number of non-zero elements in a single fiber of the current dimension. In other scenarios, the generated loop length is the number of non-zero elements in the current dimension.

4. The method for implementing a high-performance sparse computing programming framework according to claim 3, characterized in that: The transformed access index is matched with the iteration space on the sparse storage according to the attributes of the current sparse storage dimension, as follows: If it is a dense attribute, the index on the storage space does not change; If it is a sparse attribute, the sparse dimension bound in the index is the same as the dimension bound to the sparse storage. Use the iterator bound to the sparse dimension to replace the index array to access the sparse storage. If the index contains a dimension with sparse attributes, and the sparse dimension bound in the index is different from the dimension bound by the sparse storage, the non-zero element positions of the two sparse dimensions are unioned, and the union result array is used for replacement access.

5. The method for implementing a high-performance sparse computing programming framework according to claim 4, characterized in that: The flattened index on the sparse storage space is calculated sequentially, and the flattened index is initially 0; the details are as follows: If a fixed-length dimension is bound and has an inheritance relationship with the next layer, it will not be processed; Bind the fixed-length dimension and make it a non-inherited relationship with the next layer. Flatten the index and add the index of the dimension multiplied by the number of non-zero elements of the next layer of dimensions. If it is the first dimension, multiply it by 1. Bind the dimension of the variable-length attribute, find the dimension of the upper layer of the current dimension in the sparse information storage structure to form an inheritance relationship, and flatten the index accumulation , the calculation formula is as follows: ， in, Index for this dimension.

6. The method for implementing a high-performance sparse computing programming framework according to claim 1, characterized in that: Use the memory access statements generated by the sparse-aware scheduling primitive to optimize memory access. The specific steps are as follows: Enter the code block to be cached and the storage area to be cached; Check the storage level of the storage area to be cached in the code to determine whether the current cache level is at the top level. If it is not the top level, then determine its corresponding upper-level high-speed storage unit. For storage areas without special declarations, consider their storage level as memory. Traverse the intermediate representation stored in the computation code, find the input code block to be cached, and determine the position of the left sibling node in the abstract syntax tree as the position to insert the cache node; Record the iterators that appear inside the code block area to be cached and add them to the active iterator variable set; Traverse the operation statements to be cached, analyze the BufferLoad nodes or BufferStore nodes, and calculate the cache operation area; specifically as follows: if the traversed index is an iterator variable, if it is in the iterator set, the range of the iterator is used as the memory access range, if it is not in the iterator set, the value of the iterator in the code block does not change and is not processed; if the traversed index is a sparse index array, if the iterator on the sparse index array is in the iterator set, the length of the sparse dimension is used as the range; if the traversed index is a pointer array, if the iterator on the pointer array is in the iterator set, search the sparse information storage structure for the upper layer of the current dimension to form an inheritance relationship, and the calculation range is the product of the lengths of all dimensions found and the length of the previous dimension; if the traversed index is other array types, the memory access range is determined according to the maximum value range of the data type in the array; Generate a data transfer code from the storage level to be cached to the upper-level cache according to the calculated memory access range, and insert the operation statement block into the determined insertion cache node position; Traverse the statements to be cached and replace the storage space access in the BufferLoad node or BufferStore node with the access to the cache area.

7. A system for a high-performance sparse computing programming framework that implements the method of any one of claims 1 to 6, characterized in that: include: An interface module, used to receive and process the dimension information description, storage description and operation description of the sparse tensor; The storage module saves the dimension information and storage description using a sparse information storage structure, and stores the operation description using an intermediate representation; A decoding memory access statement generation module uses the stored sparse information to translate the calculation description and generate a decoding memory access statement in the sparse calculation; The sparse-aware scheduling optimization module is used to optimize memory access statements and generate optimized intermediate representations. The compiler interface module is used to convert the optimized intermediate representation into an intermediate representation recognizable by the target platform compiler.

Images