Tensor quantization program analysis method and device based on machine learning, and storage medium

Abstract

The present invention discloses a tensor program analysis method and device based on machine learning, and a storage medium, and relates to the technical field of tensor program analysis. The method comprises: embedding a tensor program to be analyzed into a preset program behavior feature template, and constructing a unified feature representation corresponding to the tensor program to be analyzed; inputting the unified feature representation into a pre-trained performance evaluation model, and obtaining a performance prediction result output by the performance evaluation model. The present invention can design a program behavior feature template according to the unified program behavior of the tensor program, so as to cope with a variety of hardware platforms and efficiently obtain a unified feature representation; and then, through a preset performance evaluation model, the performance of the tensor program is analyzed according to the unified feature representation, and the performance prediction result of the tensor program to be analyzed is obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of tensor quantization program analysis, and in particular to a tensor quantization program analysis method and device based on machine learning, and a storage medium. Background Art

[0002] With the rise of artificial intelligence algorithms such as deep learning and large models, customized accelerators have been proven to be a key means to solve the performance problems of related applications. Under the trend of hardware customization, a large number of hardware accelerator platforms and hardware primitives have emerged. Tensor programs use hardware primitives on accelerators to accelerate tensor calculations. In order to make full use of the hardware primitives on the accelerator, deep learning compilers have begun to explore automatic tuning of tensor operators. The core of program tuning relies on performance evaluation methods, but there is currently a lack of performance analysis methods for tensor programs, which seriously hinders the collaborative optimization of deep learning compilers and hardware accelerators for artificial intelligence applications. Summary of the invention

[0003] The present invention provides a tensor quantization program analysis method and device, and a storage medium based on machine learning, which can analyze the performance of a tensor quantization program.

[0004] In order to solve the above technical problems, the first aspect of the present invention discloses a tensor quantization program analysis method based on machine learning, the method comprising:

[0005] Embed the tensor program to be analyzed into a preset program behavior feature template to construct a unified feature representation corresponding to the tensor program to be analyzed; the unified feature representation is used to express the features of the tensor program to be analyzed from multiple dimensions;

[0006] The unified feature representation is input into a pre-trained performance evaluation model to obtain a performance prediction result output by the performance evaluation model, and the performance prediction result is used to measure the performance of the tensor quantization program to be analyzed.

[0007] As an optional implementation, in the first aspect of the present invention, embedding the tensor program to be analyzed into a preset program behavior feature template to construct a unified feature representation corresponding to the tensor program to be analyzed includes:

[0008] Abstractly represent the three stages of data loading into dedicated storage, primitive calculation and writing back dedicated storage data corresponding to the tensor program to be analyzed, and construct a program behavior feature template based on the abstract representation of the three stages of data loading into dedicated storage, primitive calculation and writing back dedicated storage data;

[0009] The program features corresponding to the tensorized program to be analyzed are embedded into the program behavior feature template to construct a unified feature representation corresponding to the tensorized program to be analyzed; wherein the program features include key information that can be extracted from the program and is used to describe program properties and behaviors.

[0010] As an optional embodiment, in the first aspect of the present invention, the method further comprises:

[0011] According to a preset auxiliary performance evaluation model, key scheduling primitive features corresponding to the tensor program to be analyzed are obtained;

[0012] Obtaining a key abstract syntax tree feature corresponding to the tensor program to be analyzed according to the key scheduling primitive feature and the abstract syntax tree corresponding to the tensor program to be analyzed;

[0013] And, embedding the program features corresponding to the tensorized program to be analyzed into the program behavior feature template to construct a unified feature representation corresponding to the tensorized program to be analyzed includes:

[0014] The hardware primitive features and the key abstract syntax tree features corresponding to the tensor program to be analyzed are embedded in the program behavior feature template to construct a unified feature representation corresponding to the tensor program to be analyzed.

[0015] As an optional implementation, in the first aspect of the present invention, obtaining the key scheduling primitive features corresponding to the tensor quantization program to be analyzed according to the preset auxiliary performance evaluation model includes:

[0016] According to the tensor quantization program data set corresponding to the tensor quantization program to be analyzed, an initial auxiliary performance evaluation model is trained to obtain a target auxiliary performance evaluation model trained to convergence; wherein the tensor quantization program data set includes a comparison tensor quantization program having the same function as the tensor quantization program to be analyzed but different performance and performance information corresponding to each of the comparison tensor quantization programs, and the initial auxiliary performance evaluation model includes an attention layer and a plurality of fully connected layers;

[0017] According to the attention matrix of the attention layer in the target-assisted performance evaluation model, the key scheduling primitive features corresponding to the tensor program to be analyzed are determined.

[0018] As an optional implementation, in the first aspect of the present invention, before embedding the hardware primitive features and the key abstract syntax tree features corresponding to the tensor program to be analyzed into the program behavior feature template, the method further includes:

[0019] Construct an abstract syntax tree corresponding to the tensor program to be analyzed, extract a sub-abstract syntax tree for representing hardware primitive features from the abstract syntax tree based on a feature definition method in the abstract syntax tree, and determine that the sub-abstract syntax tree is the hardware primitive feature corresponding to the tensor program to be analyzed.

[0020] As an optional embodiment, in the first aspect of the present invention, the method further comprises:

[0021] Acquire a training hardware primitive for upgrading the performance evaluation model, and perform a cross-semantic generalization operation and a cross-performance generalization operation on the performance evaluation model based on the training hardware primitive to obtain an upgraded performance evaluation model;

[0022] The cross-semantic generalization operation includes:

[0023] The performance evaluation model is trained according to the training hardware primitive, and a regularization term is added to the loss function. ,in Indicates the importance of the parameter. are the parameters that need to be learned. are the model parameters that converged on the old task;

[0024] The cross-performance generalization operation includes:

[0025] The final fully connected layer of the performance evaluation model is reinitialized according to the training hardware primitive, and the remaining model parameters remain unchanged.

[0026] As an optional implementation, in the first aspect of the present invention, the tensor quantization program data set includes:

[0027] Neural network models;

[0028] Operators, the operators comprising basic operating unit operators of the neural network model obtained by running corresponding algorithms on the neural network model;

[0029] A tensor quantization program sample, the tensor quantization program sample including a comparison tensor quantization program having the same function but different performance obtained by running automatic tuning of a tensor quantization operator on the operator;

[0030] A hardware platform, wherein the hardware platform includes at least one of a CPU, a GPU, and an NPU;

[0031] Hardware primitives, which are customized hardware acceleration cores on the hardware platform;

[0032] A performance information set, wherein the performance information set includes a tensor quantization program sample with a delay tag obtained by running the tensor quantization program sample on the corresponding hardware primitive and the hardware platform.

[0033] A second aspect of the present invention discloses a tensor quantization program analysis device based on machine learning, the device comprising:

[0034] A behavior template module, used to embed the tensor program to be analyzed into a preset program behavior feature template, and construct a unified feature representation corresponding to the tensor program to be analyzed; the unified feature representation is used to express the features of the tensor program to be analyzed from multiple dimensions;

[0035] A performance evaluation module is used to input the unified feature representation into a pre-trained performance evaluation model to obtain a performance prediction result output by the performance evaluation model, and the performance prediction result is used to measure the performance of the tensor quantization program to be analyzed.

[0036] As an optional implementation, in the second aspect of the present invention, the behavior template module embeds the tensor program to be analyzed into a preset program behavior feature template, and constructs a specific method of unified feature representation corresponding to the tensor program to be analyzed, including:

[0037] Abstractly represent the three stages of data loading into dedicated storage, primitive calculation and writing back dedicated storage data corresponding to the tensor program to be analyzed, and construct a program behavior feature template based on the abstract representation of the three stages of data loading into dedicated storage, primitive calculation and writing back dedicated storage data;

[0038] The program features corresponding to the tensorized program to be analyzed are embedded into the program behavior feature template to construct a unified feature representation corresponding to the tensorized program to be analyzed; wherein the program features include key information that can be extracted from the program and is used to describe program properties and behaviors.

[0039] As an optional implementation, in the second aspect of the present invention, the device further includes:

[0040] A key primitive analysis module, used to obtain key scheduling primitive features corresponding to the tensor program to be analyzed according to a preset auxiliary performance evaluation model;

[0041] A key syntax tree module, used to obtain a key abstract syntax tree feature corresponding to the tensor program to be analyzed according to the key scheduling primitive feature and the abstract syntax tree corresponding to the tensor program to be analyzed;

[0042] And, the behavior template module embeds the program features corresponding to the tensor program to be analyzed into the program behavior feature template to construct a specific method of unified feature representation corresponding to the tensor program to be analyzed, including:

[0043] The hardware primitive features and the key abstract syntax tree features corresponding to the tensor program to be analyzed are embedded in the program behavior feature template to construct a unified feature representation corresponding to the tensor program to be analyzed.

[0044] As an optional implementation, in the second aspect of the present invention, the key primitive analysis module obtains the key scheduling primitive features corresponding to the tensor program to be analyzed according to the preset auxiliary performance evaluation model in a specific manner, including:

[0045] According to the tensor quantization program data set corresponding to the tensor quantization program to be analyzed, an initial auxiliary performance evaluation model is trained to obtain a target auxiliary performance evaluation model trained to convergence; wherein the tensor quantization program data set includes a comparison tensor quantization program having the same function as the tensor quantization program to be analyzed but different performance and performance information corresponding to each of the comparison tensor quantization programs, and the initial auxiliary performance evaluation model includes an attention layer and a plurality of fully connected layers;

[0046] According to the attention matrix of the attention layer in the target-assisted performance evaluation model, the key scheduling primitive features corresponding to the tensor program to be analyzed are determined.

[0047] As an optional implementation, in the second aspect of the present invention, the device further includes:

[0048] A sub-syntax tree module is used to construct an abstract syntax tree corresponding to the tensor program to be analyzed before the behavior template module embeds the hardware primitive features and the key abstract syntax tree features corresponding to the tensor program to be analyzed into the program behavior feature template, extract a sub-abstract syntax tree for representing the hardware primitive features from the abstract syntax tree based on the feature definition method in the abstract syntax tree, and determine that the sub-abstract syntax tree is the hardware primitive features corresponding to the tensor program to be analyzed.

[0049] As an optional implementation, in the second aspect of the present invention, the device further includes:

[0050] A model upgrade module, used to obtain a training hardware primitive for upgrading the performance evaluation model, and perform a cross-semantic generalization operation and a cross-performance generalization operation on the performance evaluation model based on the training hardware primitive to obtain an upgraded performance evaluation model;

[0051] The cross-semantic generalization operation includes:

[0052] The performance evaluation model is trained according to the training hardware primitive, and a regularization term is added to the loss function. ,in Indicates the importance of the parameter. are the parameters that need to be learned. are the model parameters that converged on the old task;

[0053] The cross-performance generalization operation includes:

[0054] The final fully connected layer of the performance evaluation model is reinitialized according to the training hardware primitive, and the remaining model parameters remain unchanged.

[0055] As an optional implementation, in the second aspect of the present invention, the tensor quantization program data set includes:

[0056] Neural network models;

[0057] Operators, the operators comprising basic operating unit operators of the neural network model obtained by running corresponding algorithms on the neural network model;

[0058] A tensor quantization program sample, the tensor quantization program sample including a comparison tensor quantization program having the same function but different performance obtained by running automatic tuning of a tensor quantization operator on the operator;

[0059] A hardware platform, wherein the hardware platform includes at least one of a CPU, a GPU, and an NPU;

[0060] Hardware primitives, which are hardware instructions or hardware operations corresponding to the hardware platform;

[0061] A performance information set, wherein the performance information set includes a tensor quantization program sample with a delay tag obtained by running the tensor quantization program sample on the corresponding hardware primitive and the hardware platform.

[0062] The third aspect of the present invention discloses another tensor quantization program analysis device based on machine learning, the device comprising:

[0063] A memory storing executable program code;

[0064] a processor coupled to the memory;

[0065] The processor calls the executable program code stored in the memory to execute the tensor quantization program analysis method based on machine learning disclosed in the first aspect of the present invention.

[0066] The fourth aspect of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute the tensor quantization program analysis method based on machine learning disclosed in the first aspect of the present invention.

[0067] Compared with the prior art, the present invention has the following beneficial effects:

[0068] The present invention embeds the tensor program to be analyzed into a preset program behavior feature template, and constructs a unified feature representation corresponding to the tensor program to be analyzed; the unified feature representation is used to express the features of the tensor program to be analyzed from multiple dimensions; the unified feature representation is input into a pre-trained performance evaluation model to obtain a performance prediction result output by the performance evaluation model, and the performance prediction result is used to measure the performance of the tensor program to be analyzed. It can be seen that the present invention can design a program behavior feature template according to the unified program behavior of the tensor program, so as to cope with a variety of hardware platforms and efficiently obtain a unified feature representation; then, through the preset performance evaluation model, the performance of the tensor program is analyzed according to the unified feature representation, and the performance prediction result of the tensor program to be analyzed is obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0070] Figure 1 It is a flowchart of a tensor quantization program analysis method based on machine learning disclosed in an embodiment of the present invention;

[0071] Figure 2 is a schematic diagram of an operation of determining key abstract syntax tree features using an attention matrix disclosed in an embodiment of the present invention;

[0072] Figure 3 is a schematic diagram of a generalization problem in a diverse and differentiated hardware environment of orthogonal decomposition disclosed in an embodiment of the present invention;

[0073] Figure 4 is a schematic diagram of improving the generalization ability of a performance evaluation model based on a machine learning strategy disclosed in an embodiment of the present invention;

[0074] Figure 5 is a schematic diagram of a data set structure of a tensor quantization program disclosed in an embodiment of the present invention;

[0075] Figure 6 It is a structural schematic diagram of a tensor quantization program analysis device based on machine learning disclosed in an embodiment of the present invention;

[0076] Figure 7 It is a structural schematic diagram of another tensor quantization program analysis device based on machine learning disclosed in an embodiment of the present invention;

[0077] Figure 8 It is a structural schematic diagram of another tensor program analysis device based on machine learning disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0078] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0079] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or end including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or ends.

[0080] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0081] The present invention discloses a tensor program analysis method and device based on machine learning, and a storage medium. The main inventive concept is: according to the unified program behavior of the tensor program, a program behavior feature template is designed; through a preset performance evaluation model, the performance of the tensor program is analyzed according to the unified feature representation, and the performance prediction result of the tensor program to be analyzed is obtained.

[0082] Embodiment 1

[0083] See also Figure 1 , Figure 1 is a flow chart of a tensor quantization program analysis method based on machine learning disclosed in an embodiment of the present invention. Figure 1 The described tensor program analysis method based on machine learning can be applied to a tensor program analysis device based on machine learning, and the tensor program analysis device based on machine learning can be integrated in a cloud server or a local server, which is not limited in the embodiments of the present invention. Figure 1 As shown, the tensor quantization program analysis method based on machine learning may include the following operations:

[0084] Step 101: embed the tensor program to be analyzed into a preset program behavior feature template to construct a unified feature representation corresponding to the tensor program to be analyzed.

[0085] Existing program performance evaluation methods ignore the characteristics of tensor programs and the diversity of their operating environments. Therefore, they are difficult to accurately evaluate the relative performance of tensor programs, difficult to generalize in complex operating environments with diverse hardware platforms and hardware primitives, and difficult to adapt to the rapid development of customized hardware accelerators.

[0086] There is a huge search space for automatic tuning of tensor operators for specific computational loads. Due to scheduling differences, all points in the space are programs with the same computational function but different performance. Due to the rapid development of customized hardware accelerators, the tensor program space has hardware platform diversity, hardware primitive diversity, and scheduling diversity. Deep learning compilers need to search for the best performing tensor program in this vast space. As the core of program tuning, the design of performance evaluation methods is crucial to searching for the optimal program.

[0087] Optionally, the tensor program in the tensor program space constructed by automatic tuning of the tensor operator is presented in the intermediate representation of the deep learning compiler. The intermediate representation of the mainstream deep learning compiler can adopt an abstract syntax tree structure.

[0088] In an embodiment of the present invention, a unified feature representation is used to express the features of a tensor program to be analyzed from multiple dimensions; the tensor program is converted into a unified feature representation for tensor program performance evaluation, thereby coping with the hardware platform diversity, hardware primitive diversity, and scheduling diversity of the tensor program space.

[0089] According to the uniform program behavior of tensor programs, the embodiment of the present invention designs a program behavior feature template. This fixed program behavior does not change with the hardware platform. Therefore, by constructing a program behavior template, tensor programs running on different platforms can be uniformly represented, thereby coping with the diversity of hardware platforms. The embodiment of the present invention embeds the tensor program to be analyzed into a preset program behavior feature template, thereby constructing a unified feature representation corresponding to the tensor program to be analyzed.

[0090] Step 102: input the unified feature representation into a pre-trained performance evaluation model to obtain a performance prediction result output by the performance evaluation model.

[0091] In an embodiment of the present invention, the performance prediction result is used to measure the performance of the tensor quantization program to be analyzed, wherein the performance evaluation model is obtained after training with a predetermined training data set, and the training data set may be a unified feature representation corresponding to multiple preset tensor quantization programs and a performance indicator corresponding to each tensor quantization program. Optionally, the performance indicator may be the delay of the program.

[0092] It can be seen that the tensor program analysis method based on machine learning in the embodiment of the present invention can design a program behavior feature template according to the unified program behavior of the tensor program, so as to efficiently obtain a unified feature representation to cope with a variety of hardware platforms; and then through a preset performance evaluation model, the performance of the tensor program is analyzed according to the unified feature representation to obtain the performance prediction result of the tensor program to be analyzed.

[0093] In an optional embodiment, embedding the tensor program to be analyzed into a preset program behavior feature template to construct a unified feature representation corresponding to the tensor program to be analyzed may include:

[0094] Abstractly represent the three stages of data loading into dedicated storage, primitive calculation and data writing back to dedicated storage corresponding to the tensor program to be analyzed, and construct a program behavior feature template based on the abstract representation of the three stages of data loading into dedicated storage, primitive calculation and data writing back to dedicated storage;

[0095] The program features corresponding to the tensorized program to be analyzed are embedded into the program behavior feature template to construct a unified feature representation corresponding to the tensorized program to be analyzed; wherein the program features include key information that can be extracted from the program and is used to describe the program attributes and behaviors.

[0096] In this optional embodiment, the cross-platform unified program behavior is abstracted into a program behavior feature template. The tensor program has a unified program behavior, which can be abstracted into a three-stage execution model: data loading dedicated storage, primitive calculation, and dedicated storage data writeback. This fixed program behavior does not change with the hardware platform. Therefore, by constructing a program behavior template, the tensor program running on different platforms can be uniformly represented, thereby coping with the diversity of hardware platforms. This optional embodiment describes any tensor program as a combination of three stages: data loading dedicated storage, primitive calculation, and dedicated storage data writeback, and abstracts it into a unified program behavior feature template; based on the template, the features of different hardware primitives and different scheduling are embedded to construct a unified feature representation.

[0097] It can be seen that this optional embodiment can abstractly represent the tensor program to be analyzed from three stages: data loading dedicated storage, primitive calculation, and dedicated storage data writing back, and perform fixed operations in the form of program behavior feature templates. Subsequently, it is only necessary to embed the program features corresponding to the tensor program to be analyzed into the program behavior feature template to construct a unified feature representation corresponding to the tensor program to be analyzed. On the one hand, it expands the scope of application of tensor program analysis based on machine learning, and on the other hand, it improves the efficiency of tensor program analysis based on machine learning.

[0098] In another optional embodiment, the method may further include:

[0099] According to the preset auxiliary performance evaluation model, the key scheduling primitive features corresponding to the tensor program to be analyzed are obtained;

[0100] According to the key scheduling primitive features and the abstract syntax tree corresponding to the tensor program to be analyzed, a key abstract syntax tree feature corresponding to the tensor program to be analyzed is obtained;

[0101] And, embedding the program features corresponding to the tensorized program to be analyzed into the program behavior feature template, and constructing a unified feature representation corresponding to the tensorized program to be analyzed, including:

[0102] The hardware primitive features and key abstract syntax tree features corresponding to the tensor program to be analyzed are embedded in the program behavior feature template to construct a unified feature representation corresponding to the tensor program to be analyzed.

[0103] In this optional embodiment, the program features include key information that can be extracted from the program and used to describe the program attributes and behaviors, and the program features mainly include hardware primitive features and scheduling primitive features. For scheduling primitive features, different scheduling primitives have different effects on the performance of the tensor program to be analyzed, and it is necessary to screen out key scheduling primitive features with high impact, so as to improve the efficiency of tensor program analysis.

[0104] This optional embodiment obtains the key scheduling primitive features corresponding to the tensor program to be analyzed according to a preset auxiliary performance evaluation model, and obtains the key abstract syntax tree features corresponding to the tensor program to be analyzed according to the key scheduling primitive features and the abstract syntax tree corresponding to the tensor program to be analyzed: the abstract syntax tree transformation introduced by the key scheduling is the key abstract syntax tree feature that affects the performance of the tensor program, that is, the key scheduling primitives are represented by means of an abstract syntax tree, thereby providing more efficient data for subsequent operations.

[0105] It can be seen that this optional embodiment can screen out key scheduling primitive features with high influence, and express the key scheduling primitive features in the form of an abstract syntax tree, so as to more efficiently and reasonably construct a unified feature representation corresponding to the tensor program to be analyzed, thereby improving the efficiency and accuracy of tensor program analysis.

[0106] In yet another optional embodiment, obtaining key scheduling primitive features corresponding to the tensor quantization program to be analyzed according to a preset auxiliary performance evaluation model may include:

[0107] According to the tensor quantization program data set corresponding to the tensor quantization program to be analyzed, the initial auxiliary performance evaluation model is trained to obtain a target auxiliary performance evaluation model that is trained to convergence;

[0108] According to the attention matrix of the attention layer in the target-assisted performance evaluation model, the key scheduling primitive features corresponding to the tensor program to be analyzed are determined.

[0109] In this optional embodiment, key scheduling primitive features are obtained through a preset auxiliary performance evaluation model, wherein the tensor quantization program data set includes comparison tensor quantization programs with the same function as the tensor quantization program to be analyzed but different performance and performance information corresponding to each comparison tensor quantization program, and the initial auxiliary performance evaluation model includes an attention layer and several fully connected layers.

[0110] In this optional embodiment, an initial auxiliary performance evaluation model is trained on a tensor program data set, including only one attention layer and multiple fully connected layers. The initial auxiliary performance evaluation model takes the scheduling primitive features as input and outputs the performance prediction results for the tensor program. After the initial auxiliary performance evaluation model converges, the target auxiliary performance evaluation model is obtained, and the key scheduling primitives are determined by the attention matrix of the attention layer in the target auxiliary performance evaluation model. Optionally, the scheduling primitives with high attention scores in the attention matrix are key scheduling primitives that affect performance.

[0111] It can be seen that this optional embodiment can train the initial auxiliary performance evaluation model according to the tensor program data set corresponding to the tensor program to be analyzed, obtain the target auxiliary performance evaluation model trained to convergence, and finally determine the key scheduling primitives through the attention matrix of the attention layer in the target auxiliary performance evaluation model. The scheduling attention matrix is ​​used to mine the key abstract syntax tree features that affect the performance of the tensor program, improve the accuracy of evaluating the relative relationship of the tensor program performance, and thus more accurately determine the key scheduling primitives.

[0112] In yet another optional embodiment, Figure 2 As shown, according to the attention matrix of the attention layer in the target-assisted performance evaluation model, determining the key scheduling primitive features corresponding to the tensorization program to be analyzed may include the following operations:

[0113] Figure 2 This is a flowchart of using the scheduling attention matrix to mine the key abstract syntax tree features that affect the performance of tensor programs. The key scheduling primitives include Split, SamplePerfectTile, SampleCategorical, CacheWrite, and ComputeInline. Since scheduling generates programs with different performance by transforming the abstract syntax tree, the abstract syntax tree transformation introduced by the above key scheduling is the key abstract syntax tree feature that affects the performance of tensor programs.

[0114] The SamplePerfectTile scheduling primitive and the Split scheduling primitive jointly determine the slicing, which transforms the length of the loop in the abstract syntax tree. After the Split scheduling primitive, the loop iteration changes from (128, 128) to (2, 64, 128); in addition, slicing also transforms the affine of the loop, just as the access to the first dimension of Buffer A is transformed from i to i0×64+i1. Such affine transformation is particularly critical to the performance of programs in parallel scenarios. For example, 64 threads process a piece of data. When the data is sliced ​​into 64 or 128 pieces, the data processed by each thread is different, which is reflected by the affine. The integers sampled by the SampleCategorical scheduling primitive are applied to vectorization, unrolling, parallelization, etc., which will transform the optimization bound to the loop in the abstract syntax tree. The Bind scheduling primitive selects the iteration to which the thread is bound according to the instructions of SampleCategorical. Other key scheduling primitives are related to program behavior. For example, the CacheWrite scheduling primitive and the ComputeInline scheduling primitive will increase and decrease the behavior block respectively. In summary, the key abstract syntax tree features that affect the performance of tensorized programs include loops, loop binding optimization, and affine.

[0115] Based on this optional embodiment, further optionally, constructing a unified feature representation corresponding to the tensor quantization program to be analyzed may include the following operations:

[0116] The unified program behavior template of tensorized programs can be a sequence of three behaviors: load, compute, and store, which comes from the simplified execution model of tensorized programs. Each behavior is represented by three vectors, all of which start with a one-hot vector to indicate the behavior type.

[0117] The first vector is loopnest, which represents all loops of the current behavior; each of the four dimensions describes a layer of loops, namely loopid, extent, parallelism, and binding scheduling.

[0118] The second vector is affine, which stands for affine. It consists of loops and five operations. Affine may appear in multiple dimensions of multiple buffers. It is encoded from left to right in the order from write buffer to read buffer. The vectors loopnest and affine come from the mining of the scheduling attention matrix in this optional embodiment.

[0119] The third vector is buffer or intrinsic, depending on whether the current behavior relies on hardware primitives. The vector buffer consists of arithops and loop_touched_bytes, which are used to describe the computational intensity of the innermost assignment statement of the loop and the number of bytes accessed by each buffer. The vector intrinsic consists of semantics and implementation. The semantics describes the computational function, and the implementation describes the call to the underlying acceleration core. The sub-abstract syntax tree of the semantics can be further represented as a vector loopnest and a vector buffer.

[0120] In this optional embodiment, the unified features for tensor program performance evaluation are scalable and can be extended to three types of features: the implementation of hardware primitives, such as throughput, frequency, and power consumption; more scheduling in deep learning compilers; and abstract syntax tree features for buffer access. The feature extractor is implemented based on the abstract syntax tree intermediate representation TensorIR of the deep learning compiler TVM, but can be generalized to the abstract syntax tree intermediate representation of any deep learning compiler.

[0121] Further optionally, the process of extracting features based on the unified feature representation of the tensor quantization program performance evaluation may include:

[0122] S510, extracting computational behavior abstract syntax tree features from the tensorization program.

[0123] S520: Calculate the encoding step of the abstract syntax tree feature of the behavior.

[0124] S530: Calculate the final feature code of the behavior.

[0125] In this optional embodiment, the abstract syntax tree features of S510 are extracted from the abstract syntax tree of the deep learning compiler, and S520 is the encoding process of the abstract syntax tree features of the computational behavior of S510.

[0126] Based on this optional embodiment, further optionally, the structure of the performance evaluation model based on deep learning includes an attention layer and a fully connected layer, and at least includes an attention layer for learning the connection between program behaviors. In the inference phase of the performance evaluation model based on deep learning, the performance evaluation result of the tensor program is a normalized score; the normalized score refers to the delay of the best performance tensor program in the tensor program data set generated by the automatic tuning of the tensor operator divided by the delay of the current tensor program. In the training phase, the performance evaluation model based on deep learning uses ranking loss.

[0127] In yet another optional embodiment, before embedding the hardware primitive features and key abstract syntax tree features corresponding to the tensor program to be analyzed into the program behavior feature template, the method may further include:

[0128] An abstract syntax tree corresponding to the tensor program to be analyzed is constructed. Based on the feature definition method in the abstract syntax tree, a sub-abstract syntax tree for representing the hardware primitive features is extracted from the abstract syntax tree, and the sub-abstract syntax tree is determined to be the hardware primitive feature corresponding to the tensor program to be analyzed.

[0129] In this optional embodiment, the hardware primitive features are included as sub-abstract syntax trees in the complete tensorized program abstract syntax tree, so the same feature definition method can be reused, avoiding additional design efforts while ensuring information alignment. As a comprehensive program abstraction, the abstract syntax tree can not only intuitively display program behavior, but also contain hardware primitive information. Therefore, extracting features from the abstract syntax tree can more efficiently represent hardware primitive features.

[0130] In yet another optional embodiment, the method may further include:

[0131] Acquire a training hardware primitive for upgrading the performance evaluation model, perform a cross-semantic generalization operation and a cross-performance generalization operation on the performance evaluation model based on the training hardware primitive, and obtain an upgraded performance evaluation model;

[0132] Among them, cross-semantic generalization operations include:

[0133] Train the performance evaluation model based on the training hardware primitives and add a regularization term to the loss function ,in Indicates the importance of the parameter. are the parameters that need to be learned. are the model parameters that converged on the old task;

[0134] Cross-performance generalization operations include:

[0135] The last fully connected layer of the performance evaluation model is reinitialized according to the training hardware primitives, and the remaining model parameters remain unchanged.

[0136] The operating environment of tensor quantization programs has both hardware primitive diversity and hardware platform diversity, which poses a great challenge to the generalization ability of performance evaluation models on differentiated hardware. Figure 3 A schematic diagram of the orthogonal decomposition of the generalization problem in a diverse and differentiated hardware environment is shown. In general, the hardware primitives corresponding to different platforms are as follows:

[0137] X86 hardware primitives: i3.avx512.dot.int8; i4,vnni,dot,int8;

[0138] ARM hardware primitives: i5,neon,dot,int8; i6.sdot.dot.int8;

[0139] GPU hardware primitives: i1, tscore, m8n32k16; i2, tscore, m16n16k16;

[0140] There are two main directions for hardware manufacturers to develop hardware primitives: (1) Extending semantics to flexibly support computing loads, including different semantics on the same platform, such as Tensor Core supports matrix multiplication of multiple data types, shapes and layouts, and X86AVX512 supports point multiplication operations of multiple data types and different semantics on different platforms; (2) Improving performance while keeping the semantics unchanged, generally by changing the implementation of hardware primitives through microarchitecture upgrades, for example, from AVX512 to VNNI on X86, and from NEON to SDOT on ARM. In this optional embodiment, based on the above development rules, the originally complex generalization problem is decomposed into two orthogonal sub-problems.

[0141] In this optional embodiment, for semantic expansion, how to make the performance evaluation model support new semantic predictions flexibly and at low cost while always maintaining the generalization ability of multiple semantics, such problems are called cross-semantic generalization. For performance improvement with unchanged semantics, how to make the performance evaluation model originally trained on low-performance implementation migrate to high-performance implementation flexibly and at low cost, that is, let the model perceive the impact of changes in hardware primitive implementation on optimal scheduling, such problems are called cross-performance generalization.

[0142] Figure 4 A schematic diagram of improving the generalization ability of a performance evaluation model based on a machine learning strategy is shown. In this optional embodiment, cross-performance represents hardware primitive upgrades, and generally only the prediction after the upgrade is concerned, so cross-performance generalization can be solved based on transfer learning.

[0143] In this optional embodiment, cross-semantics needs to take into account both the existing massive semantics and possible new semantics in the future. There are multiple optional machine learning strategies. The premise of multi-task learning is that a data sample needs to have labels on multiple tasks. However, a tensorization program on one semantic cannot be run on other semantics to obtain running delay labels, so multi-task learning cannot be applied to cross-semantic generalization scenarios; transfer learning has catastrophic forgetting, and the performance evaluation model will forget the old semantics when migrating to new semantics, resulting in poor flexibility; lifelong learning as an incremental learning method is suitable for cross-semantic scenarios. The performance evaluation model will not forget the old semantics when learning new semantics, which greatly reduces storage and training costs and can flexibly respond to new semantic expansion.

[0144] In this optional embodiment, a fine-tuning-based transfer learning method solves the cross-performance generalization problem. Specifically, the model is pre-trained on hardware primitives with low performance implementation. After the hardware primitives are upgraded, their semantically unchanged performance is improved. The last fully connected layer of the performance evaluation model is reinitialized, and the remaining model parameters remain unchanged. The performance evaluation model is then fine-tuned on hardware primitives with high performance implementation, and the model parameters are used to perceive changes in hardware implementation.

[0145] In this optional embodiment, a lifelong learning method based on selective synaptic remodeling is optionally used to solve the cross-semantic generalization problem. Specifically, it is a regularization-based method whose basic assumption is that some parameters in the model are important for the old tasks, so when training the new task, only the parameters that are not important for the old tasks are changed. The performance evaluation model uses a basic ranking loss on the first semantic task, and the subsequent semantic expansion adds a regularization term to the loss function. ,in Indicates the importance of the parameter. are the parameters that need to be learned. are the model parameters that converged on the old task.

[0146] It can be seen that this optional embodiment improves the generalization ability of the performance evaluation method in diverse and differentiated hardware environments by constructing a unified feature representation for tensor program performance evaluation and adopting a machine learning strategy based on lifelong learning and transfer learning, thereby making the performance evaluation method adapt to the rapid development of customized hardware accelerators.

[0147] In yet another optional embodiment, the tensor quantization program data set may include:

[0148] Neural network models;

[0149] Operators, including basic operation unit operators of the neural network model obtained by running corresponding algorithms on the neural network model;

[0150] Tensor quantization program samples, which include comparison tensor quantization programs that have the same functions but different performances obtained by running automatic tuning of tensor quantization operators on operators;

[0151] A hardware platform, which includes at least one of a CPU, a GPU, and an NPU;

[0152] Hardware primitives: Hardware primitives are customized hardware acceleration cores on hardware platforms;

[0153] The performance information set includes running the tensor quantization program sample on the corresponding hardware primitive and hardware platform to obtain the tensor quantization program sample with delay label.

[0154] The tensor program data set in this optional embodiment may include: neural network models, common computer vision models and natural language processing models; operators, running the graph partitioning algorithm on the neural network model to obtain the basic operating unit operator of the neural network model; tensor program samples, running the tensor operator automatic tuning on the operator to obtain tensor programs with the same functions but different performance; hardware primitives and hardware platforms, the hardware platform includes CPU, GPU and NPU, and the hardware primitive is a customized hardware acceleration core on the hardware platform. Tensor program samples with delay labels, running the tensor program samples on the corresponding hardware primitives and hardware platforms to obtain tensor program samples with delay labels.

[0155] Figure 5 The structure of the tensor program data set in this optional embodiment is shown. Tensor programs with the same semantics in the data set share the same schedule, but can also be different; tensor programs with different semantics have different schedules. The network structure includes computer vision (CV) and natural language processing (NLP) models with different output shapes. For each set of neural networks and hardware primitives, run the graph partitioning algorithm to obtain operators; for each operator, use TVM MetaSchedule to generate tensor program samples; run all tensor program samples to obtain tensor program samples with delay labels.

[0156] In this optional embodiment, the tensor quantization program data set includes common operators, and program tuning is based on operators as basic units, so the performance evaluation model trained and tested on this data set can be extended to complex network structures constructed by basic operators.

[0157] Embodiment 2

[0158] See also Figure 6 , Figure 6 Schematic diagram of a tensor quantization program analysis device based on machine learning disclosed in an embodiment of the present invention. Figure 6As shown, the tensor quantization program analysis device based on machine learning may include:

[0159] The behavior template module 201 is used to embed the tensor program to be analyzed into a preset program behavior feature template to construct a unified feature representation corresponding to the tensor program to be analyzed; the unified feature representation is used to express the features of the tensor program to be analyzed from multiple dimensions;

[0160] The performance evaluation module 202 is used to input the unified feature representation into a pre-trained performance evaluation model to obtain a performance prediction result output by the performance evaluation model, and the performance prediction result is used to measure the performance of the tensorization program to be analyzed.

[0161] The machine learning-based tensor program analysis device in the embodiment of the present invention can design a program behavior feature template based on the unified program behavior of the tensor program, so as to efficiently obtain a unified feature representation for a variety of hardware platforms; and then, through a preset performance evaluation model, the performance of the tensor program is analyzed based on the unified feature representation to obtain a performance prediction result of the tensor program to be analyzed.

[0162] In an optional embodiment, the behavior template module 201 embeds the tensor program to be analyzed into a preset program behavior feature template, and the specific manner of constructing a unified feature representation corresponding to the tensor program to be analyzed may include:

[0163] Abstractly represent the three stages of data loading into dedicated storage, primitive calculation and data writing back to dedicated storage corresponding to the tensor program to be analyzed, and construct a program behavior feature template based on the abstract representation of the three stages of data loading into dedicated storage, primitive calculation and data writing back to dedicated storage;

[0164] The program features corresponding to the tensorized program to be analyzed are embedded in the program behavior feature template to construct a unified feature representation corresponding to the tensorized program to be analyzed; wherein the program features may include key information that can be extracted from the program and is used to describe the program attributes and behaviors.

[0165] This optional embodiment can abstractly represent the tensor program to be analyzed from three stages: data loading dedicated storage, primitive calculation, and dedicated storage data writing back, and perform fixed operations in the form of program behavior feature templates. Subsequently, it is only necessary to embed the program features corresponding to the tensor program to be analyzed into the program behavior feature template to construct a unified feature representation corresponding to the tensor program to be analyzed. On the one hand, it expands the scope of application of tensor program analysis based on machine learning, and on the other hand, it improves the efficiency of tensor program analysis based on machine learning.

[0166] In another optional embodiment, Figure 7 As shown, the device may also include:

[0167] The key primitive analysis module 203 is used to obtain key scheduling primitive features corresponding to the tensor program to be analyzed according to a preset auxiliary performance evaluation model;

[0168] A key syntax tree module 204 is used to obtain key abstract syntax tree features corresponding to the tensor program to be analyzed according to the key scheduling primitive features and the abstract syntax tree corresponding to the tensor program to be analyzed;

[0169] Furthermore, the behavior template module 201 embeds the program features corresponding to the tensorized program to be analyzed into the program behavior feature template, and the specific manner of constructing the unified feature representation corresponding to the tensorized program to be analyzed may include:

[0170] The hardware primitive features and key abstract syntax tree features corresponding to the tensor program to be analyzed are embedded in the program behavior feature template to construct a unified feature representation corresponding to the tensor program to be analyzed.

[0171] This optional embodiment can screen out key scheduling primitive features with high influence, and express the key scheduling primitive features in the form of an abstract syntax tree, so as to more efficiently and reasonably construct a unified feature representation corresponding to the tensor program to be analyzed, thereby improving the efficiency and accuracy of tensor program analysis.

[0172] In another optional embodiment, the specific method of obtaining the key scheduling primitive features corresponding to the tensor program to be analyzed by the key primitive analysis module 203 according to the preset auxiliary performance evaluation model may include:

[0173] According to the tensor quantization program data set corresponding to the tensor quantization program to be analyzed, the initial auxiliary performance evaluation model is trained to obtain a target auxiliary performance evaluation model that is trained to convergence; wherein the tensor quantization program data set may include a comparison tensor quantization program having the same function as the tensor quantization program to be analyzed but different performance and performance information corresponding to each comparison tensor program, and the initial auxiliary performance evaluation model may include an attention layer and a plurality of fully connected layers;

[0174] According to the attention matrix of the attention layer in the target-assisted performance evaluation model, the key scheduling primitive features corresponding to the tensor program to be analyzed are determined.

[0175] This optional embodiment can train the initial auxiliary performance evaluation model according to the tensor program data set corresponding to the tensor program to be analyzed, obtain the target auxiliary performance evaluation model trained to convergence, and finally determine the key scheduling primitives through the attention matrix of the attention layer in the target auxiliary performance evaluation model. The scheduling attention matrix is ​​used to mine the key abstract syntax tree features that affect the performance of the tensor program, improve the accuracy of evaluating the relative relationship of the tensor program performance, and thus more accurately determine the key scheduling primitives.

[0176] In yet another optional embodiment, the device may further include:

[0177] The sub-syntax tree module 205 is used to construct an abstract syntax tree corresponding to the tensor program to be analyzed before the behavior template module 201 embeds the hardware primitive features and key abstract syntax tree features corresponding to the tensor program to be analyzed into the program behavior feature template, and based on the feature definition method in the abstract syntax tree, extract the sub-abstract syntax tree used to represent the hardware primitive features from the abstract syntax tree, and determine the sub-abstract syntax tree as the hardware primitive features corresponding to the tensor program to be analyzed.

[0178] In this optional embodiment, the hardware primitive features are included as sub-abstract syntax trees in the complete tensorized program abstract syntax tree, so the same feature definition method can be reused, avoiding additional design efforts while ensuring information alignment. As a comprehensive program abstraction, the abstract syntax tree can not only intuitively display program behavior, but also contain hardware primitive information. Therefore, extracting features from the abstract syntax tree can more efficiently represent hardware primitive features.

[0179] In yet another optional embodiment, the device may further include:

[0180] A model upgrade module 206 is used to obtain a training hardware primitive for upgrading the performance evaluation model, and perform a cross-semantic generalization operation and a cross-performance generalization operation on the performance evaluation model based on the training hardware primitive to obtain an upgraded performance evaluation model;

[0181] Among them, cross-semantic generalization operations may include:

[0182] Train the performance evaluation model based on the training hardware primitives and add a regularization term to the loss function ,in Indicates the importance of the parameter. are the parameters that need to be learned. are the model parameters that converged on the old task;

[0183] Cross-performance generalization operations can include:

[0184] The last fully connected layer of the performance evaluation model is reinitialized according to the training hardware primitives, and the remaining model parameters remain unchanged.

[0185] This optional embodiment constructs a unified feature representation for tensor program performance evaluation and adopts a machine learning strategy based on lifelong learning and transfer learning, thereby improving the generalization ability of the performance evaluation method in diverse and differentiated hardware environments, thereby enabling the performance evaluation method to adapt to the rapid development of customized hardware accelerators.

[0186] In yet another optional embodiment, the tensor quantization program data set may include:

[0187] Neural network models;

[0188] Operators, which may include basic operating unit operators of the neural network model obtained by running corresponding algorithms on the neural network model;

[0189] A sample tensor quantization program, which may include a comparison tensor quantization program with the same functions but different performance obtained by running automatic tuning of a tensor quantization operator on an operator;

[0190] A hardware platform, which may include at least one of a CPU, a GPU, and an NPU;

[0191] Hardware primitives: Hardware primitives are hardware instructions or hardware operations corresponding to the hardware platform;

[0192] The performance information set may include running the tensor quantization program sample on the corresponding hardware primitive and hardware platform to obtain the tensor quantization program sample with delay label.

[0193] Embodiment 3

[0194] The embodiment of the present invention provides a tensor quantization operator automatic tuning framework. According to the embodiment of the present invention, the tensor quantization operator automatic tuning framework includes: tensor quantization program space generation, generating tensor quantization programs with the same functions but different performance for specific operator loads; performance evaluation, using the performance evaluation method of the present invention to predict the performance of the above tensor quantization program in a differentiated hardware environment; performance measurement, measuring the tensor quantization program selected by the performance evaluation method in an actual hardware environment to obtain a tensor quantization program with the best performance.

[0195] Furthermore, in the embodiment of the present invention, the data set simulates multiple tuning scenarios based on the tuning tasks to test the effectiveness of the performance evaluation method. The tuning tasks are shown in Table 1. m8n32k16 and others are multiple hardware primitives of GPU Tensor Core. nn represents that both the A matrix and the B matrix of matrix multiplication are not transposed. nt represents that the A matrix of matrix multiplication is not transposed and the B matrix is ​​transposed. avx512.dot and vnni.dot are two hardware primitives with the same dot multiplication semantics on the X86 platform. neon.dot and sdot.dot are two hardware primitives with the same dot multiplication semantics on the ARM platform. During the experiment, the model is trained and tested on the data set of the hardware primitives included in the task.

[0196] Table 1:

[0197]

[0198] According to an embodiment of the present invention, Task 0-Task 4 are used to simulate the automatic tuning scenario of the tensor quantization operator of a single hardware primitive on a single hardware platform; Task 5 and Task 6 are used to simulate the automatic tuning scenario of the tensor quantization operator of multiple hardware primitives on a single hardware platform; Task 7 is used to simulate the automatic tuning scenario of the tensor quantization operator of multiple hardware primitives on multiple hardware platforms; Task 9, Task 0, Task 1, Task 2 and Task 3 are used in sequence to simulate the automatic tuning scenario of the tensor quantization operator across semantic generalization; Task 6 and Task 8 are used in sequence to simulate the automatic tuning scenario of the tensor quantization operator across performance generalization.

[0199] Experiments show that compared with the prior art, the performance evaluation method of the embodiment of the present invention has the most advanced performance prediction accuracy and significantly improves the ability to cope with the diversity of operating environments. In the most advanced tensor operator automatic tuning framework, on GPU Tensor Core, X86 AVX512 / VNNI, ARM NEON / SDOT hardware primitives and platforms, the performance evaluation method of this optional embodiment accelerates the search time by an average of 9.5 times and improves the reasoning speed by an average of 2.1 times.

[0200] Embodiment 4

[0201] See also Figure 8 , Figure 8 FIG8 is a schematic diagram of a structure of another tensor program analysis device based on machine learning disclosed in an embodiment of the present invention. As shown in FIG8, the tensor program analysis device based on machine learning may include:

[0202] A memory 301 storing executable program codes;

[0203] a processor 302 coupled to the memory 301;

[0204] The processor 302 calls the executable program code stored in the memory 301 to execute the steps in the tensor quantization program analysis method based on machine learning described in the first embodiment of the present invention.

[0205] Embodiment 5

[0206] An embodiment of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute the steps in the tensor quantization program analysis method based on machine learning described in the first embodiment of the present invention.

[0207] Embodiment 6

[0208] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps in the machine learning-based tensor program analysis method described in Example 1.

[0209] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0210] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0211] Finally, it should be noted that the tensor quantization program analysis method and device based on machine learning and the storage medium disclosed in the embodiments of the present invention are only preferred embodiments of the present invention, which are only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A tensor quantization program analysis method based on machine learning, characterized in that: The method comprises: Abstractly represent the three stages of data loading into dedicated storage, primitive calculation and writing back dedicated storage data corresponding to the tensor program to be analyzed, and construct a program behavior feature template based on the abstract representation of the three stages of data loading into dedicated storage, primitive calculation and writing back dedicated storage data; According to a preset auxiliary performance evaluation model, key scheduling primitive features corresponding to the tensor program to be analyzed are obtained; Obtaining a key abstract syntax tree feature corresponding to the tensor program to be analyzed according to the key scheduling primitive feature and the abstract syntax tree corresponding to the tensor program to be analyzed; Embed the hardware primitive features and the key abstract syntax tree features corresponding to the tensor program to be analyzed into the program behavior feature template to construct a unified feature representation corresponding to the tensor program to be analyzed; wherein the unified feature representation is used to express the features of the tensor program to be analyzed from multiple dimensions; The unified feature representation is input into a pre-trained performance evaluation model to obtain a performance prediction result output by the performance evaluation model, and the performance prediction result is used to measure the performance of the tensor quantization program to be analyzed.

2. The tensor quantization program analysis method based on machine learning according to claim 1, characterized in that: The step of obtaining key scheduling primitive features corresponding to the tensor program to be analyzed according to a preset auxiliary performance evaluation model includes: According to the tensor quantization program data set corresponding to the tensor quantization program to be analyzed, an initial auxiliary performance evaluation model is trained to obtain a target auxiliary performance evaluation model trained to convergence; wherein the tensor quantization program data set includes a comparison tensor quantization program having the same function as the tensor quantization program to be analyzed but different performance and performance information corresponding to each of the comparison tensor quantization programs, and the initial auxiliary performance evaluation model includes an attention layer and a plurality of fully connected layers; According to the attention matrix of the attention layer in the target-assisted performance evaluation model, the key scheduling primitive features corresponding to the tensor program to be analyzed are determined.

3. The tensor quantization program analysis method based on machine learning according to claim 1, characterized in that: Before embedding the hardware primitive features and the key abstract syntax tree features corresponding to the tensor program to be analyzed into the program behavior feature template, the method further includes: Construct an abstract syntax tree corresponding to the tensor program to be analyzed, extract a sub-abstract syntax tree for representing hardware primitive features from the abstract syntax tree based on a feature definition method in the abstract syntax tree, and determine that the sub-abstract syntax tree is the hardware primitive feature corresponding to the tensor program to be analyzed.

4. The tensor quantization program analysis method based on machine learning according to any one of claims 1 to 3, characterized in that: The method further comprises: Acquire a training hardware primitive for upgrading the performance evaluation model, and perform a cross-semantic generalization operation and a cross-performance generalization operation on the performance evaluation model based on the training hardware primitive to obtain an upgraded performance evaluation model; The cross-semantic generalization operation includes: The performance evaluation model is trained according to the training hardware primitive, and a regularization term is added to the loss function. ,in Indicates the importance of the parameter. are the parameters that need to be learned. are the model parameters that converged on the old task; The cross-performance generalization operation includes: The final fully connected layer of the performance evaluation model is reinitialized according to the training hardware primitive, and the remaining model parameters remain unchanged.

5. The tensor quantization program analysis method based on machine learning according to claim 2, characterized in that: The tensor quantization program data set includes: Neural network models; Operators, the operators comprising basic operating unit operators of the neural network model obtained by running corresponding algorithms on the neural network model; A tensor quantization program sample, the tensor quantization program sample comprising a comparison tensor quantization program having the same function but different performance obtained by running automatic tuning of a tensor quantization operator on the operator; A hardware platform, wherein the hardware platform includes at least one of a CPU, a GPU, and an NPU; Hardware primitives, which are customized hardware acceleration cores on the hardware platform; A performance information set, wherein the performance information set includes a tensor quantization program sample with a delay tag obtained by running the tensor quantization program sample on the corresponding hardware primitive and the hardware platform.

6. A tensor quantization program analysis device based on machine learning, characterized in that: The device comprises: A key primitive analysis module is used to obtain key scheduling primitive features corresponding to the tensor program to be analyzed according to a preset auxiliary performance evaluation model; A key syntax tree module, used to obtain a key abstract syntax tree feature corresponding to the tensor program to be analyzed according to the key scheduling primitive feature and the abstract syntax tree corresponding to the tensor program to be analyzed; A behavior template module abstractly represents the three stages of data loading into dedicated storage, primitive calculation, and writing back dedicated storage data corresponding to the tensor program to be analyzed, and constructs a program behavior feature template based on the abstract representation of the three stages of data loading into dedicated storage, primitive calculation, and writing back dedicated storage data; Embed the hardware primitive features and the key abstract syntax tree features corresponding to the tensor program to be analyzed into the program behavior feature template to construct a unified feature representation corresponding to the tensor program to be analyzed; wherein the unified feature representation is used to express the features of the tensor program to be analyzed from multiple dimensions; A performance evaluation module is used to input the unified feature representation into a pre-trained performance evaluation model to obtain a performance prediction result output by the performance evaluation model, and the performance prediction result is used to measure the performance of the tensor quantization program to be analyzed.

7. A tensor quantization program analysis device based on machine learning, characterized in that: The device includes: a memory storing executable program code; a processor coupled to the memory; the processor calls the executable program code stored in the memory to execute the machine learning-based tensor program analysis method as described in any one of claims 1-5.

8. A computer storage medium, characterized in that: The computer storage medium stores computer instructions, which, when called, are used to execute the tensor quantization program analysis method based on machine learning as described in any one of claims 1-5.

Images