Search space pruning method and device for deep learning operator

Abstract

The invention provides a search space pruning method and device for a deep learning operator, and the method comprises the steps: constructing a corresponding target decision tree based on a target training set and a preset decision tree algorithm, training the target decision tree based on a preset scheduling sample, and obtaining a trained target decision tree, performing performance prediction on preset scheduling to be optimized based on the trained target decision-making tree to obtain a prediction result of the target decision-making tree, traversing all target decision-making sub-trees in the target decision-making tree based on the prediction result of the target decision-making tree, obtaining a pruning rule of each target decision-making sub-tree, and distinguishing the scheduling to be optimized based on the pruning rules to obtain the scheduling to be optimized. And determining a first schedule conforming to the pruning rule and a second schedule not conforming to the pruning rule, and performing hardware actual measurement on the first schedule and the second schedule to respectively obtain corresponding hardware actual measurement results.

Description

Technical Field

[0001] The present application relates to the field of deep learning technology, and more specifically, to a method and device for pruning the search space of a deep learning operator. Background Art

[0002] Deep learning models often contain millions or even billions of parameters, and a single run requires several G or even terabytes of floating-point operations. Such a huge amount of computing requires unprecedented computing efficiency for computing devices. In order to maximize the computing efficiency of hardware devices and maximize the computing performance of hardware, some hardware manufacturers have written hardware-specific manual operator libraries for high-frequency deep learning operators. Experienced engineers in hardware manufacturers have a full understanding of the architecture of related hardware, so the operator libraries they write have high computing performance that ordinary kernels do not have. However, the operator types and hardware adaptation range of manual operator libraries are limited, which restricts their use scenarios. In order to improve the flexibility of deep learning model deployment, deep learning compilers can automatically generate executable code for tensor programs on various hardware for various operators. In order to cope with the challenges of increasingly complex deep learning models and a wide variety of operators, deep learning compilers use a variety of different technical means to automatically implement operator tuning.

[0003] At present, to address the above problems, a widely applicable search-based operator tuning method is often used, that is, all scheduling methods of the operator are regarded as a search space, and its search goal is to find the scheduling method with the highest or approximately optimal computing performance.

[0004] However, the current search-based operator tuning method relies on a large number of scheduling traversals, resulting in a large tuning search space and a long search time, which makes the operator tuning efficiency low. In addition, compared with the convenience of manual operator libraries that can be used directly, since the search-based tuning method uses black-box tuning, the compiler cannot theoretically determine which scheduling method has higher execution efficiency and can only rely on hardware testing to accurately obtain scheduling performance, which makes the hardware testing time longer and increases the time cost of deep learning compilers. Summary of the invention

[0005] In view of this, the purpose of the present application is to provide a search space pruning method and device for deep learning operators, by constructing and training a target decision tree, performing performance prediction on the scheduling to be tuned based on the trained target decision tree to obtain pruning rules, and performing hardware testing on the screened scheduling based on the pruning rules, no longer relying on a large number of scheduling traversals, thereby reducing the tuning search space and search time, improving the efficiency of operator tuning, and at the same time, reducing the time for hardware testing and reducing the time cost of the deep learning compiler.

[0006] In a first aspect, an embodiment of the present application provides a search space pruning method for a deep learning operator, the method comprising:

[0007] Constructing a corresponding target decision tree based on a target training set and a preset decision tree algorithm, and training the target decision tree based on a preset scheduling sample to obtain a trained target decision tree; wherein the target decision tree includes a plurality of target decision subtrees;

[0008] Based on the trained target decision tree, a performance prediction is performed on the preset scheduling to be optimized to obtain a prediction result of the target decision tree; wherein the prediction result of the target decision tree includes a prediction result of each target decision subtree;

[0009] Traverse all target decision subtrees in the target decision tree based on the prediction result of the target decision tree to obtain a pruning rule for each target decision subtree;

[0010] The to-be-optimized schedules are differentiated based on the pruning rules, a first schedule that complies with the pruning rules and a second schedule that does not comply with the pruning rules are determined, and hardware measurements are performed on the first schedule and the second schedule to obtain corresponding hardware measurement results.

[0011] In a possible implementation, the target decision subtree includes a branch node and a leaf node; the branch node includes a root node; and the performance prediction of the to-be-tuned scheduling based on the trained target decision tree to obtain a prediction result of the target decision tree includes:

[0012] Based on the branch nodes and leaf nodes of the target decision subtree, a binary judgment is performed on the scheduling feature vector to be optimized to obtain a score for each leaf node;

[0013] The prediction result of the target decision subtree is determined based on the score of each leaf node, and the prediction results of each target decision subtree are added together to obtain the prediction result of the target decision tree.

[0014] In a possible implementation, traversing all target decision subtrees in the target decision tree based on the prediction result of the target decision tree to obtain a pruning rule for each target decision subtree includes:

[0015] Traversing all target decision subtrees in the target decision tree to obtain the score of each leaf node, traversing the score of each leaf node to find the target leaf node with the highest score, and determining the target path from the root node to the target leaf node;

[0016] A pruning rule of the trained target decision tree is obtained based on the target leaf node and the target path.

[0017] In a possible implementation, the pruning rule of the trained target decision tree obtained based on the target leaf node and the target path includes:

[0018] Traverse the scores of each leaf node to find the target leaf node with the highest score, and determine the target path from the root node to the target leaf node;

[0019] A pruning rule of the trained target decision tree is obtained based on the target leaf node and the target path.

[0020] In a possible implementation, the pruning rule of the trained target decision tree obtained based on the target leaf node and the target path includes:

[0021] Analyze the branch direction of the target leaf node and the attribute type of each branch node on the target path to obtain the attribute value of the rule item of the target decision subtree corresponding to the target leaf node;

[0022] In response to acquiring the attribute values ​​of the rule items of each target decision subtree, a pruning rule of the trained target decision tree is obtained based on the attribute values ​​of the rule items of each target decision subtree.

[0023] In a possible implementation manner, obtaining the rule item list of the trained target decision tree based on the attribute values ​​of the rule items of each target decision subtree includes:

[0024] For each target decision subtree's rule items, multiple groups of rule items with the same feature dimensions are determined;

[0025] The intersection of each group of rule items with the same feature dimension is taken respectively to obtain the intersection of each group of rule items, and the intersection of each group of rule items is merged to obtain the pruning rules of the target decision subtree.

[0026] In a possible implementation, the method further includes: when pruning the to-be-optimized schedule based on the pruning rule, determining a target schedule that satisfies all constraints in the pruning rule, and calculating a schedule survival rate based on the number of the target schedules and the number of the to-be-optimized schedules;

[0027] In response to the scheduling survival rate being lower than a preset first threshold, reducing the number of the target decision subtrees to dynamically adjust the pruning rules to obtain the adjusted pruning rules; or,

[0028] In response to the scheduling survival rate being higher than a preset second threshold, the number of the target decision subtrees is increased to dynamically adjust the pruning rules to obtain the adjusted pruning rules.

[0029] In a second aspect, an embodiment of the present application further provides a search space pruning device for a deep learning operator, the device comprising:

[0030] A training module, used to construct a corresponding target decision tree based on a target training set and a preset decision tree algorithm, and train the target decision tree based on a preset scheduling sample to obtain a trained target decision tree; wherein the target decision tree includes a plurality of target decision subtrees;

[0031] A prediction module, used to perform performance prediction on a preset scheduling to be optimized based on the trained target decision tree, and obtain a prediction result of the target decision tree; wherein the prediction result of the target decision tree includes a prediction result of each target decision subtree;

[0032] An acquisition module, used to traverse all target decision subtrees in the target decision tree based on the prediction result of the target decision tree, and acquire a pruning rule for each target decision subtree;

[0033] A test module is used to distinguish the to-be-optimized schedules based on the pruning rules, determine a first schedule that complies with the pruning rules and a second schedule that does not comply with the pruning rules, and perform hardware testing on the first schedule and the second schedule to obtain corresponding hardware testing results.

[0034] In a possible implementation, the target decision subtree includes a branch node and a leaf node; the branch node includes a root node; and the prediction module is specifically used to:

[0035] Based on the branch nodes and leaf nodes of the target decision subtree, a binary judgment is performed on the scheduling feature vector to be optimized to obtain a score for each leaf node;

[0036] The prediction result of the target decision subtree is determined based on the score of each leaf node, and the prediction results of each target decision subtree are added together to obtain the prediction result of the target decision tree.

[0037] In a possible implementation manner, the acquisition module is specifically configured to:

[0038] Traversing all target decision subtrees in the target decision tree to obtain the score of each leaf node, traversing the score of each leaf node to find the target leaf node with the highest score, and determining the target path from the root node to the target leaf node;

[0039] A pruning rule of the trained target decision tree is obtained based on the target leaf node and the target path.

[0040] In a possible implementation manner, the acquisition module is specifically configured to:

[0041] Traverse the scores of each leaf node to find the target leaf node with the highest score, and determine the target path from the root node to the target leaf node;

[0042] A pruning rule of the trained target decision tree is obtained based on the target leaf node and the target path.

[0043] In a possible implementation manner, the acquisition module is specifically configured to:

[0044] Analyze the branch direction of the target leaf node and the attribute type of each branch node on the target path to obtain the attribute value of the rule item of the target decision subtree corresponding to the target leaf node;

[0045] In response to acquiring the attribute values ​​of the rule items of each target decision subtree, a pruning rule of the trained target decision tree is obtained based on the attribute values ​​of the rule items of each target decision subtree.

[0046] In a possible implementation manner, the acquisition module is specifically configured to:

[0047] For each target decision subtree's rule items, multiple groups of rule items with the same feature dimensions are determined;

[0048] The intersection of each group of rule items with the same feature dimension is taken respectively to obtain the intersection of each group of rule items, and the intersection of each group of rule items is merged to obtain the pruning rules of the target decision subtree.

[0049] In a possible implementation manner, the search space pruning device for the deep learning operator further includes:

[0050] A determination module, configured to determine a target schedule that satisfies all constraints in the pruning rule when pruning the schedule to be optimized based on the pruning rule, and calculate a schedule survival rate based on the number of the target schedules and the number of the schedules to be optimized;

[0051] a first adjustment module, configured to reduce the number of the target decision subtrees in response to the scheduling survival rate being lower than a preset first threshold, so as to dynamically adjust the pruning rule to obtain the adjusted pruning rule; or

[0052] The second adjustment module is used to increase the number of the target decision subtrees in response to the scheduling survival rate being higher than a preset second threshold, so as to dynamically adjust the pruning rules to obtain the adjusted pruning rules.

[0053] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of the search space pruning method of the deep learning operator as described in any one of the first aspects.

[0054] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the search space pruning method for the deep learning operator described in any one of the first aspects are executed.

[0055] The embodiment of the present application provides a search space pruning method and device for a deep learning operator, which constructs a corresponding target decision tree based on a target training set and a preset decision tree algorithm, and trains the target decision tree based on a preset scheduling sample to obtain a trained target decision tree, performs performance prediction on a preset scheduling to be tuned based on the trained target decision tree, obtains the prediction result of the target decision tree, traverses all target decision subtrees in the target decision tree based on the prediction result of the target decision tree, obtains the pruning rule of each target decision subtree, distinguishes the scheduling to be tuned based on the pruning rule, determines the first scheduling that meets the pruning rule and the second scheduling that does not meet the pruning rule, and performs hardware testing on the first scheduling and the second scheduling to obtain the corresponding hardware testing results. The present application, by constructing and training a target decision tree, performs performance prediction on the scheduling to be tuned based on the trained target decision tree to obtain pruning rules, and performs hardware testing on the selected scheduling based on the pruning rules, no longer relies on a large number of scheduling traversals, thereby reducing the search space and search time for tuning, improving the efficiency of operator tuning, and at the same time, reducing the time of hardware testing and reducing the time cost of the deep learning compiler.

[0056] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0058] Figure 1 is a flowchart of a search space pruning method for a deep learning operator according to an embodiment of the present application;

[0059] Figure 2 It is a schematic diagram of search space pruning;

[0060] Figure 3 is a flowchart of a search space pruning method for a deep learning operator according to another embodiment of the present application;

[0061] Figure 4 This is a schematic diagram of the final execution performance comparison of a single operator;

[0062] Figure 5 It is a schematic diagram of the end-to-end model execution performance comparison;

[0063] Figure 6 It is a schematic diagram of the comparison of tuning time;

[0064] Figure 7 It is a schematic diagram of the experimental results of dynamically adjusting the pruning rules;

[0065] Figure 8(a) is a diagram showing the operator scheduling performance during the tuning process. Figure 1 ;

[0066] Figure 8(b) is a diagram showing the operator scheduling performance during the tuning process. Figure 2 ;

[0067] Figure 8(c) is a diagram showing the operator scheduling performance during the tuning process. Figure 3 ;

[0068] Fig. 9 is a structural schematic diagram of a search space pruning device for a deep learning operator according to an embodiment of the present application;

[0069] Fig.10 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0070] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0071] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0072] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0073] Considering that deep learning models often contain millions or even billions of parameters, a single run requires several G or even TB of floating-point operations. Such a huge amount of computing has brought unprecedented computing efficiency requirements to computing devices. In order to maximize the computing efficiency of hardware devices and maximize the computing performance of hardware, some hardware manufacturers have written hardware-specific manual operator libraries for high-frequency deep learning operators. Experienced engineers in hardware manufacturers have a full understanding of the architecture of related hardware, so the operator libraries they have written have high computing performance that ordinary kernels do not have. However, the operator types and hardware adaptation range of manual operator libraries are limited, which restricts their use scenarios. In order to improve the flexibility of deep learning model deployment, deep learning compilers can automatically generate executable code for tensor programs on various hardware for various operators. In order to cope with the challenges of increasingly complex deep learning models and a wide variety of operators, deep learning compilers use a variety of different technical means to automatically implement operator tuning.

[0074] At present, to address the above problems, a widely applicable search-based operator tuning method is often used, that is, all scheduling methods of the operator are regarded as a search space, and its search goal is to find the scheduling method with the highest or approximately optimal computing performance.

[0075] Specifically, based on the manually defined or automatically generated operator scheduling template, the operator tuner applies multiple scheduling primitives on the basis of the original schedule to generate a new schedule without changing the correctness of the calculation results. In order to evaluate the specific performance of different schedules, the compiler will compile different schedules during the tuning process and actually execute them on the hardware to measure their time delay data. In addition, the tuner usually sets a cost model based on machine learning, using the features extracted from the schedule and the hardware measurement results as the input and true value of the model training to predict the theoretical performance of the schedule and assist the tuner in selecting and generating new schedules. For example, the cost model is used as a population fitness evaluation tool in the genetic algorithm to guide the tuner to generate new and most promising new schedules. By combining hardware measurement with the cost model, the search-based tuner can implement the iterative process of obtaining the schedule-hardware measurement-generating a new schedule, and gradually approach the optimal solution in the progressive search process.

[0076] However, the current search-based operator tuning method relies on a large number of scheduling traversals, resulting in a large tuning search space and a long search time, which makes the operator tuning efficiency low. In addition, compared with the convenience of manual operator libraries that can be used directly, since the search-based tuning method uses black-box tuning, the compiler cannot theoretically determine which scheduling method has higher execution efficiency and can only rely on hardware testing to accurately obtain scheduling performance, which makes the hardware testing time longer and increases the time cost of deep learning compilers.

[0077] To address this problem, the present application provides a method and device for pruning the search space of a deep learning operator. By constructing and training a target decision tree, a performance prediction is made for the scheduling to be tuned based on the trained target decision tree to obtain pruning rules, and hardware testing is performed on the screened scheduling based on the pruning rules. This method no longer relies on a large number of scheduling traversals, thereby reducing the tuning search space and search time, improving the efficiency of operator tuning, and at the same time reducing the time for hardware testing and the time cost of the deep learning compiler.

[0078] Figure 1 is a flowchart of a search space pruning method for a deep learning operator according to an embodiment of the present application. Figure 1 As shown, the search space pruning method of the deep learning operator in the embodiment of the present application may specifically include:

[0079] S101, constructing a corresponding target decision tree based on a target training set and a preset decision tree algorithm, and training the target decision tree based on a preset scheduling sample to obtain a trained target decision tree.

[0080] S102, performing performance prediction on a preset scheduling to be optimized based on the trained target decision tree to obtain a prediction result of the target decision tree.

[0081] S103, traversing all target decision subtrees in the target decision tree based on the prediction result of the target decision tree, and obtaining a pruning rule for each target decision subtree.

[0082] S104, distinguishing the schedules to be optimized based on the pruning rules, determining a first schedule that complies with the pruning rules and a second schedule that does not comply with the pruning rules, and performing hardware measurements on the first schedule and the second schedule to obtain corresponding hardware measurement results respectively.

[0083] In the search space pruning method of the above deep learning operator, by constructing and training a target decision tree, a performance prediction is made for the scheduling to be tuned based on the trained target decision tree to obtain pruning rules, and hardware testing is performed on the screened scheduling based on the pruning rules. It no longer relies on a large number of scheduling traversals, thereby reducing the tuning search space and search time, improving the efficiency of operator tuning, and at the same time, reducing the time for hardware testing and reducing the time cost of the deep learning compiler.

[0084] The above exemplary steps of the embodiment of the present application are described below with reference to specific examples:

[0085] S101, constructing a corresponding target decision tree based on a target training set and a preset decision tree algorithm, and training the target decision tree based on a preset scheduling sample to obtain a trained target decision tree.

[0086] In the embodiments of the present application, the decision tree algorithm is a pre-set decision tree algorithm. The present application describes XGBoost as an example, but does not constitute a limitation on the decision tree algorithm and can be set according to actual conditions. The target training set is a training set for training the target decision tree. The size of the target training set is set to a fixed value. The present application describes the target decision tree as an XGBoost decision tree as an example. The target decision tree includes multiple target decision subtrees, and the target decision subtree includes branch nodes and leaf nodes. The branch nodes include root nodes. The number of target decision subtrees is determined by a preset gain. A corresponding target decision tree is constructed based on the target training set and the preset decision tree algorithm, and the target decision tree is trained based on a preset scheduling sample to obtain a trained target decision tree.

[0087] It should be noted that for the types of decision trees, this application uses XGBoost decision trees to provide reference information for pruning, which is mainly due to considerations of training data and prediction accuracy. First, in order to train the decision tree, it is necessary to extract features from relevant scheduling samples, use the hardware measured results of the scheduling as labels, and use multiple data pairs to train the decision tree to accurately evaluate the pros and cons of the scheduling. The input of a set of training data is the features extracted from the scheduling, which can be either manually designed parameterized features or parameter sequences in the scheduling. The labels of a set of training data are usually raw data or normalized data based on the actual execution time of the scheduling. For these parameter features, they are all continuous numerical features. In order to achieve the purpose of predicting scheduling performance, this application should construct a regression tree, so the decision tree needs to support continuous target variables. Secondly, the purpose of constructing the decision tree is to provide reliable reference information for subsequent pruning strategies, and the accuracy of the decision tree will directly affect the reliability of the pruning strategy. Among many decision tree algorithms, XGBoost meets the dual requirements of regression prediction and high precision. Experimental results show that the average relative error of XGboost in predicting scheduling performance can be less than 3%, and using XGBoost as a decision tree can provide accurate pruning constraint information.

[0088] When building a decision tree, the parameter sequence extracted from the scheduling sample is used as a feature, and the hardware measured data is used as the true value. Compared with the complex scheduling feature representation designed manually, the scheduling parameter feature can be obtained directly from the scheduling expression, without the need for a complex extraction process, and has a smaller computational overhead. In the initial stage of tuning, the scheduling samples are collected and the training set is constructed. The size of the training set is set to a fixed value. According to experience, it is set to 100, which can ensure that the tuning process performs as little hardware testing as possible while making the decision tree have a high prediction accuracy. Moreover, this training set size can make the decision tree have a sufficiently high generalization, and the pruning rules extracted from it can efficiently screen low-quality scheduling and retain the optimal scheduling.

[0089] S102, performing performance prediction on a preset scheduling to be optimized based on the trained target decision tree to obtain a prediction result of the target decision tree.

[0090] In the embodiment of the present application, the prediction result of the target decision tree includes the prediction result of each target decision subtree. Based on the trained target decision tree obtained in step S101, the performance of the scheduling to be optimized is predicted to obtain the prediction result of the target decision tree for subsequent processing.

[0091] As a possible implementation method, a scheduling feature vector to be optimized is binary judged based on the branch nodes and leaf nodes of the target decision subtree to obtain the score of each leaf node; the prediction result of the target decision subtree is determined based on the score of each leaf node, and the prediction results of each target decision subtree are added to obtain the prediction result of the target decision tree. The scheduling feature vector is the feature vector of the scheduling to be optimized.

[0092] Specifically, the target decision tree, i.e., the XGBoost decision tree, is a gradient boosting decision tree. The complete XGBoost decision tree contains several subtrees. When using XGBoost to predict the performance of scheduling, each subtree, i.e., the target decision subtree, will score the scheduling to be optimized. The sum of the prediction results of all subtrees represents the prediction result of the complete XGBoost decision tree. For each subtree, starting from the root node, each internal node will perform a binary judgment on a certain dimension of the scheduling feature vector until the leaf node, and obtain the score of each leaf node.

[0093] S103, traversing all target decision subtrees in the target decision tree based on the prediction result of the target decision tree, and obtaining a pruning rule for each target decision subtree.

[0094] In the embodiment of the present application, the pruning rules include a feature dimension (FeatureIndex), a feature lower limit (LowerBound, LB for short) and a feature upper limit (UpperBound, UB for short). Based on the prediction result of the target decision tree obtained in step S102, all target decision subtrees in the target decision tree are traversed to obtain the pruning rules of each target decision subtree for subsequent processing.

[0095] As a possible implementation method, all target decision subtrees in the target decision tree are traversed to obtain the score of each leaf node, and the score of each leaf node is traversed to find the target leaf node with the highest score, and the target path from the root node to the target leaf node is determined; based on the target leaf node and the target path, the pruning rules of the trained target decision tree are obtained.

[0096] Among them, in the process of obtaining the pruning rules of the trained target decision tree based on the target leaf nodes and the target path, traverse the scores of each leaf node to find the target leaf node with the highest score, and determine the target path from the root node to the target leaf node; the pruning rules of the trained target decision tree are obtained based on the target leaf nodes and the target path.

[0097] It should be noted that the purpose of pruning the search space is to retain the most promising high-performance scheduling solutions for hardware testing, and discard those ordinary scheduling and low-performance scheduling that have no effect on improving the upper limit of the acceleration ratio, in the hope of saving tuning time, especially the most time-consuming hardware testing part. This puts forward requirements for the selection and design of pruning rules. The pruning method should have a certain a priori and analyze the potential performance of each scheduling without the need for hardware testing. This application explores potential pruning strategies from the branch judgment conditions and leaf node scores of the decision tree.

[0098] Specifically, after the above performance prediction obtains the prediction results of the target decision tree, including the score of each leaf node, several characteristic attributes and their numerical ranges are recorded on the path from the root node to the leaf node. These combinations point to the search area of ​​a specific schedule in the search space, and the leaf nodes under the corresponding path represent the potential performance of these search areas. When the leaf nodes under a certain path have a higher score, it means that the schedule that meets the attribute constraints in the path can have higher performance, and the optimal solution of the operator optimization is more likely to be hidden under these areas. Similarly, the leaf nodes with lower scores represent that the search space under its path constraints is unlikely to further improve the acceleration ratio of operator execution.

[0099] For example, Figure 2 As shown in the figure, it is assumed that the length of the feature vector used to train the target decision tree is 2, and its value range is 0 to 10. The target decision tree is trained and constructed using several training data. This decision tree contains two subtrees. Subtree 1 contains two branch nodes and three leaf nodes. The score of leaf node 2 is higher than that of other leaf nodes. This leaf node is the target leaf node. The search space is established according to the feature vector dimension and the value interval, that is, the first dimension of the feature vector is used as the horizontal axis and the second dimension is used as the vertical axis. The complete search space is divided into three subspaces according to the decision tree structure, corresponding to the three leaf nodes respectively. Since leaf node 2 has the highest score, the scheduling method in subspace 2 is more likely to obtain a higher prediction score than the scheduling method in subspace 1 and 3. From another perspective, constraints can be extracted from the decision tree, and the search space can be pruned according to the structure of the decision tree. Search spaces 1 and 3 are pruned, and only search space 2 is retained. Similarly, for subtree 2, by superimposing the highest-scoring areas in the two subtrees and taking the intersection of the two, the retained search space can be obtained, and the remaining search space will be pruned.

[0100] To continue, the pruning rules extracted from the decision tree can be represented by a simple data structure, including feature dimension, feature lower limit and feature upper limit. For each item of the pruning rule, the feature dimension indicates the dimension of the input feature, the feature lower limit and feature upper limit represent the lower and upper bounds of the corresponding dimension feature, respectively. By default, the feature upper and lower bounds are infinite, that is, there is no limit, which also means that there is no pruning in the search space. The extraction of pruning rules from the decision tree is mainly to determine the three attributes in the data structure.

[0101] For a trained decision tree and the number of subtrees N, the number of subtrees N is mainly used to control the pruning granularity, which is a dynamically changing variable. The pruning rule of the target decision tree is a list of several rule items (RuleItem). Initially, this list is empty. For the first N subtrees in the decision tree, find the target leaf node with the highest score max_score_node by traversing, and get the target path path from the root node to this target leaf node.

[0102] It should be noted that, in the process of obtaining the pruning rules of the trained target decision tree based on the target leaf nodes and the target path, optionally, the branch directions of the target leaf nodes and the attribute types of each branch node on the target path are parsed to obtain the attribute values ​​of the rule items of the target decision subtree corresponding to the target leaf nodes; in response to obtaining the attribute values ​​of the rule items of each target decision subtree, the pruning rules of the trained target decision tree are obtained based on the attribute values ​​of the rule items of each target decision subtree.

[0103] Here, for each branch node on the target path, the attribute value of the rule item RuleItem is obtained by parsing the attribute type on each branch node and the branch direction of the target leaf node with the highest score. Specifically, if the target leaf node is in the left subtree of the branch, the feature attribute of the corresponding coordinate will generate an upper bound, that is, the conditional judgment value of the branch, for example Figure 2 The root node in will generate a RuleItem = {1, -Inf, 5}. Conversely, if the leaf node is in the right subtree of the branch, a lower bound will be generated. By traversing all N subtrees, the preliminary pruning rules can be obtained.

[0104] Optionally, in the process of obtaining a list of rule items of the trained target decision tree based on the attribute values ​​of the rule items of each target decision subtree, multiple groups of rule items with the same feature dimensions are determined for the rule items of each target decision subtree; the intersection of each group of rule items with the same feature dimension is taken to obtain the intersection of each group of rule items, and the intersection of each group of rule items is merged to obtain the pruning rules of the target decision subtree.

[0105] Here, through the Merge_by_Index function, for RuleItems with the same feature dimension, the intersection of their intervals is taken, and these collections are merged. If the RuleItem intervals with the same feature dimension have no intersection, these rules are directly discarded. In this way, a set of scheduling pruning rules for the trained target decision tree can be finally obtained and applied to scheduling pruning.

[0106] S104, distinguishing the schedules to be optimized based on the pruning rules, determining a first schedule that complies with the pruning rules and a second schedule that does not comply with the pruning rules, and performing hardware measurements on the first schedule and the second schedule to obtain corresponding hardware measurement results respectively.

[0107] In the embodiment of the present application, the schedules to be optimized are distinguished according to the pruning rules obtained in step S103, a first schedule that complies with the pruning rules and a second schedule that does not comply with the pruning rules are determined, and hardware measurements are performed on the first schedule and the second schedule to obtain corresponding hardware measurement results.

[0108] Optionally, the first schedule is compiled and executed on preset hardware, and the delay time of the first schedule is measured; the second schedule is sampled twice to obtain a sampled second schedule, and a target number of second schedule samples are randomly screened out from the sampled second schedule based on a preset sampling ratio, and the target number of second schedule samples are compiled and executed on hardware.

[0109] Specifically, after obtaining the pruning rules extracted from the decision tree, the pruning rules are speculatively tested on hardware. Specifically, if the schedule meets the pruning rules, it is the first schedule. This schedule has similar characteristics to the high-performance schedule and has the potential to become the best schedule. Therefore, the schedule is compiled and executed on the hardware to measure the delay time. For schedules that do not meet the pruning rules, they are the second schedules. These schedules are more likely to be low-performance schedules and will not be directly tested on hardware. Considering the complexity of the search space, the pruning rules can prune away some high-performance schedules. In order to ensure that the final tuning effect is not affected, the pruned schedule can be sampled twice. By setting the sampling ratio, a small number of samples can be randomly selected for hardware testing.

[0110] The search space pruning method of the deep learning operator provided in the embodiment of the present application constructs a corresponding target decision tree based on the target training set and the preset decision tree algorithm, and trains the target decision tree based on the preset scheduling sample to obtain the trained target decision tree, performs performance prediction on the preset scheduling to be tuned based on the trained target decision tree, obtains the prediction result of the target decision tree, traverses all target decision subtrees in the target decision tree based on the prediction result of the target decision tree, obtains the pruning rule of each target decision subtree, distinguishes the scheduling to be tuned based on the pruning rule, determines the first scheduling that meets the pruning rule and the second scheduling that does not meet the pruning rule, and performs hardware testing on the first scheduling and the second scheduling to obtain the corresponding hardware testing results respectively. The search space pruning method of the deep learning operator of the present application, by constructing and training the target decision tree, performs performance prediction on the scheduling to be tuned based on the trained target decision tree to obtain the pruning rule, and performs hardware testing on the selected scheduling based on the pruning rule, no longer relies on a large number of scheduling traversals, thereby reducing the search space and search time for tuning, improving the efficiency of operator tuning, and at the same time, reducing the time of hardware testing and reducing the time cost of the deep learning compiler.

[0111] Further, such as Figure 3 As shown, the search space pruning method of the deep learning operator of the present application also includes the following steps:

[0112] S301, when pruning the schedules to be optimized based on the pruning rules, determining the target schedules that satisfy all constraints in the pruning rules, and calculating the schedule survival rate based on the number of target schedules and the number of schedules to be optimized.

[0113] In the embodiment of the present application, the target scheduling is the scheduling that satisfies all constraints in the pruning rules, the scheduling survival rate is the parameter for measuring the surviving scheduling, and the scheduling survival rate represents the severity of pruning. When pruning the scheduling to be optimized based on the pruning rules obtained in the above embodiment, the target scheduling that satisfies all constraints in the pruning rules is determined, and the scheduling survival rate is calculated based on the number of target scheduling and the number of scheduling to be optimized.

[0114] It should be noted that the pruning rules are extracted from the decision tree. XGBoost uses a forward step-by-step algorithm for training. In each round of iteration, XGBoost generates new subtrees based on the fitting residuals of the current model. As the new subtrees are built, the overall prediction error of XGBoost shows a downward trend, and the prediction scores of the new subtrees also show a downward trend. In general, the highest score of each subtree is negatively correlated with the subtree number. When extracting pruning rules, the selected schedule needs to have higher performance, so the leaf node with the highest score is selected, that is, from the N subtrees at the front of the decision tree. The larger N is, the more subtrees it contains, and the higher the score of the schedule that meets these rules. However, as the rules become more stringent and the number of surviving schedules is smaller, the degree of pruning is greater at this time, and the optimal schedule or the approximately optimal schedule may also be excluded, resulting in a decrease in the operator tuning acceleration ratio.

[0115] Therefore, this application uses the scheduling survival rate as a measurement factor. In each round of tuning, the tuning framework generates a scheduling set S through a genetic algorithm, and the number of schedules it contains is schedule_number. When applying pruning rules to this set, only the target schedules that meet all constraints in the pruning rules can be retained, which is called survivor in this application. The scheduling survival rate can be calculated by the number of target schedules survivor_number and the number of schedules to be tuned schedule_number to characterize the severity of pruning.

[0116] It should be noted that by setting the parameter feature lower limit LB and feature upper limit UB, as well as the scheduling survival rate lower limit and scheduling survival rate upper limit, the range of the number of scheduling survivors is specified, and these two parameters also determine the degree of pruning.

[0117] S302, in response to the scheduling survival rate being lower than a preset first threshold, reducing the number of target decision subtrees to dynamically adjust the pruning rules to obtain adjusted pruning rules.

[0118] In the embodiment of the present application, when the scheduling survival rate calculated in step S301 is lower than the preset first threshold, it means that the scheduling survival rate is too low, and it is necessary to reduce the number of target decision subtrees to dynamically adjust the pruning rules to obtain the adjusted pruning rules. Specifically, when the scheduling survival rate is too low, the pruning at this time may prune the best schedule together, so by reducing the number of subtrees N, the pruning constraints are reduced, so that more schedules can be retained.

[0119] S303, in response to the scheduling survival rate being higher than a preset second threshold, increasing the number of target decision subtrees to dynamically adjust the pruning rules to obtain adjusted pruning rules.

[0120] In the embodiment of the present application, when the scheduling survival rate calculated in step S301 is higher than the preset second threshold, it means that the scheduling survival rate is too high, and it is necessary to increase the number of target decision subtrees to dynamically adjust the pruning rules to obtain the adjusted pruning rules. Specifically, when the scheduling survival rate is too high, the number of subtrees N is increased to increase the pruning range.

[0121] It should also be noted that the number of subtrees N will be adjusted after each round of tuning. In the initial stage of tuning, the initial value of N is determined by the parameter gain. N gradually increases from 0 until the sum of the highest scores of the leaf nodes of each subtree in the first N subtrees under the current conditions is greater than gain. Experience shows that setting LB to 0.1, UB to 0.5, and gain to 0.9 (the highest score of XGBoost is 1) can achieve better pruning results.

[0122] In short, after obtaining the scheduling pruning rules, for each round of newly generated scheduling in the search framework, the scheduling needs to be screened through the pruning rules. Only the surviving scheduling that meets each constraint in the pruning rules can be retained and tested on the hardware, and the rest of the scheduling will be discarded. Since the computational density and amount of computation of different tensor programs vary with the type and shape of operators, the pruning methods of their search spaces are also different. In order to ensure that different operators can be pruned effectively without affecting the maximum speedup ratio of operator tuning, it is necessary to dynamically adjust the pruning rules. Therefore, this application uses the scheduling survival rate as a measurement factor and dynamically adjusts the number of subtrees to maintain the degree of pruning in a reasonable range.

[0123] Please add here that the decision tree in this application is the target decision tree, and the decision subtree is the target decision subtree.

[0124] Therefore, we collect training sets to build a decision tree, identify and analyze the characteristic attributes of high-performance scheduling, summarize pruning rules, conduct hardware testing for scheduling that meets the pruning rules, perform secondary sampling on the remaining scheduling, and select a small number of scheduling for hardware testing. During the tuning process, we dynamically adjust the pruning rules to maximize the degree of pruning without affecting the tuning efficiency, thus achieving efficient operator tuning.

[0125] The search space pruning method of the deep learning operator of the present application is described below with reference to specific experimental examples.

[0126] This application conducts experiments on three platforms to fully test the adaptability of the search space pruning algorithm to different hardware. This application uses three widely used GPUs (Graphics Processing Units) V100, T4, and 3090 as test platforms. The three hardware use the same software and system environment, equipped with Ubuntu 20.04 operating system, CUDA (Compute Unified Device Architecture) Version 11.2, and driver version 535.129.03.

[0127] This application compares this method with operator libraries provided by hardware vendors, such as cuDNN and other search-based automatic tuning frameworks, such as Ansor and Heron. By comparing different baselines, it can be demonstrated that this method has advantages in operator performance and tuning efficiency.

[0128] Table 1 Operator test cases

[0129]

[0130] For operator test cases, as shown in Table 1, this application uses various types of operators such as GEMM, BatchMatmul and Conv2d, and sets different tensor size conditions to fully demonstrate the feasibility of the pruning algorithm. The operator use case description is shown in the table. For end-to-end models, this application uses models such as BERT and VGG as test cases. Among them, the BERT model, as a commonly used model in the field of natural language processing, mainly includes operators such as BatchMatmul, while VGG, as a CNN model commonly used in traditional computer vision, mainly includes convolution operators.

[0131] This application mainly sets three experimental indicators: (1) the execution time of operators and models (2) the tuning time of operators and models (3) the effectiveness of pruning strategies, which mainly include the size of the search space and the number of scheduling for hardware testing.

[0132] like Figure 4As shown, in terms of operator types, for the three operators of GEMM, BMM, and C2d, this application achieved 0.99 times, 1.02 times, and 0.97 times of Heron in V100, respectively. This performance is almost the same as Heron, which is the advanced level of the current search-based deep learning compiler; in terms of operator input size, this application sets a variety of shapes for each operator. These shapes are mainly extracted from common models, covering tensor shapes of different sizes and different computational amounts. Through experimental results, it can be obtained that the method of this application has stable performance for input tensors of different shapes, and there is no problem of performance degradation due to shape changes; in terms of hardware equipment, this application uses common hardware platforms in deep learning deployment, covering hardware devices with complementary computing resources under multiple architectures. The experimental results show that the method of this application has no significant effect on the final performance of operator tuning for different platforms. It should be pointed out that the method of this application far exceeds Ansor in GEMM operator performance, because Ansor does not support Tensor Core while the method of this application does.

[0133] The present application notes that the performance of some operators even exceeds that of Heron. This is because the method of the present application allows the tuner to focus on the high-quality search space, while the schedule generated by Heron is distributed across the entire search space. By generating schedules only in high-quality search spaces, the method of the present application is likely to achieve a higher upper limit on the acceleration ratio. At the same time, the present application also notes that the method of the present application has a performance degradation of about 5% in operator performance on the convolution operator. The present application speculates that this is because the convolution operator has a more complex scheduling method. Compared with the triple loop of GEMM and the quadruple loop structure of BMM, Conv2D has seven basic loop nests. This makes the convolution operator scheduling space dimensionality higher and the division of the search space more difficult. At the same time, the two-dimensional convolution scheduling has longer pending parameters, that is, the feature dimension of the decision tree is larger, and the pruning rules extracted from it may have certain limitations.

[0134] like Figure 5As shown, for the Bert model, the method of the present application achieved 1.08 times, 1.00 times, and 0.93 times of Heron on V100, T4, and 3090, respectively. BERT mainly includes operators such as BatchMatmul, and the method of the present application can achieve performance that is almost consistent with or slightly higher than Heron during the operator tuning process. Therefore, for end-to-end reasoning latency, its performance is basically consistent with Heron; for the VGG model, the method of the present application achieved 0.85 times, 0.98 times, and 0.94 times of Heron on V100, T4, and 3090, respectively. Since VGG mainly includes Con2D operators, the method of the present application has a certain degree of performance degradation in the process of Conv2D operator tuning, so for the entire network reasoning, its performance is slightly lower than Heron. However, compared to Ansor, the method of the present application still has a faster reasoning speed.

[0135] like Figure 6 As shown, by pruning the search space, the method of the present application can significantly shorten the tuning time to 59% of the original time. For operators tuned on V100, the present application screened out an average of 967 schedules out of 2000 schedules. This number of pruning is mainly controlled by the parameter UB. By increasing UB, the number of pruning decreases accordingly. On the contrary, the number of pruning increases, but the probability of the optimal solution being pruned increases simultaneously, which may affect the final tuning effect.

[0136] In order to verify the function of dynamic pruning rules, this application sets up a control experiment, and the experimental results are as follows: Figure 7 As shown. This application sets Our-fixed as a control, which uses fixed pruning rules. Specifically, after the decision tree training is completed, this application stipulates that all pruning rules are extracted from the first three subtrees and applied to prune during the entire tuning process. In contrast, Our-dynamic performs dynamic pruning rule adjustments as described above. This application observed that when using static pruning rules, the execution time of G2 and C1 operators increased significantly. This is because the optimal scheduling method and properties of operator tuning are different for different operators and different input shapes. If these differences are ignored, the design of the pruning rules will be biased, resulting in the retained search space not being the highest quality search space, and the area where the optimal solution is located being pruned. By dynamically changing the pruning rules, the operator performance is almost consistent with Heron, eliminating the randomness of the pruning process. The search space where the optimal solution is located can almost always be retained. Therefore, the dynamic pruning rule adjustment set in this application is effective.

[0137] The scheduling generated by the existing tuning method still has the problem of a large number of low-quality and low-performance scheduling. It can be found that the number of invalid scheduling, that is, the number of scheduling with performance equal to 0, has been greatly reduced. In the rest, this application reduces the number of redundant scheduling, but does not affect the acceleration effect of the optimal scheduling. On the contrary, in some cases, the upper limit of the acceleration ratio can be increased, such as the B1 operator. This is because the method designed in this application successfully screens out the low-quality search space, thereby improving the quality of the overall scheduling.

[0138] By performing operator tuning in a high-quality search space, the method of the present application can achieve a faster convergence speed, as shown in, for example, FIG8(a), FIG8(b), and FIG8(c). The method of the present application can achieve higher tuning performance in a shorter tuning round.

[0139] Through experiments, it can be proved that the search space pruning method of the deep learning operator of this application can effectively improve the scheduling quality, thereby shortening the tuning time without affecting the tuning efficiency.

[0140] It should be noted that the search space pruning method of the deep learning operator of the present application is a deep learning operator automatic tuning method, and also a search-based deep learning operator automatic tuning method.

[0141] Fig. 9 is a flow chart of a search space pruning device for a deep learning operator according to an embodiment of the present application. Fig. 9 As shown, the search space pruning device 900 of the deep learning operator in the embodiment of the present application may specifically include:

[0142] The training module 901 is used to construct a corresponding target decision tree based on a target training set and a preset decision tree algorithm, and train the target decision tree based on a preset scheduling sample to obtain a trained target decision tree; wherein the target decision tree includes multiple target decision subtrees.

[0143] The prediction module 902 is used to perform performance prediction on the preset scheduling to be optimized based on the trained target decision tree to obtain the prediction result of the target decision tree; wherein the prediction result of the target decision tree includes the prediction result of each target decision subtree.

[0144] The acquisition module 903 is used to traverse all target decision subtrees in the target decision tree based on the prediction result of the target decision tree, and acquire the pruning rule of each target decision subtree.

[0145] The testing module 904 is used to distinguish the schedules to be optimized based on the pruning rules, determine a first schedule that complies with the pruning rules and a second schedule that does not comply with the pruning rules, and perform hardware testing on the first schedule and the second schedule to obtain corresponding hardware testing results respectively.

[0146] In a possible implementation, the target decision subtree includes a branch node and a leaf node; the branch node includes a root node; and the prediction module is specifically used to:

[0147] Based on the branch nodes and leaf nodes of the target decision subtree, a binary judgment is made on the scheduling feature vector to be optimized and scheduled, and a score of each leaf node is obtained;

[0148] The prediction result of the target decision subtree is determined based on the score of each leaf node, and the prediction results of each target decision subtree are added together to obtain the prediction result of the target decision tree.

[0149] In a possible implementation, the acquisition module is specifically configured to:

[0150] Traverse all target decision subtrees in the target decision tree, obtain the score of each leaf node, traverse the score of each leaf node, find the target leaf node with the highest score, and determine the target path from the root node to the target leaf node;

[0151] The pruning rules of the trained target decision tree are obtained based on the target leaf nodes and the target path.

[0152] In a possible implementation, the acquisition module is specifically configured to:

[0153] Traverse the scores of each leaf node, find the target leaf node with the highest score, and determine the target path from the root node to the target leaf node;

[0154] The pruning rules of the trained target decision tree are obtained based on the target leaf nodes and the target path.

[0155] In a possible implementation, the acquisition module is specifically configured to:

[0156] Analyze the branch direction of the target leaf node and the attribute type of each branch node on the target path to obtain the attribute value of the rule item of the target decision subtree corresponding to the target leaf node;

[0157] In response to acquiring the attribute values ​​of the rule items of each target decision subtree, a pruning rule of the trained target decision tree is obtained based on the attribute values ​​of the rule items of each target decision subtree.

[0158] In a possible implementation, the acquisition module is specifically configured to:

[0159] For each target decision subtree's rule items, multiple groups of rule items with the same feature dimensions are determined;

[0160] The intersection of each group of rule items with the same feature dimension is taken separately to obtain the intersection of each group of rule items, and the intersection of each group of rule items is merged to obtain the pruning rules of the target decision subtree.

[0161] In a possible implementation, the search space pruning device for a deep learning operator further includes:

[0162] A determination module, used for determining a target schedule that satisfies all constraints in the pruning rule when pruning the schedule to be optimized based on the pruning rule, and calculating a schedule survival rate based on the number of target schedules and the number of schedules to be optimized;

[0163] A first adjustment module is configured to reduce the number of target decision subtrees in response to the scheduling survival rate being lower than a preset first threshold, so as to dynamically adjust the pruning rules to obtain adjusted pruning rules; or

[0164] The second adjustment module is used to increase the number of target decision subtrees in response to the scheduling survival rate being higher than a preset second threshold, so as to dynamically adjust the pruning rules to obtain adjusted pruning rules.

[0165] The search space pruning device of the deep learning operator provided in the embodiment of the present application constructs a corresponding target decision tree based on the target training set and the preset decision tree algorithm, and trains the target decision tree based on the preset scheduling sample to obtain the trained target decision tree, performs performance prediction on the preset scheduling to be tuned based on the trained target decision tree, obtains the prediction result of the target decision tree, traverses all target decision subtrees in the target decision tree based on the prediction result of the target decision tree, obtains the pruning rule of each target decision subtree, distinguishes the scheduling to be tuned based on the pruning rule, determines the first scheduling that meets the pruning rule and the second scheduling that does not meet the pruning rule, and performs hardware testing on the first scheduling and the second scheduling to obtain the corresponding hardware testing results respectively. The search space pruning device of the deep learning operator of the present application, by constructing and training the target decision tree, performs performance prediction on the scheduling to be tuned based on the trained target decision tree to obtain the pruning rule, and performs hardware testing on the selected scheduling based on the pruning rule, no longer relies on a large number of scheduling traversals, thereby reducing the search space and search time for tuning, improving the efficiency of operator tuning, and at the same time, reducing the time of hardware testing and reducing the time cost of the deep learning compiler.

[0166] like Fig.10As shown, an electronic device 1000 provided in an embodiment of the present application includes: a processor 1001, a memory 1002 and a bus, wherein the memory 1002 stores machine-readable instructions executable by the processor 1001. When the electronic device is running, the processor 1001 communicates with the memory 1002 through the bus, and the processor 1001 executes the machine-readable instructions to perform the steps of the search space pruning method of the deep learning operator as described above.

[0167] Specifically, the above-mentioned memory 1002 and processor 1001 can be general-purpose memory and processor, which are not specifically limited here. When the processor 1001 runs the computer program stored in the memory 1002, it can execute the search space pruning method of the above-mentioned deep learning operator.

[0168] Corresponding to the search space pruning method of the above-mentioned deep learning operator, an embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the search space pruning method of the above-mentioned deep learning operator are executed.

[0169] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0170] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0171] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0172] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the deployment method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0173] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A search space pruning method for a deep learning operator, characterized in that: The search space pruning method comprises: Constructing a corresponding target decision tree based on a target training set and a preset decision tree algorithm, and training the target decision tree based on a preset scheduling sample to obtain a trained target decision tree; wherein the target decision tree includes a plurality of target decision subtrees; Based on the trained target decision tree, a performance prediction is performed on the preset scheduling to be optimized to obtain a prediction result of the target decision tree; wherein the prediction result of the target decision tree includes a prediction result of each target decision subtree; Traverse all target decision subtrees in the target decision tree based on the prediction result of the target decision tree to obtain a pruning rule for each target decision subtree; The to-be-optimized schedules are differentiated based on the pruning rules, a first schedule that complies with the pruning rules and a second schedule that does not comply with the pruning rules are determined, and hardware measurements are performed on the first schedule and the second schedule to obtain corresponding hardware measurement results.

2. The method according to claim 1, characterized in that The target decision subtree includes a branch node and a leaf node; the branch node includes a root node; the performance prediction of the scheduling to be optimized is performed based on the trained target decision tree to obtain the prediction result of the target decision tree, including: Based on the branch nodes and leaf nodes of the target decision subtree, a binary judgment is performed on the scheduling feature vector to be optimized to obtain a score for each leaf node; The prediction result of the target decision subtree is determined based on the score of each leaf node, and the prediction results of each target decision subtree are added together to obtain the prediction result of the target decision tree.

3. The method according to claim 2, characterized in that The step of traversing all target decision subtrees in the target decision tree based on the prediction result of the target decision tree to obtain a pruning rule for each target decision subtree includes: Traversing all target decision subtrees in the target decision tree to obtain the score of each leaf node, traversing the score of each leaf node to find the target leaf node with the highest score, and determining the target path from the root node to the target leaf node; A pruning rule of the trained target decision tree is obtained based on the target leaf node and the target path.

4. The method according to claim 3, characterized in that The pruning rule of the trained target decision tree obtained based on the target leaf node and the target path includes: Traverse the scores of each leaf node to find the target leaf node with the highest score, and determine the target path from the root node to the target leaf node; A pruning rule of the trained target decision tree is obtained based on the target leaf node and the target path.

5. The method according to claim 4, characterized in that The pruning rule of the trained target decision tree obtained based on the target leaf node and the target path includes: Analyze the branch direction of the target leaf node and the attribute type of each branch node on the target path to obtain the attribute value of the rule item of the target decision subtree corresponding to the target leaf node; In response to acquiring the attribute values ​​of the rule items of each target decision subtree, a pruning rule of the trained target decision tree is obtained based on the attribute values ​​of the rule items of each target decision subtree.

6. The method according to claim 5, characterized in that The rule item list of the trained target decision tree is obtained based on the attribute values ​​of the rule items of each target decision subtree, including: For each target decision subtree's rule items, multiple groups of rule items with the same feature dimensions are determined; The intersection of each group of rule items with the same feature dimension is taken respectively to obtain the intersection of each group of rule items, and the intersection of each group of rule items is merged to obtain the pruning rules of the target decision subtree.

7. The method according to claim 1, characterized in that The method further comprises: When pruning the to-be-optimized schedule based on the pruning rule, determining a target schedule that satisfies all constraints in the pruning rule, and calculating a schedule survival rate based on the number of the target schedules and the number of the to-be-optimized schedules; In response to the scheduling survival rate being lower than a preset first threshold, reducing the number of the target decision subtrees to dynamically adjust the pruning rules to obtain the adjusted pruning rules; or, In response to the scheduling survival rate being higher than a preset second threshold, the number of the target decision subtrees is increased to dynamically adjust the pruning rules to obtain the adjusted pruning rules.

8. A search space pruning device for a deep learning operator, characterized in that: The search space pruning device comprises: A training module, used to construct a corresponding target decision tree based on a target training set and a preset decision tree algorithm, and train the target decision tree based on a preset scheduling sample to obtain a trained target decision tree; wherein the target decision tree includes a plurality of target decision subtrees; A prediction module, used to perform performance prediction on a preset scheduling to be optimized based on the trained target decision tree, and obtain a prediction result of the target decision tree; wherein the prediction result of the target decision tree includes a prediction result of each target decision subtree; An acquisition module, used to traverse all target decision subtrees in the target decision tree based on the prediction result of the target decision tree, and acquire a pruning rule for each target decision subtree; A test module is used to distinguish the to-be-optimized schedules based on the pruning rules, determine a first schedule that complies with the pruning rules and a second schedule that does not comply with the pruning rules, and perform hardware testing on the first schedule and the second schedule to obtain corresponding hardware testing results.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the search space pruning method and device for the deep learning operator as described in any one of claims 1 to 7 are performed.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the search space pruning method and device for a deep learning operator as described in any one of claims 1 to 7.

Images