A high-dimensional microarchitecture design space exploration method based on Monte Carlo tree search

Abstract

The present invention discloses a high-dimensional micro-architecture design space exploration method based on Monte Carlo tree search, comprising: constructing a design space; dynamically evaluating the importance of each parameter according to a method combining Monte Carlo tree search with Bayesian optimization, and selecting parameters with higher importance for Bayesian optimization according to the current result of the importance evaluation, and iterating until the time budget is exhausted to obtain the final Pareto design set. The method of the present invention dynamically evaluates the importance of each parameter and selects parameters with high importance for Bayesian optimization through Monte Carlo tree exploration, effectively avoiding the problem of poor effect of high-dimensional Bayesian optimization and improving the efficiency of high-dimensional parameter design space exploration. Moreover, the method is a set of automated algorithm processes, which only requires the user to provide corresponding adjustable parameter settings and an interface of an EDA simulation tool, thereby facilitating user use.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and specifically relates to a high-dimensional micro-architecture design space exploration method based on Monte Carlo tree search. Background Art

[0002] The rapid development of artificial intelligence has created a huge demand for customized hardware for neural network computing acceleration. Large language models and generative artificial intelligence require a lot of computing resources for model training and deployment. It costs a lot to design a customized and well-optimized artificial intelligence accelerator for a specific neural network. To achieve high-quality design requires professional expertise and a lot of engineering testing and revisions, which usually takes several years.

[0003] Therefore, in recent years, some systematic, fast and universal hardware development processes have been proposed and adopted by academia to reduce design overhead and speed up the design cycle. For example, Chipyard allows users to easily complete the entire design process through configurable modules and design parameters. These design parameters create a huge design space, making the optimization process a typical design space exploration (DSE) problem. Users are able to explore the design space and achieve a design that meets their specific needs by appropriately selecting and adjusting parameters.

[0004] While many tools simplify hardware design and turn it into a process of setting parameters, identifying the best parameter combination from an exponentially vast space remains a significant challenge. On the one hand, the wide design space makes it infeasible to test all potential combinations exhaustively. On the other hand, synthesizing the design and running power, performance, and area (PPA) simulations is time-consuming and can take anywhere from hours to days. Furthermore, the three metrics are highly correlated. Parameter tuning may improve one metric while adversely affecting another. This dilemma in DSE presents a significant obstacle to achieving an optimal design that balances each metric.

[0005] The latest methods for exploring the design space of hardware configurable parameters include BOOM-Explorer, GRL-DSE, and Rl-Explorer. The following will introduce the processes of these three existing technologies and their defects.

[0006] BOOM-Explorer adopts an initialization sampling method based on transductive experimental design (TED) and a heuristic sampling strategy based on multi-objective Bayesian optimization. Its process includes: 1) According to the engineer's prior knowledge, a weight is assigned to each parameter, and the entire design space is clustered by Kmeans according to the different weights assigned to each parameter, and the design space is divided into several clusters; 2) For each cluster in step 1, the most representative parameter combinations are sampled from them using the TED sampling method and these parameters are simulated by EDA to obtain PPA data; 3) According to the existing parameter combinations and corresponding PPA data, a deep kernel-based Gaussian process model is trained to fit the relationship between parameter combinations and PPA; 4) Based on the probability model of PPA for unknown combinations given by the Gaussian process model, the expected hypervolume increase (EHVI) acquisition function is used to select the next candidate combination for evaluation, and the candidate combination is simulated by EDA to obtain PPA data; 5) Steps 3 and 4 are repeated until the time cost is exhausted or the expected effect is achieved. However, the BOOM-Explorer method still has the following problems: 1) For the design space of high-dimensional parameters, the Gaussian process cannot achieve a good fit for PPA, resulting in poor performance of the entire method; 2) During the initialization process, BOOM-Explorer needs to assign weights to each parameter based on the engineer's prior knowledge. However, for the emerging novel hardware designs, engineers may not have relevant experience and knowledge and cannot make effective judgments.

[0007] GRL-DSE uses a variational graph autoencoder (VGAE) to map parameter combinations to a continuous embedding space and perform Bayesian optimization on the embedding space. Its process includes: 1) Based on the data dependencies and running order between the various components of the microprocessor, the microprocessor is mapped into a directed acyclic graph (DAG), and each component is a node in the graph; 2) Based on the collected microprocessor DAG data, VGAE is used for self-supervised training to map each microprocessor parameter combination to an embedding vector; 3) Three proxy models (Gaussian process model, deep integration model and random forest model) are trained according to the embedding vectors corresponding to the existing parameter combinations and the corresponding PPA data to fit the relationship between the embedding vector and PPA; 4) Based on the probability model of PPA of the embedding vector corresponding to the unknown combination given by the Gaussian process model, the expected hypervolume increase (EHVI) acquisition function is used to select the next candidate combination for evaluation, and the candidate combination is simulated by EDA to obtain PPA data; 5) Steps 3 and 4 are repeated until the time cost is exhausted or the expected effect is achieved. However, the GRL-DSE method still has the following problems: 1) The method of mapping the space of parameter combinations to an embedding space is not universal. For a novel circuit, it is necessary to redesign a conversion method from parameter combinations to corresponding graphs and re-collect the dataset for training VGAE, which is undoubtedly time-consuming; 2) The embedded space mapped in GRL-DSE still cannot effectively represent the design space of high-dimensional parameters.

[0008] RL-Explorer uses a multi-objective reinforcement learning method to train an agent model, which can sequentially select appropriate parameters for each microprocessor component to obtain a good PPA indicator. Its process includes: 1) Collecting parameter combinations and corresponding PPA data sets, supervising the training of a regression model, and subsequently using the output of the model as a reward in reinforcement learning training; 2) Dividing the selection of microprocessor parameter combinations into several steps, each step selects a parameter of a microprocessor component, the determined parameters are regarded as a state in reinforcement learning, and the selection of undetermined component parameters is regarded as an action in reinforcement learning. When all parameters are determined, the regression model is used to obtain PPA as a reward (only the last action will receive a reward); 3) Collecting several trajectories, training actor and critic networks; 4) Based on the trained agent model, select the parameters of the microprocessor and perform EDA simulation to obtain real PPA data. However, the RL-Explorer method still has the following problems: 1) The practice of performing reinforcement learning training on a specific circuit is not universal, because for a novel circuit, it is necessary to retrain a regression model to predict rewards (which also requires re-collecting the dataset for training the regression model) and retrain the reinforcement learning model, which is undoubtedly extremely time-consuming; 2) For the design space of high-dimensional parameters, RL-Explorer has to design more complex states and actions, which will greatly reduce the final effect of the method. Summary of the invention

[0009] In order to make up for the deficiencies of the prior art, the present invention aims to provide a high-dimensional micro-architecture design space exploration method based on Monte Carlo tree search, so as to enable the proxy model to better fit the potential relationship between parameter combinations and PPA under limited samples, and improve the efficiency of high-dimensional parameter design space exploration. To achieve this purpose, the specific technical solutions of this application are as follows:

[0010] The high-dimensional micro-architecture design space exploration method based on Monte Carlo tree search includes:

[0011] Construct design space;

[0012] According to the method combining Monte Carlo tree search and Bayesian optimization, the importance of each parameter is dynamically evaluated, and the parameters with higher importance are selected for Bayesian optimization according to the current results of importance evaluation. This iteration is repeated until the time budget is exhausted to obtain the final Pareto design set.

[0013] Furthermore, in order to quantify the importance of each parameter, a parameter index set is set , where d represents the number of parameters, x is a microarchitecture parameter combination embedding, and each dimension represents the value of the corresponding parameter. for A subset of I To simplify the embedding of microarchitecture parameters, the calculation of parameter importance is shown in the following formula:

[0014] ,

[0015] in, is a constructed set whose elements are a pair , It contains multiple groups according to Generated microarchitecture embedded PPA value A collection of Indicated by and a selected reference point The hypervolume enclosed by Will Map to , i.e. a d-dimensional vector, where at the i-th position, 0 means the i-th parameter is selected, and 1 means the opposite; yes middle The number of How many sets of PPA data are included? is a vector.

[0016] Furthermore, the specific steps of the method combining Monte Carlo tree search with Bayesian optimization are:

[0017] S1. From Random Selection Index subset and the corresponding complement , for each and , random sampling parameter combinations and use EDA simulation tools to obtain their PPA values;

[0018] S2. Initialization ,in and express The PPA value of the sampling point is initialized using Calculate the importance score ;

[0019] S3. Score by importance Calculate the root node and the corresponding UCB,initialize the root node in the Monte Carlo tree;

[0020] S4. Loop and execute the Bayesian optimization algorithm iterations until the time budget is exhausted.

[0021] Furthermore, the specific steps of each algorithm iteration in step S4 are as follows:

[0022] S41. Select the node to be operated in this loop according to the UCB value of each node in the Monte Carlo tree;

[0023] S42. Evaluate the importance of the parameters in the node selected in step S41, Randomly selected Index subset and the corresponding complement ;

[0024] S43. For each and , simplified microarchitecture embedding using all explored parameter combinations and the corresponding PPA data To train a Gaussian process as a proxy model;

[0025] S44. Use joint entropy search as the acquisition function in Bayesian optimization, select A simplified microarchitecture embedding that maximizes access functionality;

[0026] S45. Embed the newly acquired simplified microarchitecture Completed to full microarchitecture embedding , and perform EDA simulation on the complete microarchitecture embedding to obtain PPA data;

[0027] S46. Each group in steps S43 to S45 Record to collection and recalculate the importance of each variable;

[0028] S47. According to the updated scores, the parameters in the current node are divided into two child nodes, the more important parameters are divided into the left child node, and the other parameters are divided into the right child node.

[0029] Furthermore, in step S41, starting from the root node of the Monte Carlo tree, a child node with a larger UCB is selected when a branch is encountered until a leaf node is encountered. The leaf node is the node to be operated. The UCB value of each node X can be calculated by the following formula:

[0030] ,

[0031] In the formula, Indicates the quality evaluation of the node. is a hyperparameter used for balancing, Indicates the total number of times X's parent node is visited, Indicates the number of times X is visited; It can be calculated by the following formula:

[0032] ,

[0033] In the formula, It is the index set of the variables represented by each node in the Monte Carlo tree. The nodes in the Monte Carlo tree and the variables represented by the nodes are dynamically generated during the operation of the algorithm.

[0034] Furthermore, in step S43, the simplified micro-architecture embedding definition is as follows: , where d represents the number of parameters, is a microarchitecture parameter combination embedding (each dimension represents the value of the corresponding parameter), for a subset of (indices representing some parameters), To simplify the embedding of microarchitectural parameters (whose elements are based on from selected in ).

[0035] Furthermore, in step S44, the joint entropy search obtains the function formula as follows:

[0036] ,

[0037] In the formula, is a simplified microarchitectural embedding of the true Pareto optimal set of design configurations, is the PPA of the true Pareto optimal set; since the true Pareto optimal set is unknown, in order to calculate the formula, Monte Carlo sampling is used to obtain an approximate Pareto set and , is the number of samples.

[0038] Furthermore, in step S45, the newly acquired simplified micro-architecture needs to be embedded into Completed to full microarchitecture embedding , specifically: for The missing parameters in are filled by averaging the parameters of the current Pareto point set.

[0039] Compared with the prior art, the present invention has the following advantages:

[0040] (1) The present invention dynamically evaluates the importance of each parameter through Monte Carlo tree exploration and selects the parameters with high importance for Bayesian optimization, which effectively avoids the problem of poor effect of high-dimensional Bayesian optimization and improves the efficiency of high-dimensional parameter design space exploration.

[0041] (2) The present invention is an automated algorithm process that only requires the user to provide corresponding adjustable parameter settings and an interface for an EDA simulation tool, thereby facilitating user use and having strong practicality.

[0042] (3) The present invention has strong versatility and can be used for parameter selection of various hardware.

[0043] (4) The results of the algorithm of the present invention can provide an evaluation of the importance of each parameter, provide interpretability during the algorithm execution process, and provide designers with corresponding feedback to guide subsequent hardware design. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0045] The following is a further description of a high-dimensional micro-architecture design space exploration method based on Monte Carlo tree search of the present invention in conjunction with the accompanying drawings.

[0046] like Figure 1 As shown in the figure, a high-dimensional microarchitecture design space exploration method based on Monte Carlo tree search is firstly performed to construct the design space. Secondly, according to the method combining Monte Carlo tree search with Bayesian optimization, the importance of each parameter is dynamically evaluated, and the parameters with higher importance are selected for Bayesian optimization according to the current result of the importance evaluation. This iteration is performed until the time budget is exhausted to obtain the final Pareto design set.

[0047] In order to quantify the importance of each parameter, the present invention defines a parameter index set , where d represents the number of parameters, is a microarchitecture parameter combination embedding (each dimension represents the value of the corresponding parameter), for a subset of (indices representing some parameters), To simplify the embedding of microarchitectural parameters (whose elements are based on from The calculation of parameter importance is shown in the following formula:

[0048] ,

[0049] in is a set we construct, whose elements are a pair , It contains multiple groups according to Generated microarchitecture embedded PPA value A collection of Indicated by and a selected reference point The hypervolume enclosed by Will Map to (a d-dimensional vector where, at the i-th position, 0 means the i-th parameter is selected and 1 means the opposite), yes middle The number of How many groups of PPA data are included in it), is a vector; the above calculation adopts the one-to-one division of the elements in the numerator and denominator vectors.

[0050] The specific steps of the method combining Monte Carlo tree search and Bayesian optimization are:

[0051] (1) From Random Selection Index subset and the corresponding complement , for each and , random sampling parameter combinations, and use EDA simulation tools to obtain their PPA values; for example, in a specific example, =2 and =2, determine the parameter dimension d=10 of the current design space exploration task and obtain = .

[0052] (2) Initialization ,in and express The PPA value of the sampling point is initialized using Calculate the importance score .

[0053] (3) Score based on importance Calculate the root node and the corresponding UCB, initializing the root node in the Monte Carlo tree.

[0054] The Bayesian optimization algorithm is iterated in a loop until the time budget is exhausted.

[0055] Specifically, the specific steps of the Bayesian optimization algorithm iteration are as follows:

[0056] ① Select the node to be operated in this cycle according to the UCB value of each node in the Monte Carlo tree. The process starts from the root node of the Monte Carlo tree, and selects the child node with a larger UCB when encountering a branch until a leaf node is encountered. The leaf node is the node to be operated. The UCB value of each node X can be calculated by the following formula:

[0057] ,

[0058] In the formula, Indicates the quality evaluation of the node. is a hyperparameter used for balancing, Indicates the total number of times X's parent node is visited, represents the number of times X is visited. Taking into account the quality of nodes and the access frequency, UCB balances exploration and utilization. It can be calculated by the following formula:

[0059] ,

[0060] In the formula, It is the index set of the variables represented by each node in the Monte Carlo tree. The nodes in the Monte Carlo tree and the variables represented by the nodes are dynamically generated during the operation of the algorithm.

[0061] ② Further evaluate the importance of the parameters in the nodes selected in step 1. In order to more clearly distinguish the importance of the parameters in the nodes, the present invention performs additional simulations on the parameters in the nodes. Randomly selected Index subset and the corresponding complement .

[0062] ③For each and , simplified microarchitecture embedding using all explored parameter combinations and the corresponding PPA data To train a Gaussian process as a proxy model.

[0063] Simplified microarchitecture embedding is defined as: , where d represents the number of parameters, is a microarchitecture parameter combination embedding (each dimension represents the value of the corresponding parameter), for a subset of (indices representing some parameters), To simplify the embedding of microarchitectural parameters (whose elements are based on from selected in ).

[0064] ④Use joint entropy search as the acquisition function in Bayesian optimization and select A simplified microarchitecture embedding that maximizes the acquisition function. The joint entropy search acquisition function formula is as follows:

[0065] ,

[0066] In the formula, is a simplified microarchitectural embedding of the true Pareto optimal set of design configurations, is the PPA of the true Pareto optimal set. Since the true Pareto optimal set is unknown, in order to calculate the formula, Monte Carlo sampling is used to obtain an approximate Pareto set. and , is the number of samples.

[0067] ⑤Embed the newly acquired simplified microarchitecture Completed to full microarchitecture embedding , and perform EDA simulation on the complete microarchitecture embedding to obtain PPA data.

[0068] Among them, the newly acquired simplified microarchitecture needs to be embedded in Completed to full microarchitecture embedding . Specifically, for The missing parameters in are filled by averaging the parameters of the current Pareto point set.

[0069] ⑥ Each group in steps ③ to ⑤ Record to collection , and recalculate the importance of each variable.

[0070] ⑦ According to the updated score, the parameters in the current node are divided into two child nodes. The more important parameters are divided into the left child node, and the other parameters are divided into the right child node. In addition, since the updated importance score will change the UCB of the ancestor node of the current node; therefore, the present invention updates the UCB of all nodes on the path from the current node to the root node through back propagation.

[0071] The method of the present invention dynamically evaluates the importance of each parameter through Monte Carlo tree exploration and selects the parameters with high importance for Bayesian optimization, which effectively avoids the problem of poor effect of high-dimensional Bayesian optimization and improves the efficiency of high-dimensional parameter design space exploration. In addition, the method is an automated algorithm flow, which only requires the user to provide the corresponding adjustable parameter settings and the interface of the EDA simulation tool, so as to facilitate the use of the user.

[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A high-dimensional microarchitecture design space exploration method based on Monte Carlo tree search, characterized in that: The method comprises: Construct design space; According to the method combining Monte Carlo tree search and Bayesian optimization, the importance of each parameter is dynamically evaluated, and the parameters with higher importance are selected for Bayesian optimization according to the current results of importance evaluation. This process is repeated until the time budget is exhausted, and the final Pareto design set is obtained. In order to quantify the importance of each parameter, a parameter index set is set Where d represents the number of parameters, x is a microarchitecture parameter combination embedding, and each dimension represents the value of the corresponding parameter. for A subset of I To simplify the embedding of microarchitecture parameters, the calculation of parameter importance is shown in the following formula: in, is a constructed set whose elements are a pair It contains multiple groups according to The set of PPA values ​​y generated for the microarchitecture embedding, HV(f ref , y) represents the relationship between y and a selected reference point f ref The hypervolume enclosed by Mapping to {0, 1} d , i.e. a d-dimensional vector, where at the i-th position, 0 means the i-th parameter is selected, and 1 means the opposite; yes The number of y in How many groups of PPA data are included; s is a vector; The specific steps of the method combining Monte Carlo tree search and Bayesian optimization are: S1. From Randomly select N v Index subset and the corresponding complement For each and Random sampling N s parameter combinations and use EDA simulation tools to obtain their PPA values; S2. Initialization in and N s The PPA value of the sampling point is initialized using Calculate the importance score s; S3. Calculate the root node’s υ based on the importance score s X and the corresponding UCB,initialize the root node in the Monte Carlo tree; S4. Loop and execute the Bayesian optimization algorithm iterations until the time budget is exhausted.

2. The high-dimensional micro-architecture design space exploration method based on Monte Carlo tree search according to claim 1, characterized in that: The specific steps of each algorithm iteration in step S4 are as follows: S41. Select the node to be operated in this loop according to the UCB value of each node in the Monte Carlo tree; S42. Evaluate the importance of the parameters in the node selected in step S41, Randomly select N υ Index subset and the corresponding complement S43. For each and Simplified microarchitecture embedding using all explored parameter combinations and the corresponding PPA data Y to train the Gaussian process as a proxy model; S44. Use joint entropy search as the acquisition function in Bayesian optimization, select N s A simplified microarchitecture embedding that maximizes access functionality; S45. Embed the newly acquired simplified microarchitecture into Completed to a complete microarchitecture embed x j , and perform EDA simulation on the complete microarchitecture embedding to obtain PPA data; S46. Each group in steps S43 to S45 Record to collection and recalculate the importance of each variable; S47. According to the updated scores, the parameters in the current node are divided into two sub-nodes, the more important parameters are divided into the left sub-node, and the other parameters are divided into the right sub-node.

3. The high-dimensional micro-architecture design space exploration method based on Monte Carlo tree search according to claim 2, characterized in that: In step S41, starting from the root node of the Monte Carlo tree, a child node with a larger UCB is selected when a branch is encountered until a leaf node is encountered. The leaf node is the node to be operated. The UCB value of each node X can be calculated by the following formula: In the formula, υ X Indicates the quality evaluation of the node, C p is a hyperparameter used for balancing, n p Indicates the total number of times X's parent node is visited, n X Indicates the number of times X is visited; υ X It can be calculated by the following formula: In the formula, It is the index set of the variables represented by each node in the Monte Carlo tree. The nodes in the Monte Carlo tree and the variables represented by the nodes are dynamically generated during the operation of the algorithm.

4. The high-dimensional micro-architecture design space exploration method based on Monte Carlo tree search according to claim 2, characterized in that: In step S43, the simplified micro-architecture embedding definition is as follows: Where d represents the number of parameters, x is a microarchitecture parameter combination embedding, for A subset of I For simplified microarchitectural parameter embedding.

5. The high-dimensional micro-architecture design space exploration method based on Monte Carlo tree search according to claim 2, characterized in that: In step S44, the joint entropy search obtains the function formula as follows: In the formula, is a simplified microarchitectural embedding of the true Pareto optimal set of design configurations, is the PPA of the true Pareto optimal set; since the true Pareto optimal set is unknown, in order to calculate the formula, Monte Carlo sampling is used to obtain an approximate Pareto set and S is the number of samples.

6. The high-dimensional micro-architecture design space exploration method based on Monte Carlo tree search according to claim 2, characterized in that: In step S45, the newly acquired simplified micro-architecture needs to be embedded into Completed to a complete microarchitecture embed x j , specifically: for The missing parameters in are filled by averaging the parameters of the current Pareto point set.

Images