Process mapping cutset optimization method based on reinforcement learning and bidirectional graph attention

Abstract

The application discloses a process mapping cut set optimization method based on reinforcement learning and bidirectional graph attention, comprising: performing multi-stage mapping on a non-graph, constructing an action space for each AND gate node; extracting node features, constructing a forward and reverse edge set to obtain a graph state; using a bidirectional graph attention network to transmit messages along the forward and reverse edge set, splicing and fusing to obtain the embedding of each AND gate node; outputting the probability distribution of all AND gate nodes on the action space and sampling to obtain a global action vector; outputting a state value estimate; completing netlist generation and timing evaluation, and calculating a reward; taking the graph state, action vector and reward as inputs, performing genetic algorithm expert search, imitation learning pre-training, proximal policy optimization exploration and frozen encoder adaptive fine-tuning to obtain a final policy network. The method directly takes the quality of service after mapping as an optimization target, does not require pre-labeled data, significantly reduces delay, and supports delay-area trade-off optimization under area constraints.

Description

Technical Field

[0001] This invention mainly relates to the field of integrated circuit electronic design automation technology, specifically to a process mapping cut set optimization method based on reinforcement learning and bidirectional graph attention. Background Technology

[0002] Electronic Design Automation (EDA) refers to the technology of achieving efficient and high-quality design through the use of automated tools in the integrated circuit design process. In the integrated circuit design flow, technology mapping is the process of generating a NAND graph from technology-independent logic synthesis. Inverter Graph (AIG) mapping is a crucial step in the gate-level netlist of the standard cell library, and its mapping quality directly determines the delay, area, and power consumption of the final circuit.

[0003] Modern process mappers generally employ algorithms based on cut-set enumeration and dynamic programming: for each node in the AIG, enumerate all k... Feasible cut sets are mapped to feasible implementations in the standard cell library through Boolean matching, and then dynamic programming selects the optimal coverage under a specified delay or area objective. However, the above traditional methods have the following shortcomings: First, the cut set evaluation stage relies on a simplified linear delay model. There is a significant deviation between the cost estimate of the mapper when selecting cut sets and the actual delay obtained by the time series engine after mapping, which makes the mapper prone to getting trapped in local optima and cut set selection errors are widespread. Second, existing supervised learning methods that introduce machine learning into the mapper model cut set selection as a classification or regression task. The labels still come from the heuristic mapper itself and are limited by the quality upper bound of the heuristic labels. At the same time, each node is treated as an independent task, and the structural dependencies between nodes caused by shared fan-out and critical path propagation are not explicitly modeled. Third, existing methods that introduce reinforcement learning into the EDA process are mostly applied outside the mapper, such as tuning multi-parameter integrated command sequences or pruning subsets of the cell library in a multi-armed slot machine manner. These methods treat the mapper as a black box and do not touch the cut set selection decision inside the mapper.

[0004] Cut set selection optimization in existing process mapping flows faces two intertwined fundamental challenges: first, the joint action space exhibits a combinatorial explosion on the order of O(K^n), where... Here, n represents the number of candidate cut sets for each node, and n is the number of AND gate nodes in the AIG. Secondly, the cut set selections of different nodes influence each other through shared fan-out and critical paths, and the locally optimal decisions made independently by each node do not constitute a globally optimal mapping. How to efficiently search for globally cooperative cut set schemes within the mapper and in the exponential combination space, and explicitly model the structural dependencies between nodes, is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] To address the technical problems of existing technologies, this invention provides a method that directly optimizes the quality of service after mapping, eliminates the need for pre-labeled data, significantly reduces latency, and supports latency under area constraints. An optimization method for process mapping cut sets based on reinforcement learning and bidirectional graph attention to achieve area trade-offs.

[0006] To solve the above-mentioned technical problems, the technical solution proposed by this invention is as follows: A process mapping cutset optimization method based on reinforcement learning and bidirectional graph attention includes the following steps: S1. For the NAND graph, call the process mapper to perform multi-stage mapping, taking the best cut set selected for each AND gate node in each stage as a candidate action for that AND gate node, and constructing a process map for each AND gate node. The node-level action space consists of candidate cut sets; S2. Extract the node feature vector of each AND gate node in the NAND graph, and construct the forward edge set and the reverse edge set according to the NAND graph to obtain the graph state; S3. A bidirectional graph attention network is used to perform multi-layer message passing on the node feature vectors along the forward edge set and the reverse edge set respectively to obtain the forward hidden embedding and the reverse hidden embedding. The forward hidden embedding and the reverse hidden embedding are concatenated and then linearly fused to obtain the fused embedding of each AND gate node. S4. The Actor-Critic architecture in reinforcement learning is adopted. The embeddings corresponding to the gate nodes in the fusion embedding are used as inputs. The Actor head samples the probability distribution of all gate node outputs in the node-level action space in one forward propagation to obtain the global action vector. At the same time, the Critic head outputs the state value estimate of each gate node. S5. Inject the global action vector back into the process mapper through the extended interface of the process mapper to complete the netlist generation and timing evaluation, and calculate the reward signal based on the delay improvement rate of the mapping result relative to the baseline mapping result. S6. Using the graph state as the environment state, the global action vector as the policy output, and the reward signal as the fitness, the four training stages of genetic algorithm expert search, imitation learning pre-training, proximal policy optimization exploration, and frozen encoder adaptive fine-tuning are executed in sequence to update the policy network parameters and obtain the final policy network for process mapping cut set optimization.

[0007] Preferably, the specific process of step S1 is as follows: S101, For NAND graphs, call the process mapper to execute sequentially. Each of the mapping stages includes cut set enumeration, Boolean matching, and dynamic programming coverage, and each stage has its own independent optimization objective. S102. After each stage of matching is completed, the best cut set finally selected for each AND gate node in that stage is saved to the cut set record field of that node in that stage. S103, Connect each AND gate node in The optimal cut sets at each stage are aggregated to form the candidate action set for that node. The entire joint action space with the gate node is ,in This represents the total number of AND gate nodes in the AND-NOT graph; in The four mapping stages correspond to stages 0, 1, 2 and 3 of the ABC process mapper's map command. Stage 0 aims for the lowest latency, stage 1 reselects based on area flow under latency constraints, stage 2 greedily optimizes in a precise area manner under latency constraints, and stage 3 performs area recovery again after considering phase allocation.

[0008] Preferably, the specific process of step S2 is as follows: S201. Extract a 7-dimensional feature vector [TypePI, TypeInner, TypePO, Level, RLevel, Fanout, Inv] for each node in the AND-NOT graph, where TypePI, TypeInner, and TypePO are the one-hot encodings of the node type, Level and RLevel are the normalized forward and reverse levels, respectively, Fanout is the fan-out number after being transformed by the natural logarithm plus one, and Inv is the binary indicator of the inverse attribute. S202. Construct a positive edge set based on the connection relationships between nodes in and outside the graph. With reverse edge set , In the diagram, each edge points from the fan-in node to the current node. Each edge in the middle is in the opposite direction; S203. Based on the node type, mark the set of AND gate nodes to obtain the AND gate node mask, which is used to extract the embedding row corresponding to the AND gate node from the fused embedding.

[0009] Preferably, the specific process of step S3 is as follows: S301. Taking the node feature vector as input, and then... The layered graph attention network uses convolutional layers to pass messages along the forward edge set, with each layer using... H Each attention head uses ELU as the activation function, and the output is the positive hidden embedding. ; S302, Using the same node feature vector as input, and passing through another set of... The layered graph attention network performs message passing along the inverse edge set in its convolutional layers. The network structure is symmetrical with the first path, and the output is the inverse hidden embedding. ; S303, will and After being stitched along the channel dimension, it is mapped by a linear fusion layer to... Dimensional Fusion Embedding ;in The weight matrix is ​​a linear fusion matrix; The total number of all nodes; To fuse the embedded feature dimensions; in , , The fused embedding simultaneously encodes the driving cone context of the node along the forward edge and the load cone context along the reverse edge.

[0010] Preferably, the specific process of step S4 is as follows: S401. Extract the embedding matrix corresponding to the AND gate node from the fused embedding based on the AND gate node mask. ; S402, will Input by The Actor head, composed of a fully connected multilayer perceptron and Softmax, outputs each AND gate node. Probability distribution of candidate actions And sample the global action vector from each distribution. ; S403, will The input structure is symmetrical The Critic head, composed of a fully connected multilayer sensing mechanism, outputs a scalar state value estimate for each AND gate node. .

[0011] Preferably, the specific process of step S5 is as follows: S501. Write the global action vector line by line into the external selection file in the format of "node identifier, stage index". S502: Read the selection file through the injection interface of the process mapper and replace the cut set of each AND gate node with the best cut set of that node in the corresponding stage. S503. Based on the replaced cut set, perform netlist generation and timing evaluation, and parse the delay of this mapping from the evaluation report. With area ; S504, Delay based on baseline mapping results Calculate the delay improvement rate and use it as a reward signal .

[0012] Preferably, in step S504, the reward signal for:

[0013] or

[0014] in The area mapped to the baseline. For the area budget ratio, This is the area penalty coefficient; when The reward is equivalent to the pure delay improvement rate; when At that time, any amount exceeding the budget will be penalized proportionally; through adjustments... The value can be optimized for delay under different area budgets.

[0015] Preferably, the specific process of step S6 is as follows: S601, Genetic Algorithm Expert Search: Treat the global action vector as an individual, use the reward signal obtained in step S5 as the fitness, and perform [analysis] on the node-level action space. Through generational population evolution, individuals with positive rewards and their reward values ​​in the last generation of the population are selected to form an expert sample set. S602, Imitation Learning Pre-training: Using the expert sample set as supervision, the bidirectional graph attention network, Actor head and Critic head are jointly pre-trained according to the weighted sum of behavior cloning loss and value function mean square error loss; S603, Proximity Strategy Optimization Exploration: The trajectory is collected through interaction with the process mapper according to the proximity strategy optimization algorithm, and the parameters of the bidirectional graph attention network, Actor head and Critic head are updated. S604, Freeze Encoder Adaptive Fine-Tuning: Freeze the parameters of the bidirectional graph attention network, and continue to train only the Actor head and Critic head according to the proximal policy optimization algorithm to obtain the final policy network.

[0016] Preferably, the specific process of step S601 is as follows: S6011, Initialization size is A random population of individuals, where each individual is a global action vector; S6012. Perform an environmental assessment on each individual in the current population to obtain the fitness value of that individual; S6013, sequentially perform tournament selection, uniform crossover, and sort by mutation rate. random variation and proportional variation The elite retention operation produces the next generation of the population; S6014, Repeat S6012 and S6013 together Evolution will cease after this; S6015. Select individuals with positive rewards from the last generation of the population and their reward values ​​to form an expert sample set. , as a monitoring signal.

[0017] Preferably, the loss function for joint pre-training described in step S602... for:

[0018] in For expert sample set The number of samples, The total number of AND gate nodes, An index for the expert sample; For the index of the AND gate node; For the policy network in state Next, for the first Each AND gate node outputs expert actions. The probability of; is the natural logarithm of the probability; The loss weight coefficients are for the value function; Representing state Next The expected cumulative reward of each AND gate node; Indicates the first The reward signal corresponding to each expert sample.

[0019] Compared with the prior art, the advantages of the present invention are as follows: The method of this invention models the cutset selection of each AND gate node in the AND-NOT graph as a reinforcement learning problem, directly using the mapped real service quality as the reward signal; it simultaneously captures the driving cone and load cone context of the node through a bidirectional graph attention network; and it uses Actor... The Critic head makes joint decisions on all AND gate nodes in a single forward propagation, implicitly modeling the structural dependencies between nodes. The progressive training paradigm of "genetic algorithm → imitation learning → proximal policy optimization → frozen encoder fine-tuning" effectively overcomes the cold start problem in a large-scale combined action space.

[0020] This invention formalizes the cut set selection in process mapping into a graph-based combinatorial optimization problem, and solves it directly through reinforcement learning. It uses the real service quality obtained by the time-series engine after mapping as the reward signal, avoiding dependence on pre-labeled data and breaking through the bottleneck of supervised learning paradigms being limited by the upper bound of heuristic label quality.

[0021] The bidirectional graph attention network encoder proposed in this invention simultaneously performs message passing and fusion along the forward and reverse edges of the AIG, and the fusion embedding includes context information of both the driving cone and the load cone; the Actor head outputs the action probability distribution to all AND gate nodes in one forward propagation, and the structural dependencies between different nodes are implicitly modeled through the multi-hop message passing of the bidirectional graph attention network, overcoming the problem that node-by-node independent decision-making cannot form a globally optimal mapping.

[0022] The progressive training paradigm proposed in this invention, namely "genetic algorithm search → imitation learning pre-training → reinforcement learning exploration → frozen encoder fine-tuning", utilizes the genetic algorithm to locate the optimal solution region under zero prior conditions and distill it into the policy network as a good initialization, effectively overcoming the low cold-start efficiency problem of reinforcement learning in the O(K^n)-level combined action space; the frozen encoder fine-tuning stage further enables the policy to quickly adapt to specific designs while retaining the learned graph representation. Attached Figure Description

[0023] Figure 1 This is a flowchart of an embodiment of the reinforcement learning-based process mapping cutset selection optimization method of the present invention.

[0024] Figure 2 The bidirectional graph attention network and Actor of this invention A schematic diagram of the Critic strategy network.

[0025] Figure 3 The cut set selection and combination action space utilized in this invention is in area Scatter plot of distribution over a two-dimensional delay.

[0026] Figure 4 To superimpose the area of ​​the results from each training stage of this invention with different area constraints. Delayed scatter plot.

[0027] Figure 5This is a graph showing the relationship between the delay improvement rate of the present invention under different area budget ratios α.

[0028] Figure 6 This is a comparison of the convergence curves of the complete progressive training pipeline of this invention and the reinforcement learning-only scheme. Detailed Implementation

[0029] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0030] like Figure 1 and Figure 2 As shown, the process mapping cutset optimization method based on reinforcement learning and bidirectional graph attention provided in this embodiment of the invention includes the following steps: S1. For the NAND graph, call the process mapper to perform multi-stage mapping, taking the best cut set selected for each AND gate node in each stage as a candidate action for that AND gate node, and constructing a process map for each AND gate node. The node-level action space consists of candidate cut sets; S2. Extract the node feature vector of each AND gate node in the NAND graph, and construct the forward edge set and the reverse edge set according to the NAND graph to obtain the graph state; S3. A bidirectional graph attention network is used to perform multi-layer message passing on the node feature vectors along the forward edge set and the reverse edge set respectively to obtain the forward hidden embedding and the reverse hidden embedding. The forward hidden embedding and the reverse hidden embedding are concatenated and then linearly fused to obtain the fused embedding of each AND gate node. S4. The Actor-Critic architecture in reinforcement learning is adopted. The embeddings corresponding to the gate nodes in the fusion embedding are used as inputs. The Actor head samples the probability distribution of all gate node outputs in the node-level action space in one forward propagation to obtain the global action vector. At the same time, the Critic head outputs the state value estimate of each gate node. S5. Inject the global action vector back into the process mapper through the extended interface of the process mapper to complete the netlist generation and timing evaluation, and calculate the reward signal based on the delay improvement rate of the mapping result relative to the baseline mapping result. S6. Using the graph state as the environment state, the global action vector as the policy output, and the reward signal as the fitness, the four training stages of genetic algorithm expert search, imitation learning pre-training, proximal policy optimization exploration, and frozen encoder adaptive fine-tuning are executed in sequence to update the policy network parameters and obtain the final policy network for process mapping cut set optimization.

[0031] In one specific embodiment, the specific process of step S1 is as follows: S101, The NAND mapper is invoked and executed sequentially. Each of the mapping stages includes cut set enumeration, Boolean matching, and dynamic programming coverage, and each stage has its own independent optimization objective. S102. After each stage of matching is completed, the best cut set finally selected for each AND gate node in that stage is saved to the cut set record field of that node in that stage. S103, Connect each AND gate node in The optimal cut sets at each stage are aggregated to form the candidate action set for that node. The entire joint action space with the gate node is ,in This represents the total number of AND gate nodes in the AND-NOT graph; in The four mapping stages correspond to stages 0, 1, 2 and 3 of the ABC process mapper's map command. Stage 0 aims for the lowest latency, stage 1 reselects based on area flow under latency constraints, stage 2 greedily optimizes in a precise area manner under latency constraints, and stage 3 performs area recovery again after considering phase allocation.

[0032] In one specific embodiment, the specific process of step S2 is as follows: S201. Extract a 7-dimensional feature vector [TypePI, TypeInner, TypePO, Level, RLevel, Fanout, Inv] for each node in the AND-NOT graph. Here, TypePI, TypeInner, and TypePO are the one-hot encodings of the node type, Level and RLevel are the normalized forward and reverse levels, respectively, Fanout is the fan-out number after ln(nRefs+1) transformation (natural logarithm plus one transformation), and Inv is the binary indicator of the inverse attribute. S202. Construct a positive edge set based on the connection relationships between nodes in and outside the graph. With reverse edge set , In the diagram, each edge points from the fan-in node to the current node. Each edge in the middle is in the opposite direction; S203. Based on the node type, mark the set of gate nodes to obtain the gate node mask, which is used to extract the embedding row corresponding to the gate node from the fusion embedding in step S4.

[0033] In one specific embodiment, the specific process of step S3 is as follows: S301. Taking the node feature vector as input, and then... The layered graph attention network uses convolutional layers to pass messages along the forward edge set, with each layer using... Each attention head uses ELU as the activation function, and the output is the positive hidden embedding. ; S302, Using the same node feature vector as input, and passing through another set of... The layered graph attention network performs message passing along the inverse edge set in its convolutional layers. The network structure is symmetrical with the first path, and the output is the inverse hidden embedding. ; S303, will and After being stitched along the channel dimension, it is mapped by a linear fusion layer to... Dimensional Fusion Embedding ; in , , The fused embedding simultaneously encodes the driving cone context of the node along the forward edge and the load cone context along the reverse edge.

[0034] In one specific embodiment, the specific process of step S4 is as follows: S401. Extract the embedding matrix corresponding to the AND gate node from the fused embedding based on the AND gate node mask. ; S402, will Input by The Actor head, composed of a fully connected multilayer perceptron and Softmax, outputs each AND gate node. Probability distribution of candidate actions And sample the global action vector from each distribution. ; S403, will The input structure is symmetrical The Critic head, composed of a fully connected multilayer sensing mechanism, outputs a scalar state value estimate for each AND gate node. ; The action probability distribution and state value estimation of all AND gate nodes are obtained simultaneously in one forward propagation, and the structural dependencies between different AND gate nodes are implicitly modeled through multi-hop neighborhood aggregation of the bidirectional graph attention network in step S3.

[0035] In one specific embodiment, the specific process of step S5 is as follows: S501. Write the global action vector line by line into the external selection file in the format of "node identifier, stage index". S502: Read the selection file through the injection interface of the process mapper and replace the cut set of each AND gate node with the best cut set of that node in the corresponding stage. S503. Based on the replaced cut set, perform netlist generation and timing evaluation, and parse the delay of this mapping from the evaluation report. With area ; S504, Delay based on baseline mapping results Calculate the delay improvement rate and use it as a reward signal :

[0036] The extended interfaces of the process mapper include an export interface and a back-injection interface. When the process mapper is called to obtain the graph state, the export interface outputs the node features of the AND and NOT graphs and the connection relationships between nodes in the form of a structured file. The back-injection interface replaces the cut set of each AND gate node in step S502.

[0037] In step S504, the reward signal supports area constraint expansion, and the reward function is changed to:

[0038] in The area mapped to the baseline. For the area budget ratio, This is the area penalty coefficient; when The reward is equivalent to the pure delay improvement rate; when At that time, any amount exceeding the budget will be penalized proportionally; through adjustments... The value can be optimized for delay under different area budgets.

[0039] In one specific embodiment, step S6 is performed as follows: S601, Genetic Algorithm Expert Search: Treat the global action vector as an individual, use the reward signal obtained in step S5 as the fitness, and perform [analysis] on the node-level action space. Through generational population evolution, individuals with positive rewards and their reward values ​​in the last generation of the population are selected to form an expert sample set. S602, Imitation Learning Pre-training: Using expert sample sets as supervision, the bidirectional graph attention network, Actor head and Critic head are jointly pre-trained according to the weighted sum of behavior cloning loss and value function mean square error loss; S603, Proximity Strategy Optimization Exploration: The trajectory is collected through interaction with the process mapper according to the proximity strategy optimization algorithm, and the parameters of the bidirectional graph attention network, Actor head and Critic head are updated. S604, Freeze Encoder Adaptive Fine-Tuning: Freeze the parameters of the bidirectional graph attention network, and continue to train only the Actor head and Critic head according to the proximal policy optimization algorithm to obtain the final policy network; The near-end policy optimization algorithm models cut set selection as a single-step Markov decision process, and the advantage estimation is simplified to A=RV.

[0040] The specific process of step S601 is as follows: S6011, Initialization size is A random population of individuals, where each individual is a global action vector; S6012. Perform the environmental assessment of step S5 on each individual in the current population to obtain the fitness value of that individual. S6013, sequentially perform tournament selection, uniform crossover, and sort by mutation rate. random variation and proportional variation The elite retention operation produces the next generation of the population; S6014, Repeat S6012 and S6013 together Evolution will cease after this; S6015. Select individuals with positive rewards from the last generation of the population and their reward values ​​to form an expert sample set. This serves as a monitoring signal for step S602.

[0041] The loss function for joint pre-training in step S602 for:

[0042] in For expert sample set The number of samples, The total number of AND gate nodes, An index for the expert sample; For the index of the AND gate node; For the policy network in state Next, for the first Each AND gate node outputs expert actions. The probability of; is the natural logarithm of the probability; The loss weight coefficients are for the value function; Representing state Next The expected cumulative reward of each AND gate node; Indicates the first The reward signal corresponding to each expert sample.

[0043] Each round of execution in step S603 The global action and its reward are obtained through multiple rollout samplings and reused on each batch of data. Multiple rounds of gradient updates are performed. The total loss includes the pruning target loss, the mean squared error loss of the value function, and the entropy regularization term. Gradients are pruned, and the historical best reward is continuously tracked and the corresponding policy parameters are saved in real time during the training process. In step S604, the frozen bidirectional graph attention network parameters account for more than 80% of the trainable parameters of the entire policy network, and only the Actor head and Critic head are updated.

[0044] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0045] like Figure 1 As shown, the process mapping cutset optimization method based on reinforcement learning and bidirectional graph attention provided in this embodiment of the invention includes the following steps: S1. For the input NAND graph, call the process mapper to perform multi-stage mapping, taking the best cut set selected for each AND gate node in each stage as a candidate action for that node, and constructing a process map for each AND gate node. The node-level action space consists of candidate cut sets; the specific process of step S1 includes: In the source code mapperInt.h of the ABC process mapper New Structure [5][2] Fields are used to record the best cut set pointer for each AND gate node at each mapping stage and for each polarity; pCutBestPhase[mode][polarity] is saved after each stage matching process in mapperMatch.c. The four serial stages of calling the process mapper to execute the map command on the AND-NOT graph are as follows: S101. Perform cut set enumeration, Boolean matching and dynamic programming coverage in stages 0 to 3 in sequence. Stage 0 selects cut sets with the goal of minimizing delay. Stage 1 reselects cut sets according to area flow under the delay constraints obtained in stage 0. Stage 2 optimizes greedily in the form of precise area under the same delay constraints. Stage 3 performs area recovery again after considering phase allocation. S102. After each stage of the matching process is completed, save the best cut set finally selected for each AND gate node in that stage to the pCutBestPhase[mode][polarity] field. S103. Aggregate the best cut sets of each AND gate node in the four stages to form a candidate action set. The entire joint action space with the gate node is , This represents the total number of AND gate nodes in the AND-NOT graph. Since the optimization objectives differ at each stage, the optimal cut set for the same node often differs at different stages; this difference provides the basis for the action space design in this embodiment.

[0046] S2. Extract the node feature vector for each node in the NAND graph, and construct the forward edge set and reverse edge set according to the NAND graph to obtain the graph state; the specific process includes: Extended export interface for the map command of the ABC process mapper -N <path>When calling the mapper to obtain the graph state, the node characteristics and connection relationships of both the non-graph nodes are output to a structured CSV file. <path>Each row corresponds to a node, and the fields include NodeID, Node1ID, Node2ID, TypePI, TypeInner, TypePO, Level, RLevel, Fanout, and Inv; where Node1ID and Node2ID are the identifiers of the two fan-in nodes of this node, and are set to -1 for the main input node.

[0047] The reinforcement learning agent then reads the CSV file and performs the following sub-steps: S201. Construct a 7-dimensional feature vector f(v)=[TypePI,TypeInner,TypePO,Level,RLevel,Fanout,Inv] for each node. Stack the feature vectors of all nodes to form a feature matrix. , where Level and RLevel are normalized to [0,1] across the entire graph; S202. Traverse each row and construct a forward edge set based on the parent-child relationship between NodeID and Node1ID and Node2ID. With reverse edge set , In the diagram, each edge points from the fan-in node to the current node. In the opposite direction; S203. Mark the set of non-main input nodes as the set of AND gate nodes, and construct an AND gate node mask, which is used in step S4 to extract the embedding rows corresponding to the AND gate nodes from the fused embedding.

[0048] S3. A bidirectional graph attention network is used to perform multi-layer message passing on the node feature vectors along both the forward and reverse edge sets. The two outputs are then concatenated and linearly fused to obtain the fused embedding of each node. The specific process of step S3 includes: like Figure 2 As shown, the bidirectional graph attention network encoder consists of two parallel paths. The GATConv layer is used in this embodiment. Hidden Dimensions Number of attention heads in each layer The forward propagation process of the bidirectional graph attention network encoder is as follows: S301, Order , for the layer implement:

[0049] through Forward hidden embedding is obtained after layer message passing. ; S302, Order , for the Layer (l=1,...,L) execution:

[0050] through Reverse hidden embedding is obtained after layer message passing. The network structures of the first and second paths are symmetrical but their parameters are independent. S303, will and After being stitched along the channel dimension, it is then processed through a linear blending layer. Mapped to fusion embedding:

[0051] The fused embedding simultaneously encodes the driving cone context of a node along the forward edge and the load cone context along the reverse edge. An attention mechanism is employed to enable the encoder to adaptively distinguish the information contributions of different neighbors on critical and non-critical paths.

[0052] S4. Using the embeddings corresponding to the gate nodes in the fused embedding as input, the Actor head simultaneously outputs the probability distribution of candidate actions for all gate nodes in one forward propagation and samples to obtain the global action vector. The Critic head outputs the state value estimate of each gate node. The specific process of step S4 includes: like Figure 2 As shown on the right, the decision-making head consists of a symmetrical Actor head and a Critic head, both of which are... A fully connected multilayer perceptron in this embodiment The decision-making process is as follows: S401, Based on the AND gate node mask obtained in step S203, from the fusion embedding Extract the embedding matrix corresponding to the gate node. ; S402, will Input the Actor header, obtain the logits of each gate node on the 4-dimensional candidate actions, and then use Softmax to obtain the action probability distribution:

[0053] The global action vector is obtained by sampling from each distribution. ; S403, will Input the Critic header to obtain scalar state value estimates for each AND gate node. .

[0054] In a forward propagation, the encoder completes message passing for the entire AND-NOT graph. The Actor head simultaneously outputs the action probability distribution for all AND gate nodes, and the Critic head simultaneously outputs the state value estimates for all AND gate nodes. The structural dependencies between different AND gate nodes are determined by the encoder. Skip neighborhood aggregation is implicitly modeled.

[0055] S5. Inject the global action vector back into the process mapper through the extended interface of the process mapper to complete the netlist generation and timing evaluation, and calculate the reward signal based on the delay improvement rate of the mapping result relative to the baseline mapping result. In one specific embodiment, step S5 includes the following specific process: Extended injection interface for the map command of the ABC process mapper - C <path>This is used to read the external selection file and replace the cut set of each AND gate node with the best cut set for the specified stage. The specific process of step S5 is as follows: S501. The agent writes the global action vector obtained in step S4 into the selection file line by line in the format of "node identifier, stage index". <path>; S502, via map-C <path>The mapper is invoked. Inside the mapper, each AND gate node is traversed according to the file content, and the cut set of the node is replaced with the best cut set recorded by pCutBestPhase[mode][polarity] for the corresponding stage. S503. Based on the replaced cut set, perform netlist generation and timing evaluation. The evaluation results are output as a report by the timing engine. The agent parses the report to obtain the delay of this mapping. With area , Figure 3 This is a scatter plot of the distribution of the cut set selection combination action space utilized in this invention in the area-delay two-dimensional plane; S504, Delay based on baseline mapping results Calculate the reward signal:

[0056] in The baseline latency is obtained from the native ABC four-stage mapping without introducing external selection. This reward directly reflects the relative latency improvement rate of this global action relative to the baseline.

[0057] Since the topology of NAND graphs remains unchanged throughout a training round, and the final mapping result is directly determined by the global action in each round, this invention models cut set selection as a single-step Markov decision process, without intermediate state transitions; advantage estimation is simplified to... .

[0058] Optionally, such as Figure 4 As shown, this is an area-delay scatter plot overlaying the results of each training stage of the present invention with different area constraints. Step S504 supports area constraint expansion, and the reward function is changed to:

[0059] in The area mapped to the baseline. For the area budget ratio, The penalty coefficient is used in this embodiment. , Typical values ​​for include 1.00, 1.02, 1.05, 1.08, 1.10, 1.15, 1.20, and +∞; when The reward is equivalent to the pure delay improvement rate, and any amount exceeding the budget will be penalized proportionally.

[0060] S6. The process sequentially executes four training phases: genetic algorithm expert search, imitation learning pre-training, proximal policy optimization exploration, and frozen encoder adaptive fine-tuning, to obtain the final policy network. Step S6 specifically involves independently executing four training phases for each design: S601, Genetic Algorithm Expert Search: Treat the global action vector as an individual, use the reward signal obtained in step S5 as the fitness, and perform [analysis] on the node-level action space. The population evolves, and individuals with positive rewards and their reward values ​​from the last generation are selected to form an expert sample set; the specific process of step S601 is as follows: S6011, Initialization size is A random population of individuals, each individual being a sequence of lengths... The global action vector, in this embodiment ; S6012, For each individual in the current population Perform the environmental assessment in step S5 to obtain its fitness value. ; S6013. Perform the following genetic operations sequentially to generate the next generation population: Tournament selection, selecting parents from the current population with a fixed tournament size; Uniform crossover, performing positional crossover of parent genes with a fixed probability; Random mutation, with a mutation rate... Each individual is mutated independently; in this embodiment... Elite preservation, retaining those with the highest contemporary adaptability. Individuals in this embodiment directly enter the next generation. ; S6014, Repeat S6012 and S6013 together Evolution then terminates, in this embodiment. ; S6015. Select individuals with positive rewards from the last generation of the population and their reward values ​​to form an expert sample set:

[0061] Genetic algorithms The secondary mapping evaluation locates the region of better solutions in the high-dimensional discrete space, providing an anchor point for subsequent imitation learning.

[0062] S602, Imitation Learning Pre-training: Using expert sample sets as supervision, the bidirectional graph attention network, Actor head, and Critic head are jointly pre-trained according to the weighted sum of behavior cloning loss and value function mean square error loss; the specific process of step S602 is as follows: by To supervise the signal, the bidirectional graph attention network encoder, Actor head, and Critic head are jointly pre-trained. Let... The joint pre-training loss function is:

[0063] in For expert sample set The number of samples, The total number of AND gate nodes, An index for the expert sample; For the index of the AND gate node; For the policy network in state Next, for the first Each AND gate node outputs expert actions. The probability of; is the natural logarithm of the probability; The loss weight coefficients are for the value function; Representing state Next The expected cumulative reward of each AND gate node; Indicates the first The reward signal corresponding to each expert sample; The behavior cloning loss is used to focus the strategy on the high-yield regions discovered by the genetic algorithm. The mean squared error loss of the value function provides a reasonable baseline for the advantage estimation A=RV in subsequent near-end policy optimization; The loss weights are value functions. In this embodiment, the number of pre-training epochs is 100, and the learning rate is... .

[0064] S603, Proximity Strategy Optimization Exploration: Trajectories are collected through interaction with the process mapper according to the proximity strategy optimization algorithm, and parameters of the bidirectional graph attention network, Actor head, and Critic head are updated; the specific process of step S603 is as follows: Each round of execution Sub-rollout sampling: For each rollout, obtain the global action and its log probability from step S402 according to the current strategy. With state value estimation Rewards are obtained through step S5. Let importance ratio be set. Advantage estimation The target loss for cropping is:

[0065] The total loss is:

[0066] Each batch of data is reused The algorithm updates the algorithm in batches after shuffling, and prunes the gradients. During training, it continuously tracks the historical best reward and saves the corresponding policy parameters in real time. In this embodiment, the learning rate is... , , , , , =16, total number of episodes is 1000.

[0067] S604, Freeze Encoder Adaptive Fine-Tuning: Freeze the parameters of the bidirectional graph attention network, and continue training only on the Actor head and Critic head using the proximal policy optimization algorithm to obtain the final policy network; where the proximal policy optimization algorithm models cut set selection as a single-step Markov decision process, and the advantage estimation is simplified to A=RV. The specific process of step S604 is as follows: All parameters of the bidirectional graph attention network encoder are frozen, and training continues only on the Actor and Critic heads using the proximal policy optimization algorithm. Since the parameters of the bidirectional graph attention network encoder account for more than 80% of the trainable parameters of the entire policy network, the number of parameters updated after freezing is significantly reduced, thus significantly lowering the risk of overfitting. The encoder has already learned general graph representations such as critical paths and fan-out patterns in the preceding stages, so fine-tuning only the Actor / Critic heads is sufficient to quickly adapt the policy to the current design. In this embodiment, the number of fine-tuning episodes is 50.

[0068] The method of this invention models the cutset selection of each AND gate node in the AND-NOT graph as a reinforcement learning problem, directly using the mapped real service quality as the reward signal; it simultaneously captures the driving cone and load cone context of the node through a bidirectional graph attention network; and it uses Actor... The Critic head makes joint decisions on all AND gate nodes in a single forward propagation, implicitly modeling the structural dependencies between nodes. The progressive training paradigm of "genetic algorithm → imitation learning → proximal policy optimization → frozen encoder fine-tuning" effectively overcomes the cold start problem in a large-scale combined action space.

[0069] This invention performs a complete progressive training pipeline on 16 designs of the EPFL combinational logic benchmark set. Experimental results show that this invention achieves approximately 13% latency reduction on the geometric mean of the 16 designs compared to the ABC baseline, with latency improvements achieved in 14 of the designs; on the same ASAP7 technology library and the same set of benchmarks, this invention outperforms the representative supervised learning method on 11 designs.

[0070] This invention optimizes the area budget ratio in the design of int2float. Each component independently executed the complete training pipeline. Experimental results are as follows: Figure 5 As shown, the horizontal axis represents the area budget ratio, and the vertical axis represents the delay improvement rate. In the int2float design, different area budget ratios... The delay improvement rate obtained by this invention exhibits an "area wall" characteristic: a delay improvement of 3.1% is achieved when α=1.00; it jumps to 11.4% when α=1.02; and the delay improvement tends to saturate to about 13% after α≥1.08. Under all constraint modes, the actual area increase is far lower than the budget limit, verifying that this invention generates delay under a given area budget. The ability to achieve area trade-offs.

[0071] This invention compares the reinforcement learning scheme (RL-Only (RL+FT)) that only executes S603 and S604 with the full progressive training scheme (Full Pipeline (GA+IL+RL+FT)) that completes S601 to S604, based on the int2float design. Experimental results are as follows: Figure 6 As shown, the horizontal axis represents the number of training rounds, and the vertical axis represents the best reward percentage. Figure 6 The results show that the initial reward of the complete pipeline is significantly higher than that of the reinforcement learning-only scheme, the convergence speed is faster, and the average reward is maintained throughout the training process.

[0072] This invention formalizes the cut set selection in process mapping into a graph-based combinatorial optimization problem, and solves it directly through reinforcement learning. It uses the real service quality obtained by the time-series engine after mapping as the reward signal, avoiding dependence on pre-labeled data and breaking through the bottleneck of supervised learning paradigms being limited by the upper bound of heuristic label quality.

[0073] The bidirectional graph attention network encoder proposed in this invention simultaneously performs message passing and fusion along the forward and reverse edges of the AIG, and the fusion embedding includes context information of both the driving cone and the load cone; the Actor head outputs the action probability distribution to all AND gate nodes in one forward propagation, and the structural dependencies between different nodes are implicitly modeled through the multi-hop message passing of the bidirectional graph attention network, overcoming the problem that node-by-node independent decision-making cannot form a globally optimal mapping.

[0074] The progressive training paradigm proposed in this invention, consisting of "genetic algorithm search → imitation learning pre-training → reinforcement learning exploration → frozen encoder fine-tuning," utilizes a genetic algorithm to locate a relatively optimal solution region under zero prior conditions and distills it into the policy network as a good initialization, effectively overcoming the limitations of reinforcement learning in O( The problem of low cold-start efficiency in the combined action space of the order of magnitude; the frozen encoder fine-tuning stage further enables the strategy to quickly adapt to specific designs while retaining the learned graph representation.

[0075] Experiments on the EPFL combinational logic reference set and the ASAP7 7nm FinFET standard cell library show that the present invention achieves a latency reduction of approximately 13% on the geometric mean of 16 designs and a latency improvement in 14 designs compared to the ABC baseline; in area-constrained extended scenarios, more than 11% latency improvement can be achieved with only about 2% of the area budget.

[0076] The method of this invention, for each AND gate node in the AND-NOT graph, invokes the multi-stage mapping of the process mapper to collect the best cut sets of each stage as candidate actions, constructing a node-level joint action space; node feature vectors are constructed using node type, forward / backward hierarchy, fan-out, and inverse attributes, while simultaneously preserving the forward and backward edge sets; a bidirectional graph attention network is used to perform multi-layer message passing along the forward and backward edges and linearly fuse them to obtain node embeddings that simultaneously contain the context of the driving cone and the load cone; Actor In a single forward propagation, the Critic head jointly outputs the action probability distribution and state value estimate for all AND gate nodes. It completes state acquisition and mapping evaluation through the extended export and back-injection interfaces of the process mapper, using the latency improvement rate relative to the baseline as the reward. The policy network is obtained through a progressive training process involving genetic algorithm expert search, imitation learning pre-training, proximal policy optimization exploration, and frozen encoder fine-tuning. This method directly uses the real service quality after mapping as the optimization objective, requires no pre-labeled data, and can significantly reduce process mapping latency while supporting latency under area constraints. Area compromise optimization.

[0077] Those skilled in the art will understand that the following modifications can be made to the above-described specific embodiments without departing from the concept and scope of the present invention: such as the number of stages. Not limited to 4, the value can be flexibly selected according to the specific multi-stage mapping process of the process mapper used; the GATConv layer in the graph attention network encoder can be replaced with other graph neural network layers such as graph convolutional networks and graph isomorphic networks; the near-end policy optimization algorithm can be replaced with other Actors The Critic-type policy gradient algorithm; the progressive training pipeline can be further extended to a training mode that combines cross-design pre-training and design-by-design fine-tuning; the extended interface scheme can be adapted to other process mappers with multi-stage mapping processes; the reward function can be further extended to a multi-objective form that simultaneously considers power consumption, line load and other service quality indicators.

[0078] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.< / path> < / path> < / path> < / path> < / path>

Claims

1. A process mapping cutset optimization method based on reinforcement learning and bidirectional graph attention, characterized in that, Including the following steps: S1. For the NAND graph, call the process mapper to perform multi-stage mapping, taking the best cut set selected for each AND gate node in each stage as a candidate action for the AND gate node, and constructing a process map for each AND gate node. The node-level action space consists of candidate cut sets; S2. Extract the node feature vector of each AND gate node in the NAND graph, and construct the forward edge set and the reverse edge set according to the NAND graph to obtain the graph state; S3. A bidirectional graph attention network is used to perform multi-layer message passing on the node feature vectors along the forward edge set and the reverse edge set respectively to obtain the forward hidden embedding and the reverse hidden embedding. The forward hidden embedding and the reverse hidden embedding are concatenated and then linearly fused to obtain the fused embedding of each AND gate node. S4. The Actor-Critic architecture in reinforcement learning is adopted. The embeddings corresponding to the gate nodes in the fusion embedding are used as inputs. The Actor head samples the probability distribution of all gate node outputs in the node-level action space in one forward propagation to obtain the global action vector. At the same time, the Critic head outputs the state value estimate of each gate node. S5. Inject the global action vector back into the process mapper through the extended interface of the process mapper to complete the netlist generation and timing evaluation, and calculate the reward signal based on the delay improvement rate of the mapping result relative to the baseline mapping result. S6. Using the graph state as the environment state, the global action vector as the policy output, and the reward signal as the fitness, the four training stages of genetic algorithm expert search, imitation learning pre-training, proximal policy optimization exploration, and frozen encoder adaptive fine-tuning are executed in sequence to update the policy network parameters and obtain the final policy network for process mapping cut set optimization.

2. The process mapping cutset optimization method based on reinforcement learning and bidirectional graph attention according to claim 1, characterized in that, The specific process of step S1 is as follows: S101, For NAND graphs, call the process mapper to execute sequentially. Each of the mapping stages includes cut set enumeration, Boolean matching, and dynamic programming coverage, and each stage has its own independent optimization objective. S102. After each stage of matching is completed, the best cut set finally selected for each AND gate node in that stage is saved to the cut set record field of that node in that stage. S103, aggregate the best cut sets of each AND node in each stage to form the candidate action set of the node The joint action space of all AND nodes is where is the total number of AND nodes in the NOR graph in The four mapping stages correspond to stages 0, 1, 2 and 3 of the ABC process mapper's map command. Stage 0 aims for the lowest latency, stage 1 reselects based on area flow under latency constraints, stage 2 greedily optimizes in a precise area manner under latency constraints, and stage 3 performs area recovery again after considering phase allocation.

3. The process mapping cutset optimization method based on reinforcement learning and bidirectional graph attention according to claim 1, characterized in that, The specific process of step S2 is as follows: S201. Extract a 7-dimensional feature vector [TypePI, TypeInner, TypePO, Level, RLevel, Fanout, Inv] for each node in the AND-NOT graph, where TypePI, TypeInner, and TypePO are the one-hot encodings of the node type, Level and RLevel are the normalized forward and reverse levels, respectively, Fanout is the fan-out number after being transformed by the natural logarithm plus one, and Inv is the binary indicator of the inverse attribute. S202. Construct a positive edge set based on the connection relationships between nodes in and outside the graph. With reverse edge set , In the diagram, each edge points from the fan-in node to the current node. Each edge in the middle is in the opposite direction; S203. Based on the node type, mark the set of AND gate nodes to obtain the AND gate node mask, which is used to extract the embedding row corresponding to the AND gate node from the fused embedding.

4. The process mapping cutset optimization method based on reinforcement learning and bidirectional graph attention according to claim 1 or 2 or 3, characterized in that, The specific process of step S3 is as follows: S301, Taking the node feature vector as input, and then... The layered graph attention network uses convolutional layers to pass messages along the forward edge set, with each layer using... H Each attention head uses ELU as the activation function, and the output is the positive hidden embedding. ; S302, Using the same node feature vector as input, and passing through another set of... The layered graph attention network performs message passing along the inverse edge set in its convolutional layers. The network structure is symmetrical with the first path, and the output is the inverse hidden embedding. ; S303, will and After being stitched along the channel dimension, it is mapped by a linear fusion layer to... Dimensional Fusion Embedding ;in The weight matrix is ​​a linear fusion matrix; The total number of all nodes; To fuse the embedded feature dimensions; in , , The fused embedding simultaneously encodes the driving cone context of the node along the forward edge and the load cone context along the reverse edge.

5. The process mapping cutset optimization method based on reinforcement learning and bidirectional graph attention of claim 3, wherein, The specific process of step S4 is as follows: S401、According to the AND gate node mask, the embedding matrix corresponding to the AND gate node is taken out from the fusion embedding ; S402, will Input by The Actor head, composed of a fully connected multilayer perceptron and Softmax, outputs each AND gate node. Probability distribution of candidate actions And sample the global action vector from each distribution. ; S403, will The input structure is symmetrical The Critic head, composed of a fully connected multilayer sensing mechanism, outputs a scalar state value estimate for each AND gate node. .

6. The process mapping cutset optimization method based on reinforcement learning and bidirectional graph attention according to claim 1 or 2 or 3, characterized in that, The specific process of step S5 is as follows: S501. Write the global action vector line by line into the external selection file in the format of "node identifier, stage index". S502: Read the selection file through the injection interface of the process mapper and replace the cut set of each AND gate node with the best cut set of that node in the corresponding stage. S503, performing netlist generation and timing evaluation based on the replaced cut set, and obtaining the delay of this mapping from an evaluation report with area ; S504、delay according to the baseline mapping result Computing the delay improvement rate and as a reward signal .

7. The process mapping cutset optimization method based on reinforcement learning and bidirectional graph attention according to claim 6, characterized in that, In step S504, the reward signal is: or in The area mapped to the baseline. For the area budget ratio, This is the area penalty coefficient; when The reward is equivalent to the pure delay improvement rate; when At that time, any amount exceeding the budget will be penalized proportionally; through adjustments... The value can be optimized for delay under different area budgets.

8. The process mapping cutset optimization method based on reinforcement learning and bidirectional graph attention according to claim 1 or 2 or 3, characterized in that, The specific process of step S6 is as follows: S601, genetic algorithm expert search: take the global action vector as an individual, take the reward signal obtained in step S5 as fitness, and perform population evolution on the node-level action space The last population evolution is screened, and individuals with positive rewards and their reward values in the last population are selected to form an expert sample set. S602, Imitation Learning Pre-training: Using the expert sample set as supervision, the bidirectional graph attention network, Actor head and Critic head are jointly pre-trained according to the weighted sum of behavior cloning loss and value function mean square error loss; S603, Proximity Strategy Optimization Exploration: The trajectory is collected through interaction with the process mapper according to the proximity strategy optimization algorithm, and the parameters of the bidirectional graph attention network, Actor head and Critic head are updated. S604, Freeze Encoder Adaptive Fine-Tuning: Freeze the parameters of the bidirectional graph attention network, and continue to train only the Actor head and Critic head according to the proximal policy optimization algorithm to obtain the final policy network.

9. The process mapping cutset optimization method based on reinforcement learning and bidirectional graph attention of claim 8, wherein, The specific process of step S601 is as follows: S6011、initialize a random individual population with a size of , each individual being a global action vector; S6012. Perform an environmental assessment on each individual in the current population to obtain the fitness value of that individual; S6013, sequentially perform tournament selection, uniform crossover, and sort by mutation rate. random variation and proportional variation The elite retention operation produces the next generation of the population; S6014、circulating execution S6012 and S6013 evolution is terminated after the next S6015. Select individuals with positive rewards from the last generation of the population and their reward values ​​to form an expert sample set. , as a monitoring signal.

10. The process mapping cutset optimization method based on reinforcement learning and bidirectional graph attention of claim 9, wherein, The loss function for the joint pre-training in step S602 is: in For expert sample set The number of samples, The total number of AND gate nodes, An index for the expert sample; For the index of the AND gate node; For the policy network in state Next, for the first Each AND gate node outputs an expert action. The probability of; is the natural logarithm of the probability; The loss weight coefficients are for the value function; Representing state Next The expected cumulative reward of each AND gate node; Indicates the first The reward signal corresponding to each expert sample.

Images