Deep reinforcement learning manhattan and non-manhattan structure steiner minimum tree construction method
Patent Information
- Application Number
- CN202311627063.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-30
- Publication Date
- 2026-09-22
- Estimated Expiration
- 2043-11-30
AI Technical Summary
[0035]本发明提出了一种基于深度强化学习(Deep Reinforcement Learning, DRL)的方法框架用于解决曼哈顿结构下的直线Steiner最小树(Rectilinear Steiner minimumTree, RSMT)与非曼哈顿结构下的八角Steiner最小树(Octilinear Steiner MinimumTree，OSMT)构造问题。首先，为解决此构造问题提出了一个深度学习模型,以Steiner最小树(Steiner Minimum Tree, SMT)的线长作为奖励通过DRL对模型进行训练。其次，根据RSMT与OSMT的结构特征设计了边点序列(Edge Point Sequence, EPS)用于弥合深度学习模型的输出以表示Steiner最小树的结构。最后，为构造方案的质量评估提供了相应的快速精确线长计算算法，以加速模型的训练，本发明能够在超大规模集成电路的物理设计中得到一个最小化线长的布线解方案。
Smart Images
Figure CN117610487B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer-aided design technology for integrated circuits, and in particular to a deep reinforcement learning method for constructing a Steiner minimum tree with Manhattan and non-Manhattan structures, namely, a deep reinforcement learning approach for constructing a Steiner minimum tree with Manhattan and non-Manhattan structures. Background Technology
[0002] The construction of Steiner Minimum Tree (SMT) is a fundamental problem in the physical design of Very Large Scale Integration (VLSI) and is widely used in the early stages of VLSI design. SMT can be divided into Manhattan and non-Manhattan types according to the routing direction of the wiring tree. The linear Steiner Minimum Tree (RSMT) under the Manhattan structure is the shortest method of connecting a given pin by introducing Steiner points and using vertical and horizontal edges. The construction of RSMT has been proven to be an NP-complete problem [1]. Octal Steiner Minimum Tree (OSMT) is a representative of the non-Manhattan structure. Compared with RSMT, it has 45° edges and 135° edges, which can make fuller use of wiring resources, but also has greater problem complexity. With the development of Machine Learning (ML), it has also shown great potential to generate high-quality solutions for many NP-complete problems. Deep Reinforcement Learning (DRL) combines the perceptual capabilities of deep learning with the decision-making capabilities of reinforcement learning (RL), often exhibiting more advantages compared to traditional methods. Summary of the Invention
[0003] This invention proposes a deep reinforcement learning method for constructing a minimum tree for Manhattan and non-Manhattan structures using Steiner, which can obtain a wiring solution that minimizes line length in the physical design of very large-scale integrated circuits.
[0004] The present invention adopts the following technical solution.
[0005] A deep reinforcement learning method for constructing Manhattan and non-Manhattan Steiner minimum trees is proposed to solve the problem of constructing linear Steiner minimum trees (RSMT) for Manhattan structures and octagonal Steiner minimum trees (OSMT) for non-Manhattan structures in VLSI physical design. The method includes the following steps.
[0006] Step A1: Define the routing methods of RSMT and OSMT; propose a deep learning model, and train the model using DRL with the line length of the Steiner minimum tree SMT as the reward.
[0007] Step A2: Design an edge sequence EPS that can be used to encode both RSMT and OSMT structures to bridge the output of the deep learning model;
[0008] Step A3: Design a neural network model; use an attention mechanism to select the wiring method between the two pins in the pair;
[0009] Step A4: Design a fast and accurate line length calculation algorithm for RSMT and OSMT;
[0010] Step A5: Train the neural network model using a deep reinforcement learning algorithm.
[0011] In step A1, the input to the RSMT and OSMT construction problem is considered as a set of pins. pins From coordinate pairs Composition; Connection points introduced during the wiring process, other than pins, are called pseudo-Steiner points; such as Figure 1 As shown, white squares are used to represent them.
[0012] In step A1, given two pins a and b within the pin set, the various wiring methods available between the pins are as follows:
[0013] Wiring method 0: First, run a vertical line from pin a to point s, then run a horizontal line from point s to pin b, such as... Figure 1 As shown in (a);
[0014] Wiring method 1: First, run a horizontal line from pin a to point s, then run a vertical line from point s to pin b, as follows: Figure 1 As shown in (b);
[0015] Wiring method 2: First, run a vertical or horizontal edge from pin a to point s, then run a 45° or 135° edge from point s to pin b, such as... Figure 1 As shown in (c);
[0016] Wiring method 3: First, draw a 45° or 135° edge from pin a to point s, and then draw a vertical or horizontal edge from point s to pin b.
[0017] In step A1, the goal of RSMT construction is to build a straight tree connecting all pins in P and using only routing mode 0 or routing mode 1, with the shortest possible wire length. ,Right now yes superset Newly introduced The points in the diagram are Steiner points; by introducing Steiner points, overlapping edges are automatically merged, thus reducing line length; pseudo-Steiner points only exist during the routing optimization process and do not exist in the constructed RSMT.
[0018] In step A2, the edge point sequence is as follows:
[0019] For a given set of pins , An EPS has A pair ,
[0020] in, , That is, in the first In the pairing, the first The pin and the first Wiring is used between the pins. Interconnection; in RSMT problems In the OSMT problem .
[0021] For example, given a pin set The positions of each pin are as follows Figure 2 As shown in (a), a feasible RSMT solution for P is: ,like Figure 2 (b) shows the optimal RSMT. A feasible solution for OSMT is... , is the best OSMT.
[0022] In step A3, the neural network model uses an encoder-decoder architecture; the encoder used mimics the Transformer structure, mapping the 2D coordinates of the pins to a 128-dimensional feature space through a linear layer, thereby mapping the pins from low-dimensional coordinates to high-dimensional vectors, and using batch normalization to make training more stable.
[0023] The decoder encodes the result. and As input, the output is the SMT construction scheme EPS; the neural network first starts from... Choose one And Marking a pin as accessed means that the next pin from... When an accessed pin is selected, the pin can be selected; when an unaccessed pin is selected, the pin can no longer be selected.
[0024] In the next In the process of selecting the first pair, the first pair... The selection rule for each pair is as follows: The neural network will first select an unvisited pin each time. Then from Choose one ,in ,mark Pins that have already been accessed;
[0025] The pairs are added to the EPS in sequence; the neural network in the decoder uses an attention mechanism to select the two pins in the pair and the wiring method between the pins.
[0026] In step A3, the encoder of the neural network model uses a three-layer Transformer block, in which the multi-head attention mechanism uses 16 heads, the hidden layer of the feedforward neural network uses 512 dimensions, and the layer normalization is replaced with batch normalization.
[0027] In the RSMT and OSMT construction problems, a linear layer is used in the RSMT problem to learn the feature vectors of unaccessed pins. After learning the characteristics of the accessed pins under routing mode 0 and routing mode 1 through two linear mappings, the features are then concatenated to form... During OSMT construction, the characteristics of accessed pins under four routing methods are learned through four linear mappings and then spliced together. The overall structure of the encoder is as follows: Figure 3 As shown in (a).
[0028] The fast and accurate line length calculation method used in step A4 is as follows:
[0029] The input to the RSMT cable length calculation method is the pin set. as well as An RSMT construction scheme eps, from which horizontal edges are extracted. With vertical side The extraction formats for horizontal and vertical edges are [ordinate, starting x-coordinate, ending x-coordinate] and [x-coordinate, starting ordinate, ending ordinate], respectively; where the starting point of the horizontal edge's x-coordinate is on the left and the ending point is on the right, and the starting point of the vertical edge's ordinate is at the bottom and the ending point is at the top; respectively for... and Sort the data in ascending order, starting from the first point and ending at the last point. Then, you can use the sorted data as a guide. and Merge overlapping portions and calculate the bus length accordingly;
[0030] The input to the fast and accurate wire length calculation method for OSMT is the pin set. as well as An OSMT construction scheme ,according to Extract horizontal edges vertical side 45° edge and 135° edge The extraction methods for horizontal and vertical edges are the same as the fast and accurate calculation method for RSMT. and The extraction format is [starting x-coordinate, starting y-coordinate, ending x-coordinate, ending y-coordinate]; and Rotate counterclockwise by 45° and adjust to have horizontal and vertical sides respectively; , , and Sort the data from smallest to largest according to the order of starting point and ending point, and then merge overlapping parts to calculate the bus length.
[0031] The neural network model consists of two components: Actor and Critic;
[0032] The role of the Actor is to generate EPS for a given set of pins through a neural network. That is, the Actor's action is to select a pair according to the probability generated by the model at each time step, and the state at the next time step is the SMT subtree determined by the pair in the current EPS, with the SMT line length as the reward.
[0033] The role of the Critic is to predict the line length of the SMT constructed by the Actor, and use this as a baseline to help the Actor learn and improve its performance.
[0034] In step A5, SMT is constructed using the DRL framework. Figure 3 (b) illustrates the DRL framework for constructing SMT; the training method used in this step is as follows: the Actor and Critic parameters are trained using a policy gradient-based Actor-Critic algorithm, with the goal of improving the probability of the neural network generating excellent EPS; the Critic uses the same encoder structure as the Actor, but does not use ProjectionU and ProjectionV; using pin sets As input, the encoder generates the encoded result. The attention mechanism is used to calculate the weight of each vectorized pin distribution in the overall encoding result; then, the baseline is calculated through two fully connected layers and the ReLU activation function.
[0035] This invention proposes a Deep Reinforcement Learning (DRL) framework to solve the construction problems of Rectilinear Steiner Minimum Trees (RSMTs) under Manhattan structures and Octilinear Steiner Minimum Trees (OSMTs) under non-Manhattan structures. First, a deep learning model is proposed to solve this construction problem, using the line length of the Steiner Minimum Tree (SMT) as a reward and training the model through DRL. Second, based on the structural characteristics of RSMTs and OSMTs, an Edge Point Sequence (EPS) is designed to bridge the output of the deep learning model to represent the structure of the Steiner Minimum Tree. Finally, a fast and accurate line length calculation algorithm is provided for the quality evaluation of the construction scheme to accelerate model training. This invention can obtain a wiring solution with minimized line length in the physical design of VLSI. Attached Figure Description
[0036] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:
[0037] Appendix Figure 1 This is a schematic diagram of the wiring method of the present invention;
[0038] Appendix Figure 2 This is a schematic diagram illustrating the edge point sequence of the present invention;
[0039] Appendix Figure 3 This is a schematic diagram of the deep reinforcement learning framework in this invention. Detailed Implementation
[0040] As shown in the figure, a deep reinforcement learning method for constructing Manhattan and non-Manhattan Steiner minimum trees is used to solve the problem of constructing linear Steiner minimum trees (RSMT) for Manhattan structures and octagonal Steiner minimum trees (OSMT) for non-Manhattan structures in the physical design of VLSI. The method includes the following steps.
[0041] Step A1: Define the routing methods of RSMT and OSMT; propose a deep learning model, and train the model using DRL with the line length of the Steiner minimum tree SMT as the reward.
[0042] Step A2: Design an edge sequence EPS that can be used to encode both RSMT and OSMT structures to bridge the output of the deep learning model;
[0043] Step A3: Design a neural network model; use an attention mechanism to select the wiring method between the two pins in the pair;
[0044] Step A4: Design a fast and accurate line length calculation algorithm for RSMT and OSMT;
[0045] Step A5: Train the neural network model using a deep reinforcement learning algorithm.
[0046] In step A1, the input to the RSMT and OSMT construction problem is considered as a set of pins. pins From coordinate pairs Composition; Connection points introduced during the wiring process, other than pins, are called pseudo-Steiner points; such as Figure 1 As shown, white squares are used to represent them.
[0047] In step A1, given two pins a and b within the pin set, the various wiring methods available between the pins are as follows:
[0048] Wiring method 0: First, run a vertical line from pin a to point s, then run a horizontal line from point s to pin b, such as... Figure 1 As shown in (a);
[0049] Wiring method 1: First, run a horizontal line from pin a to point s, then run a vertical line from point s to pin b, as follows: Figure 1 As shown in (b);
[0050] Wiring method 2: First, run a vertical or horizontal edge from pin a to point s, then run a 45° or 135° edge from point s to pin b, such as... Figure 1 As shown in (c);
[0051] Wiring method 3: First, draw a 45° or 135° edge from pin a to point s, and then draw a vertical or horizontal edge from point s to pin b.
[0052] In step A1, the goal of RSMT construction is to build a straight tree connecting all pins in P and using only routing mode 0 or routing mode 1, with the shortest possible wire length. ,Right now yes superset Newly introduced The points in the diagram are Steiner points; by introducing Steiner points, overlapping edges are automatically merged, thus reducing line length; pseudo-Steiner points only exist during the routing optimization process and do not exist in the constructed RSMT.
[0053] In step A2, the edge point sequence is as follows:
[0054] For a given set of pins , An EPS has A pair ,
[0055] in, , That is, in the first In the pairing, the first The pin and the first Wiring is used between the pins. Interconnection; in RSMT problems In the OSMT problem .
[0056] For example, given a pin set The positions of each pin are as follows Figure 2 As shown in (a), a feasible RSMT solution for P is: ,like Figure 2 (b) shows the optimal RSMT. A feasible solution for OSMT is... , is the best OSMT.
[0057] In step A3, the neural network model uses an encoder-decoder architecture; the encoder used mimics the Transformer structure, mapping the 2D coordinates of the pins to a 128-dimensional feature space through a linear layer, thereby mapping the pins from low-dimensional coordinates to high-dimensional vectors, and using batch normalization to make training more stable.
[0058] The decoder encodes the result. and As input, the output is the SMT construction scheme EPS; the neural network first starts from... Choose one And Marking a pin as accessed means that the next pin from... When an accessed pin is selected, the pin can be selected; when an unaccessed pin is selected, the pin can no longer be selected.
[0059] In the next In the process of selecting the first pair, the first pair... The selection rule for each pair is as follows: The neural network will first select an unvisited pin each time. Then from Choose one ,in ,mark Pins that have already been accessed;
[0060] The pairs are added to the EPS in sequence; the neural network in the decoder uses an attention mechanism to select the two pins in the pair and the wiring method between the pins.
[0061] In step A3, the encoder of the neural network model uses a three-layer Transformer block, in which the multi-head attention mechanism uses 16 heads, the hidden layer of the feedforward neural network uses 512 dimensions, and the layer normalization is replaced with batch normalization.
[0062] In the RSMT and OSMT construction problems, a linear layer is used in the RSMT problem to learn the feature vectors of unaccessed pins. After learning the characteristics of the accessed pins under routing mode 0 and routing mode 1 through two linear mappings, the features are then concatenated to form... During OSMT construction, the characteristics of accessed pins under four routing methods are learned through four linear mappings and then spliced together. The overall structure of the encoder is as follows: Figure 3 As shown in (a).
[0063] The fast and accurate line length calculation method used in step A4 is as follows:
[0064] The input to the RSMT cable length calculation method is the pin set. as well as An RSMT construction scheme eps, from which horizontal edges are extracted. With vertical side The extraction formats for horizontal and vertical edges are [ordinate, starting x-coordinate, ending x-coordinate] and [x-coordinate, starting ordinate, ending ordinate], respectively; where the starting point of the horizontal edge's x-coordinate is on the left and the ending point is on the right, and the starting point of the vertical edge's ordinate is at the bottom and the ending point is at the top; respectively for... and Sort the data in ascending order, starting from the first point and ending at the last point. Then, you can use the sorted data as a guide. and Merge overlapping portions and calculate the bus length accordingly;
[0065] The input to the fast and accurate wire length calculation method for OSMT is the pin set. as well as An OSMT construction scheme ,according to Extract horizontal edges vertical side 45° edge and 135° edge The extraction methods for horizontal and vertical edges are the same as the fast and accurate calculation method for RSMT. and The extraction format is [starting x-coordinate, starting y-coordinate, ending x-coordinate, ending y-coordinate]; and Rotate counterclockwise by 45° and adjust to have horizontal and vertical sides respectively; , , and Sort the data from smallest to largest according to the order of starting point and ending point, and then merge overlapping parts to calculate the bus length.
[0066] The neural network model consists of two components: Actor and Critic;
[0067] The role of the Actor is to generate EPS for a given set of pins through a neural network. That is, the Actor's action is to select a pair according to the probability generated by the model at each time step, and the state at the next time step is the SMT subtree determined by the pair in the current EPS, with the SMT line length as the reward.
[0068] The role of the Critic is to predict the line length of the SMT constructed by the Actor, and use this as a baseline to help the Actor learn and improve its performance.
[0069] In step A5, SMT is constructed using the DRL framework. Figure 3 (b) illustrates the DRL framework for constructing SMT; the training method used in this step is as follows: the Actor and Critic parameters are trained using a policy gradient-based Actor-Critic algorithm, with the goal of improving the probability of the neural network generating excellent EPS; the Critic uses the same encoder structure as the Actor, but does not use ProjectionU and ProjectionV; using pin sets As input, the encoder generates the encoded result. The attention mechanism is used to calculate the weight of each vectorized pin distribution in the overall encoding result; then, the baseline is calculated through two fully connected layers and the ReLU activation function.
Claims
1. A deep reinforcement learning method for constructing Manhattan and non-Manhattan Steiner minimum trees, used to solve the problem of constructing linear Steiner minimum trees (RSMT) for Manhattan structures and octagonal Steiner minimum trees (OSMT) for non-Manhattan structures in integrated circuit physical design. Its features include: Includes the following steps; Step A1: Define the routing methods of RSMT and OSMT; propose a deep learning model, and train the model using DRL with the line length of the Steiner minimum tree SMT as the reward. Step A2: Design an edge sequence EPS that can be used to encode both RSMT and OSMT structures to bridge the output of the deep learning model; Step A3: Design a neural network model; use an attention mechanism to select the wiring method between the two pins in the pair; Step A4: Design a fast and accurate line length calculation algorithm for RSMT and OSMT; Step A5: Train the neural network model using a deep reinforcement learning algorithm; The fast and accurate line length calculation method used in step A4 is as follows: The input to the RSMT cable length calculation method is the pin set. as well as An RSMT construction scheme eps, from which horizontal edges are extracted. With vertical side The extraction formats for horizontal and vertical edges are [ordinate, starting x-coordinate, ending x-coordinate] and [x-coordinate, starting ordinate, ending ordinate], respectively; where the starting point of the horizontal edge's x-coordinate is on the left and the ending point is on the right, and the starting point of the vertical edge's ordinate is at the bottom and the ending point is at the top; respectively for... and Sort the data in ascending order, starting from the first point and ending at the last point. Then, you can use the sorted data as a guide. and Merge overlapping portions and calculate the bus length accordingly; The input to the fast and accurate wire length calculation method for OSMT is the pin set. as well as An OSMT construction scheme ,according to Extract horizontal edges vertical side 45° edge and 135° edge The extraction methods for horizontal and vertical edges are the same as the fast and accurate calculation method for RSMT. and The extraction format is [starting x-coordinate, starting y-coordinate, ending x-coordinate, ending y-coordinate]; and Rotate counterclockwise by 45° and adjust to have horizontal and vertical sides respectively; , , and Sort the data from smallest to largest according to the order of starting point and ending point, and then merge overlapping parts to calculate the bus length.
2. The method for constructing Steiner minimum trees for Manhattan and non-Manhattan structures in deep reinforcement learning according to claim 1, characterized in that: In step A1, the input to the RSMT and OSMT construction problem is considered as a set of pins. pins From coordinate pairs Composition; Connection points introduced during the wiring process, other than pins, are called pseudo-Steiner points.
3. The method for constructing Steiner minimum trees for Manhattan and non-Manhattan structures in deep reinforcement learning according to claim 2, characterized in that: In step A1, given two pins a and b within the pin set, the various wiring methods available between the pins are as follows: Wiring method 0: First, draw a vertical edge from pin a to point s, and then draw a horizontal edge from point s to pin b; Wiring method 1: First, run the horizontal edge from pin a to point s, and then run the vertical edge from point s to pin b; Wiring method 2: First, draw a vertical or horizontal edge from pin a to point s, and then draw a 45° or 135° edge from point s to pin b; Wiring method 3: First, draw a 45° or 135° edge from pin a to point s, and then draw a vertical or horizontal edge from point s to pin b.
4. The method for constructing Steiner minimum trees for Manhattan and non-Manhattan structures in deep reinforcement learning according to claim 3, characterized in that: In step A1, the goal of RSMT construction is to build a straight tree connecting all pins in P and using only routing mode 0 or routing mode 1, with the shortest possible wire length. ,Right now yes superset Newly introduced The points in the diagram are Steiner points; by introducing Steiner points, overlapping edges are automatically merged, thus reducing line length; pseudo-Steiner points only exist during the routing optimization process and do not exist in the constructed RSMT.
5. The method for constructing Steiner minimum trees for Manhattan and non-Manhattan structures in deep reinforcement learning according to claim 2, characterized in that: In step A2, the edge point sequence is as follows: For a given set of pins , An EPS has A pair , in, , That is, in the first In the pairing, the first The pin and the first Wiring is used between the pins. Interconnection; in RSMT problems In the OSMT problem .
6. The method for constructing Steiner minimum trees for Manhattan and non-Manhattan structures in deep reinforcement learning according to claim 5, characterized in that: In step A3, the neural network model uses an encoder-decoder architecture; the encoder used mimics the Transformer structure, mapping the 2D coordinates of the pins to a 128-dimensional feature space through a linear layer, thereby mapping the pins from low-dimensional coordinates to high-dimensional vectors, and using batch normalization to make training more stable. The decoder encodes the result. and As input, the output is the SMT construction scheme EPS; the neural network first starts from... Choose one And Marking a pin as accessed means that the next pin from... When an accessed pin is selected, the pin can be selected; when an unaccessed pin is selected, the pin can no longer be selected. In the next In the process of selecting the first pair, the first pair... The selection rule for each pair is as follows: The neural network will first select an unvisited pin each time. Then from Choose one ,in ,mark Pins that have already been accessed; The pairs are added to the EPS in sequence; the neural network in the decoder uses an attention mechanism to select the two pins in the pair and the wiring method between the pins.
7. The method for constructing Steiner minimum trees for Manhattan and non-Manhattan structures in deep reinforcement learning according to claim 5, characterized in that: In step A3, the encoder of the neural network model uses a three-layer Transformer block, in which the multi-head attention mechanism uses 16 heads, the hidden layer of the feedforward neural network uses 512 dimensions, and the layer normalization is replaced with batch normalization. In the RSMT and OSMT construction problems, a linear layer is used in the RSMT problem to learn the feature vectors of unaccessed pins. After learning the characteristics of the accessed pins under routing mode 0 and routing mode 1 through two linear mappings, the features are then concatenated to form... During OSMT construction, the characteristics of accessed pins under four routing methods are learned through four linear mappings and then spliced together. .
8. The method for constructing Steiner minimum trees for Manhattan and non-Manhattan structures in deep reinforcement learning according to claim 1, characterized in that: The neural network model consists of two components: Actor and Critic; The role of the Actor is to generate EPS for a given set of pins through a neural network. That is, the Actor's action is to select a pair according to the probability generated by the model at each time step, and the state at the next time step is the SMT subtree determined by the pair in the current EPS, with the SMT line length as the reward. The role of the Critic is to predict the line length of the SMT constructed by the Actor, and use this as a baseline to help the Actor learn and improve its performance.
9. The method for constructing Steiner minimum trees for Manhattan and non-Manhattan structures in deep reinforcement learning according to claim 8, characterized in that: In step A5, an SMT is constructed using the DRL framework. The specific training method employed in this step is as follows: The Actor and Critic parameters are trained using a policy gradient-based Actor-Critic algorithm. The training objective is to improve the probability of the neural network generating excellent EPS. The Critic uses the same encoder structure as the Actor, but does not use ProjectionU and ProjectionV. The pin set... As input, the encoder generates the encoded result. ; The attention mechanism is used to calculate the weight of each vectorized pin distribution in the overall encoding result; then, the baseline is calculated through two fully connected layers and the ReLU activation function.