Obstacle-avoiding straight line Steiner minimum tree construction method based on deep reinforcement learning
Through a deep reinforcement learning method, combined with graph convolution network and deep reinforcement learning, the complexity problem of the minimum tree structure of the linear Steiner in VLSI circuit design is solved, and the wiring solution that minimizes the line length is realized, which improves the design Efficiency and chip performance.
Patent Information
- Application Number
- CN202411542390.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-31
- Publication Date
- 2025-02-18
AI Technical Summary
The prior art is difficult to effectively construct the Steiner minimal tree around the barrier line, especially in VLSI circuit design, where there are obstacles, resulting in increased wiring path complexity and inability to effectively minimize line length.
Using a deep reinforcement learning-based method, combined with graph convolutional network (GCN) and deep reinforcement learning (DRL), we automatically learn and generate a heuristic algorithm to solve the problem of the smallest tree of the Steiner in the obstacle line. This method constructs a wiring solution that minimizes line length by constructing a barrier generation graph, generating a shortest path forest, designing an encoder and decoder for neural network models, a fast line length evaluation strategy and a local optimization strategy.
In the process of constructing the minimum tree of the linear Steiner in VLSI circuit design, it automatically learns and generates a wiring solution that minimizes the length of the line, reducing the complexity and energy consumption of the wiring path and improving the efficiency of chip design.
Smart Images
Figure CN119476175A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of integrated circuit computer-aided design, and in particular to a Steiner minimum tree construction method for obstacle-avoiding lines based on deep reinforcement learning. Background Art
[0002] With the rapid development of the Internet, higher requirements are also put forward for hardware devices, and chips, as the core of hardware devices, play a vital role. The Rectilinear Steiner Minimal Tree (RSMT) problem has always been the focus of active research in academia, not only because of its theoretical importance, but also because of its important practical significance in the wiring stage of VLSI physical design. Chip designs with shorter wires help reduce chip delays and energy consumption.
[0003] Existing methods for the RSMT problem all assume an obstacle-free wiring plane. However, as the design of modern VLSI circuits shifts to the System-on-a-Chip paradigm, many reusable components (such as hard IPCore and macro, etc.) are incorporated into the circuit to improve design efficiency. These components are pre-designed and the wiring path is not allowed to cross them. Therefore, the Obstacle-Avoiding Rectilinear Steiner Minimal Tree (OARSMT) structure is proposed as a more practical problem and has attracted new attention in VLSI physical design. In addition, even the RSMT construction problem without considering obstacles has been proven to be np-hard, and the existence of obstacles will further increase the complexity of the problem.
[0004] Deep reinforcement learning is one of the paradigms and methodologies of machine learning, which is used to describe and solve the problem of how intelligent agents can maximize rewards or achieve specific goals by learning strategies during their interaction with the environment. Summary of the invention
[0005] The present invention proposes a Steiner minimum tree construction method for obstacle avoidance lines based on deep reinforcement learning, which can obtain a wiring solution that minimizes the line length.
[0006] The present invention adopts the following technical solutions.
[0007] A method for constructing a minimum Steiner tree of obstacle-avoiding lines based on deep reinforcement learning is used to obtain a wiring solution with minimized line length in a VLSI circuit design process; the method includes a framework based on deep reinforcement learning (DRL) for automatically learning and generating a heuristic algorithm for solving the OARSMT problem, which combines a graph convolution network (GCN) with DRL, extracts graph state feature information of the entire process of generating an obstacle-avoiding Steiner tree (OAST) through GCN, and uses DRL to learn strategic decisions under different graph states; The method comprises the following steps: Step S1, constructing an obstacle-avoiding spanning graph (OASG): that is, constructing an undirected connected graph of pins and obstacles, where any edge of the connected graph does not intersect with obstacles; using OASG as the original topological structure of deep reinforcement learning (DRL), the agent can ignore obstacles when constructing OAST on OASG; Step S2, generating a shortest path forest (SPF); Step S3, designing an encoder of the neural network model; Step S4, designing a decoder of the neural network model; Step S5, rapid line length assessment strategy, i.e., evaluating the quality of OAST by rapidly assessing the line length of OARSMT; Step S6, parameter update during neural network training; Step S7: local optimization: quickly optimize the line length of the OARSMT generated by the DRL through a local optimization strategy based on edge movement.
[0008] In step S1, when defining the construction of OASG, a set of pins and a set of rectangular obstacles are first given. For an undirected graph G connecting all pins and obstacle corners, if no edge intersects with the obstacle, the graph G is called OASG, and the line connecting any two nodes in the graph G is called an edge. Then, four scan line algorithms are performed on all nodes. The scan line algorithm includes horizontal scan lines, that is, quadrants 2 and 3 are determined from left to right, and quadrants 1 and 4 are determined from right to left. It also includes vertical scan lines, that is, quadrants 3 and 4 are determined from bottom to top, and quadrants 1 and 2 are determined from top to bottom.
[0009] In step S2, when there are pins and obstacle corners in OASG, the pins are connected through legal OARSMT, and the obstacle corners are used as relay points to connect auxiliary pins; SPF is constructed in OASG through multi-source Dijkstra algorithm, and each pin is a root of the shortest path tree (SPT), which is connected to the obstacle corner closest to the pin to capture the corner information. Step S2 also includes defining SPF and defining bridge; The specific method of defining SPF is as follows: in a given OASG, the path from a corner point to the nearest pin is called the shortest path of the corner point. The shortest paths of all corner points constitute multiple trees SPT with the pin as the root. The set of these trees is called SPF. The specific method of defining a bridge is to define the edges whose two ends do not belong to an SPT as a bridge. Figure 1 The OASG in (a) is constructed by the multi-source Dijkstra algorithm to obtain Figure 1 SPF in (b); in SPF, OAST is generated by selecting bridges.
[0010] In step S3, OAST is generated by DRL through the encoder, and the information of the wiring state is represented by serialized coding. In this step, the multi-head self-attention mechanism and GCN are used as feature extractors to extract the edge point interconnection features of each state in the process of generating OAST, so as to obtain the feature representation of the point embedding under the wiring state. Specifically, the encoder first uses a linear layer to embed the nodes, uses two layers of GCN to extract the interconnection features of the nodes, and uses three layers of multi-head attention layers to capture the dependencies of each node.
[0011] In step S4, the features of OAST point embedding are used as the input of the decoder. The decoder evaluates the value of the bridge according to the state encoding, and then selects the bridge. The selection of the bridge is implemented according to the ε-greedy algorithm. Specifically, the decoder splices the features of the two end nodes of an edge, applies a fully connected network and nonlinear transformation to the spliced vector, and obtains the edge embedding vector. The average value of the OAST edge embedding vector is calculated by row, and it is passed through a fully connected layer and nonlinear transformation to obtain a graph embedding representation. A multi-layer perceptron is used to calculate the value of all bridges in the current state; and an epsilon greedy strategy is used to select bridges.
[0012] In step S5, the fast evaluation strategy of line length includes defining a connection paradigm and defining overlapping benefits; using the connection paradigm to calculate the connection mode of each edge under different connection paradigms to maximize the overlapping benefits; The connection paradigm is defined as follows: the connection paradigm of an edge is represented by the numbering order of the nodes at both ends. The nodes with the first number lead to horizontal edges, and the nodes with the last number lead to vertical lines. The definition of overlapping gain is as follows: in OAST, edges adjacent to an edge, that is, edges that share the same endpoint are called adjacent edges, and the edges belonging to the set of OAST are called overlapping edges. After the transformation, the overlapping length of an edge of a certain connection paradigm and an edge in the set after horizontal and vertical projection is the overlapping gain of the edge under the connection paradigm.
[0013] In step S6, in terms of network training, specifically: the deep Q network algorithm is used to train the neural network, and the epsilon parameter of the epsilon greedy strategy is dynamically increased, with an initial value of 0.1, and multiplied by a coefficient of 1.005 after each episode.
[0014] In step S6, in terms of the loss function, specifically: the loss function loss value is calculated using the mean square error function, the Euclidean distance between the evaluation Q value and the target value is calculated, the gradient equation is obtained by differentiating the network weight parameters through the loss function, and the Adam optimizer is used for stochastic gradient descent training.
[0015] In step S7, the scenarios that can be optimized by the local optimization strategy based on edge movement are: Scenario A1: The line segment must be an auxiliary point with one end being a pin and the other end being a non-pin, or both ends being auxiliary points, and the line segment does not contain any auxiliary points on the pin.
[0016] Scene A2: Both ends of the line segment must be auxiliary points. The line segment can only contain one pin or auxiliary point, and the degrees of both endpoints of the line segment are 2.
[0017] In step S7, a local optimization strategy based on edge movement is used to shorten the length of the line segment.
[0018] The solution described in the present invention adopts deep reinforcement learning as the paradigm and methodology of machine learning, which is used to describe and solve the problem of how an intelligent agent can maximize its reward value or achieve a specific goal through learning strategies during its interaction with the environment. When used in the VLSI circuit design process, a wiring solution that minimizes line length can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments: Attached Figure 1 is a schematic diagram of the shortest path tree forest in the present invention; Attached Figure 2 is a schematic diagram of a neural network model in the present invention; Attached Figure 3 is a schematic diagram of the connection paradigm in the present invention; Attached Figure 4 Schematic diagram of the local optimization situation. DETAILED DESCRIPTION
[0020] As shown in the figure, a Steiner minimum tree construction method for obstacle-avoiding lines based on deep reinforcement learning is used to obtain a wiring solution for minimizing line length in the VLSI circuit design process; the method includes a framework based on deep reinforcement learning (DRL) for automatically learning and generating a heuristic algorithm for solving the OARSMT problem, which combines a graph convolution network (GCN) with DRL, extracts graph state feature information of the entire process of generating an obstacle-avoiding Steiner tree (OAST) through GCN, and uses DRL to learn strategic decisions under different graph states; The method comprises the following steps: Step S1, constructing an obstacle-avoiding spanning graph (OASG): that is, constructing an undirected connected graph of pins and obstacles, where any edge of the connected graph does not intersect with obstacles; using OASG as the original topological structure of deep reinforcement learning (DRL), the agent can ignore obstacles when constructing OAST on OASG; Step S2, generating a shortest path forest (SPF); Step S3, designing an encoder of the neural network model; Step S4, designing a decoder of the neural network model; Step S5, rapid line length assessment strategy, i.e., evaluating the quality of OAST by rapidly assessing the line length of OARSMT; Step S6, parameter update during neural network training; Step S7: local optimization: quickly optimize the line length of the OARSMT generated by the DRL through a local optimization strategy based on edge movement.
[0021] In step S1, when defining the construction of OASG, a set of pins and a set of rectangular obstacles are first given. For an undirected graph G connecting all pins and obstacle corners, if no edge intersects with the obstacle, the graph G is called OASG, and the line connecting any two nodes in the graph G is called an edge. Then, four scan line algorithms are performed on all nodes. The scan line algorithm includes horizontal scan lines, that is, quadrants 2 and 3 are determined from left to right, and quadrants 1 and 4 are determined from right to left. It also includes vertical scan lines, that is, quadrants 3 and 4 are determined from bottom to top, and quadrants 1 and 2 are determined from top to bottom.
[0022] In step S2, when there are pins and obstacle corners in OASG, the pins are connected through legal OARSMT, and the obstacle corners are used as relay points to connect auxiliary pins; SPF is constructed in OASG through multi-source Dijkstra algorithm, and each pin is a root of the shortest path tree (SPT), which is connected to the obstacle corner closest to the pin to capture the corner information. Step S2 also includes defining SPF and defining bridge; The specific method of defining SPF is as follows: in a given OASG, the path from a corner point to the nearest pin is called the shortest path of the corner point. The shortest paths of all corner points constitute multiple trees SPT with the pin as the root. The set of these trees is called SPF. The specific method of defining a bridge is to define the edges whose two ends do not belong to an SPT as a bridge. Figure 1 The OASG in (a) is constructed by the multi-source Dijkstra algorithm to obtain Figure 1 SPF in (b); in SPF, OAST is generated by selecting bridges.
[0023] In step S3, OAST is generated by DRL through the encoder, and the information of the wiring state is represented by serialized coding. In this step, the multi-head self-attention mechanism and GCN are used as feature extractors to extract the edge point interconnection features of each state in the process of generating OAST, so as to obtain the feature representation of the point embedding under the wiring state. Specifically, the encoder first uses a linear layer to embed the nodes, uses two layers of GCN to extract the interconnection features of the nodes, and uses three layers of multi-head attention layers to capture the dependencies of each node.
[0024] In step S4, the features of OAST point embedding are used as the input of the decoder. The decoder evaluates the value of the bridge according to the state encoding, and then selects the bridge. The selection of the bridge is implemented according to the ε-greedy algorithm. Specifically, the decoder splices the features of the two end nodes of an edge, applies a fully connected network and nonlinear transformation to the spliced vector, and obtains the edge embedding vector. The average value of the OAST edge embedding vector is calculated by row, and it is passed through a fully connected layer and nonlinear transformation to obtain a graph embedding representation. A multi-layer perceptron is used to calculate the value of all bridges in the current state; and an epsilon greedy strategy is used to select bridges.
[0025] In step S5, the fast evaluation strategy of line length includes defining a connection paradigm and defining overlapping benefits; using the connection paradigm to calculate the connection mode of each edge under different connection paradigms to maximize the overlapping benefits; The connection paradigm is defined as follows: the connection paradigm of an edge is represented by the numbering order of the nodes at both ends. The nodes with the first number lead to horizontal edges, and the nodes with the last number lead to vertical lines. The definition of overlapping gain is as follows: in OAST, edges adjacent to an edge, that is, edges that share the same endpoint are called adjacent edges, and the edges belonging to the set of OAST are called overlapping edges. After the transformation, the overlapping length of an edge of a certain connection paradigm and an edge in the set after horizontal and vertical projection is the overlapping gain of the edge under the connection paradigm.
[0026] In step S6, in terms of network training, specifically: the deep Q network algorithm is used to train the neural network, and the epsilon parameter of the epsilon greedy strategy is dynamically increased, with an initial value of 0.1, and multiplied by a coefficient of 1.005 after each episode.
[0027] In step S6, in terms of the loss function, specifically: the loss function loss value is calculated using the mean square error function, the Euclidean distance between the evaluation Q value and the target value is calculated, the gradient equation is obtained by differentiating the network weight parameters through the loss function, and the Adam optimizer is used for stochastic gradient descent training.
[0028] In step S7, the scenarios that can be optimized by the local optimization strategy based on edge movement are: Scenario A1: The line segment must be an auxiliary point with one end being a pin and the other end being a non-pin, or both ends being auxiliary points, and the line segment does not contain any auxiliary points on the pin.
[0029] Scene A2: Both ends of the line segment must be auxiliary points. The line segment can only contain one pin or auxiliary point, and the degrees of both endpoints of the line segment are 2.
[0030] In step S7, a local optimization strategy based on edge movement is used to shorten the length of the line segment.
Claims
1. A Steiner minimum tree construction method for obstacle-avoiding straight lines based on deep reinforcement learning, which is used in the VLSI circuit design process to obtain a wiring solution with minimized line length; it is characterized by: The method includes a framework based on deep reinforcement learning (DRL) for automatically learning and generating a heuristic algorithm for solving the OARSMT problem. The method combines a graph convolutional network (GCN) with DRL, extracts graph state feature information of the entire process of generating an obstacle-avoiding Steiner tree (OAST) through GCN, and uses DRL to learn policy decisions under different graph states. The method comprises the following steps: Step S1, constructing an obstacle avoidance generation graph OASG: that is, constructing an undirected connected graph of pins and obstacles, where any edge of the connected graph does not intersect with the obstacle; Taking OASG as the original topology of deep reinforcement learning (DRL), the agent can ignore obstacles when constructing OAST on OASG. Step S2, generating the shortest path forest SPF; Step S3, designing an encoder of the neural network model; Step S4, designing a decoder of the neural network model; Step S5, rapid line length assessment strategy, i.e., evaluating the quality of OAST by rapidly assessing the line length of OARSMT; Step S6, parameter update during neural network training; Step S7: local optimization: quickly optimize the line length of the OARSMT generated by the DRL through a local optimization strategy based on edge movement.
2. The Steiner minimum tree construction method for obstacle avoidance lines based on deep reinforcement learning according to claim 1, characterized in that: In step S1, when defining the construction of OASG, a set of pins and a set of rectangular obstacles are first given. For an undirected graph G connecting all pins and obstacle corners, if no edge intersects with the obstacle, the graph G is called OASG, and the line connecting any two nodes in the graph G is called an edge. Then, four scan line algorithms are performed on all nodes. The scan line algorithm includes horizontal scan lines, that is, quadrants 2 and 3 are determined from left to right, and quadrants 1 and 4 are determined from right to left. It also includes vertical scan lines, that is, quadrants 3 and 4 are determined from bottom to top, and quadrants 1 and 2 are determined from top to bottom.
3. The Steiner minimum tree construction method for obstacle avoidance lines based on deep reinforcement learning according to claim 1, characterized in that: In step S2, when there are pins and obstacle corners in OASG, the pins are connected through legal OARSMT, and the obstacle corners are used as relay points to connect auxiliary pins; SPF is constructed in OASG through multi-source Dijkstra algorithm, and each pin is a root of the shortest path tree SPT, which is connected to the obstacle corner closest to the pin to capture the corner information. Step S2 also includes defining SPF and defining bridge; The specific method of defining SPF is as follows: in a given OASG, the path from a corner point to the nearest pin is called the shortest path of the corner point. The shortest paths of all corner points constitute multiple trees SPT with the pin as the root. The set of these trees is called SPF. The specific method of defining a bridge is: define the edge whose two ends do not belong to an SPT as a bridge, construct it on OASG through the multi-source Dijkstra algorithm, and thus obtain SPF; in SPF, generate OAST by selecting the bridge.
4. The Steiner minimum tree construction method for obstacle avoidance lines based on deep reinforcement learning according to claim 3, characterized in that: In step S3, OAST is generated by DRL through the encoder, and the information of the wiring state is represented by serialized coding. In this step, the multi-head self-attention mechanism and GCN are used as feature extractors to extract the edge point interconnection features of each state in the process of generating OAST, so as to obtain the feature representation of the point embedding under the wiring state. Specifically, the encoder first uses a linear layer to embed the nodes, uses two layers of GCN to extract the interconnection features of the nodes, and uses three layers of multi-head attention layers to capture the dependencies of each node.
5. The Steiner minimum tree construction method for obstacle avoidance lines based on deep reinforcement learning according to claim 4, characterized in that: In step S4, the point embedding features of OAST are used as the input of the decoder. The decoder evaluates the value of the bridge according to the state code, and then selects the bridge. The selection of the bridge is implemented according to the ε-greedy algorithm. Specifically, the decoder splices the features of the two end nodes of an edge, applies a fully connected network and nonlinear transformation to the splicing vector, and obtains the edge embedding vector; The embedding vectors of the OAST edges are averaged row by row, and then go through a fully connected layer and nonlinear transformation to obtain the graph embedding representation. Use a multilayer perceptron to calculate the value of all bridges in the current state; use the epsilon greedy strategy to select bridges.
6. The Steiner minimum tree construction method for obstacle avoidance lines based on deep reinforcement learning according to claim 1, characterized in that: In step S5, the fast evaluation strategy of line length includes defining a connection paradigm and defining overlapping benefits; using the connection paradigm to calculate the connection mode of each edge under different connection paradigms to maximize the overlapping benefits; The connection paradigm is defined as follows: the connection paradigm of an edge is represented by the numbering order of the nodes at both ends. The nodes with the first number lead to horizontal edges, and the nodes with the last number lead to vertical lines. The definition of overlapping gain is as follows: in OAST, edges adjacent to an edge, that is, edges that share the same endpoint are called adjacent edges, and the edges belonging to the set of OAST are called overlapping edges. After the transformation, the overlapping length of an edge of a certain connection paradigm and an edge in the set after horizontal and vertical projection is the overlapping gain of the edge under the connection paradigm.
7. The Steiner minimum tree construction method for obstacle avoidance lines based on deep reinforcement learning according to claim 1, characterized in that: In step S6, in terms of network training, specifically: the deep Q network algorithm is used to train the neural network, and the epsilon parameter of the epsilon greedy strategy is dynamically increased, with an initial value of 0.1, and multiplied by a coefficient of 1.005 after each episode.
8. The Steiner minimum tree construction method for obstacle avoidance lines based on deep reinforcement learning according to claim 1, characterized in that: In step S6, in terms of the loss function, specifically: the loss function loss value is calculated using the mean square error function, the Euclidean distance between the evaluation Q value and the target value is calculated, the gradient equation is obtained by differentiating the network weight parameters through the loss function, and the Adam optimizer is used for stochastic gradient descent training.
9. The Steiner minimum tree construction method for obstacle avoidance lines based on deep reinforcement learning according to claim 1, characterized in that: In step S7, the scenarios that can be optimized by the local optimization strategy based on edge movement are: Scenario A1: The line segment must be an auxiliary point with one end being a pin and the other end being a non-pin, or both ends being auxiliary points, and the line segment does not contain any auxiliary points on the pin; Scene A2: Both ends of the line segment must be auxiliary points. The line segment can only contain one pin or auxiliary point, and the degrees of both endpoints of the line segment are 2.
10. The Steiner minimum tree construction method for obstacle avoidance lines based on deep reinforcement learning according to claim 9, characterized in that: In step S7, a local optimization strategy based on edge movement is used to shorten the length of the line segment.