Unification method for straight line and octagonal Steiner minimum tree construction based on deep reinforcement learning

Abstract

The invention relates to a straight line and octagonal Steiner minimum tree construction unification method based on deep reinforcement learning, and belongs to the technical field of integrated circuit computer aided design. The method comprises the following steps of: firstly, designing an Edge Point Sequence (EPS) according to the structural characteristics of the SMT to be used for bridging the difference between the output of a deep learning model and the SMT structure; secondly, a deep learning model is designed for EPS, and the model is trained through DRL with the negative line length of SMT as an award; secondly, a corresponding rapid and accurate line length calculation algorithm is provided for quality evaluation of the construction scheme, so that model training is accelerated; and finally, utilizing the randomness of a machine learning method to construct diversified SMT construction schemes. According to the method, the RSMT and OSMT problems can be solved, and diversified wiring topologies can be generated.

Description

Technical Field

[0001] The present invention belongs to the technical field of integrated circuit computer-aided design, and relates to a unified method for constructing straight-line and octagonal Steiner minimum trees based on deep reinforcement learning. Background Art

[0002] The construction of Steiner Minimum Tree (SMT) is a basic problem in Very Large Scale Integration (VLSI). It is widely used in the early stages of VLSI design, such as physical synthesis, floorplanning, interconnection planning and layout, and can be used to estimate wire load, congestion estimation and interconnection delay. SMT can be divided into Manhattan and non-Manhattan according to the routing direction of the wiring tree. Constructing a rectilinear Steiner Minimum Tree (RSMT) under the Manhattan structure has been proven to be an NP-hard problem. Octilinear Steiner Minimum Tree (OSMT) is a representative of the non-Manhattan structure. OSMT adds 45° and 135° wiring angles to the RSMT wiring method and is widely used in heterogeneous integration. Thanks to the existing manufacturing process, OSMT can effectively reduce wire length, interconnection delay and number of vias, thereby speeding up the design process. However, the increase in wiring angles also brings greater problem complexity. In addition, a given net may have an infinite number of SMTs. During the routing process, SMT is often used to construct the initial routing topology of the net to optimize constraints such as bus length, routing congestion, and critical path delay. Interconnect optimization such as buffer insertion also uses SMT to optimize timing and reduce dynamic power consumption. However, due to the complexity of the problem, traditional algorithms often only focus on constructing an optimal SMT for a given net, and the generated single routing topology is difficult to balance multiple constraints. SMT that generates multiple routing topologies for each net can effectively optimize timing, reduce routing overflow, reduce routing congestion, and reduce total coupling capacitance. Therefore, diversified SMTs construction schemes are very important in the chip design process. With the development of machine learning (ML), it also shows great potential to generate high-quality solutions for many NP-complete problems. Deep reinforcement learning (DRL) combines the perception ability of deep learning with the decision-making ability of reinforcement learning (RL), and often shows more advantages than traditional methods. Summary of the invention

[0003] The purpose of this invention is to solve the RSMT and OSMT problems and generate diversified wiring topologies, providing a unified method for the construction of straight and octagonal Steiner minimum trees based on deep reinforcement learning,

[0004] To achieve the above object, the technical solution of the present invention is: a unified method for constructing straight and octagonal Steiner minimum trees based on deep reinforcement learning, comprising:

[0005] Design edge point sequence EPS according to the structural characteristics of Steiner minimum tree SMT;

[0006] Design a deep learning model for EPS and train it through DRL using the negative line length of SMT as a reward;

[0007] The randomness of machine learning methods ML is used to construct diverse SMT construction schemes.

[0008] In one embodiment of the present invention, during the deep learning model training process, a corresponding fast and accurate line length calculation algorithm is provided for the quality assessment of the construction scheme to accelerate the training of the deep learning model.

[0009] In one embodiment of the present invention, during the deep learning model training process, the input of the deep learning model is a pin sequence in the form of two-dimensional coordinates in a given wire network, and the output of the deep learning model is converted into an SMT structure through EPS.

[0010] In one embodiment of the present invention, in the process of constructing diversified SMT construction schemes, a construction scheme with the highest probability is selected through a greedy strategy, and the parallelism of ML is utilized to generate diversified construction schemes based on random sampling of probabilities generated by a deep learning model.

[0011] In one embodiment of the present invention, an edge point sequence EPS is designed according to the structural characteristics of the Steiner minimum tree SMT, with the purpose of being able to be used for encoding both RSMT and OSMT structures, which is specifically implemented as follows:

[0012] The input of RSMT and OSMT construction problems is a set of line networks P = {p1,...,p n}, where pin p i From the coordinate pair (x i ,y i);The connection points other than pins introduced in the wiring process are called pseudo-Steiner points, which are represented by white squares; Given two pins a and b, there are four wiring methods between the pins as follows: Wiring method 1 first draws the vertical edge from pin a to the pseudo-Steiner point, i.e., point s, and then draws the horizontal edge from point s to pin b; Wiring method 2 first draws the horizontal edge from pin a to point s, and then draws the vertical edge from point s to pin b; Wiring method 3 first draws the vertical edge or horizontal edge from pin a to point s, and then draws the 45° edge or 135° edge from point s to pin b; Wiring method 4 first draws the 45° edge or 135° edge from pin a to point s, and then draws the vertical edge or horizontal edge from point s to pin b;

[0013] The construction goal of RSMT is to construct a straight line tree T(R) that connects all pins in the wire net P and has the shortest line length using only wiring method 1 or wiring method 2, that is, R is a superset of P The newly introduced points in R are Steiner points. The introduction of Steiner points can automatically merge overlapping edges to achieve the effect of reducing line length. Pseudo Steiner points only exist in the optimization process of routing, not in the constructed RSMT. Compared with RSMT, OSMT adds routing method 3 or routing method 4 in the construction process. The goal of constructing diversified RSMTs and diversified OSMTs is to construct as many high-quality routing topologies as possible for a given wire network.

[0014] For a given network P = {(x1,y1),...,(x n ,y n )}, an EPS of P has n-1 pairs {[a1,(b1,e1)],...,[a n-1 ,(b n-1 ,e n-1 )]}, where a t ,b t ∈{1,...,n},e t ∈{1,...,m}, for t=1,...,n-1, that is, in the tth pairing, the ath t Pin b t The wiring method e is used between the pins t Interconnection; in the RSMT problem m=2, in the OSMT problem m=4.

[0015] In one embodiment of the present invention, the deep learning model uses an encoder-decoder architecture. Specifically,

[0016] The encoder is similar to the Transformer structure. It maps the 2D coordinates of the pins to a 128-dimensional feature space through a linear layer, thereby mapping the pins from low-dimensional coordinate representation to high-dimensional vectors, and uses batch normalization to make the training more stable. In the RSMT and OSMT construction problems, a linear layer is used to learn the feature vector U of the unvisited pins. In the RSMT problem, the features of the visited pins under wiring mode 0 and wiring mode 1 are learned through two linear mappings and then concatenated into V. In the OSMT problem, the features of the visited pins under four wiring modes are learned through four linear mappings and then concatenated into V.

[0017] The decoder uses the encoding result U = [u1, ..., u n ] T and V=[v1,...,v m×n ] T As input, the output is the SMT construction scheme EPS; the deep learning model first selects one from U And mark a0 as an accessed pin, that is, this pin can be selected when selecting an accessed pin from V next, but cannot be selected when selecting an unaccessed pin; in the next n-1 pairing selection process, the selection rule for the tth pairing is as follows: The deep learning model will first select an unaccessed pin each time Then select one from V where b t =c t %n, Mark a t are visited pins; the pairs are added to the EPS in sequence; the deep learning model of the decoder part uses the attention mechanism to select the two pins in the pair and the wiring method between the pins.

[0018] In one embodiment of the present invention, the encoder uses a three-layer Transformer block, in which the multi-head attention mechanism uses 16 heads, the hidden layer of the feedforward neural network uses 512 dimensions, and the layer normalization is replaced by batch normalization.

[0019] In one embodiment of the present invention, during the deep learning model training process, a fast and accurate line length calculation algorithm is proposed, which is specifically implemented as follows:

[0020] The input of the algorithm is the line network P, the construction scheme eps and the type m used to confirm the constructed SMT. When eps is m=2, it is the construction scheme of RSMT, and when m=4, it is the construction scheme of OSMT. Extract the horizontal edge E according to eps x , vertical side E y 、45° side E z and 135° side E k Among them, E z With Ek Only exists in the calculation process of OSMT line length. The extraction form of horizontal edge is [ordinate, starting point horizontal coordinate, end point horizontal coordinate], the extraction form of vertical edge is [horizontal coordinate, starting point ordinate, end point ordinate], and the extraction form of hypotenuse is [starting point horizontal coordinate, starting point ordinate, end point horizontal coordinate, end point ordinate]. Among them, the horizontal coordinate of horizontal edge starts at the left and ends at the right, and the vertical coordinate of vertical edge starts at the bottom and ends at the top. When calculating the line length of OSMT, E z With E k Rotate 45° counterclockwise around the origin and adjust them to horizontal and vertical edges respectively, so as to merge overlapping edges. Sort different types of edges according to coordinates from small to large. The edges that may overlap are gathered in the first and last positions after sorting. The overlapping parts can be merged according to the sorted edges to calculate the total length.

[0021] In one embodiment of the present invention, the deep learning model includes two components: Actor and Critic; the role of Actor is to generate EPS for a given wire network through a neural network, that is, the action of Actor is to select a pairing according to the probability generated by the model in each time step, and the state in the next time step is the SMT subtree determined by the pairing in the current EPS, with the line length of SMT as a reward; the role of Critic is to predict the line length of SMT constructed by Actor, and use it as a baseline to help Actor learn and improve its performance.

[0022] In one embodiment of the present invention, the Actor-Critic algorithm based on policy gradient is used to train the Actor parameters and the Critic parameters, and the goal of the training is to improve the probability of the neural network generating excellent EPS; the Critic adopts the same encoder structure as the Actor, but does not use ProjectionU and ProjectionV; the wire mesh P is used as input, and the encoding result E′ is generated through the encoder; the attention mechanism is used to calculate the weight of each vectorized pin distribution to the overall encoding result; then, the baseline is calculated through two fully connected layers and the ReLU activation function.

[0023] Compared with the prior art, the present invention has the following beneficial effects: The method of the present invention: First, an edge point sequence (EPS) is designed according to the structural characteristics of SMT to bridge the gap between the output of the deep learning model and the SMT structure. Secondly, a deep learning model is designed for EPS, and the model is trained through DRL using the negative line length of SMT as a reward. Secondly, a corresponding fast and accurate line length calculation algorithm is provided for the quality evaluation of the construction scheme to accelerate the training of the model. Finally, the randomness of the machine learning method is used to construct a variety of SMT construction schemes. The method of the present invention can solve the RSMT and OSMT problems and generate a variety of wiring topologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 For wiring method.

[0025] Figure 2 An explanation of the edge point sequence.

[0026] Figure 3 It is a deep reinforcement learning framework. DETAILED DESCRIPTION

[0027] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings.

[0028] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0029] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0030] The present invention provides a unified method for constructing straight and octagonal Steiner minimum trees based on deep reinforcement learning, comprising:

[0031] Design edge point sequence EPS according to the structural characteristics of Steiner minimum tree SMT;

[0032] Design a deep learning model for EPS and train it through DRL using the negative line length of SMT as a reward;

[0033] The randomness of machine learning methods ML is used to construct diverse SMT construction schemes.

[0034] During the deep learning model training process, a corresponding fast and accurate line length calculation algorithm is provided for the quality evaluation of the construction scheme to accelerate the training of the deep learning model.

[0035] The following is the specific implementation process of the present invention.

[0036] This embodiment provides a unified method for constructing straight and octagonal Steiner minimum trees based on deep reinforcement learning, which is specifically implemented as follows:

[0037] 1. Wiring method

[0038] The input of RSMT and OSMT construction problems is a set of line networks P = {p1,...,p n}, where pin p i From the coordinate pair (x i ,y i );The connection points other than pins introduced in the wiring process are called pseudo-Steiner points, represented by white squares;Given two pins a and b, there are four wiring methods between the pins as follows: Wiring method 1 first draws the vertical edge from pin a to the pseudo-Steiner point, i.e., point s, and then draws the horizontal edge from point s to pin b; Wiring method 2 first draws the horizontal edge from pin a to point s, and then draws the vertical edge from point s to pin b; Wiring method 3 first draws the vertical or horizontal edge from pin a to point s, and then draws the 45° edge or 135° edge from point s to pin b; Wiring method 4 first draws the 45° edge or 135° edge from pin a to point s, and then draws the vertical or horizontal edge from point s to pin b.

[0039] The construction goal of RSMT is to construct a straight line tree T(R) that connects all pins in the wire net P and has the shortest line length using only wiring method 1 or wiring method 2, that is, R is a superset of P The newly introduced points in R are Steiner points. The introduction of Steiner points can automatically merge overlapping edges to achieve the effect of reducing line length. Pseudo Steiner points only exist in the optimization process of routing and do not exist in the constructed RSMT. Compared with RSMT, OSMT adds routing method 3 or routing method 4 in the construction process. The goal of constructing diversified RSMTs and diversified OSMTs is to construct as many high-quality routing topologies as possible for a given wire network.

[0040] 2. Edge point sequence

[0041] For a given network P = {(x1,y1),...,(x n ,y n )}, an EPS of P has n-1 pairs {[a1,(b1,e1)],...,[an-1 ,(b n-1 ,e n-1 )]}, where a t ,b t ∈{1,...,n},e t ∈{1,...,m}, for t=1,...,n-1, that is, in the tth pairing, the ath t Pin b t The wiring method e is used between the pins t interconnect; in RSMT problem m=2, in OSMT problem m=4. For example, given a net P={(0,2),(2,5),(4,0),(5,4)}, the pin positions are as follows Figure 2 (a). A RSMT feasible solution of P is eps1 = {[1, (3, 2)], [4, (1, 1)], [2, (1, 1)]}, as Figure 2 (b) shows the optimal RSMT. A feasible solution of OSMT is eps2 = {[1, (2, 2)], [4, (2, 2)], [3, (1, 4)]}, as Figure 2 (c) shows the optimal OSMT.

[0042] 3. Neural Network Model

[0043] The neural network model uses an encoder-decoder architecture. The encoder used is similar to the Transformer structure, which maps the 2-dimensional coordinates of the pin to a 128-dimensional feature space through a linear layer, thereby mapping the pin from a low-dimensional coordinate representation to a high-dimensional vector, and using batch normalization to make the training more stable. In the specific implementation, a three-layer Transformer block is used, in which the multi-head attention mechanism uses 16 heads, the feedforward neural network hidden layer uses 512 dimensions, and the layer normalization is replaced with batch normalization. In the RSMT and OSMT construction problems, a linear layer is used to learn the feature vector U of the unvisited pins. In the RSMT problem, the features of the visited pins under wiring method 1 and wiring method 2 are learned through two linear mappings and then spliced ​​into V. In the OSMT construction process, the features of the visited pins under 4 wiring methods are learned through four linear mappings and then spliced ​​into V. The overall structure of the encoder is as follows Figure 3 (a) shown.

[0044] The decoder uses the encoding result U = [u1, ..., u n ] T and V=[v1,...,v m×n ] T As input, the output is the SMT construction scheme EPS. The neural network first selects a And mark a0 as a visited pin (that is, this pin can be selected when selecting visited pins from V next, but cannot be selected when selecting unvisited pins). In the next n-1 pairing selection process, the selection rule for the tth pairing is as follows: The neural network will first select an unvisited pin each time Then select one from V (where b t =c t %n, ), mark a t The pairings are added to the EPS in sequence. The neural network in the decoder part uses the attention mechanism to select the two pins in the pairing and the wiring method between the pins.

[0045] 4. Fast and accurate line length calculation method

[0046] The input of the algorithm is the line network P, the construction scheme eps and the type m used to confirm the constructed SMT. When eps is m=2, it is the construction scheme of RSMT, and when m=4, it is the construction scheme of OSMT. Extract the horizontal edge E according to eps x , vertical side E y 、45° side E z and 135° side E k Among them, E z With E k Only exists in the calculation process of OSMT line length. The extraction form of horizontal edge is [ordinate, starting point horizontal coordinate, end point horizontal coordinate], the extraction form of vertical edge is [horizontal coordinate, starting point ordinate, end point ordinate], and the extraction form of hypotenuse is [starting point horizontal coordinate, starting point ordinate, end point horizontal coordinate, end point ordinate]. Among them, the horizontal coordinate of horizontal edge starts at the left and ends at the right, and the vertical coordinate of vertical edge starts at the bottom and ends at the top. When calculating the line length of OSMT, E z With E k Rotate 45° counterclockwise around the origin and adjust them to horizontal and vertical edges respectively, so as to merge overlapping edges. Sort different types of edges according to coordinates from small to large. The edges that may overlap are gathered in the first and last positions after sorting. The overlapping parts can be merged according to the sorted edges to calculate the total length.

[0047] 5. Training methods

[0048] Figure 3(b) shows the DRL framework for constructing SMT. The model mainly consists of two components: Actor and Critic. The role of Actor is to generate EPS for a given wire network through a neural network, that is, the action of Actor is to select a pairing according to the probability generated by the model in each time step, and the state in the next time step is the SMT subtree determined by the pairing in the current EPS, with the wire length of SMT as a reward. The role of Critic is to predict the wire length of the SMT constructed by Actor, and use this as a baseline to help Actor learn and improve its performance. The Actor-Critic algorithm based on policy gradient is used to train the Actor parameters and the Critic parameters. The goal of training is to increase the probability of the neural network generating an excellent EPS. Critic uses the same encoder structure as Actor, but does not use ProjectionU and ProjectionV. With the wire network P as input, the encoder generates the encoding result E′. The attention mechanism is used to calculate the weight of each vectorized pin distribution to the overall encoding result. Subsequently, the baseline is calculated through two fully connected layers and ReLU activation function.

[0049] The present invention also provides a unified system for constructing straight and octagonal Steiner minimum trees based on deep reinforcement learning, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the method steps described above can be implemented.

[0050] The present invention also provides a computer-readable storage medium, on which computer program instructions that can be executed by a processor are stored. When the processor executes the computer program instructions, the method steps described above can be implemented.

[0051] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0052] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0053] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0054] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0055] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A unified method for constructing linear and octagonal Steiner minimum trees based on deep reinforcement learning, characterized in that: include: Design edge point sequence EPS according to the structural characteristics of Steiner minimum tree SMT; Design a deep learning model for EPS and train it through DRL using the negative line length of SMT as a reward; The randomness of machine learning methods ML is used to construct diverse SMT construction schemes.

2. A unified method for constructing straight and octagonal Steiner minimum trees based on deep reinforcement learning according to claim 1, characterized in that: During the deep learning model training process, a corresponding fast and accurate line length calculation algorithm is provided for the quality evaluation of the construction scheme to accelerate the training of the deep learning model.

3. The unified method for constructing straight and octagonal Steiner minimum trees based on deep reinforcement learning according to claim 1, characterized in that: During the deep learning model training process, the input of the deep learning model is a pin sequence in the form of two-dimensional coordinates in a given wire network, and the output of the deep learning model is converted into an SMT structure through EPS.

4. The unified method for constructing straight and octagonal Steiner minimum trees based on deep reinforcement learning according to claim 1, characterized in that: In the process of constructing diverse SMT construction schemes, the construction scheme with the highest probability is selected through a greedy strategy, and the parallelism of ML is utilized to randomly sample the probabilities generated by the deep learning model to generate diverse construction schemes.

5. The unified method for constructing straight and octagonal Steiner minimum trees based on deep reinforcement learning according to claim 1, characterized in that: The edge sequence EPS is designed based on the structural characteristics of the Steiner minimum tree SMT, with the aim of being able to encode both RSMT and OSMT structures. The specific implementation is as follows: The input of RSMT and OSMT construction problems is a set of line networks P = {p1,...,p n }, where pin p i From the coordinate pair (x i ,y i );The connection points other than pins introduced in the wiring process are called pseudo-Steiner points, which are represented by white squares; Given two pins a and b, there are four wiring methods between the pins as follows: Wiring method 1 first draws the vertical edge from pin a to the pseudo-Steiner point, i.e., point s, and then draws the horizontal edge from point s to pin b; Wiring method 2 first draws the horizontal edge from pin a to point s, and then draws the vertical edge from point s to pin b; Wiring method 3 first draws the vertical edge or horizontal edge from pin a to point s, and then draws the 45° edge or 135° edge from point s to pin b; Wiring method 4 first draws the 45° edge or 135° edge from pin a to point s, and then draws the vertical edge or horizontal edge from point s to pin b; The construction goal of RSMT is to construct a straight line tree T(R) that connects all pins in the wire net P and has the shortest line length using only wiring method 1 or wiring method 2, that is, R is a superset of P The newly introduced points in R are Steiner points. The introduction of Steiner points can automatically merge overlapping edges to achieve the effect of reducing line length. Pseudo Steiner points only exist in the optimization process of routing, not in the constructed RSMT. Compared with RSMT, OSMT adds routing method 3 or routing method 4 in the construction process. The goal of constructing diversified RSMTs and diversified OSMTs is to construct as many high-quality routing topologies as possible for a given wire network. For a given network P = {(x1,y1),...,(x n ,y n )}, an EPS of P has n-1 pairs {[a1,(b1,e1)],...,[a n-1 ,(b n-1 ,e n-1 )]}, where a t ,b t ∈{1,...,n},e t ∈{1,...,m}, for t=1,...,n-1, that is, in the tth pairing, the ath t Pin b t The wiring method e is used between the pins t Interconnection; in the RSMT problem m=2, in the OSMT problem m=4.

6. A unified method for constructing straight and octagonal Steiner minimum trees based on deep reinforcement learning according to claim 5, characterized in that: Deep learning models use an encoder-decoder architecture. Specifically, The encoder is similar to the Transformer structure. It maps the 2D coordinates of the pins to a 128-dimensional feature space through a linear layer, thereby mapping the pins from low-dimensional coordinate representation to high-dimensional vectors, and uses batch normalization to make the training more stable. In the RSMT and OSMT construction problems, a linear layer is used to learn the feature vector U of the unvisited pins. In the RSMT problem, the features of the visited pins under wiring mode 0 and wiring mode 1 are learned through two linear mappings and then concatenated into V. In the OSMT problem, the features of the visited pins under four wiring modes are learned through four linear mappings and then concatenated into V. The decoder uses the encoding result U = [u1, ..., u n ] T and V=[v1,...,v m×n ] T As input, the output is the SMT construction scheme EPS; the deep learning model first selects one from U And mark a0 as an accessed pin, that is, this pin can be selected when selecting an accessed pin from V next, but cannot be selected when selecting an unaccessed pin; in the next n-1 pairing selection process, the selection rule for the tth pairing is as follows: The deep learning model will first select an unaccessed pin each time Then select one from V in Mark a t are visited pins; the pairs are added to the EPS in sequence; the deep learning model of the decoder part uses the attention mechanism to select the two pins in the pair and the wiring method between the pins.

7. The unified method for constructing straight and octagonal Steiner minimum trees based on deep reinforcement learning according to claim 6, characterized in that: The encoder uses a three-layer Transformer block, in which the multi-head attention mechanism uses 16 heads, the feedforward neural network hidden layer uses 512 dimensions, and the layer normalization is replaced by batch normalization.

8. The unified method for constructing straight and octagonal Steiner minimum trees based on deep reinforcement learning according to claim 5, characterized in that: During the deep learning model training process, a fast and accurate line length calculation algorithm is proposed, which is implemented as follows: The input of the algorithm is the line network P, the construction scheme eps and the type m used to confirm the constructed SMT; eps is the construction scheme of RSMT when m=2 and the construction scheme of OSMT when m=4; according to eps, the horizontal edge E is extracted. x , vertical side E y 、45° side E z and 135° side E k ; Among them, E z With E k Only exists in the calculation process of OSMT line length; the extraction form of horizontal edge is [ordinate, starting point horizontal coordinate, end point horizontal coordinate], the extraction form of vertical edge is [horizontal coordinate, starting point ordinate, end point ordinate], and the extraction form of hypotenuse is [starting point horizontal coordinate, starting point ordinate, end point horizontal coordinate, end point ordinate]; among them, the horizontal coordinate of horizontal edge starts at the left and ends at the right, and the vertical coordinate of vertical edge starts at the bottom and ends at the top; when calculating the line length of OSMT, E z With E k Rotate 45° counterclockwise around the origin and adjust them into horizontal and vertical edges respectively to facilitate merging overlapping edges; sort different types of edges according to coordinates from small to large. The edges that may overlap will be gathered in the first and last positions after the sorting is completed. The overlapping parts can be merged according to the sorted edges to calculate the total length.

9. The unified method for constructing straight and octagonal Steiner minimum trees based on deep reinforcement learning according to claim 6, characterized in that: The deep learning model consists of two components: Actor and Critic; the role of Actor is to generate EPS for a given wire network through a neural network, that is, the action of Actor is to select a pairing according to the probability generated by the model in each time step, and the state in the next time step is the SMT subtree determined by the pairing in the current EPS, with the line length of SMT as the reward; the role of Critic is to predict the line length of SMT constructed by Actor, and use it as a baseline to help Actor learn and improve its performance.

10. A unified method for constructing straight and octagonal Steiner minimum trees based on deep reinforcement learning according to claim 9, characterized in that: The Actor-Critic algorithm based on policy gradient is used to train the Actor parameters and the Critic parameters. The goal of the training is to improve the probability of the neural network generating excellent EPS. The Critic uses the same encoder structure as the Actor, but does not use ProjectionU and ProjectionV. The wire net P is used as input and the encoding result E′ is generated through the encoder. The attention mechanism is used to calculate the weight of each vectorized pin distribution to the overall encoding result. Subsequently, the baseline is calculated through two fully connected layers and the ReLU activation function.

Images