Integrated circuit wiring method and system based on deep reinforcement learning and RSMT

Abstract

The present invention relates to the field of integrated circuit wiring technology, and more particularly to an integrated circuit wiring method and system based on deep reinforcement learning and RSMT. The present invention proposes an improved circuit wiring model to RES The solution not only introduces a selective convolution layer in the encoder, but also incorporates a feature processing layer designed based on the macaron deformation layer: the former realizes the extraction and dynamic fusion of multi-scale features, effectively captures the local details and multi-scale features of the point set, thereby improving the model's adaptability to complex point distributions and providing a more accurate feature representation for solution construction; the latter adds residual paths and hierarchical feature processing, further improving the global optimization capability of solution construction, making the generated solution more accurate and with lower errors. The present invention solves the problem that the existing REST method still needs to be improved in terms of feature expression ability, training efficiency and adaptability to large-scale point sets.

Description

Technical Field

[0001] The present invention relates to the field of integrated circuit wiring technology, and more specifically to: 1. An integrated circuit wiring method based on deep reinforcement learning and RSMT; 2. An integrated circuit wiring system based on deep reinforcement learning and RSMT. The present invention is mainly used in circuit wiring, chip layout optimization and other combinatorial optimization problems involving network connections, aiming to optimize connection problems in circuit design through deep reinforcement learning technology and improve wiring efficiency and accuracy. Background Art

[0002] The Rectilinear Steiner Minimum Tree (RSMT) problem is a fundamental problem in the field of electronic design automation (EDA). Its goal is to connect a given set of points (called pins) using the shortest straight line segments on a two-dimensional plane, and allow the introduction of additional auxiliary points (called Steiner points) to further optimize the total line length. This problem is widely used in circuit board routing, global routing in chip design, clock tree synthesis, and routing delay estimation. However, since the RSMT problem has been proven to be an NP-complete problem, finding the optimal solution when the number of nodes is large is extremely computationally challenging.

[0003] With the rapid development of deep learning and reinforcement learning technologies, researchers have tried to introduce these technologies into the RSMT problem. For example, the existing REST method provides a pioneering method based on reinforcement learning, which uses the Actor-Critic architecture combined with the Transformer encoder to gradually generate a node connection sequence through reinforcement learning to solve RSMT. However, after experiments, it was found that the REST method only performs well on smaller point sets (number of nodes ≤ 40), and there is still room for improvement in feature expression ability, training efficiency, and adaptability to large-scale point sets (number of nodes > 40). Summary of the invention

[0004] Based on this, it is necessary to provide an integrated circuit wiring method and system based on deep reinforcement learning and RSMT to address the problems that the existing REST method still needs to be improved in terms of feature expression ability, training efficiency and adaptability to large-scale point sets.

[0005] The present invention is implemented by the following technical solutions:

[0006] In a first aspect, the present invention discloses an integrated circuit wiring method based on deep reinforcement learning and RSMT, comprising:

[0007] Step 1: Get all the M Coordinates of nodes to be processed Point 1~Point M ;

[0008] Step 2: Use the trained circuit wiring model to Point 1~ Point M Processing to obtain circuit wiring results RES ; M >1.

[0009] Among them, the trained circuit wiring model is trained using a self-critical sequence training algorithm.

[0010] The circuit wiring model includes: encoder Encoder and decoder Decoder. Encoder is used to Point 1~ Point M Encode to get M Encoded features Encoding 1~ Encoding M ;Decoder is used to Encoding 1~ Encoding M Decode and gradually generate node connection sequences to obtain RES .

[0011] Among them, Encoder includes: 1 embedding layer Embedder, 1 selective convolution layer SKConv, and 1 feature processing layer MT.

[0012] Embedder is used for m Nodes to be processed Point m Extract preliminary features and generate m Embedding Embedding m SKConv is used to Embedding m Extract multi-scale features and perform dynamic fusion to obtain the m The fused multi-scale features MS-features m MT is used for MS-features m Perform global feature enhancement and obtain m Encoded features Encoding m .

[0013] Among them, MT includes: N Macaron deformation layer mt1~mt N ; N ≥1;mt n The output of mtn+1 The input of mt1 is MS-features m ;mt N The output is Encoding m ; n ∈[1, N -1].

[0014] This integrated circuit wiring method based on deep reinforcement learning and RSMT implements the method or process according to the embodiment of the present disclosure.

[0015] In a second aspect, the present invention discloses an integrated circuit wiring system based on deep reinforcement learning and RSMT, which uses the integrated circuit wiring method based on deep reinforcement learning and RSMT disclosed in the first aspect.

[0016] The integrated circuit wiring system based on deep reinforcement learning and RSMT includes: a point set acquisition module and a circuit wiring module.

[0017] The point set acquisition module is used to obtain all M Coordinates of nodes to be processed Point 1~ Point M The circuit routing module is used to use the trained circuit routing model to Point 1~ Point M Processing to obtain circuit wiring results RES .

[0018] This integrated circuit wiring system based on deep reinforcement learning and RSMT implements the method or process according to the embodiment of the present disclosure.

[0019] Compared with the prior art, the present invention has the following beneficial effects:

[0020] 1. The present invention proposes an improved circuit wiring model to carry out RES In order to solve the problem, the encoder not only introduces a selective convolution layer, but also incorporates a feature processing layer designed based on the macaron deformation layer: the former realizes the extraction and dynamic fusion of multi-scale features, effectively captures the local details and multi-scale features of the point set, thereby improving the model's adaptability to complex point distributions and providing a more accurate feature representation for the construction of the solution; the latter adds residual paths and hierarchical feature processing to further enhance the global optimization capability of the solution construction, making the generated solution more accurate and with lower errors.

[0021] 2. The present invention adopts the self-critical sequential training (SCST) algorithm, which optimizes the generation strategy by using the reward difference between the benchmark solution and the generated solution, avoids strong dependence on high-quality benchmark solutions, improves the efficiency and convergence speed of model training, and ensures that the generated solution has higher quality and robustness.

[0022] 3. After simulation comparison, under the condition of similar test time consumption, the RES results of the present invention under different numbers of nodes are better than those of the REST method, and show more stable performance - as the number of nodes increases, the line length error of the present invention remains stable. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0024] Figure 1 A flowchart of an integrated circuit wiring method based on deep reinforcement learning and RSMT provided in Example 1 of the present invention;

[0025] Figure 2 for Figure 1 The structure diagram of the encoder in the coder;

[0026] Figure 3 for Figure 2 The nth macaron deformation layer MT n Structure diagram of

[0027] Figure 4 for Figure 3 The structure diagram of the feed-forward processing layer in the middle;

[0028] Figure 5 for Figure 1 The structure diagram of the decoder in the decoder;

[0029] Figure 6 This is a simulation comparison result diagram proposed in Example 1 of the present invention. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] It should be noted that when a component is referred to as being "mounted on" another component, it may be directly on the other component or there may be a central component. When a component is considered to be "set on" another component, it may be directly set on the other component or there may be a central component at the same time. When a component is considered to be "fixed to" another component, it may be directly fixed on the other component or there may be a central component at the same time.

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "or / and" used herein includes any and all combinations of one or more related listed items.

[0033] Example 1

[0034] See also Figure 1 , Figure 1 A flowchart of an integrated circuit wiring method based on deep reinforcement learning and RSMT is shown, which includes:

[0035] Step 1: Get all the M Coordinates of nodes to be processed Point 1~ Point M .

[0036] Since this method is aimed at routing in a plane area, it is necessary to first specify the target plane area and all the nodes to be processed that need to be connected to the target plane area (a total of M indivual, M >1). The coordinates of each node to be processed are two-dimensional and known quantities - they are used as inputs for subsequent models.

[0037] Step 2: Use the trained circuit wiring model to Point 1~ Point M Processing to obtain circuit wiring results RES (Rectilinear Edge Sequence).

[0038] For the above-mentioned circuit wiring model, its specific structure is an innovation of the present invention, and its training method is another innovation of the present invention.

[0039] (I) First, let’s look at the structure of the circuit wiring model. Figure 1 As shown, the circuit wiring model can be divided into two parts according to its function: encoder and decoder.

[0040] I, encoder Encoder is used to M Coordinates of nodes to be processed Point 1~ Point M Encode to get M Encoded features Encoding 1~ Encoding M .

[0041] It should be emphasized that the present invention adjusts the specific structure of the encoder:

[0042] See also Figure 2 , the encoder Encoder includes: 1 embedding layer Embedder, 1 selective convolution layer SKConv, and 1 feature processing layer MT.

[0043] Specific:

[0044] ①The embedding layer is used to m Nodes to be processed Point m Extract preliminary features and generate m Embedding Embedding m . m ∈[ 1, M ].

[0045] The embedding layer is designed to perform preliminary feature extraction and is a one-dimensional convolutional layer. That is, in the embedding layer, Point m Perform a one-dimensional convolution to obtain Embedding m .

[0046] The above process can be formulated as:

[0047] ;

[0048] In the formula, Conv (.) represents one-dimensional convolution.

[0049] ② The selective convolution layer SKConv is used to m Embedding Embedding m Extract multi-scale features and perform dynamic fusion to obtain the m The fused multi-scale features MS-features m .

[0050] The selective convolution layer SKConv converts the firstm Embedding Embedding m as input.

[0051] Among them, m Embedding Embedding m The dimension is ( B , C in , N in ); B represents the batch size, C in represents the number of input feature channels, N in Indicates the number of sample points.

[0052] In SKConv, on the one hand, K The branch convolutions of different scales are m Embedding Embedding m Perform convolution operation to obtain K On the other hand, the channel attention mechanism CAM is used to dynamically generate K The branch convolution corresponds to K convolution weights, and K The convolution results are weighted fused to get the m The fused multi-scale features MS- features m .

[0053] Among them, m The fused multi-scale features MS-features m The dimension is ( B , N in , C out ); C out Indicates the number of output feature channels.

[0054] The above process can be formulated as:

[0055] ;

[0056] In the formula, w i Indicates i Dynamic weights corresponding to branch convolutions; Conv kernel=3+2(i-1) Indicates i branch convolution; kernel=3+2( i-1) indicates the i The convolution kernel specifications used by the branch convolution; Conv kernel=3+2(i-1) ( Embedding m ) indicates the i The convolution result.

[0057] It should be noted that, in this embodiment 1, K It is recommended to take 3 (the convolution kernel sizes of the three branches are 3, 5, and 7 respectively) - this can achieve a balance between model computing consumption and usage effect.

[0058] ③The feature processing layer MT is used to m The fused multi-scale features MS-features m Perform global feature enhancement and obtain m Encoded features Encoding m .

[0059] The feature processing layer MT converts the m The fused multi-scale features MS-features m as input.

[0060] like Figure 2 As shown, the feature processing layer MT includes: N Macaron Transformer layers mt1~mt N ; N ≥1.

[0061] It should be noted that, in this embodiment 1, N It is recommended to choose 3, which can achieve the best balance between model computing consumption and usage effect.

[0062] Among them, the nth macaron deformation layer mt n The output is used as the n+1th macaron deformation layer mt n+1 Input; n ∈[1, N -1].

[0063] The input of the first macaron deformation layer mt1 is MS-features m ; Nth macaron deformation layer mt N The output is Encoding m .

[0064] First of all, see Figure 3 , N Macaron deformation layer mt1~mt NThe structures are the same, including: 2 feedforward network layers FFN1~FFN2 and 1 attention network layer Attention.

[0065] For any macaron deformation layer:

[0066] 301, the feed-forward network layer FFN1 is used to input the macaron deformation layer IN Perform preliminary enhancement processing;

[0067] 302. The attention network layer Attention is used to weight the output of the feedforward network layer FFN1 through a multi-head self-attention mechanism to capture global dependencies;

[0068] 303. The feed-forward network layer FFN2 is used to further enhance the output of the attention network layer Attention to obtain the output of the macaron deformation layer OUT .

[0069] The above process can be formulated as:

[0070] ;

[0071] In the formula, FFN 1(.) represents the processing process of the feedforward network layer FFN1; Att (.) represents the processing process of the attention network layer Attention; FFN 2(.) represents the processing process of the feedforward network layer FFN2.

[0072] That is to say, N Macaron deformation layer mt1~mt N All follow the above structural design.

[0073] Secondly, see Figure 3 The feedforward network layers FFN1~FFN2 have the same structure, including: 1 feedforward processing layer Feed-Forward, 1 residual connection layer Plus, and 1 batch normalization layer Batch-Norm.

[0074] For any feed-forward network layer:

[0075] 3011, the feed-forward processing layer Feed-Forward is used to input the feed-forward network layer in 1. Perform linear processing first, then scale it to half to get the feedforward feature ff 1.

[0076] For details, see Figure 4The feed-forward processing layer Feed-Forward includes: 2 linear change layers Linear1~Linear2, 1 ReLU activation layer Relu, 1 random inactivation layer Dropout, and 1 scaling layer Scaling-Layer.

[0077] Among them, the linear change layer Linear1 is used to in 1 Perform linear transformation to achieve dimensionality increase;

[0078] The ReLU activation layer Relu is used to activate the output of Linear1 using the ReLU activation function;

[0079] The linear change layer Linear2 is used to perform linear transformation on the output of the ReLU activation layer Relu to achieve dimensionality reduction;

[0080] The random dropout layer Dropout is used to randomly drop the output of the linear change layer Linear2;

[0081] The scaling layer is used to scale the output of the random dropout layer to half, and we get ff 1.

[0082] The above process can be formulated as:

[0083] ;

[0084] In the formula, Dropout (.) represents the process of Dropout; Linear 1(.) represents the processing of the linear change layer Linear1; ReLU(.) represents the processing of the ReLU activation layer Relu; Linear 2(.) represents the processing process of the linear change layer Linear2.

[0085] 3012, the residual connection layer Plus is used to connect the output of the feed-forward processing layer Feed-Forward with ff 1. Overlay;

[0086] 3013. Batch-Norm is used to perform batch normalization on the output of the residual connection layer to obtain the output of the feedforward network layer. out 1.

[0087] The above process can be formulated as:

[0088] ;

[0089] In the formula, Norm(.) represents the processing process of the batch normalization layer Batch-Norm; FF (.) represents the processing process of the feed-forward processing layer.

[0090] Next, see Figure 3 The attention network layer Attention includes: 1 multi-head self-attention mechanism Multi-Head-Attention, 1 residual connection layer Plus, and 1 batch normalization layer Batch-Norm.

[0091] For any attention network layer:

[0092] 311. Multi-Head-Attention mechanism is used for the input of the attention network layer in 2 Perform multi-head attention calculation.

[0093] The multi-head self-attention mechanism aims to capture the global dependencies between features. in 2. Processing is performed in three dimensions: query, key, and value. The calculation process can be expressed as follows:

[0094] ;

[0095] In the formula, Q represents the linear mapping from in Query generated in 2; K represents the linear mapping from in 2; V represents the key generated by linear mapping from in The Value generated in 2; d k represents the scaling factor; Softmax(.) represents the Softmax function.

[0096] 312. The residual connection layer Plus is used to connect the output of Multi-Head-Attention with in 2. Overlay;

[0097] 313. Batch-Norm is used to batch normalize the output of the residual connection layer Plus to obtain the output of the attention network layer. out 2.

[0098] The above process can be formulated as:

[0099] ;

[0100] In the formula, Norm(.) represents the Batch-Norm processing process; MHA(.) represents the Multi-Head-Attention processing process.

[0101] Ⅱ, decoder is used to M Encoded features Encoding 1~ Encoding M Decode and gradually generate node connection sequences to obtain circuit wiring results RES .

[0102] See also Figure 5 The structure of the decoder adopts the same design as the REST method, and can be divided into: feature extraction layer FE, path selection and optimization layer Patch-SU according to its function.

[0103] The feature extraction layer FE is used to Encoding 1~ Encoding M The path selection and optimization layer Patch-SU is used to perform multiple path selections on the output of the feature extraction layer FE, and update and iterate to gradually optimize the path, and finally obtain the circuit wiring result. RES .

[0104] That is, the output of the encoder is used as the input of the decoder. The decoder first selects a node to be processed as the starting point through the pointing mechanism, and gradually generates edges in the subsequent deployment. After that, the edge pairs generated in each step are updated to RES In the process, the selected node to be processed in the current step is marked as visited; in this way, through recursive calls, the connection of all nodes to be processed is gradually completed, and the complete RES , ensure that the connection requirements of RSMT are met.

[0105] (ii) Next, let's look at the training method of the circuit wiring model. Different from the conventional training method using a training set with real labels, the trained circuit wiring model in the present invention is trained using a self-critical sequential training (SCST) algorithm.

[0106] Specifically, the training method of the circuit wiring model includes:

[0107] S1, initialize the model hyperparameters and model parameters of the circuit wiring model, and randomly generate several two-dimensional node coordinates to form a training set.

[0108] It should be noted that S1 does not need to construct a sample set with real labels, but instead constructs a training set by randomly generating two-dimensional node coordinates.

[0109] S2, circuit wiring model based on training set L In each round of training, the model loss is calculated to update the model parameters by gradient until a trained circuit wiring model is obtained.

[0110] In S2, each round of training uses U sample sets, each sample set includes the same number of two-dimensional node coordinates.

[0111] Since the processing of each round of training is similar, l Take round training as an example to illustrate:

[0112] S201, first go through l -1 round of training circuit wiring model for the l The first round of training used j The sample set generates a benchmark sequence based on a deterministic strategy and uses the negative value of the length of the benchmark sequence as the first l Baseline reward for a training round ;

[0113] Then through the l -1 round of training circuit wiring model for the l The first round of training used j The sample set generates an exploration sequence based on a random strategy and uses the negative value of the length of the exploration sequence as the first l Generation reward for training round ; j ∈[1, U ].

[0114] It should be noted that:

[0115] 1. A deterministic strategy means that given a model input, the model produces the same output each time; the model selects the optimal action each time through a greedy strategy.

[0116] The benchmark sequence includes: a benchmark node connection sequence and a corresponding logarithmic probability; the length of the benchmark sequence can be calculated using an evaluation function.

[0117] 2. Random strategy means that each time the model executes, it randomly selects an action from a set of possible actions, and no longer selects the action with the highest probability.

[0118] The exploration sequence includes: the explored node connection sequence and the corresponding logarithmic probability; the length of the exploration sequence can be calculated using the evaluation function.

[0119] It should be noted that this algorithm pursues: the larger the reward, the better; in the RSMT problem, the line length should be as small as possible, so its negative value (i.e. the opposite number) is used as the reward.

[0120] S202, based on , Construct the l The loss function of the training round L oss l , and based on L oss l Back propagation to update the l -1 round of training to obtain the model parameters of the circuit wiring model l Circuit wiring model for round training.

[0121] in, ;

[0122] In the formula, Indicates l the advantages of round training (i.e., relative gains); Indicates l The first round of training used j Sample sets; express The corresponding correct RES ; Indicates based on generate probability.

[0123] In this embodiment 1, it is recommended to use the Adam optimizer to L oss l Update the model parameters. Then, after multiple rounds of training, the model parameters reach a stable convergence, and a trained circuit wiring model is obtained.

[0124] The SCST algorithm used in the present invention combines the advantages of the policy gradient method and the flexibility of the advantage function, and can achieve efficient optimization in complex scenarios without labeled data. Compared with the Actor-Critic algorithm, the SCST algorithm directly uses the deterministic output of the current policy as the benchmark reward without the need to train a separate Critic network, thus avoiding the problem of unstable training of the Critic network, reducing the number of parameters, and making the training process more direct.

[0125] Simulation Verification

[0126] This embodiment 1 conducts a simulation comparison between the above-mentioned method (abbreviated as Ours method) and the REST method to illustrate the effectiveness and advantages of Ours method:

[0127] The same test set was processed by Ours method and REST method, and the obtained RESThe corresponding average percentage error of line length (which characterizes the error between the output solution and the optimal solution, the smaller the value, the better) - the result comparison is shown in Table 1. Among them, the test set uses 5*i (i=1,2,…,10) two-dimensional test nodes, which are in the same plane area and have known two-dimensional coordinates.

[0128] Table 1 RES Error comparison

[0129]

[0130] As can be seen from Table 1, Ours method shows stable performance in the entire test range: as the number of two-dimensional test nodes increases, the error growth remains gentle, indicating that it is more robust in dealing with large-scale RSMT problems.

[0131] Moreover, Ours method has shown superiority over REST method when the number of 2D test nodes reaches 10: as the number of 2D test nodes increases, the increase in error percentage is lower, indicating that Ours method has significant improvements over REST method and is more suitable for large-scale point sets.

[0132] In addition, when the number of two-dimensional test nodes is 50, Ours method is used to obtain RES Compared with the optimal solution, the results are as follows Figure 6 As shown. Among them, Figure 6 The area (a) in the middle shows the optimal solution (Optimal) - its total length is 5.262; Figure 6 The middle (b) area shows the Ours method. RES ——The total length is 5.263; it can be seen that the Ours method obtains RES It is very close to the optimal solution, which also illustrates the effectiveness and superiority of the method proposed in Example 1.

[0133] Example 2

[0134] This embodiment 2 provides an integrated circuit wiring system based on deep reinforcement learning and RSMT, which uses the integrated circuit wiring method based on deep reinforcement learning and RSMT disclosed in embodiment 1.

[0135] The integrated circuit wiring system based on deep reinforcement learning and RSMT includes: a point set acquisition module and a circuit wiring module.

[0136] The point set acquisition module is used to obtain all M Coordinates of nodes to be processed Point 1~ Point M The circuit routing module is used to use the trained circuit routing model to Point 1~ PointM Processing to obtain circuit wiring results RES .

[0137] Since this system uses the integrated circuit wiring method based on deep reinforcement learning and RSMT in Example 1, it also has the same effect and will not be repeated here.

[0138] Example 3

[0139] This embodiment 3 discloses a computer device, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the integrated circuit wiring method based on deep reinforcement learning and RSMT disclosed in embodiment 1 are implemented.

[0140] This embodiment 3 also discloses a readable storage medium, which stores computer program instructions. When the computer program instructions are read and executed by a processor, the steps of the integrated circuit wiring method based on deep reinforcement learning and RSMT disclosed in embodiment 1 are executed.

[0141] This embodiment 3 also discloses a computer program product, including a computer program. When the computer program is executed by a processor, the steps of the integrated circuit wiring method based on deep reinforcement learning and RSMT disclosed in embodiment 1 are implemented.

[0142] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. An integrated circuit wiring method based on deep reinforcement learning and RSMT, characterized in that: include: Step 1: Get all the M Coordinates of nodes to be processed Point 1~ Point M ; Step 2: Use the trained circuit wiring model to Point 1~ Point M Processing to obtain circuit wiring results RES ; M >1; Among them, the trained circuit wiring model is obtained by training using a self-critical sequence training algorithm; The circuit wiring model includes: encoder Encoder, decoder Decoder; Encoder is used to Point 1~ Point M Encode to get M Encoded features Encoding 1~ Encoding M ;Decoder is used to Encoding 1~ Encoding M Decode and gradually generate node connection sequences to obtain RES ; Encoder includes: 1 embedding layer Embedder, 1 selective convolution layer SKConv, and 1 feature processing layer MT; Embedder is used to m Nodes to be processed Point m Extract preliminary features and generate m Embedding Embedding m SKConv is used to Embedding m Extract multi-scale features and perform dynamic fusion to obtain the m The fused multi-scale features MS-features m MT is used for MS-features m Perform global feature enhancement and obtain m Encoded features Encoding m ; m ∈[1, M ]; Among them, MT includes: N Macaron deformation layer mt1~mt N ; N ≥1;mt n The output of mt n+1 The input of mt1 is MS-features m ;mt N The output is Encoding m ; n ∈[1, N -1].

2. The integrated circuit wiring method based on deep reinforcement learning and RSMT according to claim 1, characterized in that: In Embedder, Point m Perform a one-dimensional convolution to obtain Embedding m .

3. The integrated circuit wiring method based on deep reinforcement learning and RSMT according to claim 1, characterized in that: In SKConv, on the one hand, K Branch convolutions of different scales Embedding m Perform convolution operation to obtain K On the other hand, the channel attention mechanism CAM is used to dynamically generate K The branch convolution corresponds to K convolution weights, and K The convolution results are weighted fused to obtain MS-features m .

4. The integrated circuit wiring method based on deep reinforcement learning and RSMT according to claim 1, characterized in that: mt1~mt N The structures of are the same, including: 2 feedforward network layers FFN1~FFN2, 1 attention network layer Attention; In any macaron deformation layer: FFN1 is used as the input for the macaron deformation layer IN Perform preliminary enhancement processing; Attention is used to weight the output of FFN1 through a multi-head self-attention mechanism to capture global dependencies; FFN2 is used to further enhance the output of Attention to obtain the output of the macaron deformation layer. OUT .

5. The integrated circuit wiring method based on deep reinforcement learning and RSMT according to claim 4, characterized in that: FFN1~FFN2 have the same structure, including: 1 feed-forward processing layer Feed-Forward, 1 residual connection layer Plus, and 1 batch normalization layer Batch-Norm; In any feed-forward network layer: Feed-Forward is used to input the feed-forward network layer in 1. Perform linear processing first, then scale it to half to get the feedforward feature ff 1; Plus is used to combine the output of Feed-Forward with ff 1. Perform superposition; Batch-Norm is used to batch normalize the output of the residual connection layer to obtain the output of the feedforward network layer. out 1.

6. The integrated circuit wiring method based on deep reinforcement learning and RSMT according to claim 5, characterized in that: Feed-Forward includes: 2 linear change layers Linear1~Linear2, 1 ReLU activation layer Relu, 1 random loss layer Dropout, and 1 scaling layer Scaling-Layer; Linear1 is used to in 1 Perform linear transformation to achieve dimensionality increase; Relu is used to activate the output of Linear1 using the ReLU activation function; Linear2 is used to perform linear transformation on the output of Relu to achieve dimensionality reduction; Dropout is used to randomly inactivate the output of Linear2; Scaling-Layer is used to scale the output of Dropout to half, ff 1.

7. The integrated circuit wiring method based on deep reinforcement learning and RSMT according to claim 4, characterized in that: Attention includes: 1 multi-head self-attention mechanism Multi-Head-Attention, 1 residual connection layer Plus, and 1 batch normalization layer Batch-Norm; In any attention network layer: Multi-Head-Attention is used as the input to the attention network layer in 2 Perform multi-head attention calculation; Plus is used to combine the output of Multi-Head-Attention with in 2. Overlay; Batch-Norm is used to batch normalize the output of Plus to obtain the output of the attention network layer out 2.

8. The integrated circuit wiring method based on deep reinforcement learning and RSMT according to claim 1, characterized in that: The training method of the circuit wiring model includes: S1, initializing the model hyperparameters and model parameters of the circuit wiring model, and randomly generating a number of two-dimensional node coordinates to form a training set; S2, circuit wiring model based on training set L In each round of training, the model loss is calculated to update the model parameters by gradient until a trained circuit wiring model is obtained; In S2, each round of training uses U sample sets, each sample set includes the same number of two-dimensional node coordinates; For l Round of training, first go through the l -1 round of training circuit wiring model for the l The first round of training used j The sample set generates a benchmark sequence based on a deterministic strategy and uses the negative value of the length of the benchmark sequence as the first l Baseline reward for a training round , and then through the l -1 round of training circuit wiring model for the l The first round of training used j The sample set generates an exploration sequence based on a random strategy and uses the negative value of the length of the exploration sequence as the first l Generation reward for training round ;based on , Construct the l The loss function of the training round L oss l , and based on L oss l Back propagation to update the l -1 round of training to obtain the model parameters of the circuit wiring model l Circuit wiring model for round training; j ∈[ 1, U ]; in, ; In the formula, Indicates l Advantages of wheel training; Indicates l The first round of training used j Sample sets; express The corresponding correct RES ; Indicates based on generate probability.

9. An integrated circuit wiring system based on deep reinforcement learning and RSMT, characterized in that: It uses the integrated circuit wiring method based on deep reinforcement learning and RSMT as described in any one of claims 1-8; The integrated circuit wiring system based on deep reinforcement learning and RSMT includes: Point set acquisition module, which is used to obtain all points in the target plane area. M Coordinates of nodes to be processed Point 1~ Point M ; as well as The circuit routing module is used to use the trained circuit routing model to Point 1~ Point M Processing to obtain circuit wiring results RES .

10. A computer program product, characterized in that It includes a computer program; when the computer program is executed by a processor, it implements the steps of the integrated circuit wiring method based on deep reinforcement learning and RSMT as described in any one of claims 1 to 8.

Images