Digital back-end detailed wiring method based on reinforcement learning

Abstract

The invention provides a digital back-end detailed wiring method based on reinforcement learning, and belongs to the technical field of digital integrated circuit back-end wiring. Comprising the following steps: constructing a reinforcement learning environment based on a multilayer metal layer wiring grid, and encoding a wiring area, a forbidden area, a starting point and an ending point into matrix data; defining an action space comprising up-down, left-right and cross-layer actions; designing a reward mechanism fusing physical constraint and time sequence optimization, and respectively punishing or rewarding entering a forbidden zone, deviating from a preferential wiring direction, generating interconnection line delay and successfully arriving at an end point; building a deep reinforcement learning network model to sense an environment state and outputting an optimal action; and through training, intelligent agent learning realizes end-to-end automatic wiring under complex constraints. According to the method, DRC / LVS rule constraints can be met, the interconnection line time sequence is optimized, winding attempts and manual intervention are reduced, the wiring efficiency and the chip performance are remarkably improved, and the method is suitable for high-complexity digital integrated circuit rear-end detailed wiring under advanced process nodes.

Description

Technical Field

[0001] This invention relates to the fields of digital integrated circuit back-end routing and reinforcement learning technology, and particularly to a digital back-end detailed routing method based on reinforcement learning. Background Technology

[0002] In the back-end design flow of digital integrated circuits, routing is a critical step, requiring two stages: global routing and detailed routing. The core task of the detailed routing stage is to achieve pin pair connectivity in a complex physical space, while simultaneously meeting manufacturing constraints such as DRC (Design Rule Check) and LVS (Layout and Schematic Compliance Check), and also taking into account timing performance indicators such as interconnect delay, which directly affects the normal operation and performance ceiling of the chip.

[0003] Traditional detailed routing relies on classic algorithms, but it faces significant drawbacks: routing is essentially a complex optimization problem with multiple constraints. Traditional algorithms need to go through multiple rounds of routing attempts to find a feasible path, which not only consumes a lot of time and hardware resources, but also easily leads to insufficient timing optimization. Moreover, as chip process nodes continue to advance, the number of metal layers increases and routing space becomes limited, further reducing the efficiency and adaptability of traditional methods. Manual intervention is often required to fix routing violations or optimize timing, which increases design costs and time.

[0004] The rise of reinforcement learning technology has provided new insights into decision optimization problems under complex constraints. By constructing a reasonable training environment, action space, and reward mechanism, reinforcement learning agents can learn optimal decision-making strategies during training, and the integration of deep learning can further enhance their ability to perceive and process complex environments. Therefore, applying reinforcement learning technology to detailed wiring in digital back-end systems holds promise for overcoming the efficiency and performance bottlenecks of traditional methods, enabling automated and high-precision wiring. Summary of the Invention

[0005] This invention aims to provide a detailed routing method for digital back-end systems based on reinforcement learning. By combining reinforcement learning and deep learning, the intelligent agent can autonomously learn routing strategies that meet physical constraints and timing optimization, reducing routing attempts and manual intervention, improving routing efficiency, quality and chip performance, and adapting to the high-complexity routing requirements under advanced process nodes.

[0006] To achieve the above objectives, the technical solution of the present invention is implemented as follows: A reinforcement learning-based detailed routing method for digital back-end systems, including S1. Establish a reinforcement learning training environment, construct a three-dimensional wiring grid based on the wiring tracks on each metal layer, encode the wired area, pin and via area as forbidden areas, and encode the wired area, start point and end point as state matrix; S2. Define the action space of the intelligent agent, which includes up, down, left, right, cross-layer, and descending actions; S3. Design a reward function, which includes a forbidden zone penalty, a wiring direction reward, an interconnection delay penalty, and a destination arrival reward, to guide the agent to optimize timing while satisfying the physical design rules; S4. Build a deep reinforcement learning network model to perceive the state matrix and output the optimal action corresponding to each state. S5. Train the deep reinforcement learning network model until the model converges to obtain the trained wired agent. S6. Call the trained wiring agent to automatically wire the pin pairs to be connected in the actual wiring environment.

[0007] Furthermore, the establishment of the reinforcement learning training environment includes: dividing the three-dimensional wiring mesh into basic units based on the minimum wiring track spacing allowed by the process; modeling different metal layers using differentiated mesh spacing; and encoding the wiring environment into a multi-dimensional matrix, wherein the wiring area is encoded as the first value, the restricted area is encoded as the second value, the starting point is encoded as the third value, and the ending point is encoded as the fourth value.

[0008] Furthermore, the encoding is a multi-dimensional matrix, specifically: for wiring tracks in the upper metal layer that do not actually exist due to the doubling of wiring track spacing, their corresponding positions are encoded as the second value in the multi-dimensional matrix.

[0009] Furthermore, the design reward function is specifically as follows: when the agent selects a restricted area or exceeds the environmental boundary as the next state, a first penalty reward is given; when the agent selects the preferred routing direction of the current metal layer, a first positive reward is given; when the agent selects a destination as the next state, a second positive reward is given; when the agent selects a routingable area as the next state, the delay is estimated based on the parasitic parameters per unit length of the interconnect, and a second penalty reward related to the delay is given; the total reward is calculated by weighted combination: reward = a1r1 + a2r2 + a3r3 + a4r4, where a1-a4 are preset weight coefficients, and r1-r4 correspond to the individual reward values ​​of the restricted area penalty item, the routing direction reward item, the destination arrival reward item, and the interconnect delay penalty item, respectively.

[0010] Furthermore, the preferred routing direction is set as follows: the conventional routing experience of prioritizing vertical routing for odd-numbered metal layers and horizontal routing for even-numbered metal layers, or vice versa, is taken into consideration as a reward mechanism and state space design.

[0011] Furthermore, the delay estimation based on parasitic parameters per unit length of interconnects specifically includes: obtaining the statistical values ​​of parasitic capacitance and parasitic resistance per unit length of the current metal layer. The parasitic capacitance and resistance per unit length are calculated using the 3σ method, i.e., C=C mean_j +C sigma_j R=R mean_j +R sigma_j C mean_j C sigma_j Let R be the mean and standard deviation of the parasitic capacitance per unit length on the j-th metal layer, respectively. mean_j R sigma_j The mean and standard deviation of the parasitic resistance per unit length on the metal layer of the j-th layer are respectively; the interconnect delay per unit length is calculated as the benchmark for the second penalty reward based on the Elmore delay model.

[0012] Furthermore, the deep reinforcement learning network model is a DoubleDQN network, which includes an online network and a target network with identical structures and periodically synchronized parameters; the online network is used to select actions and update network parameters, and the target network is used to calculate the target Q-value.

[0013] Furthermore, the DoubleDQN network includes: a convolutional feature extraction module for extracting features from the input state matrix and outputting a feature vector; a state feature fusion module for concatenating the feature vector with the current state coordinates, historical states, historical actions, and historical reward information; and a fully connected decision module for calculating the Q-value of each action based on the concatenated feature vector.

[0014] Furthermore, the convolutional feature extraction module includes at least two convolutional layers, each followed by a batch normalization layer and an activation function.

[0015] Furthermore, the automatic wiring process, which involves calling upon the trained wiring agent, specifically includes: The actual wiring environment is converted into the state matrix; Sort the multiple pin pairs to be connected according to the urgency of the wiring; The wiring agent is invoked sequentially to perform wiring for each pin pair. After each pair of pins is wired, the state matrix is ​​updated, the newly added wired path is encoded as a no-go zone, and then the next pair of pins is wired.

[0016] Beneficial effects: (1) Better wiring quality: Taking into account physical constraints (such as no-go zones, line width and spacing requirements), timing constraints (interconnect delay optimization) and space utilization constraints (priority wiring direction) in actual wiring, it effectively avoids DRC / LVS violations, reduces later repair costs, and improves the utilization rate of wiring resources. (2) Strong algorithm stability: The DoubleDQN algorithm is adopted, and the Q-value overestimation problem is avoided through the structure of online network and target network separation; a special reward mechanism is designed for wiring scenarios to ensure that the agent's learning direction is accurate. (3) Wide range of applications: The network structure supports multi-layer metal wiring (4 layers in the example), can handle wiring meshes of 100×100 scale, and can be extended to larger scale meshes to adapt to the wiring needs of digital integrated circuits of different complexities. (4) Significant efficiency and resource advantages: After training, the model weight parameters are retained. When actually wiring, the weight parameters are directly called. The optimal wiring decision can be quickly output by reading environmental data without multiple rounds of wire routing attempts, which greatly saves computing time and hardware resource consumption. Attached Figure Description

[0017] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a flowchart of the main body of the reinforcement learning-based digital back-end detailed routing method according to an embodiment of the present invention; Figure 2 This is the real-world environment for the back-end wiring of the digital integrated circuit described in the embodiments of the present invention.

[0018] Figure 3 This refers to the model input data matrix described in this embodiment of the invention.

[0019] Figure 4 This is the DDQN network structure in an embodiment of the present invention.

[0020] Figure 5 This is an iterative graph showing the success rate of wiring in training the reinforcement learning model in an embodiment of the present invention. Detailed Implementation

[0021] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0022] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0023] Example 1 See Figures 1-5 A reinforcement learning-based detailed routing method for digital back-end systems, including... S1. Establish a reinforcement learning training environment, construct a three-dimensional wiring grid based on the wiring tracks on each metal layer, encode the wired area, pin and via area as forbidden areas, and encode the wired area, start point and end point as state matrix; It should be noted that in this embodiment, the areas occupied by power lines, clock lines, signal lines, pins (ports), and vias in this 3D wiring mesh environment can be considered as forbidden areas in reinforcement learning. To prevent rule violations such as DRC and LVS, the line width and minimum spacing between lines are considered, i.e., the spacing between grid lines when dividing the mesh.

[0024] S2. Define the action space of the intelligent agent, which includes up, down, left, right, cross-layer, and descending actions; S3. Design a reward function, which includes a forbidden zone penalty, a wiring direction reward, an interconnection delay penalty, and a destination arrival reward, to guide the agent to optimize timing while satisfying the physical design rules; It should be noted that this embodiment sets up a reasonable reward mechanism, penalizing actions that enter forbidden areas or exceed boundaries. Simulating real-world routing experience, it prioritizes horizontal or vertical routing on corresponding metal layers to effectively utilize routing space resources. A reward is given when an action aligns with the preferred routing direction of its metal layer. Considering the electrical effects of interconnects, interconnect delay is estimated based on statistical parasitic capacitance and resistance per unit length, serving as one of the reference standards for routing quality. For each grid traversed, the corresponding delay loss is calculated, and points are deducted as a penalty according to certain standards. Finally, a reasonable reward score is given upon reaching the destination.

[0025] S4. Build a deep reinforcement learning network model to perceive the state matrix and output the optimal action corresponding to each state. S5. Train the deep reinforcement learning network model until the model converges to obtain the trained wired agent. S6. Call the trained wiring agent to automatically wire the pin pairs to be connected in the actual wiring environment.

[0026] This embodiment uses deep learning methods to train a reinforcement learning agent, setting a reasonable reward mechanism and action space, enabling it to achieve optimal end-to-end detailed wiring in complex environments.

[0027] In a specific example, establishing the reinforcement learning training environment includes: dividing the three-dimensional wiring mesh using the minimum wiring track spacing allowed by the process as the basic unit; modeling different metal layers using differentiated mesh spacing; and encoding the wiring environment into a multi-dimensional matrix, wherein the wiring area is encoded as the first value, the restricted area is encoded as the second value, the starting point is encoded as the third value, and the ending point is encoded as the fourth value.

[0028] In the specific implementation, multiple metal layers are used, and the mesh is constructed based on track routing with the minimum track spacing as the basic unit. Considering the environment in real routing, the mesh spacing of metal layers M1 and M2 is half that of metal layers M3 and M4. When the track spacing in the upper metal layer increases by a factor of 2, the track of the upper metal layer is still set according to the spacing of the lowest layer, but the track that disappears after the proportional increase is set as a forbidden area in the mesh. In order to make effective use of routing space resources and reduce inter-line interference, the routing strategy of odd horizontal and even vertical or odd vertical and even horizontal is usually adopted. The areas occupied by power lines, clock lines, signal lines and pins and vias in this three-dimensional environment are regarded as forbidden areas in reinforcement learning. The data is converted into matrix data and used as input for reinforcement learning. The wireable area is encoded as 0, the forbidden area is encoded as 1, the single-ended start point is encoded as 2, and the single-ended end point is encoded as 3.

[0029] In a specific example, the encoding is a multidimensional matrix, specifically: for wiring tracks in the upper metal layer that do not actually exist due to the doubling of wiring track spacing, their corresponding positions are encoded as the second value in the multidimensional matrix.

[0030] This embodiment solves the "fake mesh" problem of the upper metal layer by supplementing the network coding rules: the line spacing of the upper metal layer is wider than that of the lower layer (for example, 1 micrometer for the lower layer and 2 micrometers for the upper layer). In this way, some "mesh positions" that should exist in the upper layer do not actually exist. These non-existent positions are directly marked as "no lines allowed" (with the same code as the restricted area) to prevent the agent from going to these places.

[0031] In a specific example, the design reward function is as follows: when the agent selects a restricted area or exceeds the environmental boundary as the next state, a first penalty reward is given; when the agent selects the preferred routing direction of the current metal layer, a first positive reward is given; when the agent selects a destination as the next state, a second positive reward is given; when the agent selects a routable area as the next state, the delay is estimated based on the parasitic parameters per unit length of the interconnect, and a second penalty reward related to the delay is given; the total reward is calculated by weighted combination: reward = a1r1 + a2r2 + a3r3 + a4r4, where a1-a4 are preset weight coefficients, and r1-r4 correspond to the individual reward values ​​of the restricted area penalty item, the routing direction reward item, the destination arrival reward item, and the interconnect delay penalty item, respectively.

[0032] In a practical implementation, the corresponding reward mechanism can be set as follows: Entry into the forbidden area is prohibited. When entering state2 from state1, if state2 is a forbidden area, the reward of r1 is given and state2 is set to state1. That is, if you hit the forbidden area, you will bounce back to the original position. If a priority routing action occurs in the corresponding layer, an additional reward of r2 will be given; If the next state reaches the target coordinates (end point), then the reward of r3 is given; If an action is taken from the current state to enter the next state, and the corresponding state is a routable area, then the unit delay of the interconnect is estimated based on the electrical parameters on the digital back-end metal layer. Considering the electrical effects of the interconnect, the interconnect delay is estimated according to the statistical parasitic capacitance and statistical parasitic resistance per unit length, which serves as one of the reference standards for the quality of routing.

[0033] The final total reward is calculated as follows: reward = a1r1 + a2r2 + a3r3 + a4r4.

[0034] In a specific example, the preferred routing direction is set as follows: the conventional routing experience of prioritizing vertical routing for odd-numbered metal layers and horizontal routing for even-numbered metal layers, or vice versa, is taken into consideration as a reward mechanism and state space design.

[0035] It should be noted that, in order to train a reasonable reinforcement learning body to achieve optimal routing between single ends, constraints are imposed based on reality: to prevent short circuits in interconnects, routing is prohibited in areas that have been routed or have vias or pins; to simulate real routing experience, odd-numbered layers are prioritized for vertical routing and even-numbered layers are prioritized for horizontal routing, or vice versa, to effectively utilize routing space resources.

[0036] In a specific example, the delay estimation based on parasitic parameters per unit length of interconnects specifically includes: obtaining the statistical values ​​of parasitic capacitance and parasitic resistance per unit length of the current metal layer. The parasitic capacitance and resistance per unit length are calculated using the 3σ method, i.e., C=C mean_j +C sigma_j R=R mean_j +R sigma_j C mean_j C sigma_j Let R be the mean and standard deviation of the parasitic capacitance per unit length on the j-th metal layer, respectively. mean_j R sigma_j The mean and standard deviation of the parasitic resistance per unit length on the metal layer of the j-th layer are respectively; the interconnect delay per unit length is calculated as the benchmark for the second penalty reward based on the Elmore delay model.

[0037] It should be noted that estimating interconnect delay based on statistical parasitic capacitance and statistical parasitic resistance per unit length can serve as a reference standard for evaluating the quality of wiring. Specifically: Using the physical parameters provided by the simulation software and process library, the parasitic capacitance and parasitic resistance per unit length on each metal layer are statistically analyzed. If the mean and standard deviation of the parasitic capacitance per unit length on the j-th metal layer are C... mean_j and C sigma_j The mean and standard deviation of the parasitic resistance per unit length are R1 and R2, respectively. mean_j and R sigma_j The 3σ calculation method is used. The calculation yielded: C=C mean_j +C sigma_j R=R mean_j +R sigma_j Then, the delay per unit length of interconnect is estimated using the Elmore model for calculating interconnect delay, as follows:

[0038] Where L is the unit interconnect length, τ DN That is, r4.

[0039] In a specific example, the deep reinforcement learning network model is a DoubleDQN network, comprising an online network and a target network with identical structures and periodically synchronized parameters. The online network is used to select actions and update network parameters, and the target network is used to calculate the target Q-value. The DoubleDQN network includes: a convolutional feature extraction module for extracting features from the input state matrix and outputting a feature vector; a state feature fusion module for concatenating the feature vector with current state coordinates, historical states, historical actions, and historical reward information; and a fully connected decision module for calculating the Q-value of each action based on the concatenated feature vector. The convolutional feature extraction module includes at least two convolutional layers, each followed by a batch normalization layer and an activation function.

[0040] In the specific implementation, a convolutional neural network is used to convolve the encoding matrix composed of four metal layers. Through two convolutions, each with a 3×3 kernel, the result of the last convolution layer is flattened into a one-dimensional vector, and the current state coordinates, the previous state coordinates, the action taken in the previous step, and the reward obtained after taking that action are added. This vector is then output through a fully connected layer to obtain the action that should be taken in the current state. This embodiment uses the DDQN (double deep q-learning) method, with two identical network structures: one network updates the action, and the other updates the target. Under the action of these two networks, the model gradually converges.

[0041] In a specific instance, invoking the trained wiring agent to perform automatic wiring includes: The actual wiring environment is converted into the state matrix; Sort the multiple pin pairs to be connected according to the urgency of the wiring; The wiring agent is invoked sequentially to perform wiring for each pin pair. After each pair of pins is wired, the state matrix is ​​updated, the newly added wired path is encoded as a no-go zone, and then the next pair of pins is wired.

[0042] In the specific implementation, to achieve the goal of using reinforcement learning methods for detailed single-ended routing of digital back-ends, a 4×100×100 grid is selected from a real routing environment to make it suitable as input for the DoubleDQN model for training.

[0043] Therefore, when a pair of pins needs to be connected in a real wiring environment, a trained reinforcement learning agent can analyze the wiring environment to obtain the optimal path from one pin to another. This embodiment provides a single-ended detailed routing method for a digital backend based on reinforcement learning, which involves two steps. The invention will be described in detail through the following embodiments.

[0044] During the model training process, a batch of data will be obtained, such as... Figure 2 The 4×100×100 real-world routing environment shown is encoded as 0, 1, 2, and 3 according to unrouted areas, routed areas, start pin, and end pin, and converted into a data format for reinforcement learning input, such as... Figure 3 As shown.

[0045] Constraints are implemented based on real-world conditions to prevent short circuits in interconnects. Routing is prohibited in areas where wiring has already occurred or where vias or pins exist; entering a forbidden area incurs a penalty of -2 points. Simulating real-world wiring experience, odd-numbered layers are prioritized for vertical routing, and even-numbered layers for horizontal routing, effectively utilizing wiring space resources. Failure to adhere to the corresponding metal layer's priority routing direction incurs a penalty of -2 points. The electrical effects of interconnects are considered, and interconnect delay is estimated based on statistical parasitic capacitance and resistance per unit length, serving as one of the reference standards for wiring quality. Each step deducts a corresponding delay score as a penalty. Reaching the target endpoint awards a +10 score as a reward. Following this reward mechanism, the reinforcement learning agent is guided to explore the environment, gradually converging to make reasonable action choices in the corresponding states through training to perceive environmental changes.

[0046] Model structure as follows Figure 4 As shown, through two convolutions, each with a 3×3 kernel, the result of the last convolution layer is flattened into a one-dimensional vector. The current state coordinates, the previous state coordinates, the action taken in the previous step, and the reward obtained after taking that action are added. This vector is then output through a fully connected layer to obtain the action that should be taken in the current state. This embodiment uses the DDQN (double deep q-learning) method, where two networks have the same structure: one network updates the action, and the other updates the target. Under the action of these two networks, the model gradually converges.

[0047] The epsilon-greedy algorithm was used during training, with the following hyperparameter settings: capacity = 10000 experiences, batch size = 32 samples, learning rate = 3e-4, discount rate gamma = 0.99, and the exploration rate epsilon decreasing from 1 to 0 at a rate of 0.998. During training, the weights of the target net were updated every 20 steps. The Adam optimizer was used, and the loss function was MSELoss. After training for 1000 episodes, the wiring success rate on the test set gradually increased to 100%. Figure 5 As shown.

[0048] After the reinforcement learning agent has been trained, the model is invoked to implement single-end to single-end routing in a real detailed routing environment. The specific method includes the following steps: Convert the wiring environment into the data format read into the model; The multiple pairs of pins that need to be connected in the area are sorted in order of increasing slack value obtained from the timing analysis before routing, and the routing is carried out accordingly. After routing the connection between the i-th pair of pins, update the network environment by changing the area code value occupied by the i-th pair of routing to 1. The subsequent interconnections between the (i+1)-th pair of pins will continue to be routed according to the updated mesh.

[0049] In summary, this embodiment has the following advantages: Cabling quality optimization takes into account various physical constraints in actual cabling, improves cabling resource utilization, effectively avoids DRC / LVS violations, and reduces subsequent repair costs.

[0050] The DoubleDQN algorithm is adopted to avoid the problem of Q-value overestimation, and a reward mechanism specifically designed for wiring problems is designed.

[0051] The network structure is suitable for routing problems of different sizes, supports multi-layer metal routing (4-layer example), can handle 100×100 size routing meshes, and can be extended to larger size meshes for routing.

[0052] After training, the model's weight parameters are retained. When routing, the weight parameters are called and the routing environment is read to make the optimal routing decision, saving computation time and hardware resource consumption.

[0053] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A detailed routing method for a digital back-end based on reinforcement learning, characterized in that, include S1. Establish a reinforcement learning training environment, construct a three-dimensional wiring grid based on the wiring tracks on each metal layer, encode the wired area, pin and via area as forbidden areas, and encode the wired area, start point and end point as state matrix; S2. Define the action space of the intelligent agent, which includes up, down, left, right, cross-layer, and descending actions; S3. Design a reward function, which includes a forbidden zone penalty, a wiring direction reward, an interconnection delay penalty, and a destination arrival reward, to guide the agent to optimize timing while satisfying the physical design rules; S4. Build a deep reinforcement learning network model to perceive the state matrix and output the optimal action corresponding to each state. S5. Train the deep reinforcement learning network model until the model converges to obtain the trained wired agent. S6. Call the trained wiring agent to automatically wire the pin pairs to be connected in the actual wiring environment.

2. The method according to claim 1, characterized in that, The establishment of the reinforcement learning training environment includes: dividing the three-dimensional wiring mesh into basic units based on the minimum wiring track spacing allowed by the process; modeling different metal layers using differentiated mesh spacing; and encoding the wiring environment into a multi-dimensional matrix, wherein the wiring area is encoded as the first value, the restricted area is encoded as the second value, the starting point is encoded as the third value, and the ending point is encoded as the fourth value.

3. The method according to claim 2, characterized in that, The encoding is a multidimensional matrix, specifically: for wiring tracks in the upper metal layer that do not actually exist due to the doubling of wiring track spacing, their corresponding positions are encoded as the second value in the multidimensional matrix.

4. The method according to claim 1, characterized in that, The design reward function is as follows: when the agent selects a restricted area or exceeds the environmental boundary as the next state, a first penalty reward is given; when the agent selects the preferred routing direction of the current metal layer, a first positive reward is given; when the agent selects a destination as the next state, a second positive reward is given; when the agent selects a routingable area as the next state, the delay is estimated based on the parasitic parameters per unit length of the interconnect, and a second penalty reward related to the delay is given; the total reward is calculated by weighted combination: reward = a1r1 + a2r2 + a3r3 + a4r4, where a1-a4 are preset weight coefficients, and r1-r4 correspond to the individual reward values ​​of the restricted area penalty item, the routing direction reward item, the destination arrival reward item, and the interconnect delay penalty item, respectively.

5. The method according to claim 4, characterized in that, The preferred routing direction is set as follows: the conventional routing experience of prioritizing vertical routing for odd-numbered metal layers and horizontal routing for even-numbered metal layers, or vice versa, is used as a reward mechanism and state space design consideration.

6. The method according to claim 4, characterized in that, The delay estimation based on parasitic parameters per unit length of interconnects specifically includes: obtaining the statistical values ​​of parasitic capacitance and parasitic resistance per unit length of the current metal layer. The parasitic capacitance and resistance per unit length are calculated using the 3σ method, i.e., C=C0. mean_j +C sigma_j R=R mean_j +R sigma_j C mean_j C sigma_j Let R be the mean and standard deviation of the parasitic capacitance per unit length on the j-th metal layer, respectively. mean_j R sigma_j The mean and standard deviation of the parasitic resistance per unit length on the metal layer of the j-th layer are respectively; the interconnect delay per unit length is calculated as the benchmark for the second penalty reward based on the Elmore delay model.

7. The method according to claim 1, characterized in that, The deep reinforcement learning network model is a DoubleDQN network, which includes an online network and a target network with identical structures and periodically synchronized parameters; the online network is used to select actions and update network parameters, and the target network is used to calculate the target Q-value.

8. The method according to claim 7, characterized in that, The DoubleDQN network includes: a convolutional feature extraction module for extracting features from the input state matrix and outputting a feature vector; a state feature fusion module for concatenating the feature vector with the current state coordinates, historical states, historical actions, and historical reward information; and a fully connected decision module for calculating the Q-value of each action based on the concatenated feature vector.

9. The method according to claim 8, characterized in that, The convolutional feature extraction module includes at least two convolutional layers, each followed by a batch normalization layer and an activation function.

10. The method according to claim 1, characterized in that, The process of calling upon the trained wiring agent to perform automatic wiring specifically includes: The actual wiring environment is converted into the state matrix; Sort the multiple pin pairs to be connected according to the urgency of the wiring; The wiring agent is invoked sequentially to perform wiring for each pin pair. After each pair of pins is wired, the state matrix is ​​updated, the newly added wired path is encoded as a no-go zone, and then the next pair of pins is wired.

Images