A reinforcement learning-based analog IC layout wiring sequence optimization and wiring method

Abstract

The present invention discloses a reinforcement learning-based routing sequence optimization and routing method for analog IC layouts. The method comprises: obtaining routing data, including routing areas, obstacle areas, a set of wire nets, and the starting and ending points of each wire net; gridding the routing space and constructing a multi-channel image based on the routing data; using the multi-channel image as a state, solving the analog IC wire net sorting problem based on a Markov decision process, including: constructing a wire sequence selection model that takes the current multi-channel image as input and outputs a probability vector and a predicted value to complete the wire net sorting; based on the sorting results of all wire nets, combining the routing areas, obstacle areas, and the starting and ending points of each wire net, routing each wire net in sequence to obtain a routing result; calculating a performance indicator of the routing result as a reward, and constructing a total loss function based on the predicted value to optimize the wire sequence selection model. The present invention effectively improves the efficiency of analog IC routing and improves the performance of the layout.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic design automation, and in particular to an analog IC layout wiring sequence optimization and wiring method based on reinforcement learning. Background Art

[0002] The integrated circuit (IC) industry is currently experiencing rapid growth. With the continuous advancement of technology, market demands for higher performance from ICs are increasing, and circuit complexity and scale are also increasing. Against this backdrop, design assistance using electronic design automation (EDA) technology is becoming increasingly important. Digital signals have well-defined high and low levels, making them easy to process and analyze. After years of development, digital EDA tools have become powerful and reliable in logic synthesis, place and route, and their development has reached a relatively complete stage. However, analog ICs are highly sensitive to noise and interference. This difference in performance leads to more constraints during the routing process of analog IC layouts, thus creating certain challenges in the design of automated routing technologies for analog ICs.

[0003] Current automated routing technologies primarily utilize analytical and heuristic algorithms. A key advantage of analytical routing is its ability to precisely determine the optimal routing path using mathematical formulas, thereby ensuring a certain level of routing accuracy and reliability. However, this method's high computational complexity limits its application to complex circuit structures, and it may be impossible to find a viable solution for large-scale problems. In such cases, heuristic routing becomes a more practical option. Similarly, in heuristic routing, when the problem size reaches a certain level, different routing sequences will affect the overall routing results.

[0004] Artificial intelligence (AI), a powerful technology, is playing an increasingly important role in our daily lives. By analyzing and learning from large amounts of data, AI can automatically extract features and discover patterns, enabling efficient decision-making and prediction. Due to these outstanding features, machine learning is gradually gaining prominence and application in the field of IC physical design. AI technology can assist in optimizing chip layout and routing, improving design efficiency and performance, and bringing new ideas and methods to IC physical design. Summary of the Invention

[0005] The purpose of the present invention is to provide an analog IC layout wiring sequence optimization and wiring method based on reinforcement learning, a line sequence selection method based on reinforcement learning, and a bidirectional Routing algorithms effectively improve the efficiency of analog IC routing and improve the performance results of the layout.

[0006] The technical solution adopted in the present invention is:

[0007] The first aspect of the present invention provides a method for optimizing and routing analog IC layout routing sequences based on reinforcement learning, the method comprising:

[0008] Obtain wiring data, including wiring areas, obstacle areas, wire net collections, and the start and end points of each wire net;

[0009] Gridding the wiring space and constructing a multi-channel image based on the wiring data;

[0010] Using multi-channel images as states, we solve the analog IC net sorting problem based on a Markov decision process, including:

[0011] Construct a line sequence selection model. The line sequence selection model takes the current multi-channel image as input and outputs a probability vector and a predicted value for each action decision. The probability vector is a vector composed of the probabilities of each line network being selected.

[0012] Each time the line sequence selection model makes an action decision, it selects the line net with the highest probability in the probability vector for line sorting and simultaneously updates the multi-channel image to complete the sorting of all line nets. The predicted value refers to the value of the current line net sorting result after selecting the line net with the highest probability for sorting.

[0013] According to the sorting results of all the nets, combined with the routing area, obstacle area and the starting and ending points of each net, each net is routed in turn to obtain the routing results of the simulated IC layout;

[0014] The performance indicators of the simulated IC layout routing results are calculated as rewards, and the total loss function of the wire sequence selection model is constructed in combination with the predicted value to optimize the wire sequence selection model; wherein the total loss function includes the loss function of the policy network and the loss function of the value network.

[0015] In some embodiments, gridding the wiring space and constructing a multi-channel image based on the wiring data includes:

[0016] Grid the wiring space of the analog IC, and the number of grids in the length and width directions corresponds to the length and width of the multi-channel image;

[0017] If the analog IC uses a double-layer wiring method, that is, the horizontal and vertical directions are wired on different metal layers and connected through through holes, then the number of channels in the multi-channel image is twice the number of wire nets plus two. The different metal layers of each wire net correspond to two channels, and the last two channels represent the wiring area and the obstacle area.

[0018] If the analog IC uses a single-layer wiring method, the number of channels in the multi-channel image is the number of wire nets plus one. The metal layer of each wire net corresponds to a channel, and the last channel represents the wiring area and obstacle area.

[0019] In some embodiments, gridding the wiring space and constructing a multi-channel image based on the wiring data further includes:

[0020] The grid cells are marked with a binary method, where the available routing grid area is marked as 0, and the unroutable grid area that has been routed or occupied by other obstacles is marked as 1;

[0021] The grid area where each mesh starts and ends is marked as 0. If a mesh is selected for mesh sorting, the grid area where the start and end points of that mesh are located is marked as 1, and the multi-channel image is updated. Therefore, the multi-channel image contains information about the currently sorted mesh and the remaining unsorted meshes.

[0022] In some embodiments, building a line sequence selection model includes:

[0023] (1) The feature map of the multi-channel image is processed through the attention mechanism to adaptively adjust the weight of each channel, including:

[0024] ① Feature map of multi-channel image x Perform global average pooling and global maximum pooling operations to generate two different channel descriptors:

[0025]

[0026] in, represents the feature map of global average pooling, Represents the feature map of the global maximum pooling, represents global average pooling, represents global maximum pooling;

[0027] ②The feature map of global average pooling and the feature map of global maximum pooling are passed through a shared fully connected network; the network first reduces the number of channels to , then restore the number of channels to C , and finally use the Sigmoid function to generate channel attention weights:

[0028]

[0029] in, C is the original number of channels of the feature map of the multi-channel image, r is the ratio of reduction, FC represents the fully connected network, s is the Sigmoid activation function, is the average attention weight, is the maximum attention weight;

[0030] ③Calculate attention weight A :

[0031]

[0032] ④The attention weight A Feature maps with multi-channel images x Multiply by channel to get the weighted feature map output:

[0033]

[0034] in, is the weighted feature map;

[0035] (2) The weighted feature map Feature maps with multi-channel images x Splicing to obtain a spliced ​​feature map;

[0036] (3) Construct a neural network that takes the concatenated feature map as input and outputs a probability vector and a predicted value.

[0037] In some embodiments, building the line sequence selection model further includes:

[0038] Process multi-channel images and scale them into images with a specific length, width, and number of channels.

[0039] In some embodiments, the neural network is a residual neural network.

[0040] In some embodiments, for each net, a unidirectional Algorithmic or Bidirectional The algorithm routes the net and marks the routed grid area as 1 after routing is completed;

[0041] If the analog IC uses a double-layer wiring method, then

[0042] unidirectional Algorithmic or Bidirectional When searching for a node, the algorithm determines whether the neighboring nodes in the six search directions (up, down, left, right, front, and back) are available routing grid areas. The algorithm then selects the optimal moving direction based on a cost function.

[0043] When a path transitions between layers at the same planar location on different metal layers, that location is a via location.

[0044] In some embodiments, a performance indicator of a simulated IC layout and routing result is calculated as a reward, including:

[0045] Calculate various performance indicators of analog IC layout and routing results, including:

[0046] (1) Line length length , that is, the total length of the wiring path;

[0047] (2) Number of through holes , that is, the total number of through holes in the wiring path when double-layer wiring;

[0048] (3) Coupling noise ;

[0049] Based on the performance indicators of the simulated IC layout and routing results, the relative rewards are calculated respectively:

[0050]

[0051] in, Indicates the baseline value; Indicates specific variable values, corresponding to various performance indicators of the simulated IC layout and wiring results; Indicates the relative reward value of the corresponding performance indicator, including the relative reward value of line length , Relative reward value of the number of through holes and coupling noise relative reward value ;

[0052] The total reward value is:

[0053]

[0054] in, is the total reward value, a、b、c is the weight coefficient;

[0055] If the analog IC uses a single-layer wiring method, the total reward value does not include the relative reward value of the number of through holes.

[0056] In some embodiments, constructing a total loss function of a line sequence selection model includes:

[0057] The discounted cumulative reward calculation method is used to distribute the reward to the state of each time step as follows:

[0058]

[0059] Where, t is the time step, the value range is [0, N], N is the total number of all network; γ is the discount factor; Rewards for simulating IC layout and routing results; Indicates that at time stept Discount accumulated rewards when

[0060] Based on the discounted cumulative reward, the loss function of the policy network and the value network is calculated; for the policy network, the gradient-based clipping policy loss function is used as follows:

[0061]

[0062]

[0063]

[0064] in, i represents the weight parameters of the policy network, represents the loss function of the policy network, Represents the time step t is the ratio of the probability that the old model selects a certain network for network sorting in a certain state to the probability that the new model selects the same network for network sorting in the same state; Represents the time step t The predictive value of Indicates the difference; is a hyperparameter, Indicates that Restricted to [ , ] range, Indicates the limit value;

[0065] For the value network, the mean square error is used as the loss function, as shown below:

[0066]

[0067] Where Φ is the weight parameter of the value network, represents the loss function of the value network;

[0068] Total loss function As shown below:

[0069]

[0070] in, It is a regular term used to prevent parameter overfitting. Represents the weight matrix of the model.

[0071] According to a second aspect of the present invention, an analog IC is provided, which is wired using the analog IC layout wiring sequence optimization and wiring method based on reinforcement learning described in any one of the first aspects.

[0072] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0073] (1) Optimize the wiring sequence. The analog IC line network sorting problem is modeled as a Markov process. Using a reinforcement learning framework with an attention mechanism, the model automatically learns key channel information, enhances feature expression, and suppresses interference from irrelevant or redundant information, providing a more accurate basis for wiring decisions. This solves the problem of non-optimal wiring sequence specified by humans, thereby improving the rationality and effectiveness of the wiring sequence.

[0074] (2) Improve the quality of routing results. Optimize routing results by setting rewards based on line length, number of vias, and coupling noise. Reducing line length can reduce parasitic capacitance, inductance, resistance, and power consumption, improving electrical performance and energy efficiency. Reducing the number of vias can reduce the negative impact of parasitic parameters on high-frequency performance, signal transmission quality, power consumption, stability, manufacturability, and reliability. Reducing coupling noise can reduce the impact of electrical or magnetic interactions between adjacent wires. Through these optimizations, the quality of routing results is significantly improved, and layout performance is improved.

[0075] (3) Improve wiring efficiency. Use bidirectional The routing algorithm searches for paths from both the starting point and the end point of a double-ended network. Under reasonable cost function settings, the search process is accelerated. Ideally, the routing time can be shortened to one-way routing. The first half of the algorithm greatly improves the routing efficiency. At the same time, a higher routing layer change cost is added to the heuristic function, which reduces the number of through holes and further optimizes the routing effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 This is an overall framework diagram of an analog IC layout wiring sequence optimization and wiring method based on reinforcement learning provided in an embodiment of the present application;

[0077] Figure 2 A two-way Flowchart of the algorithm. DETAILED DESCRIPTION

[0078] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments provided in this application, all other embodiments obtained by those of ordinary skill in the art without making creative work are within the scope of protection of the present invention.

[0079] Obviously, the drawings described below are merely examples or embodiments of the present application. Those skilled in the art can, without inventive effort, apply the present application to other similar scenarios based on these drawings. Furthermore, it is also understood that, although the effort involved in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, changes in design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as an insufficiency of the content disclosed in this application.

[0080] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments unless there is a conflict.

[0081] Unless otherwise defined, the technical or scientific terms used in this application should have the ordinary meaning understood by a person of ordinary skill in the technical field to which this application belongs. The words "one", "a", "the" and the like used in this application do not indicate a limit on quantity and may indicate the singular or plural. The terms "include", "comprise", "have" and any variations thereof used in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units that are inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The word "multiple" used in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0082] Currently, automated routing for analog ICs presents significant challenges. Analog ICs are extremely sensitive to noise and interference, and the integrity and accuracy of their internal signals are easily affected by external factors. During the routing process, not only must correct electrical connections between components be ensured, but noise levels and electromagnetic interference during signal transmission must also be strictly controlled. These unique requirements make automated routing for analog ICs more challenging during design and implementation.

[0083] Commonly used automatic routing algorithms are mainly divided into analytical algorithms and heuristic algorithms. Analytical routing algorithms rely on complex mathematical models and formulas to calculate the optimal routing path. When dealing with complex circuit structures, as the circuit scale increases, the factors that need to be considered increase, resulting in a sharp increase in computational complexity. In this case, the application of analytical algorithms in complex circuit structures will be limited, and feasible solutions may not be found when dealing with large-scale problems. When heuristic routing algorithms deal with routing problems, when the problem scale reaches a certain level, the routing order becomes a key factor affecting the overall routing results. Currently, the routing order is mostly manually specified, but this manually specified approach often lacks comprehensive optimization considerations. Different routing orders can lead to completely different routing results and also have a certain impact on the performance of the circuit.

[0084] According to the characteristics of analog IC wiring, this application provides an analog IC layout wiring sequence optimization and bidirectional The algorithmic wiring method improves the efficiency of analog IC wiring and improves the performance of the layout. The method includes the following steps: 1) obtaining wiring data and performing corresponding processing; 2) constructing a multi-channel image of the wire network information as the environment state; 3) the intelligent agent performs actions, explores and learns from experience in the environment, and records the corresponding actions and states; 4) if the training indicators are met, the existing data is input into the neural network to start the training process; 5) after completing a round of wire sequence arrangement, a bidirectional method is used according to the wiring order at this time. This application addresses the issue of manually prescribing wiring sequences that are not always optimal by modeling the analog IC wire net sorting problem as a Markov process and utilizing a specific reinforcement learning framework, enabling the agent to autonomously learn the wiring sequence. In the reinforcement learning reward setting, routing results are optimized based on multiple factors, including wire length, number of vias, and coupling noise. A relative value reward approach is used to address the issue of large differences in the order of magnitude of reward indicators, thereby obtaining a more reasonable basis for routing decisions.

[0085] The present invention provides an analog IC layout wiring sequence optimization and bidirectional The wiring method of the algorithm includes the following steps:

[0086] 1) Obtain routing data and perform corresponding processing. The routing data includes the routing area and obstacle area (such as the area occupied by the device) obtained from the layout results, the device connection relationship netlist and the corresponding pin coordinates, and split all wire nets into two-terminal wire nets;

[0087] 2) Construct a multi-channel image and its feature map based on the line network information, process the feature map through the attention mechanism, and then input it into the neural network;

[0088] 3) The agent (i.e., the line sequence selection model) performs actions, explores, and learns from experience in the environment, and records the corresponding actions and states;

[0089] 4) If the training indicators are met, the existing data is input into the neural network to start the training process;

[0090] 5) After completing a round of line sequence arrangement, use bidirectional wiring according to the wiring sequence at this time Algorithm for path planning.

[0091] Based on the above, this application models the analog IC line network sorting problem as a Markov decision process and uses a reinforcement learning framework with an attention mechanism to process it. The reinforcement learning framework splits all circuit lines into two-pin nets. The feature map is processed by the attention mechanism, and the multi-channel image is processed by a residual neural network. The attention mechanism adaptively adjusts the weights of each channel in the feature map to highlight important channel information, thereby enhancing the representation ability of the model. In the reinforcement learning reward setting, the wiring results are optimized from three aspects: line length, number of through-holes, and coupling noise, and rewards are set for the intelligent agent in the form of relative value rewards.

[0092] With the help of reinforcement learning, the intelligent agent can autonomously learn the wiring sequence, effectively solving the problem of non-optimal wiring sequence specified by humans, improving wiring efficiency and layout performance. Its reward setting comprehensively optimizes the wiring results based on multiple factors, and is bidirectional. The algorithm accelerates the search process and provides new ideas for analog IC layout and routing.

[0093] Specifically, such as Figure 1 As shown, the present application maps the wiring sequence selection problem to a Markov decision problem, and the action selection of the current wiring sequence depends only on the current state.

[0094] The present invention firstly performs uniform gridding on the layout space and constructs a multi-channel image of the wiring information. MConsidering that analog ICs use two-layer routing, where horizontal and vertical routing are done on different metal layers and connected via vias, the number of channels in the multi-channel image is set to twice the number of two-ended nets plus two, and the grid cells are labeled using a binary method. The bottom two layers represent the routing area and obstacle area, and the remaining layers contain labeling information for each net.

[0095] In the embodiment of the present application, the wiring adopts a grid wiring method. First, the coordinate information of the layout image is obtained to know the range of the wiring space, and then the wiring space is divided into uniform grids. Taking into account factors such as the space utilization of the wiring layer, circuit performance and circuit manufacturability, the analog IC adopts a double-layer wiring mode, that is, the horizontal and vertical directions use different metal layers and are connected through through holes. When the grid cells are marked with a binary method, the available wiring area can be marked as 0, and the grid resources that have been wired or occupied by other obstacles and cannot be wired are marked as 1. The bottom two layers of the image mark the pin information of all the wire nets and the final wiring information after the obstacles are formed after the subsequent actual wiring. The remaining layers will mark the starting and ending positions one by one according to the pin information of all the original wire nets. Whenever an action decision is made, the image layer group where the corresponding wire net is located will be marked.

[0096] It should be noted that the present application is also applicable to analog ICs with a single-layer wiring method. For analog ICs with a single-layer wiring method, each wire net corresponds to only one layer, that is, it only occupies one channel of a multi-channel image.

[0097] Set status S = {M, J, H} ,in J Represents the current set to be routed, H Represents an arranged line network sequence. Action space A The size corresponds to the current state J The number of, each time after making a decision, A The size of will be reduced by one. Define the currently selected network action as a t , the wiring strategy is π : S → A . Build a line sequence selection model and make action decisions based on the line sequence selection model.

[0098] The line sequence selection model process is as follows:

[0099] (1) After obtaining the wiring-related information, the constructed wiring multi-channel image is scaled to adapt to the input format of the neural network, and the feature map is obtained after scaling x , sent to the attention mechanism for corresponding processing. After the corresponding processing, it is combined with the original feature map x After splicing, it is passed into the neural network.

[0100] (2) The neural network outputs a probability vector and estimated value for each action decision. The probability vector refers to the probability of each line being selected and added to the network. H The vector of probabilities in the , select the line network with the largest probability value as the current actual action a t , the estimated value is to choose this action a t Then the value will be generated according to the current line sequence.

[0101] (3) After selecting the result of a round of line sequence, use bidirectional The algorithm performs wiring and calculates the actual reward value based on the wiring results, which serves as reward feedback for reinforcement learning and is used to optimize the quality of wire sequence selection.

[0102] Among them, the attention mechanism can highlight important channel information by adaptively adjusting the weights of each channel in the feature map, thereby enhancing the representation ability of the model. In this work, the embodiment of the present application uses average pooling and maximum pooling to extract global channel features.

[0103] For the input feature map ,in C is the number of channels, H and W is the spatial dimension, then the output of global average pool (GAP) The average value of each channel is calculated as:

[0104] (1)

[0105] Where, Represents the position in the feature map aisle c The value at .

[0106] Use the global max pool (GMP) method to calculate the maximum value of each channel and output , as shown below:

[0107] (2)

[0108] By inputting feature maps x Perform average pooling and maximum pooling operations to generate two different channel descriptors:

[0109] (3)

[0110] Both pooled features are passed through a shared fully connected network. This network first reduces the number of channels to (where r is the reduction ratio), and then restore the number of channels to C , and finally use the Sigmoid function to generate channel attention weights:

[0111] (4)

[0112] Among them, FC represents the fully connected layer, and σ is the Sigmoid activation function, which ensures that the output is in the range of [0, 1].

[0113] Then add these two attention maps:

[0114] (5)

[0115] Attention weight A With the original input feature map x Multiply channel-wise to produce a reweighted output:

[0116] (6)

[0117] In the overall reinforcement learning framework, weighted output The attention mechanism is integrated into the feature processing of the model to enhance the model's recognition and utilization of important features of multi-channel images.

[0118] Because the length and width of circuit layouts vary across different cases, this application example processes multi-channel images, scaling them to a uniform length, width, and height (i.e., number of channels) before feeding them into the residual neural network. Of course, other neural network structures could also be used to replace the residual neural network to process multi-channel images.

[0119] The fusion of attention mechanism and neural network enables the model to automatically learn which channels are more important to the current task, thereby enhancing the feature expression of key channels and suppressing irrelevant or redundant information.

[0120] It should be noted that rewards can only be calculated based on the wiring results after at least one round of wire sorting has been completed. If a round of wire sorting is not completed, wiring cannot be performed based on the wire sorting results, and no wiring results can be obtained.

[0121] Regarding the reward setting in reinforcement learning, the present invention evaluates the wiring results from the following perspectives:

[0122] (1) Wire length:

[0123] Excessively long metal traces increase parasitic capacitance and inductance, inhibiting signal transmission speed and integrity, ultimately reducing electrical performance. Furthermore, longer traces lead to higher resistance and power consumption, reducing energy efficiency. In the reward setting, the total length of the trace is directly calculated as a component of the reward.

[0124] To calculate the wire length, this application will take the total length Wire_length It is defined as the sum of the Manhattan distances between points in each routing path. and The path, Wire_length As shown in formula (7):

[0125] (7)

[0126] in, n Represents the total number of segments on the path. The z coordinate is ignored in this calculation because only the two-dimensional distance between each segment is considered.

[0127] (2) Number of through holes:

[0128] In analog ICs, through-holes will introduce parasitic capacitance, inductance, and resistance. A large number of through-holes will increase parasitic parameters, which will have a negative impact on high-frequency performance, signal transmission quality, power consumption, and stability. Through-holes will also affect the manufacturability of the circuit. Considering the physical discontinuity between the through-holes and the surrounding materials, as well as the requirement for additional processing steps, the presence of through-holes will lead to increased manufacturing complexity and cost, and ultimately reduce the yield rate. In addition, through-holes are the cause of thermal stress concentration during circuit operation, thereby reducing the reliability of the circuit. In the example of this application, the number of times the metal wire layer changes in the wiring result is approximated as the number of through-holes, which is another bonus indicator.

[0129] The number of vias is illustrated by calculating the z-coordinate change between each turning point of the routing path. and ,when When , a through hole will be added, as shown in equation (8):

[0130] (8)

[0131] in, is an indicator function, when (represents a layer change, i.e. a via should be added) is 1, otherwise it is 0.

[0132] (3) Coupling noise:

[0133] In analog IC routing, the smaller the spacing between metal wires, the more significant the increase in coupled noise. Coupling noise typically arises from electrical or magnetic interactions between adjacent wires, and the degree of coupling depends on the proximity of the wires. To reduce coupled noise in routing results, this embodiment of the present invention determines a penalty term by calculating the coupled noise of parallel wires that are too close together. Equation (9) (parameter meanings) provides a method for calculating correlated noise.

[0134] (9)

[0135] in, and A function of the coupled noise between two edges, MD represents the maximum distance between two parallel line segments that have coupled noise effects. and It is a variable related to the grid line segment in the wiring diagram, which is used to determine whether the line segment is selected for wiring. The logical AND operation (&) is used to determine whether two specific line segments are selected for wiring at the same time.

[0136] Considering the inconsistency of the sizes of different reward components, this application uses relative values ​​to calculate each reward category, as shown in formula (10). The standard value is calculated based on the heuristic network order and the corresponding wiring results in the current advanced wiring work.

[0137] (10)

[0138] in, Indicates the reference value, x This calculation is used to determine the value of a variable. x The degree of change relative to the baseline value is converted into a reward value. , then the reward value is positive, indicating an improvement relative to the reference value. x Greater than , then the reward value is negative, indicating a deterioration relative to the reference value.

[0139] According to the above analysis, the total reward value is shown in formula (11).

[0140] (11)

[0141] If the analog IC uses a single-layer wiring method, the total reward value does not include the relative reward value of the number of through holes.

[0142] Considering that reinforcement learning requires the intelligent agent to continuously explore the environment, the wiring process needs to be executed multiple times, which results in a high time cost.

[0143] Based on this, the present application example designs a bidirectional Routing algorithms, such as Figure 2 As shown, the search starts from the start and end of each double-ended net at the same time to improve the search efficiency. When the paths at both ends reach the same intersection, they will be backtracked and combined into a complete path. Ideally, the routing process takes one-way time. In addition, to reduce the path irregularity that may be caused by bidirectional search, a higher routing layer change cost is added to the heuristic function to minimize the number of vias.

[0144] Figure 2 It is bidirectional The flow of the routing algorithm. Bidirectional The algorithm first creates two node sets, representing the start and end points. Next, the algorithm checks whether the open set (the set of nodes to be processed) is empty. If the open set is empty, no path connecting the start and end points can be found, and the algorithm declares the routing failed. If the open set is not empty, the algorithm selects the node with the lowest total cost (actual and estimated costs) from each set. It then expands these two nodes, exploring their neighboring nodes and updating the total cost of these neighboring nodes. Simultaneously, the algorithm updates the costs of the neighboring nodes and adds them to the open set for further processing. During the expansion process, the algorithm continuously checks for a common node between the two sets, known as a confluence point. If a confluence point is found, the algorithm starts from this node and searches backwards towards the start and end points to construct a complete path. Once the path is constructed, the algorithm outputs the routing result and terminates.

[0145] It should be noted that this application can also use one-way Algorithm, but the time consumption will be higher than bidirectional Of course, this application can also use other path planning algorithms. In addition, if the analog IC is a double-layer wiring method, then the one-way Algorithms and Bidirectional The routing algorithm must adhere to design rules. This means that when searching for the current node, it also adds the node at the same location on the other metal layer to the open set, provided that the node at the same location on the other metal layer is routable. If the routing path passes through the same location on two metal layers, that location is a via.

[0146] Specifically, one-way Algorithmic or Bidirectional When searching for the current node, the algorithm determines whether the adjacent nodes in the six search directions of the current node are available routing grid areas. The six search directions include up, down, left, right, front, and back, and selects the better movement direction through cost function calculation; when the path undergoes inter-layer transition at the same plane position on different metal layers, that position is the through-hole position.

[0147] It should be noted that when the current node is located on the upper metal layer, its search direction does not include the upper area or the upper area is directly set as a non-routable area; the same applies to the lower area.

[0148] Finally, this application example uses a proximal strategy optimization algorithm to update the neural network model parameters. First, based on the bidirectional The reward is calculated based on the wiring results obtained by the algorithm, and the discounted cumulative reward calculation method is used to allocate the reward to the state of each time step, as shown in Equation (12).

[0149] (12)

[0150] Where t is the time step, and its value range is [0, N]. This time step means that after the network is arranged in sequence, the bidirectional The reward value obtained by the routing algorithm based on the routing results generated by this routing sequence. γ is the discount factor.

[0151] Based on the discounted cumulative reward, the present invention calculates the loss function of the policy network and the value network. For the policy network, the gradient clipping policy loss function is used, as shown in equations (13) to (15).

[0152] (13)

[0153] (14)

[0154] (15)

[0155] in, i represents the weight parameters of the network, Yes The clipping value of ϵ is a hyperparameter used to limit the update range of the policy. (x, A, B) restricts x to the range [A, B]. It represents the probability ratio of the old model output to the current model output. The probability ratio is explained as follows:

[0156] Current Policy Network It means in the state Select Action The probability of is the probability of the old policy network in the same state and action, express and The ratio reflects the relative change between the new and old strategies.

[0157] For the value network, the mean square error is used as the loss function, as shown in Equation (16).

[0158] (16)

[0159] Where Φ is the weight parameter of the value network. The total loss function is shown in Equation (17).

[0160] (17)

[0161] in, It is a regularization term used to prevent parameter overfitting. represents the model parameters to be regularized (the weight matrix of the neural network), is the L2 norm of the parameter, that is, the square root of the sum of the squares of the weights, and γ is a hyperparameter of the regularization strength.

[0162] In addition, this application example uses an adaptive learning rate method to dynamically adjust the step size based on gradient information, preventing the fixed learning rate from converging too slowly or oscillating during training, thereby improving the optimization effect of the model at different stages. The use of an adaptive learning rate can ensure stability and generalization capabilities during training, making the model more resistant to noise and outliers. The use of the RMSProp optimizer in this application example can minimize losses and optimize the weight parameters of the neural network to maximize the reward of the wire sequence selection results, allowing the model to adaptively learn to arrange the wire network sequence with high quality, further improving the wiring quality.

[0163] The reinforcement learning-based analog IC layout wiring order optimization and wiring method of this application is based on the reinforcement learning framework of line sequence selection, which models the analog IC line network sorting problem as a Markov process. This framework splits all circuit lines into two-pin nets and uses a reinforcement learning framework with an attention mechanism to process them, thereby solving the problem that the current wiring order is not always optimal due to human regulations. In the reinforcement learning reward setting, the wiring results are optimized from three aspects: line length, number of through holes, and coupling noise, and rewards are set for the intelligent agent in the form of relative value rewards to solve the problem of excessive differences in the order of magnitude of reward indicators. Based on bidirectional The routing algorithm designs a path planning method that starts searching from both the routing starting point and the end point at the same time. With a reasonable setting of the cost function, it can speed up the routing and improve the routing efficiency while meeting the design rule check of the corresponding process and ensuring the routing effect.

[0164] An embodiment of the present application also provides an analog IC, which is wired using the analog IC layout wiring sequence optimization and wiring method based on reinforcement learning in the above method embodiment.

[0165] In summary, the present invention integrates the attention mechanism into the analog IC layout wiring sequence optimization and bidirectional The algorithm's routing method achieves refined and intelligent processing of network information. The attention mechanism enables the model to automatically focus on key channel information, thereby effectively enhancing feature expression and suppressing the interference of irrelevant or redundant information, providing a more accurate basis for subsequent routing decisions. Under the reinforcement learning framework, the routing results are comprehensively optimized from multiple aspects such as line length, number of vias, and coupling noise, and with the help of bidirectional The algorithm accelerates the search process, not only solving the problem that the manually specified wiring order is not always optimal, but also significantly improving wiring efficiency and layout performance.

[0166] It should be noted that the various technical features of the above-described embodiments can be combined in any manner. To simplify the description, not all possible combinations of the various technical features in the above-described embodiments are described. However, as long as there are no contradictions in the combination of these technical features, they should be considered to be within the scope of this specification. In addition, according to the needs of implementation, the various steps / components described in this application can be split into more steps / components, and two or more steps / components or partial operations of steps / components can be combined into new steps / components to achieve the purpose of the present invention.

[0167] Those skilled in the art will readily understand that the above-described embodiments merely represent several implementation methods of the present application, and their descriptions are relatively specific and detailed, but should not be construed as limiting the scope of the invention. It should be noted that a person of ordinary skill in the art may make several variations and improvements without departing from the concept of the present application, and these variations and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be based on the appended claims.

Claims

1. A method for optimizing and routing analog IC layout wiring sequence based on reinforcement learning, characterized in that: The method includes: Obtain wiring data, including wiring areas, obstacle areas, wire net collections, and the start and end points of each wire net; Gridding the wiring space and constructing a multi-channel image based on the wiring data; Using multi-channel images as states, we solve the analog IC net sorting problem based on a Markov decision process, including: Construct a line sequence selection model. The line sequence selection model takes the current multi-channel image as input and outputs a probability vector and a predicted value for each action decision. The probability vector is a vector composed of the probabilities of each line network being selected. Each time the line sequence selection model makes an action decision, it selects the line net with the highest probability in the probability vector for line sorting and simultaneously updates the multi-channel image to complete the sorting of all line nets. The predicted value refers to the value of the current line net sorting result after selecting the line net with the highest probability for sorting. According to the sorting results of all the nets, combined with the routing area, obstacle area and the starting and ending points of each net, each net is routed in turn to obtain the routing results of the simulated IC layout; Calculating the performance index of the simulated IC layout routing result as a reward, and constructing a total loss function of the wire sequence selection model in combination with the predicted value to optimize the wire sequence selection model; wherein the total loss function includes the loss function of the policy network and the loss function of the value network; Among them, building a line sequence selection model includes: (1) The feature map of the multi-channel image is processed through the attention mechanism to adaptively adjust the weight of each channel, including: ① Feature map of multi-channel image x Perform global average pooling and global maximum pooling operations to generate two different channel descriptors: in, represents the feature map of global average pooling, Represents the feature map of the global maximum pooling, represents global average pooling, represents global maximum pooling; ②The feature map of global average pooling and the feature map of global maximum pooling are passed through a shared fully connected network; the network first reduces the number of channels to , then restore the number of channels to C , and finally use the Sigmoid function to generate channel attention weights: in, C is the original number of channels of the feature map of the multi-channel image, r is the ratio of reduction, FC represents the fully connected network, σ is the Sigmoid activation function, is the average attention weight, is the maximum attention weight; ③Calculate attention weight A : ④The attention weight A Feature maps with multi-channel images x Multiply by channel to get the weighted feature map output: in, is the weighted feature map; (2) The weighted feature map Feature maps with multi-channel images x Splicing to obtain a spliced ​​feature map; (3) Construct a neural network that takes the concatenated feature map as input and outputs a probability vector and a predicted value.

2. The method for optimizing and routing analog IC layout wiring sequence based on reinforcement learning according to claim 1, wherein: Grid the routing space and construct a multi-channel image based on the routing data, including: Grid the wiring space of the analog IC, and the number of grids in the length and width directions corresponds to the length and width of the multi-channel image; If the analog IC uses a double-layer wiring method, that is, the horizontal and vertical directions are wired on different metal layers and connected through through holes, then the number of channels in the multi-channel image is twice the number of wire nets plus two. The different metal layers of each wire net correspond to two channels, and the last two channels represent the wiring area and the obstacle area. If the analog IC uses a single-layer wiring method, the number of channels in the multi-channel image is the number of wire nets plus one. The metal layer of each wire net corresponds to a channel, and the last channel represents the wiring area and obstacle area.

3. The analog IC layout wiring sequence optimization and wiring method based on reinforcement learning according to claim 2, characterized in that: Gridding the routing space and constructing a multi-channel image from the routing data, also includes: The grid cells are marked with a binary method, where the available routing grid area is marked as 0, and the unroutable grid area that has been routed or occupied by other obstacles is marked as 1; The grid area where the starting point and end point of each line grid are located is marked as 0; if a grid is selected for line grid sorting, the grid area where the starting point and end point of the grid are located is marked as 1, thereby updating the multi-channel image.

4. The analog IC layout wiring sequence optimization and wiring method based on reinforcement learning according to claim 1, characterized in that: Building a line sequence selection model also includes: Process multi-channel images and scale them into images with a specific length, width, and number of channels.

5. The analog IC layout wiring sequence optimization and wiring method based on reinforcement learning according to claim 1 or 4, characterized in that: The neural network is a residual neural network.

6. The analog IC layout wiring sequence optimization and wiring method based on reinforcement learning according to claim 3, characterized in that: For each net, combine the routing area, obstacle area, and its start and end points, using a one-way Algorithmic or Bidirectional The algorithm routes the net and marks the routed grid area as 1 after routing is completed; If the analog IC uses a double-layer wiring method, then unidirectional Algorithmic or Bidirectional When searching for a node, the algorithm determines whether the neighboring nodes in the six search directions (up, down, left, right, front, and back) are available routing grid areas. The algorithm then selects the optimal moving direction based on a cost function. When a path transitions between layers at the same planar location on different metal layers, that location is a via location.

7. The method for optimizing and routing analog IC layout wiring sequence based on reinforcement learning according to claim 2, wherein: Calculate performance indicators of analog IC layout and routing results as rewards, including: Calculate various performance indicators of analog IC layout and routing results, including: (1) Line length length , that is, the total length of the wiring path; (2) Number of through holes , that is, the total number of through holes in the wiring path when double-layer wiring; (3) Coupling noise ; Based on the performance indicators of the simulated IC layout and routing results, the relative rewards are calculated respectively: in, Indicates the baseline value; Indicates specific variable values, corresponding to various performance indicators of the simulated IC layout and wiring results; Indicates the relative reward value of the corresponding performance indicator, including the relative reward value of line length , Relative reward value of the number of through holes and coupling noise relative reward value ; The total reward value is: in, is the total reward value, α, β, γ is the weight coefficient; If the analog IC uses a single-layer wiring method, the total reward value does not include the relative reward value of the number of through holes.

8. The method for optimizing and routing analog IC layout wiring sequence based on reinforcement learning according to claim 1, wherein: Construct the total loss function of the line sequence selection model, including: The discounted cumulative reward calculation method is used to distribute the reward to the state of each time step as follows: Where, t is the time step, the value range is [0, N], N is the total number of all network; γ is the discount factor; Rewards for simulating IC layout and routing results; Indicates that at time step t Discount accumulated rewards when Based on the discounted cumulative reward, the loss function of the policy network and the value network is calculated; for the policy network, the gradient-based clipping policy loss function is used as follows: in, θ represents the weight parameters of the policy network, represents the loss function of the policy network, Represents the time step t is the ratio of the probability that the old model selects a certain network for network sorting in a certain state to the probability that the new model selects the same network for network sorting in the same state; Represents the time step t The predictive value of Indicates the difference; is a hyperparameter, Indicates that Restricted to [ , ] range, Indicates the limit value; For the value network, the mean square error is used as the loss function, as shown below: Where Φ is the weight parameter of the value network, represents the loss function of the value network; Total loss function As shown below: in, It is a regular term used to prevent parameter overfitting. Represents the weight matrix of the model.

9. An analog IC, characterized in that: The analog IC is wired using the analog IC layout wiring sequence optimization and wiring method based on reinforcement learning as described in any one of claims 1 to 8.

Images