A method for automatic generation of standard cell layout with transferability under reinforcement learning

Abstract

The present invention discloses a method for automatically generating a standard cell layout with transferability under reinforcement learning, which relates to electronic design automation. This solution is proposed to solve the problem of poor transferability in the prior art. The layout features of the standard cell layout are extracted and the design specifications are set as the reward function under the reinforcement learning framework; the transistor monomer acts as an intelligent agent, directly interacts with the layout scene, accumulates different circuit layouts and corresponding reward values, and autonomously learns the mobile strategy based on this, thereby realizing the automatic optimization of the standard cell layout under multiple constraints. The advantage is that it can not only realize batch generation of layouts under a single process node, but also adapt to the layout generation requirements of different process nodes, and realize the migration capability across process nodes.

Description

Technical Field

[0001] The present invention relates to integrated circuit EDA technology, and in particular to a method for automatically generating a standard cell layout with mobility under reinforcement learning. Background Art

[0002] In the scope of digital integrated circuit design, the layout of standard unit circuits is not only complex but also follows a strict standardized process. At present, this work is mainly completed through manual wiring, and the entire standard unit layout drawing process often takes several weeks. For standard unit circuits with a high degree of customization and complex structure, manual wiring may not achieve the optimal design solution, while algorithm optimization helps to improve the overall quality of the layout. At the same time, different process nodes lead to differences in design rules. The standard unit layout needs to be customized according to their respective process rules, which undoubtedly increases the time cost of the design.

[0003] In view of this, developing an automatic layout generation algorithm that can adapt to multiple process nodes and has migration capabilities is crucial to accelerate the research and development process of chip design. Existing layout algorithms mostly use heuristic search, but these algorithms have limitations in search accuracy, applicability and efficiency.

[0004] Over the past two decades, although some scholars have explored this field, the strategy of automatically generating standard cell layouts is still mainly limited to academic research due to the large number of standard cells and the diversity and complexity of design rules. In 2004, Ma Qi et al. [1] used a simulated annealing algorithm to optimize the area and delay of cell layout, which can handle circuits not limited to static series-parallel structures. In 2006, Lazzari C et al. [2] used the Euler algorithm for layout planning and explored the shortest path for wiring. In 2014, Ziesemer A et al. [3] used mixed linear programming (MILP) for layout and optimized the layout area. In 2021, Hao Rui et al. [4] used a deep learning algorithm for layout and roughly completed the overall layout of a deployable linear actuator. However, their method did not consider the constraints of design rules and was not applied in actual engineering.

[0005] In the design of the layout, it is necessary to consider the design rules related to the manufacturability of chip production. In 2019, Kyeongrok Jo et al. from Seoul National University used graph theory methods for transistor layout based on design rules [5]. In 2021, Daeyeal Lee et al. from the University of California proposed a satisfiability model theory that dynamically allocates pins to solve location and path problems [6]. Although their method takes into account the constraints of design rules, it does not show the DRC detection situation, and readers cannot understand the errors in their method. In addition, the process design rules of different foundries may vary, so when facing different processes, traditional methods need to process and copy the standard unit circuit layout design according to specific rules, which also requires considerable time and cost.

[0006] None of the above layout automatic generation methods consider transferability.

[0007] [1]MA Qi,WANG Xu.An Algorithm for Transistor Cell Layout Synthesis[J].Journal of Circuits and Systems, 2004(04):121-124.

[0008] [2]Lazzari C, Santos C, Reis RA new transistor-level layout generation strategy for static CMOS circuits[C] / / 2006 13th IEEE International Conference on Electronics, Circuits and Systems. IEEE, 2006: 660-663.

[0009] [3]Ziesemer A,Reis R,Moreira MT,et al.Automatic layout synthesis with ASTRAN applied to asynchronous cells[C] / / 2014IEEE 5th Latin AmericanSymposium on Circuits and Systems.IEEE,2014:1-4.

[0010] [4]Hao Rui,Cai Yici,et al.DrPlace:ADeep Learning Based Routability-Driven VLSIPlacement Algorithm[J].Journal of Computer-Aided Design&ComputerGraphics,2021,33(04):624-631.

[0011] [5]Jo K,Ahn S,Do J,et al.Design Rule Evaluation Framework UsingAutomatic Cell Layout Generator for Design Technology Co-Optimization[J].IEEETransactions on Very Large Scale Integration(VLSI)Systems,2019,PP(99):1-14.

[0012] [6] Lee D, Park D, Ho CT, et al. SP&R: SMT-based Simultaneous Place-&-Route for Standard Cell Synthesis of Advanced Nodes[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2020, PP(99):1-1. Summary of the invention

[0013] The purpose of the present invention is to provide a method for automatically generating a standard cell layout with mobility under reinforcement learning, so as to solve the problems existing in the above-mentioned prior art.

[0014] The present invention discloses a method for automatically generating a standard cell layout with mobility under reinforcement learning, which extracts the layout features of the standard cell layout and sets the design specifications as a reward function under the reinforcement learning framework; the transistor monomer acts as an intelligent agent, directly interacts with the layout scene, accumulates different circuit layouts and corresponding reward values, and autonomously learns the mobile strategy based on the accumulated values, thereby realizing the automatic optimization of the standard cell layout under multiple constraints.

[0015] After the front-end standard unit circuit design is completed and the netlist is exported, the transistors in the netlist are randomly placed as the initial layout input model, and the transistor monomer is used as the intelligent agent; at the tth step, the state value of the environment is s t , and calculate the current reward value r according to the setting of constraints t ; Reinforcement learning based on the current state value s t Output action strategy a t , by moving the position information of the current agent, the new layout state value s is obtained t+1 ; Combine the current environment state value, action strategy, reward value and new environment into a four-tuple (s t ,a t , r t ,s t+1 ) and stored in the transistor layout experience pool; continuously accumulate experience of different layouts to train the reinforcement learning network for self-learning and self-optimization, gradually find the standard unit layout that meets the constraints and is better, and obtain the standard unit circuit layout automatic design model that meets the preset constraints.

[0016] The Q-learning algorithm is used to realize automatic optimization of standard cell layout under multiple constraints.

[0017] The state space s observed by the agent is defined as:

[0018] s = [vx, vy, x-xi, y{0, 1}];

[0019] In the formula, v x , v y Indicates the current action speed of the agent; x i Represents the position information of the current intelligent agent, x={x1,x2,...,xN} represents the position information of all N transistors in the standard cell layout, y=0 represents that the transistor is in the N region, and y=1 represents that the transistor is in the P region.

[0020] The movement of the agent is a deterministic behavior, and the agent's action strategy a is to exchange the source and drain under the fixed gate condition and look for sharing; for each transistor in the current group, check whether its source is in the drain list of any transistor in the previous group.

[0021] Create an array that records the source-drain sharing situation to count the number of source-drain sharing, and reward the action strategy with the largest number of sharing; if the drain of the current transistor is the same as the source of the next transistor, and their y coordinates are the same, and the x coordinate of the current transistor plus 1 is equal to the x coordinate of the next transistor, then the two transistors are considered to be shared, and the reward value r t +1.

[0022] Update the Q value according to the Q-learning formula:

[0023]

[0024] Among them, α is the learning rate, γ is the discount factor, max(Q(S t+1 ,a t+1 )) represents the maximum Q value that can be obtained in the next state, and A(*) represents all action spaces in each state.

[0025] Go back and forth to explore different states and get rewards. The reward value is r t The larger it is, the more sources and drains there are. This layout is the optimal layout. Through training, a series of layout action strategies with the largest Q value are obtained.

[0026] In each round of training, each gate is moved in a fixed order until it switches to the last gate, and then reset to the initial layout to start a new round; the collected samples (s t , a t , r t ,s t+1 ) is input into the main network for training, and the sampling results are stored in the layout experience pool; the data in the layout experience pool is continuously updated; samples are randomly sampled from the local experience pool as the input of the target network; the parameters of the main network are updated through rounds, and after several rounds, the parameters of the main network are assigned to the target network to update the parameters of the target network.

[0027] After completing the optimal layout design, the wiring step is performed. According to the JSON format layout content output by the target network, the same connection points are connected; according to the current process rules, the wiring work uses the first metal layer to connect the corresponding pins. If the pins overlap, the wiring needs to be performed on the second metal layer; the channel length of the transistor and several design rule parameters are defined as adjustable variables to adapt to the changes in design rules under different process nodes; the design rule parameters include standard cell height, line width, via width, distance between gates, and VDD / VSS metal strip height.

[0028] The method for automatically generating a standard cell layout with migration under reinforcement learning described in the present invention has the advantage that it can not only realize batch generation of layouts under a single process node, but also adapt to the layout generation requirements of different process nodes and realize migration capabilities across process nodes. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a flow chart of the method for automatically generating a standard cell layout in the present invention.

[0030] Figure 2It is a schematic diagram of the algorithm of the method for automatically generating a standard cell layout in the present invention.

[0031] Figure 3 It is a working schematic diagram of the method for automatically generating a standard cell layout in the present invention. DETAILED DESCRIPTION

[0032] The layout design of the standard cell layout must meet multiple design specifications such as source-drain sharing and minimum area. At the same time, there are multiple design scenarios such as layout differences of different circuit modules and irregular transistor sizes. In response to this multi-constrained target optimization problem, the present invention proposes an automatic generation method for standard cell layouts based on reinforcement learning, which extracts the layout features of the standard cell layout and sets the design specifications as a reward function under the reinforcement learning framework. As an intelligent agent, the transistor monomer directly interacts with the layout scenario, accumulates different circuit layouts and corresponding reward values, and autonomously learns movement strategies based on this, thereby realizing automatic optimization of the standard cell layout under multiple constraints. This method can meet the migration of standard cell layouts under different process nodes. The main process flowchart is as follows: Figure 1 shown.

[0033] In the layout of standard cell layout, the diversity of transistor sizes leads to area waste, especially in cells with limited height. To solve this problem, folding technology is used to split and parallel large transistors to arrange them horizontally to reduce wiring complexity. The transistor pair chain consists of PMOS and NMOS tubes, with the aim of minimizing the number of chains to optimize the cell width. The relative position and placement direction of the chain must be considered during layout to reduce the difficulty of wiring and improve the routing rate. The order and flipping of the chain must also be determined during layout, and a 180° flip decision is made based on the principle of source and drain sharing. Dummy gates are inserted between unshared MOS tubes to maximize source and drain sharing. The optimization goal is to obtain the best transistor ordering through rule learning through reinforcement learning algorithms while meeting the height restriction to obtain a cell with the most optimized area.

[0034] As disclosed in CN117875253A, the process from reading the transistor netlist to generating the standard cell layout in json format mainly goes through the following layout modes:

[0035] (1) The standard cell layout includes two rows of horizontal diffusion strips, P-type and N-type. All PMOS transistors are located in the P-type strips, and NMOS transistors are located in the N-type strips.

[0036] (2) A pair of PMOS and NMOS transistors with a common gate are placed vertically in alignment and share a polycrystalline gate. Such a pair of PMOS and NMOS transistors is called a transistor pair, while a pair of PMOS and NMOS transistors that do not share a common gate but are placed vertically in alignment are also called a transistor pair.

[0037] (3) If the source and drain regions of the connected MOS tubes in the circuit are adjacent, they are connected by diffusion regions, which is called source-drain sharing. Multiple MOS tubes arranged in a row and sharing the source and drain are called a diffusion chain. Since MOS tubes are often arranged in pairs of P and N, the diffusion chain is also called a transistor pair chain.

[0038] (4) The power supply VSS / ground line VDD is laid in parallel outside the two rows of horizontal strips.

[0039] (5) The wire mesh other than the power supply VSS / ground line VDD is laid between the P-type and N-type horizontal strips.

[0040] The Q-learning algorithm is a reinforcement learning algorithm based on value iteration in the prior art. It can solve the optimal strategy problem in discrete state spaces and action strategies, and does not require prior knowledge of the state transformation model of the environment. The main idea is briefly described as follows: the intelligent agent interacts with the surrounding environment, and the intelligent agent makes multiple attempts for each possible state space and / or action strategy, and continuously learns and optimizes a value function Q(s,a) to achieve autonomous learning, where s represents the state space and a represents the action strategy. Specifically, the action strategy a is to perform layout operations based on the state space s. Through continuous information exchange between the intelligent agent and the environment, the algorithm will construct a Q-table based on the action strategy and state space, and store the state value of each state space in the table, referred to as Q value. At startup, the Q-table is empty, and the intelligent agent only knows a set of possible state spaces and action strategies. At the tth step, the current state space s t Next, select action strategy a t , through the action of the environment, a new state space s is formed t+1 , and generates a reward or penalty r t+1 .

[0041] When the Q-learning algorithm is applied in the present invention, after the front-end standard unit circuit design is completed and the netlist is derived, the transistors in the netlist can be randomly placed or placed based on experience as the initial layout input model, and the transistor monomer is used as the intelligent agent. At the tth step, the state value of the environment is s t , and calculate the current reward value r according to the setting of constraints t . Reinforcement learning is based on the current state value s t Output action strategy a t , by moving the position information of the current agent, the new layout state value s is obtained t+1 . Combine the current environment state value, action strategy, reward value and new environment into a four-tuple (s t ,a t , r t ,s t+1) and stored in the transistor layout experience pool. The experience of different layouts is continuously accumulated to train the reinforcement learning network for self-learning and self-optimization, and gradually find a standard cell layout that meets the constraints and is more optimal, so as to achieve an automatic design model of standard cell circuit layout that meets multiple constraints and can be used for multiple process nodes.

[0042] The main process of the Q-learning algorithm applied in the present invention is as follows:

[0043] (1) State space

[0044] When a transistor cell moves in a scene as an agent, it will decide the next action to perform based on the observed layout scene. The interaction between the agent and the layout scene is the basis and key to autonomous learning and training of reinforcement learning. The layout scene refers to the transistor element pool environment where the agent exists and interacts. Therefore, the state space observed by the agent and the action strategy executed need to be reasonably designed and represented.

[0045] The state space s observed by the agent is defined as:

[0046] s = [vx, vy, x-xi, y{0, 1}];

[0047] In the formula, v x , v y Indicates the current action speed of the agent; x i represents the position information of the current intelligent agent, x={x1,x2,...,xN} represents the position information of all N transistors in the standard cell layout, and y=0 or 1 represents whether the transistor is in the N region or the P region respectively.

[0048] (2) Action space design

[0049] The movement of the agent is a deterministic behavior, and the action strategy a of the agent is mainly to exchange the source and drain under the condition of fixed gate and find the sharing situation. For each transistor in the current group, check whether its source is in the drain list of any transistor in the previous group.

[0050] (3) Reward function design

[0051] Create an array that records the source-drain sharing situation to count the number of source-drain sharing, and reward the action strategy with the largest number of sharing. If the drain of the current transistor is the same as the source of the next transistor, and their y coordinates are the same, and the x coordinate of the current transistor plus 1 is equal to the x coordinate of the next transistor, then the two transistors are considered to be shared, and the reward value r t +1.

[0052]

[0053] If the above three equations are satisfied, it is considered as source-drain sharing, and the reward value is r t +1.

[0054] (4) Update Q value

[0055] Update the Q value according to the Q-learning formula:

[0056]

[0057] Go back and forth like this, explore different states, and get rewards. The reward value is r t The larger the value, the more sources and drains there are, and this layout is the optimal layout. Among them, α is the learning rate, γ is the discount factor, max(Q(S t+1 ,a t+1 )) represents the maximum Q value that can be obtained in the next state, and A(*) represents all action spaces in each state. The training process is the learning process of judging the quality of action strategies, and a series of layout action strategies with the maximum Q value are obtained through training.

[0058] In each round of training, each gate is moved in a fixed order until the last gate is switched, and then reset to the initial layout to start a new round. t , a t , r t ,s t+1 ) is input into the main network for training, and the sampling results are stored in the layout experience pool. The data in the layout experience pool is continuously updated. Samples are randomly sampled from the local experience pool as the input of the target network. The parameters of the main network are updated in rounds. After several rounds, the parameters of the main network are assigned to the target network to update the parameters of the target network. The algorithm structure is as follows: Figure 2 shown.

[0059] After completing the optimal layout design, the next step is wiring. It is necessary to connect the same connection points according to the JSON format layout content output by the target network. This process critically involves the routing work of the metal layer and the via layer. The accuracy of the wiring is evaluated by considering the degree of deviation from the wiring guidance and the cost on a specific metal layer. According to the current process rules, the wiring work uses the first metal layer to connect the corresponding pins. If the pins overlap, the wiring needs to be performed on the second metal layer, usually involving two layers of metal wires. The present invention defines the channel length of the transistor and a series of design rule parameters as adjustable variables to adapt to the changes in design rules under different process nodes, ensuring the flexibility and portability of the design. The design rule parameters include standard cell height, line width length, via width, distance between gates, and VDD / VSS metal strip height.

[0060] like Figure 3 As shown, taking AN2D0 as an example, the automatic generation process of the standard cell layout is demonstrated.

[0061] The method for automatically generating a standard cell layout with mobility under reinforcement learning described in the present invention has at least the following advantages:

[0062] (1) By applying a reinforcement learning algorithm, this algorithm aims to maximize the source-drain sharing of the device and achieve batch optimization layout of transistor layouts (stored in JSON format), thereby reducing time costs and improving design efficiency.

[0063] (2) The algorithm takes maximizing the number of source-drain sharing as the core optimization goal, and uses this strategy to minimize the transistor layout area.

[0064] (3) Automatically identify adjacent devices that cannot share source and drain and insert gate dummies between them to isolate these devices, ensuring layout consistency and enhancing reliability.

[0065] (4) The designed layout automatic generation algorithm follows the standard cell layout design rules to ensure that the layout can pass the design rule check DRC.

[0066] (5) The algorithm sets device information and design rules as adjustable parameters to adapt to the changes in design rules at different process nodes and achieve the portability of the algorithm at different process nodes.

[0067] The method proposed in the present invention can not only realize the batch generation of layouts under a single process node, but also adapt to the layout generation requirements of different process nodes, and realize the migration capability across process nodes. Specifically, the method has successfully migrated the standard cell layout automatic generation technology of the 180nm process node to more advanced process nodes such as 55nm, 40nm, and 28nm, proving its wide applicability. Key layout and wiring parameters such as design rules are fully considered. For transistor networks adapted to different process nodes, the algorithm allows device characteristics and design rules to be set as adjustable parameters. In this way, when facing changes in process nodes, only these parameters need to be appropriately adjusted to achieve cross-process migration and adaptation.

[0068] For those skilled in the art, various other corresponding changes and deformations can be made according to the technical solutions and concepts described above, and all of these changes and deformations should fall within the protection scope of the claims of the present invention.

Claims

1. A method for automatically generating a standard cell layout with transferability under reinforcement learning, characterized in that: The layout features of the standard cell layout are extracted and the design specifications are set as the reward function under the reinforcement learning framework; the transistor monomer acts as an intelligent agent, directly interacts with the layout scene, accumulates different circuit layouts and corresponding reward values, and autonomously learns the movement strategy based on this, thereby realizing the automatic optimization of the standard cell layout under multiple constraints; After the front-end standard unit circuit design is completed and the netlist is exported, the transistors in the netlist are randomly placed as the initial layout input model, and the transistor monomer is used as the intelligent agent; at the tth step, the state value of the environment is st, and the current reward value rt is calculated according to the setting of the constraint conditions; Reinforcement learning outputs the action strategy at according to the current state value st, and obtains the new layout state value st+1 by moving the position information of the current intelligent agent; the current environment state value, action strategy, reward value and new environment are combined into a four-tuple (st, at, rt, st+1) and stored in the transistor layout experience pool; the experience of different layouts is continuously accumulated to train the reinforcement learning network for self-learning and self-optimization, and gradually finds the standard unit layout layout that meets the constraints and is better, and obtains the standard unit circuit layout automatic design model that meets the preset constraints; Use Q-learning algorithm to realize automatic optimization of standard cell layout under multiple constraints; The state space s observed by the agent is defined as: s=[v x ,v y ,x-x i ,y|{0,1}]; Where, vx, vy represent the action speed of the current agent; xi represents the position information of the current agent, x = {x1, x2, ..., xN} represents the position information of all N transistors in the standard cell layout, y = 0 represents that the transistor is in the N region, and y = 1 represents that the transistor is in the P region; The movement of the agent is a deterministic behavior. The action strategy a of the agent is to exchange the source and drain under the fixed gate condition and find the sharing situation; for each transistor of the current group, check whether its source is in the drain list of any transistor of the previous group; Create an array that records the source-drain sharing situation to count the number of source-drain sharing, and reward the action strategy with the largest number of sharing; if the drain of the current transistor is the same as the source of the next transistor, and their y coordinates are the same, and the x coordinate of the current transistor plus 1 is equal to the x coordinate of the next transistor, then the two transistors are considered to be shared, and the reward value is rt+1; Update the Q value according to the Q-learning formula: Among them, α is the learning rate, γ is the discount factor, max(Q(S t+1 ,a t+1 )) represents the maximum Q value that can be obtained in the next state, and A(*) represents all action spaces in each state.

2. The method for automatically generating a standard cell layout with mobility under reinforcement learning according to claim 1, characterized in that: Go back and forth to explore different states and get rewards. The reward value is r t The larger it is, the more sources and drains there are. This layout is the optimal layout. Through training, a series of layout action strategies with the largest Q value are obtained.

3. The method for automatically generating a standard cell layout with mobility under reinforcement learning according to claim 2, characterized in that: In each round of training, each gate is moved in a fixed order until it switches to the last gate, and then reset to the initial layout to start a new round; the collected samples (s t , a t , r t ,s t+1 ) is input into the main network for training, and the sampling results are stored in the layout experience pool; The data in the layout experience pool is continuously updated; samples are randomly sampled from the local experience pool as the input of the target network; the parameters of the main network are updated through rounds, and after several rounds, the parameters of the main network are assigned to the target network to update the parameters of the target network.

4. The method for automatically generating a standard cell layout with mobility under reinforcement learning according to claim 3, characterized in that: After completing the optimal layout design, the wiring step is performed. According to the JSON format layout content output by the target network, the same connection points are connected; according to the current process rules, the wiring work uses the first metal layer to connect the corresponding pins. If the pins overlap, the wiring needs to be performed on the second metal layer; the channel length of the transistor and several design rule parameters are defined as adjustable variables to adapt to the changes in design rules under different process nodes; the design rule parameters include standard cell height, line width, via width, distance between gates, and VDD / VSS metal strip height.

Images