Chip macro element placement method for layout adjustment based on reinforcement learning

Abstract

The invention discloses a chip macro element placement method for layout adjustment based on reinforcement learning, which uses reinforcement learning to learn how to adjust the existing layout instead of placing elements from the beginning, thereby providing enough information for strategies and obtaining relatively dense and accurate rewards. Besides, the concept of regularity is introduced in the training process, which is regarded as an important index in the chip design industry, but is often neglected in the current reinforcement learning layout method. The method is evaluated in an ICCAD 2015 benchmark test, and compared with several competitive methods, the semi-perimeter wiring length and regularity can be improved. The power consumption, performance and area performance of the chip are tested by using commercial software, and the result shows that the layout adjustment method provided by the invention can realize remarkable PPA improvement. In addition, the layout of any method can be finely adjusted, and the quality of the layout is improved.

Description

Technical Field

[0001] The present invention relates to a chip macro component placement method, and in particular to a chip macro component placement method adjusted based on reinforcement learning. Background Art

[0002] Electronic design automation (EDA) technology has been widely used to assist engineers in chip design. However, the rapid development of very large-scale integration (VLSI) technology has led to an exponential growth in chip size, which has brought great challenges to existing EDA technology. In particular, an important stage in the design process: chip floor planning, which aims to place a series of components on the chip canvas while complying with constraints such as congestion and density to optimize indicators such as power consumption, performance, and area. The number of components to be placed on the chip canvas has increased rapidly, and the routing relationship between them has become more complex, which requires more novel and efficient methods. In the placement stage of floor planning, we need to determine the location of each component on the chip canvas. Modern chips typically contain thousands of macro components (individual building blocks such as memory) and millions of standard cells (smaller basic components such as logic gates). In the chip design process, the upstream step usually provides a netlist to the chip layout task, outlining the requirements for layout placement. The netlist can be represented as a large-scale hyper-graph, which contains many hyper-edges representing the routing relationships between macro components and standard components. Each hyper-edge contains multiple components, indicating that these components need to be connected by a set of wires during the placement process. Component placement is usually divided into two stages: macro component placement, which is usually solved by heuristic methods or learning-based methods; standard component placement, which is usually solved by a solver. After all components (including macro components and standard components) are placed, the routing stage begins. The general process of chip layout planning and routing is as follows: Figure 1 shown.

[0003] Researching the placement of chip macro components can improve the technical level and competitiveness of China's chip industry and promote industrial development. This is of great significance for alleviating the contradiction between supply and demand in the domestic chip industry, reducing external dependence, and achieving industrial autonomy and control.

[0004] The macro component placement problem is usually modeled as an optimization problem, which is evaluated iteratively to obtain better layout results. The optimization problem needs to satisfy the non-overlapping constraint, that is, there should be no overlapping parts between macro components, otherwise they cannot be used in actual manufacturing.

[0005] Reinforcement learning has recently become a promising technology for improving the quality of macro component placement. It provides a new path to solve various challenges of macro component placement by learning strategies through interaction with complex environments. Specifically, the reinforcement learning method divides the chip layout into discrete grids and gradually places macro components into the grid, modeling the problem as a Markov decision process in a discrete action space. At each step of the Markov process, the reinforcement learning agent determines the placement position of the current macro component. After placing the current component, the system gives a reward and the state is transferred. The process is repeated until each macro component is placed on the chip layout. The chip macro component placement method based on reinforcement learning can obtain a layout result with a lower line length, and the trained layout strategy can be generalized to other chips, and the placement speed is faster than the traditional method. However, the current reinforcement learning-based method has problems such as difficulty in satisfying non-overlapping constraints, sparse rewards, little available information during placement, and easy congestion of layout results, which makes it difficult to handle complex modern chips.

[0006] Therefore, for the chip macro component layout problem based on reinforcement learning, it is necessary to design a method that meets the constraints, is efficient and reliable. Summary of the invention

[0007] Purpose of the invention: In view of the problems existing in the existing reinforcement learning-based chip macro component layout method, such as the difficulty in satisfying the non-overlapping constraints, sparse rewards, little available information during the placement process, and easy congestion of layout results, the present invention proposes a reinforcement learning layout method based on adjustment.

[0008] Technical solution: A chip macro component placement method based on reinforcement learning for layout adjustment, which uses reinforcement learning to gradually adjust each macro component of the existing layout instead of placing it from the same place. The optimization problem of macro component placement is stated as: Under the constraint that components do not overlap, determine the position of each component so that the wiring length of these components is the shortest. Use half-perimeter wirelength (HPWL) to estimate the macro component wiring length.

[0009] The optimization problem of macro component placement (chip component layout problem) can be modeled as follows: the input includes a netlist H = (V, E), where V represents the information of all components (i.e., height and width) that need to be placed on the chip layout; E is a network consisting of hyperedges e i∈E, each hyperedge contains two or more components, indicating that these components need to be connected by a set of wires during the layout process. The component layout solution s includes all components The position of k, where k represents the number of elements, each element v i ∈V on the chip layout is represented by (x i ,y i ). For a component layout solution s, the specific calculation method of its HPWL index is as follows: each hyperedge e j ∈E corresponds to a rectangular area with coordinates The lower left endpoint and coordinates The upper right endpoint is marked, which means that the rectangle encloses e j The minimum rectangle of all components in it. and Represent the width and height of the rectangle respectively, then the calculation method of the semi-perimeter wiring length is:

[0010]

[0011] In the optimization process, we hope to minimize the HPWL objective function while satisfying the non-overlapping constraint. Therefore, the chip component layout problem can be formalized as:

[0012] min HPWL(s,H),stD(s)≤∈

[0013] Where D represents the chip density, and ∈ represents the constraint value that the density needs to meet. Ultimately, we hope to improve the chip power consumption, performance, and area (PPA) performance by minimizing HPWL.

[0014] Current applications of reinforcement learning for chip placement suffer from several limitations, including long training times, failure to guarantee PPA improvements, and lack of generalization to different chip placements. In this study, we highlight that a major reason for these issues is the formulation of the problem, i.e., traditional RL methods perform macro placement from scratch, often leading to limited information state and inaccurate reward signals during the learning process. Current reinforcement learning-based placement methods face long training times, poor generalization, and failure to guarantee PPA performance. A key issue is the formulation of the problem, i.e., performing placement from scratch using reinforcement learning, which leads to a lack of useful information and inaccurate rewards during training. To address these challenges, we propose a novel reinforcement learning method, called MaskRegulate, that shifts the focus from the initial placement to the optimization of an existing placement. The reinforcement learning policy acts as a regulator rather than a placer, acting on a pre-existing placement, which allows for comprehensive state information and more precise rewards. This regulation improves the efficiency of the learning process, ultimately improving placement results.

[0015] The order of macro placement is determined based on some predefined rules, such as the number of nets, the size of macros, and the number of connections of placed modules. An episode ends when all macros are placed. Usually, the state representation includes information about the chip layout, the macros that have been placed, and the macros that are currently being placed. In early reinforcement learning methods, rewards are determined only after all macros are placed, which leads to sparse reward signals and increases the complexity of the training process. Recent studies have introduced various methods to enhance the density of reward signals. Unlike reinforcement learning placers, our chemical learning regulator focuses on optimizing the existing layout by adjusting the position of one macro at each step. Unlike placers that place from scratch, the regulator is able to exploit additional information when adjusting each macro. Specifically, the regulator considers not only the already placed macros, but also the positions of all other macros. In addition, although using a similar reward function as before, it still improves accuracy by considering all macros. Due to the above advantages in the MDP problem formulation, the reinforcement learning regulator can achieve better results than the reinforcement learning placer in experiments even without considering additional factors such as regularity. Furthermore, it is easier for the tuner to tune unseen silicon than to place it from scratch, because the placer’s incomplete state information is worse in the unseen silicon case, leading to worse performance.

[0016] The layout of chip macro elements has a significant impact on the subsequent chip design process, including standard cell layout and chip routing. If one focuses only on minimizing wire length (which is the case for most current reinforcement learning placers), some macro elements may end up being placed in the center of the chip layout, causing congestion. This will result in the available layout area being divided into separate, unconnected sub-areas. As a result, standard cells connected by the same network may be scattered across different layout sub-areas, increasing the overall wire length, which ultimately degrades timing performance. Therefore, experienced engineers usually place macro elements in the peripheral area of ​​the chip during macro element layout to prevent this from happening. We incorporate regularity into our learning-based layout method to achieve layout preferences similar to those of experienced engineers. Intuitively, macro elements close to the edge tend to have lower regularity. Therefore, we propose a simple and effective way to measure regularity. On the chip layout, the regularity of a grid located at (x,y) is calculated as min{x,X max -x}+min{y,Y max -y}, where X max and Y max Represents the actual length of the horizontal and vertical axes respectively.

[0017] During training, the reward of MaskRegulate consists of two parts: wire and r reg , respectively represent the reduction of HPWL and the improvement of regularity after optimizing the current macro element. In order to mitigate the impact of line length and regularity on the scale difference of training, r wire and r reg are all normalized. The final reward is r = α·r wire +(1-α)·r reg , where α is the trade-off coefficient, which is usually 0.5. We use the proximal policy optimization (PPO) algorithm to learn the policy.

[0018] The detailed algorithm flow is as follows:

[0019] Algorithm flow 1: Chip macro component placement method for layout adjustment based on reinforcement learning

[0020] Input: Legal initial layout result s, netlist H, number of adjustments N, initial adjustment strategy π

[0021] Output: trained regulator policy π′

[0022] Based on the netlist information H, determine the macro element adjustment order O, the current adjustment number i = 1

[0023] 1. When the number of training times i≤N:

[0024] 2. When there are still macro components that have not been adjusted:

[0025] 3. Select a component according to the macro component adjustment order O

[0026] 4. Observe the current state s and use the strategy π to adjust it, and get the action a = π(s)

[0027] 5. Update the layout state s', get reward r, and update the data set D←(s,a,s′,r)

[0028] 6. Based on data set D, use PPO for strategy training

[0029] 7. Return the trained regulator policy π′

[0030] Beneficial effect: The method proposed in the present invention uses reinforcement learning to learn how to adjust the existing layout, which can provide sufficient information for the strategy and obtain relatively dense and accurate rewards, thereby improving the algorithm training efficiency of reinforcement learning for chip macro component layout and obtaining better layout results.

[0031] Compared with the existing chip macro element layout method, this method has high optimization efficiency, good performance, and is easy to implement. It can be conveniently used to adjust various existing layout results and has great practical application potential. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is the general process of chip layout;

[0033] Figure 2 is a flow chart of a chip macro component placement method for performing layout adjustment based on reinforcement learning according to an embodiment of the present invention;

[0034] Figure 3 Figure 2. Placement and congestion results of different methods on eight ICCAD 2015 benchmarks. Congestion results are obtained by Cadence Innovus. Red dots indicate critical areas of congestion, including (a) DREAMPlace, (b) AutoDMP, (c) WireMask-EA, (d) MaskPlace, and (e) MaskRegulate. DETAILED DESCRIPTION

[0035] The present invention is further explained below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0036] A chip macro component placement method for layout adjustment based on reinforcement learning, referred to as MaskRegulate. This method is a method that uses reinforcement learning to adjust the existing chip layout results. Compared with the previous method of placing components from scratch, MaskRegulate can obtain more information and more intensive and accurate rewards during the adjustment process, thereby improving training efficiency and obtaining chips with better performance. The input of the MaskRegulate method is a legal initial layout result s, a netlist H, the number of adjustments N, and a randomly initialized strategy π, and the output result is the trained adjuster strategy π′. MaskRegulate will adjust the chip components one by one based on the initial layout results. The order of adjustment O is determined by the netlist information H, and generally the macro components with larger areas are adjusted first. After determining the order of component adjustment, all components are adjusted once, which is called an episode. In each episode, each step will select a component in order, observe the current state s, and use the strategy π to adjust it, and obtain the action a=π(s). Next, update the layout state s', obtain the reward r, and update the data set D←(s,a,s′,r). After the round, PPO is used to train the strategy based on the dataset D. After a total of N rounds, the trained regulator strategy π′ is returned.

[0037] For a component layout solution s, the specific calculation method of its HPWL index is as follows: Each hyperedge e j ∈E corresponds to a rectangular area with coordinates The lower left endpoint and coordinates The upper right endpoint is marked, which means that the rectangle encloses e j The minimum rectangle of all components in it. and Represent the width and height of the rectangle respectively, then the calculation method of the semi-perimeter wiring length is:

[0038]

[0039] In the optimization process, we hope to minimize the HPWL objective function while satisfying the non-overlapping constraint. Therefore, the chip component layout problem can be formalized as:

[0040] min HPWL(s,H),stD(s)≤∈

[0041] Where D represents the chip density, and ∈ represents the constraint value that the density needs to meet. Ultimately, we hope to improve chip power consumption, performance, and area performance by minimizing HPWL.

[0042] Eight chips from the chip design dataset ICCAD2015 are selected to verify the effectiveness of the algorithm MaskRegulat.

[0043] Our comparison algorithm is as follows:

[0044] DREAMPlace (DMP): an advanced analysis-based layout method

[0045] WireMask-EA: A state-of-the-art black-box macro component placement method.

[0046] AutoDMP: Improves DREAMPlace’s macro placement method by iteratively exploring its configuration space

[0047] MaskPlace: A representative reinforcement learning method whose policy architecture, state, and HPWL reward are similar to ours.

[0048] MaskRegulate: Our proposed method

[0049] The overall evaluation results are shown in Table 1. MaskRegulate achieved the best average ranking in both proxy and PPA metrics. DREAMPlace had the worst average ranking in line length, congestion, and timing. However, after the adjustment of MaskRegulate, the obtained layout achieved the best average ranking. Compared with MaskPlace, MaskRegulate significantly improved in multiple PPA metrics: routing line length increased by 17.08%, routing horizontal and vertical congestion overflow improved by 73.08% and 38.81% respectively, the worst negative margin increased by 18.35%, the total negative margin increased by 37.89%, and the number of violation points decreased by 46.17%. By integrating regularity, MaskRegulate achieved the highest regularity on all eight chips.

[0050] Table 1: Results of proxy and PPA metrics in ICCAD 2015 benchmark. Global HPWL (1e8) and regularity (1e6) are two proxy metrics. PPA metric is evaluated by Cadence Innovus. Placement is performed by different methods and subsequent stages are completed by Cadence Innovus. The following are the definitions of each metric: rWL (m): routing line length; rO-H (%) and rO-V (%): horizontal and vertical congestion, respectively; WNS (ns): worst latency; TNS (1e5μs): total latency; NVP (1e4): number of violation points. For WNS and TNS, the larger the value, the better, while for other metrics, the smaller the value, the better.

[0051]

[0052]

[0053] In addition, we provide detailed visualization of the placement results of all methods for all eight chips in ICCAD 2015. Figure 3 As shown in Figure 2, MaskRegulate shows significant improvements in congestion metrics. In addition, the placement results of MaskRegulate are more regular than all other methods.

[0054] The generalization ability of reinforcement learning strategies is an important research issue. We pre-train MaskRegulate and MaskPlace on the first four ICCAD chips (i.e., superblue1, superblue3, superblue4, and superblue5) and test them on the remaining four chips. To further verify the ability of MaskRegulate to adjust the results of different initial layouts, we use it to adjust the results obtained by different initial layouts on the test chip.

[0055] Table 2 Generalization results of proxy indicators for four chips in ICCAD 2015 benchmark

[0056]

[0057]

Claims

1. A chip macro component placement method for layout adjustment based on reinforcement learning, characterized in that: The optimization problem of macro component placement is formulated as: Under the constraint that components do not overlap, determine the position of each component so that the wiring length of these components is the shortest; The half-perimeter wiring length is used to estimate the macro element wiring length, and the chip power consumption, performance, and area performance are improved by minimizing HPWL.

2. The chip macro component placement method for layout adjustment based on reinforcement learning according to claim 1, characterized in that: The optimization problem of macro component placement is modeled as follows: the input includes a netlist H = (V, E), where V represents the information of all components that need to be placed on the chip layout; E is a network consisting of hyperedges e i ∈E, each hyperedge contains two or more components, indicating that these components need to be connected by a set of wires during the layout process; the component layout solution s includes all components The position of k, where k represents the number of elements, each element v i ∈V on the chip layout is represented by (x i ,y i ); For a component layout solution s, the specific calculation method of its HPWL index is as follows: each hyperedge e j ∈E corresponds to a rectangular area with coordinates The lower left endpoint and coordinates The upper right endpoint is marked, which means that the rectangle encloses e j The minimum rectangle of all components within and Represent the width and height of the rectangle respectively, then In the optimization process, the HPWL objective function is minimized while satisfying the non-overlapping constraint; the chip component layout problem is formalized as: min HPWL(s,H),stD(s)≤∈ Where D represents the chip density and ∈ represents the constraint value that the density needs to meet.

Images