A mixed-size macro layout method, device and storage medium based on reinforcement learning and macro legalization

Abstract

The present invention discloses a mixed-size macro layout method, device and storage medium based on reinforcement learning and macro legalization. By combining reinforcement learning and macro module legalization algorithms, it can efficiently handle macro module layout problems of mixed sizes and pre-placed modules. Specifically, a convolutional neural network is used to extract global observation features and a graph neural network is used to extract netlist features, so that the reinforcement learning agent can capture and aggregate macro and micro circuit layout information in real time, accelerating the learning of the intelligent agent. A fast-action mask layer generation algorithm and a macro module legalization algorithm are used to adapt to the placement of macro modules of mixed sizes, alleviating the space waste and computational overhead caused by different grid division algorithms. The present invention provides a mixed-size macro layout method, device and storage medium based on reinforcement learning and macro legalization, which achieves multiple goals of improving layout quality, optimizing resource utilization, and shortening the design cycle while ensuring layout legality.

Description

Technical Field

[0001] The present invention relates to a mixed-size macro layout method, device and storage medium based on reinforcement learning and macro legalization, and belongs to the technical field of integrated circuit electronic design automation (EDA). Background Art

[0002] With the rapid advancement of integrated circuit technology and the continuous shrinking of semiconductor process nodes, chip integration is becoming increasingly higher. This presents more challenges for chip designers. Traditional chip design methods relying solely on manpower are no longer able to meet the performance, power consumption, and design cycle requirements of chips. Therefore, electronic design automation (EDA) tools must continue to improve to adapt to the increasing complexity of chip design. Modern EDA tools must support a variety of chip designs and be flexible and scalable enough to meet various design requirements.

[0003] Macro placement (floorplanning) is a crucial step in the physical design process of very large-scale integrated circuits (VLSIs). It determines the relative positions of individual macro modules in the chip netlist and significantly impacts the subsequent standard cell placement and routing. Furthermore, modern chip designs often include a large number of pre-placed modules. Ignoring these pre-placed modules in placement methods can lead to illegal placement or suboptimal solutions. The mixed-size macro placement problem with pre-placed modules can be summarized as follows: Given a predetermined chip canvas (layout area) and a netlist containing macros, standard cells, and pre-placed modules of varying sizes, the goal is to determine the positions of all macros so that there is no overlap between macro modules while minimizing wire length and routability. The quality of macro placement impacts chip performance, power consumption, and area. Initially, with simple chip structures and low complexity, macro placement could be solved manually. However, with the increasing integration density of chips, the number of design rules and requirements has increased, necessitating the need for automated placement algorithms to assist in design. Traditional optimization-based placement algorithms can be categorized into three main categories: partitioned placement algorithms, simulated annealing algorithms, and analytical placement algorithms.

[0004] With the advent of the era of big data and artificial intelligence, data volumes and computing power are exploding. Intelligent computing is highly likely to be an effective way to overcome the current bottlenecks in macromodule placement quality and efficiency. Machine learning has been applied to various research fields and has achieved significant progress. Reinforcement learning, with its excellent self-learning and generalization capabilities, is becoming one of the most active research areas in machine learning. The integration of reinforcement learning technology with the EDA field has become a trend. In recent years, some researchers have considered using reinforcement learning-based methods to solve the macromodule placement problem. Some researchers use the Proximal Policy Optimization (PPO) algorithm to train an agent to sequentially place macromodules on the chip canvas. This method only considers the case where macromodules have uniform size when gridding the canvas. Other researchers have combined multi-view embedding models with reinforcement learning algorithms and introduced random distillation networks to alleviate the problem of sparse reward values ​​and encourage exploration. However, this method does not consider macromodule size and cannot generate valid placements. Furthermore, the mask layer represents chip placement as a visual problem to fully describe the circuit modules on the chip, maximize the reward for wire length, and ensure that valid macromodule placements do not overlap. However, the mask layer is only for macro modules and does not consider the joint placement of macro modules and standard cells.

[0005] While reinforcement learning shows great promise in EDA, research in this area is relatively new, so some methods can only achieve good results when applied to smaller problems. When faced with a complex netlist with a large number of modules, it may not be possible to achieve results comparable to those of mature traditional EDA algorithms.

[0006] Therefore, further research and development of the application of reinforcement learning in the EDA field is needed to overcome its limitations in dealing with large-scale, high-complexity chip design problems. Summary of the Invention

[0007] Objective: To overcome the problem of poor macro module layout results in the prior art, the present invention provides a mixed-size macro layout method, device and storage medium based on reinforcement learning and macro legalization.

[0008] Technical solution: To solve the above technical problems, the technical solution adopted by the present invention is:

[0009] In a first aspect, a mixed-size macro layout method based on reinforcement learning and macro legalization includes the following steps:

[0010] Step S1: Obtain the netlist and layout space of the circuit to be laid out, divide the layout space into a grid, and arrange the pre-placed macro modules in the grid layout space according to their sizes and positions to obtain an initialized layout state.

[0011] Step S2: Obtain a pre-built reinforcement learning layout model.

[0012] Step S3: Based on the initialized layout state, global observation information, the sizes of the placed macromodules, and the corresponding netlist are obtained to obtain a one-dimensional feature vector. This one-dimensional feature vector is used as the current state and input into the reinforcement learning layout model, which outputs an action probability prediction matrix and value. Based on the value, intrinsic rewards are obtained and the parameters of the reinforcement learning layout model are updated based on the intrinsic rewards.

[0013] Step S4: Based on the initialized layout state and the size of the macroblock to be placed, a fast action mask generation algorithm is used to obtain a current action mask layer. The current action mask layer is superimposed on the action probability prediction matrix to obtain a prediction matrix for the macroblock to be placed. Based on the prediction matrix for the macroblock to be placed, the current action is obtained, i.e., the macroblock to be placed is arranged at the specified location. The layout state after placement is obtained based on the current action.

[0014] Step S5: Replace the initial layout state in step S3 with the layout state after placement, repeat steps S3 and S4 to arrange the macro modules to be placed one by one to the designated positions, until all macro modules are placed, and obtain a preliminary plan for partially overlapping macro layout.

[0015] Step S6: Input the preliminary plan of partially overlapping macro layout into the macro module legalization algorithm to obtain a legal macro module layout plan.

[0016] Step S7: Obtain an external reward according to the legal macromodule layout solution, and update the parameters of the reinforcement learning layout model according to the external reward to obtain an updated reinforcement learning layout model.

[0017] Step S8: Repeat steps S3-S7 until the training reaches the set maximum cycle to generate the final reinforcement learning layout model.

[0018] As a preferred solution, the reinforcement learning layout model includes: a strategy network and a value network, the strategy network π θ (A t |S t ) is composed of a deconvolutional neural network, the value network V σ (S t ) consists of a fully connected neural network.

[0019] Among them, the layout state of the macro module is used as the state space, and St Indicates that a movable macro module is placed as the action space, with A t θ represents the parameters of the policy network, and σ represents the parameters of the value network.

[0020] Based on the current intrinsic reward in the value, the loss function of the policy network is calculated, and the parameters of the policy network are updated according to the loss function of the policy network.

[0021] According to the external reward, the loss function of the reinforcement learning layout model is calculated, and the parameters of the reinforcement learning layout model are updated according to the loss function of the reinforcement learning layout model.

[0022] As a preferred solution, step S3 specifically includes:

[0023] Based on the initialized layout state, global observation information is obtained and fed into a convolutional neural network to extract a global feature embedding vector. The size of the laid-out macromodule and the corresponding netlist are fed into the graph neural network to output a graph feature embedding vector. The global feature embedding vector and the graph feature embedding vector are combined into a one-dimensional feature vector.

[0024] The one-dimensional feature vector is used as the current state and input into the policy network and value network respectively. The policy network outputs the action probability prediction matrix, and the value network outputs the value.

[0025] Store the value in a buffer, randomly replay the value during reinforcement learning layout model training, input the value into the target network and prediction network respectively, and output the feature embedding vector φ(·) and the feature embedding vector φ respectively. ∧ (·), according to the feature embedding vector φ(·), the feature embedding vector φ ∧ (·) Calculate the intrinsic reward R Intrinsic .

[0026] According to the intrinsic reward R Intrinsic Calculate the loss function of the policy network, and update the parameters of the policy network according to the loss function of the policy network to obtain the updated policy network.

[0027] As a preferred solution, the intrinsic reward R Intrinsic The calculation formula is as follows:

[0028] R Intrinsic =‖φ(·)-φ ∧ (·)‖ 2

[0029] in,‖‖ 2 represents the 2-norm.

[0030] As a preferred solution, step S4 specifically includes:

[0031] Get the current global observation layer M according to the initialized layout state obs , according to the macro module M to be placed i The size of the width and height are recorded as w i and h i .

[0032] Calculate the macro module M to be placed i The ratio of width to grid width w i / w b , the ratio of height to grid height h i / h b , round down to get the macro module M to be placed i The number of grid cells n occupied on the canvas w and n h . and Where W and H represent the width and height of the wafer canvas, respectively. N is the number of grids the wafer canvas is divided into.

[0033] Initialize the action mask layer M with an all-zero matrix mask , and assign it to the temporary matrix M tmp .

[0034] According to the current global observation layer M obs , the current global observation layer M obs The occupied grid corresponds to M tmp Add 1 to all the grids on the grid to get the grid S occupied by the macro module placed .

[0035] Traverse each unit grid and place the macro module M i The lower left corner of the grid is placed on the current grid, if the grid it occupies will overlap with S placed If a conflict occurs, the grid cannot be placed, corresponding to M tmp Add 1 to the grid value on the grid to get the conflicting grid S collision .

[0036] Traverse the outermost rows and columns of the grid, if the macro module M is to be placed i Number of grid cells occupied n w and n h If it is greater than 1, the macro module to be placed cannot be placed in that row or column, which corresponds to M tmp Add 1 to all the grid values ​​of the rows or columns above to obtain the network S that exceeds the boundary boundary .

[0037] M tmp The area composed of all grids greater than 0 in S invalid .

[0038] Mtmp Set all values ​​greater than 1 to 1, and save the rest to 0, and output the current action mask layer M mask .

[0039] The current action mask layer M mask Superimposed with the action probability prediction matrix, the prediction matrix of the macromodule to be placed is obtained.

[0040] According to the prediction matrix of the macroblock to be placed, the current action is obtained, that is, the macroblock to be placed is arranged at the specified position, and the layout state after placement is obtained according to the current action.

[0041] As a preferred solution, step S6 specifically includes:

[0042] According to the preliminary scheme P of partially overlapping macro layout illgal Initialize the horizontal constraint graph G with the overlapped part in h and vertical constraint graph G v .

[0043] According to the horizontal constraint graph G h and vertical constraint graph G v Calculate the longest horizontal path lp h The longest perpendicular path lp v .

[0044] Randomly adjust the level constraint graph G h and vertical constraint graph G v After adjustment, calculate the adjusted horizontal longest path lp h ' and the longest vertical path lp v '.

[0045] The adjusted horizontal longest path lp h ' and the longest vertical path lp v ' is compared with the width W and height H of the chip respectively until the longest path after adjustment is smaller than the chip size, then the loop is exited and the coordinates of all macro modules are determined.

[0046] Initialize the simulated annealing solver to obtain the total displacement D of all macromodules, the approximate wire length W estimated by the semi-perimeter wire length, and the unavailable area A of the macromodules between standard cells.

[0047] Use the solver based on simulated annealing algorithm to continuously adjust the macro modules in the macro layout. Each adjustment calls the objective function to calculate the target cost. When the adjustment is executed, the newly calculated cost new Less than the previous minimum target cost min When the minimum target cost is reached, the adjustment is applied, otherwise the macro module position is called back. minWhen it is less than the predetermined threshold, the loop is exited. The macromodule layout at this time is used as the legal macromodule layout scheme P legal .

[0048] As a preferred solution, the calculation formula of the objective function is as follows:

[0049] Cost = α·D + β·W + γ·A

[0050] Where: α, β, and γ are user-defined penalty coefficients, which are used to balance the contribution of displacement, line length change, and unusable area to the objective function respectively.

[0051] As a preferred solution, obtaining external rewards according to a legal macromodule layout solution specifically includes:

[0052] Obtain extrinsic rewards for legal macromodule placement solutions using DREAMPlace, an open source placer based on a parsing algorithm.

[0053] The extrinsic reward R Extrinsic =W HPWL , W HPWL Indicates the half perimeter length of the layout, calculated as follows:

[0054]

[0055] Where: x j and y j Respectively The horizontal and vertical coordinates, N netlist Indicates the number of netlists, net indicates a wire net in the netlist, Represents the jth pin in the i-th net.

[0056] In a second aspect, a computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements a mixed-size macro layout method based on reinforcement learning and macro legalization as described in any one of the first aspects.

[0057] According to a third aspect, a computer device includes:

[0058] Memory, used to store instructions.

[0059] The processor is configured to execute the instructions so that the computer device performs the operations of the mixed-size macro layout method based on reinforcement learning and macro legalization as described in any one of the first aspects.

[0060] Beneficial effects: The present invention provides a mixed-size macro layout method, device and storage medium based on reinforcement learning and macro legalization, which can efficiently handle the macro module layout problem of mixed sizes and pre-placed modules by combining reinforcement learning and macro module legalization algorithms. Specifically, a convolutional neural network is used to extract global observation features and a graph neural network is used to extract netlist features, so that the reinforcement learning agent can capture and aggregate macro and micro circuit layout information in real time, accelerating the learning of the intelligent agent. A fast-action mask layer generation algorithm and a macro module legalization algorithm are used to adapt to the placement of macro modules of mixed sizes, alleviating the space waste and computational overhead caused by different grid division algorithms. It achieves the multiple goals of improving layout quality, optimizing resource utilization, and shortening the design cycle while ensuring the legality of the layout.

[0061] This paper presents a mixed-size macro placement method based on reinforcement learning and macro legalization. This method uses reinforcement learning to generate a global layout that allows for partial overlap between macro modules. A macro module legalization algorithm then ensures the legality of the floorplan. This method outperforms existing state-of-the-art methods in terms of macro placement quality while offering numerous benefits, including a fast-action mask generation algorithm for pre-placed modules, training a policy network with a non-sparse reward feedback algorithm, and transferring pre-trained reinforcement learning models to new circuit placements. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is a flowchart of a mixed-size macro layout method based on reinforcement learning and macro legalization of the present invention.

[0063] Figure 2 Schematic diagram of the reinforcement learning layout model iteration of the present invention. DETAILED DESCRIPTION

[0064] The following is a clear and complete description of the technical solutions in the examples of the present invention, in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative work are within the scope of protection of the present invention.

[0065] The present invention will be further described below with reference to specific embodiments.

[0066] Example 1:

[0067] like Figure 1 As shown, this embodiment introduces a mixed-size macro layout method based on reinforcement learning and macro legalization, including the following steps:

[0068] Step S1: Obtain the netlist and layout space of the circuit to be laid out, divide the layout space into a grid, and arrange the pre-placed macro modules in the grid layout space according to their sizes and positions to obtain an initialized layout state.

[0069] In one embodiment, a gridded layout space is arranged with macro modules, wherein occupied grids are set to 1 and unoccupied grids are set to 0.

[0070] Step S2: Build a reinforcement learning layout model.

[0071] In one embodiment, the reinforcement learning layout model employs a Greedy Policy Optimization (PPO)-based reinforcement learning layout model. Using a deep reinforcement learning algorithm, the placement of macroblocks is considered a Markov decision process. By continuously completing macroblock placement through a reinforcement learning agent, performing gradient descent, and updating the policy network neural network parameters, the policy network in the reinforcement learning layout model ultimately learns how to maximize extrinsic rewards, thereby achieving high-quality macroblock placement.

[0072] In one embodiment, the reinforcement learning layout model includes: a policy network π θ (A t |S t ) is composed of a deconvolutional neural network, the value network V σ (S t ) consists of a fully connected neural network.

[0073] Among them, the layout state of the macro module is used as the state space, and S t Indicates that a movable macro module is placed as the action space, with A t express.

[0074] Execute a current action a t , change the current action a t and the current state s t Input policy network, output next state s t+1 .

[0075] The current state s t Input value network, output value.

[0076] Based on the current intrinsic reward in the value, the loss function of the policy network is calculated, and the parameters of the policy network are updated according to the loss function of the policy network.

[0077] According to the external reward, the loss function of the reinforcement learning layout model is calculated, and the parameters of the reinforcement learning layout model are updated according to the loss function of the reinforcement learning layout model.

[0078] like Figure 2 As shown, step S3: Based on the initialized layout state, global observation information, the sizes of the placed macromodules, and the corresponding netlist are obtained to obtain a one-dimensional feature vector. This one-dimensional feature vector is used as the current state and input into the reinforcement learning layout model, which outputs an action probability prediction matrix and value. Based on the value, an intrinsic reward value is obtained, and the parameters of the reinforcement learning layout model are updated based on the intrinsic reward value.

[0079] In one embodiment, global observation information is obtained based on the initialized layout state and input into a convolutional neural network to extract a global feature embedding vector. The size of the laid-out macromodule and the corresponding netlist are input into a graph neural network, which outputs a graph feature embedding vector. The global feature embedding vector and the graph feature embedding vector are combined into a one-dimensional feature vector.

[0080] In one embodiment, the global observation information includes: current macromodule layout, current global observation layer, and current action mask layer.

[0081] The one-dimensional feature vector is used as the current state and input into the policy network and value network respectively. The policy network outputs the action probability prediction matrix, and the value network outputs the value.

[0082] In one embodiment, chip layout evaluation can only be performed after all macroblocks are placed. The process of placing macroblocks one by one can be considered a scenario with sparse reward values, which often makes reinforcement learning algorithms difficult to train and suffer from poor performance and inefficient sample complexity.

[0083] Inspired by curiosity-based reward mechanisms, this paper provides timely feedback to the reinforcement learning agent by providing intrinsic rewards each time a macromodule is deployed. This method combines the learning efficiency provided by the Experience Replay (ER) algorithm with the exploration performance of the Random Network Distillation (RND) network to achieve more efficient and robust exploration and accelerate model convergence.

[0084] The experience pool replay algorithm stores the value of each step of the reinforcement learning agent's operation in a buffer and then randomly replays this value during reinforcement learning model training. The value mainly includes the actions performed by the consecutive placement of macro modules, the resulting states, and the feedback intrinsic rewards.

[0085] The key idea behind the experience pool replay algorithm is to store important and uncommon batches, giving priority to replaying important values. The random distillation network (RND) involves two networks: a randomly initialized and frozen target network, and a prediction network trained on the current global observation. The feature embedding vectors generated by the target network and the prediction network are φ(·) and φ respectively. ∧ (·). At this time, the intrinsic reward R IntrinsicThe calculation formula is:

[0086] R Intrinsic =‖φ(·)-φ ∧ (·)‖ 2

[0087] in,‖‖ 2 represents the 2-norm.

[0088] As the prediction network continues to place macromodules, its parameters are trained to minimize the expected mean squared error (MSE). As the prediction network trains, the outputs of the two networks for the same repeated state become increasingly similar, resulting in less intrinsic reward. Conversely, if the state differs, the prediction and target networks provide higher intrinsic rewards, encouraging the agent to explore new states.

[0089] Step S4: Based on the initialized layout state and the size of the macroblock to be placed, a fast action mask generation algorithm is used to obtain a current action mask layer. The current action mask layer is superimposed on the action probability prediction matrix to obtain a prediction matrix for the macroblock to be placed. Based on the prediction matrix for the macroblock to be placed, the current action is obtained, i.e., the macroblock to be placed is arranged at the specified location. The layout state after placement is obtained based on the current action.

[0090] In one embodiment, the method of obtaining a current action mask layer using a fast action mask generation algorithm based on the initialized layout state and the size of the macroblock to be placed, and superimposing the current action mask layer with the action probability prediction matrix to obtain a prediction matrix for the macroblock to be placed specifically includes:

[0091] Get the current global observation layer M according to the initialized layout state obs , according to the macro module M to be placed i The size of the width and height are recorded as w i and h i .

[0092] Calculate the macro module M to be placed i The ratio of width to grid width w i / w b , the ratio of height to grid height h i / h b , round down to get the macro module M to be placed i The number of grid cells n occupied on the canvas w and n h . and Where W and H represent the width and height of the wafer canvas, respectively. N is the number of grids the wafer canvas is divided into.

[0093] Initialize the action mask layer M with an all-zero matrix mask, and assign it to the temporary matrix M tmp .

[0094] According to the current global observation layer M obs , the current global observation layer M obs The occupied grid corresponds to M tmp Add 1 to all the grids on the grid to get the grid S occupied by the macro module placed .

[0095] Traverse each unit grid and place the macro module M i The lower left corner of the grid is placed on the current grid, if the grid it occupies will overlap with S placed If a conflict occurs, the grid cannot be placed, corresponding to M tmp Add 1 to the grid value on the grid to get the conflicting grid S collision .

[0096] Traverse the outermost rows and columns of the grid, if the macro module M is to be placed i Number of grid cells occupied n w and n h If it is greater than 1, the macro module to be placed cannot be placed in that row or column, which corresponds to M tmp Add 1 to all the grid values ​​of the rows or columns above to obtain the network S that exceeds the boundary boundary .

[0097] M tmp The area composed of all grids greater than 0 in S invalid .

[0098] M tmp Set all values ​​greater than 1 to 1, and save the rest to 0, and output the current action mask layer M mask .

[0099] In one embodiment, the procedure of the fast action mask generation algorithm is shown in Table 1 below:

[0100] Table 1 Fast motion mask generation algorithm

[0101]

[0102] S invalid =S boundary +S placed +S collision

[0103] Among them, S boundary Defined as the current macro module M i A set of grid coordinates that would extend beyond the grid boundaries if placed on the grid; S placed Defined as the set of grid coordinates occupied by previously placed macromodules; S collisionDefined as if occupied, it will cause M i A set of grid coordinates that overlap with previously placed macroblocks.

[0104] In one embodiment, the algorithm uses m∈[0,1,2,..,N-1] and n∈[0,1,2,..,N-1] to represent the order of the grid cells in the x and y directions, respectively.

[0105] By putting M tmp The occupied grid cells in are set to 1 to get the current action mask layer M mask , determines whether the grid cell can be used for macro module placement. mask Block the positions that cannot be placed, and the reinforcement learning agent will mask The next action to be executed is sampled based on the probability distribution of the remaining legal positions in the policy network. This significantly increases the number of available actions sampled by the policy network and avoids the exponential growth of the solution space. After adopting a new algorithm for generating action masks that allow for partial overlap, a macromodule legalization algorithm is applied to the resulting layout to obtain a high-quality layout.

[0106] Step S5: Replace the initial layout state in step S3 with the layout state after placement, repeat steps S3 and S4 to arrange the macro modules to be placed one by one to the designated positions, until all macro modules are placed, and obtain a preliminary plan for partially overlapping macro layout.

[0107] Step S6: Input the preliminary plan of partially overlapping macro layout into the macro module legalization algorithm to obtain a legal macro module layout plan.

[0108] In one embodiment, the step of inputting the preliminary plan of partially overlapping macro layout into a macro module legalization algorithm to obtain a legal macro module layout plan includes:

[0109] According to the preliminary scheme P of partially overlapping macro layout illgal Initialize the horizontal constraint graph G with the overlapped part in h and vertical constraint graph G v .

[0110] According to the horizontal constraint graph G h and vertical constraint graph G v Calculate the longest horizontal path lp h The longest perpendicular path lp v .

[0111] Randomly adjust the level constraint graph G h and vertical constraint graph G v After adjustment, calculate the adjusted horizontal longest path lp h ' and the longest vertical path lp v '.

[0112] The adjusted horizontal longest path lp h ' and the longest vertical path lp v ' is compared with the width W and height H of the chip respectively until the longest path after adjustment is smaller than the chip size, then the loop is exited and the coordinates of all macro modules are determined.

[0113] Initialize the simulated annealing solver to obtain the total displacement D of all macromodules, the approximate wire length W estimated by the semi-perimeter wire length, and the unusable area A of the macromodule between standard cells, which are used to balance the contributions of displacement, wire length change, and unusable area to the objective function.

[0114] Use the solver based on simulated annealing algorithm to continuously adjust the macro modules in the macro layout. Each adjustment calls the objective function to calculate the target cost. When the adjustment is executed, the newly calculated cost new Less than the previous minimum target cost min When the minimum target cost is reached, the adjustment is applied, otherwise the macro module position is called back. min When it is less than the predetermined threshold, the loop is exited. The macromodule layout at this time is used as the legal macromodule layout scheme P legal .

[0115] In one embodiment, the objective function is calculated as follows:

[0116] Cost = α·D + β·W + γ·A

[0117] Where: α, β, and γ are user-defined penalty coefficients, which are used to balance the contribution of displacement, line length change, and unusable area to the objective function respectively.

[0118] In one embodiment, the procedure of the macromodule legalization algorithm is shown in Table 2:

[0119] Table 2 shows the macro module legalization algorithm

[0120]

[0121] In one embodiment, the total displacement D of all macromodules is calculated as follows:

[0122]

[0123] Where V represents the set of pre-placed macro modules, T represents the set of movable macro modules, (x i ,y i ) represents the current coordinate of the macro module, (x i ′ ,y i ′) represents the adjusted coordinates of the macromodule.

[0124] In one embodiment, the calculation formula of the unusable area A of the macro module between standard cells is as follows:

[0125]

[0126] Where T represents the set of movable macromodules, M i Indicates the macro module to be placed, S channel Denotes the expected macromodule channel spacing, A ij Represents the distance between the i-th macroblock to be placed and the j-th movable macroblock.

[0127] To generate new solutions during the simulated annealing solver, the algorithm uses two operations to perturb the constraint graph. First, the algorithm randomly performs macroblock swap operations, swapping the positions of two macroblocks in the horizontal and vertical graphs. Second, edges are moved in opposite directions to introduce additional variation. These perturbations allow the simulated annealing solver to explore different macroblock layouts within the constraints imposed by the graph. Notably, the macroblock passage space between macroblocks generated by the macroblock legalization algorithm is well controlled. channel The controllability of the macroblocks helps in subsequent steps of model training, such as standard cell placement and routing. The macroblock legalization adjusts the position of the macroblocks to ensure the legality of the placement while minimizing the impact of macroblock movement.

[0128] Step S7: Obtain an external reward according to the legal macromodule layout solution, and update the parameters of the reinforcement learning layout model according to the external reward to obtain an updated reinforcement learning layout model.

[0129] In one embodiment, obtaining an external reward based on a legal macromodule layout solution specifically includes:

[0130] The extrinsic reward is the evaluation feedback on the overall layout quality after reinforcement learning completes the layout of all macro modules and the layout of standard cells. The ultimate goal of chip placement is to minimize indicators such as power consumption, which requires the continued placement of standard cells for evaluation. However, it takes several hours to achieve a single evaluation using industry-standard EDA tools, which is unacceptable for reinforcement learning agents that require tens of thousands of attempts to learn the optimal placement strategy. In order to ensure that the running time of each iteration is within an acceptable and reasonable range, this method uses the most advanced open source layout tool DREAMPlace based on analytical algorithms to place standard cells after each macro module is placed. Macro modules in fixed positions will affect the quality of the subsequent detailed layout, which can be improved over time through training. The combination of reinforcement learning agents and gradient-based optimization layout tools will enhance each other. In addition, DREAMPlace can fully utilize the computing power of the GPU and complete the placement of standard cells within a few seconds. Therefore, this method simplifies the extrinsic reward value and defines it as R Extrinsic =W HPWL To calculate the Half-Perimeter Wirelength (HPWL) of the layout.

[0131]

[0132] Where: x j and y j Respectively The horizontal and vertical coordinates, N netlist Indicates the number of netlists, net indicates a wire net in the netlist, Represents the jth pin in the i-th net.

[0133] Step S8: Repeat steps S3-S7 until the training reaches the set maximum cycle to generate the final reinforcement learning layout model.

[0134] Step S9: pre-process the netlist of the input circuit layout, and input the layout space with pre-placed macro modules and the macro modules to be placed into the final reinforcement learning layout model, so that a high-quality macro layout can be obtained at one time.

[0135] Example 2:

[0136] This embodiment introduces a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the mixed-size macro layout method based on reinforcement learning and macro legalization as described in any one of the embodiments 1 is implemented.

[0137] Example 3:

[0138] This embodiment introduces a computer device, including:

[0139] Memory, used to store instructions.

[0140] The processor is configured to execute the instructions so that the computer device performs the operations of the mixed-size macro layout method based on reinforcement learning and macro legalization as described in any one of the embodiments 1.

[0141] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0142] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0143] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0144] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0145] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A mixed-size macro placement method based on reinforcement learning and macro legalization, characterized by: The following steps are involved: Step S1: Obtain a netlist and a layout space of a circuit to be laid out, divide the layout space into a grid, and obtain a gridded layout space; arrange the pre-placed macro modules in the gridded layout space according to the size and position of the pre-placed macro modules in the netlist to obtain an initialized layout state; Step S2: Obtain a pre-built reinforcement learning layout model; Step S3: Obtain global observation information and the size of the laid-out macromodule and the corresponding netlist according to the initialized layout state, and obtain a one-dimensional feature vector; The one-dimensional feature vector is used as the current state and input into the reinforcement learning layout model, which outputs the action probability prediction matrix and value. The intrinsic reward is obtained based on the value, and the parameters of the reinforcement learning layout model are updated based on the intrinsic reward. Step S4: Based on the initialized layout state and the size of the macroblock to be placed, a fast action mask generation algorithm is used to obtain a current action mask layer. The current action mask layer is superimposed on the action probability prediction matrix to obtain a prediction matrix for the macroblock to be placed. Based on the prediction matrix for the macroblock to be placed, the current action is obtained, i.e., the macroblock to be placed is arranged at a specified position. The layout state after placement is obtained based on the current action. Step S5: replacing the initial layout state in step S3 with the layout state after placement, repeating steps S3 and S4 to arrange the macro modules to be placed one by one to the designated positions until all macro modules are placed, thereby obtaining a preliminary plan for partially overlapping macro layout; Step S6: Inputting the partially overlapping macro layout preliminary plan into the macro module legalization algorithm to obtain a legal macro module layout plan; Step S7: Obtaining an external reward based on the legal macromodule layout solution, and updating the parameters of the reinforcement learning layout model based on the external reward to obtain an updated reinforcement learning layout model; Step S8: Repeat steps S3-S7 until the training reaches the set maximum cycle, generating the final reinforcement learning layout model; The step S3 specifically includes: Based on the initialized layout state, global observation information is obtained and input into the convolutional neural network to extract the global feature embedding vector. The size of the laid out macromodule and the corresponding netlist are input into the graph neural network to output the graph feature embedding vector. The global feature embedding vector and the graph feature embedding vector are merged into a one-dimensional feature vector. Take the one-dimensional feature vector as the current state and input it into the policy network and value network respectively. The policy network outputs the action probability prediction matrix, and the value network outputs the value. Store the value in a buffer, randomly replay the value during reinforcement learning layout model training, input the value into the target network and prediction network respectively, and output the feature embedding vector respectively. , feature embedding vector , according to the feature embedding vector , feature embedding vector Calculating intrinsic rewards ; Based on intrinsic rewards Calculate the loss function of the policy network and update the parameters of the policy network according to the loss function of the policy network to obtain the updated policy network; The intrinsic reward The calculation formula is as follows: ； in, represents the 2-norm; The external rewards obtained according to the legal macro module layout plan specifically include: Obtaining extrinsic rewards for legal macromodule placement solutions using DREAMPlace, an open source layouter based on analytical algorithms; The extrinsic rewards , Indicates the half perimeter length of the layout, calculated as follows: ； in: and Respectively The horizontal and vertical coordinates, Indicates the number of netlists, net indicates a wire net in the netlist, Represents the jth pin in the i-th net.

2. The mixed-size macro placement method based on reinforcement learning and macro legalization according to claim 1, characterized in that: The reinforcement learning layout model includes: a strategy network and a value network, a strategy network Composed of deconvolutional neural networks, value networks It is composed of a fully connected neural network; Among them, the layout state of the macro module is used as the state space, Indicates that a movable macro module is placed as the action space, and express; represents the parameters of the policy network, Parameters representing the value network; Based on the current intrinsic reward in the value, calculate the loss function of the policy network and update the parameters of the policy network according to the loss function of the policy network; According to the external reward, the loss function of the reinforcement learning layout model is calculated, and the parameters of the reinforcement learning layout model are updated according to the loss function of the reinforcement learning layout model.

3. The mixed-size macro placement method based on reinforcement learning and macro legalization according to claim 1, characterized in that: The step S4 specifically includes: Get the current global observation layer according to the initialized layout state , according to the macro module to be placed The size of the width and height are recorded as and ; Calculate the macro modules to be placed Ratio of width to grid width , the ratio of height to grid height , round down to get the macro module to be placed The number of grid cells to occupy on the canvas and and , where W and H represent the width and height of the wafer canvas respectively; N is the number of grids into which the wafer canvas is divided; Initialize the action mask layer with an all-zero matrix , and assign it to the temporary matrix ; According to the current global observation layer , the current global observation layer The occupied grid corresponds to Add 1 to all the grids above to get the grids occupied by the macro module ; Traverse each unit grid and place the macro module to be placed The lower left corner of the grid is placed on the current grid, if the grid it occupies will overlap with If a conflict occurs, the grid cannot be placed, corresponding to Add 1 to the grid value on the grid to get the grid where the conflict occurs ; Traverse the outermost rows and columns of the grid, if a macro module is to be placed Number of grids occupied and If it is greater than 1, the macro module to be placed cannot be placed in that row or column, and the corresponding Add 1 to all the grid values ​​of the rows or columns above to obtain the network that exceeds the boundary ; Will The area composed of all grids greater than 0 in ; Will Set all values ​​greater than 1 to 1, and save the rest to 0, and output the current action mask layer. ; Set the current action mask layer Superimposed with the action probability prediction matrix, the prediction matrix of the macro module to be placed is obtained; According to the prediction matrix of the macro modules to be placed, the current action is obtained, that is, the macro modules to be placed are arranged at the specified position; and the layout state after placement is obtained according to the current action.

4. The mixed-size macro placement method based on reinforcement learning and macro legalization according to claim 1, characterized in that: The step S6 specifically includes: Based on the preliminary plan of partially overlapping macro layout Initialize the horizontal constraint graph for the overlap in and vertical constraint graph ; According to the horizontal constraint diagram and vertical constraint graph Calculate the longest horizontal path The longest perpendicular path ; Randomly adjust the level constraint graph and vertical constraint graph , after adjustment, calculate the adjusted horizontal longest path The longest perpendicular path ; The adjusted horizontal longest path The longest perpendicular path Compare them with the chip's width W and height H respectively until the longest path after adjustment is smaller than the chip size, then exit the loop and determine the coordinates of all macro modules; Initialize the simulated annealing solver to obtain the total displacement D of all macromodules, the approximate line length W estimated by the semi-perimeter line length, and the unavailable area A of the macromodule between standard cells; Use the solver based on simulated annealing algorithm to continuously adjust the macro modules in the macro layout. Each adjustment calls the objective function to calculate the target Cost. Smaller than the previous minimum target When the minimum target is reached, the adjustment is applied, otherwise the macro module position is called back; When it is less than the predetermined threshold, the loop is exited; the macro module layout at this time is used as the legal macro module layout scheme .

5. The mixed-size macro placement method based on reinforcement learning and macro legalization according to claim 4, characterized in that: The calculation formula of the objective function is as follows: ； in: 、 、 are user-defined penalty coefficients used to balance the contributions of displacement, line length change, and unusable area to the objective function.

6. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the computer program implements a mixed-size macro layout method based on reinforcement learning and macro legalization as claimed in any one of claims 1 to 5.

7. A computer device, characterized in that: include: a memory for storing instructions; The processor is configured to execute the instructions so that the computer device performs the operation of the mixed-size macro layout method based on reinforcement learning and macro legalization as claimed in any one of claims 1 to 5.

Images