Generating integrated circuit layouts using neural networks

Abstract

A method, system, and apparatus for generating a computer chip layout, including a computer program encoded on a computer storage medium. One method includes obtaining netlist data for a computer chip, and generating a computer chip layout including placing, at each time step in a sequence including a plurality of time steps, a respective macro node, for each time step, the placing including generating an input representation for the time step, processing the input representation using a node placement neural network having a plurality of network parameters, wherein the node placement neural network is configured to process the input representation according to current values of the network parameters to generate a score distribution over a plurality of locations on a surface of the computer chip, and assigning, using the score distribution, a macro node to be placed at the time step to a location of the plurality of locations.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Patent Application No. 63 / 014,021, filed April 22, 2020, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This specification relates to the use of neural networks for electronic design automation, and more specifically, for generating computer chip layouts. Background Technology

[0004] Computer chip layout is a schematic representation of the arrangement of a portion or all of the circuitry of a computer chip on the surface of the chip (i.e., the chip area).

[0005] A neural network is a machine learning model that uses one or more non-linear units to predict the output from a received input. Some neural networks include one or more hidden layers in addition to the output layer. The output of each hidden layer is used as the input to the next layer in the network (i.e., the next hidden layer or output layer). Each layer of the network generates an output from the received input based on the current values ​​of its corresponding parameter set. Summary of the Invention

[0006] This specification describes a system implemented as a computer program on one or more computers at one or more locations, which generates a chip layout of an integrated circuit. In this specification, the integrated circuit for which the chip layout is generated is referred to as a "computer chip," but it should generally be understood to mean any collection of electronic circuits fabricated on a piece of semiconductor material. The chip layout places each node in a netlist of nodes at a corresponding location on the surface of the computer chip.

[0007] Specific embodiments of the subject matter described in this specification may be implemented, thereby achieving one or more of the following advantages.

[0008] Placement planning is a critical step in the chip design process, involving the placement of chip components on the chip's surface. The placement of components should optimize metrics such as area, total wire length, and congestion. If the placement planning performs poorly in terms of these metrics, the resulting integrated circuit chip will perform poorly. For example, the integrated circuit chip may fail to function, consume excessive power, have unacceptable latency, or exhibit any of a variety of other undesirable characteristics due to poor placement of components on the chip.

[0009] The technology allows for the automatic generation of high-quality chip layout plans with minimal user intervention through the use of the node placement neural network and the training technique. As a specific example, when using distributed training, high-quality (i.e., exceeding human capabilities) layouts can be generated within hours without any human expert involvement.

[0010] Unlike the system described above, traditional layout planning solutions involve a lengthy process requiring significant human involvement and often lasting several weeks. Due to the vast potential space for node layout combinations, traditional automated methods cannot reliably generate high-quality layout plans without consuming excessive computing power and time, without requiring human experts, or neither. However, by effectively utilizing reinforcement learning to train the node layout neural network, this technique can rapidly generate high-quality layout plans.

[0011] Furthermore, compared to integrated circuit chips produced using conventional methods, those produced using this method can have lower power consumption. For a given surface area, it can also have higher computing power, or conversely, for a given amount of computing power, it can be produced using fewer resources.

[0012] Furthermore, when trained as described in this specification—that is, when the encoder neural network is trained through supervised learning and the policy neural network is trained through reinforcement learning—the node placement neural network can be rapidly generalized to new netlists and new integrated circuit chip dimensions. This significantly reduces the amount of computational resources required to generate layouts for new netlists, because generating high-quality layout plans for new netlists requires almost no computationally expensive fine-tuning.

[0013] Details of one or more embodiments of the subject matter described herein are set forth in the following drawings and description. Other features, aspects, and advantages of this subject matter will become apparent from the description, drawings, and claims. Attached Figure Description

[0014] Figure 1 An exemplary layout generation system is shown.

[0015] Figure 2 This illustrates the processing of the node layout neural network at time steps.

[0016] Figure 3 This is a flowchart illustrating an exemplary process for training node layout in a neural network.

[0017] Figure 4 This is a flowchart of an exemplary process for placing macro nodes at a given time step.

[0018] In different accompanying drawings, the same reference numerals and names denote the same elements. Detailed Implementation

[0019] Figure 1 An exemplary layout generation system 100 is illustrated. The layout generation system 100 is an example of a system implemented as a computer program on one or more computers in one or more locations where the systems, components and technologies described below are implemented.

[0020] System 100 receives netlist data 102 from a computer chip, which is a very large-scale integrated circuit (VLSI) chip to be manufactured, comprising multiple integrated circuit components such as transistors, resistors, and capacitors. These integrated circuit components can differ depending on the chip's intended function. For example, the chip can be a dedicated chip, i.e., an application-specific integrated circuit (ASIC), used for machine learning computation, video processing, encryption, or other computationally intensive functions.

[0021] Netlist data 102 is data describing the connectivity of integrated circuit components of a computer chip. Specifically, netlist data 102 specifies the connectivity on a computer chip among multiple nodes, each node corresponding to one or more of the multiple integrated circuit components of the computer chip. That is, each node corresponds to a corresponding appropriate subset of integrated circuit components, and these subsets do not overlap. In other words, netlist data 102 identifies, for each of the multiple nodes, which other nodes (if any) need to be connected to by the node via one or more wires in the manufactured computer chip. In some cases, the integrated circuit components have been clustered, for example, through an external system or by using existing clustering technology, and each node in the netlist data represents a different cluster.

[0022] System 100 generates a final computer chip layout 152 as output, which places some or all of the nodes in netlist data 102 at corresponding locations on the surface of the computer chip. That is, the final computer chip layout 152 identifies the corresponding locations on the surface of the computer chip for some or all of the nodes in netlist data 102 and therefore for the integrated circuit components represented by the nodes.

[0023] As an example, netlist data 102 can identify two types of nodes: nodes representing macro components and nodes representing standard cell components.

[0024] Macro components are large blocks of IC components, such as static random access memory (SRAM) or other memory blocks, represented as individual nodes in a netlist. For example, a node representing a macro component may include nodes that each represent a corresponding instance of SRAM. As another example, a node representing a macro component may include a hard macro consisting of a fixed number of standard cells, such as a macro consisting of a fixed number of instances of a register file. As yet another example, a node representing a macro component may include one or more nodes, each representing a phase-locked loop (PLL) circuit to be placed on the chip. As yet another example, a node representing a macro component may include one or more nodes, each representing a sensor to be placed on the chip.

[0025] Standard cell components are groups of transistors and interconnect structures, such as groups that provide Boolean logic functions (e.g., AND, OR, XOR, XNOR, inverters) or groups that provide storage functions (e.g., flip-flops or latches).

[0026] In some implementations, nodes in the netlist data represent individual standard cell components. In other implementations, nodes in the netlist data represent clustered standard cell components.

[0027] Typically, layout 152 assigns each node to a grid square in an N×M grid that is covered on the chip surface, where N and M are integers.

[0028] In some implementations, the values ​​of N and M are provided to the system as input 100.

[0029] In other implementations, system 100 generates values ​​for N and M.

[0030] For example, system 100 can treat the selection of the optimal number of rows and columns as a binning problem and sort the different combinations of rows and columns by the amount of wasted space generated on the chip surface. System 100 can then select the combination that results in the minimum amount of wasted space as the values ​​of N and M.

[0031] As another example, system 100 may use a mesh generation machine learning model to process inputs derived from netlist data, data characterizing the surface of an integrated circuit chip, or both, the mesh generation machine learning model being configured to process the inputs to generate outputs that define how the surface of the integrated circuit chip is divided into an N×M mesh.

[0032] System 100 includes a node layout neural network 110 and a graphics layout engine 130.

[0033] System 100 uses a node layout neural network 110 to generate a macro node layout 122.

[0034] Specifically, the macro node layout 122 places each macro node (i.e., each node representing a macro) in the netlist data 102 at a corresponding position on the surface of the computer chip.

[0035] By placing the corresponding macro nodes from netlist data 102 at each time step in a sequence of multiple time steps, system 100 generates macro node layout 122.

[0036] In other words, system 100 generates a macro node layout node by node across multiple time steps, where each macro node is placed at a position at a different time step according to the macro node order. The macro node order sorts the macro nodes, and in the macro node order, each node preceding any given macro node is placed before the given macro node.

[0037] At each specific time step in the sequence, system 100 generates an input representation for that specific time step and processes the input representation using a node-layout neural network 110.

[0038] The input representation of a specific time step typically characterizes at least (i) the corresponding position of any macronode on the chip surface preceding the specific macronode to be placed at the specific time step in the macronode sequence, and (ii) the specific macronode to be placed at the specific time step.

[0039] The input representation may also optionally include data characterizing the connectivity between nodes specified in the netlist data 102. For example, for some or all of the nodes, the input representation may characterize one or more other nodes that a node is connected to, based on the netlist. For example, the input representation may represent each connection between any two nodes as an edge connecting the two nodes.

[0040] The following is for reference. Figure 2 A more detailed description of the exemplary input representation is provided.

[0041] In the first time step of the sequence, the input indicates that no node has been placed yet, so for each node in the netlist, it indicates that the node has not yet been placed on the surface of the chip.

[0042] The node-layout neural network 110 is a neural network with parameters (referred to as "network parameters" in this specification) and is configured to process the input representation based on the current values ​​of the network parameters to generate a score distribution, such as a probability distribution or a logarithmic distribution, at multiple locations on the surface of a computer chip. For example, in an N×M grid covered on the chip surface, the distribution can be on top of a grid square.

[0043] Then, system 100 uses the score distribution generated by the neural network to assign the macro nodes to be placed at a specific time step to the locations among multiple locations.

[0044] The following is for reference. Figures 2 to 4 The operation performed by the neural network 110 at a given time step is described in more detail, as well as the placement of nodes at that time step using the score distribution.

[0045] By adding macro nodes one by one to the layout, after the last time step in the sequence, the macro node layout will include the corresponding layouts of all macro nodes in netlist data 102.

[0046] Once system 100 generates macro node layout 122, graphics layout engine 130 generates initial computer chip layout 132 by placing each standard cell at a corresponding position on the surface of a partially placed integrated circuit chip, the integrated circuit chip including macro components represented by macro nodes placed according to the macro node layout, i.e., placed as in macro node layout 122.

[0047] In some implementations, engine 130 clusters standard cells into a set of standard cell clusters (or obtains data identifying already generated clusters), and then uses a graphical layout technique to place each standard cell cluster at a corresponding location on the surface of the partially placed integrated circuit chip. As a specific example, engine 130 can use a partitioning technique based on a normalized minimum cut objective to cluster the standard cells. An example of this technique is hMETIS, described in "Karypis, G. and Kumar, V. A hypergraph partitioning package. In HMETIS, 1998".

[0048] In some other implementations, Engine 130 does not cluster the standard cells, but instead uses graphical layout technology to place each standard cell directly at the corresponding position on the surface of the partially placed integrated circuit chip.

[0049] The graph placement technique can be any suitable technique used to place the nodes of the graph. For example, engine 130 can use a force-based technique, namely force-directed technique. Specifically, when using a force-based technique, engine 130 represents the netlist as a spring system, with the springs applying a force to each node according to the weight x distance formula, such that closely connected nodes attract each other. Optionally, engine 130 also introduces repulsive forces between overlapping nodes to reduce the placement density. After applying all the forces, engine 130 moves the nodes in the direction of the force vector. To reduce oscillations, engine 130 can set a maximum distance for each move. The use of force-directed techniques to place nodes is described in more detail in “Shahookar, K. and Mazumder, P. Vlsi cell placement techniques. ACM Comput. Surv., 23(2):143220, June 1991. ISSN 0360-0300. doi:10.1145 / 103724.103725”.

[0050] In some implementations, system 100 uses the initial layout 132 as the final layout 152.

[0051] In some other implementations, system 100 provides an initial layout 132 as input to legalization engine 150, which adjusts the initial layout 132 to generate a final layout 152.

[0052] Specifically, the legalization engine 150 can generate a legalized integrated circuit chip layout by applying a greedy legalization algorithm to the initial integrated circuit chip layout. For example, the engine 150 can perform a greedy legalization step to capture macros to the nearest legal position while adhering to the minimum gap constraint.

[0053] Optionally, engine 150 can further refine the legalized layout, or it can directly refine the initial layout 132 without generating a legalized layout (e.g., by performing simulated annealing on a reward function). An exemplary reward function will be described in more detail below. As a specific example, engine 150 can perform simulated annealing by applying a hill-climbing algorithm to iteratively adjust the layout in the legalized layout or initial layout 132, thereby generating the final computer chip layout 152. Hill-climbing algorithms and other simulated annealing techniques that can be used to adjust the macro node layout 122 are described in more detail in “S. Kirkpatrick, CD Gelatt, and MPVecchi. Optimization by simulated annealing. SCIENCE, 220(4598):671–680, 1983”. As another example, system 100 can further refine the legalized layout by providing the legalized layout or initial layout 132 to electronic design automation (EDA) software tools for evaluation and fine-tuning, or it can directly refine the initial layout 132 without generating a legalized layout.

[0054] Optionally, system 100 or an external system can then manufacture (produce) a chip (integrated circuit) based on the final layout 152. This integrated circuit may exhibit better performance, for example, having one or more of lower power consumption, lower latency, or smaller surface area compared to integrated circuits designed using conventional design processes, and / or being manufactured using fewer resources. Manufacturing can be performed using any known technology. In some cases, manufacturing a chip based on the final layout may include presenting data identifying the layout to a user to allow the user to modify the final layout 152 prior to manufacturing, or providing the final layout 152 to electronic design automation (EDA) for fine-tuning prior to manufacturing.

[0055] System 100 can receive netlist data 102 through any of a number of methods.

[0056] For example, system 100 can receive netlist data 102 as an upload from a remote user of the system via a data communication network (e.g., using an application programming interface (API) available to system 100). In some cases, system 100 can then provide the final layout 152 to the remote user via the API provided by system 100, for example, for manufacturing a chip based on the final layout 152.

[0057] As another example, system 100 may be part of an electronic design automation (EDA) software tool and may receive netlist data 102 from the tool's user or other components of the tool. In this example, system 100 may provide a final layout 152 for evaluation by other components of the EDA software tool prior to the fabrication of a computer chip.

[0058] Figure 2 This illustrates the processing of the node layout neural network 110 at a given time step.

[0059] As referenced above Figure 1 At each time step during layout generation, the node layout neural network 110 is configured to receive and process input representations to generate score distributions, such as probability distributions or logarithmic distributions, at multiple locations on the surface of the computer chip.

[0060] Typically, the input representation includes at least (i) data representing the corresponding position of the chip surface of any macronode preceding a specific macronode to be placed at a specific time step in the macronode sequence, and (ii) data representing the specific macronode to be placed at a specific time step.

[0061] like Figure 2 As shown, the node layout neural network 110 includes an encoder neural network 210, a policy neural network 220, and a selective value neural network 230.

[0062] The encoder neural network 210 is configured to process the input representation at each specific time step to generate an encoded representation 212 of the input representation. The encoded representation is a numerical representation in a fixed-dimensional space, i.e., an ordered set of a fixed number of numerical values. For example, the encoded representation can be a vector or matrix of floating-point values ​​or other types of numerical values.

[0063] The policy neural network 220 is configured to process the encoded representation 212 at each specific time step to generate a score distribution.

[0064] Typically, the policy neural network 220 can have any suitable architecture that allows the policy neural network 220 to map the encoded representation 212 to the score distribution. For example... Figure 2As shown in the example, the policy neural network 220 is a deconvolutional neural network, which consists of a fully connected neural network followed by a set of deconvolutional layers. The policy neural network 220 may optionally include other types of neural network layers, such as batch normalization layers or other types of normalization layers. However, in other examples, the policy neural network 220 may be, for example, a recurrent neural network, i.e., a neural network comprising one or more recurrent neural network layers, such as Long Short-Term Memory (LSTM) layers, gated recurrent unit (GRU) layers, or other types of recurrent layers, with an output layer that generates positional scores. For example, when the scores are probabilities, the output layer may be a softmax layer.

[0065] In use, the value neural network 230 is configured to process the encoded representation 212 at each specific time step to generate a value estimate, which estimates the value of the current state of the layout at that specific time step. The value of the current state is an estimate of the output of the reward function of the layout generated from the current state (i.e., from the current partial layout). For example, the value neural network 230 may be a recurrent neural network or a feedforward neural network, such as a neural network including one or more fully connected layers.

[0066] This value estimate can be used during the training of the neural network 110, i.e., when using reinforcement learning techniques that rely on the available value estimate. In other words, when the reinforcement learning technique used to train the node layout neural network requires a value estimate, the node layout neural network 110 also includes a value neural network 230 that generates the value estimate required by the reinforcement learning technique.

[0067] The training node layout neural network 110 will be described in more detail below.

[0068] like Figure 2 As illustrated in the example, the input feature representation includes a netlist (“macrofeature”), “netlist graph data” that represents the connectivity between nodes in the netlist as edges connecting two corresponding nodes in the netlist data, and a corresponding vectorized representation of some or all nodes in a “current macro id” that identifies the macro node placed at a specific time step. As a specific example, the input feature representation may include corresponding vectorized representations of macro nodes only, macro nodes and standard cell clusters, or macro nodes and standard cell nodes.

[0069] Each vectorized representation characterizes the corresponding node. Specifically, for each node that has been placed, the vectorized representation includes data identifying the node's location on the chip surface, such as the coordinates of the node's center or some other specified portion of the node, and for each node that has not yet been placed, the vectorized representation includes data indicating that the node has not yet been placed, such as default coordinates indicating that the node has not yet been placed on the chip surface. The vectorized representation may also include other information characterizing the node, such as the node's type, node dimensions, such as the node's height and width, etc.

[0070] exist Figure 2 In the example, encoder neural network 210 includes graph encoder neural network 214, which processes vectorized representations of nodes in a netlist to generate (i) netlist embeddings of the vectorized representations of nodes in the netlist, and (ii) current node embeddings representing macronodes to be placed at a specific time step. An embedding is a numerical representation in a fixed-dimensional space, i.e., an ordered set of a fixed number of numerical values. For example, an embedding can be a vector or matrix of floating-point values ​​or other types of numerical values.

[0071] Specifically, the graph encoder neural network 214, for example, randomly initializes the corresponding edge embedding of each edge in the netlist data and initializes the corresponding node embedding of each node in the netlist data, that is, makes the node embedding equal to the corresponding vectorized representation of the node.

[0072] Then, the graph encoder neural network 214 repeatedly updates the node and edge embeddings by updating the embeddings at each of the multiple message passing iterations.

[0073] After the final message passing iteration, the graph encoder neural network 214 generates netlist embeddings and current node embeddings based on node and edge embeddings.

[0074] As a specific example, neural network 214 can generate netlist embeddings by combining edge embeddings after the final message-passing iteration. For example, the system can compute netlist embeddings by applying a reduced averaging function to the edge embeddings after the final message-passing iteration.

[0075] As another specific example, neural network 214 can set the current node embedding of the current node to be equal to the current node embedding after the last message passing iteration.

[0076] Neural network 214 can use any of a variety of messaging techniques to update node and edge embeddings at each messaging iteration.

[0077] As a specific example, at each message passing iteration, neural network 214 updates the edge embedding of each edge using the corresponding node embeddings of the two nodes connected by the edge.

[0078] At each iteration, to update the embedding of a given edge, network 214 generates a summary representation based at least on the node embeddings of the two nodes connected by the edge, and processes the summary representation using a first fully connected neural network to generate an updated edge embedding for the given edge. In some implementations, each edge has the same weight in the netlist data, i.e., 1. In some other implementations, each edge is associated with a corresponding weight in the netlist data, and the system generates the summary representation based on the node embeddings of the two nodes connected by the edge and the weight associated with the edge in the netlist data. The weight of each edge can, for example, be learned jointly with the training of the neural network.

[0079] To update the embedding of a given node at a given message passing iteration, the system updates the node embedding using the corresponding edge embeddings of the edges connected to the node. For example, the system could average the corresponding edge embeddings of the edges connected to the node.

[0080] The input feature representation may also optionally include “netlist metadata” that characterizes the netlist of the node. Netlist metadata may include any appropriate information characterizing the netlist. For example, this information may include any information about the underlying semiconductor technology (horizontal and vertical routing capacity), the nets (edges) in the netlist, the total number of macro and standard cell clusters, the canvas size (i.e., the size of the chip surface), or the number of rows and columns in the mesh.

[0081] When the input feature representation includes netlist metadata, the encoder neural network 210 may include a fully connected neural network that processes the metadata to generate netlist metadata embeddings.

[0082] The encoder neural network 210 generates encoded representations based at least on netlist embeddings of the vectorized representations of nodes in the netlist and current node embeddings representing the macronodes to be placed at a specific time step. When the encoder neural network 210 also generates netlist metadata embeddings, the system also uses these embeddings to generate encoded representations.

[0083] As a specific example, neural network 210 can concatenate netlist embeddings, current node embeddings, and netlist metadata embeddings, and then process the concatenation using a fully connected neural network to generate an encoded representation.

[0084] The system also tracks the density of locations on the chip, i.e., the density of squares in the grid. Specifically, the system maintains a density value for each location, indicating how much of that location is occupied. When a node is placed at a given location, the density value for that location is set to 1 (or set to a different maximum value indicating that the location is fully occupied). When no node is placed at a given location, the density value for that location indicates the number of edges passing through that location. The density value for a given location can also reflect congestion, such as clock bands or other structures blocking certain portions of the chip surface, by setting the values ​​for those locations to 1.

[0085] Once the policy neural network 220 generates a score distribution at a time step, the system uses that density to generate a modified score distribution, and then uses the modified score distribution to assign nodes corresponding to the time steps. Specifically, the system modifies the score distribution by setting the score at any location with a density value that satisfies (e.g., exceeds) a threshold to zero.

[0086] For example, the system can assign a node to the position with the highest score in the modified score distribution, or sample positions from the modified score distribution such that the probability of each position being selected is equal to that probability, and then assign the node to the sampled position.

[0087] This is Figure 2 The value in the diagram is represented as a grid density mask that can be applied to the score distribution, i.e., a mask in which the value is zero at any location with a density above a threshold and 1 at any location with a density not above a threshold, to generate a modified score distribution.

[0088] As a specific example, the threshold can be equal to 1, and the system can set the score to zero for any location where a node has been placed (i.e., a location with a density value of 1). As another example, the threshold can be less than 1, instructing the system to also set the score to zero for any location where there are no nodes but too many wires pass through it (i.e., the number of wires associated with the location is higher than the threshold).

[0089] In order to use the neural network 110 to generate high-quality layouts, the system (or other system) trains the neural network on training data.

[0090] In some implementations, the system uses reinforcement learning to train the neural network 110 end-to-end to maximize the received expected reward, as measured by a reward function. The reward function typically measures the quality of the layout generated using the node-layout neural network 110. References will follow below. Figure 3 A more detailed description of the reward function.

[0091] However, training a neural network solely through reinforcement learning can lead to poor generalization to new netlist data, so the system may have to retrain the neural network each time a new netlist is received, for example, from scratch.

[0092] To improve the generalization of neural network 110, the system can train encoder neural network 210 through supervised learning and then train policy neural network 220 through reinforcement learning. See below for reference. Figure 3 This training process will be described in more detail.

[0093] Figure 3 This is a flowchart of an exemplary process 300 for training a node layout neural network. For convenience, process 300 is described as being executed by a system of one or more computers located in one or more locations. For example, a properly programmed layout generation system (e.g., Figure 1 The layout generation system 100 can execute process 300.

[0094] The system can execute process 300 to train the node layout neural network, that is, to determine the training values ​​of the network parameters.

[0095] In some implementations, the system distributes the training of the node placement neural network among many different workers, i.e., among many homogeneous or heterogeneous computing devices, such as devices using CPUs, GPUs, or ASICs to perform training computations. In some of these implementations, part or all of step 300 can be executed in parallel by many different workers operating asynchronously to accelerate the training of the node placement neural network. In other implementations, different workers operate synchronously to execute part or all of the steps of process 300 in parallel to accelerate the training of the neural network.

[0096] The system can use process 300 to train any node layout neural network, including (i) an encoder neural network configured to receive and process an input representation at each of a plurality of time steps to generate an encoder output, the input representation including data representing the current state of the layout of a netlist of nodes on the surface of the integrated circuit chip at time step, and (ii) a policy neural network configured to receive and process an encoded representation generated based on the encoder output generated by the encoder neural network at each of a plurality of time steps to generate a score distribution at multiple locations on the surface of the integrated circuit chip.

[0097] An example of this type of neural network is the reference above. Figure 2 The aforementioned neural network.

[0098] Another example of such a neural network is described in patent application No. 16 / 703,837, filed on December 4, 2019, entitled “GENERATINGINTEGRATED CIRCUIT FLOORPLANS USING NEURAL NETWORKS”, the entire contents of which are incorporated herein by reference.

[0099] The system obtains supervised training data (step 302).

[0100] The supervised training data includes (i) multiple training input representations, each representing the layout of the corresponding netlist of a node, and (ii) for each training input representation, the target value of a reward function that measures the quality of the layout of the corresponding netlist of a node.

[0101] More specifically, the reward function measures certain characteristics of the generated layout, which, when optimized, cause the chip manufactured using the generated layout to exhibit good performance, for example, in one or more aspects of power consumption, heat generation, or timing performance.

[0102] Specifically, the reward function includes corresponding terms of one or more features. For example, when there are multiple terms, the reward function can be the sum or weighted sum of the multiple terms.

[0103] As an example, the reward function could include a wire length metric, which measures the wire length of wires on the chip surface. This metric is higher when the wire length between nodes on the chip surface is shorter.

[0104] For example, the wire length can be a negative of the Manhattan distance or another distance metric between all adjacent nodes on the chip surface.

[0105] As another example, the wire length metric can be based on the half-perimeter wire length (HPWL), which approximates the wire length using half the perimeter of the bounding boxes of all nodes in the netlist. When calculating the HPWL, the system can assume that all wires leaving the standard cell cluster originate from the center of the cluster. Specifically, the system can calculate the HPWL for each edge in the netlist and then calculate the wire length metric as the negative of the normalized sum of the HPWLs of all edges in the netlist.

[0106] Including a term for measuring wire length in the reward function has the advantage that the write length roughly measures the wiring cost and is also related to other important metrics such as power and timing.

[0107] As another example, the reward function could include a congestion metric, where a higher congestion measure is used when congestion is lower on the surface of a computer chip. Congestion is a measure of the difference between available wiring resources in a given region (not necessarily a contiguous region) on a chip and the actual wires that cross that region. For example, congestion could be defined as the ratio of wires crossing a region in a generated layout to available wiring resources (e.g., the maximum number of wires that can cross that region). As a specific example, a congestion metric could track the wire density across the horizontal and vertical edges of a surface.

[0108] Specifically, the system can utilize a routing model from a netlist (e.g., net bounding box, upper L, lower L, A*, minimum spanning tree, or actual routing net). Based on this routing model, a congestion metric can be calculated by determining the ratio of available cabling resources in the layout to the cabling estimate from the routing model at each location on the surface.

[0109] As another example, the system can compute a congestion metric by tracking the vertical and horizontal distribution of each location separately, for example, as described above. The system can then smooth the congestion estimate by running convolutional filters (e.g., a 5x1 convolutional filter or filters of different sizes), depending on each direction in the vertical and horizontal directions and the number of locations in both. The system can then compute the congestion metric as the negative of the average of the top 10%, 15%, or 20% of the congestion estimates.

[0110] As another example, the reward function can include timing terms; that is, terms measuring the timing of digital logic are higher when chip performance is better (e.g., the reward function uses correspondingly higher values ​​for individual chip layouts that require less time to perform certain computational tasks). Static timing analysis (STA) can be used to measure the timing or performance of a layout. This measurement can include calculating level delays (including internal cell delays and wire delays) on the logic path and finding the critical path, which determines the maximum speed at which the clock can run for safe operation. For a practical view of timing, logic optimization may be necessary to adapt to paths that become longer or shorter as node placement progresses.

[0111] As another example, the reward function may include one or more terms that measure the power or energy the chip will consume, i.e., one or more terms that are higher when the chip will consume less power.

[0112] As another example, the reward function may include one or more items that measure the area of ​​the layout, i.e., one or more items that are higher when the area occupied by the layout is low.

[0113] In some cases, the system receives supervised training data from other systems.

[0114] In other cases, the system generates supervised training data. As a specific example, different node layout neural networks can be trained on different netlists at different time points based on different node layout neural networks, using multiple training input representations. Figure 2 The node placement neural network described above generates the output of a node placement neural network with a simpler structure. This ensures that the placement has different qualities.

[0115] For example, the system can generate supervised training data by selecting a different set of accelerator netlists and then generating a layout for each netlist. To generate different layouts for each netlist, the system can train a simpler policy network with various congestion weights (ranging from 0 to 1) and random seeds on the netlist data (e.g., through reinforcement learning), and collect snapshots of each layout during the policy training process. Each snapshot includes a representation of the layout and a reward value generated by a reward function used for the layout. An untrained policy network starts with random weights and generates low-quality layouts, but as the policy network is trained, the quality of the generated layouts improves, allowing the system to collect different datasets with layouts of varying quality.

[0116] In some implementations, the training input representation can completely represent the final layout, that is, the layout of all macro nodes in the corresponding netlist. In other implementations, the training input representation can represent the layout at various stages of the layout generation process, that is, some representations can represent a partial layout with only a subset of macro nodes placed.

[0117] The system trains the encoder neural network in conjunction with the reward prediction neural network on supervised training data through supervised learning (step 304).

[0118] The reward prediction neural network is configured to receive the encoder output generated by the encoder neural network from the training input representation for each training encoder input, and process the encoded representation to generate a predicted value of the reward function of the layout represented by the training input representation.

[0119] A reward prediction neural network can be, for example, a fully connected neural network that receives the encoder output and processes it to generate reward predictions. When the encoder neural network has the above reference... Figure 2 In the aforementioned architecture, the encoder output can be a cascade of netlist graph embeddings and metadata embeddings.

[0120] For example, the system can train an encoder neural network and a reward prediction neural network to optimize an objective function (e.g., mean squared error loss), which measures the error between the target value of the reward function and the predicted value of the reward function used to train the input representation for a given training representation.

[0121] The system then trains a policy neural network using reinforcement learning to generate a score distribution, thereby obtaining a layout that maximizes the reward function (step 306). The system can use any of a variety of reinforcement learning techniques to train the node layout neural network.

[0122] For example, the system can use policy gradient techniques (such as REINFORCE or Proximity Policy Optimization (PPO)) for training. In these cases, when the neural network includes a value prediction neural network, the value predictions generated by the value neural network can be used to compute a baseline value, which modifies the reward function value when the gradient of the reinforcement learning loss function is computed.

[0123] When training a policy neural network through reinforcement learning, the system can keep the parameter values ​​of the encoder neural network fixed to the values ​​determined by training on supervised training data.

[0124] Specifically, when training a policy neural network through reinforcement learning on a given netlist for a given chip, the system can use a placement neural network to place macronodes one by one in the given netlist, as described above. After placing the macronodes, the system can place standard cell nodes as described above to determine the final layout. Then, the system can, for example, compute the required number of reward functions for the final layout as described above, and train the placement neural network through reinforcement learning using the reward values, the macronode layout, and the score distribution generated by the placement neural network. Therefore, although the placement neural network is only used to place macronodes, the reward value is calculated only after the standard cell nodes have also been placed, thereby ensuring that the placement neural network generates a macronode layout that still allows for a high-quality layout of the standard cell nodes.

[0125] The system receives new netlist data (step 308).

[0126] In some implementations, the system uses a trained node placement neural network to generate an integrated circuit layout for the new netlist data. That is, the system places the corresponding nodes from the new netlist data at each time step in multiple time steps using the score distribution generated by the trained node placement neural network (step 310). In other words, the system generates the layout of the new netlist data without further training of the node placement neural network.

[0127] In other words, the encoder neural network is trained through supervised learning, and the policy neural network is trained through reinforcement learning. The system trains the node layout neural network to generalize to the new netlist without any additional training.

[0128] In some other implementations, to further improve the quality of the layout generated for the new netlist, the system first fine-tunes the trained node layout neural network on the new netlist data using reinforcement learning (step 312), and then uses the fine-tuned node layout neural network to generate an integrated circuit layout for the new netlist data (step 314), as described above. During fine-tuning, the system can use the same reinforcement learning techniques as described above, and depending on the implementation, the parameter values ​​of the encoder neural network can be kept fixed or updated during fine-tuning.

[0129] Figure 4 This is a flowchart of an exemplary process 400 for placing macro nodes at a given time step. For convenience, process 400 is described as being executed by a system of one or more computers located in one or more locations. For example, a properly programmed layout generation system (e.g., Figure 1 The layout generation system 100 can execute process 400.

[0130] The system can execute process 400 for each time step in the time step sequence to place each macro node according to the macro node order.

[0131] In some implementations, the system receives the macro node order along with the netlist data as input.

[0132] In some other implementations, the system can generate the macro node order based on netlist data.

[0133] As an example, the system can sort macro nodes by size, for instance, by decreasing the size and using topological sorting to break connections. By placing larger macros first, the system reduces the likelihood that there will be no feasible layout for subsequent macros. Topological sorting helps the policy network learn to place connected nodes close to each other.

[0134] As another example, the system can process inputs derived from netlist data using a macro node order prediction machine learning model, which is configured to process inputs derived from netlist data to generate outputs with a defined macro node order.

[0135] As yet another example, the node placement neural network can be further configured to generate probability distributions on macronodes. The system can then dynamically generate the macronode order by selecting the macronode to be placed in the next time step after a specific time step, based on the probability distributions on the macronodes, for each specific time step across multiple time steps. For example, the system could select the macronode with the highest probability that has not yet been placed.

[0136] The system generates an input representation based on the netlist data. The input representation at least characterizes (i) the corresponding position on the chip surface of any macronode preceding the specific macronode to be placed at a given time step in the macronode sequence, and (ii) the specific macronode to be placed at the given time step (step 402). Optionally, the input representation may also include additional information about the nodes in the netlist, netlist metadata, or both. (Refer to above) Figure 2 An example of the input representation is described.

[0137] The system uses a node-layout neural network with multiple parameters (“network parameters”) to process the input representation (step 404). The node-layout neural network is configured to process the input representation based on the current values ​​of the network parameters to generate a score distribution at multiple locations on the surface of the integrated circuit chip.

[0138] The system uses the score distribution to assign the macro node to be placed at a specific time step to a location among multiple locations (step 406). As mentioned above, the system can modify the score distribution based on the tracking density of the current layout, i.e., by setting the score of any location with a density value that meets a threshold to zero, and then selecting a location from the modified score distribution.

[0139] In some implementations, the system can use additional information to further modify the score distribution.

[0140] Specifically, as described above, in some implementations, neural networks are trained on multiple different layouts for multiple different netlists on multiple different chips. This may require the neural network to generate score distributions on chip surfaces of different sizes. That is, when multiple locations are grid squares from an N×M grid covering the surface of an integrated circuit chip, different chips can have different N and M values. To address this issue, the system can configure the neural network to generate scores on a fixed-size maxN×MaxM grid. When the N value of the current chip is less than maxN, the system can set the score of the extra rows to zero. Similarly, when the M value of the current chip is less than maxM, the system can set the score of the extra columns to zero.

[0141] This specification uses the term "configured to" in relation to systems and computer program components. A system of one or more computers configured to perform a specific operation or action means that software, firmware, hardware, or a combination thereof are installed on the system that causes the system to perform those operations or actions during operation. One or more computer programs to be configured to perform a specific operation or action means that the one or more programs include instructions that, when executed by a data processing device, cause the device to perform the operation or action.

[0142] Embodiments of the subject matter and functional operation described in this specification may be implemented in digital electronic circuits, in computer software or firmware embodied in a tangible manner, in computer hardware (including the structures disclosed herein and their structural equivalents), or in one or more combinations thereof. Embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transient storage medium, for executing or controlling the operation of a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or one or more combinations thereof. Alternatively or additionally, program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated as coded information for transmission to a suitable receiver device for execution by the data processing device.

[0143] The term "data processing device" refers to data processing hardware, including all types of devices, apparatuses, and machines for processing data, such as programmable processors, computers, or multiple processors or computers. The device may also be or further include special-purpose logic circuitry, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits). In addition to hardware, the device may optionally include code that creates the execution environment for computer programs, such as code constituting processor firmware, protocol stacks, database management systems, operating systems, or combinations thereof.

[0144] A computer program, which may also be referred to or described as a program, software, software application, application, module, software module, script, or code, may be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages; and it may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A program may, but does not need to, correspond to a file in a file system. A program may be stored as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), as a single file dedicated to the program in question, or as multiple coordinating files (e.g., files storing one or more modules, subroutines, or code portions). A computer program may be deployed to execute on a single computer or multiple computers located at a single site or distributed across multiple sites and interconnected via a data communication network.

[0145] In this specification, the term "database" is used broadly to refer to any collection of data that does not need to be structured in any particular way, or does not need to be structured at all, and can be stored on storage devices in one or more locations. Thus, for example, an index database may include multiple collections of data, each of which can be organized and accessed in different ways.

[0146] Similarly, in this specification, the term "engine" is used broadly to refer to a software-based system, subsystem, or process programmed to perform one or more specific functions. Typically, an engine can be implemented as one or more software modules or components installed on one or more computers at one or more locations. In some cases, one or more computers will be dedicated to a particular engine; in other cases, multiple engines can be installed and run on the same computer or on multiple computers.

[0147] The processes and logic flows described in this specification can be executed by one or more programmable computers, which execute one or more computer programs to perform functions by manipulating input data and generating output. The processes and logic flows can also be executed by dedicated logic circuitry (such as FPGAs or ASICs) or by a combination of dedicated logic circuitry and one or more programmable computers.

[0148] A computer suitable for executing computer programs can be based on a general-purpose microprocessor or a special-purpose microprocessor or both, or any other type of central processing unit (CPU). Typically, the CPU receives instructions and data from read-only memory or random access memory or both. The components of a computer are the CPU for making or executing instructions and one or more memory devices for storing instructions and data. The CPU and memory may be supplemented by or incorporated into special-purpose logic circuitry. Typically, a computer will also include, or be operatively coupled to, devices for receiving data from or transferring data to one or more mass storage devices (e.g., magnetic disks, magneto-optical disks, or optical disks) for storing data, or both. However, a computer does not necessarily need to have these devices. Furthermore, a computer can be embedded in other devices such as mobile phones, personal digital assistants (PDAs), mobile audio or video players, game controllers, global positioning system (GPS) receivers, or portable storage devices such as universal serial bus (USB) flash drives, to name a few.

[0149] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, such as semiconductor memory devices, such as EPROM, EEPROM and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0150] To provide interaction with the user, embodiments of the subject matter described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user, and a keyboard and pointing device (e.g., a mouse or trackball) from which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback, such as visual, auditory, or tactile feedback; and input from the user can be received in any form, including acoustic, voice, or tactile input. Furthermore, the computer can interact with the user by sending documents to and receiving documents from the device used by the user; for example, by sending a webpage to a web browser on the user's device in response to a request received from a web browser. Additionally, the computer can interact with the user by sending text messages or other forms of messages to a personal device (e.g., a smartphone running a messaging application) and subsequently receiving response messages from the user.

[0151] Data processing devices used to implement machine learning models may also include, for example, dedicated hardware accelerator units for handling the general and computationally intensive parts of machine learning training or production, namely inference and workloads.

[0152] Machine learning frameworks can be used to implement and deploy machine learning models, such as the TensorFlow framework, the Microsoft Cognitive Toolkit framework, the Apache Singa framework, or the Apache MXNet framework.

[0153] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes backend components, such as a data server, or middleware components, such as an application server, or frontend components, such as a client computer with a graphical user interface, a web browser, or an application that allows the user to interact with the implementation of the subject matter described in this specification, or any combination of one or more such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication (e.g., a communication network) of any form or medium. Examples of communication networks include local area networks (LANs) and wide area networks (WANs), such as the Internet.

[0154] A computing system may include clients and servers. Clients and servers are typically geographically separated and usually interact via a communication network. The client-server relationship arises from computer programs running on their respective computers and having a client-server relationship with each other. In some embodiments, the server transmits data (e.g., HTML pages) to a user device, for example, to display data to a user interacting with the device acting as a client and to receive user input from the user. Data generated on the user device (e.g., the result of user interaction) may be received from the device at the server.

[0155] While this specification contains numerous details of specific implementations, these should not be construed as limiting the scope of any invention or the scope that may be claimed, but rather as descriptions of features that may be specific to particular embodiments of a particular invention. Certain features described in this specification in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, while the features described above may be described as functioning in certain combinations, and even claimed in this way initially, in some cases one or more features from the claimed combination may be removed, and the claimed combination may be intended for sub-combinations or variations thereof.

[0156] Similarly, although the operations are described in a specific order in the accompanying drawings and listed in the claims, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all the shown operations to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the program components and systems can generally be integrated into a single software product or encapsulated in multiple software products.

[0157] Specific embodiments of this subject matter have been described. Other embodiments fall within the scope of the following claims. For example, the actions listed in the claims can be performed in a different order and the desired result can still be achieved. As an example, the processes described in the figures do not necessarily require the specific order or sequence shown to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous.

Claims

1. A method executed by one or more computers, the method comprising: Obtain netlist data of an integrated circuit chip, wherein the netlist data specifies connectivity between multiple nodes on the integrated circuit chip, each node corresponding to one or more of multiple integrated circuit components of the integrated circuit chip, and wherein the multiple nodes include macro nodes representing macro components and standard cell nodes representing standard cell components; and Generating an integrated circuit chip layout, wherein the integrated circuit chip layout places each node in the netlist data at a corresponding position on the surface of the integrated circuit chip, including: According to the macronode sequence, corresponding macronodes are placed at each of multiple time steps to generate a macronode layout on the surface of the chip. For each specific time step among the multiple time steps, the placement includes: An input representation is generated based on the netlist data, the input representation at least characterizing (i) the corresponding position of any macronodes on the surface of the chip before the specific macronode to be placed at the specific time step in the macronode sequence, and (ii) the specific macronode to be placed at the specific time step; The input representation is processed using a node-layout neural network with multiple network parameters, wherein the node-layout neural network is configured to process the input representation based on the current values ​​of the network parameters to generate a score distribution at multiple locations on the surface of the integrated circuit chip; and Using the score distribution, the macronodes to be placed at the specific time step are assigned to positions among the plurality of locations; and Generating an initial integrated circuit chip layout includes placing each standard cell at a corresponding position on the surface of a partially placed integrated circuit chip, the partially placed integrated circuit chip including macro components represented by macro nodes placed according to the macro node layout.

2. The method according to claim 1, wherein, Placing each of the standard units includes: The standard units are clustered to generate multiple clusters of standard units; and The standard cells of each cluster are placed at their respective positions on the surface of the integrated circuit chips where the section is located using graphical layout techniques.

3. The method according to claim 1, wherein, Placing each of the standard units includes: Graphical layout techniques are used to place each standard cell at its corresponding position on the surface of the integrated circuit chip in the portion.

4. The method according to claim 2, wherein, The graphic layout technology is based on force.

5. The method of claim 1, further comprising, before placing the corresponding macro node: The macro nodes are generated in the specified order.

6. The method according to claim 5, wherein, Generating the macro node order includes processing the input derived from the netlist data using a macro node order prediction machine learning model, the macro node order prediction machine learning model being configured to process the input derived from the netlist data to generate an output that defines the macro node order.

7. The method according to claim 5, wherein, The order in which the macro nodes are generated includes: The macro components are sorted according to their size.

8. The method according to claim 1, wherein, The plurality of locations include grid squares from an N×M grid covering the surface of the integrated circuit chip.

9. The method of claim 8, further comprising: A mesh generation machine learning model is used to process inputs derived from the netlist data, data characterizing the surface of the integrated circuit chip, or both. The mesh generation machine learning model is configured to process the inputs to generate an output that defines how the surface of the integrated circuit chip is divided into the N×M mesh.

10. The method according to claim 1, wherein: The node layout neural network includes: An encoder neural network configured to receive the input representation and process the input representation to generate an encoded representation; and A policy neural network configured to process the encoded representation to generate the score distribution.

11. The method of claim 10, wherein The input representation includes the corresponding vectorized representation of each node in the netlist data, and The encoder neural network includes a graph encoder neural network that processes the vectorized representation of nodes in the netlist data to generate (i) a netlist embedding of the vectorized representation of nodes in the netlist data, and (ii) a current node embedding representing the macro node to be placed at the specific time step.

12. The method according to claim 11, wherein, The netlist data represents the connectivity between nodes as edges, where each edge connects two corresponding nodes in the netlist data, and wherein the graph encoder neural network is configured as follows: For each edge in the netlist data, the corresponding edge is embedded and initialized; For each node in the netlist data, the corresponding node embedding of the node in the netlist data is initialized to be equal to the corresponding vectorized representation of the node; and Repeat the following two steps: For each edge in the netlist data, the edge embedding is updated using the corresponding node embeddings of the two nodes connected by the edge, and For each node in the netlist data, the node embedding of the node is updated using the corresponding edge embedding of the edges connected to the node.

13. The method according to claim 12, wherein, Updating the edge embedding of the edge includes: A summary representation is generated based on at least the node embeddings of the two nodes connected by the edge, and the summary representation is processed using a first fully connected neural network to generate the updated edge embedding.

14. The method according to claim 13, wherein, Generating the summary representation includes: A second fully connected neural network is used to process the embedding of each node to generate the corresponding intermediate embedding for each node, and The summary representation is generated by combining at least the corresponding intermediate embeddings.

15. The method according to claim 13, wherein: Each edge is associated with a weight in the netlist data, and Generating the summary representation includes generating the summary representation based on the node embeddings of the two nodes connected by the edge and the weights associated with the edge in the netlist data.

16. The method according to claim 11, wherein, Updating the node embedding of a node using the corresponding edge embedding of the edges connected to the node includes: The edge embeddings of the edges connected to the node are averaged.

17. The method according to claim 12, wherein, The netlist embedding is a combination of edge embeddings after repeating the two-step operation.

18. The method according to claim 12, wherein, The current node embedding is the node embedding of the macro node to be placed at the specific time step after the two-step operation is repeated.

19. The method according to claim 11, wherein: For each node that has been placed, the vectorized representation includes data identifying the location of the node on the surface of the chip, and For each node that has not yet been placed, the vectorized representation includes data indicating that the node has not yet been placed.

20. The method according to claim 11, wherein, The encoder neural network is configured to generate the encoded representation based at least on the netlist embedding of the vectorized representation of the nodes in the netlist data and the current node embedding representing the macro node to be placed at the specific time step.

21. The method according to claim 10, wherein, The encoder neural network has been trained through supervised learning.

22. The method according to claim 10, wherein, The policy neural network has been trained through reinforcement learning.

23. The method according to claim 1, wherein, Assigning the node to a location among the plurality of locations using the score distribution includes: Generate a modified score distribution that sets the score to zero for each location on the chip surface where the density exceeds a threshold, based on the corresponding location of any macronode preceding the particular macronode in the macronode sequence; and The nodes are assigned using the modified score distribution.

24. The method according to claim 23, wherein, Assigning the nodes using the modified score distribution includes: The node is assigned to the position with the highest score in the modified score distribution.

25. The method according to claim 23, wherein, Assigning the nodes using the modified score distribution includes: Positions are sampled from the modified score distribution, and The node is assigned to the sampled location.

26. The method according to any one of claims 1-25, wherein, Generating the integrated circuit chip layout further includes: A valid integrated circuit chip layout is generated by applying a greedy validation algorithm to the initial integrated circuit chip layout.

27. The method according to claim 26, wherein, Generating the integrated circuit chip layout further includes: The integrated circuit chip layout plan is generated by performing simulated annealing on the reward function starting from the legalized integrated circuit chip layout.

28. The method according to any one of claims 5-7, wherein, The node placement neural network is further configured to generate probability distributions on the macro nodes, wherein the generation order of the macro nodes includes: for each specific time step among the plurality of time steps, selecting a macro node to be placed in the next time step after the specific time step based on the probability distributions on the macro nodes.

29. A method for training a node placement neural network, the node placement neural network comprising: An encoder neural network is configured to receive, at each of a plurality of time steps, an input representation including data representing the current state of the layout of a netlist of nodes on the surface of an integrated circuit chip at said time step, and to process said input representation to generate an encoder output. A policy neural network, configured to receive, at each of the plurality of time steps, an encoded representation generated based on an encoder output generated by the encoder neural network, and to process the encoded representation to generate a score distribution at a plurality of locations on the surface of the integrated circuit chip, the method comprising: Obtain supervised training data, wherein the supervised training data includes: Multiple training input representations, each representing the corresponding layout of the corresponding netlist for a node, and For each training input representation, the corresponding objective value of the reward function that measures the quality of the layout of the corresponding netlist of the node; and The encoder neural network is trained jointly with the reward prediction neural network on the supervised training data through supervised learning, wherein the reward prediction neural network is configured to receive, for each trained encoder input, an encoder output generated by the encoder neural network based on the training input representation, and process the encoded representation to generate a predicted value of the reward function for the layout represented by the training input representation.

30. The method according to claim 29, wherein, The reward function includes a higher wire length metric when the wire length between nodes on the surface of the chip is shorter.

31. The method according to claim 29, wherein, The reward function includes a higher congestion metric when the congestion on the surface of the integrated circuit chip is low.

32. The method of claim 29, further comprising: The policy neural network is trained using reinforcement learning to generate a score distribution that leads to a layout that maximizes the reward function.

33. The method according to claim 32, wherein, The node layout neural network further includes a value neural network configured to receive, at each of the plurality of time steps, an encoded representation generated by the encoder neural network and process the encoded representation to generate a value estimate, wherein training the policy neural network includes jointly training the policy neural network with the value neural network using an augmentation technique that uses the value estimate.

34. The method according to claim 32, wherein, Training the policy neural network by reinforcement learning includes keeping the parameter values ​​of the encoder neural network fixed to values ​​determined through training on the supervised training data.

35. The method of claim 29, further comprising: After training the node placement neural network, the trained node placement neural network is used to generate an integrated circuit layout for new netlist data, including placing the corresponding nodes from the new netlist data at each of multiple time steps using the score distribution generated by the trained node placement neural network.

36. The method of claim 29, further comprising: After training the node layout neural network: Receive new netlist data; The trained node layout neural network is fine-tuned on the new netlist data using reinforcement learning; and Generating an integrated circuit layout for the new netlist data using the fine-tuned node layout neural network includes placing corresponding nodes from the new netlist data at each of multiple time steps using a score distribution generated by the fine-tuned node layout neural network.

37. The method according to claim 29, wherein, The encoder neural network includes a graph encoder neural network, and the encoder output includes an embedding of a netlist generated by the graph encoder neural network.

38. The method according to claim 37, wherein, The encoder neural network includes a metadata embedding neural network that embeds metadata associated with the netlist, and wherein the encoder output includes the embedded metadata.

39. The method according to claim 37, wherein, The encoded representation includes at least the embedding of the netlist and the embedding of the current node from the netlist of the node to be placed at the time step generated based on the embedding of the current node generated by the neural network of the graph encoder.

40. The method according to claim 29, wherein, Training the encoder neural network jointly with the reward prediction neural network on the supervised training data through supervised learning includes: training the encoder neural network and the reward prediction neural network to optimize an objective function, the objective function measuring the error between a target value of the reward function and a predicted value of the reward function for the training input representation, given a training representation.

41. The method according to any one of claims 29 to 40, wherein, The layout, represented by the multiple training inputs, is generated based on the outputs of the different node layout neural networks at different time points during the training of the different node layout neural networks.

42. An integrated circuit chip having a plurality of integrated circuit components, the plurality of integrated circuit components being disposed on the surface of the integrated circuit chip according to an arrangement formed by a process comprising a corresponding operation of the method of any one of claims 1 to 28 or 35 to 36.

43. A non-transitory computer-readable storage medium storing one or more instructions, which, when executed by one or more computers, cause the one or more computers to perform a corresponding operation of the method according to any one of claims 1 to 41.

44. A system comprising one or more computers and one or more storage devices storing instructions, wherein the instructions, when executed by the one or more computers, cause the one or more computers to perform a corresponding operation of the method according to any one of claims 1 to 41.

45. A method for producing integrated circuit chips, comprising: The method according to any one of claims 1 to 28 or 35 to 36 generates an integrated circuit chip layout, and the integrated circuit chip is manufactured according to the layout.

Images