Ai-enabled circuit synthesis

Abstract

Some embodiments of the invention provide an Al-enabled method for integrated circuit (IC) architecture discovery, circuit topology and parameter optimization. The method of some embodiments explores the high dimensional design space beyond human intuition and extracting circuit features, configurations and designs unidentifiable to human experts in a variety of tasks. The method of some embodiments is used for RFIC architecture discovery, circuit topology and parameter optimization. For instance, the method of some embodiments provides an automated design flow for RFICs (e.g., RF or mmWave IC) that allows: (1) architecture selection, (2) circuit topology optimization, (3) parameter optimization, and (4) inverse design EM synthesis in a non-intuitive design space. In some embodiments, the method automates the synthesis of high-performance power amplifier (PA) circuits for transmitter systems, leveraging a combination of reinforcement learning (RL) and inverse design to optimize respectively active and passive components in the design. Some embodiments of the invention provide an Al-enabled method for integrated circuit (IC) architecture discovery, circuit topology and parameter optimization. The method of some embodiments explores the high dimensional design space beyond human intuition and extracting circuit features, configurations and designs unidentifiable to human experts in a variety of tasks. The method of some embodiments is used for RFIC architecture discovery, circuit topology and parameter optimization. For instance, the method of some embodiments provides an automated design flow for RFICs (e.g., RF or mmWave IC) that allows: (1) architecture selection, (2) circuit topology optimization, (3) parameter optimization, and (4) inverse design EM synthesis in a non-intuitive design space. In some embodiments, the method automates the synthesis of high-performance power amplifier (PA) circuits for transmitter systems, leveraging a combination of reinforcement learning (RL) and inverse design to optimize respectively active and passive components in the design.

Description

AI-ENABLED CIRCUIT SYNTHESISCLAIM OF PRIORITY TO PRIOR APPLICATION

[0001] This application claims priority to U.S. Provisional Patent Application 63 / 702,977, filed October 3, 2024, and titled “Reinforcement Learning Enabled Inverse Synthesis of Radio-Frequency ICs". U.S. Provisional Patent Application 63 / 702,977 is hereby incorporated by reference.STATEMENT REGARDING FEDERALLY-SPONSORED RESEARCH OR DEVELOPMENT

[0002] This invention was made with government support under Grant No. W911NF- 21-1-0314 awarded by the U.S. Army Research Office. The government has certain rights in the invention.BACKGROUND

[0003] Next generation wireless systems are expected to enable new wireless sensor, imaging, and communication systems for smart cities, autonomous vehicles, robotics, and cyber-physical systems. However, the design of these electronic systems is extremely complex, time-consuming, and expensive. This has impeded innovation and as a result, it is very challenging for companies to relatively quickly design a complex chip with low cost.

[0004] Unlike digital systems, radio frequency integrated circuits (RFICs) suffer from strong electromagnetic (EM) interactions between all passive (matching networks, antennas, filters) and active (amplifiers, mixers) elements. Today, RFIC design (e.g., RF design or millimeter wave, mmWave, design) is a complex iterative design process that involves codesign of active circuits and passive EM structures, including matching networks (MN), combiners, splitters, hybrids, baluns, switches, diplexers, beamforming networks, antennas, and the like. Design of high-frequency circuits and EM structures has historically relied on intuitive and analytical approaches with starting template architectures. However, such preselected topologies are not close to achieving the optimal performance in the space of all possible circuit and EM topologies.

[0005] Consider the design of a typical RFIC, such as a mmWave power amplifier (PA) illustrated in Figure 1. In this typical design approach, the design decision starts from output power requirements that determine the transistor sizes given the supply voltage. For efficient generation of high output power that requires a high impedance transformation ratio, power combining may be necessary. Optimizing power combining and MN design is done through a series of trial-and-error processes taking losses, size and bandwidth into account. This iterativePTON.POOOIPCT 1process is repeated for driver cells, inter-stage matching, and input splitters, and again with extracted layouts and EM simulations. This approach not only limits the design space to a small set of pre-fixed templates, but the design time can also be significant.

[0006] Attempts at automation with ad-hoc optimization and metaheuristic algorithms have not been able to address the scale of the design space in which such systems operate. For instance, in recent years, algorithmic synthesis of analog and RF circuits has been investigated. A common approach is to train a neural network (NN) on randomized parameter sweeps around a known point. However, this requires generation of a diverse dataset which is difficult and time-consuming, and sweeping around a known point automatically limits the search space.

[0007] More recently, circuit synthesis has been viewed in the framework of a sequential decision process, leading to usage of reinforcement learning (RL) algorithms. RL is a specific type of machine learning in which an agent learns to make sequential decisions by interacting with an environment, receiving rewards or penalties for its actions, to maximize its cumulative reward over time. Unlike supervised learning, RL. doesn't rely on labeled examples but learns through trial-and-error, making it ideal for tasks like game playing, robotics, and optimizing complex systems where the optimal solution isn't known beforehand.

[0008] Observing the circuit and its performance at a time step, the model decides how to tune the parameters to move closer to a satisfactory design. This process repeats, leading to a gradual convergence, with model learning built in. RL has several advantages - first, the model learns directly through its interactions with the circuit, which circumvents the need for users to generate a dataset ahead of time. Additionally, RL relies on stochastic processes to switch between exploitative and explorative actions. By automatically exploring the action space, RL models are well suited for searching highly non-convex spaces.

[0009] However, prior implementation of RL for circuit design has focused only on digital and analog circuits. RF circuits suffer from strong EM coupling and parasitic effects, which means schematic-level optimization is less reliable. In addition, RF circuits require resonant matching topologies, which are also difficult to accurately design. As a result, RL has not been proven for successful RFIC synthesis yet.PTON.POOOIPCT 2BRIEF SUMMARY

[0010] Some embodiments of the invention provide an Al-enabled method for integrated circuit (IC) architecture discovery, circuit topology, and parameter optimization. The method of some embodiments explores the high dimensional design space beyond human intuition and extracts circuit features, configurations, and designs unidentifiable to human experts in a variety of tasks.

[0011] The method of some embodiments performs Al-driven synthesis of an IC that has active and passive circuit components. For the IC, the method of some embodiments receives a target first description of the active and passive circuit components of the IC. The method uses a set of one or more Al-trained processes (1) to iteratively examine different configurations of the active and passive circuit components, and (2) to produce a synthesized second description of the IC that includes an optimal configuration of the active and passive circuit components identified through the iterative examinations. In some embodiments, an AI- trained process uses an Al model that is trained through a learning process (e.g., a supervised or unsupervised learning process, or a reinforcement learning process).

[0012] In some embodiments, the target first description includes description of electromagnetic (EM) structures each formed by a set of passive circuit components and having a desired frequency response. Examples of EM structures include matching networks (MN), combiners, splitters, hybrids, baluns, switches, diplexers, beamforming networks, and / or antennas. In these embodiments, the optimal configuration of the passive circuit components in the produced synthesized second description includes an optimal configuration of the EM structures.

[0013] The target first description in some embodiments includes a code-based description of the IC, while the synthesized second description comprises a circuit representation of the IC. The method of some embodiments converts the code-based description into an encoded description of the IC circuit components. This encoded description is in a format that allows one or more of the Al-trained processes to process the target circuit description.

[0014] The method outputs the synthesized second description of the IC as a synthesized representation of the IC for subsequent processing by a subsequent electronic design automation (EDA) tool. The EDA tool is part of an overall EDA process that includes a set of physical design layout tools (e.g., floor planner, placer, global and / or detailed routers, and / or compactor) that from a synthesized representation of the IC define the physical design layout of the circuit components of the IC over a substrate (e.g., semiconductor substrate).PTON.P0001PCT 3

[0015] Some embodiments use one or more machine-trained networks to implement one or more of the Al-driven processes. In some embodiments, each machine-trained network includes several interconnected processing nodes each performing computations based on one or more parameters trained through a machine learning process (e.g., a supervised or unsupervised learning process or a reinforcement learning process). The set of machine-trained networks in some embodiments includes one or more neural networks. Each neural network includes several interconnected neurons with each neuron performing computations based on parameters (e.g., weight values, bias values, etc.) trained through a machine learning process.

[0016] In some embodiments, the set of machine-trained networks includes a first machine-trained network that is used iteratively to identify different sets of modifications to perform on the circuit components of the IC. The first machine-trained network identifies the different sets of modifications by exploring numerous candidate sets of modifications in a design space based on (1) a set of constraints that define viable candidate sets of modifications to identify, and (2) an objective function that quantifies a value that expresses a benefit or cost for performing the candidate set of modifications.

[0017] For instance, the method in some embodiments iteratively (1) uses a first machine-trained network to identify different sets of candidate modifications to the circuit components of the IC in view of the set of constraints, (2) for each set of identified candidate modifications, quantifies a value that expresses a benefit or cost for performing the set of modifications, and (3) accepts or rejects the set of modifications based on the quantified value. The first machine trained network in some embodiments is a policy network of a reinforcement learning (RL) engine.

[0018] To quantify the cost or benefit value for each set of candidate modifications, the method in some embodiments uses a simulator to simulate the operation of at least a portion of the IC after the identified set of modifications is performed on the IC components and to produce the quantified value for the identified set of modifications based on the simulated operation. During inference time, the policy network evaluates the values quantified for the different sets of modifications. On the other hand, during training, the method of some embodiments uses a critic network to evaluate the values quantified for the different sets of modifications and to produce an error value (e.g., temporal difference (TD) error value) that guides the policy network’s training.

[0019] In some embodiments, the first machine-trained network identifies modifications to the active circuit components of the IC. The method in some of these embodiments uses a second machine-trained network to identify and perform different sets ofPTON.POOOIPCT 4modifications to passive circuit components of the IC. For instance, for each set of activecircuit modifications that are performed on the IC (e.g., based on the first machine-trained network’s identification of the modification set and its acceptance of this set based on the quantified cost or benefit value for this set), the method of some embodiments uses a second machine-trained network to iteratively (1) identify and evaluate different sets of modifications to passive circuit components of the IC, and (2) from the evaluated passive circuit component modifications, select and perform one of the passive circuit component modifications.

[0020] To identify a set of modifications to a set of passive circuit components, the method of some embodiments uses an inverse design process, which receives a set of desired properties and characteristics for the passive circuit components and then uses an optimization process to generate a predicted solution. The optimization process in some embodiments is a genetic process (also called genetic algorithm), such as an evolutionary process (also called evolutionary algorithm). In some such embodiments, the second machine-trained network implements an EM emulator that predicts, for each set of modifications identified by the inverse design process, a frequency response of a set of circuits (such as EM structures) formed by a set of passive circuit components to quantify quality of the set of modifications.

[0021] The method of some embodiments is used for RFIC architecture discovery, circuit topology and parameter optimization. For instance, the method of some embodiments provides an automated design flow for RFICs (e.g., RF or mmWave IC) that allows: (1) architecture selection, (2) circuit topology optimization, (3) parameter optimization, and (4) inverse design EM synthesis in a non-intuitive design space.

[0022] In some embodiments, the method automates the synthesis of high-performance power amplifier (PA) circuits for transmitter systems, leveraging a combination of reinforcement learning (RL) and inverse design to optimize respectively active and passive components in the design. Given a set of target specifications (desired specifications), the method of some embodiments searches over design parameters including topology, device sizes, and voltage / current bias points to find an optimal solution. In sum, the automated design flow of some embodiments provides an end-to-end RFIC Al-enabled synthesis with both active and passive optimization, which in automated manner can take the RFIC from specifications to layout with fabricated and measured results. This automated design flow drastically reduces the total design time by eliminating unnecessary iterative design processes.

[0023] The preceding Summary is intended to serve as a brief introduction to some embodiments of the invention. It is not meant to be an introduction or overview of all inventive subject matter disclosed in this document. The Detailed Description that follows and thePTON.POOOIPCT 5Drawings that are referred to in the Detailed Description will further describe the embodiments described in the Summary as well as other embodiments. Accordingly, to understand all the embodiments described by this document, a full review of the Summary, Detailed Description, the Drawings and the Claims is needed. Moreover, the claimed subject matters are not to be limited by the illustrative details in the Summary, Detailed Description and the Drawing.PTON.POOOIPCT 6BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The novel features of the invention are set forth in the appended claims. However, for purposes of explanation, several embodiments of the invention are set forth in the following figures.

[0025] Figure 1 illustrates the design of a typical mmWave power amplifier (PA).

[0026] Figure 2 illustrates an EDA process that uses an Al-driven synthesizer of some embodiments of the invention.

[0027] Figure 3 conceptually illustrates an RFIC synthesizer of some embodiments.

[0028] Figure 4 conceptually illustrates a more detailed implementation of an RL engine of an RFIC synthesizer of some embodiments.

[0029] Figure 5 conceptually illustrates the design space of some embodiments for the Al-enabled synthesis operation.

[0030] Figure 6 conceptually illustrates the operations of the simulator and the optimization criteria evaluator to determine the reward for a given set of circuit parameters and constraints.

[0031] Figure 7 illustrates an example of embedding of a circuit into the format of an MDP with S, A, R vectors.

[0032] Figure 8 illustrates how the design parameters specified in the state enable topology search.

[0033] Figure 9 conceptually illustrates another example of topology search, for larger-scale circuit parameters rather than parameters that define the topology of a single stage.

[0034] Figure 10 conceptually illustrates a process for training the RL engine.

[0035] Figure 11 conceptually illustrates an inverse design synthesizer of some embodiments.

[0036] Figure 12 conceptually illustrates two examples of pixel structures representing synthesized passive components.

[0037] Figures 13 and 14 conceptually illustrate how a simulator of some embodiments uses the information from inverse synthesis to modify its output in some embodiments (which in turn modifies the rewards).

[0038] Figure 15 conceptually illustrates an example that correlates training time with successful designs and design / inference time.

[0039] Figure 16 conceptually illustrates the viability of the RFIC synthesis technique of some embodiments.PTON.POOOIPCT 7

[0040] Figure 17 conceptually illustrates the parameters that define a design stage, including transmission-line lengths.

[0041] Figure 18 conceptually illustrates an example of a passive component structure synthesized via the RL engine.

[0042] Figure 19 conceptually illustrates an example of a detailed process for designing and manufacturing an IC.

[0043] Figure 20 conceptually illustrates a computer system with which some embodiments of the invention are implemented.PTON.POOOIPCT 8DETAILED DESCRIPTION

[0044] In the following detailed description of the invention, numerous details, examples, and embodiments of the invention are set forth and described. However, it will be clear and apparent to one skilled in the art that the invention is not limited to the embodiments set forth and that the invention may be practiced without some of the specific details and examples discussed.

[0045] Some embodiments of the invention provide an Al-enabled method for integrated circuit (IC) architecture discovery, circuit topology and parameter optimization. The method of some embodiments explores the high dimensional design space beyond human intuition and extracts circuit features, configurations, and designs unidentifiable to human experts in a variety of tasks. The method of some embodiments is implemented by an Al-driven synthesizer that synthesizes an IC with active and passive circuit components. Through its automated exploration of the design space, this synthesizer identifies an optimal synthesized circuit representation of the IC that has an optimal combination of active and passive circuit components.

[0046] Figure 2 illustrates an EDA process 202 that uses one such Al-driven synthesizer 200 of some embodiments of the invention. For an IC design that includes both active and passive circuit components, this synthesizer performs an automated exploration of various solutions in the design space (without any input from a designer during its exploration) to produce a synthesized circuit representation of the IC design that has optimal implementations of active and passive circuit components. In other words, this synthesizer concurrently optimizes the implementations of the active and passive circuit components in its synthesized output.

[0047] As shown, the synthesizer 200 includes a user interface (UI) 212, a specification re-encoder 214, and one or more automated synthesizing processes 216. The UI 212 in some embodiments includes one or more interfaces, such as application programming interfaces (APIs), graphical user interfaces (GUIs), etc. Circuit designers interact with these interfaces through one or more services, such as webservers, that connect with the computers of the circuit designers through a network, such as a local area network, wide area network, or network of networks (such as the Internet), or a combination thereof.

[0048] Through the UI 212, the synthesizer of some embodiments receives and stores in a storage (e.g., a non-volatile storage) a target first description 222 of the IC design from one or more circuit designers. As shown, this design has both active and passive circuit components. Several automated processes 216 of the synthesizer then operate on this targetPTON.P0001PCT 9circuit specification to produce a synthesized circuit representation of the IC design described by the target specification. This synthesized circuit representation is described in a synthesized second description 224 of the IC.

[0049] One or more of these automated processes 216 are Al-trained processes. The synthesizer 200 uses these Al-trained processes (1) to iteratively examine different configurations of the active and passive circuit components and (2) to produce the synthesized second description 224 of the IC that includes an optimal configuration of the active and passive circuit components identified through the iterative examinations.

[0050] In some embodiments, an Al-trained process uses an Al model 220 that is trained through a learning process (e.g., a supervised or unsupervised learning process, or a reinforcement learning process). The Al models 220 (also referred to as machine learning (ML) models), in some embodiments, are each collectively represented by a set of configurable parameters that are trained (learned) through the learning (supervised, unsupervised, or reinforcement) process. The trained Al models 220 can generate predicted outputs for a given, previously unseen input. These Al models 220 may include machine-trained networks (e.g., neural networks), genetic and / or evolutionary algorithms, decision trees, etc.

[0051] In some embodiments, the target first description 222 includes description of electromagnetic (EM) structures of the IC design. Each EM structure comprises a set of passive circuit components and has a desired frequency response. Examples of EM structures include matching networks (MN), combiners, splitters, hybrids, baluns, switches, diplexers, beamforming networks, and / or antennas. In some embodiments, the optimal configuration of the passive circuit components in the produced synthesized second description 224 includes an optimal configuration of the EM structures.

[0052] The target first description in some embodiments includes a code-based description of the IC, while the synthesized second description comprises a circuit representation of the IC. The specification re-encoder 214 of the synthesizer 220 in some embodiments converts the code-based description 222 into an encoded description 215 of the IC circuit components. This encoded description is in a format that allows one or more of the Al-trained processes to interpret and process the target circuit description.

[0053] In some embodiments, there is not a 1 :1 correspondence between circuit components in the target first description 222 of the IC design and the synthesized second description 224 of the IC design. For instance, in some embodiments, the target first description 222 specifies a set of target characteristics for the synthesized IC design, while the synthesized second description 224 specifies a set of actual active and passive components.PTON.POOOIPCT 10

[0054] The synthesizer outputs the synthesized second description 224 of the IC as a synthesized representation of the IC for subsequent processing by a subsequent electronic design automation (EDA) tool in the EDA process 202. In different embodiments, different EDA tools can serve as the subsequent EDA tool that processes the synthesized second description 224.

[0055] Figure 2 illustrates one example of such subsequent tools. In this example, the subsequent tools include physical design tools 230 that define a physical design layout for the circuits specified in the synthesized second description. The physical design layout specifies the physical layout of the IC, i.e., arrangement of the circuit components on a substrate (e.g., semiconductor substrate) and the interconnection between these components. The physical design tools include floorplanners and placers 232 that define the position of the circuit components in the design layout, global and detailed routers 234 that define interconnect lines (routes) that connect the circuit components, and compactors 236 that compact the design layout. While Figure 2 shows the output of the synthesizer 216 being provided to a set of physical design tools 230, one of ordinary skill will realize that in other embodiments the output of the synthesizer is supplied to another synthesizer or a post-synthesizer processor for performing post processing operations.

[0056] Some embodiments use one or more machine-trained networks to implement one or more of the Al-driven processes. In some embodiments, each machine-trained network includes several interconnected processing nodes each performing computations based on one or more parameters trained through a machine learning process (e.g., supervised or unsupervised learning process or a reinforcement learning process). In some embodiments, the set of machine-trained networks includes one or more neural networks. Each neural network includes several interconnected neurons with each neuron performing computations based on parameters (e.g., weight values, bias values, etc.) trained through a machine learning process.

[0057] In some embodiments, the set of machine-trained networks includes a first machine-trained network that is used iteratively to identify different sets of modifications to perform on the circuit components of the IC. The first machine-trained network identifies the different sets of modifications by exploring numerous candidate sets of modifications in a design space based on (1) a set of constraints that define viable candidate sets of modifications to identify and (2) an objective function that quantifies a value that expresses a benefit or cost for performing the candidate set of modifications.

[0058] For instance, the synthesizer in some embodiments iteratively (1) uses a first machine-trained network to identify different sets of candidate modifications to the circuitPTON.POOOIPCT 11components of the IC in view of the set of constraints, (2) for each set of identified candidate modifications, quantifies a value that expresses a benefit or cost for performing the set of modifications, and (3) accepts or rejects the set of modifications based on the quantified value. The first machine trained network in some embodiments is a policy network of a reinforcement learning (RL) engine.

[0059] To quantify the cost or benefit value for each set of candidate modifications, the method in some embodiments uses a simulator to simulate the operation of at least a portion of the IC after the identified set of modifications is performed on the IC and to produce the quantified value for the identified set of modifications based on the simulated operation. During inference time, the policy network evaluates the values quantified for the different sets of modifications. On the other hand, during training, the method of some embodiments uses a critic network to evaluate the values quantified for the different sets of modifications and to produce an error value (e.g., temporal difference (TD) error value) that guides the policy network’s training.

[0060] In some embodiments, the first machine-trained network identifies modifications to the active circuit components of the IC. The synthesizer in some of these embodiments uses a second machine-trained network to identify and perform different sets of modifications to passive circuit components of the IC. For instance, for each set of activecircuit modifications that are performed on the IC (e.g., based on the first machine-trained network’s identification of the modification set and its acceptance of this set based on the quantified cost or benefit value for this set), the synthesizer of some embodiments uses a second machine-trained network to iteratively (1) identify and evaluate different sets of modifications to passive circuit components of the IC, and (2) from the evaluated passive circuit component modifications, select and perform one of the passive circuit component modifications.

[0061] To identify a set of modifications to a set of passive circuit components, the synthesizer of some embodiments uses an inverse design process, which receives a set of desired properties and characteristics for the passive circuit components and then uses an optimization process to generate a predicted solution. The optimization process in some embodiments is a genetic process (also called genetic algorithm), such as an evolutionary process (also called evolutionary algorithm). In some such embodiments, the second machine- trained network implements an EM emulator that predicts, for each set of modifications identified by the inverse design process, a frequency response of a set of circuits (such as EM structures) formed by a set of passive circuit components to quantify quality of the set of modifications.PTON.POOOIPCT 12

[0062] The method of some embodiments is used for RFIC architecture discovery, circuit topology and parameter optimization. For instance, the method of some embodiments provides an automated design flow for RFICs (e.g., RF or mmWave IC) that allows for (1) architecture selection, (2) circuit topology optimization, (3) parameter optimization, and (4) inverse design EM synthesis in a non-intuitive design space.

[0063] In some embodiments, the method automates the synthesis of high-performance power amplifier (PA) circuits for transmitter systems, leveraging a combination of reinforcement learning (RL) and inverse design to optimize respectively active and passive components in the design. Given a set of target specifications (desired specifications), the method of some embodiments searches over design parameters including topology, device sizes, and voltage / current bias points to find an optimal solution. In sum, the automated design flow of some embodiments provides an end-to-end RFIC Al-enabled synthesis with both active and passive optimization, which in automated manner can take the RFIC from specifications to layout with fabricated and measured results. This automated design flow drastically reduces the total design time by eliminating unnecessary iterative design processes.

[0064] Figure 3 conceptually illustrates an RFIC synthesizer 300 of some embodiments. For a target RFIC specification, the RFIC synthesizer (1) explores the design space to select an optimal architecture, circuit topology, set of circuit parameters, and EM structures for the RFIC design that implements the target RFIC specification, and (2) provides an RFIC synthesized circuit description 224 as its output. The target RFIC description in some embodiments is a code-based description, while the RFIC synthesized circuit description 224 is a circuit representation of the RFIC.

[0065] As shown, the RFIC synthesizer 300 includes (1) a reinforcement learning (RL) engine 305 with one or more machine-trained networks 307 to primarily explore and optimize active circuit components of the RFIC, and (2) an inverse design synthesizer 310 with a second machine-trained network 315 to primarily explore and optimize configuration of passive circuit components of the RFIC. To produce the RFIC synthesized circuit description, the RFIC synthesizer 300 in some embodiments performs multiple iterations between the RL engine 305 and the inverse design synthesizer 310, with each such iteration including multiple iterations of the RL engine operations followed by multiple iterations of the inverse design synthesizer operations. In other embodiments, each iteration of the RL engine operations includes one or more iterations of the inverse design synthesizer operations. Still other embodiments replace the inverse design synthesizer operations with a lookup table for a subset of the iterations ofPTON.POOOIPCT 13the RL engine operations, only using the inverse design synthesizer operations at the end of the RFIC synthesizer operations.

[0066] It should be noted that, in some embodiments, the RL engine 305 specifically only explores and optimizes parameters of the active circuit components while the inverse design synthesizer 310 only explores and optimizes the configuration of the passive circuit components. In other embodiments, the RL engine 305 may explore and optimize additional parameters that do not specifically configure the active components and / or the inverse design synthesizer 310 may explore and optimize additional configuration besides the passive components.

[0067] In each iteration of the RFIC synthesizer 300 operation, the RL engine 305 in some embodiments uses a RL policy network (e.g., one of the machine-trained networks 307) to identify different sets of candidate modifications to the active circuit components of the IC in view of the set of constraints. For each set of identified candidate modifications to the active circuit components, the RL engine uses a simulator (not shown) to quantify a value that expresses a benefit or cost for performing the set of modifications. The RL engine in some embodiments then uses this value to determine whether it should end its current iterative search of the design space for an optimal design and configuration of the active circuit components. If not, the RL policy network uses this value to produce its next set of candidate modifications to the active circuit components.

[0068] Otherwise, when the RL engine uses this value to determine that it should end its current iterative search of the design space (i.e., uses this value to determine that it has reached a set of one or more criteria to end its current exploration of the design space for its current iterative exploration), the RL engine outputs its current RFIC design to the inverse design synthesizer 310. The outputted current RFIC design in some embodiments includes the most optimal set of modifications to the active circuit components that the RL engine 305 discovered in its current iterative exploration.

[0069] The inverse design synthesizer 310 then performs multiple iterations of its operation to finds an optimal configuration for the passive circuit components in the current RFIC design. The inverse design synthesizer 310 receives a set of desired properties and characteristics for the passive circuit components and then uses an optimization process to generate a set of modifications to a set of passive circuit components (e.g., lengths of the transmission lines). The optimization process in some embodiments is a genetic process (genetic algorithm), such as an evolutionary process (evolutionary algorithm). For each set of modifications identified by the optimization process, the inverse design synthesizer 310 thenPTON.POOOIPCT 14uses a trained neural network that implements an EM emulator to predict a frequency response of the set of circuits (such as EM structures) formed by the set of passive circuit components to quantify quality of the set of modifications.

[0070] After performing several iterations and reaching a set of one or more criteria to end its current exploration of the design space for its current iterative exploration, the inverse design synthesizer 310 outputs to the RL engine 305 an optimal solution that it identified through its current iterations. In addition to this solution, the inverse design synthesizer 310 in some embodiments also outputs a reward value to the RL engine 305 that quantifies the quality of the overall current optimal solution identified by the RL engine 305 and the inverse design synthesizer 310.

[0071] Based on this reward value and a set of one or more end criteria, the RL engine 305 then determines whether it should end the operation of the RFIC synthesizer 300 (i.e., the combined search of the design space by the RL engine 305 and the inverse design circuit 310). If not, the RL engine 305 starts another iterative process for exploring the design space to identify a new optimal solution for active circuit components, factoring in the additional data provided by the RL engine 305 (i.e., the indication of the optimal solution for the passive components and / or the reward value associated with that optimal solution).

[0072] Otherwise, when the RL engine 305 determines that it should end the operation of the RFIC synthesizer 300, the RL engine 305 outputs one of the optimal complete solutions that includes one of the optimal active circuit configurations identified by the RL engine 305 and an optimal passive circuit configuration identified by the inverse design circuit 310 for this optimal active circuit configuration. This solution has been optimized for both active and passive circuit components, e.g., has an optimal circuit architecture, circuit topology, set of circuit parameters, and EM structures for an RFIC design that implements the RFIC specification.

[0073] Figure 4 conceptually illustrates a more detailed implementation of an RL engine 400 of an RFIC synthesizer of some embodiments. This synthesizer synthesizes high- performance power amplifier (PA) circuits for transmitter systems, leveraging a combination of reinforcement learning and inverse design to optimize respectively active and passive components in the design. Specifically, the RFIC synthesizer uses the RL engine 400 to optimize one set of circuits, primarily the active component circuits (e.g., transistors, etc.), and then uses the inverse design engine 310 to optimize electromagnetic passive components, offering more optimal designs through irregular, non-intuitive geometries. For both the RLPTON.POOOIPCT 15engine and the inverse design process, the synthesizer uses Al-driven processes in some embodiments.

[0074] To find the optimal design and configuration of active and passive components for a PA circuit, the RFIC synthesizer iterates several times through the RL engine and inverse design operations. During each iteration, the RL engine iteratively searches through the design space to identify an optimal active circuit design and configuration, while the inverse design iteratively searches through the design space to find (for the currently identified optimal active circuit design) optimal passive circuit configured structures, e.g., optimal multi-port EM structures, such as combiner, splitter, inter-stage MN, with desired S-parameters for the currently identified optimal active circuit configuration. The inverse design of this synthesizer will be described below by reference to Figure 11.

[0075] As shown in Figure 4, the active circuit designs and configurations explored and outputted by the RL engine 400 include different PA architectures (e.g., combiner architectures, gain-stage architectures, etc.) and different cell and driver topologies (e.g., different common-emitter (CE) topologies, common-base (CB) topologies, stacks, etc.) with varying parameters, biasing, etc. As shown, the RL engine 400 includes a policy network 405, a circuit configurator 410, a simulator 415, and an optimization criteria evaluator 420.

[0076] In some embodiments, the policy network is a neural network that is trained by using a critic network 425 during a training process as further described below. For a set of target specifications (desired specifications) and a current state of the PA circuit, the policy network iteratively identifies different active circuit designs and configurations. Based on its training, the policy network’s iterative exploration of the design space searches over numerous designs and configurations, such as design parameters including topology, device sizes, and voltage / current bias points to find an optimal solution.

[0077] During runtime (inference time), the policy network 405 is a decision making agent that interacts with the environment (i.e., simulator 415 in this example) by identifying a set of actions (e.g., adjusting circuit parameters) and then observing rewards (e.g., observing changes in performance of the circuit and how well the circuit conforms to the target specifications). In each iteration during one of its iterative explorations of the design activecircuit space, the policy network 405 in some embodiments identifies a different set of candidate modifications to the active circuit components of the IC in view of a set of constraints. To identify each set of candidate modifications, the policy network 405 takes in the current PA-circuit state and performance (as indicated by the rewards 422) and provides an action vector 426 that is a collection of probability distributions for different actions to thePTON.POOOIPCT 16circuit configurator 410. In the first iteration of each iterative exploration, the current PA- circuit is the input current state 424, while in each successive iterations, the current PA-circuit is a current state 428 retrieved from the state storage 430.

[0078] In this manner, each iteration of the policy network 405 of some embodiments operates on a 3-tuple (S, A, R) set, where ‘ S’ is the current design state, ‘A’ is the set of actions, ‘R’ is the reward function. In this formulation, S and R are input tuples with A being the output tuple. In some embodiments, the action vector 426 provides stochastic guidance on design actions to meet target specifications by providing a probability distribution of actions that maximizes future weighted rewards to be able to meet the target specifications, captured by a multi -objective composite function, as further described below.

[0079] For each iteration’s action vector from the policy network, the circuit configurator 410 modifies the current state based on the action vector and stores the new current state in the state storage 430. In some embodiments, the circuit configurator 410 samples from the probability distribution for each state parameter so as to reduce the probability distribution for each state parameter to a single value that can be provided to the simulator 415. That is, the action vector 426, in some embodiments, specifies for each state parameter a probability distribution of values (or a probability distribution of modifications to the current value). The circuit configurator 410 performs random sampling from the probability distribution for each parameter in the action vector to arrive at a new value for each state parameter. In some embodiments, the probabilities of this sampling evolves over time (e.g., to more fully explore the state parameters initially and then more focus on the high probability modifications later on as the solution is closer to the target reward values.

[0080] For the current state generated by the circuit configurator (i.e., for each set of actions performed on the active circuit components by the circuit configurator 410 based on the current action vector output by the policy network 405), the simulator 415 quantifies a reward value that expresses a benefit of the generated current state (i.e., of the performed set of actions). The simulator 415, in some embodiments, is an EM simulator that virtually analyzes electronic devices and systems by computationally solving Maxwell's equations. Such a simulator can provide detailed insights into EM behavior before physical prototyping, helping engineers uncover unintended interactions, ensure performance specifications are met, and optimize designs for high frequency products, such as antennas, RF components, and integrated circuits. Specifically, in some embodiments, the simulator 415 is a circuit simulator (i.e., rather than a field simulator) that simulates the circuit defined by the state (with the sampled actions applied) and outputs various circuit performance characteristics that are usedPTON.POOOIPCT 17to determine the reward function. The computation of this reward function will be further described below.

[0081] To ensure the simulator 415 performs its analysis accurately, some embodiments use a parameter calculator 417 that performs a black-box extraction methodology to provide the parasitics of the circuit design to the EM simulator. A primary challenge of RFIC design, especially at high frequencies, is the effects of layout parasitics. Coupling effects and parasitic inductances, capacitances, and losses can lead to major deviations between the schematic design and the layout design. Because the proposed algorithm operates on the schematic design, a methodology must be included to capture or predict parasitics at the schematic stage to ensure reliability and accuracy of the synthesized designs.

[0082] To accomplish this, some embodiments use a black-box extraction approach. A single transistor of average size is laid out and extracted in an EM solver in some embodiments. The EM solver accurately predicts all parasitics and captures them into an S-parameter view. The active element (i.e. the transistor itself), is black-boxed and paired with the S-parameter (SP) parasitic block. In black-boxing the transistor, the EM model in some embodiments relies on the PDK model for the transistor itself and applies the SP block around it. This method proves accurate even at D-band frequencies (110-170GHz), allowing for one-shot schematic- to-layout designs from the RL-optimized values and topology.

[0083] The optimization criteria evaluator 420 uses the reward data (e.g., circuit performance data) output by the simulator to determine whether it should end the RL engine’s current iterative search of the design space for an optimal design and configuration of the active circuit components. This evaluation in some embodiments not only depend on the reward value but also depend on a set of one or more other criteria. This other set of criteria is dependent on the optimization process (e.g., hill descent, simulated annealing, etc.) performed by the evaluator 420. This other set of criteria in some embodiments can include a maximum iteration number and / or a maximum search time parameter.

[0084] When the evaluator 420 determines that it should not end the RL engine’s current iterative search of the design space, the evaluator provides the reward value 422 to the policy network 405 and directs the policy network 405 to produce its next set of candidate modifications to the active circuit components. In some embodiments, the evaluator 420 also identifies the current state for which the policy network 405 should perform its operation. In some embodiments, this current state is the state on which the simulator 415 just performed its most recent operations (i.e., the state resulting from the most recent action vector 426 output by the policy network 405). However, in other embodiments, the evaluator can designate thePTON.POOOIPCT 18immediate prior state (i.e., the state most recently evaluated by the policy network) as the current state, or it can designate another one of the states stored in the state storage 430 as the current state, so that it can restart the exploration from another state. Such a restart of the search in the design space is not needed in other embodiments as the policy network’s operation implicitly can restart its design space exploration by producing a next current state that significantly differs from the prior current state.

[0085] Otherwise, when the evaluator 420 determines that it should end its current iterative search of the design space (i.e., uses the reward value and one or more other criteria to determine that it should end its current exploration of the design space for its current iterative exploration), the RL engine 400 outputs its current optimal RFIC design 434 to the inverse design synthesizer 310. As shown, this optimal RFIC design includes an optimal design and configuration of PA architecture (e.g., combiner architecture, gain-stage architecture, etc.), cell and driver topology (e.g., transistor and transistor stack topologies), parameters, and biasing, etc.

[0086] In addition to the optimal RFIC design, the RL engine 400 of some embodiments also provides the desired S-parameters of all EM structures in the design (e.g., as determined by the parameter calculator 417 to the inverse design, which then uses an AL based EM forward model to synthesize the optimal multi-port EM structures (such as combiner, splitter, inter-stage MN) with these desired S-parameters for the RFIC circuit configuration, as further described below. Multi-port EM structures, in some embodiments, are devices or circuits with two or more connection points (ports) via which electromagnetic waves can enter or exit. The multi-port EM structures can serve various functions, such as signal filtering, power spliting, amplification, and beamforming, and can be found in many applications, including high-frequency electronics and antennas.

[0087] These structures are characterized by their scattering parameters (S- parameters), which describe how electromagnetic signals are reflected and transmitted at each port. S-parameters are used in RF and microwave engineering to describe how an EM structure interacts with signals at its ports, representing reflected and transmitted power based on incident waves. The S-parameters may quantify signal magnitude and phase, aiding in the analysis of signal loss, gain, impedance mismatches, and the overall performance of linear networks. This makes S-parameters essential for designing and simulating high-frequency circuits like filters, amplifiers, and antennas.

[0088] As described further below by reference to Figure 11, the inverse design synthesizer 310 attempts to optimize the passive components for the RFIC given the currentPTON.POOOIPCT 19optimal solution for the active components and the desired S-parameters. The inverse design synthesizer 310 provides back to the RL engine, in some embodiments, (1) additional reward value(s) and / or (2) the optimal passive circuit configuration. The RL engine 400 (e.g., the optimization criteria evaluator) uses the additional reward values to either modify the reward (objective function) computations, modify the circuit parameters provided to the simulator 415 and re-run the simulator, or a combination thereof. In some embodiments, if the optimization criteria evaluator 420 (or another module of the RL engine 400) determines that the optimal active component solution determined by the RL engine 400 and the optimal passive component solution determined by the inverse design synthesizer 310 adequately meet the target specifications (or if, e.g., a threshold time or number of iterations has been reached), the RL engine 400 outputs a fully optimized synthesized solution 450 for the circuit (i.e., a synthesized RIFC design that meets the target specifications for that circuit). This fully optimal synthesized solution 450 specifies the optimal power amplifier architecture (e.g., power combining, number of gain stages), the optimal power amplifier driver and cell topology, parameters, and biasing, as well as the optimal passive circuit configuration.

[0089] As described, in some embodiments the RL engine goes through numerous iterations in which the policy network 405 generates an action vector (i.e., modifies the current state of the active components), the simulator 415 uses the sampled action vector to determine the circuit performance, and this circuit performance is used to calculate the reward that is provided back to the policy network 405 (and, during training, the critic network 425). After numerous such iterations, the current optimized architecture and desired S-parameters are provided to the inverse design synthesizer 310.

[0090] In other embodiments, the inverse design synthesizer 310 is used for each iteration of the RL engine to ensure that the circuit performance is computed accurately. In such embodiments, for each modification to the active components output by the policy network 405 and then sampled by the circuit configurator 410, not only does the simulator 415 determine circuit performance but the inverse design synthesizer determines the optimal passive circuit configuration to determine how far this optimal configuration deviates from the desired S-parameters. As further described below, this deviation can be used to update the output of the simulator 415 or as an additional input into the reward computation.

[0091] In either case, whether the inverse design is used after numerous iterations of the RL engine or during each iteration, some embodiments build look-up tables 423 that mostly replace the synthesizer 310. For instance, some embodiments build up these look-up tables 423 during training and then use the look-up tables to mostly replace the inverse design synthesizerPTON.POOOIPCT 20at inference time. Specifically, such embodiments can use a look-up table to determine the deviation from a desired set of S-parameters that can be used to feed back into the RL engine (e.g., for the simulator calculations). Once a final optimal solution for the active components has been reached by the RL engine, the inverse design synthesizer (rather than the look-up tables) can be used to actually synthesize the optimal passive components.

[0092] During training, the policy network 405 is trained as the decision-making agent that learns like a human designer. This learning is reflected in the parameters of the Al model used by the policy network. In those embodiments that implement the policy network as a neural network, this Al model is implemented by the trained parameters (e.g., the weights, bias values, etc.) of the neural network. For this learning of the policy network, the critic network 425 receives each reward value produced by the simulator and in response produces a TD error value 432 that guides the next set of candidate actions identified by the policy network.

[0093] In this way, with repeated experiments, the policy network’s model is based on exploration of different PA architectures with different cell topologies (such as commonemitter (CE), common-base (CB), stacks, etc.) with varying parameters, biasing, and constituent EM structures. This model also builds intuition relating the complex nonlinear manifold of the parameter space to the design performance. For example, this model can understand that high output power is more efficiently realized with a multi-port EM combiner, as opposed to a single two-port matching network with high impedance transformation ratio.

[0094] When trained effectively, the policy network’s model can also understand that the driver needs to be strong enough to saturate the main stage, while not being overdesigned and inefficient. Much like human designer intuition, the model then builds toward a policy function that allows it to optimize the path towards the right parameter for the PA architecture and the constituent inverse designed EM structures for the entire RFIC optimization. The training will be further described below by reference to Figure 10.

[0095] As mentioned above, some embodiments formulate the policy network 405 to operates on a 3 -tuple (S, A, R) set, by taking in the current state and performance (indicated by rewards), and providing a stochastic guidance of the best course of design actions A to take. Figure 5 conceptually illustrates the design space of some embodiments for the Al-enabled synthesis operation. As shown, some embodiments define the design state as a vector that captures the choices of power amplifier architecture (e.g., the number of stages, number of power combiner paths), circuit topology (e.g., whether a stage is a cascode, common-emitter, or common-base stage), as well as various device parameters (e.g., transistor gate length). The state can also include optimal S-parameters that will be provided to the inverse designPTON.POOOIPCT 21synthesizer in some embodiments. In some embodiments, the specification reencoder 214 defines this state vector based on the target circuit specification.

[0096] At each iteration step, some embodiments use Equation (A) below to define the reward ‘R’ as R=r (if r<0) or R=10+r (if r>0), where. i r = =i min i(.In this equation, {o i, o*sj} are the hard and soft optimization targets respectively, {Oh,i, osj} are the achieved performance during the optimization of the current state (e.g., as output by the simulator), and a is used to scale the soft target rewards when not all hard targets are met. Hard targets, or hard constraints, are targets for which the goal is to meet the constraint exactly (i.e., that must be met for a solution to be considered to have met the specification). Soft targets / constraints, on the other hand, are idealized goals that can be penalized for not being met but that the designer is okay with being violated by an optimal solution. The constraints are generally defined by the circuit specification. In some embodiments, the various constraints may be weighted differently and combined in various different ways in the reward function.

[0097] Figure 6 conceptually illustrates the operations of the simulator 415 and the optimization criteria evaluator 420 to determine the reward for a given set of circuit parameters and constraints. As shown, the simulator 415 receives the target specifications 605. In this case, the specifications include two hard constraints (i.e., a saturation power of 22 dBm and a gain of 15 dB) as well as one soft constraint (an efficiency of 30%). The simulator also receives the circuit 610 (e.g., as defined by the circuit configurator 410) and passive loss information (e.g., from the LUT 423).

[0098] The simulator 415 outputs various circuit parameters (e.g., output power as a function of input power, frequency response, efficiency, etc.). The optimization criteria evaluator 420 (or a separate module, in some embodiments) uses these parameters to calculate whether the targets (constraints) for the circuit are met, with the differences then used to calculate the rewards. The figure illustrates four examples of reward calculations based on different values output by the simulator for the target specifications. In the first example, the saturation power falls short by 3 dBm and the gain falls short by 3 dB, but the efficiency is met. In this case, the resulting reward value is -0.17 because the two hard constraints are not met. In the second example, the saturation power and gain similarly fall short, but in this case the efficiency target is also not met (i.e., the calculated efficiency is half of the target), resulting in a very low (-0.53) reward value. In the third example, the saturation power target is met but the gain still falls short while the efficiency target is met, resulting in a better (though stillPTON.P0001PCT 22negative) reward. Finally, in the last example, all of the targets (both hard and soft constraints) are met, resulting in a large (+10) reward.

[0099] The optimization problem can also be viewed in the framework of a Markov Decision Process (MDP), using in some embodiments a 4-tuple of (S, A, Pa, Ra). Here, S is the set of states, A is the set of actions, Pa(s,s’) is the state transition function (i.e., based on the set of actions), and Ra(s,s’) is the reward function for a set of actions.

[0100] Figure 7 illustrates an example of embedding of a circuit into the format of the MDP with the S, A, R vectors. This figure illustrates an example of how some embodiments convert the design of active transistor stages into a reinforcement learning problem. The state vector space is formed from the device properties, such as sizes (e.g., length, width, area, etc.), impedances, voltages, DC operating parameters, etc., for different components (e.g., different active components) of the circuit. The action vector space (e.g., parameters the model optimizes over) includes device values (e.g., modifications to at least some of the values in the state vector). The reward function in this example is formed from performance metrics, which are combined into a multi -objective reward (e.g., equation A shown above).

[0101] In some embodiments, the action set A specifies changes to the design parameters, which can include comprise transistor widths / lengths, bias voltage, and / or current levels. As noted below, the design parameters can in certain embodiments also include parameters such as capacitor / inductor values and transmission line widths / lengths. To enforce the Markov property, rather than setting these values directly through the action vector, the RL model in some embodiments chooses the amount to increment or decrement each parameter (or provides a probability distribution for incrementing and / or decrementing each parameter). This can be done in both discrete and continuous action spaces. In addition, some embodiments set lower and / or upper limits on each parameter to satisfy physical manufacturing constraints. Normally, this parameter space dictates a fixed circuit topology - all components are used, and only their values are searched. However, topology search can be integrated into the RL framework by segmenting a portion of each action {a} into a state where the device is effectively removed from the circuit.

[0102] Through this, the model can choose to eliminate any transistor within a stage or remove the stage entirely, enabling topology search. The default circuit stage has two transistors of varying sizes, forming a “cascode” stage. However, the RL model can choose to zero out either transistor, creating either a “common source” stage if the top device (the first transistor) is removed, or a “common gate” stage if the bottom device (the second transistor)PTON.POOOIPCT 23is removed. Through this, all three basic transistor topologies are encompassed within the topology search.

[0103] Figure 8 illustrates how the design parameters specified in the state enable topology search. Specifically, this figure illustrates an example of embedding topology optimization into the RL exploration process, expanding the design space searched. When an action zeroes out the value of a state parameter (in this case, the transistor length), the model can remove a device from the signal path. By default, stacking two transistors results in a cascode stage (i.e., with the values “Lei” and “Lef’ both nonzero). When an action zeroes out the top device (i.e., sets the value “Lei” to 0), this results in a common source stage. On the other hand, when an action zeroes out the bottom device (i.e., sets the value “Lei” to 0), doing so results in a common gate stage. By defining the state in terms of transistor length values for each stage, all core circuit topologies can be explored by the RL engine.

[0104] Figure 9 conceptually illustrates another example of topology search, this time for larger-scale circuit parameters rather than parameters that define the topology of a single stage. As shown, this figure illustrates four different potential topologies 905-920 that depend on two different values expressed by the state vector of some embodiments. In this case, the state vector specifies (1) the number of stages of the power amplifier (as shown by the values 925) and (2) the number of combiner paths (as shown by the values 930). The four potential topologies shown are produced based on the state vector specifying either 1 or 2 paths and 1 or 2 stages. That is, when the number of stages specified by the value 925 is 1 and the number of paths specified by the value 930 is 1, the single-stage, one-way topology 905 results. When the number of stages specified by the value 925 is 2 and the number of paths specified by the value 930 is 1, the two-stage, one-way topology 910 results. When the number of stages specified by the value 925 is 1 and the number of paths specified by the value 930 is 2, the single-stage, two-way topology 915 results. Finally, when the number of stages specified by the value 925 is 2 and the number of paths specified by the value 930 is 2, the two-stage, two- way topology 920 results. For these variables, the state vector values are limited to positive integer values, so many more combinations of paths and stages are possible (e.g., up to N stages and M combiner paths). The action vector output by the policy network, in some embodiments, can modify one or both of these values 925 or 930 in the state vector to explore different circuit topologies.

[0105] The state set S includes the circuit parameters, which comprises both circuit performance values and circuit device values in some embodiments. This allows the state to provide a full description of all factors that affect performance of the design and are used byPTON.POOOIPCT 24the RL model for both estimation of design performance and prediction of how the design parameters should be tweaked to improve the design. This approach captures transistor DC operating points (e.g., gm, Vth, region of operation, etc....), bias currents and voltages, along with frequency simulation data. For PA design specifically, this includes the output power curves with respect to input power sweeps, for all frequency points in the band of interest.

[0106] The state set S in some embodiments is fed into the policy neural network (such as policy network 405), which outputs a vector matching the dimension of the action set A that effectively updates the circuit at each iteration as described above. Convergence is iterative, meaning the circuit is repeatedly updated via the policy network until the desired specifications are reached, or until a user-imposed time limit is reached.

[0107] After each iteration, the policy network in some embodiments is updated via gradient ascent to maximize the reward that comes from the reward function, R(s,s’). Thus, the reward function in some embodiments is formulated in a way to richly encompass all metrics of the circuit that are intended to be optimized. To achieve this, some embodiments use a multiobjective composite function. Several desired metrics (such as efficiency, output power, input match, etc.) are composed into the set C*=[ct*], where [ct*] is the desired value for a specific metric. This can be a maximization or minimization problem, in addition to a soft or hard constraint. Then, the reward at each step is what was described above by reference to Equation (A). As mentioned above, Oh,i are the hard constraints, meaning they are required to be satisfied for the circuit to be considered a solution, while os,t are soft constraints which are desired to be minimized but are not a hard requirement for a solution. When r is greater than 0 on average, the model in some embodiments is consistently generating satisfactory designs.

[0108] As noted above, Figure 10 conceptually illustrates a process 1000 for training the RL engine (i.e., for training the policy network 405 of the RL engine 400). The process 1000, performs reinforcement learning to optimize the policy network 405. During training, the critic network 425 is used in order to train the policy network 405 (i.e., to modify the parameters of the policy network), while in some embodiments the critic network 425 is no longer used for inference.

[0109] As shown, the process 1000 begins by initializing (at 1005) the circuit. In some embodiments, the initial circuit uses initialized values (e.g., randomly initialized values) for the state parameters that specify the circuit description. This initial circuit description state is provided to the simulator 1010, which outputs various circuit performance parameters (e.g., the bandwidth, gain, power, noise figure, linearity, etc.). These circuit performance parameters,PTON.POOOIPCT 25in some embodiments, are used to calculate the reward (e.g., according to the multi-objective function defined for the reward).

[0110] The process 1000 determines (at 1015) whether the target specification has been achieved; that is, whether the circuit performance values computed by the simulator match those specified by the target specification. Assuming the circuit does not yet meet the target specifications, the process 1000 also determines (at 1020) whether a time limit for the current training iteration has been reached. Some embodiments impose a time limit (or a threshold number of inner loop iterations) for each iteration of the training process.

[0111] If neither of these metrics are satisfied (i.e., the time limit has not been reached and the target specification performance is not achieved), the current state vector as well as the reward (i.e., either the output of the simulator or a set of values generated from the simulator output) is provided to the RL engine (at 1020). Based on the current state vector and the reward, the critic network of the RL engine computes a value function in order to inform the actor how well the action performed (e.g., based on the reward) and how the actor should adjust in its next iteration. Some embodiments use a history of trajectories (i.e., a design’s progression from start to finish) as inputs into the critic network, which provides an estimate of the optimality of actions and is used to optimize the weights of the policy network. The value neural network may also have its own weights adjusted using the history of trajectories in some embodiments. The actor network also outputs an action vector that is used to update the state (as described above), which is provided to the simulator as the process 1000 returns to 1010.

[0112] Once the time limit has been reached (or the target specification has been achieved), the process 1000 updates (at 1025) the actor and critic networks. Some embodiments update the parameters of the actor (policy) network using gradient descent (e.g., based on the rate of change of the output of the critic network with respect to the parameters of the actor network). Some embodiments also update the critic network at this time, as the critic network is trained along with the actor network (although the critic network is not used during inference, it can be improved during the course of training so as to provide better improvements to the actor network).

[0113] The process 1000 determines (at 1030) whether performance has converged. In some embodiments, this entails determining whether the optimal policy network (actor network) has been determined, such that the target circuit specification is achieved. If not, the process 1000 returns to 1005 to re-initialize the circuit and continue training. Once performance has converged, the training process 1000 ends.PTON.POOOIPCT 26

[0114] As indicated above, the RL engine 305 works in tandem with an inverse design synthesizer 310. The inverse design synthesizer, in some embodiments, receives a set of desired properties and characteristics for the passive circuit components (with the RL engine having attempted to optimize the active components) and then uses an optimization process to generate a predicted solution (i.e., a synthesized design for the passive circuit components). For each iteration of the active circuit configuration, in some embodiments the inverse design synthesizer synthesizes the optimal multi-port EM structures (such as combiner, splitter, interstage matching networks, etc.) with desired S-parameters for that circuit configuration. That is, with the circuit parameters in place (from the RL engine), some embodiments use the inverse design synthesizer to synthesize the arbitrary-shaped but compact and efficient EM structures based on the desired S-parameters. During training, some embodiments use tabulated data to approximate insertion loss of the inverse design matching networks for given pairs of impedance points. Once the active components are finalized by the trained RL agent, several inverse synthesis runs are performed, and the best solution design is chosen (e.g., for the matching networks).

[0115] Figure 11 conceptually illustrates an inverse design synthesizer 1100 of some embodiments. As shown, the inverse design synthesizer 1100 includes an inverse design engine 1105, an EM emulator 1110, and a parameter comparator and optimization evaluator 1115. The inverse design synthesizer 1100 receives, in some embodiments, a set of parameters from the RL engine. After a set of iterations of the RL engine, in some embodiments the RL engine synthesizes the active components of the RFIC and specifies a desired set of parameters (e.g., S-parameters, parasitics, and / or impedances) for the passive components. These are stored by the inverse design synthesizer 1100 as desired / synthesized parameters 1120.

[0116] The desired parameters 1120 are provided to the inverse design engine 1105, which outputs a synthesized structure for the passive components 1125. In some embodiments, given a desired set of impedances or S-parameters, the inverse design engine 1105 synthesizes a pixel structure. Various algorithms can be used for this synthesis operation. Genetic algorithms (GA) have been demonstrated successfully for this purpose, allowing synthesis of a final design in a matter of minutes. The pixel structure represents the synthesized passive components which, per a trained inverse design engine 1105, should optimally produce the desired set of parameters 1120. This is the tenet of inverse design of some embodiments. A rectangular area is divided into grids, where each “pixel” can be filled with metal or left empty. By filling in pixels, the inverse design algorithm creates a QR-code like structure that is non- intuitive to human designers. Despite the non-intuitive nature, the grid structure enables vastlyPTON.POOOIPCT 27more designs to be generated compared to the traditional geometries. As an example, a 16x16 grid has 2A(16xl6) or approximately le77 unique designs. This can distribute the electric and magnetic fields more optimally, thanks to the exponentially larger number of design freedoms, enabling more globally optimal designs.

[0117] Figure 12 conceptually illustrates two examples of pixel structures 1205 and 1210 representing synthesized passive components. The first pixel structure 1205 represents a three-port combining / dividing structure, while the second pixel structure 1210 represents a two-port single-ended pixelated structure. In each diagram, the pixel structure (1) indicates the pixels that are filled with metal for that structure and (2) indicates the locations of input / output ports and DC voltage connections (e.g., ground, bias voltage).

[0118] This pixel structure (synthesized passive components) 1125 is provided to the EM emulator 1110. In some embodiments, this EM emulator 1110 is a machine-trained network (e.g., a forward convolutional neural network) that generates a predicted set of parameters (e.g., S-parameters, parasitics, and / or impedances) 1130. The structure of the predicted set of parameters 1130 matches that of the desired parameters 1120 in some embodiments. The EM emulator 1110, in some embodiments, is trained through gradient descent using supervised learning with randomly generated inverse design structures having labelled data indicating their S-parameter response.

[0119] The larger design set enabled by the pixel structure used for inverse design allows more optimal solutions to be found but makes it impossible for design through regular optimization algorithms or methods. These structures do not have schematic-level representations and thus must be directly simulated in EM solvers to determine their behavior. However, an accurate EM simulation of one structure can take on the order of minutes, which leads to intractable computation times if numerous designs much be searched. A convolutional neural -network surrogate has been demonstrated to predict the performance of an arbitrary pixel structure in lieu of an EM simulator. After the NN has been trained, the prediction time of each design is negligible as it is reduced to simple matrix computations.

[0120] The parameter comparator and optimization evaluator 1115 compares the predicted parameters 1130 (from the EM emulator 1110) for a set of synthesized passive components 1125 to the desired parameters 1120 to determine how closely the synthesized passive component design fits its target as specified by the RL engine.

[0121] The parameter comparator and optimization evaluator 1115 also determines whether this passive component design is adequate or should be optimized by another iteration of the inverse design engine 1105. If the inverse design engine 1105 should be run again (e.g.,PTON.POOOIPCT 28because a time or iteration number threshold has not yet been reached, and / or because the predicted parameters are too different from the desired parameters), the parameter comparator and optimization evaluator 1115 provides feedback to the inverse design engine 1105 so that it will continue attempting to synthesize a better set of passive components for the desired parameters 1130. In other embodiments, the inverse design engine (e.g., the genetic algorithm) only performs its synthesis operation once for each set of desired parameters from the RL engine.

[0122] Once the parameter comparator and optimization evaluator 1115 determines that the inverse design synthesizer operations are complete (for the current iteration), it provides to the RL engine (1) a measure of rewards (indicating to what extent the desired parameters are matched by the current synthesized passive components) and / or (2) the current set of synthesized passive components. In some embodiments, the RL engine incorporates the rewards from the RL engine into its subsequent operation. For instance, if the inverse design synthesizer determines that the circuit can’t be designed with passive components to exactly meet the specified parameters from the RL engine, some of these parameters (which directly affect the reward objective function) are adjusted based on how closely the parameters can be met. Conjunctively or alternatively, some embodiments modify the circuit specifications and re-run the simulator. Based on the output of the inverse design, some of the values input to the simulator may need to be changed, so the RL engine modifies these values and re-runs the simulator to determine new circuit parameters used to calculate the reward objective function.

[0123] Figures 13 and 14 conceptually illustrate how a simulator 1300 of some embodiments uses the information from inverse synthesis to modify its output in some embodiments (which in turn modifies the rewards). In Figure 13, the simulator 1300 receives a circuit (as determined from the state and action vectors by the circuit configurator) and an assumption that there is no loss due to the passive circuit components (an optimized assumption). As shown, this results in a 20dB gain. However, in Figure 14, the simulator 1300 receives an estimate of the passive component loss (e.g., of N dB) from the inverse design synthesizer. This is used by the simulator 1300 to modify the circuit parameter (i.e., the gain) calculation, which is decreased by N. As such, the reward calculation will be different, as the difference between the target and actual gain will be changed. In some embodiments, rather than providing this information to the simulator 1300, the inverse design synthesizer provides the information to the optimization criteria evaluator that computes the rewards based on the calculated circuit parameters.PTON.POOOIPCT 29

[0124] The inverse design synthesizer of some embodiments specifically uses a genetic algorithm (e.g., an evolutionary algorithm) for the inverse design engine 1105 and a deep convolution neural network (CNN) for the EM emulator 1110. The genetic algorithm generates candidate inverse design structures that are fed to the inverse engine’s deep CNN, which serves as a rapid EM emulator that processes each received structure to predict the S-parameters of the structure. The predicted S-parameters of each structure are then used to indicate the fit of a design as compared to the desired set of S-parameters or desired impedance matching over a frequency range. In this manner, the DL-based EM emulator and genetic algorithm of the inverse design can rapidly optimize the passive structures. At runtime (inference time), the inverse design and RL engine provide more optimal passive design through irregular, non- intuitive geometries, and can complete end-to-end automated PA designs within minutes. For each modification of the optimal active structures from the RL engine, the inverse design synthesizer then synthesizes the EM structures in the modified PA circuit to allow full end-to- end optimization. The inverse design synthesizer of some embodiments is described in further detail in International (PCT) Application PCT / US2025 / 033964, filed 6 / 17 / 2025 and titled “Al Enabled Synthesis of Electromagnetic Structures and High Frequency Circuits”, which is incorporated herein by reference.

[0125] The following describes the use of the inverse design synthesizer in the specific context of synthesizing matching networks, but it should be understood that this applies to other passive components as well in some embodiments. A matching network in an RFIC of some embodiments is a circuit of passive components (like inductors and capacitors) that connects a source (e.g., an antenna or a transistor) to a load (e.g., another transistor or an antenna) within an RF system to ensure maximum power transfer and minimize signal reflections by transforming the source and load impedances to be complex conjugates of each other at a specific operating frequency.

[0126] As part of the design parameters, lumped components (capacitors, resistors, inductors) and transmission lines can be included as generic devices, with their values (capacitance, resistance, inductance, length, width) included in the action set 4. This gives the RL engine described above the ability to tune the networks for matching at desired frequencies. In the same fashion as the transistor topology optimization, the lumped components or transmission lines can have a portion of their action vector, a, segmented for a portion that indicates removal of the selected device. This enables the matching networks to be reconfigured as needed. For example, in some embodiments, two L-matches are cascaded. In theory, this enables two resonances to be achieved, for a dual-peak design or broaderPTON.POOOIPCT 30bandwidth. However, one L-section can be removed. Or a shunt section can be removed, enabling an effectively longer series section to be implemented through two cascaded lines.

[0127] However, traditional matching networks transmission lines are limited to a cascade of series / shunt sections with different sizes and values. Each L-section (series-shunt) provides one resonance. Increasing the number of L-sections makes design of the network exponentially more difficult. This geometric requirement naturally limits the design space available and provides no guarantees on the optimality of the matching. However, if these geometric requirements were bypassed, an exponentially larger design space would be unlocked, making a globally optimal solution more likely.

[0128] In the context of the RL synthesis method, the inverse design pixel structures can directly replace the cascaded L-section approach. Rather than lumped components or transmission lines, the RL model instead optimizes a set of ideal impedances seen at the input / output of each stage. An ideal transformation between the output of one stage and the input of the next stage is assumed. Through this, the RL model effectively learns to perform a source-pull and load-pull at the impedances at each stage, providing maximum power transfer at the target frequencies. After a satisfactory design has been achieved, the ideal impedances are fed into the inverse-design algorithm. This generates a pixel structure at the input and output of each stage, effectively completing the design.

[0129] This above-described circuit-EM co-design opens up a design space beyond human intuition. For instance, Figure 15 conceptually illustrates one example that correlates training time with successful designs and design / inference time. As shown, training time in this example (which included about 35,000 explored solutions of circuit and inverse EM synthesis), took about 3.3 days (~80 hours) computing on a cluster of 192 cores to reach close to 100% success rate for the explored designs. This exampled used proximal policy optimization (PPO) for its stability and sample efficiency, and both the actor and critic networks used were 2-layer networks with 64 neurons per layer. After training, the design time of the output stage actives can take on average 6-to-7 minutes, or approximately 30 iterative designs and simulations.

[0130] In some embodiments, the output is effectively the entire PA layout, as the model trains on a layout-aware parasitic for the active devices and the inverse design synthesizes the on-chip EM structures. For a full PA design, synthesis can take a few hours (depending on specifications), and additional design time for ancillary circuits. Some embodiments partition the circuit and EM synthesis, so that the design methodology can focus on an end-to-end optimization.PTON.POOOIPCT 31

[0131] Figure 16 conceptually illustrates the viability of the above-described RFIC synthesis technique. Specifically, this figure illustrates demonstrated power amplifier simulated performance, for both large-signal and small-signal responses. For this simulation, a power amplifier in the D-band (110-170GHz) was taped-out and measured to demonstrate viability. The power amplifier was laid out and full-EM extracted for accuracy of simulation, and indicates final expected performance of the physical chip.

[0132] The above-described methodology (e.g., as shown in Figure 4) conceptually illustrates an Al-driven synthesis process that uses an RL engine to synthesize the active circuit components of an IC (e.g., an RFIC) and provides desired S-parameters for the IC to an inverse design engine that synthesizes the passive components of the IC. Other embodiments incorporate at least some of the passive component synthesis (e.g., transmission-line lengths, etc.) into the state vector that is modified by the action probability distribution output by the policy network of the RL engine.

[0133] Figure 17, for instance, illustrates the parameters that define a design stage, similar to that shown in Figure 8. In this case, however, the parameters also include transmission-line lengths (“Leni” and “Len2”). The RL engine (i.e., the policy network) works to optimize these passive component parameters along with the active component parameters (e.g., the transistors) rather than simply specifying desired S-parameters and handing off the passive component optimization to the inverse design synthesizer. Like with the transistor stages, sections of passive devices can be cascaded with the option to zero individual devices out. By removing or adding series and shunt sections to the signal path, multiple resonances or different matching properties can be unlocked.

[0134] Figure 18 conceptually illustrates an example of a passive component structure synthesized via the RL engine (i.e., rather than using inverse design to synthesize the passive components). In this case, the figure illustrates a traditional transmission-line structure (as compared to the pixelated structures shown above in Figure 12).

[0135] The above-described embodiments describe performing Al-driven synthesis of an IC (e.g., specifically an RFIC). Figure 19 conceptually illustrates an example of a detailed process 1900 for designing and manufacturing an IC. The process 1900 uses the Al-driving synthesis techniques described above to synthesize such an IC.

[0136] The process 1900 begins (at 1905) by defining the code that specifies the IC design and performing functional verification and testing on this code. In some embodiments, the process uses one of the common hardware description languages (HDL) to specify the code. The HDL code in some embodiments describes the desired structure, behavior, and timing ofPTON.POOOIPCT 32the IC. To perform functional verification and testing on the code for the IC, some embodiments specify one or more modules and / or circuit components in the code and check the specified modules and / or circuit components for functional accuracy.

[0137] Next, the process 1900 performs (at 1910) a synthesis operation, which converts the HDL description into a circuit representation that commonly includes digital circuit components, such as logic gates, flip-flops, and other larger digital components (e.g., adders, multipliers, etc.). As noted, some embodiments use the above-described Al-driven synthesis that uses an RL engine to synthesize at least the active components of the IC.

[0138] At 1915, the process 1900 performs verification and testing on the circuit representation that is produced by the synthesis operation. In some embodiments, the verification and testing checks the circuit representation to determine whether this representation meets desired timing constraints and satisfies any other constraint of the HDL code. When the verification and testing fails (e.g., if a portion of the circuit representation fails to meet a constraint), the process 1900 returns to step 1910 (as denoted by a dashed arrow line) to reperform synthesis to modify the circuit representation to resolve this failure.

[0139] When the verification and testing at 1915 passes, the process 1900 performs a set of physical design operations 1918, which include operations 1920-1935 between which the process 1900 can iterate through multiple times as further described below. At 1920, the process 1900 performs a floorplanning operation that defines a general location for some or all of the circuit blocks (e.g., for various large circuit blocks). For instance, in some embodiments, floorplanning divides the design layout into one or more sections devoted to different purposes (e.g., ALU, memory, decoding, etc.), and assigns some or all of the circuit blocks to these sections based on the purposes served by these blocks.

[0140] At 1925, the process 1900 performs a placement operation, which is based on the floorplanning data and defines a specific location and orientation in the design layout for each circuit block. The placement operation in some embodiments is an automated process that tries to find an optimal placement for each circuit block based on one or more optimization criteria, such as congestion or estimated length of interconnects (e.g., metal wires) needed for connecting the nets associated with the circuit blocks. A net in some embodiments includes a set of two or more pins of one or more circuit blocks that need to be connected electrically (e.g., through a set of wires, contacts, and / or vias). After performing the placement operation, the process 1900 might return to the floorplanning operation if it determines that the floorplanning should be revised to improve the result of the placement operation.PTON.POOOIPCT 33

[0141] Once the placement operation is completed satisfactorily, the process performs (at 1930) a routing operation to define the route needed to connect each net (i.e., to connect each set of pins that needs to be interconnected). Each defined route includes one or more interconnect segments (also called wire segments) that traverse one or more interconnect layers (also called wiring layers), and one or more vias and / or contacts that connect pins and / or wire segments on different wiring layers.

[0142] Some embodiments divide the routing operation into a global routing operation and a detailed routing operation. For each net, global routing defines a global route that more generally defines the route for the net (e.g., defines a general area in the design layout traversed by the route). For instance, in some embodiments, the global router divides an IC into individual global routing areas, called Gcells. Then, a global route (Groute) is created for each net by listing the global routing areas (the Gcells) that the Groute for the net should pass through.

[0143] The detailed routing defines the actual route for each net (e.g., the route that connects the set of pins that forms the net). As mentioned above, each defined route includes one or more interconnect segments that traverse one or more interconnect layers, and one or more vias and / or contacts that connect pins and / or wire segments on different interconnect layers. In performing its detailed routing operation, the detailed router of some embodiments uses the global router’s Groute data, e.g., by biasing its detail route search for the net to the Groute regions traversed by the Groute defined by the global router.

[0144] During or after the detailed routing operation, the process 1900 performs a design rule check (DRC) operation to ensure that the defined routes do not violate design rules. One example of the design rule check that is done for a route is to ensure that the route is not closer than an acceptable minimum spacing requirement on each layer traversed by the route to another route or another component in the design layout on that layer. Routes that violate minimum spacing constraints can cause undue capacitance and, in some cases, electrical shorts on the IC.

[0145] The process 1900 in some embodiments can iterate through the global and detailed routing multiple times to identify better Groutes for some nets in order to improve the detailed routes for these nets or other nets. Also, the process 1900 in some embodiments can return from either of these routing operations to an earlier operation in the EDA flow (e.g., to the placement operation) in order to improve the results of this earlier operation to improve the routes defined by the later routing operation.PTON.POOOIPCT 34

[0146] After routing, the process 1900 performs (at 1935) compaction operations. In some embodiments, the compaction operation compresses the design layout in one or more directions to decrease the size of the IC die (e.g., to decrease the two-dimensional area of the IC die) that would be manufactured based on the design layout. Reducing the size of the IC improves the performance of the IC in some embodiments. A compacted design layout also lowers costs of the ICs manufactured using the design layout by allowing more ICs to be produced for a given wafer size.

[0147] After the compaction operation, the process 1900 performs a layout verification operation (at 1940) to ensure that the compacted design layout (e.g., the compacted routes in this design) to ensure that the layout meets one or more verification criteria. This verification operation includes a DRC operation that ensures that the compacted design layout does not violate design rules. One example of the DRC that is done for a route is to ensure that the route is not closer than an acceptable minimum spacing requirement on each layer traversed by the route to another route or another component in the design layout on that layer. Other examples of the DRC include ensuring that items in the design layout (e.g., routes, pins, contacts, vias, or other components) do not violate minimum area, minimum width, and maximum curvature requirements.

[0148] The layout verification in some embodiments includes other operations, such as extraction. Extraction in some embodiments computes parasitic values (e.g., parasitic capacitance values or parasitic inductance values) exerted on items (e.g., wire segments) in the design layout. In some embodiments, the extraction operation computes capacitance coefficients for one or more conductive components in the design layout (e.g., for each wire segment of a route, or for the entirety of each route, in the design layout) and uses the capacitance coefficients to compute parasitic influence (e.g., capacitance, resistance, or inductance) on the conductive component(s).

[0149] After the compaction operation at 1935 or the subsequent verification operation 1940, the process 1900 in some embodiments can return to an earlier operation in the EDA flow (e.g., to the placement operation, to the routing operations) in order to improve the results of this earlier operation to improve the compacted design defined by the later compaction operation. For instance, when the design is not verified at 1940 (e.g., if a problem with the design is detected during verification), the process 1900 returns to an earlier physical design operation 1920 to 1935 to reperform this physical design operation, and any subsequent physical design operation, for a portion or for the entire design layout. In some embodiments, the design layout that exists after the compaction operation and that passes the subsequentPTON.POOOIPCT 35verification operation 1940 on this layout is the end result of the physical design process, is called the physical design layout, and is used as the input to the subsequent operations 1945- 1955 that form the manufacturing sub-process of the process 1900.

[0150] In some embodiments, the physical design sub-process includes other operations that are not displayed in Figure 19. These other operations are not displayed for purposes of brevity. Examples of such operations include partitioning, power planning, and clock tree synthesis (CTS). In some embodiments, partitioning divides the design layout into similar-sized subsets and ensures a minimum number of connections between subsections. Power planning defines the power delivery network (PDN) that includes the interconnects for delivery power from the power supply circuit to circuits defined by the IC design layout. CTS in some embodiments defines a clock delivery network for delivering one or more clock signals to circuits defined by the IC design layout. CTS in some embodiments also inserts buffers and / or inverters along the clock signal paths on the clock delivery network in order to balance the load and decrease or eliminate any clock skew or delay.

[0151] Once physical design operations 1918 are completed and the design layout is finalized, the process 1900 performs a set of mask production operations 1943, which include operations 1945-1960. These processes collectively produce a set of one or more masks for each layer of the IC based on the design layout which, when used to fabricate the IC, should result in an IC (or multiple ICs) that match the design layout as closely as possible.

[0152] At 1945, the process 1900 performs a coloring operation for each layer of the design layout. The coloring operation decomposes the design layout for a layer into multiple (e.g., two, three, etc.) separate layouts for the purpose of mask production by assigning each feature in the layout to one of multiple “colors”. For certain IC design layers, the features (e.g., the routes, pins, contacts, vias, etc.) are packed too closely for the features to be printed on a wafer using a single mask. In some embodiments, the coloring operation identifies an optimal decomposition for a layer by iteratively assigning the features in the layer to different colors (e.g., using a graph coloring algorithm) and scoring the decomposition. In some embodiments, this coloring operation is optional and can be skipped for some or all of the IC layers.

[0153] After the coloring operation, the process 1900 performs mask design (at 1950) or mask layout generation. Mask design generates, for each layer of the IC, the layout for one or more masks (i.e., one mask for each color) that will optimally create the shapes defined in the layout during fabrication of the IC. Some embodiments use commonly known techniques, such as OPC (optical proximity correction) and / or ILT (inverse lithography technology) operations. An ILT system, for instance, iteratively defines a potential mask layout, performsPTON.POOOIPCT 36lithography simulation to simulate the wafer shapes that would be manufactured using the potential mask layout, compares this simulation to a set of target wafer shapes, and updates the mask layout based on the comparison (the inverse lithography step). After numerous such iterations, the ILT system determines an optimized mask design for a given layout.

[0154] The process 1900 then performs a mask preparation operation at 1955. In some embodiments, the mask preparation operation 1955 includes operations that prepare a mask writer (e.g., an electron beam mask writer) to fabricate a particular mask based on the mask design, such as mask data preparation (MDP). MDP in some embodiments prepares the mask layout for a mask writer. This operation in some embodiments includes “fracturing” the data into trapezoids, rectangles, or triangles.

[0155] In some embodiments, after performing mask preparation (or after performing a mask rule check prior to the mask preparation), the process 1900 can return to an earlier operation in the mask production operations 1943 (e.g., to coloring 1945 or mask design 1950) in order to improve the results of this earlier operation and thereby improve the eventual fabricated mask. Due to the high expenses of fabricating a mask, it is generally desirable to have the mask designs optimized before fabrication. In some embodiments, the process 1900 can iteratively repeat the mask production operations 1943 in order to improve the quality of the overall generated mask or can return to one of the earlier physical design operations 1918, as described above.

[0156] Once the mask layout is generated and verified, the process 1900 fabricates (at 1960) the one or more masks specified for all the layers of the IC based on the mask layout. Mask generation transforms each mask image (also referred to as a mask layer, in some embodiments) of the mask layout into one or more lithographic masks in some embodiments.

[0157] Once the masks are generated, the process 1900 performs (at 1965) wafer fabrication, which uses the generated masks to manufacture multiple IC dies on an IC wafer (e.g., a silicon wafer). The masks for the substrate and each wiring layer are used to generate the devices and wiring on the substrate and each wiring layer of each IC die. Each IC die is usually tested. During the testing of the IC dies, if it is determined that the IC has a defect because of its design or its masks, the process 1900 has to return to an earlier operation to improve its design layout, its mask layout, or its mask production operation. Lastly, the process 1900 performs (at 1955) packaging, which places each IC die in one chip package. Packaging in some embodiments includes slicing a wafer into multiple IC dies and placing each die on a substrate, which is then encapsulated to form a chip package. After performing packaging, the process 1900 ends.PTON.POOOIPCT 37

[0158] Many of the above-described features and applications are implemented as software processes that are specified as a set of instructions recorded on a computer readable storage medium (also referred to as computer readable medium). When these instructions are executed by one or more processing unit(s) (e.g., one or more processors, cores of processors, or other processing units), they cause the processing unit(s) to perform the actions indicated in the instructions. Examples of computer readable media include, but are not limited to, CD- ROMs, flash drives, RAM chips, hard drives, EPROMs, etc. The computer readable media does not include carrier waves and electronic signals passing wirelessly or over wired connections.

[0159] In this specification, the term “software” is meant to include firmware residing in read-only memory or applications stored in magnetic storage, which can be read into memory for processing by a processor. Also, in some embodiments, multiple software inventions can be implemented as sub-parts of a larger program while remaining distinct software inventions. In some embodiments, multiple software inventions can also be implemented as separate programs. Finally, any combination of separate programs that together implement a software invention described here is within the scope of the invention. In some embodiments, the software programs, when installed to operate on one or more electronic systems, define one or more specific machine implementations that execute and perform the operations of the software programs.

[0160] Figure 20 conceptually illustrates a computer system 2000 with which some embodiments of the invention are implemented. The computer system 2000 can be used to implement any of the above-described hosts, controllers, and managers. As such, it can be used to execute any of the above-described processes. This computer system includes various types of non-transitory machine-readable media and interfaces for various other types of machine- readable media. Computer system 2000 includes a bus 2005, processing unit(s) 2010, a system memory 2025, a read-only memory 2030, a permanent storage device 2035, input devices 2040, and output devices 2045.

[0161] The bus 2005 collectively represents all system, peripheral, and chipset buses that communicatively connect the numerous internal devices of the computer system 2000. For instance, the bus 2005 communicatively connects the processing unit(s) 2010 with the readonly memory 2030, the system memory 2025, and the permanent storage device 2035.

[0162] From these various memory units, the processing unit(s) 2010 retrieve instructions to execute and data to process in order to execute the processes of the invention. The processing unit(s) may be a single processor or a multi-core processor in differentPTON.POOOIPCT 38embodiments. The read-only-memory (ROM) 2030 stores static data and instructions that are needed by the processing unit(s) 2010 and other modules of the computer system. The permanent storage device 2035, on the other hand, is a read-and-write memory device. This device is a non-volatile memory unit that stores instructions and data even when the computer system 2000 is off. Some embodiments of the invention use a mass-storage device (such as a magnetic or optical disk and its corresponding disk drive) as the permanent storage device 2035.

[0163] Other embodiments use a removable storage device (such as a flash drive, etc.) as the permanent storage device. Like the permanent storage device 2035, the system memory 2025 is a read-and-write memory device. However, unlike storage device 2035, the system memory is a volatile read-and-write memory, such a random-access memory. The system memory stores some of the instructions and data that the processor needs at runtime. In some embodiments, the invention’s processes are stored in the system memory 2025, the permanent storage device 2035, and / or the read-only memory 2030. From these various memory units, the processing unit(s) 2010 retrieve instructions to execute and data to process in order to execute the processes of some embodiments.

[0164] The bus 2005 also connects to the input and output devices 2040 and 2045. The input devices enable the user to communicate information and select commands to the computer system. The input devices 2040 include alphanumeric keyboards and pointing devices (also called “cursor control devices”). The output devices 2045 display images generated by the computer system. The output devices include printers and display devices, such as cathode ray tubes (CRT) or liquid crystal displays (LCD). Some embodiments include devices such as a touchscreen that function as both input and output devices.

[0165] Finally, as shown in Figure 20, bus 2005 also couples computer system 2000 to a network 2065 through a network adapter (not shown). In this manner, the computer can be a part of a network of computers (such as a local area network (“LAN”), a wide area network (“WAN”), or an Intranet, or a network of networks, such as the Internet. Any or all components of computer system 2000 may be used in conjunction with the invention.

[0166] Some embodiments include electronic components, such as microprocessors, storage and memory that store computer program instructions in a machine-readable or computer-readable medium (alternatively referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, read-only compact discs (CD-ROM), recordable compact discs (CD-R), rewritable compact discs (CD-RW), read-only digitalPTON.POOOIPCT 39versatile discs (e.g., DVD-ROM, dual-layer DVD-ROM), a variety of recordable / rewritable DVDs (e g., DVD-RAM, DVD-RW, DVD+RW, etc ), flash memory (e g., SD cards, mini-SD cards, micro-SD cards, etc.), magnetic and / or solid state hard drives, read-only and recordable Blu-Ray® discs, ultra-density optical discs, and any other optical or magnetic media. The computer-readable media may store a computer program that is executable by at least one processing unit and includes sets of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as is produced by a compiler, and files including higher-level code that are executed by a computer, an electronic component, or a microprocessor using an interpreter.

[0167] While the above discussion primarily refers to microprocessor or multi-core processors that execute software, some embodiments are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions that are stored on the circuit itself.

[0168] As used in this specification, the terms “computer”, “server”, “processor”, and “memory” all refer to electronic or other technological devices. These terms exclude people or groups of people. For the purposes of the specification, the terms display or displaying means displaying on an electronic device. As used in this specification, the terms “computer readable medium,” “computer readable media,” and “machine readable medium” are entirely restricted to tangible, physical objects that store information in a form that is readable by a computer. These terms exclude any wireless signals, wired download signals, and any other ephemeral or transitory signals.

[0169] While the invention has been described with reference to numerous specific details, one of ordinary skill in the art will recognize that the invention can be embodied in other specific forms without departing from the spirit of the invention. For instance, several figures conceptually illustrate processes. The specific operations of these processes may not be performed in the exact order shown and described. The specific operations may not be performed in one continuous series of operations, and different specific operations may be performed in different embodiments. Furthermore, the process could be implemented using several sub-processes, or as part of a larger macro process. Thus, one of ordinary skill in the art would understand that the invention is not to be limited by the foregoing illustrative details, but rather is to be defined by the appended claims.PTON.POOOIPCT 40

Claims

CLAIMS1. A method for performing artificial intelligence (Al) driven synthesis of an integrated circuit (IC) design comprising active and passive circuit components, the method comprising: receiving, for the IC design, a target first description of the active and passive circuit components of the IC design; using a set of one or more Al-driven processes to iteratively examine different configurations of the active and passive circuit components, in order to produce a synthesized second description of the IC design, said synthesized second description comprising an optimal configuration of the active and passive circuit components identified through the iterative examinations of the set of Al-driven processes, each Al-trained process using an Al model that is trained through a learning process; and providing the synthesized second description of the IC design as a synthesized representation of the IC for subsequent processing by a subsequent electronic design automation (EDA) tool.

2. The method of claim 1, wherein the passive components described in the target first description comprise electromagnetic (EM) structures each of which has a set of passive circuit components and a desired frequency response, the optimal configuration of the active and passive circuit components includes optimal configuration of the EM structures.

3. The method of claim 1, wherein the target first description comprises a code-based description of the IC design and the synthesized second description comprises a circuit representation of the IC design.

4. The method of claim 3 further comprising converting the code-based description into an encoded description of the circuit components in the IC design, said encoded description in a format for processing by one or more of the Al-driven processes.

5. The method of claim 1, wherein the subsequent EDA tool is part of an overall EDA process comprising a set of physical design layout tools that from a synthesized representation of the IC define the physical design layout of the circuit components of the IC over a substrate.

6. The method of claim 1, wherein at least one Al model is trained through a supervised learning process, an unsupervised learning process, or a reinforcement learning process.

7. The method of claim 1, wherein at least one Al-driven process is implemented by a machine-trained network comprising a plurality of processing nodes each of which perform computations based on one or more parameters trained through a learning process.PTON.POOOIPCT 418. The method of claim 7, wherein the machine-trained network is a neural network comprising a plurality of neurons performing computations based on parameters trained through the learning process.

9. The method of claim 1, wherein using the set of Al-driven processes comprises iteratively using a first machine-trained network to perform different sets of modifications to the circuit components of the IC.

10. The method of claim 9, wherein the use of the first machine-trained network identifies the different sets of modifications by exploring a plurality of candidate sets of modifications in a design space based on (i) a set of constraints that define viable candidate set of modifications to identify, and (ii) an objective function that quantifies a value that expresses a benefit or cost for performing the candidate set of modifications.

11. The method of claim 1, wherein using the set of Al-driven processes comprises: iteratively: using a first machine-trained network to identify one sets of modifications to the circuit components of the IC; for each set of identified modifications, quantifying a value that expresses a benefit or cost for performing the set of modifications; and accepting or rejecting the set of modifications based on the quantified value.

12. The method of claim 11, wherein the first machine-trained network is a policy network of a reinforcement learning (RL) engine that evaluated the values quantified for the different sets of modifications during synthesis.

13. The method of claim 12, wherein the RL engine further comprises a critic network that is used to train the policy network before synthesis.

14. The method of claim 12, wherein said quantifying comprising using, for each identified set of modifications, a simulator to simulate the operation of at least a portion of the IC after the identified set of modifications is performed on the IC and to produce the quantified value for the identified set of modifications based on the simulated operation.

15. The method of claim 9, wherein the sets of modifications performed by the first machine-trained network are modifications to the active circuit components.

16. The method of claim 15, wherein using the Al-driven processes further comprises using a second machine-trained network to identify and perform different sets of modifications to passive circuit components of the IC.

17. The method of claim 15, wherein using the Al-driven processes further comprises for each set of modifications performed by the first machine-trained network:PTON.POOOIPCT 42using a second machine-trained network to iteratively evaluate different sets of modifications to passive circuit components of the IC; and from the evaluated passive circuit component modifications, selecting and performing one of the passive circuit component modifications18. The method of claim 17, wherein the second machine-trained network implements an EM emulator that predicts, for each set of modifications, a frequency response of a set of circuits formed by a set of passive circuit components to quantify quality of the set of modifications.

19. A machine readable medium storing a program which when implemented by at least one processing unit implements the method according to any one of claims 1-18.

20. An electronic device comprising: a set of processing units; and a machine readable medium storing a program which when implemented by at least one of the processing units implements the method according to any one of claims 1-18.

21. A system comprising means for implementing the method according to any one of claims 1-18.

22. A computer program product comprising instructions which when executed by a computer cause the computer to perform the method according to any one of claims 1-18.PTON.POOOIPCT 43

Images