Method and apparatus for analog circuit size adjustment

Abstract

A system performs operations of a neural network agent and a circuit simulator for simulating circuit sizing. The system receives input indicative of a specification and design parameters of a simulated circuit. The system iteratively searches a design space until a circuit size is found that satisfies the specification and the design parameters. In each iteration, the neural network agent computes measurement estimates for random samples generated in a trust region, which is a portion of the design space. Based on the measurement estimates, the system identifies a candidate size corresponding to an optimized value indicator. The circuit simulator receives the candidate size and generates a simulation measurement value. The system computes updates to weights of the neural network agent and the trust region based at least in part on the simulation measurement value for a next iteration.

Description

TECHNICAL FIELD

[0001] Embodiments of the application generally relate to analog circuit design, and more specifically, to analog design space search with deep reinforcement learning. BACKGROUND

[0002] The annual increase in computing power described by Moore’s law is opening unprecedented possibilities. This remarkable progress has been accompanied by a dramatic increase in chip design complexity. One example of this complexity is the growth of process, voltage, and temperature (PVT) conditions. While most of the system-on-a-chip (SoC) area is occupied by digital circuits, analog circuits are also essential for a chip. However, the design work for analog counterparts is more laborious due to the need for human expert intervention and the scarcity of automated tools.

[0003] Transistor sizing is a time-consuming and labor-intensive task in analog design. Currently, transistor sizing is mainly done through trial and error. Designers first apply their knowledge of analog circuits and transistor characteristics to select a reasonable range of candidate solutions. Then, the designers explore the design space in a grid search manner and receive feedback from SPICE (Simulation Program with Integrated Circuit Emphasis) circuit simulation. The actions of the designers and the process of SPICE circuit simulation are repeated until the specifications are met. Due to the very large design space, known techniques for automatically determining transistor sizes often encounter convergence or scalability problems.

[0004] Known circuit sizing solutions, such as Bayesian optimization (BO), model-free agents, sequence-to-sequence modeling with encoder-decoder techniques, graph convolutional neural networks, etc., all have various types of defects, such as scalability, generalizability, efficiency, and reusability.

[0005] Furthermore, to ensure that a chip can work under variations of manufacturing process, power supply, and environment, many PVT conditions must be signed off before tape-out. A conventional strategy for exploring PVT conditions is to test all PVT conditions every time a new set of circuit sizing assignments is obtained. This strategy wastes computing resources and electronic design automation (EDA) tool licenses.

[0006] Therefore, there is a need to improve simulation sizing automation to address the existing problems. SUMMARY

[0007] In view of the above, one of the purposes of the present application is to provide a method and apparatus for simulating circuit sizing with higher search capability by iteratively searching a portion of the design space.

[0008] In a first aspect, the present application provides a method for simulating circuit sizing, comprising: receiving an input indicating a specification of a simulated circuit and a plurality of design parameters; and iteratively searching a design space until a circuit sizing satisfying the specification and the design parameters is found, wherein the iteratively searching further comprises: a neural network agent computing a measurement estimate for each of a plurality of samples randomly generated in a trust region to identify a candidate sizing corresponding to an optimal value indicator, wherein the trust region is a portion of the design space; and updating a weight of the neural network agent and the trust region based at least in part on simulation measurements of the candidate sizing by a circuit simulator for use in a next iteration.

[0009] In some embodiments, the method further comprises: selecting an initial candidate sizing corresponding to based on simulation measurements of an initial sample in the design space by the circuit simulator; initializing the trust region to be centered at the initial candidate sizing; and initializing the neural network agent trained using at least the initial candidate sizing and corresponding simulation measurements.

[0010] In some embodiments, the trust region searched in a current iteration is centered at a candidate sizing identified in a previous iteration.

[0011] In some embodiments, the design parameters include a plurality of process, voltage, temperature (PVT) conditions, and the method further comprises: identifying a circuit sizing satisfying the specification under a first PVT condition of the plurality of PVT conditions; testing the circuit sizing satisfying the specification under other PVT conditions of the plurality of PVT conditions except the first PVT condition by the circuit simulator; and progressively exploring PVT conditions that fail the test until a final circuit sizing satisfying the specification and all PVT conditions is found.

[0012] In some embodiments, the stepwise exploration further comprises: adding a second PVT condition of the PVT conditions that fails the test to a condition pool, wherein the condition pool initially comprises the first PVT condition; adding a second neural network agent for the second PVT condition to an agent pool, wherein the agent pool initially comprises the first neural network agent for the first PVT condition; the neural network agents in the agent pool iteratively search the same trust region to obtain updated circuit sizes that satisfy the specification under the PVT conditions in the condition pool; and, the agent pool and the condition pool are enlarged for the iterative search until the final circuit size is found to satisfy the specification and all PVT conditions.

[0013] In some embodiments, the circuit size is a solution to a constraint satisfaction problem defined by a set of constraints and a set of circuit variables, wherein each circuit variable corresponds to a set of predetermined resizing values.

[0014] In some embodiments, calculating the update further comprises: calculating a ratio to estimate accuracy of the neural network agent in the trust region according to simulation measurements in the trust region; and, calculating a change of the radius of the trust region according to the ratio.

[0015] In some embodiments, the neural network agent is a multi-layer neural network that is learned through reinforcement learning.

[0016] In some embodiments, the value indicator is an output of a value function generated by the neural network agent with the candidate size as input.

[0017] In some embodiments, the value indicator is an output of a value function that evaluates a sum of normalized measurement values.

[0018] In some embodiments, the first PVT condition is the worst PVT condition of the plurality of PVT conditions, and the second PVT condition is the worst PVT condition of the PVT conditions that fail the test.

[0019] In a second aspect, the present disclosure provides an apparatus for simulating circuit resizing, comprising: at least one processor and a memory coupled to the at least one processor and storing instructions which, when executed by the at least one processor, cause the at least one processor to perform the method for simulating circuit resizing as described in any of the above embodiments.

[0020] These and other objects of the present disclosure will no doubt become obvious to those skilled in the art after reading the following detailed description of the preferred embodiment, the drawings of which are incorporated in and constitute a part of this specification. BRIEF DESCRIPTION OF DRAWINGS

[0021] A more complete understanding of the invention can be obtained by reading the following detailed description and referring to the examples provided in the accompanying drawings. The included drawings are used to provide a further understanding of embodiments of the present disclosure, and are incorporated in and constitute a part of the present disclosure. The drawings illustrate implementations of embodiments of the present disclosure and, together with the description, serve to explain the principles of the embodiments of the present disclosure. It will be understood that the drawings are not necessarily drawn to scale, as some components may be shown out of proportion to actual dimensions in order to clearly illustrate the concepts of the embodiments of the present disclosure.

[0022] Figure 1 This is a model-based RL framework for analog circuit size adjustment, as shown in an embodiment of the present invention.

[0023] Figure 2 This is a schematic flowchart illustrating the circuit size adjustment process according to an embodiment of the present invention.

[0024] Figure 3 This is a schematic diagram illustrating an algorithm for exploring feasible circuit dimensions according to an embodiment of the present invention.

[0025] Figure 4 This is a schematic diagram of a model-based RL platform according to an embodiment of the present invention.

[0026] Figure 5 This is a flowchart illustrating the PVT exploration process according to an embodiment of the present invention.

[0027] Figure 6 This is an example of a PVT exploration strategy illustrated in an embodiment of the present invention.

[0028] Figure 7 This is a flowchart illustrating a method for adjusting the size of an analog circuit according to an embodiment of the present invention.

[0029] Figure 8 This is a schematic diagram of a system according to an embodiment of the present invention.

[0030] In the following detailed description, numerous specific details are set forth for illustrative purposes so that those skilled in the art can more thoroughly understand the embodiments of the invention. However, it will be apparent that one or more embodiments may be practiced without these specific details, and different embodiments may be combined as needed, and should not be limited to the embodiments illustrated in the accompanying drawings. Detailed Implementation

[0031] The following description is provided in relation to preferred embodiments of the present application, which are merely used to illustrate technical features of the present application, and are not used to limit the scope of the present application. Certain words are used throughout the description and claims to indicate certain features of the application. Those skilled in the art will recognize that manufacturers can use different names to refer to the same feature. Therefore, the present description and claims are not limited to the names used to refer to the features, but are limited to the features themselves. The terms "element", "system" and "device" used in the present application can be computer-related entities, which can be hardware, software, or a combination of hardware and software. The terms "comprise" and "include" used in the following description and claims are open terms, and should be interpreted to mean "including, but not limited to..." In addition, the term "coupled" means either a direct electrical connection or an indirect electrical connection via other devices or connections.

[0032] In the drawings, which are not necessarily drawn to scale, like numerals describe substantially similar components throughout the several views. Like numerals in different drawings represent like elements. The drawings illustrate only typical or exemplary embodiments of the application.

[0033] The term "substantially" or "approximately" used herein means within an acceptable range of a person skilled in the art who can solve the technical problems to be solved and substantially achieve the technical effects to be achieved. For example, "substantially equal" means that within an acceptable range, the skilled person can accept a certain error from "exactly equal" without affecting the correctness of the results.

[0034] Analog circuit sizing, also known as transistor sizing, is an iterative process for determining values of a set of sizing variables, including length, width, and multiplicities for each transistor in a given topology, to satisfy given specifications. Trade-offs are often present in design choices. For example, larger transistor sizes often result in better performance but consume more power and area.

[0035] A model-based reinforcement learning (RL) framework for analog circuit sizing is disclosed. The framework automatically adjusts analog circuit sizes under design constraints and incorporates a PVT exploration strategy. The framework includes RL agents that can quickly adapt to the environment based on learned experiences and can gradually approach optimality over time.

[0036] One aspect of the framework efficiently explores PVT conditions. At the system level, the framework improves research and development efficiency in the analog front-end sizing process. Experimental results show that the RL agents in the framework can efficiently search the design space of state-of-the-art designs with superior performance. At the algorithm level, the framework can directly mimic the dynamics of a circuit simulator (e.g., a SPICE simulator). At the verification level, the framework explores the input PVT conditions (e.g., stepwise explores the PVT conditions that fail the test and selects the worst PVT condition that fails the test to join the iteration process in the next iteration) and verifies whether the selected circuit size meets the specification for all input PVT conditions.

[0037] Figure 1 A model-based RL framework 100 for analog circuit sizing according to an embodiment of the present invention is shown (“framework 100”, also referred to as a platform). The framework 100 includes one or more RL agents 110 that interact with a circuit simulator 120, such as a SPICE simulation environment (also referred to as a SPICE simulator). Each RL agent 110 can be a neural network that learns through deep reinforcement learning during an iteration process. Each RL agent 110 can be a model-based agent that is initially trained through supervised learning in a local area, which is a portion of the design space. The local area is identified by the circuit simulator 120 at initialization to potentially contain circuit sizes that meet the specification and is dynamically updated during the iteration. The local area is also referred to as a trust region.

[0038] In an embodiment, the input to the framework 100 by the designers can include topology, specification, transistor size ranges, and PVT conditions. The topology, transistor size ranges, and PVT conditions are collectively referred to as a set of design parameters. The term “designers” used herein refers to design engineers. The framework 100 can initialize a plurality of RL agents, where each RL agent is for a different PVT condition. For example, assume the input PVT conditions are 9, then there can be 9 RL agents accordingly, each corresponding to a respective PVT condition. To simplify the description, later reference is made to Figures 4 to 6 The scenario of describing the plurality of RL agents is described. Reference is made to Figures 1 to 3 The following description of

[0039] The design space is a space of all sizing values that can be selected to determine the size of the analog circuit. Thus, the circuit sizing problem is a search problem in the given design space. Each sample in the design space is a vector of sizing variables with respective sizing values. For each sample in the design space, the RL agent 110 computes a measurement estimate that estimates the simulation measurement of the circuit simulator 120. The RL agent 110 is able to produce the measurement estimate faster than the circuit simulator 120 produces the simulation measurement.

[0040] The search problem is solved iteratively. In each iteration, the framework 100 applies a value function to each measurement estimate produced by the RL agent 110 to obtain a corresponding value metric. The sample corresponding to the measurement estimate with the highest (or optimal) value metric is selected as a candidate size. The candidate size is a set of assignments; that is, assigning the sizing values to the corresponding sizing variables. The circuit simulator 120 receives the candidate size and produces a simulation measurement to verify whether the candidate size satisfies the specification. The PVT condition manager 160 keeps track of the PVT conditions that have been verified by the circuit simulator 120 to satisfy the specification, and causes the framework 100 to initialize more RL agents for PVT conditions that do not satisfy the specification.

[0041] In an embodiment, the framework 100 further includes a gradient module 130 that computes updates to the weights of the RL agent 110. The gradient module 130 iteratively performs the updates using a gradient method based on a loss function that measures the difference (e.g., mean square error (MSE)) between the simulation measurement of the circuit simulator 120 and the measurement estimate of the RL agent 110. The framework 100 further includes a trust region update module 140 that iteratively updates a trust region for the RL agent 110 to search. The trust region is a portion of the design space. In an embodiment, the trust region is a circular region centered at the candidate size, whose radius can be dynamically expanded or shrunk in each iteration. In each iteration, the RL agent 110 receives a random sample in the trust region as input and generates a measurement estimate as output. The random sample can be generated by a random sample generator 150 using a Monte Carlo method, although the present disclosure is not limited thereto. Since the search for the candidate size is limited to the trust region rather than the entire design space in each iteration, the search can be performed at a high speed and efficiently.

[0042] Thus, the RL agent 110 identifies (or is provided with, i.e., the RL agent 110 picks) a candidate size in a random sample, and the circuit simulator 120 feeds back simulation measurements to update the RL agent 110 and the trust region, where the random sample for the next iteration is generated in the trust region. When a candidate size that satisfies the specification is found, the framework 100 outputs the circuit size. When a candidate size that satisfies the specification and all input PVT conditions is found, the circuit size is referred to as the “final circuit size”.

[0043] The interaction between the RL (reinforcement learning) agent 110 and the circuit simulator 120 (“agent-simulator loop”) replaces the traditional designer-simulator loop, in which a designer interacts with a circuit simulator to fine-tune circuit sizes through trial and error. The agent-simulator loop is more efficient and faster than the designer-simulator loop. The RL agent 110 is able to effectively identify and reject unqualified samples, allowing the circuit simulator 120 to focus on candidate sizes that potentially / possibly satisfy the specification.

[0044] Figure 2 is a flowchart illustrating a circuit sizing process 200 according to an embodiment of the present application. At step 210, the process 200 begins. In step 210, a circuit simulator (e.g., the circuit simulator 120) is initialized with a random circuit size (e.g., the circuit size 130) and a random sample (e.g., the sample 140) of PVT conditions. Figure 1The circuit simulator 120 generates simulation measurements on initial samples in the design space. For example, the initial samples can be a pre-determined number of samples randomly selected in the design space without any reduction. Based on the simulation measurements, the framework 100 identifies an initial candidate size among the initial samples (e.g., after sampling, simulation measurements are obtained by simulating each sample, and then the simulation measurements are used to determine which sample has the highest value, and the sample point corresponding to the highest value is used as a local region), and initializes a trust region and an RL agent at step 220. The trust region can be a circular region centered at the initial candidate size with an initial radius. The RL agent can be initially trained with the initial candidate size and the corresponding simulation measurements. At step 230, random samples are generated in the trust region. At step 240, the RL agent identifies a candidate size from the random samples. More specifically, for each random sample, the RL agent generates a measurement estimate. The sample corresponding to the best measurement estimate (which is related to a value indicator, and it can be understood that the best measurement estimate can be the measurement estimate corresponding to the highest value indicator) is sent to the circuit simulator as a candidate size. At step 250, the circuit simulator runs a circuit simulation on the candidate size to obtain a simulation measurement. It is determined whether the simulation measurement satisfies the specification (step 260), and if the simulation measurement satisfies the specification, the candidate size is selected (step 270). Further validation of the selected candidate size is performed for other PVT conditions. If the simulation measurement does not satisfy the specification, the trust region and the weights of the RL agent are updated (step 280). The updated trust region can be centered at the candidate size identified at step 240. Then, the process 200 returns to step 230 to start a new iteration with the updated trust region and the updated RL agent. The loop of steps 230-280 is an iterative training process of the RL agent in the trust region. In each iteration, the RL agent is trained with the candidate size and the corresponding simulation measurement. Experimental results show that the training is efficient and converges fast.

[0045] The following description provides mathematical formulations for analog circuit sizing. Generally, analog circuit sizing can be formulated as a constrained multi-objective optimization problem, which is defined in (1).

[0046]

[0047] where X is a vector of variables to be optimized; Ds is the design space; F m，c(X) is the mth objective function (e.g., power, performance, and area) under the cth PVT condition; C d，c (X) is the mth objective function (e.g., power, performance, and area) under the cth PVT condition; C

[0048] As PVT conditions grow exponentially in the fast-evolving technology, it is often infeasible to find a global optimal solution for (1). In contrast, it is more practical to satisfy the constraints specified by the designers. Therefore, the optimization problem described in (1) can be simplified as a constraint satisfaction problem (CSP). More generally, a CSP is defined as the triple <X, D, C> in (2).

[0049]

[0050] where X is a finite set of sizing variables to be searched. Each sizing variable has a non-empty domain D i , i.e., the design space, and {b1, b2,..., b1} is the possible values. C is a set of constraints. A constraint is a pair consisting of a constraint scope t j and a relation r j over the variables in the scope, which restricts the feasible assignments. The simulation performed by a circuit simulator (e.g., a SPICE simulator) is denoted as S pice , a function.

[0051] One effective method to solve the CSP in (2) is local search. The local search performed by the model-based RL framework 100 Figure 1 has the following triple advantages. 1) Faster adaptation capability: reducing the domain from the entire design space to a local region (i.e., the trust region), thus reducing the number of iterations to build up the space. In addition, the circuit space is locally continuous, i.e., the neighboring points around the known optimal value exhibit similar optimality. 2) Model-based agent has the capability of supervised learning: supervised learning works effectively in the local landscape. Since no reward is involved in the training of the model-based agent, the learning is not sensitive to reward engineering. 3) Easier implementation and convergence: the training procedure of supervised learning is relatively easier compared to model-free agents.

[0052] The model-based RL framework 100 provides a compact design space D Ldirect modeling. The model imitates the behavior of a SPICE simulator, which estimates simulation measurements S pice (X) of the circuit. In an embodiment, the model (e.g., Figure 1 The RL agent 110 in (X) can employ a feedforward neural network f N，N (X; 0) of three layers. The model can be used as a SPICE function approximator, as shown in (3).

[0053]

[0054] where, is a vector of predicted measurements (e.g., gain, phase margin, etc.) with respect to a vector of dimensions X estimated with weights 0.

[0055] The loss function J(0) can be obtained by mean squared error (MSE), as shown in (4).

[0056]

[0057] The model-based RL (reinforcement learning) agent (e.g., Figure 1 The RL (reinforcement learning) agent 110 in (X) aims to learn a prediction model f (N，N) to mimic the dynamics S pice of the environment. The prediction model f (N，N) is iteratively updated with a gradient method based on the loss function in (4). The model-based RL (reinforcement learning) agent explores feasible solutions rather than global solutions to prevent overdesigning the circuit. The RL (reinforcement learning) agent is also referred to as a neural network agent.

[0058] The value function is used to evaluate the merit of simulation measurements and measurement estimates. The output of the value function is referred to as a value merit. The value function does not participate in training the RL (reinforcement learning) agent, and thus does not affect the convergence of the neural network model. One non-limiting example of a value function (V alue ) is the sum of normalized measurements. For example, in some embodiments, there can be many value merits, such as performance, power consumption, etc. of the analog circuit, and embodiments of the present invention can sum up these normalized value merits as a comprehensive merit, i.e., as a final value merit. In particular, the present invention is not limited in this regard. Such a value function can be evaluated using readily available information. However, alternative value functions can be implemented to encode (e.g., consider / weight) the importance of each measurement with respect to tradeoffs between constraints.

[0059] Figure 3 is a schematic illustration of an algorithm (in pseudo-code) for exploring feasible circuit sizes according to an embodiment of the present application. At initialization, the algorithm starts random exploration in the design space. A SPICE simulator simulates N samples in the design space and determines the best sample (e.g., the sample with the highest value indicator of the simulation measurements) according to a value indicator of the simulation measurements. A region around the best sample is selected as the local region D TR . A model (i.e., a neural network surrogate) is constructed by exploring this local region. Monte Carlo sampling is used to randomly sample the local region, which has the advantage of fast inference time of the neural network surrogate. The neural network surrogate identifies candidate sizes in the local region based on the value indicator . A SPICE simulator runs simulations on the candidate sizes to produce simulation measurements. The local region is then updated by a trust region method (TRM) for the next iteration. The weights of the neural network surrogate are also updated.

[0060] One key factor for the performance of the neural network surrogate is the transition of the size of the search space from the global landscape to the local region. Therefore, the definition of the local property plays a role in the efficiency of the algorithm. The local region (also called trust region) is dynamically updated throughout the search process.

[0061] The trust region method defines an iteration-dependent trust region radius Δr j , where the model is considered as a proper representation of the objective function . At each iteration i, the trust region algorithm first solves the trust region subproblem (5) to obtain d *(i) . In an embodiment, this is achieved by Monte Carlo sampling.

[0062]

[0063] where d *(i) is the optimal trial step from the current center point, ||·|| is a norm. is the trust region.

[0064] The trust region method computes the ratio p 1The estimated reduction is the difference between the estimated function value at the current center point of the trust region and the estimated function value at the trial point (i.e., trial step away from the current center point). The estimated function value is a value indicator of the estimate Similarly, the actual function value is a value indicator of the simulation measurement The actual reduction is defined as the difference between the actual function value at the current center point of the trust region and the actual function value at the trial point. Based on the ratio p 1 Accept or reject the trial point. If the neural network is very close to the objective function The radius will expand. Very close is indicated by a ratio close to 1 (e.g., within a predetermined threshold). Otherwise, the radius will shrink. The update of the radius is computed based on the ratio.

[0065] Conceptually, the trust region is a circular region characterized by a center and a radius. The center is the best sample determined at initialization or during each iteration. The radius of the trust region is dynamically changed based on the accuracy of the model in the trust region with the current radius. The ratio described above is a measure or estimate of the accuracy. The radius is chosen such that it is not too large for the neural network to model, nor too small to require searching more local regions. The trust region method balances this tradeoff. If the neural network is able to model the trust region sufficiently (i.e., very close to the objective function), the radius can expand to allow searching in a larger space. If the neural network is unable to model the current trust region sufficiently, the radius is reduced or not changed to allow easier modeling.

[0066] In the algorithm of Figure 3 , the search of the circuit size is performed according to one PVT condition. The search can be extended to multiple PVT conditions as described below with reference to Figures 4-6 .

[0067] Figure 4 is a schematic diagram of a model-based RL platform 400 (“platform 400”) according to an embodiment of the present invention. The platform 400 is a model-based RL platform Figure 1An example of the framework 100 in FIG. 1 with more details. The platform 400 includes RL agents 410, SPICE environment (SPICE environment) 420, gradient update 430, trust region method 440, Monte Carlos sampling 450, and PVT exploration 460, which are examples of the RL agents 110, circuit simulator 120, gradient module 130, trust region update module 140, random sample generator 150, and PVT condition manager 160, respectively.

[0068] The PVT exploration 460 is used to maintain and update the condition pool P, where the condition pool P is part of the PVT exploration strategy. Initially, the condition pool only contains one PVT condition, e.g., the worst PVT condition among all PVT conditions specified in the designer’s input (i.e., the condition that is hardest to satisfy the specification for the simulated circuit according to prior knowledge or experience). The condition pool can be gradually expanded to include additional PVT conditions. Each PVT condition has its own independent model. That is, each PVT condition in the condition pool has a corresponding RL agent 410 (which is in the agent pool), and each RL agent 410 in the agent pool is trained to model a different PVT condition in the condition pool. Multiple RL agents 410 can simultaneously perform measurement estimation on the same set of random samples in the same trust region. In each iteration, the gradient update 430 updates the weights of each RL agent 410 based on a proxy-specific loss function (e.g., the MSE function), and the trust region method 440 determines a common trust region for all RL agents 410.

[0069] At each iteration, Monte Carlo sampling 450 generates a set of random samples for all RL agents 410 in the trust region. Each RL agent 410 computes a measurement estimate for each random sample. The platform 400 includes a value function module 470 for evaluating a value indicator for each measurement estimate from each RL agent 410. The sample corresponding to the measurement estimate with the largest value indicator is selected as the candidate size. For the case of multiple agents, each RL (reinforcement learning) agent 410 in the pool of agents first computes its best candidate size. Then, the worst candidate size among all the best candidate sizes is selected as the candidate size and sent to the SPICE environment 420. The "best" and "worst" candidate sizes are selected to maximize and minimize the value indicator of the corresponding measurement estimate, respectively. The SPICE environment 420 runs circuit simulation on the candidate size and the simulated measurement values are used to update the RL agents 410 and the trust region. The iterative process between the RL (reinforcement learning) agents 410 and the SPICE environment 420 continues until a final circuit size is found that satisfies the specification under all PVT conditions specified in the designer's input.

[0070] Figure 5 is a flowchart diagram of a PVT exploration process 500 according to an embodiment of the application. The PVT exploration process 500 is a progressive strategy. The PVT exploration process 500 first focuses the search on a single PVT condition; for example, the most difficult PVT condition. By overcoming the most difficult PVT condition, it will be easier to find a circuit size under other PVT conditions. A circuit size is a set of assignments; i.e., assigning size adjustment values to corresponding size adjustment variables. Once a circuit size is found that satisfies the specification, verification is performed to confirm that the circuit size also satisfies the specification under all other PVT conditions.

[0071] Reference Figure 5At step 510, the PVT exploration process 500 initializes an i-th RL agent (which does not yet satisfy the specification) for an i-th PVT condition (as a preferred embodiment, the present application is exemplified with the worst PVT condition), where i is a run index initialized to 1. At step 520, the i-th RL agent is added to the agent pool and the i-th worst PVT condition is added to the condition pool. At step 530, the RL agent(s) in the agent pool perform a search in the trust region to identify a candidate size under the PVT condition(s) in the condition pool. At step 540, the circuit simulator runs a circuit simulation for the candidate size. At step 550, if no circuit size satisfying the specification is found for all the PVT condition(s) in the condition pool, the weight and trust region of each RL agent are updated (step 560), and the process 500 returns to step 530, at which all the RL agents in the agent pool start another search in the updated trust region. It should be noted that the steps 530-560 in the dashed box 590 represent the operations of the circuit size tuning process 200 Figure 2 ) performed by multiple RL agents under multiple PVT conditions.

[0072] If a circuit size satisfying the specification is found for all the PVT conditions in the condition pool, the circuit simulator tests the circuit size under all other PVT conditions (step 570), i.e., not in the condition pool. At step 580, if the test indicates that all PVT conditions satisfy the specification, the circuit size is output as the final circuit size. At step 580, if the test indicates that not all PVT conditions satisfy the specification, the process 500 returns to step 510 (where the index i is incremented by 1) to initialize the next RL agent for the next worst PVT condition that fails to satisfy the specification. The process 500 continues until the final circuit size is found.

[0073] Figure 6is an example of the PVT exploration strategy according to embodiments of the present application. A first RL (Reinforcement Learning) agent is initialized and trained to search for a candidate size under the worst PVT condition (e.g., PVT3, which is the most difficult PVT condition to satisfy among the nine PVT conditions). After the first RL (Reinforcement Learning) agent identifies a candidate size that satisfies the specification under PVT3, the circuit simulator tests the candidate size against all other PVT conditions. Assume that the candidate size does not satisfy the specification under PVT5, PVT6, PVT7, and PVT9, where PVT6 is the most difficult PVT condition to satisfy / the worst PVT condition among PVT5, PVT6, PVT7, and PVT9, or the second worst PVT condition. A second RL (Reinforcement Learning) agent is initialized and trained to search for a candidate size under PVT6, while the first RL (Reinforcement Learning) agent performs a second round of search under PVT3. After the first RL (Reinforcement Learning) agent and the second RL (Reinforcement Learning) agent jointly identify a second candidate size that satisfies the specification under both PVT3 and PVT6, the circuit simulator tests the second candidate size against all other PVT conditions.

[0074] Assume that the second candidate size does not satisfy the specification only under PVT9. A third RL (Reinforcement Learning) agent is initialized and trained to search for a candidate size under PVT9, while the first RL (Reinforcement Learning) agent searches under PVT3 and the second RL (Reinforcement Learning) agent searches under PVT6. After all three RL (Reinforcement Learning) agents jointly identify a third candidate size that satisfies the specification under PVT3, PVT6, and PVT9, the circuit simulator tests the third candidate size against all other PVT conditions. Assume that the third candidate size satisfies the specification under all other PVT conditions, the third candidate size is output as the final circuit size solution to the analog circuit sizing problem. In embodiments of the present application, by exploring PVT conditions that do not pass the test / satisfy the specification step by step and adding the worst PVT condition that does not satisfy the specification and the corresponding RL agent to the test / iteration process in the process of identifying candidate sizes, the number of iterations can be reduced and the iteration process can be efficiently performed, so that the optimal solution can be quickly found.

[0075] Figure 7 is a flowchart of a method 700 for analog circuit sizing according to embodiments of the present application. The method 700 can be performed by Figure 8The system 800 performs the method 700. The method 700 begins at step 710, where the system receives input indicative of a specification of an analog circuit and a set of design parameters. At step 720, the system iteratively searches a design space until a circuit size that satisfies the specification and the design parameters is found. The iterative search includes steps 730 and 740. At step 730, the neural network agent computes a measurement estimate value for each sample randomly generated in a trust region to identify a candidate size corresponding to an optimized value indicator (or referred to as an optimal / highest value indicator, i.e., the highest one among multiple value indicators corresponding to multiple samples). The trust region is a portion of the design space. At step 740, the system computes an update to the weights of the neural network agent and the trust region based at least in part on a simulation measurement value of the candidate size by a circuit simulator for use in the next iteration.

[0076] In an embodiment, the value indicator at step 730 is an output of a value function that applies the measurement estimate value generated by the neural network agent with the candidate size as input.

[0077] In an embodiment, at initialization of the neural network agent, the system selects an initial candidate size that is used to optimize a simulation measurement value generated by a circuit simulator for an initial sample in the design space. The system initializes a trust region centered at the initial candidate size. The system also initializes the neural network agent that is trained using at least the initial candidate size and the corresponding simulation measurement value. In an embodiment, the trust region searched in a current iteration is centered at a candidate size identified in a previous iteration.

[0078] The model-based RL framework 100 and 400 can be implemented on one or more processors that execute instructions to perform the methods of the framework 100 and 400.

[0079] Figure 8 is a schematic diagram of a system 800 according to an embodiment of the present application. The system 800 includes hardware circuitry for performing the operations described in connection with Figures 1 to 7 The system 800 includes processing hardware 810. In an embodiment, the processing hardware 810 can include one or more processors 813, such as central processing units (CPUs), graphics processing units (GPUs), digital processing units (DPUs), artificial intelligence (AI) processors, and other general purpose and / or special purpose processing circuitry. Referring back to Figure 1 and Figure 4The one or more processors 813 can execute instructions stored in the memory 820 to perform operations of the model-based RL framework 100 and / or the model-based RL framework 400. The processing hardware 810 can also include non-programmable, fixed function hardware.

[0080] The memory 820 is coupled to the processing hardware 810. The memory 820 can include dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, and other non-transitory machine-readable storage media; e.g., volatile or non-volatile storage. The memory 820 can also include storage devices such as any type of disk drives, solid-state drives, or other storage devices. In an embodiment, the memory 820 can store instructions that, when executed by the processing hardware 810, cause the processing hardware 810 to perform the simulation circuit sizing operations described above, such as the method 500 in Figure 5 and the method 700 in Figure 7 .

[0081] The system 800 can also include a user interface 840 to obtain information from a designer. The designer can provide input via the user interface 840 to indicate one or more of: e.g., transistor sizes to tune, ranges of sizing variables, circuit topology, measurements to observe from a SPICE simulator, and specifications for each PVT condition. The memory 820 can store an automated script that, when executed by the processing hardware 810, builds the neural network agent and hyper-parameters of the neural network.

[0082] In some embodiments, the system 800 can also include a network interface 850 to connect to wired and / or wireless networks for transmitting and / or receiving voice, digital data, and / or media signals. It should be understood that Figure 8 Embodiments of the system 800 are simplified for illustrative purposes. Additional hardware components can be included, and the application is not limited to this example.

[0083] Figure 2 Figure 5 and Figure 7 The operations of the flow diagrams of Figure 1 Figure 4 and Figure 8 have been described with reference to the exemplary embodiments of Figure 2 Figure 5 and Figure 7 However, it should be understood that the operations of the flow diagrams of Figure 1 ,​​​Figure 4 and Figure 8 Embodiments outside of the embodiments of Figure 1 , Figure 4 and Figure 8 Embodiments of Figure 2 , Figure 5 and Figure 7 The flow diagrams of

[0084] Various functional means or blocks have been described herein. As will be readily appreciated by one skilled in the art, the functional blocks will preferably be implemented by circuitry (special purpose or general purpose) that carries out the recited functions and operations, which circuitry will typically include transistors configured to control the operation of the circuitry in accordance with the functions and operations described herein.

[0085] The use of ordinal terms such as "first", "second", "third", etc., in the claims to modify a claim element does not by itself connote any priority, precedence, or order of one claim element over another, or of an execution of one action before another, but rather merely makes a distinction between claim elements and actions. The use of the article "a" and / or "an" preceding an element or action of the claims is not intended to bear any meaning other than that of the definition of the element or action.

[0086] While the present application has been described in detail with respect to its embodiments and advantages, it should be appreciated that various modifications, substitutions, and alterations can be made to the present application without departing from the spirit and scope of the application as defined by the appended claims. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the present application is as defined by the appended claims.

Claims

1. A method for analog circuit sizing, comprising: receiving inputs indicative of a specification of an analog circuit and a plurality of design parameters; and, iteratively searching a design space until a circuit size satisfying the specification and the design parameters is found, wherein the iteratively searching further comprises: a neural network agent computing, for each of a plurality of samples randomly generated in a trust region, a measurement estimate to identify a candidate size corresponding to an optimal value indicator, wherein the trust region is a portion of the design space; and, computing, based at least in part on simulation measurements of the candidate size by a circuit simulator, an update of weights of the neural network agent and the trust region for a next iteration; wherein the design parameters include a plurality of process, voltage, temperature (PVT) conditions, and the method further comprises: identifying a circuit size satisfying the specification under a first PVT condition of the plurality of PVT conditions; the circuit simulator testing the circuit size satisfying the specification under PVT conditions other than the first PVT condition of the plurality of PVT conditions; and, gradually exploring PVT conditions that fail the test until a final circuit size satisfying the specification and all PVT conditions is found; wherein the gradually exploring comprises: adding a second PVT condition of the PVT conditions that fail the test to a condition pool, wherein the condition pool initially includes the first PVT condition; adding a second neural network agent for the second PVT condition to an agent pool, wherein the agent pool initially includes a first neural network agent for the first PVT condition; the neural network agents in the agent pool iteratively searching the same trust region to obtain updated circuit sizes satisfying the specification under the PVT conditions in the condition pool; and, increasing the agent pool and the condition pool for iteratively searching until the final circuit size satisfying the specification and all PVT conditions is found. the method further comprises:

2. The method of claim 1, wherein, selecting an initial candidate size based on simulation measurements generated by the circuit simulator for initial samples in the design space; initializing the trust region to be centered at the initial candidate size; and, initializing the neural network agent trained using at least the initial candidate size and corresponding simulation measurements. the trust region searched in a current iteration is centered at a candidate size identified in a previous iteration.

3. The method of claim 1, wherein, the circuit size is a solution to a constraint satisfaction problem defined by a set of constraints and a set of circuit variables, wherein each circuit variable corresponds to a set of predetermined sizing values.

4. The method of claim 1, wherein, computing the update further comprises:

5. The method of claim 1, wherein, computing a ratio to estimate accuracy of the neural network agent in the trust region from simulation measurements in the trust region; and, computing a change in a radius of the trust region from the ratio. the neural network agent is a multi-layer neural network learned through reinforcement learning.

6. The method of claim 1, wherein, the optimal value indicator is a highest value indicator obtained by a value function applied to the measurement estimate generated by the neural network agent.

7. The method of claim 1, wherein, the value indicator is an output of a value function that evaluates a sum of normalized measurements.

8. The method of claim 1, wherein, ​ 9. The method of claim 1, wherein, The first PVT condition is a worst PVT condition among the plurality of PVT conditions, and the second PVT condition is a worst PVT condition among the PVT conditions that fail the test, wherein the worst PVT condition is considered to be the most difficult condition to meet the specification according to prior knowledge.

10. An apparatus for analog circuit sizing, comprising: at least one processor; a memory coupled to the at least one processor and storing instructions which, when executed by the at least one processor, cause the at least one processor to perform the method for analog circuit sizing as claimed in any one of claims 1 to 9.

Images