Analog circuit size parameter optimization method based on graph neural network and reinforcement learning
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHENGZHOU UNIV
- Filing Date
- 2026-04-17
- Publication Date
- 2026-07-21
AI Technical Summary
Existing methods for optimizing the size parameters of analog circuits rely on human experience, fail to make full use of topological information, struggle to uniformly handle device coupling constraints, have low efficiency in joint optimization of multiple performance indicators, and have low training efficiency in reinforcement learning.
A graph neural network and reinforcement learning-based approach is adopted. By constructing a heterogeneous circuit diagram, the graph neural network is used to encode the analog circuit topology and device association constraints. The parameters are optimized by combining the reinforcement learning decision module. Furthermore, constraint projection and a phased reward mechanism are introduced to achieve automated optimization of dimensional parameters.
It improves state representation capabilities, reduces invalid searches, decreases manual parameter tuning workload, unifies the handling of device coupling constraints, and improves training stability and optimization efficiency.
Smart Images
Figure CN122433652A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of electronic design automation technology, and in particular to a method for optimizing the size parameters of analog circuits based on graph neural networks and reinforcement learning. Background Technology
[0002] In analog integrated circuit design, dimensional parameter optimization is a critical step affecting circuit performance and design cycle. Dimensional parameters typically include the channel width, channel length, and multiplication factor of MOS transistors, as well as the values of components such as resistors, capacitors, and inductors. Different combinations of dimensional parameters directly affect the circuit's gain, bandwidth, phase margin, power consumption, slew rate, noise, area, and stability under process voltage and temperature conditions.
[0003] Existing methods for optimizing the size parameters of analog circuits mainly rely on human experience, rule-based search, gradient methods, or heuristic optimization algorithms such as genetic algorithms and particle swarm optimization. Human experience-based methods depend on the long-term accumulation of designers and are difficult to adapt to large-scale parameter spaces; rule-based search methods are usually limited to specific circuit structures and lack versatility; gradient-based optimization methods are prone to getting trapped in local optima under strongly nonlinear, multi-constraint, and discrete coupling conditions; although heuristic optimization algorithms have a certain global search capability, they require a large number of simulation calls, which limits optimization efficiency.
[0004] Furthermore, compared to digital circuits, analog circuits have more complex topologies, device coupling relationships, and design constraints. For example, differential pair devices typically need to meet symmetry constraints, current mirror devices need to meet proportional constraints, and compensation and bias networks are strongly coupled to the main signal path. Existing optimization methods often treat each dimensional parameter as an independent variable, failing to fully utilize circuit topology and device constraint information, resulting in a large number of invalid or infeasible actions during the search process.
[0005] In recent years, reinforcement learning has been used to automate parameter search tasks. However, directly inputting size parameter vectors into a reinforcement learning model can easily overlook the connections and hierarchical dependencies between devices, resulting in insufficient state representation capabilities. On the other hand, common problems in analog circuit optimization, such as sparse reward signals, coupling conflicts between performance metrics, and high simulation costs, also lead to low training efficiency in reinforcement learning.
[0006] Therefore, it is necessary to propose an analog circuit size parameter optimization method that can explicitly express the analog circuit topology, take into account device correlation constraints, and make automatic decisions for multiple performance indicators. Summary of the Invention
[0007] The purpose of this invention is to provide a method for optimizing the size parameters of analog circuits based on graph neural networks and reinforcement learning, so as to solve the problems in the prior art of optimizing the size parameters of analog circuits that rely on human experience, lack of utilization of topological information, difficulty in uniformly handling device coupling constraints, and low efficiency of joint optimization of multiple performance indicators.
[0008] To achieve the above objectives, the present invention adopts the following technical solution: a method for optimizing analog circuit size parameters based on graph neural networks and reinforcement learning, comprising the following steps: Step S1, obtaining the netlist, device size parameter range, performance index constraints, and device association constraints of the analog circuit to be optimized; constructing a heterogeneous circuit diagram based on the netlist, and initializing node features, edge features, and the vector of size parameters to be optimized. Step S2, constructing a graph neural network encoding module to encode the heterogeneous circuit diagram, obtaining node embedding representations and circuit-level global embedding representations. Step S3, constructing a reinforcement learning decision module, using the circuit-level global embedding representation, local embedding representation, performance index deviation vector, and historical action information as the state input, and outputting the set of parameters to be adjusted and their adjustment amounts. Step S4, performing parameter updates based on the output actions and performing constraint projection to obtain the updated set of size parameters. Step S5, calling the circuit evaluation environment to obtain performance index results and calculating the reward value. Step S6, updating the reinforcement learning decision module based on the reward value, iteratively executing the above steps until the termination condition is reached, and outputting the optimized set of analog circuit size parameters.
[0009] Further, the heterogeneous circuit diagram adopts a graph structure representation of device nodes, network nodes, and connection edges to simultaneously retain device attribute information and network connection relationship information.
[0010] Furthermore, the reinforcement learning decision-making module adopts a Critic-Actor structure, uses a policy network to output action distribution, and uses a value network to evaluate the value of the current state, so as to achieve gradual optimization of the size parameters.
[0011] Furthermore, the parameter update process introduces a constraint projection operator to ensure that the action execution result always falls within the feasible region composed of boundary constraints, matching constraints, proportional constraints, and symmetry constraints.
[0012] Furthermore, the reward mechanism adopts a phased design, prioritizing the reduction of violations during the performance-infeasible phase, and further optimizing power consumption, area, and performance margin during the performance-feasible phase, in order to reduce the adverse impact of sparse rewards on the training process.
[0013] Compared with existing technologies, the present invention has at least the following beneficial effects: First, modeling the analog circuit topology using heterogeneous circuit diagrams and graph neural networks helps improve the ability of state representation to express the coupling relationship of devices; second, achieving joint decision-making of parameter sets and adjustment quantities through reinforcement learning decision modules helps reduce the workload of manual parameter tuning; third, uniformly processing the associated variables in differential pairs, current mirrors, and symmetrical structures through constrained projection mechanisms helps reduce invalid searches; fourth, the phased reward and optional proxy evaluation mechanism helps improve training stability and optimization efficiency. Attached Figure Description
[0014] Figure 1 This is a schematic diagram of the overall process of the method of the present invention.
[0015] Figure 2 This is a schematic diagram illustrating the interaction between the reinforcement learning decision module and the circuit evaluation environment of this invention.
[0016] Figure 3 This is a schematic diagram of the reinforcement learning network decision module of the present invention.
[0017] Figure 4 This is a schematic diagram of the reward calculation process of the present invention. Detailed Implementation
[0018] The present invention will be further described below with reference to the accompanying drawings. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.
[0019] This invention addresses the task of optimizing the size parameters of analog circuits. It transforms the analog circuit to be optimized from a traditional parameter representation into a graph structure representation, then uses a graph neural network to extract structure-aware features, and inputs these features into a reinforcement learning decision module. The reinforcement learning model then progressively selects the set of parameters to be optimized and their adjustment amounts. Finally, the circuit evaluation environment returns performance indicators and generates reward feedback, thus forming a closed-loop optimization process.
[0020] The invention is applicable to the task of optimizing the size parameters of analog or mixed-signal circuits such as operational amplifiers, comparators, low-noise amplifiers, bandgap references, phase-locked loop submodules, filters, and bias circuits.
[0021] In step S1, the original netlist, design specifications, and constraint information of the analog circuit to be optimized are first obtained. Design specifications may include, but are not limited to, DC gain, unity-gain bandwidth, phase margin, slew rate, quiescent power consumption, input reference noise, offset voltage, chip area, and performance constraints under process voltage and temperature conditions.
[0022] The netlist is parsed to identify MOS transistors, resistors, capacitors, inductors, current sources, and sub-modules, and a set of device nodes is constructed. Simultaneously, identify the electrical connection network between nodes and construct a network node set. If a pin of a device is connected to a network, a connection edge is established between the corresponding device node and the network node, forming a heterogeneous circuit diagram. .
[0023] In one embodiment, the node characteristics of a MOS device include at least: device type code, channel width W, channel length L, multiplier M, threshold type, drain-source current, transconductance, output conductance, operating area flag, whether it belongs to a differential pair, whether it belongs to a current mirror, and whether it belongs to a compensation branch. The node characteristics of passive devices such as resistors, capacitors, and inductors include at least: device type code, parameter value, branch category, and whether it is an adjustable parameter. The node characteristics of a network include at least: network connectivity, DC bias potential, whether it is an input terminal, whether it is an output terminal, whether it is a bias terminal, and whether it is a power supply terminal or a ground terminal.
[0024] The connection edge characteristics include at least pin type information to distinguish the gate, drain, source, and body of a MOS device, or the positive and negative ports of a passive device; they may also include parasitic estimation information, direction information, and connection weight information.
[0025] Simultaneously, the associated constraints in the netlist are identified. For differential input pairs, parameter equality constraints can be established; for current mirrors, aspect ratio constraints can be established; for symmetrical branches, mirror symmetry constraints can be established; and for preset parameter ranges, upper and lower boundary constraints can be established. Through these constraints, the feasible region Ω is formed.
[0026] All parameters to be optimized are arranged in a predetermined order to form a size parameter vector. Then, the parameters are normalized to obtain the initial optimized state.
[0027] In step S2, in order to obtain a high-dimensional representation that reflects the circuit topology and device coupling characteristics, the present invention constructs a graph neural network encoding module to perform message passing and feature aggregation on heterogeneous circuit diagrams.
[0028] During message passing at layer k, the message sent by node j to node i can be represented as:
[0029]
[0030] middle, Let j be the representation of node j at layer k. As edge features, For attention weights, This is the parameter matrix to be learned.
[0031] The node update expression is:
[0032]
[0033] in, The parameter matrix to be learned, For nodes The neighborhood group, It is a non-linear activation function.
[0034] After encoding by a K-layer graph neural network, the set of embedding representations for all nodes is obtained. To represent the global state of the entire circuit, a circuit-level global embedded representation g=READOUT is obtained through a readout function. The readout function may employ mean pooling, max pooling, attention pooling, or a combination thereof.
[0035] Through the above encoding process, the local device connection relationships, cross-branch coupling relationships, and circuit-level functional structures can all be mapped to a unified embedding space, providing a structure-aware state for subsequent reinforcement learning.
[0036] In step S3, the reinforcement learning decision module of the present invention preferably adopts an actor-critic structure, wherein the actor network outputs the action strategy and the critic network outputs the value estimate of the current state.
[0037] At time t, the state received by the reinforcement learning decision module can be represented as:
[0038]
[0039] in, This represents a circuit-level global embedding representation. This represents the local embedding representation corresponding to the candidate parameter group. This represents the current performance metric deviation vector. It represents historical action information, including but not limited to the number of times it was accessed, the magnitude of actions in recent rounds, and recent reward trends.
[0040] To improve search efficiency, instead of including all parameters in the action space at once, candidate parameter groups are constructed based on the degree of performance violation, local sensitivity, and historical access frequency. In one embodiment, the candidate parameter group may preferentially include the following types of parameters: first, parameters that are highly correlated with violation performance indicators; second, parameters whose devices are located in critical branches; and third, parameters that have not been accessed for a long time but have potential impact.
[0041] The actions output by the reinforcement learning module include the identifier of the parameter set to be adjusted. and adjustment amount .in, Used to indicate the set of parameters being optimized in this round. Used to represent the corresponding increment, decrement, or scaling amount.
[0042] In step S4, after the action is executed, the dimension parameter vector needs to be updated, and the update result must satisfy the preset feasible region constraint. The update expression is:
[0043]
[0044] in, This is the mask vector corresponding to the parameter group to be adjusted. To constrain the projection operator.
[0045] For example, when the parameter group being adjusted belongs to a differential pair, the constraint projection will ensure that the corresponding device sizes of the left and right branches remain equal; when it belongs to a current mirror structure, the constraint projection will synchronously adjust the relevant device parameters according to a preset ratio; when the parameter update exceeds the upper and lower limits, the constraint projection will truncate it to the allowable range.
[0046] This constraint projection mechanism can automatically correct parameter update results that violate constraints after the action is executed, avoiding simulator call failures or the generation of solutions without physical meaning.
[0047] In step S5, after the parameter update is completed, the circuit evaluation environment is invoked to evaluate the updated parameter set. The evaluation environment can be a circuit simulator, a performance prediction proxy model trained from historical data, or a combination of both.
[0048] In one embodiment, the surrogate model is used to quickly predict performance metrics. When the surrogate model's prediction is close to the performance boundary or its uncertainty exceeds a preset threshold, a circuit simulator is then invoked for accurate evaluation. The new simulation results can be fed back into the surrogate model's training set to continuously correct the surrogate model.
[0049] The circuit evaluation results constitute the performance index vector at the current moment. Based on the target indicator vector y* and various constraints, the normalized deviation is calculated, and the reward value is further calculated.
[0050] To alleviate the sparse reward problem in analog circuit optimization, this invention employs a phased reward mechanism. During the performance-infeasible phase, priority is given to reducing violations, and the reward can be written as:
[0051]
[0052] middle, Let be the normalized deviation of the k-th performance index at time t. This refers to the number of violations.
[0053] Once all key performance indicators meet the requirements, the performance optimization phase begins, and the reward can be written as:
[0054]
[0055] in, For performance margin, For power consumption indicators, For area indicators, Used to suppress instability caused by excessively large step sizes.
[0056] In practice, the weights of the reward items can be adjusted according to the circuit type. For example, for operational amplifiers, more attention can be paid to DC gain, unity-gain bandwidth, and phase margin; for low-noise amplifiers, more attention can be paid to noise figure, gain, and input-output matching; and for bandgap references, more attention can be paid to reference voltage accuracy and temperature coefficient.
[0057] In step S6, the reinforcement learning decision module preferably uses the proximal policy optimization algorithm for updating. Its policy objective function can be expressed as:
[0058]
[0059] in, For policy network parameters, The probability ratio between the old and new strategies. This is the estimate of the dominance function, where ε is the cutoff coefficient.
[0060] The critic network is used to estimate the state-value function to improve the stability of policy updates. By continuously sampling states, actions, and reward trajectories and updating the policy and value networks, iterative improvements to the size parameter optimization policy are achieved.
[0061] Termination conditions may include reaching the preset maximum number of iterations; the reward value changing less than a threshold for several consecutive rounds; or all target performance indicators meeting the requirements and remaining stable.
[0062] After training, the optimized set of dimensional parameters is output and written back to the original netlist or layout parameter file, generating updated analog circuit design results. For scenarios requiring post-simulation verification, layout post-simulation or corner simulation can be further invoked to perform final verification of the output results.
[0063] Taking a two-stage operational amplifier as an example, the netlist of the circuit to be optimized is first read to identify the input differential pair, mirror load, tail current source, second-stage amplification branch and compensation branch. Then, the input transistor, mirror transistor, current source transistor, resistor, capacitor and other devices are used as device nodes, and the connection networks are used as network nodes to construct a heterogeneous circuit diagram.
[0064] Then, set target indicators according to design requirements, including DC gain not lower than the preset threshold, unity gain bandwidth not lower than the preset threshold, phase margin not lower than the preset threshold, and static power consumption not higher than the preset threshold. Also, set the input differential pair parameters to be equal and the mirror branch size to be updated proportionally.
[0065] After encoding by the graph neural network, the reinforcement learning decision-making module prioritizes adjusting parameter sets that are strongly correlated with gain or phase margin deviation based on the current state. The adjustments are then evaluated using a circuit simulation environment after constraint projection, yielding new gain, bandwidth, phase margin, and power consumption results. The strategy is then updated based on the reward function. After multiple iterations, a set of dimensional parameters that meets the design specifications is output.
[0066] Those skilled in the art should understand that, without departing from the core ideas of this invention, the types of graph neural networks, reinforcement learning algorithms, reward function forms, parameter coupling rules, and circuit evaluation environments can be replaced or improved, and such improvements should all fall within the protection scope of this invention.
Claims
1. A method for optimizing the size parameters of analog circuits based on graph neural networks and reinforcement learning, characterized in that, The process includes the following steps: Step S1, obtaining the netlist, device size parameter range, performance index constraints, and device association constraints of the analog circuit to be optimized; constructing a heterogeneous circuit diagram based on the netlist, including device nodes, network nodes, and connection edges between device nodes and network nodes, and initializing node features, edge features, and the vector of size parameters to be optimized; Step S2, constructing a graph neural network encoding module, inputting the heterogeneous circuit diagram into the graph neural network encoding module, performing message passing and feature aggregation on the device nodes and network nodes to obtain node embedding representations and circuit-level global embedding representations; Step S3, constructing a reinforcement learning decision module, using the circuit-level global embedding representation, the local embedding representations corresponding to the candidate parameter groups, the current performance index deviation vector, and historical action information to form the state input, and outputting the parameter group to be adjusted and its adjustment amount; Step S4, performing parameter updates based on the actions output in Step S3, and performing constraint projection on the updated size parameter group to obtain an updated size parameter group that satisfies the device association constraints and parameter boundary constraints; Step S5: Call the circuit evaluation environment to evaluate the updated size parameter group, obtain the corresponding performance index results, and calculate the reward value based on the performance index results; Step S6: Feed the reward value back to the reinforcement learning decision module, perform iterative training on the reinforcement learning decision module, repeat steps S2 to S5 until the termination condition is reached, and output the optimized analog circuit size parameter group.
2. The method for optimizing analog circuit size parameters based on graph neural networks and reinforcement learning according to claim 1, characterized in that, In step S1, the device node includes at least a MOS transistor node, a resistor node, a capacitor node, an inductor node, a current source node, or a submodule node. The network node is used to represent the connection network in the circuit, and the connection edge is used to represent the association relationship between the device pin and the connection network. The device association constraint includes at least one or more of the following: differential pair constraint, current mirror constraint, scaling constraint, symmetry constraint, or layout matching constraint.
3. The method for optimizing analog circuit size parameters based on graph neural networks and reinforcement learning according to claim 2, characterized in that, In step S1, the device node features include at least one or more of the following: device type code, device size parameters, device working area identifier, bias feature, small signal feature, and device constraint flag; the network node features include at least one or more of the following: network type, network connectivity, network voltage feature, and network function identifier; and the connection edge features include at least one or more of the following: pin type, connection direction, and parasitic information.
4. The method for optimizing analog circuit size parameters based on graph neural networks and reinforcement learning according to claim 1, characterized in that, In step S2, the graph neural network encoding module updates the node representation using a multi-layer message passing mechanism. The update of the node representation at the k-th layer includes: sending a message from node j to node i. And updated by nodes The updated node representation is obtained; where, This represents the embedding representation of node i at layer k. Representing edge features, Indicates attention weights. Let i represent the set of neighbors of node i. and For trainable parameters, It is a non-linear activation function; after message passing is completed, the circuit-level global embedded representation is obtained through the readout function.
5. The method for optimizing analog circuit size parameters based on graph neural networks and reinforcement learning according to claim 1, characterized in that, In step S3, the state received by the reinforcement learning decision module at each moment is: ,in, This represents a circuit-level global embedding representation. This represents the local embedding representation corresponding to the candidate parameter group. This represents the current performance metric deviation vector. This represents historical action information; the candidate parameter group is obtained by filtering based on at least one of performance violation degree, parameter sensitivity, and historical access frequency.
6. The method for optimizing analog circuit size parameters based on graph neural networks and reinforcement learning according to claim 5, characterized in that, The action output in step S3 includes the identifier of the parameter group to be adjusted. and adjustment amount The parameter update expression in step S4 is: ,in, This represents the current size parameter vector. This represents the updated size parameter vector. Let Ω denote the constraint projection operator, and let Ω denote the feasible region consisting of parametric boundary constraints and device-related constraints. This represents the mask vector corresponding to the parameter group to be adjusted. This indicates element-wise multiplication.
7. The method for optimizing analog circuit size parameters based on graph neural networks and reinforcement learning according to claim 1, characterized in that, The reward value in step S5 adopts a phased reward mechanism, which includes at least a feasibility optimization phase and a performance optimization phase; in the feasibility optimization phase, the reward value is used to reduce performance violations. During the performance optimization phase, the reward value is used to optimize at least one of power consumption, area, gain, bandwidth, phase margin, slew rate, noise, or robustness while meeting performance constraints.
8. The method for optimizing analog circuit size parameters based on graph neural networks and reinforcement learning according to claim 7, characterized in that, The reward value in step S5 is defined as: when a performance metric violation occurs. When performance constraints are met, ;in, This represents the normalized deviation of the k-th performance index at time t. Indicates the number of violations. Indicates performance margin index, Indicates power consumption metrics. Indicating area index, and These are the weighting coefficients.
9. The method for optimizing analog circuit size parameters based on graph neural networks and reinforcement learning according to claim 1, characterized in that, The circuit evaluation environment in step S5 includes a circuit simulator and / or a performance prediction proxy model. When a performance prediction proxy model is used, the performance prediction proxy model first outputs the predicted performance index. When the uncertainty of the proxy model exceeds a preset threshold or the prediction result is close to the constraint boundary, the circuit simulator is called to perform accurate evaluation, and the newly obtained evaluation sample is added to the incremental update dataset of the proxy model.
10. The method for optimizing analog circuit size parameters based on graph neural networks and reinforcement learning according to claim 1, characterized in that, The termination conditions in step S6 include at least one of the following: reaching a preset number of iterations, convergence of reward values in multiple consecutive rounds, or all target performance indicators meeting preset requirements. After training, the optimized analog circuit size parameter set is written back to the original netlist or layout parameter file to generate updated analog circuit design results.