A novel circuit parameter optimization method under PVT conditions

Abstract

The application provides a novel circuit parameter optimization method under PVT conditions, characterized by the following steps: parameterized modeling of the circuit, including at least the following parameters: process P, voltage V and temperature T; model initialization: set the comprehensive performance score of the scheme under a specific PVT corner point, and set the final optimization target reward as: ; group sampling and simulation: in each step of iteration of training, the strategy network generates a group of candidate design schemes containing different candidate design schemes for the same input state, parallel simulation verification is performed on each scheme in the group, each scheme needs to traverse all preset PVT corner points, and the worst-case reward value corresponding to each scheme is calculated according to the reward function definition; evaluation and update: after the reward value set of the group of schemes is obtained, the average value and standard deviation of all rewards in the group are calculated as the baseline, the "in-group relative advantage" of each scheme is calculated by using the in-group statistical information: ; finally, the in-group relative advantage is used to update the strategy network parameters, the appearance probability of the scheme with a positive advantage value is increased, so that the model is iterated to a local better area.

Description

Technical Field

[0001] This invention belongs to the field of automated design technology for analog integrated circuits, and particularly relates to a method for optimizing analog circuit parameters based on reinforcement learning under multiple variations in process, voltage, and temperature (PVT). Background Technology

[0002] With the continuous evolution of integrated circuit manufacturing processes, analog circuit design faces severe challenges. Unlike digital circuits, the performance of analog circuits (such as gain, bandwidth, and power consumption) is extremely sensitive to parameters such as transistor size and bias current. Especially in actual manufacturing and applications, process variations, supply voltage fluctuations, and environmental temperature changes can cause circuit performance to deviate significantly from nominal values, or even lead to functional failure. Therefore, during the design process, designers must find a set of robust parameters that can still meet design specifications under all extreme PVT conditions (i.e., worst-case). Summary of the Invention

[0003] In view of the above-mentioned deficiencies of the prior art, the present invention proposes a novel method for optimizing circuit parameters under PVT conditions.

[0004] An embodiment of the present invention provides a novel method for optimizing circuit parameters under PVT conditions, characterized by comprising the following steps: Parametric modeling of the circuit includes at least the following parameters: process P, voltage V, and temperature T. Model initialization: Let For the plan At a specific PVT corner Based on the overall performance score, the final optimization target reward is set as follows:

[0005] Group sampling and simulation: In each iteration of training, the policy network generates a set of samples containing the same input state. Different candidate design schemes For each solution within the group Perform parallel simulation verification for each scheme. All preset PVT corner points need to be traversed, and the reward function definition above needs to be used to calculate the reward for each scheme. Corresponding worst-case reward value ; Evaluation and Update: After obtaining the set of reward values ​​for this group of schemes. Then, calculate the average of all rewards within the group. and standard deviation As a baseline, the "within-group relative advantage" of each option is calculated using within-group statistics. :

[0006] Finally, the relative advantage within the group is used to update the policy network parameters, increasing the probability of schemes with positive advantage values, thereby guiding the model to iterate towards locally better regions.

[0007] Preferably, the evaluation and updating includes the following steps: Calculate rewards; Computational advantages; Calculate the gradient.

[0008] Preferably, the update direction is determined by the gradient of the objective function:

[0010] Preferably, computational advantages include: computational local advantages, computational global advantages, and constructing an effective advantage function of dual fusion.

[0011] Preferably, calculating local advantages includes: For action groups generated by the policy network and its corresponding worst-case reward The calculation can reflect the local advantages that reflect the relative strengths and weaknesses within the current sampling batch. .

[0012] Preferably, the calculation formula is:

[0013] in, This represents the average reward within the current group. Standard deviation, This is a smoothing term.

[0014] Preferably, calculating global advantage includes: maintaining a historical best reward record. During the optimization process, the highest and worst historical reward values ​​appearing in all sampled samples since the start of training are recorded and updated in real time.

[0015] Preferably, for the first in the group A single sample, its global advantage The calculation formula is:

[0017] Preferably, constructing the effective advantage function of dual fusion includes: constructing the final effective advantage function. By introducing a balance coefficient ( This involves linearly weighting and integrating local and global advantages.

[0018] Preferably, the calculation function is:

[0020] While retaining the advantage of "relative optimization within the group" of the GRPO algorithm, this embodiment of the invention further introduces "global historical advantage" as a supplementary evaluation dimension, constructing a dual advantage calculation mechanism that integrates local and global factors, thereby improving the efficiency of the algorithm in converging to the global optimal solution while ensuring exploration stability. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating an embodiment of the present invention.

[0022] Figure 2 This is a flowchart illustrating a preferred embodiment of the present invention. Detailed Implementation

[0023] The embodiments of the present invention will be described in detail below. The embodiments described below are implemented based on the technical solution of the present invention, and detailed implementation methods and specific operation processes are given. However, the protection scope of the present invention is not limited to the embodiments described below.

[0024] Figure 1 This is a flowchart illustrating an embodiment of the present invention. As shown in the figure: A novel method for optimizing circuit parameters under PVT conditions is provided, characterized by the following steps: Parametric modeling of the circuit includes at least the following parameters: process P, voltage V, and temperature T. Model initialization: Let For the plan At a specific PVT corner Based on the overall performance score, the final optimization target reward is set as follows:

[0025] Group sampling and simulation: In each iteration of training, the policy network generates a set of samples containing the same input state. Different candidate design schemes For each solution within the group Perform parallel simulation verification for each scheme. All preset PVT corner points need to be traversed, and the reward function definition above needs to be used to calculate the reward for each scheme. Corresponding worst-case reward value ; Evaluation and Update: After obtaining the set of reward values ​​for this group of schemes. Then, calculate the average of all rewards within the group. and standard deviation As a baseline, the "within-group relative advantage" of each option is calculated using within-group statistics. :

[0026] Finally, the relative advantage within the group is used to update the policy network parameters, increasing the probability of schemes with positive advantage values, thereby guiding the model to iterate towards locally better regions.

[0027] This embodiment provides a robust parameter optimization method for PVT in analog circuits based on an improved GRPO algorithm. While retaining the advantage of "intra-group relative optimization" of the GRPO algorithm, this embodiment further introduces "global historical advantage" as a supplementary evaluation dimension, constructing a dual-advantage calculation mechanism that integrates local and global factors. This ensures exploration stability while improving the efficiency of the algorithm's convergence to the global optimum.

[0028] In recent years, reinforcement learning (RL) has been widely applied to the automated search of circuit parameters due to its powerful exploratory capabilities. Among them, group relative policy optimization (GRPO), as an emerging policy gradient algorithm, has shown unique advantages. Unlike traditional methods based on the actor-critic framework, GRPO does not require training an additional value network (critic) to evaluate state values, thus avoiding the estimation bias problem of the critic network in sparse reward or high-noise environments during parameter optimization.

[0029] Figure 2 This is a flowchart illustrating a preferred embodiment of the present invention.

[0030] The GRPO-based analog circuit optimization scheme of this invention employs a "relative advantage within a group" mechanism to guide strategy updates. Its specific parameter optimization application under PVT conditions is as follows: Parametric modeling: constructing the circuit's variables to be optimized as vectors This includes the transistor's aspect ratio, multiplier, bias current, capacitance, and resistance values.

[0031] Worst-case driven reward function definition: To ensure the robustness of the designed circuit under process, voltage, and temperature (PVT) drift, existing technologies define the reward function according to the "weakest link" principle. That is, for any given design scheme... Its final reward This depends on the worst overall performance of the scheme among all preset PVT corner points. Specifically, it is set... For the plan At a specific PVT corner Based on the overall performance score, the final optimization target reward for this solution is:

[0032] This definition enables optimization algorithms to implement a lower bound on the performance of circuits.

[0033] Group Sampling and Simulation: In each iteration of training, the policy network does not perform single-point sampling, but instead generates a group containing data for the same input state. Different candidate design schemes Subsequently, the system analyzed each solution within the group. Perform parallel simulation verification. For each scheme, it is necessary to traverse all preset PVT corners and calculate the worst-case reward value for each scheme according to the above reward function definition. .

[0034] Evaluation and Update: After obtaining the set of reward values ​​for this group of schemes. Then, the algorithm calculates the average of all rewards within the group. and standard deviation As a baseline, the "within-group relative advantage" of each option is calculated using within-group statistics. :

[0035] Finally, the relative advantage within the group is used to update the policy network parameters, increasing the probability of solutions with positive advantage values ​​(i.e., performance better than the group average), thereby guiding the model to iterate towards locally better regions. The update direction is determined by the gradient of the objective function.

[0036] like Figure 2 As shown, the computational advantages include: 1. Calculate Local Advantage For action groups generated by the policy network and its corresponding worst-case reward First, calculate the local advantages that reflect the relative strengths and weaknesses within the current sampling batch. .

[0037] The calculation formula is:

[0038] in, This represents the average reward within the current group. Standard deviation, This is a smoothing term. The advantage of local dominance lies in leveraging the low variance within a group to perform a fine-grained gradient search within the neighborhood of the current parameter.

[0039] 2. Calculate Global Advantage By introducing a global perspective and utilizing historical best information during the optimization process to calculate global advantages, the aim is to guide the policy network to explore known global high-performance regions.

[0040] The specific calculation steps are as follows: Maintain a historical best reward record During the optimization process, the highest and worst historical reward values ​​appearing in all sampled samples since the start of training are recorded and updated in real time.

[0041] For the first in the group A single sample, its global advantage The calculation formula is:

[0042] (Note: Normalization may be applied based on the specific implementation; the core is the comparison with historical extremes.) The role of global advantage is to provide an absolute value anchor, suppressing exploration directions below the historical best level and guiding strategies toward known global high-performance regions.

[0043] 3. Constructing an Effective Advantage Function for Dual Fusion Constructing the final effective advantage function By introducing a balance coefficient ( The local advantages and global advantages are linearly weighted and integrated:

[0044] Using this Instead of the single intra-group advantage in traditional GRPO, a policy gradient objective function (such as ClippedSurrogate Objective) is used to evaluate the policy network parameters. Perform iterative updates. Through adjustments... This invention can maintain the diversity of exploration by leveraging local advantages in the early stage of training, and accelerate convergence to the Pareto front that satisfies PVT constraints by leveraging global advantages in the later stage of training.

[0045] The above preferred embodiments are aimed at improving the effect of parameter optimization by using a novel advantage calculation (combining local and global advantages) technique of GRPO in the selection of analog circuit parameters under PVT conditions.

[0046] 4. Policy Network Update Based on the calculated effective advantage value The policy network parameters are updated using a clipping objective function. By maximizing this objective function, the generation of high-performance networks is improved. The probability of value action is determined, thereby enabling the circuit parameter configuration to continuously evolve towards stronger PVT robustness and better overall performance.

[0047] After implementing the above-mentioned circuit parameter optimization method, the inventors conducted circuit tests, and the data are as follows: Table 1: Test Results of RAMOS_PFC Circuit

[0048] Table 2: Test Results of LEUNG_NMCF Circuit

[0049] Table 3: Test Results of LEUNG_DFCFC2 Circuit

[0050] Compared with the prior art, the improved solution proposed in this invention has the following beneficial effects: 1. Improved strategy stability under worst-case optimization. In existing technologies, if the overall worst-case performance of a sampling group in a certain round is unsatisfactory, the mere intra-group advantage can still generate positive update signals, leading to the misleading phenomenon of "choosing the best among the worst." This invention introduces a global advantage, using the historical best value R_best as an absolute reference, which can effectively suppress negative feedback in such low-quality sampling groups. This "global anchor" mechanism significantly reduces the oscillation of the strategy in the suboptimal region, ensuring that the optimization direction always points towards improving the lower bound of performance.

[0051] 2. It achieves an organic unity between local fine-tuning and global guidance. This invention does not abandon the original low variance advantage of GRPO, but defines it as a local advantage for fine-tuning within the parameter neighborhood; at the same time, it utilizes global advantages to overcome the limitations of the local perspective. This fusion mechanism enables the algorithm to possess both the high sample efficiency of GRPO and the macroscopic search capability of global optimization algorithms, making it particularly suitable for solving the problems of high parameter coupling and non-convex performance surfaces in PVT design of analog circuits.

[0052] 3. Accelerated convergence speed in high-dimensional spaces. In design scenarios with stringent PVT constraints, the solution space that satisfies the conditions is extremely narrow. Introducing global advantage is equivalent to adding a continuous "gravitational field" to the stochastic search process, rapidly pulling the policy network towards historically discovered high-performance regions. Experiments show that, compared to the traditional GRPO method without global advantage, this invention can search for circuit parameter configurations with higher robustness and better overall performance with fewer simulation iterations.

[0053] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A novel method for optimizing circuit parameters under PVT conditions, characterized in that, Includes the following steps: Parametric modeling of the circuit includes at least the following parameters: process P, voltage V, and temperature T. Model initialization: Let For the plan At a specific PVT corner Based on the overall performance score, the final optimization target reward is set as follows: Group sampling and simulation: In each iteration of training, the policy network generates a set of samples containing the same input state. Different candidate design schemes For each solution within the group Perform parallel simulation verification for each scheme. All preset PVT corner points need to be traversed, and the reward function definition above needs to be used to calculate the reward for each scheme. Corresponding worst-case reward value ; Evaluation and Update: After obtaining the set of reward values ​​for this group of schemes. Then, calculate the average of all rewards within the group. and standard deviation As a baseline, the "within-group relative advantage" of each option is calculated using within-group statistics. : Finally, the relative advantage within the group is used to update the policy network parameters, increasing the probability of schemes with positive advantage values, thereby guiding the model to iterate towards locally better regions.

2. The circuit parameter optimization method according to claim 1, characterized in that, The assessment and update include the following steps: Calculate rewards; Computational advantages; Calculate the gradient.

3. The circuit parameter optimization method according to claim 2, characterized in that, The update direction is determined by the gradient of the objective function: 。 4. The circuit parameter optimization method according to claim 2, characterized in that, Computational advantages include: computational local advantages, computational global advantages, and the construction of an effective advantage function for dual fusion.

5. The circuit parameter optimization method according to claim 4, characterized in that, Calculating local advantages includes: For action groups generated by the policy network and its corresponding worst-case reward The calculation can reflect the local advantages that reflect the relative strengths and weaknesses within the current sampling batch. .

6. The circuit parameter optimization method according to claim 5, characterized in that, The calculation formula is: in, This represents the average reward within the current group. Standard deviation, This is a smoothing term.

7. The circuit parameter optimization method according to claim 4, characterized in that, Calculating global advantage includes: maintaining a historical best reward record. During the optimization process, the highest and worst historical reward values ​​appearing in all sampled samples since the start of training are recorded and updated in real time.

8. The circuit parameter optimization method according to claim 7, characterized in that, For the first in the group A single sample, its global advantage The calculation formula is: 。 9. The circuit parameter optimization method according to claim 4, characterized in that, Constructing the effective advantage function of dual fusion includes: constructing the final effective advantage function. By introducing a balance coefficient ( This involves linearly weighting and integrating local and global advantages.

10. The circuit parameter optimization method according to claim 9, characterized in that, The calculation function is: 。

Images