An improved BSIM parameter extraction method based on dueling DQN

Abstract

The application discloses an improved BSIM parameter extraction method based on a dueling DQN, aims to improve the efficiency and accuracy of parameter extraction in semiconductor device modeling, uses TCAD simulation data and WAT measurement data in the parameter extraction process, extracts parameters related to channel length, width, current and capacitance, models the BSIM parameter extraction problem as a Markov decision process, calculates target Q values and losses by creating a dueling DQN network model and performing model training, and updates the network parameters of the dueling DQN; in the training process, the root mean square error value is calculated, invalid actions are removed, even and odd action pairs are processed, the root mean square error value difference is sorted, the model converges, and the converged BSIM parameter value is obtained. The method can converge within a preset error range, and significantly improves the efficiency and accuracy of BSIM parameter extraction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of integrated circuit computer-aided design, and in particular to an improved BSIM parameter extraction method based on Dueling DQN. BACKGROUND

[0002] Compact models of transistors describe the characteristics of transistors in the form of equations. During the circuit design phase, these compact models are used for circuit simulation to ensure the normal operation of the circuit. These compact models can contain many parameters, and the values of the parameters determine the characteristics of the transistor. By fitting experimental data, multidimensional optimization is performed on Cgg-Vgs, Cgc-Vgs, Id-Vgs, Id-Vds, current first derivative (gm-Vgs), etc. curves to extract the model parameter values.

[0003] Among them, BSIM (Berkeley Short-channel IGFET Model) is usually used to simulate the charge and current behavior of MOSFET and other devices. Standard models such as BSIM4, BSIM-SOI, and BSIM-CMG contain hundreds of parameters.

[0004] Chinese invention patent CN118033385B discloses a method for constructing a compact model parameter extraction model of an integrated circuit device and an application. The constructed model includes multiple feature extraction units, each of which extracts features of only one type of electrical characteristic curve to ensure the independence and accuracy of feature extraction of different curves. In the feature fusion stage, physical features containing device multi-modal information are introduced, so that the compact model parameter extraction model can cope with the extraction of compact models of devices with different physical parameters, link device physics to circuit design, and realize end-to-end compact model parameter extraction. However, the feature extraction process of this method is complex, and its accuracy depends on the physical features containing device multi-modal information.

[0005] Currently, some deep learning algorithms are also used for parameter extraction, but deep learning methods lack explainability, are dependent on the quality of the data set, and have poor scalability. In addition, there are some methods based on genetic algorithms, but the calculation is too complex and the parameter extraction cost is high. Traditional DQN algorithms can be used for continuous state input problems, but they also face some problems, such as uniform sampling not considering the importance of different samples, low efficiency of exploring state space using ε-greedy method, and overestimation of Q value.

[0006] For BSIM parameter extraction tasks, for example, extracting n parameters, each parameter has m possible values, and the state space has n mPossibly, such a state space is huge, how to efficiently explore is a challenge, further research is still needed to improve the automation process of BSIM parameter extraction. SUMMARY

[0007] The problem to be solved by the present application is to provide an improved BSIM parameter extraction method based on Dueling DQN, which extracts multiple compact model parameters using IV and CV curves, can converge within a preset error range, significantly improves the efficiency and accuracy of BSIM parameter extraction, and realizes the automation process of BSIM parameter extraction.

[0008] The present application adopts the following technical scheme: a compact model parameter extraction method based on DQN algorithm, comprising the following steps:

[0009] S1, initializing the environment: setting the environment of BSIM parameter extraction, modeling the BSIM parameter extraction problem as a Markov decision process, and constructing a DQN model;

[0010] S2, model training: creating a Dueling DQN network model and performing model training, calculating target Q values and losses, and updating the network parameters of Dueling DQN;

[0011] S3, record and update the optimal step: record the model training process, calculate the root mean square error value in each time step, remove invalid actions and process even and odd action pairs, sort according to the difference of root mean square error value, and get the optimal parameters;

[0012] S4, model convergence: repeat steps S2 to S3, stop training when the BSIM parameter error is less than the preset requirement, get the converged BSIM parameter value, continue to iterate and converge along the last optimal parameter when the state is reset, and update the optimal solution of the parameter.

[0013] Preferably, the data used for BSIM parameter extraction mainly comes from two parts, including simulation data obtained by TCAD simulation software and WAT measurement data, for devices under different process nodes, use IV, CV characteristic curves under different sizes and temperatures as target curves, respectively used for extracting parameters related to channel length, channel width, capacitance-voltage characteristics, current-voltage characteristics and temperature.

[0014] Preferably, the BSIM parameter extraction problem is modeled as a Markov decision process, and the agent, state, action and reward are defined, the agent selects the corresponding action by perceiving the current state to maximize the reward obtained.

[0015] To achieve this goal, a deep Q network (DQN) is used as a value-based learning method, which is based on the Dueling DQN architecture, stores state-action-reward-next state tuples in a priority experience replay pool, and enables the agent to learn and optimize its policy between different state-action pairs through training neural networks, thereby improving the efficiency and accuracy of parameter extraction, and the agent interacts with the environment through Hspice simulation.

[0016] Preferably, the priority experience replay pool is implemented in the form of a binary tree; each leaf node stores the priority of the sample, the parent node is the sum of the priorities of adjacent leaf nodes, and the root node is the sum of all sample leaf nodes.

[0017] Preferably, step S2 model training includes the following sub-steps:

[0018] S2.1, create a Dueling DQN network model, define the structure of state input and output Q value, and improve the estimation of Q value by calculating state value and advantage function respectively;

[0019] S2.2, initialize the priority experience replay pool to store state s t , action a t , reward r and next state s t+1 tuple;

[0020] S2.3, reset the environment to the initial state, denoted as initial state s0, and determine whether there is an optimal parameter tuning step, if there is, randomly select to execute the optimal parameter tuning step or not, the parameter tuning step consists of a series of actions a

[0021] S2.4, in each interaction between the agent and the environment, the agent selects an action from the current state; the agent selects an action from the current state according to the greedy strategy, and the random exploration probability decreases with the number of executions.

[0022] S2.5, execute the action and obtain the new state and reward, store the obtained tuple in the experience replay pool, randomly sample a small batch of samples from the experience replay pool, and train the Dueling DQN network;

[0023] S2.6, calculate the target Q value and loss, and update the network parameters of the Dueling DQN.

[0024] Preferably, in step S3, the training process is recorded as follows:

[0025] In each time step of the Dueling DQN network model training, the recorded information includes: the current state s t , the executed action at , the obtained reward r t , the next state s after the current state t+1 , the current root mean square error value rms t , get the action sequence of the action list;

[0026] Determine whether the root mean square error value decreases, and whenever a new rms t value is recorded, check whether the rms t is less than the minimum root mean square error value recorded before, if so, find a new optimal step, clear the previously recorded action list, and update the minimum minimum root mean square error value, and record the action sequence at this time.

[0027] Further, in step S3, invalid actions are removed, as follows:

[0028] For each action in the recorded action list, check whether it leads to a value of the next state that exceeds the preset range, and if the execution of an action leads to a state that exceeds the range, remove it from the action list.

[0029] Preferably, in step S3, the even and odd action pairs are processed, and the recorded action list is traversed, and when processing each action, the following judgment is made:

[0030] If the current action is even, check the subsequent actions in the action list to find out whether there is an odd action corresponding to the current action; if the corresponding odd action is found, perform the removal operation to remove the current action and the corresponding odd action from the action list at the same time; if not, continue to move to the next action to continue checking;

[0031] If the current action is odd, check the subsequent actions in the action list to find out whether there is an even action corresponding to the current action, and perform the removal operation.

[0032] Preferably, in step S3, the sorting according to the difference in root mean square error values is performed, the difference between the current root mean square error value and the next root mean square error value is calculated, the action list is sorted in descending order according to the calculated RMS difference, and actions that have a greater improvement on the root mean square error value are given priority; only the top P best actions are retained, P is a hyperparameter, to form a new action list.

[0033] Compared with the prior art, the above technical scheme has the following technical effects:

[0034] 1. The BSIM parameter extraction method is improved, multiple compact model parameters are extracted using IV and CV curves, and Dueling DQN and priority experience pool algorithm are used to speed up model training and solve the problem of difficulty in convergence of the model.

[0035] 2、The improved BSIM parameter extraction method improves the BSIM parameter extraction method, explores the optimal parameter extraction step each time, and continues to explore along the previous optimal parameter extraction step when resetting the state next time, so that the optimal solution can be found more easily, the automatic process of BSIM parameter extraction can be realized, the target curve can be efficiently fitted, and the problem of low extraction parameter efficiency of the traditional dependence on manpower is solved. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 The improved BSIM parameter extraction method based on the Dueling DQN of the application is a flow chart;

[0037] Figure 2 The improved BSIM parameter extraction method based on the Dueling DQN of the application is a flow chart;

[0038] Figure 3 The improved BSIM parameter extraction method based on the Dueling DQN of the application is a flow chart;

[0039] Figure 4 The improved BSIM parameter extraction method based on the Dueling DQN of the application is a flow chart;

[0040] Figure 5 The improved BSIM parameter extraction method based on the Dueling DQN of the application is a flow chart. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical scheme and advantages of the application more clear, the technical scheme of the application will be further described in detail below with reference to the drawings, and the described embodiments are only a part of the embodiments involved in the application. All non-innovative embodiments of other researchers in the field on the basis of the embodiments belong to the protection scope of the application. At the same time, the step numbers in the embodiments are only set for the convenience of description and explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0042] The application relates to an improved BSIM parameter extraction method based on Dueling DQN, which extracts multiple compact model parameters by using IV and CV curves, and comprises the following steps:

[0043] S1, initializing the environment: setting the environment of BSIM parameter extraction, modeling the BSIM parameter extraction problem as a Markov decision process, and constructing a DQN model;

[0044] S2, model training: create a Dueling DQN network model and perform model training, calculate target Q value and loss, and update the network parameters of the Dueling DQN;

[0045] S3, record and update the optimal step: record the model training process, calculate the root mean square error value in each time step, remove invalid actions and process even and odd action pairs, sort according to the difference in root mean square error value, and obtain the optimal parameters;

[0046] S4, model convergence: repeat steps S2 to S3, stop training when the BSIM parameter error is less than the preset requirement, and obtain the converged BSIM parameter value. When the state is reset, continue to iterate and converge along the last optimal parameter, and update the optimal solution of the parameters.

[0047] In an embodiment of the present application, the data used for BSIM parameter extraction mainly comes from two parts, including simulation data obtained by TCAD simulation software and WAT measurement data. For devices under different process nodes, use IV and CV characteristic curves under different sizes and temperatures as target curves for extracting parameters related to channel length, channel width, capacitance-voltage characteristics, current-voltage characteristics, and temperature.

[0048] Further, the present embodiment models the BSIM parameter extraction problem as a Markov Decision Process (MDP). As shown in Figure 1 A reinforcement learning method using a Dueling DQN architecture is described, which optimizes decision-making in a simulated environment by creating and training a model. Priority experience replay is used in training to efficiently utilize experience data, and an ε-greedy strategy is used for exploration and exploitation. In this process, by recording and optimizing the optimal parameter tuning action sequence, the most effective strategy for performance improvement is selected, and the model parameters are converged, so as to realize the optimal adjustment of the environment parameters.

[0049] Four key components are defined in the reinforcement learning model: agent, state, action, and reward.

[0050] Specifically, the agent perceives the current state and selects the corresponding action to maximize the reward obtained. To achieve this goal, a deep Q network (DQN) is used as a value-based learning method. By training the neural network, DQN enables the agent to learn and optimize its strategy between different state-action pairs, thereby improving the efficiency and accuracy of parameter extraction.

[0051] The BSIM (Berkeley Short-channel IGFET Model) model is an electrical model widely used in semiconductor device modeling, particularly in the design and analysis of modern CMOS (Complementary Metal-Oxide-Semiconductor) technology. It is primarily used to describe the electrical characteristics of short-channel MOSFETs (Metal-Oxide-Semiconductor Field-Effect Transistors) and can effectively predict device performance under different operating conditions.

[0052] For each type of device characteristic curve, the parameters that can be extracted from each set of curves are not the same. A model can be trained for each set of curves to extract parameters. Each model is designed to use the specific characteristics of the parameters assigned to it, iteratively predict parameter values, and continue to correct them until the error is reduced to an acceptable value.

[0053] Reinforcement learning is a paradigm of machine learning in which an agent learns through interaction with the environment, performing actions and receiving rewards. This is different from supervised and unsupervised learning, which typically rely on data sets for training. Reinforcement learning allows continuous learning through interaction with the environment, especially when dealing with large amounts of data and complex combination problems.

[0054] The improved Dueling DQN algorithm is used in this embodiment for implementation and evaluation. The initial test involves a single parameter system change, while all other parameters are kept at their predefined values to minimize fitting errors.

[0055] For the definition of the action space, i.e., setting each extraction parameter to have two actions of increasing and decreasing, the step size of parameter change must be a reasonable number before training. If it is too large, it means an increase in the search space, and if it is too small, it will result in a loss of precision.

[0056] The state represents a vector of information such as current parameter values and voltage characteristics. The reward is set by changing the fitting error relative to the error before parameter extraction. If the current action results in a decrease in error, a positive reward is given; otherwise, a negative reward is given.

[0057] Interaction between the agent and the environment is achieved through Hspice simulation. Each action performed modifies the parameter values in the model card, and the reference curve data is obtained from Hspice.

[0058] The implementation of this embodiment is based on the improved BSIM parameter extraction method using Dueling DQN, and the specific process is as follows:

[0059] 1. Initialize the environment:

[0060] Set up the environment for BSIM parameter extraction.

[0061] The DQN model is constructed using the Dueling DQN architecture.

[0062] A priority experience replay pool is constructed to store state-action-reward-next state tuples in the form of a binary tree, with each leaf node storing the priority of a sample, the parent node being the sum of the priorities of adjacent leaf nodes, and the root node being the sum of all sample leaf nodes.

[0063] 2. Training process:

[0064] 2.1. Create a Dueling DQN network, define the structure of state input and output Q value, and improve the estimation of Q value by calculating state value and advantage function respectively.

[0065] 2.2. Initialize the priority experience replay pool to store state s t , action a t , reward r and next state s t+1 tuple.

[0066] 2.3. Reset the environment to the initial state, denoted as initial state s0, and determine whether there is an optimal tuning step. If there is an optimal tuning step, take a random number in the range of 0-1 with a hyperparameter of 0.1. If the random number is less than 0.1, execute the tuning step sequentially, which consists of a series of actions a.

[0067] 2.4. In each interaction between the agent and the environment, the agent selects an action from the current state (according to the ε-greedy strategy). In this embodiment, ε is set to 0.6 and will decrease with the number of executions, with a step size of:

[0068] ε=(ε inital -ε finall ) / n

[0069] where n is the number of times the value decreases to the predetermined value, set to 500.

[0070] 2.5. Execute the action and obtain the new state and reward, store (state s t , action a t , reward r, next state s t+1 ) in the experience replay pool, and randomly sample a small batch of samples from the experience replay pool for network training.

[0071] In this embodiment, the sample size batch_size is set to 64.

[0072] 2.6. Calculate the target Q value and loss, and update the network parameters of the Dueling DQN.

[0073] 3. Record and update the optimal step:

[0074] 3.1. Record the training process, determine if the rms is reduced

[0075] In each time step of the training, record four important information: the current state s t , the action a t executed, the reward r t obtained, and the next state s t+1 after the current state; t The current root mean square error value rms

[0076] Whenever a new RMS value is recorded, check if this value is less than the minimum RMS value recorded before. If so, it means that a new optimal step has been found: in this case, clear the action list recorded before and update the minimum RMS value. At the same time, record the action sequence at this time for subsequent analysis and use;

[0077] 3.2. Remove invalid actions:

[0078] For each action in the recorded action list, check if the action leads to the value of the next state beyond the preset range. If the execution of a certain action leads to the state beyond the range, it will be removed from the action list.

[0079] 3.3. Process even and odd action pairs:

[0080] Traverse the recorded action list, and when processing each action, make the following judgment: if the current action is even, you need to check the subsequent actions in the list: find out if there is a case where the odd action corresponds to the current action. If such a subsequent action is found, the current action and this subsequent action will be removed from the list at the same time. If not, move on to the next action and make the same check. If the current action is odd, use a similar way: find out if there is a case where the even action corresponds to the current action, and handle the relevant removal operation.

[0081] 3.4. Sort according to the RMS difference:

[0082] Calculate the difference RMS diff between the current RMS value RMS t and the next RMS value RMS t+1 , that is:

[0083] RMS diff = RMS t+1 - RMS t

[0084] According to the calculated RMS difference, the action list is sorted in descending order, so that the actions that have greater improvement on the RMS value are given priority. Finally, only the top 10 actions with the best performance are retained for subsequent training.

[0085] 4. Repeat the above steps until convergence, i.e., stop training when the parameter error reaches an acceptable value.

[0086] In this embodiment, the initial values of the parameters and the state are set as shown in the following table:

[0087]

[0088]

[0089] Further, the relationship between the target curve and the reference curve before fitting in this embodiment is shown in Figure 2 and Figure 3 , taken from a large-size device, 28nm process; Figure 2 The curve is the Cgg-Vgs curve before fitting, the dashed line is the measured data curve, and the solid line is the simulation result; Figure 3 The curve is an illustration of the Id-Vgs and its first derivative, logarithmic curve relationship before fitting.

[0090] The relationship between the target curve and the reference curve after fitting is shown in Figure 4 and Figure 5 , Figure 4 The curve is the Cgg-Vgs curve of the fitting result, Figure 5 The curve is an illustration of the Id-Vgs and its first derivative, logarithmic curve relationship of the fitting result.

[0091] In summary, the present application aims to propose a stable and effective BSIM parameter extraction method based on reinforcement learning. Each exploration records the optimal parameter extraction step, and when the state is reset next time, it is easier to find the optimal solution along the previous optimal parameter extraction step. In addition, by using Dueling DQN and priority experience pool algorithm, the model training is accelerated and the problem of model not easy to converge is solved.

[0092] The above only describes the preferred embodiments of the present application, and it should be noted that for ordinary skilled persons in the art, without departing from the principles of the present application, several improvements and refinements can be made, and these improvements and refinements should also be considered as the protection scope of the present application.

Claims

1. An improved BSIM parameter extraction method based on Dueling DQN, characterized in that, The method comprises the following steps: S1, initializing the environment: setting the environment for BSIM parameter extraction, modeling the BSIM parameter extraction problem as a Markov decision process, and constructing a DQN model; The BSIM parameter extraction uses TCAD simulation data and WAT measurement data, and uses IV and CV characteristic curves at different sizes and temperatures as target curves to extract channel length, channel width, capacitance-voltage characteristic, current-voltage characteristic, and temperature-related parameters for devices at different process nodes; The BSIM parameter extraction problem is modeled as a Markov decision process, and the agent, state, action and reward are defined, the agent selects the corresponding action by perceiving the current state to maximize the reward obtained; For the definition of the action space, set two actions of increasing and decreasing for each extracted parameter, and the step length of parameter change is determined in advance before training; The state represents the vector of the current parameter value and the voltage characteristic; The reward is set by the change of the fitting error relative to the error when the parameter extraction has not been performed initially, if the current action leads to a decrease in error, a positive reward is given, otherwise, a negative reward is given; The DQN model uses a deep Q network as a value-based learning method, is constructed based on a Dueling DQN architecture, stores state-action-reward-next state tuples in a priority experience replay pool, and learns and optimizes the policy between different state-action pairs; the interaction between the agent and the environment is realized through Hspice simulation; S2, model training: creating a Dueling DQN network model and performing model training, calculating target Q values and losses, and updating the network parameters of the Dueling DQN; S3, recording and updating the optimal step: recording the model training process, calculating the root mean square error value in each time step, removing invalid actions and processing even and odd action pairs, sorting according to the difference in root mean square error value to obtain the optimal parameters; The processing of even and odd action pairs traverses the recorded action list, and the following judgment is performed when processing each action: If the current action is even, check the subsequent actions in the action list to find out if there is a corresponding odd action for the current action; if the corresponding odd action is found, the remove operation is performed to remove the current action and the corresponding odd action from the action list at the same time; if not, continue to move to the next action to continue checking; If the current action is odd, check the subsequent actions in the action list to find out if there is a corresponding even action for the current action, and perform the remove operation; S4, model convergence: repeating steps S2 to S3, stopping training when the BSIM parameter error is less than the preset requirement, obtaining the converged BSIM parameter value, and continuing to iterate and converge along the last optimal parameter when the state is reset, and updating the optimal solution of the parameters.

2. The Dueling DQN-based improved BSIM parameter extraction method according to claim 1, wherein, The priority experience replay pool is implemented in the form of a binary tree; each leaf node stores the priority of the sample, the parent node is the sum of the priorities of the adjacent leaf nodes, and the root node is the sum of all sample leaf nodes.

3. The Dueling DQN-based improved BSIM parameter extraction method according to claim 1, wherein, Step S2 model training comprises the following substeps: S2.1, create a Dueling DQN network model, define the structure of state input and output Q value, the Dueling DQN network model improves the estimation of Q value by calculating state value and advantage function respectively; S2.2, initialize a prioritized experience replay pool storing state s t , action a t , reward r and next state s t+1 tuple; S2.3, reset the environment to the initial state, denoted as initial state s0, judge whether there is an optimal parameter adjustment step, if there is, randomly select to execute the optimal parameter adjustment step or not, the parameter adjustment step is composed of a series of actions a; S2.4, in each interaction between the agent and the environment, the agent selects an action from the current state; S2.5, execute the action and obtain the new state and reward, store the obtained tuple into the experience replay pool, randomly extract a small batch of samples from the experience replay pool, and train the Dueling DQN network; S2.6, calculate the target Q value and loss, and update the network parameters of the Dueling DQN.

4. The improved BSIM parameter extraction method based on Dueling DQN according to claim 3, characterized in that, In step S2.4, the agent selects an action from the current state according to the greedy strategy, and the random exploration probability decreases with the number of executions, and the step length of each decrease is: ε = (ε inital - ε finall ) / n where n is the number of times required to fall to a predetermined value, ε inital , ε finall The random exploration probabilities before and after the fall, respectively.

5. The improved BSIM parameter extraction method based on Dueling DQN according to claim 3, wherein, In step S3, the training process is recorded as follows: In each time step of the Dueling DQN network model training, the recorded information includes: the current state s t , the executed action a t , the obtained reward r t , the next state s t+1 after the current state, the current root mean square error value rms t , and the action sequence of the action list; Judge whether the root mean square error value decreases, and whenever a new rmst value is recorded, check whether the rmst is less than the minimum root mean square error value recorded before, if so, find a new optimal step, clear the previously recorded action list, and update the minimum root mean square error value, and record the action sequence at this time.

6. The Dueling DQN-based improved BSIM parameter extraction method according to claim 5, wherein, In step S3, the invalid actions are removed as follows: For each action in the recorded action list, check whether it leads to a state value beyond the preset range, if the execution of an action leads to a state beyond the range, remove it from the action list.

7. The improved BSIM parameter extraction method based on Dueling DQN according to claim 5, characterized in that, In step S3, the actions are sorted according to the difference between the root mean square error values, the difference between the current root mean square error value and the next root mean square error value is calculated, and the action list is sorted in descending order according to the calculated RMS difference, giving priority to the actions that have greater improvement on the root mean square error value; Only the top P optimal actions are kept to form a new action list, and P is a hyperparameter.

Images