Improved BSIM parameter extraction method based on Dueling DQN

Abstract

The invention discloses an improved BSIM parameter extraction method based on Dueling DQN, and aims to improve the efficiency and accuracy of parameter extraction in semiconductor device modeling, TCAD simulation data and WAT measurement data are used in the parameter extraction process, parameters related to channel length, width, current and capacitance are extracted, the BSIM parameter extraction problem is modeled as a Markov decision process, and the efficiency and accuracy of parameter extraction in semiconductor device modeling are improved. The method comprises the following steps: establishing a Dueling DQN network model and performing model training, calculating a target Q value and loss, and updating network parameters of the Dueling DQN; in the training process, root-mean-square error values are calculated, invalid actions are removed, even and odd action pairs are processed, sorting is carried out according to the difference of the root-mean-square error values, and converged BSIM parameter values are obtained through model convergence. According to the method, convergence can be carried out in a preset error range, and the efficiency and the accuracy of BSIM parameter extraction are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of integrated circuit computer-aided design, and in particular to an improved BSIM parameter extraction method based on Dueling DQN. Background Art

[0002] The compact model of the transistor describes the characteristics of the transistor in the form of equations. During the circuit design stage, these compact models are used for circuit simulation to ensure the normal operation of the circuit. These compact models may contain many parameters, and the values ​​of the parameters determine the characteristics of the transistor. By fitting the experimental data, multi-dimensional optimization is performed on the curves such as Cgg-Vgs, Cgc-Vgs, Id-Vgs, Id-Vds, and the first-order derivative of current (gm-Vgs), and the model parameter values ​​are extracted.

[0003] Among them, BSIM (Berkeley Short-channel IGFET Model) is usually used to simulate the charge and current behavior of devices such as MOSFET. Standard models such as BSIM4, BSIM-SOI, and BSIM-CMG contain hundreds of parameters.

[0004] Chinese invention patent CN118033385B discloses a method for constructing a compact model parameter extraction model for integrated circuit devices and its application. The constructed model includes multiple feature extraction units, each of which only extracts the features of one type of electrical characteristic curve to ensure the independence and accuracy of feature extraction of different curves. At the same time, physical features containing multimodal information of the device are introduced in the feature fusion stage, so that the compact model parameter extraction model can cope with the extraction of compact models of devices with different physical parameters, linking the device physics to circuit design, and realizing end-to-end compact model parameter extraction. However, the feature extraction process of this method is complicated, and its accuracy depends on the physical features of the multimodal information of the device.

[0005] There are also some deep learning algorithms for extracting parameters, but deep learning methods lack interpretability, rely on the quality of the data set, and have poor scalability. In addition, there are some methods based on genetic algorithms, but the calculations are too complicated and the cost of extracting parameters is high. The traditional DQN algorithm can be used for continuous state input problems, but it also faces some problems. For example, uniform sampling fails to take into account the different importance of samples, the use of ε-greedy method to explore the state space is inefficient, and the Q value is overestimated.

[0006] For the BSIM parameter extraction task, for example, if n parameters are extracted and each parameter has m possible values, the state space has n mThere are many possibilities. Such a state space is huge. How to explore it efficiently is a challenge. Further research is still needed to improve the automated process of BSIM parameter extraction. Summary of the invention

[0007] The problem to be solved by the present invention is to provide an improved BSIM parameter extraction method based on Dueling DQN, which utilizes IV and CV curves to extract multiple compact model parameters, can converge within a preset error range, significantly improve the efficiency and accuracy of BSIM parameter extraction, and realize the automated process of BSIM parameter extraction.

[0008] The present invention adopts the following technical solution: a compact model parameter extraction method based on the DQN algorithm, comprising the following steps:

[0009] S1. Initialize the environment: Set up the BSIM parameter extraction environment, model the BSIM parameter extraction problem as a Markov decision process, and build a DQN model;

[0010] S2, model training: Create Dueling DQN network model and perform model training, calculate target Q value and loss, and update Dueling DQN network parameters;

[0011] S3. Record and update the optimal steps: record the model training process, calculate the root mean square error value in each time step, remove invalid actions and process even and odd action pairs, sort according to the root mean square error value difference, and obtain the optimal parameters;

[0012] S4, model convergence: Repeat steps S2 to S3, stop training when the BSIM parameter error is less than the preset requirement, obtain the converged BSIM parameter value, and when resetting the state, continue iterating and converging along the last optimal parameter to update the optimal parameter solution.

[0013] Preferably, the data sources used for BSIM parameter extraction mainly include two parts, including simulation data obtained by simulation software such as TCAD and WAT measurement data. For devices at different process nodes, IV and CV characteristic curves of different sizes and temperatures are used as target curves, which are used to extract parameters related to channel length, channel width, capacitance-voltage characteristics, current-voltage characteristics, and temperature, respectively.

[0014] Preferably, the BSIM parameter extraction problem is modeled as a Markov decision process, defining an agent, state, action and reward, wherein the agent selects a corresponding action by sensing the current state to maximize the reward obtained.

[0015] To achieve this goal, a deep Q network (DQN) is adopted as a value-based learning method, which is built on the Dueling DQN architecture. Through the prioritized experience replay pool, the state-action-reward-next-state tuple is stored. By training the neural network, DQN enables the agent to learn and optimize its strategy between different state-action pairs, thereby improving the efficiency and accuracy of parameter extraction. The interaction between the agent and the environment is realized through Hspice simulation.

[0016] Preferably, the priority experience replay pool is implemented in the form of a binary tree; each leaf node stores the priority of the sample, the parent node is the sum of the priorities of adjacent leaf nodes, and the root node is the sum of all sample leaf nodes.

[0017] Preferably, step S2 model training includes the following sub-steps:

[0018] S2.1. Create a Dueling DQN network model and define the structure of state input and output Q value. The Dueling DQN network model improves the estimation of Q value by calculating state value and advantage function respectively.

[0019] S2.2. Initialize the priority experience replay pool and store the state s t 、Action a t , reward r and next state s t+1 tuple of ;

[0020] S2.3. Reset the environment to the initial state, recorded as the initial state s0, and determine whether there is an optimal parameter adjustment step. If so, randomly select to execute the optimal parameter adjustment step or not to execute it. The parameter adjustment step consists of a series of actions a.

[0021] S2.4. In each interaction between the agent and the environment, the agent selects an action from the current state; the agent selects an action from the current state according to the greedy strategy, and the probability of random exploration decreases with the number of executions.

[0022] S2.5. Execute the action and obtain the new state and reward, store the obtained tuple in the experience replay pool, randomly extract small batch samples from the experience replay pool, and train the Dueling DQN network;

[0023] S2.6. Calculate the target Q value and loss, and update the network parameters of Dueling DQN.

[0024] Preferably, in step S3, the training process is recorded as follows:

[0025] At each time step of Dueling DQN network model training, the recorded information includes: the current state s t , Action performed at , the reward received t , the next state after the current state s t+1 The current root mean square error value rms t , get the action sequence of the action list;

[0026] Determine whether the root mean square error value decreases, and whenever a new rms t When the value is t Is it less than the previously recorded minimum root mean square error value? If so, find a new optimal step, clear the previously recorded action list, update the minimum minimum root mean square error value, and record the action sequence at this time.

[0027] Furthermore, in step S3, invalid actions are removed as follows:

[0028] For each action in the recorded action list, check whether the value of the next state exceeds the preset range. If the execution of an action causes the state to be out of range, it is removed from the action list.

[0029] Preferably, in step S3, the even-numbered and odd-numbered action pairs are processed by traversing the recorded action list and performing the following judgment when processing each action:

[0030] If the current action is an even number, check the subsequent actions in the action list to find out whether there is an odd-numbered action corresponding to the current action; if a corresponding odd-numbered action is found, remove it and remove the current action and the corresponding odd-numbered action from the action list at the same time; if not found, move to the next action and continue checking;

[0031] If the current action is an odd number, check the subsequent actions in the action list to find out whether there is an even-numbered action corresponding to the current action, and perform the removal operation.

[0032] Preferably, in step S3, the sorting is performed according to the difference in root mean square error values, the difference between the current root mean square error value and the next root mean square error value is calculated, and the action list is sorted in descending order according to the calculated RMS difference, giving priority to actions that have a significant improvement in the root mean square error value; only the top P actions with the best performance are retained, where P is a hyperparameter, to form a new action list.

[0033] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0034] 1. The present invention improves the BSIM parameter extraction method, uses IV and CV curves to extract multiple compact model parameters, and accelerates model training and solves the problem of difficult model convergence by using Dueling DQN and priority experience pool algorithm.

[0035] 2. The present invention improves the BSIM parameter extraction method, records the optimal parameter extraction steps each time it explores, and continues to explore along the previous optimal parameter extraction steps when the state is reset next time, which makes it easier to find the optimal solution. It can realize the automated process of BSIM parameter extraction, efficiently fit the target curve, and solve the problem of low efficiency of traditional parameter extraction relying on manpower. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flow chart of the improved BSIM parameter extraction method based on Dueling DQN of the present invention;

[0037] Figure 2 Schematic diagram of the relationship between the Cgg-Vgs target curve and the reference curve before fitting in an embodiment of the present invention;

[0038] Figure 3 Schematic diagram of the relationship between Id-Vgs and its first-order derivative, logarithmic target curve and reference curve before fitting in an embodiment of the present invention;

[0039] Figure 4 This is a schematic diagram of the fitting result of the relationship between the Cgg-Vgs target curve and the reference curve of an embodiment of the present invention;

[0040] Figure 5 Schematic diagram of the fitting results of the relationship between Id-Vgs and its first-order derivative, the logarithmic target curve and the reference curve in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the application is further elaborated in detail below in conjunction with the accompanying drawings. The described embodiments are only a part of the embodiments involved in the present invention. All non-innovative embodiments of other researchers in the field on this embodiment belong to the protection scope of the present invention. At the same time, for the step numbering in the embodiment of the present invention, it is only set for the convenience of explanation, and the order between the steps is not limited in any way. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0042] The present invention relates to an improved BSIM parameter extraction method based on Dueling DQN, which uses IV and CV curves to extract multiple compact model parameters, including the following steps:

[0043] S1. Initialize the environment: Set up the BSIM parameter extraction environment, model the BSIM parameter extraction problem as a Markov decision process, and build a DQN model;

[0044] S2, model training: Create Dueling DQN network model and perform model training, calculate target Q value and loss, and update Dueling DQN network parameters;

[0045] S3. Record and update the optimal steps: record the model training process, calculate the root mean square error value in each time step, remove invalid actions and process even and odd action pairs, sort according to the root mean square error value difference, and obtain the optimal parameters;

[0046] S4, model convergence: Repeat steps S2 to S3, stop training when the BSIM parameter error is less than the preset requirement, obtain the converged BSIM parameter value, and when resetting the state, continue iterating and converging along the last optimal parameter to update the optimal parameter solution.

[0047] In one embodiment of the present invention, the data sources used for BSIM parameter extraction mainly include two parts, including simulation data obtained by simulation software such as TCAD and WAT measurement data. For devices at different process nodes, IV and CV characteristic curves of different sizes and temperatures are used as target curves (target curves), which are used to extract parameters related to channel length, channel width, capacitance-voltage characteristics, current-voltage characteristics, and temperature, respectively.

[0048] Furthermore, this embodiment models the BSIM parameter extraction problem as a Markov decision process (MDP). Figure 1 As shown in Figure 1, a reinforcement learning method using the Dueling DQN architecture is described to optimize decision making in a simulated environment by creating and training a model. Priority experience replay is used in training to efficiently utilize experience data, and the ε-greedy strategy is combined for exploration and utilization. In this process, the most effective strategy for performance improvement is selected by recording and optimizing the optimal parameter adjustment action sequence until the model parameters converge, thereby achieving optimal adjustment of the environment parameters.

[0049] Four key components are defined in the reinforcement learning model: agent, state, action, and reward.

[0050] Specifically, the agent perceives the current state and selects the corresponding action to maximize the reward obtained. To achieve this goal, a deep Q network (DQN) is used as a value-based learning method. By training the neural network, DQN enables the agent to learn and optimize its strategy between different state-action pairs, thereby improving the efficiency and accuracy of parameter extraction.

[0051] The BSIM (Berkeley Short-channel IGFET Model) model is an electrical model widely used in semiconductor device modeling, especially in the design and analysis of modern CMOS (complementary metal oxide semiconductor) technology. It is mainly used to describe the electrical characteristics of short-channel MOSFET (metal oxide semiconductor field effect transistor) and can effectively predict the performance of the device under different operating conditions.

[0052] For various characteristic curves of the device, the parameters that can be extracted from each group of curves are also different. For each group of curves, a model can be trained separately for parameter extraction. Each model is individually designed to use the specific characteristics of the parameters assigned to it, iteratively predict the parameter value, and continue to make corrections until the error is reduced to an acceptable value.

[0053] Reinforcement learning is a machine learning paradigm where an agent learns by interacting with the environment, performing actions, and receiving rewards. This is different from supervised learning and unsupervised learning, which usually rely on data sets for training. Reinforcement learning can continuously learn in the process of interacting with the environment, and is more effective when faced with large amounts of data and complex combinatorial problems.

[0054] This example was implemented and evaluated using a modified Dueling DQN algorithm, with initial trials involving systematic variations of a single parameter while keeping all other parameters at their predefined values ​​to minimize the fitting error.

[0055] For the definition of action space, that is, setting each extracted parameter to have two actions of increase and decrease, the step size of parameter change is determined in advance before training and must be a reasonable number. If it is too large, it means the increase of search space, and if it is too small, it will lead to loss of accuracy.

[0056] The state represents a vector of information such as the current parameter value and voltage characteristics. The reward is set according to the change in the fitting error relative to the error when the parameters were not extracted. If the current action leads to a decrease in the error, a positive reward is given; otherwise, a negative reward is given.

[0057] The interaction between the agent and the environment is realized through Hspice simulation. Each action will modify the parameter value corresponding to the model card, and the reference curve data will be obtained by Hspice.

[0058] This implementation is based on the improved BSIM parameter extraction method of Dueling DQN. The specific process is as follows:

[0059] 1. Initialize the environment:

[0060] Set up the environment for BSIM parameter extraction.

[0061] Build a DQN model, using the Dueling DQN architecture.

[0062] Construct a priority experience replay pool to store state-action-reward-next-state tuples, implemented in the form of a binary tree. Each leaf node stores the priority of the sample, the parent node is the sum of the priorities of adjacent leaf nodes, and the root node is the sum of all sample leaf nodes.

[0063] 2. Training process:

[0064] 2.1. Create a Dueling DQN network and define the structure of state input and output Q value. Dueling DQN improves the estimation of Q value by calculating state value and advantage function respectively.

[0065] 2.2. Initialize the priority experience replay pool to store state s t 、Action a t , reward r and next state s t+1 A tuple of .

[0066] 2.3. Reset the environment to the initial state, recorded as the initial state s0, and determine whether there is an optimal parameter adjustment step. If there is an optimal parameter adjustment step, use the hyperparameter 0.1 and take a random number in the range of 0-1. If the random number is less than 0.1, it means that the optimal parameter adjustment steps are executed in sequence. The parameter adjustment step consists of a series of actions a.

[0067] 2.4. In each interaction between the agent and the environment, the agent selects an action from the current state (according to the ε-greedy strategy). In this embodiment, ε is set to 0.6, which decreases with the number of executions, and the step size of each decrease is:

[0068] ε=(ε inital -ε finall ) / n

[0069] Wherein, n is the number of times required to drop to a predetermined value, which is set to 500.

[0070] 2.5. Execute the action and obtain the new state and reward. t , action a t , reward r, next state s t+1 ) is stored in the experience replay pool, and small batches of samples are randomly extracted from the experience replay pool for training the network.

[0071] In this embodiment, the number of samples batch_size is set to 64.

[0072] 2.6. Calculate the target Q value and loss, and update the network parameters of Dueling DQN.

[0073] 3. Record and update the best steps:

[0074] 3.1. Record the training process and determine whether the RMS decreases

[0075] At each time step of training, four important information are recorded: the current state s t , the action performed is a t , the reward r t , the next state s after the current state t+1 Current root mean square error value rms t ;

[0076] Whenever a new RMS value is recorded, check if it is less than the previously recorded minimum RMS value. If so, it means a new optimal step has been found: in this case, clear the previously recorded action list and update the minimum RMS value. At the same time, record the action sequence at this time for subsequent analysis and use;

[0077] 3.2. Remove invalid actions:

[0078] For each action in the recorded action list, check whether the action causes the value of the next state to be out of the preset range. If the execution of an action causes the state to be out of range, remove it from the action list.

[0079] 3.3. Processing even and odd action pairs:

[0080] Traverse the recorded action list and make the following judgment when processing each action: If the current action is an even number, check the subsequent actions in the list: check whether there is an odd-numbered action corresponding to the current action. If such a subsequent action is found, remove the current action and the subsequent action from the list at the same time. If not found, move to the next action and perform the same check. If the current action is an odd number, use a similar approach: check whether there is an even-numbered action corresponding to the current action and process the relevant removal operations.

[0081] 3.4) Sort by RMS difference:

[0082] Calculate the current RMS value RMS t with the next RMS value RMS t+1 The difference between RMS diff ,Right now:

[0083] RMS diff =RMS t+1 -RMS t

[0084] The action list is sorted in descending order based on the calculated RMS difference, which ensures that those actions that have a greater improvement in the RMS value are prioritized. Finally, only the top 10 best performing actions are retained for use in subsequent training.

[0085] 4. Repeat the above steps until convergence, that is, stop training when the parameter error reaches an acceptable value.

[0086] In this embodiment, the initial values ​​of the parameters and the state settings are shown in the following table:

[0087]

[0088]

[0089] Furthermore, the relationship between the target curve and the reference curve before fitting in this embodiment is as follows: Figure 2 and Figure 3 As shown, it is taken from a large-size device, 28nm process; Figure 2 The curve is the Cgg-Vgs curve before fitting, the dotted line is the measured data curve, and the solid line is the simulation result; Figure 3 The curve is a schematic diagram of the relationship between Id-Vgs and its first-order derivative and logarithmic curve before fitting.

[0090] The relationship between the fitted target curve and the reference curve is as follows: Figure 4 and Figure 5 As shown, Figure 4 The curve is the Cgg-Vgs curve of the fitting result. Figure 5 The curve is a schematic diagram of the relationship between Id-Vgs and its first-order derivative and logarithmic curve of the fitting result.

[0091] In summary, the present invention aims to propose a stable and effective BSIM parameter extraction method based on reinforcement learning. The optimal parameter extraction steps are recorded each time the exploration is carried out. When the state is reset next time, it is easier to find the optimal solution by continuing to explore along the previous optimal parameter extraction steps. In addition, by using Dueling DQN and priority experience pool algorithm, model training is accelerated and the problem of difficult model convergence is solved.

[0092] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An improved BSIM parameter extraction method based on Dueling DQN, characterized in that: The following steps are involved: S1. Initialize the environment: Set up the BSIM parameter extraction environment, model the BSIM parameter extraction problem as a Markov decision process, and build a DQN model; S2, model training: Create Dueling DQN network model and perform model training, calculate target Q value and loss, and update Dueling DQN network parameters; S3. Record and update the optimal steps: record the model training process, calculate the root mean square error value in each time step, remove invalid actions and process even and odd action pairs, sort according to the root mean square error value difference, and obtain the optimal parameters; S4, model convergence: Repeat steps S2 to S3, stop training when the BSIM parameter error is less than the preset requirement, obtain the converged BSIM parameter value, and when resetting the state, continue iterating and converging along the last optimal parameter to update the optimal parameter solution.

2. The improved BSIM parameter extraction method based on Dueling DQN according to claim 1 is characterized in that: In step S1, the BSIM parameter extraction uses TCAD simulation data and WAT measurement data. For devices at different process nodes, IV and CV characteristic curves of different sizes and temperatures are used as target curves to extract channel length, channel width, capacitance-voltage characteristics, current-voltage characteristics, and temperature-related parameters.

3. The improved BSIM parameter extraction method based on Dueling DQN according to claim 2 is characterized in that: In step S1, the BSIM parameter extraction problem is modeled as a Markov decision process, and an agent, state, action, and reward are defined. The agent selects a corresponding action by sensing the current state to maximize the reward. The DQN model uses a deep Q network as a value-based learning method, is built on the Dueling DQN architecture, stores state-action-reward-next-state tuples through a prioritized experience replay pool, and learns and optimizes strategies between different state-action pairs; the interaction between the agent and the environment is realized through Hspice simulation.

4. The improved BSIM parameter extraction method based on Dueling DQN according to claim 3 is characterized in that: The priority experience replay pool is implemented in the form of a binary tree; each leaf node stores the priority of the sample, the parent node is the sum of the priorities of the adjacent leaf nodes, and the root node is the sum of all sample leaf nodes.

5. The improved BSIM parameter extraction method based on Dueling DQN according to claim 3 is characterized in that: Step S2: Model training, including the following sub-steps: S2.

1. Create a Dueling DQN network model and define the structure of state input and output Q value. The Dueling DQN network model improves the estimation of Q value by calculating state value and advantage function respectively. S2.

2. Initialize the priority experience replay pool and store the state s t 、Action a t , reward r and next state s t+1 tuple of ; S2.

3. Reset the environment to the initial state, recorded as the initial state s0, and determine whether there is an optimal parameter adjustment step. If so, randomly select to execute the optimal parameter adjustment step or not to execute it. The parameter adjustment step consists of a series of actions a. S2.

4. In each interaction between the agent and the environment, the agent selects an action from the current state; S2.

5. Execute the action and obtain the new state and reward, store the obtained tuple in the experience replay pool, randomly extract small batch samples from the experience replay pool, and train the Dueling DQN network; S2.

6. Calculate the target Q value and loss, and update the network parameters of Dueling DQN.

6. The improved BSIM parameter extraction method based on Dueling DQN according to claim 5 is characterized in that: In step S2.4, the agent selects an action from the current state according to the greedy strategy. The probability of random exploration decreases with the number of executions, and the step length of each decrease is: e=(e inital -e finall ) / n Where n is the number of times it takes to drop to a predetermined value, ε inital , ε finall The random exploration probability before and after the decrease respectively.

7. The improved BSIM parameter extraction method based on Dueling DQN according to claim 5 is characterized in that: In step S3, the training process is recorded as follows: At each time step of Dueling DQN network model training, the recorded information includes: the current state s t , Action performed a t , the reward obtained t , the next state after the current state s t+1 The current root mean square error value rms t , get the action sequence of the action list; Determine whether the RMS error value decreases. Whenever a new RMST value is recorded, check whether RMST is less than the previously recorded minimum RMS error value. If so, find a new optimal step, clear the previously recorded action list, update the minimum RMS error value, and record the action sequence at this time.

8. The improved BSIM parameter extraction method based on Dueling DQN according to claim 5, characterized in that: The method for removing invalid actions in step S3 is as follows: For each action in the recorded action list, check whether the value of the next state exceeds the preset range. If the execution of an action causes the state to be out of range, it is removed from the action list.

9. The improved BSIM parameter extraction method based on Dueling DQN according to claim 5, characterized in that: In step S3, even-numbered and odd-numbered action pairs are processed by traversing the recorded action list and performing the following judgment when processing each action: If the current action is an even number, check the subsequent actions in the action list to find out whether there is an odd-numbered action corresponding to the current action; if a corresponding odd-numbered action is found, remove it and remove the current action and the corresponding odd-numbered action from the action list at the same time; if not found, move to the next action and continue checking; If the current action is an odd number, check the subsequent actions in the action list to find out whether there is an even-numbered action corresponding to the current action, and perform the removal operation.

10. The improved BSIM parameter extraction method based on Dueling DQN according to claim 5, characterized in that: In step S3, the action list is sorted according to the difference in the root mean square error value, the difference between the current root mean square error value and the next root mean square error value is calculated, and the action list is sorted in descending order according to the calculated RMS difference, and the action that has a greater improvement in the root mean square error value is given priority; Only the top P best performing actions are retained, where P is a hyperparameter, to form a new action list.

Images