Process parameter optimization method, system, device and computer readable storage medium

Abstract

The application discloses a process parameter optimization method, system, device and computer readable storage medium, and relates to the technical field of industrial design, and specifically comprises the following steps: determining an electrical index prediction value and a prediction uncertainty thereof according to a to-be-optimized process parameter; determining a reinforcement learning state vector based on the electrical index prediction value, an electrical index specification value, the to-be-optimized process parameter, a process constraint state and a current remaining re-inspection number, and inputting the reinforcement learning state vector into a preset policy network to generate an initial action vector; determining a to-be-simulated process parameter according to the initial action vector and the to-be-optimized process parameter, and calling a TCAD simulator or a preset agent model to execute simulation based on the to-be-simulated process parameter to obtain a simulation result; determining a combined reward value according to the simulation result and the prediction uncertainty; and controlling the policy network to output an optimal process parameter based on the reinforcement learning state vector, the initial action vector and the combined reward value. The application can stably output process parameters meeting specifications under a limited simulation budget.

Description

Technical Field

[0001] This application relates to the field of industrial design technology, specifically to a method, system, device, and computer-readable storage medium for optimizing process parameters. Background Technology

[0002] As semiconductor process nodes continue to evolve and device structures become increasingly complex, the optimal combination of process parameters that meets device performance indicators needs to be determined through simulation or experimentation during semiconductor device development and process platform modeling. Therefore, an efficient process parameter optimization method is urgently needed.

[0003] In related technologies, process parameter optimization typically relies on trial-and-error parameter tuning based on engineer experience, or employs parameter scanning or numerical optimization-based search methods, using process parameters as optimization variables and performance index errors as the objective function. However, existing process parameter optimization methods have shortcomings when facing high-dimensional parameter coupling, complex index constraints, and high simulation costs. They are prone to optimizing parameters that violate process limitations or produce invalid simulations due to a lack of effective control over process boundaries and simulation costs, resulting in wasted resources and making it difficult to stably output process parameters that meet specifications under limited resources.

[0004] Therefore, how to provide a process parameter optimization method to stably output process parameters that meet specifications under a limited simulation budget is an urgent problem to be solved. Summary of the Invention

[0005] This application provides a method, system, device, and computer-readable storage medium for optimizing process parameters, which can stably output process parameters that meet specifications under limited simulation budget.

[0006] In a first aspect, embodiments of this application provide a method for optimizing process parameters, including: Determine the predicted values ​​of electrical indicators and their prediction uncertainties based on the process parameters to be optimized; The reinforcement learning state vector is determined based on the predicted values ​​of electrical indicators, the specified values ​​of electrical indicators, the process parameters to be optimized, the process constraint status, and the current remaining number of iterations. The reinforcement learning state vector is then input into the preset policy network to generate the initial action vector. The process parameters to be simulated are determined based on the initial motion vector and the process parameters to be optimized, and the simulation results are obtained by calling the TCAD simulator or a preset proxy model based on the process parameters to be simulated. The combined reward value is determined based on the simulation results and prediction uncertainty; The policy network is controlled to output optimal process parameters based on the reinforcement learning state vector, initial action vector, and combined reward value.

[0007] In conjunction with the first aspect, in one implementation, determining the reinforcement learning state vector based on the predicted electrical index value, the specified electrical index value, the process parameters to be optimized, the process constraint state, and the current remaining number of iterations includes: The specification normalized deviation vector is determined based on the predicted values ​​and the specified values ​​of electrical indicators; The reinforcement learning state vector is determined based on the normalized vector of the process parameters to be optimized, the specification normalized deviation vector, the process constraint state, and the current remaining number of iterations.

[0008] In conjunction with the first aspect, in one implementation, determining the process parameters to be simulated based on the initial action vector and the process parameters to be optimized includes: The initial motion vector is mapped to the process parameter increment according to the preset motion mapping strategy, and the initial process parameters are obtained based on the process parameter increment and the process parameters to be optimized. The initial process parameters are constrained and made compliant based on the preset process constraints to obtain compliant process parameters. The process parameters to be simulated are obtained by pre-screening compliant process parameters.

[0009] In conjunction with the first aspect, in one implementation, the simulation results include actual electrical index values, simulation convergence status indicators, simulation cost, constraint violation degree, and oscillation metric. The determination of the combined reward value based on the simulation results and prediction uncertainty includes: The current total specification deviation is determined based on the actual electrical index values ​​and the specified electrical index values; The severity of failure is determined based on the simulation convergence status indicators; The cost of retesting is determined based on the cost of simulation. The combined reward value is determined based on the current total specification deviation, the total specification deviation at the previous time step, the prediction uncertainty, the degree of constraint violation, the severity of failure, the backtesting cost, and the oscillation metric.

[0010] In conjunction with the first aspect, in one implementation, determining the combined reward value based on the current total specification deviation, the previous total specification deviation, the prediction uncertainty, the degree of constraint violation, the severity of failure, the backtesting cost, and the oscillation metric includes: The combined reward value is obtained by substituting the current total specification deviation, the previous total specification deviation, the prediction uncertainty, the degree of constraint violation, the severity of failure, the backtesting cost, the target weight coefficient, and the oscillation metric into the following calculation formula:

[0011] In the formula, This represents the current total specification deviation. This represents the total specification deviation at the previous moment. The first preset threshold for compliance; The current time represents the uncertainty of the forecast. The degree of constraint violation at the current moment; The severity of the failure at the current moment; The cost of backtesting at the current moment; , , , , , , The target weight coefficient; This measures the oscillation at the current moment. This represents the combined reward value at the current moment.

[0012] In conjunction with the first aspect, in one implementation method, the method for determining the target weight coefficient is as follows: If the current total specification deviation is greater than the second preset target value, the preset coarse search stage weight coefficient will be used as the target weight coefficient. If the current total specification deviation is greater than the first preset compliance threshold but not greater than the second preset compliance threshold, then the preset weight coefficient for the near-specification stage will be used as the target weight coefficient. If the current total specification deviation is not greater than the first preset compliance threshold and the prediction uncertainty is greater than the preset uncertainty threshold, then the preset refinement stage weight coefficient will be used as the target weight coefficient. If the current total specification deviation is not greater than the first preset compliance threshold and the prediction uncertainty is not greater than the preset uncertainty threshold, then the preset convergence stage weight coefficient is used as the target weight coefficient, and the first preset compliance threshold is less than the second preset compliance threshold.

[0013] In conjunction with the first aspect, in one embodiment, the method further includes: If the preset termination condition is not met, the process parameters to be simulated will be updated to the process parameters to be optimized in the next iteration, and the remaining number of iterations will be updated according to the simulation cost. Continue with the steps described above: determining the predicted values ​​of electrical parameters and their prediction uncertainties based on the process parameters to be optimized.

[0014] Secondly, embodiments of this application provide a process parameter optimization system, the process parameter optimization system comprising: The first processing module is used to determine the predicted values ​​of electrical indicators and their prediction uncertainties based on the process parameters to be optimized. The second processing module is used to determine the reinforcement learning state vector based on the predicted value of the electrical index, the specification value of the electrical index, the process parameters to be optimized, the process constraint status, and the current remaining number of iterations, and input the reinforcement learning state vector into the preset policy network to generate the initial action vector. The third processing module is used to determine the process parameters to be simulated based on the initial action vector and the process parameters to be optimized, and to call the TCAD simulator or a preset proxy model to perform the simulation based on the process parameters to be simulated to obtain the simulation results. The fourth processing module is used to determine the combined reward value based on simulation results, prediction uncertainty, constraint violation degree, and oscillation metric. The fifth processing module is used to update the policy network based on the reinforcement learning state vector, the initial action vector, and the combined reward value until a preset termination condition is met, and then output the optimal process parameters.

[0015] Thirdly, embodiments of this application provide a process parameter optimization device, which includes a processor, a memory, and a process parameter optimization program stored in the memory and executable by the processor. When the process parameter optimization program is executed by the processor, it implements the steps of the process parameter optimization method as described in any of the preceding claims.

[0016] Fourthly, embodiments of this application provide a computer-readable storage medium storing a process parameter optimization program, wherein when the process parameter optimization program is executed by a processor, it implements the steps of the process parameter optimization method as described in any of the preceding claims.

[0017] The beneficial effects of the technical solutions provided in this application include: Based on the process parameters to be optimized, the predicted values ​​and uncertainties of electrical indicators are determined. A low-cost surrogate model is used for pre-evaluation to reduce simulation costs. Then, based on the predicted values ​​of electrical indicators, the specifications of electrical indicators, the process parameters to be optimized, the process constraint state, and the current remaining number of iterations, a reinforcement learning state vector is determined. The budget and constraints are incorporated into the state space to achieve perception and control of resource constraints and process boundaries. Subsequently, the reinforcement learning state vector is input into a preset policy network to generate an initial action vector. Based on the initial action vector and the process parameters to be optimized, the process parameters to be simulated are determined, replacing trial and error to improve search efficiency. Then, based on the process parameters to be simulated, the TCAD simulator or the preset surrogate model is called to execute the simulation to obtain the simulation results. Based on the simulation results and the prediction uncertainty, a combined reward value is determined. A multi-factor reward function is used to penalize simulation failures and budget consumption, guiding the policy network to avoid invalid simulation regions. On this basis, the control policy network outputs the optimal process parameters based on the reinforcement learning state vector, the initial action vector, and the combined reward value, improving the ability to search for the optimal solution under limited resources. Through closed-loop iteration and reward guidance, stable output of process parameters that meet specifications is achieved under limited simulation budget. Attached Figure Description

[0018] Figure 1This is a schematic flowchart of an embodiment of the process parameter optimization method of this application; Figure 2 This is a schematic diagram of the constraint compliance processing in the embodiments of the process parameter optimization method of this application; Figure 3 This is a detailed flowchart illustrating an embodiment of the process parameter optimization method of this application; Figure 4 This is a schematic diagram of the reinforcement learning state-action-reward collaborative framework optimized for TCAD in this application; Figure 5 This is a schematic diagram of the hardware structure of the process parameter optimization equipment involved in the embodiments of this application. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0021] In a first aspect, embodiments of this application provide a method for optimizing process parameters.

[0022] In one embodiment, reference is made to Figure 1 , Figure 1 This is a schematic flowchart illustrating an embodiment of the process parameter optimization method of this application. Figure 1 As shown, the process parameter optimization methods include: Step S10: Determine the predicted values ​​of electrical parameters and their prediction uncertainties based on the process parameters to be optimized.

[0023] As an example, this application embodiment pre-defines a surrogate model in its specific implementation. This surrogate model can be a machine learning model such as a neural network, Gaussian process regression, or Krigin model, and its training data comes from historical TCAD simulation data or experimental data. The current process parameters to be optimized (e.g., a vector containing multiple dimensions such as doping concentration, oxide layer thickness, and annealing temperature) are input into the surrogate model. After forward computation, the surrogate model outputs predicted values ​​of electrical indicators and their prediction uncertainties. The predicted values ​​of electrical indicators refer to the device performance indicators predicted by the surrogate model, such as threshold voltage, drive current, and off-state current. The prediction uncertainty refers to the reliability assessment of the above predicted values ​​by the surrogate model, such as prediction variance, confidence interval width, or the degree of divergence between ensemble learning models. Through this step, a low-cost assessment and risk estimate of the performance of the process parameters to be optimized can be quickly obtained without calling expensive TCAD simulations, providing a data foundation for subsequent state construction.

[0024] Step S20: Determine the reinforcement learning state vector based on the predicted value of the electrical index, the specified value of the electrical index, the process parameters to be optimized, the process constraint status, and the current remaining number of iterations, and input the reinforcement learning state vector into the preset policy network to generate the initial action vector.

[0025] In this exemplary embodiment, the input information is first processed to construct a reinforcement learning state vector; the predicted values ​​of electrical indicators are compared with the specified values ​​of electrical indicators to obtain performance deviation information; the process parameters to be optimized are numerically processed; the current process constraint state (e.g., whether the previous simulation converged, whether the parameters exceeded the limits, etc.) and the current remaining number of iterations (i.e., the remaining simulation budget) are read; the deviation information, parameter information, constraint state, and budget information are concatenated or encoded to form the reinforcement learning state vector at the current moment; then, this state vector is input to a preset policy network, which can be a deep neural network (DNN), a convolutional neural network (CNN), or a recurrent neural network (RNN), etc. Based on the input state, the policy network calculates an initial action vector through its internal weights, which represents the parameter adjustment direction or adjustment amount suggested by the policy network, used to guide the subsequent update of process parameters.

[0026] Step S30: Determine the process parameters to be simulated based on the initial motion vector and the process parameters to be optimized, and call the TCAD simulator or a preset proxy model to perform the simulation based on the process parameters to be simulated to obtain the simulation results.

[0027] In this exemplary embodiment, the initial action vector is first applied to the current process parameters to be optimized to obtain candidate process parameters. During this process, the candidate parameters can be checked for compliance to ensure that they conform to physical meaning and process rules. The processed process parameters are then determined as the process parameters to be simulated. Next, an input file recognizable by the TCAD simulator is generated based on the process parameters to be simulated, and the TCAD simulator is called to perform physical simulation. Alternatively, the system inputs the process parameters to be simulated into a preset proxy model, which outputs the prediction results. After the simulation is completed, the system collects the simulation output file or model prediction data, parses it to obtain the simulation results, which include at least the actual performance index values ​​(actual electrical indices), the simulation convergence status (whether there is an error or divergence), and the resource cost consumed by this simulation (such as time or number of attempts).

[0028] Step S40: Determine the combined reward value based on the simulation results and prediction uncertainty.

[0029] As an example, in this embodiment, the performance compliance status or deviation of the current process parameters is calculated based on the actual performance index values ​​and electrical index specifications in the simulation results, and the risk of the current exploration area is assessed in conjunction with the prediction uncertainty. In addition, factors such as constraint violations during the simulation process (e.g., whether the parameters are close to the boundary), simulation failure, budget consumption, and the degree of oscillation in the optimization process can also be comprehensively considered. The above indicators are weighted and combined to calculate a scalar or vector-based combined reward value. This reward value is used to quantify the merits of the current action. A positive value indicates that it is beneficial to the optimization objective, and a negative value indicates that it is detrimental to the optimization objective, thereby providing feedback signals for updating the policy network.

[0030] Step S50: Control the policy network to output optimal process parameters based on the reinforcement learning state vector, initial action vector and combined reward value.

[0031] In this exemplary embodiment, reinforcement learning state vectors, initial action vectors, and combined reward values ​​are used to construct training samples. Reinforcement learning algorithms (such as policy gradient methods) are employed to update the internal weights of the policy network, enabling it to generate better actions in subsequent iterations. During iteration, the system maintains an optimal parameter record. After each simulation result, the current simulation result is compared with the historical best result. If the current result is better, the process parameters stored in the optimal parameter record are updated. The iteration terminates when a preset termination condition is met (such as exhausting the remaining iteration count, continuously meeting performance targets, or reaching the maximum number of iterations). At this point, the process parameters stored in the optimal parameter record are output as the final optimal process parameters. Through this closed-loop iteration and historical best-in-class tracking mechanism, it is ensured that the final output parameters are the best solution verified within a limited budget and meeting specifications.

[0032] This application determines the predicted values ​​and uncertainties of electrical indicators based on the process parameters to be optimized, and uses a low-cost surrogate model for pre-evaluation to reduce simulation costs. Then, based on the predicted electrical indicator values, the specified electrical indicator values, the process parameters to be optimized, the process constraint state, and the current remaining number of iterations, a reinforcement learning state vector is determined, incorporating the budget and constraints into the state space to achieve perception and control of resource limitations and process boundaries. Subsequently, the reinforcement learning state vector is input into a pre-set policy network to generate an initial action vector, and the process parameters to be simulated are determined based on the initial action vector and the process parameters to be optimized, replacing trial and error to improve search efficiency. Then, based on the process parameters to be simulated, a TCAD simulator or a pre-set surrogate model is called to execute the simulation to obtain simulation results. A combined reward value is determined based on the simulation results and the predicted uncertainty, and a multi-factor reward function is used to penalize simulation failures and budget consumption, guiding the policy network to avoid invalid simulation regions. Based on this, the control policy network outputs the optimal process parameters based on the reinforcement learning state vector, the initial action vector, and the combined reward value, improving the ability to search for the optimal solution under limited resources. Through closed-loop iteration and reward guidance, stable output of process parameters meeting specifications is achieved under limited simulation budget.

[0033] Further, in one embodiment, determining the reinforcement learning state vector based on the predicted electrical index value, the specified electrical index value, the process parameters to be optimized, the process constraint state, and the current remaining number of iterations includes: The specification normalized deviation vector is determined based on the predicted values ​​and the specified values ​​of electrical indicators; The reinforcement learning state vector is determined based on the normalized vector of the process parameters to be optimized, the specification normalized deviation vector, the process constraint state, and the current remaining number of iterations.

[0034] As an example, in the embodiments of this application, different formulas are first used to calculate the specification normalization deviation vector for different types of electrical indicators, specifically including the following four cases: (1) Target-type indicators (such as threshold voltage Vth): Calculate the absolute value of the difference between the predicted value and the specified value, divide it by the preset specification tolerance, that is, the target value. For target-type indicators (such as Vth), it is required to be close to the target value. ) ,in, Let m be the predicted value of the m-th indicator; Let m be the specification value of the m-th indicator; This is a preset specification tolerance; The specification normalization deviation of the m-th indicator; (2) Upper limit type indicators (such as off-state current Ioff): If the predicted value is less than the upper limit, the deviation is normalized to 0; if it is greater than the upper limit, the ratio of the excess to the specification tolerance is calculated, and the upper limit type indicators (such as...) are applied to the upper limit type indicators. ), ,in, This represents the upper limit of the m-th indicator; (3) Lower limit indicators (such as drive current Ion): If the predicted value is greater than the lower limit, the deviation is normalized to 0; if it is less than the lower limit, the ratio of the insufficient part to the specification tolerance is calculated, and the lower limit indicator (such as...) is applied. ) ,in, This represents the lower limit of the m-th indicator; (4) Interval-type indicators (such as parasitic capacitance): If the predicted value is within the target interval, the deviation is normalized to 0; if it is outside the interval, the normalized distance to the nearest boundary is calculated for interval-type indicators. :like ,but ;like ,but ;like ,but (This does not constitute a deviation and is not subject to penalty); The normalized deviations of all the above indicators are concatenated into a specification normalized deviation vector.

[0035] Next, a normalized vector of the process parameters to be optimized is constructed. The current process parameters (such as doping concentration, oxide layer thickness, annealing temperature, etc.) are divided by their respective physical upper limits to eliminate the influence of dimensions, resulting in a normalized parameter vector. Then, the process constraint state is determined, including the convergence flag of the previous TCAD simulation (e.g., convergence is 0, non-convergence or error is 1) and whether the current parameters violate design rules. Finally, the remaining number of backtests is obtained. For example, if the total budget is 100 TCAD simulations, and 20 have been completed, then the remaining number of backtests is 80. The normalized parameter vector, specification normalized deviation vector, process constraint state scalar, and remaining backtest number scalar are concatenated in a preset order to form the final reinforcement learning state vector St, which serves as the input to the policy network.

[0036] Further, in one embodiment, determining the process parameters to be simulated based on the initial action vector and the process parameters to be optimized includes: The initial motion vector is mapped to the process parameter increment according to the preset motion mapping strategy, and the initial process parameters are obtained based on the process parameter increment and the process parameters to be optimized. The initial process parameters are constrained and made compliant based on the preset process constraints to obtain compliant process parameters. The process parameters to be simulated are obtained by pre-screening compliant process parameters.

[0037] As an example, in the embodiments of this application, the preset action mapping strategy includes at least one of continuous incremental mapping or hierarchical selection mapping, used to transform the initial action vector output by the strategy network into specific process parameter update amounts (i.e., process parameter increments), as follows: (1) Continuous incremental action mode: The policy network outputs the initial action vector. It contains n normalized variables (corresponding to n process parameters to be optimized). The system will use the initial action vector. The process parameter increment is obtained by performing element-wise multiplication with the preset parameter scale vector S. The parameter scaling vector S contains the maximum allowable adjustment step size for each process parameter. Subsequently, the process parameter increment is compared with the current process parameter to be optimized. Add them together to obtain the initial process parameters. .

[0038] (2) Hierarchical action pattern: The policy network outputs the initial action vector. Its structure is a quadruple. .

[0039] First level (group selection): Based on Select the target parameter group (such as structure group, doping group, material group, or physical model group) from all process parameters; Second layer (parameter selection): According to Within the selected group, further refine the selection by 1 to 3 specific parameter subsets; Third layer (direction and amplitude): according to Determine the adjustment direction (positive or negative), based on Determine the adjustment range; Fourth layer (preliminary projection): Based on the above selection and amplitude, a preliminary parameter update is generated and superimposed on the current process parameters to be optimized to obtain the initial process parameters.

[0040] It should be understood that, since the initial process parameters may violate physical limits or process rules, the system needs to perform constraint compliance processing, which includes at least one of the following methods: (1) Boundary trimming: Check whether the initial process parameters exceed the physical allowable range (e.g., the oxide layer thickness cannot be negative, and the annealing temperature cannot exceed the equipment limit). If a parameter value exceeds the limit, it is forcibly trimmed to the nearest boundary value.

[0041] (2) Manufacturability Projection: Detects whether the combination of parameters violates the process rules (e.g., an imbalance in the ratio of injected energy to dosage will lead to manufacturing failure). If it violates the rules, the parameter combination is corrected to the nearest legal region using a projection algorithm. This algorithm is common knowledge in the field and will not be described in detail here for the sake of brevity.

[0042] (3) Dependency Correction: Detect whether there are preset dependencies between parameters (e.g., when modifying the fence length, the sidewall width needs to be adjusted in conjunction). If they exist, correct the relevant parameters in conjunction according to the preset function relationship to ensure the physical consistency of the parameter combination.

[0043] (4) Action scaling: Detect whether the current parameter is close to the process boundary or in a region with high uncertainty in the surrogate model prediction. If so, automatically reduce the adjustment step size (i.e., reduce the action amplitude) and regenerate candidate parameters to avoid violent oscillations or entering a high-risk region.

[0044] (5) Illegal action rejection: Detect whether the candidate parameters obviously violate physical constraints (e.g., negative thickness, exceeding the allowable range of the process platform). If it is determined to be illegal, the action is directly rejected, no TCAD task is generated, and a constraint penalty is imposed on the policy network to force the network to learn to avoid such actions.

[0045] The parameters obtained after at least one of the above processing steps are called compliant process parameters. To further save on the expensive TCAD simulation budget, the system calls a preset proxy model to quickly evaluate the compliant process parameters, obtain predicted values ​​of electrical indicators, and calculate the difference between the predicted values ​​and the specification values. If the difference is less than a preset screening threshold (e.g., 50% of the specification tolerance), the parameter is considered to have optimization potential and is sent to the TCAD simulation stage as a process parameter to be simulated. If the difference is too large, the parameter is discarded and TCAD simulation is not called, thereby saving the simulation budget.

[0046] It should be noted that, referring to Figure 2 As shown, the original action of the input policy a t = (Δ x 1,...,Δ x n Starting from this point, five different processing branches are initiated in parallel; among them, the boundary clipping branch limits the action value clipping to a specific range. Within the interval, the manufacturability projection branch projects actions to the nearest legal process area, the dependency correction branch adjusts related variables with parameter dependencies, and the action scaling branch automatically reduces the action step size when parameters approach the boundary. After processing by these four branches, they jointly output compliant process parameters, expressed by the formula... The corrected state update is completed, along with a complete record of the constraint and boundary fields; the fifth illegal action rejection branch will directly determine and punish the violation action, not generate the corresponding TCAD simulation task, apply constraint penalties by following the rejection path, and not trigger TCAD task generation throughout the process. The entire process avoids illegal actions from the source, ensuring that the output process parameters fully comply with manufacturing constraints and boundary requirements.

[0047] Further, in one embodiment, the simulation results include actual electrical index values, simulation convergence state indicators, simulation cost, constraint violation degree, and oscillation metric. The determination of the combined reward value based on the simulation results and prediction uncertainty includes: The current total specification deviation is determined based on the actual electrical index values ​​and the specified electrical index values; The severity of failure is determined based on the simulation convergence status indicators; The cost of retesting is determined based on the cost of simulation. The combined reward value is determined based on the current total specification deviation, the total specification deviation at the previous time step, the prediction uncertainty, the degree of constraint violation, the severity of failure, the backtesting cost, and the oscillation metric.

[0048] As an example, in the embodiments of this application, the simulation results include actual electrical index values, simulation convergence status indicators, simulation costs, constraint violation degree and oscillation metric, and the current total specification deviation. The severity of failure can be calculated based on the normalized distance between the actual electrical parameters returned by TCAD simulation and the specified electrical parameters; This can be determined based on the simulation convergence status indicator. If the TCAD simulation reports an error, fails to converge, or fails to generate a mesh, then... Set to 1, otherwise The cost of verification is 0; The cost can be determined based on the simulation's consumption; for example, the cost of each TCAD simulation call. Counted as 1 unit of budget consumption; constraint on the degree of violation Calculate the closest distance between compliant process parameters and the process boundary; the closer the distance, the better. The higher the value, the greater the prediction uncertainty. The prediction variance or confidence interval width output by the surrogate model can be directly used; oscillation measure It can be obtained by statistically analyzing the variance of the action vectors or reward values ​​of the most recent N (e.g., 5) iterations, and is used to punish the instability of the policy.

[0049] It should be understood that, through the above multi-factor combination, if the current total specification deviation meets the target, a target achievement reward is obtained; if the deviation decreases compared to the previous time step, a progress reward is obtained; at the same time, uncertainty penalties, constraint penalties, failure penalties, cost penalties, and oscillation penalties are deducted. Through this reward function, the policy network is guided to find a robust optimal solution within a limited budget.

[0050] Further, in one embodiment, determining the combined reward value based on the current total specification deviation, the previous total specification deviation, the prediction uncertainty, the degree of constraint violation, the severity of failure, the backtesting cost, and the oscillation metric includes: The combined reward value is obtained by substituting the current total specification deviation, the previous total specification deviation, the prediction uncertainty, the degree of constraint violation, the severity of failure, the backtesting cost, the target weight coefficient, and the oscillation metric into the following calculation formula:

[0051] In the formula, This represents the current total specification deviation. This represents the total specification deviation at the previous moment. The first preset threshold for compliance; The current time represents the uncertainty of the forecast. The degree of constraint violation at the current moment; The severity of the failure at the current moment; The cost of backtesting at the current moment; , , , , , , The target weight coefficient; This measures the oscillation at the current moment. This represents the combined reward value at the current moment.

[0052] As an example, in the embodiments of this application, the current total specification deviation is... Total specification deviation at the previous moment Prediction uncertainty Degree of violation of constraints Severity of failure , return inspection cost Target weight coefficient , , , , , , and oscillation measurement Substitute into the following formula to obtain the combined reward value. :

[0053] Furthermore, in one embodiment, the method for determining the target weight coefficient is as follows: If the current total specification deviation is greater than the second preset target value, the preset coarse search stage weight coefficient will be used as the target weight coefficient. If the current total specification deviation is greater than the first preset compliance threshold but not greater than the second preset compliance threshold, then the preset weight coefficient for the near-specification stage will be used as the target weight coefficient. If the current total specification deviation is not greater than the first preset compliance threshold and the prediction uncertainty is greater than the preset uncertainty threshold, then the preset refinement stage weight coefficient will be used as the target weight coefficient. If the current total specification deviation is not greater than the first preset compliance threshold and the prediction uncertainty is not greater than the preset uncertainty threshold, then the preset convergence stage weight coefficient is used as the target weight coefficient, and the first preset compliance threshold is less than the second preset compliance threshold.

[0054] As an example, in this embodiment, the specific value of the preset uncertainty threshold can be determined according to actual needs and is not limited here; the specific values ​​of the first preset compliance threshold and the second preset compliance threshold can be determined according to actual needs, as long as the first preset compliance threshold is less than the second preset compliance threshold, and are not limited here. For example, the first preset compliance threshold can be set as a specification tolerance. The second preset threshold is set to To ensure a smooth transition between stages, the determination of the target weight coefficients adopts a phased strategy, with specific thresholds and weight configurations as follows: (1) Coarse search stage: If the current total specification deviation is greater than the second preset target value (e.g., set to ), When the parameters are considered to be far from the target, the preset coarse search stage weight coefficients are used as the target weight coefficients. The configuration features are as follows: the weight α of the specification compliance item is set to 0 (because it is difficult to meet the target, to avoid the gradient vanishing due to sparse rewards), the weight β of the deviation reduction item is set to a higher value (such as 1.0, to encourage large step size to approach), the weight γ of the prediction uncertainty item is set to a lower value (such as 0.05), the weight λ of the constraint violation item is set to a lower value (such as 0.2), the weight μ of the failure severity item is set to a preset penalty value (such as 1.0, to ensure that simulation failure is avoided), the weight η of the backtesting cost is set to a lower value (such as 0.0, to allow some waste), and the weight ν of the oscillation metric is set to a lower value (such as 0.05).

[0055] (2) Approaching Specification Stage: If the current total specification deviation is greater than the first preset compliance threshold but not greater than the second preset compliance threshold (e.g. < ≤ When the parameters are considered to be close to the feasible region, the preset weight coefficient for the near-specification stage is used as the target weight coefficient. The configuration features are as follows: the weight α for specification compliance is set to a medium value (e.g., 0.5), the weight β for deviation reduction is set to a medium value (e.g., 0.5), the weight γ for prediction uncertainty is set to a medium value (e.g., 0.2), the weight λ for constraint violation is increased (e.g., 0.8, prioritizing the search for legal candidates), the weight μ for failure severity is set to a preset penalty value (e.g., 1.0), the weight η for backtesting cost is set to a low value (e.g., 0.05), and the weight ν for oscillation measurement is set to a medium value (e.g., 0.1).

[0056] (3) Refinement stage: If the current total specification deviation is not greater than the first preset compliance threshold and the prediction uncertainty is greater than the preset uncertainty threshold, it is considered that the parameters meet the standard but the model credibility is low. At this time, the preset refinement stage weight coefficient is used as the target weight coefficient. The configuration features are: the specification compliance item weight α is set to a high value (e.g., 1.0), the deviation reduction item weight β is set to a low value (e.g., 0.3), the prediction uncertainty item weight γ is increased (e.g., 0.5, to encourage the exploration of low uncertainty region), the constraint violation item weight λ is kept at a high value (e.g., 0.8), the failure severity item weight μ is set to the preset penalty value (e.g., 1.0), the backtesting cost weight η is increased (e.g., 0.2, to start saving budget), and the oscillation measurement weight ν is set to a medium value (e.g., 0.2).

[0057] (4) Convergence Phase: If the current total specification deviation is not greater than the first preset threshold and the prediction uncertainty is not greater than the preset uncertainty threshold, a robust solution is considered to have been found. At this time, the preset convergence phase weight coefficient is used as the target weight coefficient. The configuration features are: the termination reward weight is added on the basis of the previous phase reward to encourage the end of the search. The basic weight coefficients (α,β,γ,λ,μ,η,ν) are maintained in the refinement phase configuration or the exploration item weight is further reduced, while the termination reward weight is increased.

[0058] Furthermore, in one embodiment, the method further includes: If the preset termination condition is not met, the process parameters to be simulated will be updated to the process parameters to be optimized in the next iteration, and the remaining number of iterations will be updated according to the simulation cost. Continue with the steps described above: determining the predicted values ​​of electrical parameters and their prediction uncertainties based on the process parameters to be optimized.

[0059] As an example, in the embodiments of this application, the determination of the preset termination condition specifically includes the following situations: (1) The current remaining number of backtests has reached zero (budget exhausted); (2) The current total specification deviation is below the compliance threshold for K consecutive times (e.g., 5 times); (3) The combined reward value did not increase significantly for K consecutive times.

[0060] Specifically, if the termination condition is not met, the process proceeds to the update step; the validated process parameters to be simulated in this round (i.e., compliant and pre-screened parameters) are directly assigned to the process parameter variables to be optimized in the next iteration. This serves as the input baseline for the next round of the policy network, ensuring the continuity of the optimization process. Subsequently, the cost incurred in this round of simulation is subtracted from the current remaining number of iterations to obtain the updated remaining number of iterations. This value will be used as part of the state vector for the next round of reinforcement learning, reducing the policy network's awareness budget and thus automatically adjusting the exploration strategy later (such as switching from exploration to exploitation). Finally, the loop executes the process, carrying the updated process parameters to be optimized and the remaining number of iterations, and returns to execute the step of determining the predicted values ​​of electrical indicators and their prediction uncertainties based on the process parameters to be optimized, starting the next round of iteration. Through this closed-loop update mechanism, it is ensured that the optimization process continuously approaches the optimal solution before the budget is exhausted.

[0061] It should be noted that, referring to Figure 3 As shown, the current round state s is generated from the reinforcement learning state construction. t Start by proceeding sequentially until the strategy outputs the corresponding action 'a'. t Entering the constraint compliance process to obtain the next state. Then, pre-screening is performed, and simulation is conducted using either a real TCAD simulation or a preset proxy model. The process then proceeds to calculate the combined reward value to obtain the current round's reward r. t The process begins by recording and storing all interaction information (s, a, r, s') on the trajectory, followed by updating and iterating the execution policy network. Finally, it proceeds to the convergence judgment stage. If convergence is achieved, the final result is output directly; otherwise, the updated state s is updated. {t+1} As the initial state for the next round, jump back to start a new round of iterative optimization, and continue the loop until the convergence condition is met.

[0062] It should be understood that, with reference Figure 4 As shown, this reinforcement learning state-action-reward construction independent process for TCAD optimization can be connected to the optional external task definition process on the left to obtain task and specification inputs related to task definition and specification tolerance definition; at the same time, it can be connected to the optional external simulation feedback process to obtain feedback and budget inputs related to retrieval results, convergence and budget status. The main body sequentially completes the normalization deviation processing steps for each indicator and specification, and finally outputs trajectory samples and candidate retrieval results. It can also be connected to the optional external prediction model management process to complete the reception and subsequent management of trajectory samples.

[0063] It should be noted that the specific implementation process of the process parameter optimization method provided in this application will be described in detail below with reference to specific embodiments.

[0064] I. Constructing a Reinforced Learning State Taking MOS devices as an example, the parameters to be optimized include gate oxide thickness (tox), channel doping concentration (Nch), source / drain extension region doping (Next), and annealing parameters. Interface density of states (Dit), mobility model correction parameters It has 6 dimensions, which are formed after normalization. ∈[0,1] 6.

[0065] Proxy model for the current Output prediction With uncertainty Specification normalization deviation is calculated using the following formula: (1) Target-oriented indicators (such as Vth):

[0066] (2) Lower limit indicators (such as gmmax):

[0067] (3) Upper limit indicators (such as Ioff):

[0068] Standard mark .

[0069] Historical best deviation Take the smallest total deviation from the historical trajectory Deviation Improvement Amount Recheck budget status Take the remaining budget percentage; TCAD convergence flag Manufacturability mark Failure mode flags Return from the previous simulation; historical actions This is the previous continuous increment. Final state. It is composed of the following components:

[0070] II. Action Construction and Parameter Generation (1) Continuous incremental actions: Policy network output = (Δtox, ΔNch, ΔNext, ΔAnneal, ΔDit, Δmu), each component is activated by the tanh function and mapped to [-1, 1], then compared with the parameter scale vector. The element-wise multiplication yields the actual parameter increment. Preliminary parameter update formula: Boundary clipping formula: Layered Actions: Policy Network Output First floor ∈{structure group, dopant group, material group, physical model group}; second layer Select 1-3 parameters within the selected group; third level: direction ∈ {-1, +1}; fourth level... The continuous amplitude is given by the policy network.

[0071] (2) Constraint compliance processing: After the action is executed, if the candidate parameters If it goes out of bounds, it will be clipped to the specified area. Combinations that violate preset process window constraints (e.g., Nch>Next +) Correct the parameter combination according to the preset projection rules; if tox and If monotonicity constraints exist, adjustments are made in tandem according to the dependency relationship. The processed parameters are called compliant process parameters.

[0072] (3) Pre-screening of proxy model: Input the compliant process parameters into the preset proxy model to obtain the predicted values ​​of electrical indicators, and calculate the difference between the predicted values ​​and the specification values; if the difference is less than the preset screening threshold (e.g., 50% of the specification tolerance), the parameter is considered to have optimization potential and is sent to the TCAD simulation stage as the process parameter to be simulated; if the difference is too large, the parameter is discarded, TCAD simulation is not called, and the next iteration is directly entered or a negative reward is given.

[0073] III. Reward Function Calculation (1) Basic combination reward: The reward value calculation formula is as follows

[0074] (2) Multi-objective vectorized reward: It can be returned as a multi-objective vector, where the multi-head value function of the policy network learns the components of achievement, improvement, risk, and cost respectively, and then synthesizes the training objective in a weighted manner.

[0075] IV. Strategy Training and Execution Closed Loop Policy training: RL training is performed using a proxy model as the environment; when the uncertainty of the proxy model in a certain region is too high, a real TCAD backtest can be triggered as a state / reward correction signal; after every N rounds, the trajectory batch samples are used for policy network updates.

[0076] Strategy execution and optimal parameter tracking: (1) Generation and screening: The trained strategy generates candidate TCAD parameters based on the current state. After constraint compliance processing and pre-screening by proxy model, the process parameters to be simulated are obtained.

[0077] (2) Simulation and Recording: Based on the process parameters to be simulated, the TCAD simulator is called to perform the simulation and obtain the actual electrical indicators. The system compares the current actual electrical indicators with the historical best performance indicators. If the current indicators are better, the recorded best process parameters and corresponding best performance indicators are updated.

[0078] (3) State update: Calculate the combined reward value based on the simulation results and update the policy network. Update the process parameters to be simulated to the process parameters to be optimized in the next iteration, and update the current remaining number of iterations based on the simulation cost.

[0079] (4) Termination and Output: Determine whether the preset termination conditions are met (such as budget exhaustion or continuous achievement of targets). If met, output the recorded optimal process parameters, rather than the parameters of the last iteration. This step ensures that the final output is the best result verified throughout the optimization process.

[0080] (5) Iterative loop: If the termination condition is not met, return to the generation and filtering steps and continue to the next iteration.

[0081] External Interface: The candidates output by the strategy in this application can be used as input to the external simulation feedback process; the feedback results from the external simulation feedback process and the remaining budget update the state vector in this application. Equal components; the samples output by the trajectory recording module of this application can be used as input for active learning and incremental updates in the external prediction model management process.

[0082] V. TCAD Mapping and Preferred Implementation Methods (1) Parameter Mapping: The parameter changes generated by the action are mapped to the corresponding variables in the TCAD process simulation input file, device simulation input file, geometric parameters, doping distribution parameters, material model parameters, physical model parameters (mobility, recombination, interface states, quantum correction, etc.) or configuration file through a parameter mapping table. The parameter mapping table must contain at least the following fields: variable name, unit, upper and lower limits, scaling method, default value, step size, constraint conditions, parameter group, list of dependent parameters, and TCAD engine compatibility flag. Refer to the simplified example fragment shown in Table 1 (the specific values ​​are related to the device platform): Table 1. Mapping Relationship between Process Parameter Action Dimensions and TCAD Simulation Variables

[0083] (2) Preferred implementation method: (a) Threshold and weight adaptation: α / β / γ / λ / μ / η / ν and It can be adaptively adjusted during training based on specification satisfaction rate, retracement pass rate, oscillation metric, etc.

[0084] (b) Multi-fidelity environment: The surrogate model can provide high / low fidelity predictions at the same time; policy training can prioritize low fidelity, and key candidates can be switched to high fidelity or real TCAD.

[0085] (c) Abnormal process parameter detection: perform Out-of-Distribution (OOD) detection on candidates output by the strategy; reject candidate trigger constraint projections or illegal actions in the OOD region as a supplement to pre-screening.

[0086] (d) Cross-device migration: This method is applicable to silicon-based logic devices such as CMOS, FinFET and GAA, power devices such as SiCMOSFET, GaN HEMT and IGBT, BCD, analog devices, memory devices, etc.; the specific components of state / action / reward can be configured independently according to device category.

[0087] (5) Manual intervention interface: Process engineers can insert manually specified candidates, modify the weights of α / β / γ / λ, or fix certain process parameters in the closed loop. The system records such interventions and retains traceability.

[0088] Secondly, embodiments of this application also provide a process parameter optimization system, the process parameter optimization system comprising: The first processing module is used to determine the predicted values ​​of electrical indicators and their prediction uncertainties based on the process parameters to be optimized. The second processing module is used to determine the reinforcement learning state vector based on the predicted value of the electrical index, the specification value of the electrical index, the process parameters to be optimized, the process constraint status, and the current remaining number of iterations, and input the reinforcement learning state vector into the preset policy network to generate the initial action vector. The third processing module is used to determine the process parameters to be simulated based on the initial action vector and the process parameters to be optimized, and to call the TCAD simulator or a preset proxy model to perform the simulation based on the process parameters to be simulated to obtain the simulation results. The fourth processing module is used to determine the combined reward value based on simulation results, prediction uncertainty, constraint violation degree, and oscillation metric. The fifth processing module is used to update the policy network based on the reinforcement learning state vector, the initial action vector, and the combined reward value until a preset termination condition is met, and then output the optimal process parameters.

[0089] Furthermore, in one embodiment, the second processing module is specifically used for: The specification normalized deviation vector is determined based on the predicted values ​​and the specified values ​​of electrical indicators; The reinforcement learning state vector is determined based on the normalized vector of the process parameters to be optimized, the specification normalized deviation vector, the process constraint state, and the current remaining number of iterations.

[0090] Furthermore, in one embodiment, the third processing module is specifically used for: The initial motion vector is mapped to the process parameter increment according to the preset motion mapping strategy, and the initial process parameters are obtained based on the process parameter increment and the process parameters to be optimized. The initial process parameters are constrained and made compliant based on the preset process constraints to obtain compliant process parameters. The process parameters to be simulated are obtained by pre-screening compliant process parameters.

[0091] Furthermore, in one embodiment, the simulation results include actual electrical index values, simulation convergence status indicators, simulation cost, constraint violation degree, and oscillation metric. The fourth processing module is specifically used for: The current total specification deviation is determined based on the actual electrical index values ​​and the specified electrical index values; The severity of failure is determined based on the simulation convergence status indicators; The cost of retesting is determined based on the cost of simulation. The combined reward value is determined based on the current total specification deviation, the total specification deviation at the previous time step, the prediction uncertainty, the degree of constraint violation, the severity of failure, the backtesting cost, and the oscillation metric.

[0092] Furthermore, in one embodiment, the fourth processing module is specifically used for: The combined reward value is obtained by substituting the current total specification deviation, the previous total specification deviation, the prediction uncertainty, the degree of constraint violation, the severity of failure, the backtesting cost, the target weight coefficient, and the oscillation metric into the following calculation formula:

[0093] In the formula, This represents the current total specification deviation. This represents the total specification deviation at the previous moment. The first preset threshold for compliance; The current time represents the uncertainty of the forecast. The degree of constraint violation at the current moment; The severity of the failure at the current moment; The cost of backtesting at the current moment; , , , , , , The target weight coefficient; This measures the oscillation at the current moment. This represents the combined reward value at the current moment.

[0094] Furthermore, in one embodiment, the fourth processing module is specifically used for: If the current total specification deviation is greater than the second preset target value, the preset coarse search stage weight coefficient will be used as the target weight coefficient. If the current total specification deviation is greater than the first preset compliance threshold but not greater than the second preset compliance threshold, then the preset weight coefficient for the near-specification stage will be used as the target weight coefficient. If the current total specification deviation is not greater than the first preset compliance threshold and the prediction uncertainty is greater than the preset uncertainty threshold, then the preset refinement stage weight coefficient will be used as the target weight coefficient. If the current total specification deviation is not greater than the first preset compliance threshold and the prediction uncertainty is not greater than the preset uncertainty threshold, then the preset convergence stage weight coefficient is used as the target weight coefficient, and the first preset compliance threshold is less than the second preset compliance threshold.

[0095] Furthermore, in one embodiment, the fifth processing module is specifically used for: If the preset termination condition is not met, the process parameters to be simulated will be updated to the process parameters to be optimized in the next iteration, and the remaining number of iterations will be updated according to the simulation cost. Continue with the steps described above: determining the predicted values ​​of electrical parameters and their prediction uncertainties based on the process parameters to be optimized.

[0096] This application determines the predicted values ​​and uncertainties of electrical indicators based on the process parameters to be optimized, and uses a low-cost surrogate model for pre-evaluation to reduce simulation costs. Then, based on the predicted electrical indicator values, the specified electrical indicator values, the process parameters to be optimized, the process constraint state, and the current remaining number of iterations, a reinforcement learning state vector is determined, incorporating the budget and constraints into the state space to achieve perception and control of resource limitations and process boundaries. Subsequently, the reinforcement learning state vector is input into a pre-set policy network to generate an initial action vector, and the process parameters to be simulated are determined based on the initial action vector and the process parameters to be optimized, replacing trial and error to improve search efficiency. Then, based on the process parameters to be simulated, a TCAD simulator or a pre-set surrogate model is called to execute the simulation to obtain simulation results. A combined reward value is determined based on the simulation results and the predicted uncertainty, and a multi-factor reward function is used to penalize simulation failures and budget consumption, guiding the policy network to avoid invalid simulation regions. Based on this, the control policy network outputs the optimal process parameters based on the reinforcement learning state vector, the initial action vector, and the combined reward value, improving the ability to search for the optimal solution under limited resources. Through closed-loop iteration and reward guidance, stable output of process parameters meeting specifications is achieved under limited simulation budget.

[0097] The functions of each module in the above-mentioned process parameter optimization system correspond to the steps in the above-mentioned process parameter optimization method embodiment, and their functions and implementation processes will not be described in detail here.

[0098] Thirdly, embodiments of this application provide a process parameter optimization device, which can be a personal computer (PC), laptop computer, server, or other device with data processing capabilities.

[0099] Reference Figure 5 , Figure 5This is a schematic diagram of the hardware structure of the process parameter optimization device involved in the embodiments of this application. In the embodiments of this application, the process parameter optimization device may include a processor, a memory, a communication interface, and a communication bus.

[0100] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.

[0101] Communication interfaces include input / output (I / O) interfaces, physical interfaces, and logical interfaces used for interconnecting devices within the process parameter optimization equipment, as well as interfaces used for interconnecting the process parameter optimization equipment with other devices (such as other computing devices or user equipment). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user equipment can be displays, keyboards, etc.

[0102] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0103] The processor can be a general-purpose processor, which can call a process parameter optimization program stored in memory and execute the process parameter optimization method provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the process parameter optimization program is called can be referred to in the various embodiments of the process parameter optimization method of this application, and will not be repeated here.

[0104] Those skilled in the art will understand that Figure 5 The hardware structure shown does not constitute a limitation of this application and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0105] Fourthly, embodiments of this application also provide a readable storage medium.

[0106] The present application has a process parameter optimization program stored on a readable storage medium, wherein when the process parameter optimization program is executed by a processor, the steps of the process parameter optimization method described above are implemented.

[0107] The method implemented when the process parameter optimization program is executed can be referred to in various embodiments of the process parameter optimization method of this application, and will not be repeated here.

[0108] The terms "comprising" and "having," and any variations thereof, in the specification and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.

[0109] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.

[0110] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0111] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.

[0112] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0113] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.

[0114] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for optimizing process parameters, characterized in that, include: Determine the predicted values ​​of electrical indicators and their prediction uncertainties based on the process parameters to be optimized; The reinforcement learning state vector is determined based on the predicted values ​​of electrical indicators, the specified values ​​of electrical indicators, the process parameters to be optimized, the process constraint status, and the current remaining number of iterations. The reinforcement learning state vector is then input into the preset policy network to generate the initial action vector. The process parameters to be simulated are determined based on the initial motion vector and the process parameters to be optimized, and the simulation results are obtained by calling the TCAD simulator or a preset proxy model based on the process parameters to be simulated. The combined reward value is determined based on the simulation results and prediction uncertainty; The policy network is controlled to output optimal process parameters based on the reinforcement learning state vector, initial action vector, and combined reward value.

2. The process parameter optimization method as described in claim 1, characterized in that, The process of determining the reinforcement learning state vector based on the predicted values ​​of electrical indicators, the specified values ​​of electrical indicators, the process parameters to be optimized, the process constraint status, and the current remaining number of iterations includes: The specification normalized deviation vector is determined based on the predicted values ​​and the specified values ​​of electrical indicators; The reinforcement learning state vector is determined based on the normalized vector of the process parameters to be optimized, the specification normalized deviation vector, the process constraint state, and the current remaining number of iterations.

3. The process parameter optimization method as described in claim 1, characterized in that, The step of determining the process parameters to be simulated based on the initial action vector and the process parameters to be optimized includes: The initial motion vector is mapped to the process parameter increment according to the preset motion mapping strategy, and the initial process parameters are obtained based on the process parameter increment and the process parameters to be optimized. The initial process parameters are constrained and made compliant based on the preset process constraints to obtain compliant process parameters. The process parameters to be simulated are obtained by pre-screening compliant process parameters.

4. The process parameter optimization method as described in claim 1, characterized in that, The simulation results include actual electrical index values, simulation convergence status indicators, simulation costs, constraint violation degrees, and oscillation metrics. The determination of the combined reward value based on the simulation results and prediction uncertainty includes: The current total specification deviation is determined based on the actual electrical index values ​​and the specified electrical index values; The severity of failure is determined based on the simulation convergence status indicators; The cost of retesting is determined based on the cost of simulation. The combined reward value is determined based on the current total specification deviation, the total specification deviation at the previous time step, the prediction uncertainty, the degree of constraint violation, the severity of failure, the backtesting cost, and the oscillation metric.

5. The process parameter optimization method as described in claim 4, characterized in that, The determination of the combined reward value based on the current total specification deviation, the previous total specification deviation, the prediction uncertainty, the degree of constraint violation, the severity of failure, the backtesting cost, and the oscillation metric includes: The combined reward value is obtained by substituting the current total specification deviation, the previous total specification deviation, the prediction uncertainty, the degree of constraint violation, the severity of failure, the backtesting cost, the target weight coefficient, and the oscillation metric into the following calculation formula: In the formula, This represents the current total specification deviation. This represents the total specification deviation at the previous moment. The first preset threshold for compliance; The current time represents the uncertainty of the forecast. The degree of constraint violation at the current moment; The severity of the failure at the current moment; The cost of backtesting at the current moment; , , , , , , The target weight coefficient; This measures the oscillation at the current moment. This represents the combined reward value at the current moment.

6. The process parameter optimization method as described in claim 5, characterized in that, The method for determining the target weight coefficient is as follows: If the current total specification deviation is greater than the second preset target value, the preset coarse search stage weight coefficient will be used as the target weight coefficient. If the current total specification deviation is greater than the first preset compliance threshold but not greater than the second preset compliance threshold, then the preset weight coefficient for the near-specification stage will be used as the target weight coefficient. If the current total specification deviation is not greater than the first preset compliance threshold and the prediction uncertainty is greater than the preset uncertainty threshold, then the preset refinement stage weight coefficient will be used as the target weight coefficient. If the current total specification deviation is not greater than the first preset compliance threshold and the prediction uncertainty is not greater than the preset uncertainty threshold, then the preset convergence stage weight coefficient is used as the target weight coefficient, and the first preset compliance threshold is less than the second preset compliance threshold.

7. The process parameter optimization method as described in claim 1, characterized in that, The method further includes: If the preset termination condition is not met, the process parameters to be simulated will be updated to the process parameters to be optimized in the next iteration, and the remaining number of iterations will be updated according to the simulation cost. Continue with the steps described above: determining the predicted values ​​of electrical parameters and their prediction uncertainties based on the process parameters to be optimized.

8. A process parameter optimization system, characterized in that, The process parameter optimization system includes: The first processing module is used to determine the predicted values ​​of electrical indicators and their prediction uncertainties based on the process parameters to be optimized. The second processing module is used to determine the reinforcement learning state vector based on the predicted value of the electrical index, the specification value of the electrical index, the process parameters to be optimized, the process constraint status, and the current remaining number of iterations, and input the reinforcement learning state vector into the preset policy network to generate the initial action vector. The third processing module is used to determine the process parameters to be simulated based on the initial action vector and the process parameters to be optimized, and to call the TCAD simulator or a preset proxy model to perform the simulation based on the process parameters to be simulated to obtain the simulation results. The fourth processing module is used to determine the combined reward value based on simulation results, prediction uncertainty, constraint violation degree, and oscillation metric. The fifth processing module is used to update the policy network based on the reinforcement learning state vector, the initial action vector, and the combined reward value until a preset termination condition is met, and then output the optimal process parameters.

9. A process parameter optimization device, characterized in that, The process parameter optimization device includes a processor, a memory, and a process parameter optimization program stored in the memory and executable by the processor, wherein when the process parameter optimization program is executed by the processor, it implements the steps of the process parameter optimization method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a process parameter optimization program, wherein when the process parameter optimization program is executed by a processor, it implements the steps of the process parameter optimization method as described in any one of claims 1 to 7.

Images