Method for preparing thin film capacitor dielectric layer based on real-time adaptive control atomic layer deposition

Abstract

The application discloses a method for preparing a thin film capacitor dielectric layer based on real-time adaptive control, and relates to the technical field of thin film capacitor manufacturing. A digital twin model of an atomic layer deposition process is constructed, a process parameter sequence is mapped to a real-time state sequence and a final performance vector, a Markov decision process is established based on the digital twin model, and a deep deterministic policy gradient algorithm is used to train an optimal policy network of an ideal process trajectory containing an expected process parameter sequence and an expected real-time state sequence. A training data set is generated based on the digital twin model, an inversion model is trained, a real-time correction amount of a process parameter is output according to the deviation between a real-time actual state and an expected state by using the inversion model, and the corrected process parameter is applied to a production process cycle, so that the atomic layer deposition process is realized from offline global optimization to online precise and stable full-process adaptive control.

Description

Technical Field

[0001] This invention relates to the field of thin-film capacitor manufacturing technology, and in particular to a method for preparing the dielectric layer of a thin-film capacitor based on atomic layer deposition with real-time adaptive control. Background Technology

[0002] As a core component in modern electronic devices, the performance of thin-film capacitors (such as energy density, reliability, and operating frequency) directly depends on the quality of the dielectric layer. Atomic layer deposition (ALD) technology, due to its superior thickness control precision, excellent step coverage, and large-area uniformity, has become a key process for preparing high-performance nanoscale thin-film capacitor dielectric layers (such as Al2O3, HfO2, ZrO2, and their composites).

[0003] However, existing atomic layer deposition (ALD) technologies still face significant technical challenges in the industrial-scale fabrication of dielectric layers for high-performance, highly consistent thin-film capacitors. First, traditional process parameter optimization relies on a trial-and-error approach, which is inefficient and struggles to achieve global optimum. Process development heavily depends on engineers' experience, requiring repeated experiments to adjust parameters such as temperature, pressure, precursor pulse time, and plasma power. This is not only time-consuming and material-intensive, but also, due to the complex nonlinear coupling relationships between parameters, human experience struggles to identify the truly global optimal process window, thus hindering the improvement of film performance. Second, most existing ALD equipment operates in an "open-loop" or simple "program-controlled" mode, lacking real-time feedback and active compensation capabilities. In actual production, key conditions such as precursor concentration, plasma state, and cavity temperature field can experience unpredictable micro-drifts. Due to the lack of effective online monitoring and real-time compensation mechanisms, these micro-drifts lead to uncontrollable fluctuations in film thickness, chemical composition, and microstructure within and between batches, resulting in dispersion in capacitor electrical performance (such as breakdown voltage and leakage current), severely impacting product yield and reliability.

[0004] In recent years, machine learning and artificial intelligence technologies have shown potential in process modeling and optimization. However, existing research has largely focused on offline modeling and static optimization, failing to deeply integrate intelligent algorithms with the real-time execution system of deposition equipment. Therefore, how to construct an intelligent atomic layer deposition (ALD) fabrication system that combines global optimization and real-time correction remains a pressing technical challenge in this field. Thus, developing an innovative ALD fabrication method to overcome these shortcomings and achieve high-performance, high-consistency, and high-efficiency manufacturing of thin-film capacitor dielectric layers has significant industrial implications and application value. To this end, a method for fabricating thin-film capacitor dielectric layers based on real-time adaptive control using ALD is proposed. Summary of the Invention

[0005] The main objective of this invention is to provide a method for preparing the dielectric layer of a thin-film capacitor based on atomic layer deposition with real-time adaptive control, which can effectively solve the problems in the background art.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A method for fabricating thin-film capacitor dielectric layers based on real-time adaptive control atomic layer deposition includes the following steps: S1: Construct a digital twin model of the atomic layer deposition process, which is used to map the process parameter sequence. To real-time state sequence and final performance vector Forward prediction model To represent; S2: Based on the digital twin model, establish a Markov decision process and train the optimal policy network using a deep deterministic policy gradient algorithm. The optimal policy network Used to generate an ideal process trajectory that includes the expected sequence of process parameters and the expected real-time state sequence. ; S3: Generate a training dataset based on the digital twin model, and train the inversion model. The inversion model Used according to Deviation between actual state and expected state at any given moment Real-time correction of output process parameters ; S4: During the production process, following the ideal process trajectory described above. As a baseline, the following steps are performed for each atomic layer deposition cycle: The actual state vector is obtained by measuring with in-situ sensors. ; Calculate state deviation ,in For the ideal process trajectory The corresponding expected state; State deviation Input the inversion model To obtain the real-time correction amount ; The revised process parameters Apply to the next loop, where It is a saturation function. For the ideal process trajectory Expected parameters in; S5: Periodically collect actual data from production batches, calibrate the digital twin model, and apply the calibrated model to the optimal policy network. Fine-tuning is performed to update the ideal process trajectory. .

[0007] Furthermore, in step S1, the forward prediction model To achieve the hidden state of a hybrid structure employing recurrent neural networks and multilayer perceptrons The update and output are represented as follows: ; ; ; in, For the first A vector of process parameters for each cycle. This is the predicted real-time state vector. For the predicted final performance vector, It is a recurrent neural network. These are the model parameters for the recurrent neural network. , These are the weights and biases of the recurrent neural network, respectively. It is a multilayer perceptron network. These are the model parameters for a multilayer perceptron network. , These represent the weights and biases of the multilayer perceptron network.

[0008] Furthermore, in step S2, the state of the Markov decision process... Including current process parameters The current real-time state predicted by the digital twin model and cycle counting ;action Adjustment amount for process parameters in the next cycle The reward function includes single-step rewards. and endgame rewards .

[0009] Furthermore, the single-step reward Used to penalize deviations from the ideal stable growth state, defined as: , , All are penalty coefficients; For the first The increase in film mass per cycle after one cycle; This represents the increase in film mass per cycle under ideal conditions. For the first The average surface roughness of the film after one cycle; This represents the average surface roughness of the thin film under ideal conditions.

[0010] Furthermore, the final reward The weighted function of the breakdown field strength, leakage current density, and dielectric constant of the thin film is expressed as: , To measure the breakdown electric field strength, the unit is MV / cm. To achieve the preset target value of the breakdown field strength, the unit is MV / cm. Leakage current density, unit: A / cm² 2 , The preset target value for leakage current density, in A / cm². 2 , Where is the dielectric constant. The dielectric constant is the preset target value. The uniformity standard deviation is a statistic used to characterize the uniformity of film thickness or capacitance distribution across the entire substrate. , , , These are weighting coefficients, and they satisfy... .

[0011] Furthermore, in step S2, the deep deterministic policy gradient algorithm includes an Actor network, a Critic network, and a corresponding target network. and .

[0012] Furthermore, the update of the Critic network is achieved by minimizing the temporal difference error loss, defined as: Its target value Defined as: ,in The parameters of the Critic network are The loss function value at that time; For mathematical expectation operators, representing the cache from experience replay. Transfer costs sampled from Calculate its average value; For the Critic network with network parameters of At that time, in the state The predicted long-term return obtained by executing the next action; For the Critic network with network parameters of At that time, in the state Next action The obtained long-term return forecast; As a discount factor, and ; For the Actor network with parameters of At that time, the next state The next action to choose.

[0013] Furthermore, the Actor network is updated using policy gradients, which are defined as follows: ,in For the performance objective function Regarding Actor network parameters The gradient of is updated according to the following rule: ; The learning rate; Assignment operator; The output of the Critic network is the long-term return prediction relative to the input action. The gradient, and in The value is taken at the location; The output action of the Actor network relative to its own parameters The gradient, and in The value is taken at that location.

[0014] Furthermore, in step S3, the inversion model The training dataset was generated in the following way: In the digital twin model, at the nominal process parameters Apply random perturbation nearby ; Calculate random disturbances Corresponding state changes , constitute data pairs ; The model parameters are trained by minimizing the loss function through supervised learning. Where the loss function is minimized Defined as: , The number of training samples. It is a nonlinear function.

[0015] Furthermore, in step S5, the ideal process trajectory is... The update adopts a smooth switching strategy, specifically: The trajectory to be executed within L production batches From the old trajectory Gradually transitioning to a new trajectory , represented as ,in It is a transition function that monotonically increases from 0 to 1.

[0016] Furthermore, the in-situ sensor includes at least two of the following: a quartz crystal microbalance, a mass spectrometer, and a plasma atomic emission spectrometer; the actual state vector It includes at least two of the following: film mass deposition amount, peak concentration of reaction byproducts, and spectral intensity of plasma active groups.

[0017] Furthermore, the triggering condition for step S5 is: it is automatically triggered after every M production batches are completed, or it is triggered when the sliding average value of the final performance index of the film is lower than a preset threshold.

[0018] Furthermore, the method also includes a predictive maintenance step: monitoring and recording the real-time correction amount. If the correction amount corresponding to the process parameter shows a monotonically increasing trend and exceeds the warning threshold in the time series, a maintenance warning signal for the corresponding equipment component will be generated.

[0019] A real-time adaptive control-based atomic layer deposition system for fabricating the dielectric layer of a thin-film capacitor includes: Atomic layer deposition equipment is used to perform the deposition of dielectric layers; An in-situ sensor array, connected to the atomic layer deposition equipment, is used to monitor the deposition process status in real time; The computing server communicates with the atomic layer deposition equipment and in-situ sensor array. The computing servers include: The digital twin module is used to store and run digital twin models; The strategy optimization module is used to run the deep deterministic policy gradient algorithm and store the optimal policy network and ideal process trajectory. The real-time control module, with an embedded inversion model, is used to calculate and output real-time correction values ​​based on sensor data. The system scheduler coordinates the operation of the strategy optimization module and the real-time control module, and manages the updating and switching of the ideal process trajectory.

[0020] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for preparing a thin-film capacitor dielectric layer based on real-time adaptive control atomic layer deposition.

[0021] The present invention has the following beneficial effects: Compared with existing technologies, this solution uses deep reinforcement learning algorithms to conduct virtual experiments in a digital twin environment. It can systematically explore the complex process parameter space of high-dimensional and nonlinear processes, and autonomously discover process formulations that surpass human experience and are globally optimal or near optimal. This improves the key performance indicators of the dielectric layer of the prepared thin-film capacitor (such as breakdown field strength and dielectric constant), thereby increasing the yield.

[0022] Compared with existing technologies, this solution, through the integration of in-situ sensors for real-time monitoring and inversion models, can proactively detect and instantly compensate for various disturbances such as precursor concentration drift, plasma inhomogeneity, and equipment status fluctuations, maintaining the process state within an ideal trajectory. This results in a fundamental improvement in the uniformity of film thickness, chemical composition, and structural consistency.

[0023] Traditional process development relies on physical experiments, which is time-consuming. Compared with existing technologies, this invention, through a set digital twin and reinforcement learning framework, transfers most of the process exploration and optimization work to virtual space, enabling multiple virtual experiments to be completed in a very short time, thereby significantly shortening the process development cycle. Attached Figure Description

[0024] Figure 1 A schematic diagram of the process for fabricating the dielectric layer of a thin-film capacitor using atomic layer deposition based on real-time adaptive control; Figure 2 A schematic diagram of the structure of a thin-film capacitor dielectric layer system based on real-time adaptive control atomic layer deposition. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0026] Example 1: See also Figure 1 The flowchart shown is a schematic diagram of the method for preparing a thin-film capacitor dielectric layer based on real-time adaptive control atomic layer deposition according to the present invention, which specifically includes the following stages and steps: Phase 1: Digital Twin Model Construction and Data Preparation 1.1: Historical Data Collection and Processing: Collect historical operating data of the atomic layer deposition equipment, including detailed process parameter logs, in-situ sensor time-series data (such as quartz crystal microbalance, spectroscopic ellipsometry data), and electrical performance test data of the deposited thin film.

[0027] 1.2: Constructing a high-fidelity digital twin model: Based on the collected data, train a forward prediction model. This model, taking a sequence of process parameters as input, can accurately predict the dynamic state evolution of the deposition process (such as real-time thickness and roughness) and the final properties of the thin film (such as breakdown field strength and leakage current). This model will serve as the virtual simulation environment for training all subsequent algorithms, and will be used as a forward prediction model. The specific training process is as follows: 1.21: Data Preparation and Preprocessing Data source collection: Process parameter data: Raw data from multiple complete process batches were collected, each batch containing a complete dielectric layer deposition process. The data includes the key controllable parameter vector x for each deposition cycle t. t For example: cavity temperature T t Work pressure P t Precursor A pulse time t A,t Precursor B pulse time t B,t , purging time t purge,t Plasma power RF t wait.

[0028] In-situ sensor timing data: Synchronously collect the in-situ sensor reading vector o corresponding to each cycle. t For example: the mass change Δm measured by a quartz crystal microbalance. t Thickness d retrieved by spectral ellipsometer inversion t With roughness R t Peak intensity of specific byproducts detected by mass spectrometer I ms,t Specific spectral line intensities I in plasma emission spectra oes,t wait.

[0029] Final thin film performance data: After each process batch is completed, the performance vector Y is obtained by offline detection of the deposited thin film, such as the breakdown field strength E. b Leakage current density J l , dielectric constant k, etc.

[0030] Data cleaning and alignment: Invalid batches with obvious equipment malfunctions or serious data omissions will be removed.

[0031] Sensor data is filtered and smoothed to reduce the impact of random noise.

[0032] Data from different sampling frequencies (such as high-speed mass spectrometry and QCM once per cycle) are aligned to a unified deposition cycle time step t using timestamps.

[0033] Construct sequence pairs from each batch of data: X (i) =[x1,x2,...,x T ] (i) and O (i) =[o1,o2,...,o T ] (i) And the corresponding scalar performance Y (i) .

[0034] Feature engineering and sequence construction: Define the model's "state" vector s t . st Not only includes the current in-situ readings o t It can also include derived features extracted from historical data, such as the moving average of process parameters over the most recent N cycles and the trend of growth rate changes, to better characterize the dynamic properties of the process.

[0035] Therefore, each sample sequence is reconstructed as: input sequence X (i) The corresponding output sequence S (i) =[s1,s2,...,s T ] (i) and terminal output Y (i) .

[0036] The datasets of all valid batches are randomly divided into training set, validation set and test set.

[0037] 1.22: Model Architecture Definition and Initialization Network architecture design: Forward prediction model It employs a hybrid structure of recurrent neural networks and multilayer perceptrons.

[0038] The cyclic encoding section employs either a gated cyclic unit (GRU) or a long short-term memory (LSTM) network. Its input is the process parameter x for the current cycle. t and the hidden state h from the previous moment t-1 The output is the hidden state h at the current moment. t This section is used to capture the time-series dependencies of process parameters and the dynamic memory effect of the process: ; Real-time state decoder: A multilayer perceptron, with the current hidden state h t As input, output the predicted current real-time state vector ŝ t : ; Final performance predictor: A standalone multilayer perceptron with the final hidden state h of the entire process sequence. T As input, the output is the predicted final performance vector Y: ; Parameter initialization: Randomly initialize all weights and bias parameters θ={θ_{t}} in the model using standard methods (such as Xavier or He initialization). rnn W s ,b s W y ,b y ,θ mlp}

[0039] 1.23: Iterative Model Training Loss function definition: Total loss L total It is a weighted sum of the real-time state prediction loss and the final performance prediction loss.

[0040] Real-time state prediction loss: The mean squared error is used to measure the difference between the predicted state and the actual state at each time step.

[0041] Final performance prediction loss: also using mean squared error or mean absolute error (selected according to the characteristics of the performance index).

[0042] Training cycle: For the set number of training rounds N epoch Repeat the following process: Forward propagation: Take a batch of input sequence X from the training set, and pass it through the RNN part of the model, the state decoder and the performance predictor in sequence to obtain the predicted state and final performance of the entire sequence.

[0043] Loss Calculation: Using the loss function formula above, calculate the total loss L of this batch of data. total .

[0044] Backpropagation and parameter update: Calculate the total loss L total The gradient with respect to all trainable parameters θ of the model.

[0045] Gradient descent optimization algorithms (such as Adam) and their variants are used to update parameters based on the calculated gradients in order to minimize the loss.

[0046] Verification and early cessation: After a certain number of training iterations or rounds, evaluate the model performance on an independent validation set and calculate the validation loss.

[0047] Monitor changes in validation loss. If the validation loss increases instead of decreasing over multiple consecutive evaluation periods, trigger the "early stop" mechanism to prevent overfitting and save the model parameters with the minimum validation loss as the optimal parameters.

[0048] 1.24: Model Testing and Evaluation Performance evaluation: Forward predictions are performed on a test set that has never been used for training and validation, using the saved best model parameters.

[0049] Evaluation metrics calculation: Calculate metrics such as root mean square error (RMSE), mean absolute percentage error (MAPE), and correlation coefficient of the final performance prediction on the test set.

[0050] Model validation: If the evaluation metrics meet the preset accuracy requirements, the model is considered to have been successfully trained, has a sufficiently high fidelity, and can be used as a reliable digital twin.

[0051] The process is complete. The trained forward prediction model can accurately predict the dynamic state evolution of the deposition process and the final film performance based on a given sequence of historical process parameters, providing a core virtual simulation environment for subsequent reinforcement learning optimization and real-time inversion control.

[0052] Phase Two: Offline Learning of Global Optimization Strategies 2.1: Defining the Reinforcement Learning Environment: The above digital twin model is encapsulated into a reinforcement learning simulation environment. Define the state space (containing process parameters and simulated sensor states), the action space (continuous adjustments to process parameters), and the reward function (rewarding high performance and penalizing unstable growth).

[0053] 2.2: Training the Deep Deterministic Policy Gradient Model: The deep deterministic policy algorithm is trained in this virtual environment. The agent learns how to adjust process parameters to obtain the highest long-term performance reward through extensive trial and error. After training convergence, the optimal policy network μ is saved. The training process includes the following steps: 2.21: Algorithm Initialization Network construction: Randomly initialize the parameters of four neural networks: Actor online network (μ): parameter θ μ The input is the state s. t The output is a deterministic action a. t .

[0054] Critic online network (Q): parameter θ Q The input is the state s. t and action a t The output is the scalar Q value.

[0055] Actor target network (μ'): parameters are θ μ’ Initially set to θ μ’ ←θ μ .

[0056] Critic target network (Q'): parameter θ Q’ Initially set to θ Q’ ←θ Q .

[0057] Experience replay cache initialization: Create a first-in-first-out data storage area D to store the transferred samples (s) t ,a t ,r t ,s t+1 ).

[0058] Initialization of the exploration noise process: Initialize a stochastic process N (usually an Ornstein-Uhlenbeck process) for action exploration to facilitate full exploration of the action space in the early stages of training.

[0059] 2.22: Iterative Training Loop For the total number of training rounds (i.e., the simulated complete process batch) N episode Repeat the following steps: Environment and round initialization: Reset the digital twin environment to obtain the initial process state s1.

[0060] Initialize the noise process N.

[0061] Timing Interaction and Data Collection (Loop within a Complete Process Batch): For each atomic layer deposition cycle step t=1toT in this process batch: Action selection: The Actor online network generates a baseline action based on the current state: a t =μ(s t |θ μ To facilitate exploration, noise is added: a t =a t +N t .

[0062] Action execution and feedback: Executing actions a in a digital twin model t (That is, adjusting process parameters). The model simulation generates the next state s. t+1 and instant rewards t r t Based on the single-step reward function, the penalty for deviation from stable growth is defined as: .

[0063] Data storage: Transferring samples (s) t ,a t ,r t ,s t+1 Store in the experience replay cache D.

[0064] Status update: Let s t =s t+1 Then proceed to the next cycle.

[0065] Network parameter updates (learning from experience): After collecting a certain number of samples, the network is updated multiple times. Each update involves: Batch sampling: Randomly select a small batch of N transfer samples (s) from the experience replay buffer D. i ,a i ,r i ,si+1 ).

[0066] Calculate the target value: For each sample, calculate the target value using the target network: Target value Defined as: .

[0067] Updating the Critic online network: The parameters θ are updated by minimizing the mean square time difference error of the Critic network. Q The loss function is: .

[0068] Use a gradient descent algorithm (such as Adam) to minimize this loss.

[0069] Updating the Actor online network: Updating the Actor network parameters θ using the sampled policy gradient. μ The goal is to maximize the expected Q-value given by the Critic network. The gradient is approximated as: Update θ along this gradient direction μ .

[0070] Soft update of the target network: The parameters of the target network are slowly updated with a very small step size τ (τ<<1) to keep track of changes in the online network, thereby stabilizing training. θ Q’ ←τθ Q +(1-τ)θ Q’ ; θ μ’ ←τθ μ +(1-τ)θ μ’ .

[0071] 2.23: Convergence Determination and Policy Output Performance evaluation: Every fixed number of training rounds, freeze the current Actor online network μ, run multiple test batches in a digital twin environment (without adding exploration noise), and calculate the average cumulative discounted reward obtained.

[0072] Convergence determination: Training is considered converged when the average cumulative discount return no longer increases significantly in multiple consecutive evaluations, or when it reaches a preset threshold.

[0073] Policy Output: The parameters θ of the converged Actor online network μ. μ Fixed, as the optimal policy network μ * The network can receive any given process state and output the optimal process parameter adjustments that maximize long-term performance rewards.

[0074] 2.3: Generating the Ideal Process Trajectory Using a pre-trained optimal policy network μ, in a digital twin environment, starting from a standard initial state s0, a complete process batch is run according to the decisions made by μ. Each step in this process is recorded by the optimal policy network μ. * The expected process parameters x generated t * And the corresponding expected real-time state s predicted by the digital twin model. t * Sequence {(x)} t * ,s t * )} T t=1 That is, to form an ideal process trajectory It will serve as the tracking benchmark for subsequent real-time adaptive control.

[0075] The process is now complete. This process ensures that the policy network learns through trial and error, autonomously discovering high-performance, highly stable complex process formulations in a virtual environment.

[0076] Phase 3: Offline Training of Real-Time Inversion Controller 3.1: Generating Inversion Training Data: In the digital twin model, focusing on the "ideal process trajectory" At each operating point, a large number of random parameter perturbations are applied, and the state output changes caused by these perturbations are calculated, thereby generating a massive amount of (state deviation, parameter perturbation) data pairs.

[0077] 3.2: Training the Inversion Model: Using the above data pairs as the training set, a lightweight feedforward neural network, i.e., the inversion model, is trained under supervised training. Inversion model The training process includes the following steps: 3.21: Training Data Generation This step involves the already trained high-fidelity forward prediction model. The process is conducted within a digital twin model and does not involve actual production equipment.

[0078] Define the nominal operating point: One or more high-performance ideal process trajectories generated by a deep deterministic policy gradient algorithm are selected as nominal references. Each time step t on the trajectory corresponds to a nominal process parameter x. t * and a nominal expected state s t * .

[0079] Apply system parameter perturbations: For nominal parameter x t * Apply a large number of regular random perturbations δa Disturbance δ a It is a vector whose dimension is the same as the controllable process parameters (action space).

[0080] Perturbation method: Independent sampling is performed from a predetermined distribution (such as a uniform distribution or a truncated normal distribution) to generate a large number of perturbation samples {δ a (i) The disturbance range should cover the range of parameter drift that may occur in actual production (e.g., each parameter varies within the range of ±5% to ±15% of its nominal value).

[0081] Simulated disturbance response: For each perturbation sample δ a (i) In the forward prediction model Simulate its impact: Calculate the parameters after the disturbance: x t’ =x t * +δ a (i) .

[0082] x t’ With the current state s t * (Or an extended state containing historical information) are input into the model together. .

[0083] Operating Model Predict the system's state output at the next (or several) time steps after a disturbance is applied, using one time step (or a few time steps). t+1’ .

[0084] Calculate the change in state (deviation): δ s (i) =ŝ t+1’ -s t+1 * Here s t+1 * It is the expected state at time t+1 on the nominal trajectory.

[0085] Building the training dataset: The above process was performed at different time steps t and different perturbation samples δ. a (i) Repeat this process multiple times to generate a large pairing dataset D. inv .

[0086] Each sample in the dataset is an (input, output) pair: Input: State deviation vector δ s (i) .

[0087] Output: The process parameter disturbance vector δ that caused this deviation. a (i) .

[0088] The dataset needs to cover a sufficiently wide range (δ) s ,δ a Mapping space to ensure the generalization ability of the inversion model.

[0089] 3.22: Definition and Initialization of Inversion Model Architecture Network architecture design: Taking a feedforward neural network (multilayer perceptron, MLP) as an example, it includes: Input layer: Dimension and state deviation vector δ s They have the same dimensions.

[0090] Hidden layers: typically contain 1 to 3 fully connected layers, each followed by a non-linear activation function (such as ReLU).

[0091] Output layer: Dimension and process parameter correction vector δ a Since the dimensions of the action space are the same, linear activation functions are typically used.

[0092] Parameter initialization: All parameters θ of the model are randomly initialized using standard methods (such as He initialization). inv .

[0093] 3.23: Supervised Model Training Loss function definition: The mean squared error loss function is used to directly minimize the difference between the correction predicted by the model and the actual applied perturbation: This loss function forces the model to learn the inverse mapping relationship of parameter perturbations from the state bias.

[0094] Training loops and optimization: Dataset D inv It is divided into training set and validation set.

[0095] Iterative training is performed using mini-batch gradient descent and its optimizers (such as Adam).

[0096] Forward propagation: Take a small batch (δ) s ,δ a ) sample, δ s Input inversion model The predicted disturbance is obtained.

[0097] Loss Calculation and Backpropagation: Calculate the loss between the predicted perturbation and the true value, and then use the backpropagation algorithm to calculate the loss relative to the model parameters θ. inv The gradient.

[0098] Parameter update: Update θ based on gradient using the optimizer. inv In order to minimize losses.

[0099] Validation and Early Stopping: Periodically evaluate model performance on the validation set. When the validation loss no longer decreases, trigger early stopping and save the parameters of the best-performing model on the validation set.

[0100] 3.24: Model Testing and Performance Verification Closed-loop simulation test: Virtual closed-loop control tests were conducted in a digital twin environment to evaluate the trained inversion model. The actual control effectiveness.

[0101] Simulation Method: A continuous random disturbance is artificially introduced along the nominal trajectory to simulate real-world noise and drift. At each time step, based on the deviation between the currently simulated state and the nominal state, an inversion model is used. Predict the correction amount and apply it to the virtual process.

[0102] Evaluation metric: Track the cumulative error between the actual and nominal states throughout the entire simulation batch. A successful inversion model should be able to quickly suppress the bias and maintain it at a level close to zero.

[0103] Generalization ability assessment: Repeat the above closed-loop simulation test on a different nominal trajectory than that used for training (representing different process formulations or objectives) to verify the effectiveness of the model near different operating points.

[0104] Model validation and deployment: Training is considered complete when the model can stably and accurately map state deviations into effective corrective actions during testing and demonstrates good generalization.

[0105] The trained inversion model The parameters are fixed and integrated into the real-time control system, serving as the core decision-making module of the real-time adaptive control engine.

[0106] The process is complete. The inversion model trained through this process... It possesses the ability to quickly perform reverse reasoning based on online observations, making it a key intelligent component for realizing a millisecond-level real-time control closed loop of "monitoring-decision-execution". Phase Four: Online Production Initialization and Deployment 4.1 System Deployment: Combine the trained optimal policy network μ (and its defined ideal process trajectory) with the inversion model. Deployed to a real-time computing server connected to a real atomic layer deposition device.

[0107] 4.2: Setting the baseline: Set the parameter sequence in the ideal process trajectory as the initial baseline recipe for equipment operation, and set the expected state sequence as the target for real-time tracking.

[0108] Phase 5: Online Adaptive Production Operation (Dual-Loop Control) 5.1: Inner Ring Real-time adaptive control: Start production. For each atomic layer deposition cycle: a. The in-situ sensor array collects the actual state data s of the current cycle. meas .

[0109] b. s meas The deviation Δs is obtained by comparing it with the expected state of the current cycle in the ideal trajectory.

[0110] c. Input the deviation Δs into the inversion model The model instantly outputs the process parameter correction amount Δa.

[0111] d. After the correction amount Δa is checked for safety boundaries, it is added to the reference parameters in the ideal trajectory to generate the execution instructions for the next cycle, and then sent to the atomic layer deposition equipment.

[0112] 5.2: Outer Ring Periodic strategy optimization: during production breaks (e.g., after every 10 batches): a. Data aggregation and performance evaluation: Collect complete data (actual parameters, sensor data, final performance) for all production batches during this period.

[0113] b. Digital twin model calibration: Fine-tune the digital twin model using new production data to make it more closely match the current equipment status.

[0114] c. Strategy Re-optimization: Starting from the current average process state, run a deep deterministic policy gradient algorithm for short-term fine-tuning training in a calibrated digital twin environment to explore whether a new process strategy with better performance can be obtained.

[0115] d. Smooth Trajectory Update: If the new strategy is proven to be superior to the existing strategy through virtual verification, the system will smoothly and seamlessly transition the current ideal process trajectory to the new optimal trajectory in subsequent batches using a weighted averaging method. The inner loop inversion controller will automatically adapt to and track the new trajectory.

[0116] Phase Six: Monitoring, Early Warning, and Maintenance 6.1 System Monitoring: Real-time monitoring of the time series of the correction amount Δa of the inner loop output. If the correction amount of a certain parameter continues to increase in a single direction and exceeds the preset threshold, it indicates that the corresponding equipment component (such as gas valve, radio frequency power supply) may experience performance degradation.

[0117] 6.2 Predictive Maintenance: The system automatically generates early warning reports to prompt preventive maintenance, thereby changing "post-failure repair" to "predictive maintenance" and ensuring the continuity and stability of production.

[0118] The in-situ sensor array includes at least two of the following: a quartz crystal microbalance, a mass spectrometer, and a plasma emission spectrometer.

[0119] The triggering conditions for outer loop optimization can be flexibly set according to the production plan, such as triggering at a fixed batch interval, or triggering when the rolling average of product performance indicators shows a downward trend.

[0120] Through the above process, this invention achieves an intelligent upgrade of the atomic layer deposition process: in the offline stage (stages one to three), digital twins and artificial intelligence are used to find the global optimal solution; in the online stage (stages four to six), dual-loop control is used to ensure the extreme stability of production at the moment (inner loop) and to achieve the long-term continuous evolution of the process (outer loop), ultimately achieving a synergistic improvement in the performance, consistency and production efficiency of the thin film capacitor dielectric layer.

[0121] Example 2: This invention also provides a system for fabricating the dielectric layer of a thin-film capacitor based on real-time adaptive control, see [link to relevant documentation]. Figure 2 The system architecture diagram shown includes: Atomic layer deposition equipment is used to perform the deposition of dielectric layers; An in-situ sensor array, connected to the atomic layer deposition equipment, is used to monitor the deposition process status in real time; The computing server communicates with the atomic layer deposition equipment and in-situ sensor array. The computing servers include: The digital twin module is used to store and run digital twin models; The strategy optimization module is used to run the deep deterministic policy gradient algorithm and store the optimal policy network and ideal process trajectory. The real-time control module, with an embedded inversion model, is used to calculate and output real-time correction values ​​based on sensor data. The system scheduler coordinates the operation of the strategy optimization module and the real-time control module, and manages the updating and switching of the ideal process trajectory.

[0122] Example 3: The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the steps of the above-described method for preparing a thin-film capacitor dielectric layer by atomic layer deposition based on real-time adaptive control.

[0123] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A method for preparing a thin-film capacitor dielectric layer based on real-time adaptive control atomic layer deposition, characterized in that, Includes the following steps: S1: Construct a digital twin model of the atomic layer deposition process, which is used to map the process parameter sequence. To real-time state sequence and final performance vector Forward prediction model To represent; S2: Based on the digital twin model, establish a Markov decision process and train the optimal policy network using a deep deterministic policy gradient algorithm. The optimal policy network Used to generate an ideal process trajectory that includes the expected sequence of process parameters and the expected real-time state sequence. ; S3: Generate a training dataset based on the digital twin model, and train the inversion model. The inversion model Used according to Deviation between actual state and expected state at any given moment Real-time correction of output process parameters ; S4: During the production process, following the ideal process trajectory described above. As a baseline, the following steps are performed for each atomic layer deposition cycle: The actual state vector is obtained by measuring with in-situ sensors. ; Calculate state deviation ,in For the ideal process trajectory The corresponding expected state; State deviation Input the inversion model To obtain the real-time correction amount ; The revised process parameters Apply to the next loop, where It is a saturation function. For the ideal process trajectory Expected parameters in; S5: Periodically collect actual data from production batches, calibrate the digital twin model, and apply the calibrated model to the optimal policy network. Fine-tuning is performed to update the ideal process trajectory. .

2. The method for preparing a thin-film capacitor dielectric layer based on real-time adaptive control of atomic layer deposition according to claim 1, characterized in that, In step S1, the forward prediction model To achieve the hidden state of a hybrid structure employing recurrent neural networks and multilayer perceptrons The update and output are represented as follows: ； ； ； in, For the first A vector of process parameters for each cycle. This is the predicted real-time state vector. For the predicted final performance vector, It is a recurrent neural network. These are the model parameters for the recurrent neural network. , These represent the weights and biases of the recurrent neural network, respectively. It is a multilayer perceptron network. These are the model parameters for a multilayer perceptron network. , These represent the weights and biases of the multilayer perceptron network.

3. The method for preparing a thin-film capacitor dielectric layer based on real-time adaptive control of atomic layer deposition according to claim 1, characterized in that, In step S2, the state of the Markov decision process Including current process parameters Current real-time state predicted by digital twin model and cycle counting ;action Adjustment amount for process parameters in the next cycle The reward function includes single-step rewards. and endgame rewards The single-step reward Used to penalize deviations from the ideal stable growth state, defined as: , , All are penalty coefficients; For the first The increase in film mass after one cycle; This represents the increase in film mass per cycle under ideal conditions. For the first The average surface roughness of the film after one cycle; The average roughness of the thin film surface under ideal conditions; the final reward It is a weighted function of the breakdown field strength, leakage current density and dielectric constant of the thin film.

4. The method for preparing a thin-film capacitor dielectric layer based on real-time adaptive control of atomic layer deposition according to claim 3, characterized in that, In step S2, the deep deterministic policy gradient algorithm includes an Actor network, a Critic network, and a corresponding target network. and The Critic network is updated by minimizing the temporal difference error loss, and the Actor network is updated using the policy gradient; the target value of the Critic network is... Defined as: ,in For the Critic network with network parameters of At that time, in the state The predicted long-term return obtained by executing the next action; As a discount factor, and ; For the Actor network with parameters of At that time, the next state The next action to choose.

5. The method for preparing a thin-film capacitor dielectric layer based on real-time adaptive control of atomic layer deposition according to claim 1, characterized in that, In step S3, the inversion model The training dataset was generated in the following way: In the digital twin model, at the nominal process parameters Apply random perturbation nearby ; Calculate random disturbances Corresponding state changes , constitute data pairs ; The model parameters are trained by minimizing the loss function through supervised learning. Where the loss function is minimized Defined as: , The number of training samples. It is a nonlinear function.

6. The method for preparing a thin-film capacitor dielectric layer based on real-time adaptive control of atomic layer deposition according to claim 1, characterized in that, In step S5, the ideal process trajectory is... The update adopts a smooth switching strategy, specifically: The trajectory to be executed within L production batches From the old trajectory Gradually transitioning to a new trajectory , represented as ,in It is a transition function that monotonically increases from 0 to 1.

7. The method for preparing a thin-film capacitor dielectric layer based on real-time adaptive control of atomic layer deposition according to claim 1, characterized in that, The in-situ sensor includes at least two of the following: a quartz crystal microbalance, a mass spectrometer, and an inductively coupled plasma atomic emission spectrometer; the actual state vector It includes at least two of the following: film mass deposition amount, peak concentration of reaction byproducts, and spectral intensity of plasma active groups.

8. The method for preparing a thin-film capacitor dielectric layer based on real-time adaptive control of atomic layer deposition according to claim 1, characterized in that, The triggering conditions for step S5 are: automatically triggered after each M production batches are completed, or triggered when the sliding average value of the final performance index of the film is lower than a preset threshold.

9. The method for preparing a thin-film capacitor dielectric layer based on real-time adaptive control of atomic layer deposition according to claim 1, characterized in that, The method further includes: monitoring and recording the real-time correction amount. If the correction amount corresponding to the process parameter shows a monotonically increasing trend and exceeds the warning threshold in the time series, a maintenance warning signal for the corresponding equipment component will be generated.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method for preparing a thin-film capacitor dielectric layer based on real-time adaptive control as described in any one of claims 1-9.

Images