Atomic layer deposition process management method and system based on digital twinning

Abstract

The application is suitable for the field of atomic layer deposition, and discloses an atomic layer deposition process management method and system based on digital twinning, which comprises the following steps: constructing a digital twinning model of an atomic layer deposition production system based on a multi-scale physical information neural network model; acquiring real-time reaction cavity process parameters and real-time state variables, and inputting the real-time reaction cavity process parameters and real-time state variables into the digital twinning model to obtain the physical field distribution in the reaction cavity and the predicted growth state of the substrate surface film; based on the physical field distribution in the reaction cavity and the predicted growth state of the substrate surface film, performing decision analysis through a pre-trained reinforcement learning intelligent agent to obtain correction parameters for optimizing the film growth quality; generating correction instructions based on the correction parameters, and updating the reaction cavity process parameters based on the correction instructions to make the substrate surface film in the reaction cavity reach the target film thickness and uniformity, so that a real-time intelligent closed-loop control of perception-prediction-decision-execution can be formed.

Description

Technical Field

[0001] This invention relates to the field of atomic layer deposition, and more particularly to a method and system for managing atomic layer deposition processes based on digital twins. Background Technology

[0002] Atomic layer deposition (ALD), a key technology for achieving atomic-level precision thin film growth, involves a multi-scale, strongly coupled physicochemical mechanism, ranging from macroscopic fluid dynamics, mass and heat transfer within the reaction chamber to chemical adsorption and reactions on the microscopic substrate surface. While digital twin technology offers the possibility of intelligent ALD processes, existing modeling methods still face significant bottlenecks. Physical mechanism models based on the coupling of computational fluid dynamics (CFD) and surface reaction dynamics, although offering high physical fidelity, suffer from enormous computational costs in solving high-dimensional nonlinear partial differential equations, making it difficult to meet the real-time and online optimization control requirements of digital twins. Furthermore, solving the model's inverse problem is challenging. Purely data-driven machine learning models, on the other hand, suffer from poor generalization ability due to the high cost and scarcity of data for key ALD quality parameters (such as film thickness and uniformity). In data-sparse regions or under extreme process conditions, predictions may even violate fundamental physical laws. While Physics-Informed Neural Networks (PINNs) can integrate data and physical constraints, combining the multi-scale, strongly coupled processes of Alternating Dynamics (ALD) into a single PINN model often leads to unstable training and convergence difficulties due to the vast differences in residual magnitudes between different physical processes (macroscopic flow fields and microscopic reactions) and extremely high equation complexity. Furthermore, existing digital twin systems, when achieving closed-loop optimization, often rely on traditional optimization algorithms separated from the model for parameter search, resulting in low efficiency and an inability to achieve truly real-time dynamic control.

[0003] Therefore, existing technologies still need improvement and development. Summary of the Invention

[0004] The first objective of this invention is to provide a digital twin-based atomic layer deposition process management method, which aims to solve the technical problem that the process parameters of existing atomic layer deposition systems cannot be optimized and controlled in real time.

[0005] To achieve the above objectives, the solution provided by the present invention is as follows:

[0006] A digital twin-based method for managing atomic layer deposition (ALD) processes includes: constructing a digital twin model of an ALD production system based on a multi-scale physical information neural network model; acquiring real-time reaction chamber process parameters and real-time state variables, and inputting these parameters and variables into the digital twin model to obtain the physical field distribution within the reaction chamber and the predicted growth state of the thin film on the substrate surface; performing decision analysis using a pre-trained reinforcement learning agent based on the physical field distribution within the reaction chamber and the predicted growth state of the thin film on the substrate surface to obtain correction parameters for optimizing the film growth quality; generating correction instructions based on the correction parameters, and updating the reaction chamber process parameters based on the correction instructions to ensure that the thin film on the substrate surface within the reaction chamber achieves the target film thickness and uniformity.

[0007] Preferably, the digital twin model of the atomic layer deposition production system based on the multi-scale physical information neural network model includes: constructing a macroscopic sub-network based on a fully connected neural network, which is used to solve the fluid dynamics, mass transfer, and heat transfer problems at the macroscopic reaction chamber scale, and whose physical constraints include mass conservation, momentum conservation, energy conservation, and species transport equations; constructing a microscopic sub-network based on a fully connected neural network, which is used to solve the chemical adsorption and thin film growth kinetics problems at the microscopic scale of the substrate surface, and whose physical constraints include surface reaction kinetic equations; coupling the macroscopic and microscopic sub-networks through physical boundary conditions so that the predicted output of the macroscopic sub-network is used as the input of the microscopic sub-network to obtain the multi-scale physical information neural network model; acquiring training data and using the training data to train the multi-scale physical information neural network model to obtain the digital twin model.

[0008] Preferably, the total loss function of the digital twin model is L. total L total Represented as:

[0009] L total =L macro +L micro

[0010] L macro =λ phys,macro L phys,macro +λ data,macro L data,macro +λ bc,macro L bc,macro

[0011] L micro =λ phys,micro L phys,micro +λ data,micro L data,micro +λ bc,micro L bc,micro

[0012] In the formula, L macro L represents the overall loss function of the macroscopic subnetwork. micro L represents the overall loss function of the micro-subnetwork. data,macro L represents the mean square error loss between macroscopic measurement data and network predictions. bc,macro For the initial and boundary condition constraint loss, L phys,macro L represents the residual loss of the macroscopic physical equations. data,micro L represents the mean square error loss between the microscopic measurement data and the network predictions. bc,micro For the initial and boundary condition constraint loss, L phys,micro λ represents the residual loss of the microscopic surface reaction kinetic equation. phys,macro , λ data,macro , λ bc,macro λ represents the hyperparameters of the physical loss term, data loss term, and initial and boundary condition loss term of the macroscopic subnetwork, respectively. phys,micro , λ data,micro , λ bc,micro These are the hyperparameters for the physical loss term, data loss term, and initial and boundary condition loss term of the micro-subnetwork, respectively.

[0013] Preferably, L phys,macro and L phys,micro They are represented as follows:

[0014]

[0015] In the formula, F mc F ns F ec F st Representing the residual operators of the mass conservation equation, momentum conservation equation, energy conservation equation, and species transport equation, respectively, n θ It is a deep neural network with parameter θ, {X macro,i} is the location point sampled within the macroscopic computational domain, N macro F represents the total number of configuration points. sk For the residual operator of the surface reaction kinetic equation, m γ It is a deep neural network with parameter y, {X micro,j} is the configuration point sampled within the microscopic surface domain, N micro This represents the total number of configuration points.

[0016] Preferably, the step of acquiring training data and using the training data to train the multi-scale physical information neural network model to obtain a digital twin model specifically includes: training the macroscopic sub-network using fluid dynamics simulation data combined with collected historical macroscopic physical field data, and optimizing the parameters of the macroscopic sub-network by minimizing the loss function of the macroscopic sub-network; inputting historical reaction chamber process parameters and historical state variables into the trained macroscopic scale sub-model to obtain surface thin film growth state prediction data, training the microscopic sub-network using experimentally measured surface thin film growth state data, and optimizing the parameters of the microscopic sub-network by minimizing the loss function of the microscopic sub-network; jointly fine-tuning the parameters of the macroscopic sub-network and the microscopic sub-network, and optimizing the parameters of the macroscopic sub-network and the microscopic sub-network by minimizing the total loss function to obtain the digital twin model.

[0017] Preferably, the step of predicting the growth state of the thin film based on the physical field distribution within the reaction chamber and the substrate surface thin film, and then performing decision analysis through a pre-trained reinforcement learning agent to obtain correction parameters for optimizing the thin film growth quality, further includes: constructing an agent based on an Actor network and a Critic network; instantiating the digital twin model into a reinforcement learning environment, and defining a state space, action space, and reward function. The state space includes the predicted thin film thickness, predicted thin film uniformity, and real-time process parameters at the current moment; defining the action space includes the adjustment amount of controllable process parameters; designing a reward function R(t) to evaluate the effect of the control action and guide the agent to learn the optimal strategy; employing a proximal policy optimization algorithm, generating experience tuples through the interaction between the agent and the reinforcement learning environment and storing them in an experience replay buffer, and sampling experience data to update the parameters of the Actor network and the Critic network until the agent converges, thus obtaining a pre-trained agent.

[0018] Preferably, the reward function is defined as R(t), then R(t) is expressed as:

[0019] R(t) = w1exp[-|d pred (t+1)-d target |ξ d -w2max(0,δ th -δ pred (t+1))

[0020] In the formula, d pred (t) represents the predicted value of the thin film on the substrate surface, d target ξ represents the target film thickness. d δ represents the thickness convergence adjustment factor. th To meet the minimum acceptable uniformity threshold, δ pred (t) represents the predicted film uniformity, δ pred(t+1) represents the predicted film uniformity at the next moment, and w1 and w2 are weighting coefficients used to balance the importance of thickness accuracy and uniformity.

[0021] Preferably, the correction parameters include an adjustment amount ΔT of the reaction chamber temperature T and an adjustment amount Δt of the precursor pulse time. pulse Adjustment amount Δt for inert gas purging time purge .

[0022] The second objective of this invention is to provide a digital twin-based atomic layer deposition (ALD) process management system. This system implements the digital twin-based ALD process management method described above. The digital twin-based ALD process management system includes a physical module, a digital twin module, a decision control module, and an interaction module. The physical module includes an ALD device, an integrated sensor network, and a programmable actuator. The integrated sensor network includes a chamber pressure sensor, a multi-point temperature sensor, and a residual gas analysis mass spectrometer. The programmable actuator is used to precisely control the precursor pulse valve, the purge gas valve, and the heater power. The digital twin module... The growth module includes a digital twin model, which is used to predict the physical field distribution inside the reaction chamber and the predicted growth state of the thin film on the substrate surface based on real-time reaction chamber process parameters and real-time state variables. The decision control module interacts with the digital twin model and performs decision analysis based on the physical field distribution inside the reaction chamber and the predicted growth state of the thin film on the substrate surface predicted by the digital twin model to obtain correction parameters for optimizing the quality of thin film growth. The interaction module communicates with the physical module, the digital twin module and the decision control module respectively to provide a human-computer interaction page and display the reaction chamber process parameters and real-time state variables, the physical field distribution inside the reaction chamber and the predicted growth state of the thin film on the substrate surface and the correction parameters.

[0023] In this scheme, a digital twin model of the atomic layer deposition (ALD) production system is constructed based on a multi-scale physical information neural network model. This model utilizes multi-scale PINN to explicitly decouple the macroscopic flow field and microscopic surface reaction, and couples them through physical boundary conditions. This ensures that the digital twin model possesses both high physical fidelity and excellent training stability, effectively overcoming the convergence problem caused by scale differences in traditional single models. Secondly, the trained digital twin model is used as a virtual training environment for the reinforcement learning agent, enabling the agent to autonomously learn complex nonlinear optimal control strategies in a low-cost, risk-free digital space, achieving adaptive optimization without explicitly solving inverse problems. Finally, this architecture places time-consuming high-fidelity simulation calculations in the offline training stage, enabling online decision-making speeds to reach sub-second levels. This not only significantly improves optimization efficiency and robustness and avoids the predicament of traditional algorithms easily getting trapped in local optima, but also forms a real-time intelligent closed-loop control of perception-prediction-decision-execution, thereby achieving autonomous optimization and high-quality production of the ALD process in the face of equipment aging and raw material fluctuations. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0025] Figure 1 This is a flowchart of the atomic layer deposition process management method based on digital twin provided in an embodiment of the present invention;

[0026] Figure 2 This is a structural block diagram of the atomic layer deposition process management system based on digital twin provided in an embodiment of the present invention.

[0027] Explanation of icon numbers:

[0028] 201. Entity Module; 202. Digital Twin Module; 203. Decision Control Module; 204. Interaction Module. Detailed Implementation

[0029] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” or “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 In this embodiment of the invention, the atomic layer deposition process management method based on digital twins includes:

[0031] S101. Constructing a digital twin model of an atomic layer deposition production system based on a multi-scale physical information neural network model;

[0032] S102. Obtain the real-time reaction chamber process parameters and real-time state variables, and input the real-time reaction chamber process parameters and real-time state variables into the digital twin model to obtain the physical field distribution in the reaction chamber and the predicted growth state of the thin film on the substrate surface.

[0033] S103. Based on the physical field distribution in the reaction chamber and the predicted growth state of the thin film on the substrate surface, a pre-trained reinforcement learning agent is used for decision analysis to obtain the correction parameters for optimizing the growth quality of the thin film.

[0034] S104. Generate correction instructions based on correction parameters, and update the reaction chamber process parameters based on correction instructions, so that the substrate surface film in the reaction chamber achieves the target film thickness and uniformity.

[0035] In this embodiment, step S101, in some embodiments, exemplarily, involves constructing a digital twin model of an atomic layer deposition production system based on a multi-scale physical information neural network model, including: constructing a macroscopic sub-network based on a fully connected neural network, where the macroscopic sub-network is used to solve fluid dynamics, mass transfer, and heat transfer problems at the macroscopic reaction chamber scale, and the physical constraints of the macroscopic sub-network include mass conservation, momentum conservation, energy conservation, and species transport equations; constructing a microscopic sub-network based on a fully connected neural network, where the microscopic sub-network is used to solve chemical adsorption and thin film growth kinetics problems at the microscopic scale of the substrate surface, and the physical constraints of the microscopic sub-network include surface reaction kinetic equations; coupling the macroscopic sub-network and the microscopic sub-network through physical boundary conditions, so that the predicted output of the macroscopic sub-network is used as the input of the microscopic sub-network, thereby obtaining a multi-scale physical information neural network model; acquiring training data and using the training data to train the multi-scale physical information neural network model, thereby obtaining a digital twin model. By constructing two sub-networks, one macroscopic and one microscopic, and embedding mass, momentum, and energy conservation equations and surface reaction kinetic equations as physical constraints, this method innovatively achieves effective decoupling and accurate modeling of multi-scale, strongly coupled physicochemical mechanisms in atomic layer deposition. Furthermore, by using physical boundary conditions to apply the predicted output of the macroscopic sub-network as the input of the microscopic sub-network, the physical consistency of cross-scale information transmission and the self-consistency of the model are ensured. Combined with a staged training strategy, this method can efficiently integrate CFD simulation data and sparse experimental data, significantly reducing the dependence on expensive measurement data. The resulting digital twin model not only possesses high physical fidelity and excellent data fitting ability but also exhibits outstanding generalization performance and stability, laying a solid foundation for achieving real-time high-precision prediction and intelligent optimization control of the process.

[0036] It should be noted that the species transport equation is one of the core equations in fields such as fluid mechanics, chemical engineering, and environmental science. It is used to describe the changes in the concentration of a certain species (such as molecules, ions, or particles) in a multi-component mixture over time and space. Essentially, it is a concrete manifestation of the "law of conservation of mass" in the context of multi-component flow. By quantifying the three core processes of "convection, diffusion, and generation / consumption," it reveals the evolutionary mechanism of species concentration.

[0037] In this embodiment, the overall loss function of the macroscopic subnetwork is L. macro Then L macro Represented as:

[0038] L macro =λ phys,macro L phys,macro +λ data,macro L data,macro +λ bc,macro L bc,macro

[0039] In the formula, L data,macro L represents the mean square error loss between macroscopic measurement data and network predictions. bc,macro For the initial and boundary condition constraint loss, L phys,macro λ represents the residual loss of the macroscopic physical equations. phys,macro , λ data , macro , λ bc,macro These are the hyperparameters for the Macro-PINN physical loss term, data loss term, and initial and boundary condition loss terms, respectively.

[0040] In this embodiment, the overall loss function of the micro-subnetwork is L. micro Then L micro Represented as:

[0041] L micro =λ phys,micro L phys,micro +λ data,micro L data,micro +λ bc,micro L bc,micro

[0042] In the formula, L data,micro L represents the mean square error loss between the microscopic measurement data and the network predictions. bc,micro For the initial and boundary condition constraint loss, L phys,micro λ represents the residual loss of the microscopic surface reaction kinetic equation. phys , micro , λ data,micro , λ bc,micro These are the hyperparameters for the Micro-PINN physical loss term, data loss term, and initial and boundary condition loss term, respectively.

[0043] The total loss function of the digital twin model is L total L total Represented as:

[0044] L total =L macro +L micro .

[0045] In this embodiment, in the thermal ALD growth process of alumina (Al2O3) thin films, the atomic layer deposition system (ALD system) uses trimethylaluminum (TMA) and deionized water (H2O) as precursors, and nitrogen (N2) as purging and carrier gas. One growth cycle of ALD includes four steps: First, a TMA pulse enters the reaction chamber and reacts with surface hydroxyl groups to generate methane and surface-bound aluminum species; second, an inert gas purging is performed to remove the TMA precursor and gas-phase reaction byproducts; then, a water vapor pulse is introduced to react with the aluminum species to complete the growth of the alumina layer and regenerate the surface hydroxyl groups; finally, an inert gas is introduced again to remove the H2O precursor and gas-phase reaction byproducts.

[0046] In this embodiment, training data is acquired and used to train a multi-scale physical information neural network model to obtain a digital twin model. Specifically, this includes: training the macroscopic sub-network using fluid dynamics simulation data combined with collected historical macroscopic physical field data, and optimizing the parameters of the macroscopic sub-network by minimizing its loss function; inputting historical reaction chamber process parameters and historical state variables into the trained macroscopic sub-model to obtain surface thin film growth state prediction data, and training the microscopic sub-network using experimentally measured surface thin film growth state data, and optimizing the parameters of the microscopic sub-network by minimizing its loss function; jointly fine-tuning the parameters of the macroscopic and microscopic sub-networks, and optimizing the parameters of both the macroscopic and microscopic sub-networks by minimizing the total loss function to obtain the digital twin model.

[0047] Specifically, the input vector of the macro-PINN subnetwork is X. macro = [x,y,z,t,q], where (x,y,z) are three-dimensional spatial coordinates, t is time, and q is a process parameter vector, which includes chamber temperature T, pressure P, and precursor pulse time t. pulse Inert gas purging time t purge etc.; the output is the macroscopic physical field Φ macro (X macro )=[c(x,y,z,t),u(x,y,z,t),T(x,y,z,t)], where c is the precursor concentration and u is the flow field velocity vector. The physical constraints of Macro-PINN consist of the mass conservation equation, momentum conservation equation, energy conservation equation, and species transport equation describing continuous fluids, with the residual loss function L of the corresponding partial differential equation system. phys,macro Defined as:

[0048]

[0049] In the formula, F mc F ns F ecF st Representing the residual operators of the mass conservation equation, momentum conservation equation (i.e., the Navier-Stokes equation), energy conservation equation, and species transport equation, respectively, n θ It is a deep neural network with parameter θ, {X macro,i} is the location point sampled within the macroscopic computational domain, N macro This represents the total number of configuration points.

[0050] Furthermore, the loss function L of Macro-PINN macro It also includes data loss L data,macro and initial and boundary condition loss L bc,macro Data loss L data,macro The mean square error between the neural network predictions and the actual measurements, and the initial and boundary condition loss L. bc,macro The mean square error between the neural network output value and the true initial and boundary condition values ​​is...

[0051] L macro =λ phys,macro L phys,macro +λ data,macro L data,macro +λ bc,macro L bc,macro

[0052] In the formula, λ phys,macro , λ data,macro , λ bc,macro These are the hyperparameters for the Macro-PINN physical loss term, data loss term, and initial and boundary condition loss term, respectively, used to balance the contributions of each loss term.

[0053] Understandably, the initial and boundary condition loss L bc,macro This is the mean squared error between the neural network output value and the actual initial and boundary condition values. It can be set according to the actual situation. It can be the mean squared error between the neural network output value (predicted initial condition value) and the actual initial condition value, or the mean squared error between the neural network output value (predicted boundary condition value) and the actual boundary condition value, or it can be a weighted sum of the mean squared errors between the neural network output value (predicted initial condition value) and the actual initial condition value and the mean squared errors between the neural network output value (predicted boundary condition value) and the actual boundary condition value.

[0054] Specifically, the Micro-PINN subnetwork focuses on solving the chemisorption and thin film growth kinetics at the microscale on the substrate surface. Its input vector is X. micro =[x s ,y s ,t,q,c s (x s ,ys ,t),T s (x s ,y s ,t)], where (x s ,y s ) represents the coordinates of the substrate surface, c s (x s ,y s ,t)T s (x s ,y s ,t) represent the positions (x,t) of the Macro-PINN on the substrate surface. s ,y s The predicted concentration and temperature at the specified location are crucial for multi-scale coupling; the output is the predicted growth state Ψ of the thin film on the substrate surface. micro (X micro )=[d(x s ,y s ,t),β(x s ,y s ,t)], where d is the film thickness and β is the precursor surface coverage. The physical constraints of Micro-PINN are constituted by the surface reaction kinetic equation, and its residual loss function is defined as:

[0055]

[0056] In the formula, F sk For the residual operator of the surface reaction kinetic equation, m γ It is a deep neural network with parameter y, {X micro,j} is the configuration point sampled within the microscopic surface domain, N micro This represents the total number of configuration points.

[0057] Similarly, the loss function L of Micro-PINN micro It also includes data loss L data,micro Initial and boundary condition loss L bc,micro Two items, namely

[0058] L micro =λ phys,micro L phys,micro +λ data,micro L data,micro +λ bc,micro L bc,micro

[0059] In the formula, λ phys,micro , λ data,micro , λ bc,micro These are the hyperparameters for the Micro-PINN physical loss term, data loss term, and initial and boundary condition loss term, respectively.

[0060] The two sub-networks are coupled through physical boundary conditions, namely the output c(x) of Macro-PINN on the substrate surface. s ,y s ,t) and T(x s ,y s The input (t) is directly used as the input to the Micro-PINN. This coupling ensures the physical driving effect of the macroscopic flow field on the microscopic surface processes. The total loss function L of the multi-scale PINN is... total The sum of the loss functions of the two subnetworks:

[0061] L total =L macro +L micro .

[0062] In this embodiment, in step S102, in some embodiments, exemplaryly, in the thermal ALD growth of alumina (Al2O3) thin film, the reaction chamber process parameters include: reaction chamber temperature T, pressure P, and TMA pulse time t. pulse1 H2O pulse time t pulse2 Inert gas purging time t purge .

[0063] The state variables were obtained as follows: the instantaneous concentrations of TMA and H2O were obtained by a residual gas analysis mass spectrometer in the reaction chamber, and the film thickness d and uniformity δ were intermittently measured by an integrated in-situ ellipsometry.

[0064] In this embodiment, the physical field distribution within the reaction chamber is the precursor concentration distribution c(x,y,z,t), flow velocity vector distribution u(x,y,z,t), and temperature distribution T(x,y,z,t) in three-dimensional space output by the macroscopic sub-network.

[0065] The predicted growth state of the thin film on the substrate surface is the predicted film thickness d(x) at different locations on the substrate surface output by the micro-subnetwork. s ,y s ,t), Predicted value of precursor surface coverage β(x) s ,y s ,t), film thickness uniformity prediction value δ pred (t).

[0066] Furthermore, Macro-PINN is a 9-layer fully connected neural network, including an input layer, 7 hidden layers, and an output layer. The input parameters have a dimension of 9, including spatial coordinates (x, y, z), time t, and the aforementioned 5 process parameters q = [T, P, t]. pulse1 ,t pulse2 ,t purgeThe output parameters have four dimensions: TMA concentration c1, H2O concentration c2, flow rate u, and temperature T. Each hidden layer contains 386 neurons and uses the Swish activation function.

[0067] The macroscopic subnetwork is constrained by the conservation of mass, momentum, energy, and the species transport equation; the corresponding residual operator can be expressed as:

[0068]

[0069] In the formula, ρ is the gas density, μ is the gas viscosity, Γ is the external volume force, and C p κ is the specific heat capacity, D is the thermal conductivity. i Ω i Let represent the diffusion coefficient of the i gaseous components in the reaction chamber and the net flux from the surface chemical reaction or mass transfer boundary, respectively.

[0070] Substituting the above parameters into the overall loss function L of the macroscopic subnetwork macro .

[0071] Micro-PINN is a 7-layer fully connected neural network, including an input layer, 5 hidden layers, and an output layer. The input parameters have a 10-dimensional dimension, including the substrate surface coordinates (x, y, φ). s ,y s ), time t, process parameter q = [T, P, t pul se1 ,t pulse2 ,t purge ] and the TMA concentration c1(x) at the substrate surface provided by Macro-PINN prediction. s ,y s ,t) and H2O concentration c2(x s ,y s The output parameters have a dimension of 2, namely the film thickness d and the surface coverage β. Each hidden layer also contains 386 neurons and uses the Swish activation function.

[0072] Micro-PINN is constrained by surface reaction kinetics. Taking the classic condensation-hydrolysis reaction mechanism of Al2O3 AlD as an example, its physical residual can be expressed as:

[0073]

[0074] In the formula, α1 and α2 represent the reaction rates of the TMA and H2O half-reactions, respectively, exhibiting self-limiting characteristics. The film growth rate is proportional to α1, and β... OH The thickness d is the coverage of surface hydroxyl groups, and is obtained by integrating the growth rate over time.

[0075] Substituting the above parameters into the overall loss function L of the micro-subnetwork micro .

[0076] In this embodiment, in step S103, in some embodiments, exemplarily, based on the physical field distribution within the reaction chamber and the predicted growth state of the thin film on the substrate surface, a pre-trained reinforcement learning agent performs decision analysis to obtain correction parameters for optimizing the thin film growth quality. Prior to this, the method further includes: constructing an agent based on an Actor network and a Critic network; instantiating the digital twin model into a reinforcement learning environment, and defining the state space s(t), action space a(t), and reward function R(t). The state space s(t) includes the predicted thin film thickness d at the current moment. pred (t), Predicted film uniformity δ pred Given the controllable process parameters q(t) and q(t), the action space a(t) is defined, which includes the adjustment amount of the controllable process parameters. A reward function R(t) is designed to evaluate the effect of the control action and guide the agent to learn the optimal policy. The proximal policy optimization algorithm is adopted to generate experience tuples through the interaction between the agent and the reinforcement learning environment and store them in the experience replay buffer. The experience data is sampled to update the parameters of the Actor network and the Critic network until the agent converges, thus obtaining the pre-trained agent.

[0077] In this embodiment, a proximal policy optimization algorithm is used to train the agent's optimization strategy by having it interact with the environment, thereby maximizing long-term cumulative rewards. Using a reward function, after approximately 1000 iterations of training, the agent learns a strategy to stably approach the target thickness and maintain high uniformity by coordinating pulse / purge times and fine-tuning the temperature. Coordinating the pulse / purge times ensures sufficient reaction and removal of byproducts, while fine-tuning the temperature regulates the reaction rate.

[0078] Specifically, when the buffer accumulates enough data or reaches the update cycle, the policy updater samples a batch of empirical data from the buffer, uses the near-end policy optimization algorithm to calculate the policy gradient, and updates the parameters of the Actor network and the Critic network so that they can obtain higher cumulative rewards in future decisions.

[0079] In this embodiment, the reward function R(t) is expressed as:

[0080] R(t) = w1exp[-|d pred (t+1)-d target |ξ d -w2max(0,δ th -δ pred (t+1))

[0081] In the formula, d pred(t) represents the predicted value of the thin film on the substrate surface, d target ξ represents the target film thickness. d δ represents the thickness convergence adjustment factor. th To meet the minimum acceptable uniformity threshold, δ pred (t) represents the predicted film uniformity, δ pred (t+1) represents the predicted film uniformity at the next time step, and w1 and w2 are weighting coefficients used to balance the importance of thickness accuracy and uniformity. The first term of the reward function R(t) uses an exponential decay function to encourage the agent to quickly and smoothly approach the target thickness; the second term is a uniformity penalty term, which applies a negative reward when the uniformity is below a threshold, forcing the agent to ensure high quality at the same time.

[0082] In this embodiment, the real-time dynamic process optimization process based on reinforcement learning in the thermal ALD growth of alumina (Al2O3) thin films is as follows:

[0083] 1) Initialization: Set initial process parameters q0 and target thickness d. target Uniformity threshold δ th And reinforcement learning hyperparameters.

[0084] 2) State Observation: At the beginning of each control cycle, the system acquires the current process parameter q(t) and sensor data from the physical entity layer. This data is input into the digital twin model for forward inference to obtain the predicted growth state of the thin film on the substrate surface at the current moment: s(t) = [d...]. pred (t),δ pred (t)].

[0085] 3) Action Decision: The pre-trained reinforcement learning agent receives the state s(t) and makes a decision based on its currently learned policy. Output a control action a(t). This action defines the adjustment amount for one or more process parameters.

[0086] 4) Environment interaction and reward calculation: The environment (digital twin model) receives action a(t), updates the process parameters to q(t+1)=q(t)+a(t), and uses q(t+1) to perform forward inference again to predict the state s(t+1) at the next moment.

[0087] In this embodiment, in step S104, in some embodiments, exemplarily, the correction parameters include the adjustment amount ΔT of the reaction chamber temperature T and the adjustment amount Δt of the precursor pulse time. pulse Adjustment amount Δt for inert gas purging time purge .

[0088] The correction parameters are numerical adjustment quantities that need to be converted into control commands that the physical actuators can directly respond to. The core is to adapt to the actuator type and control logic. Based on the controlled object of the actuator, the correction parameters are mapped into specific operation commands and then transmitted to the actuator. The conversion of correction parameters into correction commands is shown in Table 1.

[0089] Table 1 Correction Parameters and Correction Commands

[0090]

[0091] After the instruction is generated, it is sent to the actuator using an industrial-grade communication interface, such as industrial Ethernet or RS485 serial port.

[0092] Before issuing a command, it must undergo a legality verification process to confirm whether the adjustment amount is within the actuator's safety threshold, such as ensuring that the heater power does not exceed the rated value and the pulse time is not less than the minimum safe duration, in order to avoid equipment damage.

[0093] After the actuator completes the parameter update, it returns a success / failure signal to ensure closed-loop controllability.

[0094] In this embodiment, a digital twin model of the atomic layer deposition (ALD) production system is constructed based on a multi-scale physical information neural network model. This model utilizes multi-scale PINN to explicitly decouple the macroscopic flow field and microscopic surface reaction, and couples them through physical boundary conditions. This ensures that the digital twin model possesses both high physical fidelity and excellent training stability, effectively overcoming the convergence problem caused by scale differences in traditional single models. Secondly, the trained digital twin model is used as a virtual training environment for the reinforcement learning agent, enabling the agent to autonomously learn complex nonlinear optimal control strategies in a low-cost, risk-free digital space, achieving adaptive optimization without explicitly solving inverse problems. Finally, this architecture places time-consuming high-fidelity simulation calculations in the offline training stage, enabling online decision-making speeds to reach sub-second levels. This not only significantly improves optimization efficiency and robustness and avoids the predicament of traditional algorithms easily getting trapped in local optima, but also forms a real-time intelligent closed-loop control of perception-prediction-decision-execution, thereby achieving autonomous optimization and high-quality production of the ALD process in the face of equipment aging and raw material fluctuations.

[0095] This invention also provides an atomic layer deposition process management system based on digital twins, including a physical module 201, a digital twin module 202, a decision control module 203, and an interaction module 204. The physical module 201 includes an ALD device, an integrated sensor network, and a programmable actuator. The integrated sensor network includes a chamber pressure sensor, a multi-point temperature sensor, and a residual gas analysis mass spectrometer. The programmable actuator is used to precisely control the precursor pulse valve, the purge gas valve, and the heater power. The digital twin module 202 includes a digital twin model, which is used to adjust the real-time reaction chamber process parameters and real-time state changes. The physical field distribution within the reaction chamber and the predicted growth state of the thin film on the substrate surface are obtained through quantity prediction. The decision control module 203 interacts with the digital twin model and performs decision analysis based on the physical field distribution within the reaction chamber and the predicted growth state of the thin film on the substrate surface obtained by the digital twin model to obtain correction parameters for optimizing the quality of thin film growth. The interaction module 204 communicates with the physical module 201, the digital twin module 202 and the decision control module 203 respectively to provide a human-computer interaction page and display the process parameters and real-time state variables of the reaction chamber, the physical field distribution within the reaction chamber and the predicted growth state of the thin film on the substrate surface and the correction parameters.

[0096] In this embodiment, through the deep integration and closed-loop linkage of the physical module 201, the digital twin module 202, the decision control module 203, and the interaction module 204, a complete intelligent system is constructed, from physical world perception to virtual space decision-making and then reverse control. First, by integrating a sensor network and a high-fidelity digital twin model, the system achieves accurate and real-time mapping and prediction of the macroscopic flow field and microscopic surface reaction process inside the reaction chamber, completely solving the inherent defects of slow calculation of traditional mechanism models and poor generalization of pure data models. Second, the decision control module 203, relying on a pre-trained reinforcement learning agent, can... Twin data autonomously and rapidly generates correction instructions for optimizing process parameters and issues adjustments in real time through programmable actuators. This transforms the traditional passive optimization that relies on offline trial and error into proactive control with online autonomous decision-making, greatly improving the efficiency and robustness of process control. Finally, the system achieves transparent visualization and interactive management of the entire process data through the interactive module 204. This not only provides operators with real-time insight into the process status and virtual metering capabilities, but also enables the atomic layer deposition process to adaptively respond to changes in equipment status and production disturbances, truly achieving the intelligent and autonomous goal of high-quality, high-uniformity thin film production.

[0097] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural transformations made using the contents of the present invention's specification and drawings under the inventive concept of the present invention, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. A method for managing atomic layer deposition processes based on digital twins, characterized in that, include: A digital twin model of an atomic layer deposition production system was constructed based on a multi-scale physical information neural network model. The real-time reaction chamber process parameters and real-time state variables are obtained and input into the digital twin model to obtain the physical field distribution in the reaction chamber and the predicted growth state of the thin film on the substrate surface. Based on the physical field distribution within the reaction chamber and the predicted growth state of the thin film on the substrate surface, a pre-trained reinforcement learning agent is used for decision analysis to obtain corrected parameters for optimizing the quality of thin film growth. Based on the correction parameters, correction instructions are generated, and the process parameters of the reaction chamber are updated based on the correction instructions, so that the thin film on the substrate surface in the reaction chamber reaches the target film thickness and uniformity. The process of predicting the growth state based on the physical field distribution within the reaction chamber and the thin film on the substrate surface, and then using a pre-trained reinforcement learning agent to perform decision analysis to obtain corrected parameters for optimizing the thin film growth quality, previously included: constructing an agent based on an Actor network and a Critic network. The digital twin model is instantiated as a reinforcement learning environment, and a state space, action space, and reward function are defined. The state space includes the predicted film thickness, predicted film uniformity, and real-time process parameters at the current moment. The action space includes the adjustment amount of controllable process parameters, and the reward function is designed. R ( t It is used to evaluate the effect of control actions and guide the agent to learn the optimal policy; it adopts a proximal policy optimization algorithm, generates experience tuples through the interaction between the agent and the reinforcement learning environment and stores them in the experience replay buffer, and samples the experience data to update the parameters of the Actor network and the Critic network until the agent converges, thus obtaining a pre-trained agent.

2. The atomic layer deposition process management method based on digital twin as described in claim 1, characterized in that, The digital twin model of the atomic layer deposition production system, constructed based on a multi-scale physical information neural network model, includes: A macroscopic subnetwork is constructed based on a fully connected neural network. The macroscopic subnetwork is used to solve fluid dynamics, mass transfer and heat transfer problems at the macroscopic reaction chamber scale. The physical constraints of the macroscopic subnetwork include mass conservation, momentum conservation, energy conservation and species transport equations. A micro-subnetwork is constructed based on a fully connected neural network. The micro-subnetwork is used to solve the chemical adsorption and thin film growth kinetics problems at the microscale on the substrate surface. The physical constraints of the micro-subnetwork include surface reaction kinetic equations. By coupling the macroscopic subnetwork and the microscopic subnetwork through physical boundary conditions, the predicted output of the macroscopic subnetwork is used as the input of the microscopic subnetwork, thus obtaining a multi-scale physical information neural network model. Acquire training data and use the training data to train a multi-scale physical information neural network model to obtain a digital twin model.

3. The atomic layer deposition process management method based on digital twin as described in claim 2, characterized in that, The total loss function of the digital twin model is: L total , L total Represented as: In the formula, L macro This represents the overall loss function of the macroscopic subnetwork. L micro This represents the overall loss function of the micro-subnetwork. L data, macro The mean squared error loss between macroscopic measurement data and network predictions. L bc, macro For initial and boundary condition constraint losses, L phys, macro This represents the residual loss of the macroscopic physical equations. L data, micro The mean square error loss between the microscopic measurement data and the network predictions. L bc, micro For initial and boundary condition constraint losses, The residual loss of the microscopic surface reaction kinetic equation, λ phys, macro , λ data, macro , λ bc, macro These are the hyperparameters for the physical loss term, data loss term, and initial and boundary condition loss term of the macroscopic subnetwork, respectively. λ phys, micro , λ data, micro , λ bc, micro These are the hyperparameters for the physical loss term, data loss term, and initial and boundary condition loss term of the micro-subnetwork, respectively.

4. The atomic layer deposition process management method based on digital twin as described in claim 1, characterized in that, L phys, macro and They are represented as follows: In the formula, F mc , F ns , F ec , F st These represent the residual operators for the mass conservation equation, momentum conservation equation, energy conservation equation, and species transport equation, respectively. n θ The parameter is Deep neural networks, {X macro, i } is a configuration point sampled within the macroscopic computational domain. N macro For the total number of configuration points, F sk The residual operator of the surface reaction kinetic equation, The parameter is Deep neural networks, {X micro, j } is the configuration point sampled within the microscopic surface domain. N micro This represents the total number of configuration points.

5. The atomic layer deposition process management method based on digital twin as described in claim 2, characterized in that, The process of acquiring training data and using the training data to train a multi-scale physical information neural network model to obtain a digital twin model specifically includes: The macroscopic subnetwork is trained using fluid dynamics simulation data combined with collected historical macroscopic physical field data, and the parameters of the macroscopic subnetwork are optimized by minimizing the loss function of the macroscopic subnetwork. Historical reaction chamber process parameters and historical state variables are input into the trained macroscopic sub-model to obtain surface thin film growth state prediction data. The micro-sub-network is trained by combining the surface thin film growth state data obtained from experimental measurements, and the parameters of the micro-sub-network are optimized by minimizing the loss function of the micro-sub-network. The parameters of the macroscopic subnetwork and the microscopic subnetwork are jointly fine-tuned, and the parameters of the macroscopic subnetwork and the microscopic subnetwork are optimized by minimizing the total loss function to obtain the digital twin model.

6. The atomic layer deposition process management method based on digital twin as described in claim 1, characterized in that, Define the reward function as follows R ( t ),but R ( t ) is represented as: In the formula, d pred ( t () represents the predicted value of the thin film on the substrate surface. d target Indicates the target film thickness. ξ d This represents the thickness convergence adjustment factor. δ th The minimum acceptable uniformity threshold, δ pred ( t ) indicates the predicted film uniformity. δ pred ( t+ 1) Represents the predicted film uniformity at the next moment. w 1 and w 2 is a weighting factor used to balance the importance of thickness accuracy and uniformity.

7. The atomic layer deposition process management method based on digital twin as described in claim 1, characterized in that, The correction parameters include adjustments to the reaction chamber temperature, the precursor pulse time, and the inert gas purging time.

8. A digital twin-based atomic layer deposition process management system, characterized in that, The digital twin-based atomic layer deposition process management system is used to implement the digital twin-based atomic layer deposition process management method as described in any one of claims 1-7. The digital twin-based atomic layer deposition process management system includes a physical module, a digital twin module, a decision control module, and an interaction module. The physical module includes an atomic layer deposition device, an integrated sensor network, and a programmable actuator. The integrated sensor network includes a chamber pressure sensor, a multi-point temperature sensor, and a residual gas analysis mass spectrometer. The programmable actuator is used to precisely control the precursor pulse valve, the purge gas valve, and the heater power. The digital twin module includes a digital twin model. The digital twin model is used to predict the physical field distribution within the reaction chamber and the predicted growth state of the thin film on the substrate surface based on real-time reaction chamber process parameters and real-time state variables. The decision control module interacts with the digital twin model and performs decision analysis based on the physical field distribution within the reaction chamber and the predicted growth state of the thin film on the substrate surface obtained by the digital twin model to obtain correction parameters for optimizing the quality of thin film growth. The interaction module communicates with the physical module, the digital twin module, and the decision control module respectively to provide a human-computer interaction page and display the reaction chamber process parameters and real-time state variables, the physical field distribution within the reaction chamber, the predicted growth state of the thin film on the substrate surface, and the correction parameters.

Images