Atomic layer deposition process management method and system based on digital twinning

Abstract

The invention is suitable for the field of atomic layer deposition, and discloses an atomic layer deposition process management method and system based on digital twinning, and the method comprises the steps: building a digital twinning model of an atomic layer deposition production system based on a multi-scale physical information neural network model; acquiring real-time reaction chamber process parameters and real-time state variables, and inputting the real-time reaction chamber process parameters and the real-time state variables into the digital twin model to obtain physical field distribution in the reaction chamber and a predicted growth state of a film on the surface of the substrate; based on the physical field distribution in the reaction cavity and the substrate surface film prediction growth state, decision analysis is carried out through a pre-trained reinforcement learning agent, and correction parameters used for optimizing the film growth quality are obtained; and generating a correction instruction based on the correction parameter, and updating the process parameter of the reaction cavity based on the correction instruction, so that the film on the surface of the substrate in the reaction cavity reaches the target film thickness and uniformity, and thus real-time intelligent closed-loop control of perception-prediction-decision-execution can be formed.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of atomic layer deposition, and in particular to an atomic layer deposition process management method and system based on digital twinning. BACKGROUND

[0002] Atomic layer deposition (ALD) is a key technology for achieving atomic-level precision thin film growth. Its process involves multiple scales and strong coupling of physical and chemical mechanisms, from fluid dynamics, mass and heat transfer in the macroscopic reaction chamber to chemical adsorption and reaction on the microscale substrate surface. Although digital twinning technology makes it possible to realize the intelligentization of ALD process, existing modeling methods still have significant bottlenecks. The physical mechanism model based on the coupling of computational fluid dynamics (CFD) and surface reaction kinetics, although high in physical fidelity, has a huge computational cost for solving high-dimensional nonlinear partial differential equations, making it difficult to meet the real-time and online optimization control requirements of digital twinning, and the inverse problem of the model is difficult to solve. Pure data-driven machine learning models have poor model generalization ability due to high measurement cost and data scarcity of ALD key quality parameters (such as film thickness and uniformity), and the prediction results in sparse data regions or extreme process conditions may even violate basic physical laws. Although Physics-Informed Neural Networks (PINN) can integrate data and physical constraints, integrating the multi-scale and strong coupling processes of ALD into a single PINN model often leads to unstable model training and convergence difficulties due to the large difference in residual magnitudes and the high complexity of equations for different physical processes (macroscopic flow field and microscopic reaction).

[0003] Therefore, the prior art still needs to be improved and developed. SUMMARY

[0004] A first object of the present application is to provide an atomic layer deposition process management method based on digital twinning, which aims to solve the technical problem that the process parameters of the existing atomic layer deposition system cannot be optimized and controlled in real time.

[0005] To achieve the above-mentioned object, the scheme provided by the present application is:

[0006] The application discloses an atomic layer deposition process management method based on digital twinning, and relates to the atomic layer deposition field.

[0007] Preferably, the digital twin model of the atomic layer deposition production system is constructed based on a multi-scale physical information neural network model, which comprises the following steps: a macroscopic sub-network is constructed based on a full connection neural network, the macroscopic sub-network is used to solve fluid dynamics, mass transfer and heat transfer problems at a macroscopic reaction cavity scale, and physical constraints of the macroscopic sub-network include mass conservation, momentum conservation, energy conservation and species transport equations; a microscopic sub-network is constructed based on a full connection neural network, the microscopic sub-network is used to solve chemical adsorption and thin film growth kinetics problems at a microscopic scale of a substrate surface, and physical constraints of the microscopic sub-network include surface reaction kinetics equations; the macroscopic sub-network and the microscopic sub-network are coupled through physical boundary conditions, so that the predicted output of the macroscopic sub-network is used as the input of the microscopic sub-network, and a multi-scale physical information neural network model is obtained; training data is obtained, and the multi-scale physical information neural network model is trained using the training data, so that the digital twin model is obtained.

[0008] Preferably, the total loss function of the digital twin model is L total , L total is expressed as:

[0009] L total =L macro +L micro

[0010] L macro =λ phys,macro L phys,macro +λ data,macro L data,macro +λ bc,macro L bc,macro

[0011] L micro =λ phys,micro L phys,micro +λ data,micro L data,micro +λ bc,micro L bc,micro

[0012] where L macro represents the overall loss function of the macroscopic subnetwork, L micro represents the overall loss function of the microscopic subnetwork, L data,macro is the mean square error loss between macroscopic measurement data and network predicted values, L bc,macro is the initial, boundary condition constraint loss, L phys,macro is the macroscopic physical equation residual loss, L data,micro is the mean square error loss between microscopic measurement data and network predicted values, L bc,micro is the initial, boundary condition constraint loss, L phys,micro is the microscopic surface reaction kinetics equation residual loss, λ phys,macro , λ data,macro , λ bc,macro are the hyperparameters of the physical loss term, data loss term and initial, boundary condition loss term of the macroscopic subnetwork, respectively. phys,micro , λ data,micro , λ bc,micro are the hyperparameters of the physical loss term, data loss term and initial, boundary condition loss term of the microscopic subnetwork, respectively.

[0013] Preferably, L phys,macro and L phys,micro are represented as follows, respectively:

[0014]

[0015] where F mc , F ns , F ec , F st represent the residual operators of the mass conservation equation, momentum conservation equation, energy conservation equation and species transport equation, respectively, n θ is a deep neural network with parameters θ, {X macro,i} are collocation points sampled in the macroscopic computational domain, N macro is the total number of collocation points, F sk is the residual operator of the surface reaction kinetics equation, m γ is a deep neural network with parameters y, {X micro,j} are collocation points sampled in the microscopic surface domain, N micro is the total number of collocation points.

[0016] Preferably, the training data is acquired, and the multi-scale physical information neural network model is trained using the training data to obtain the digital twin model, specifically comprising: training the macroscopic sub-network using fluid dynamics simulation data combined with collected historical macroscopic physical field data, and optimizing the parameters of the macroscopic sub-network by minimizing the loss function of the macroscopic sub-network; inputting the historical reaction cavity process parameters and historical state variables into the trained macroscopic scale sub-model to obtain surface film growth state prediction data, training the microscopic sub-network combined with the surface film growth state data measured by experiment, and optimizing the parameters of the microscopic sub-network by minimizing the loss function of the microscopic sub-network; jointly fine-tuning the parameters of the macroscopic sub-network and the parameters of the microscopic sub-network, and optimizing the parameters of the macroscopic sub-network and the microscopic sub-network by minimizing the total loss function to obtain the digital twin model.

[0017] Preferably, based on the physical field distribution in the reaction cavity and the predicted growth state of the surface film of the substrate, the pre-trained reinforcement learning agent is used for decision analysis to obtain the correction parameter for optimizing the film growth quality, and the previous method further comprises: constructing an agent based on an Actor network and a Critic network; instantiating the digital twin model as a reinforcement learning environment, and defining a state space, an action space and a reward function, the state space comprising the predicted film thickness, the predicted film uniformity and the real-time process parameters at the current time, the defined action space comprising the adjustment amount of the controllable process parameters, and the designed reward function R(t) being used for evaluating the effect of the control action and guiding the agent to learn the optimal strategy; using a proximal policy optimization algorithm, generating experience tuples through the interaction between the agent and the reinforcement learning environment and storing them into an experience replay buffer, and sampling experience data to update the parameters of the Actor network and the Critic network until the agent converges, thereby obtaining the pre-trained agent.

[0018] Preferably, the reward function is defined as R(t), and R(t) is expressed as:

[0019] R(t) = w1exp[-|d pred (t+1)-d target |ξ d ]-w2max(0,δ th -δ pred (t+1))

[0020] In the formula, d pred (t) represents the predicted value of the surface film of the substrate, d target represents the target film thickness, ξ d represents the thickness convergence adjustment factor, δ th is the acceptable minimum uniformity threshold, δ pred (t) represents the predicted film uniformity, and δ pred(t+1) denotes the predicted film uniformity at the next time, w1 and w2 are weight coefficients for balancing the importance of thickness accuracy and uniformity.

[0021] Preferably, the correction parameters include an adjustment amount ΔT of the reaction chamber temperature T, an adjustment amount Δt of the precursor pulse time pulse and an adjustment amount Δt of the inert gas purge time purge .

[0022] A second object of the present application is to provide a digital-twin-based atomic layer deposition process management system for implementing the digital-twin-based atomic layer deposition process management method as described above, the digital-twin-based atomic layer deposition process management system comprising a physical module, a digital-twin module, a decision control module and an interaction module, the physical module comprising an atomic layer deposition device, an integrated sensor network and a programmable actuator, the integrated sensor network comprising a chamber pressure sensor, a multi-point temperature sensor, a residual gas analysis mass spectrometer, the programmable actuator being used to accurately control the precursor pulse valve, the purge gas valve and the heater power, the digital-twin module comprising a digital-twin model, the digital-twin model being used to predict the physical field distribution in the reaction chamber and the predicted growth state of the film on the substrate surface according to the real-time reaction chamber process parameters and real-time state variables, the decision control module interacting with the digital-twin model and making decision analysis according to the physical field distribution in the reaction chamber and the predicted growth state of the film on the substrate surface predicted by the digital-twin model, to obtain correction parameters for optimizing the film growth quality, the interaction module communicating with the physical module, the digital-twin module and the decision control module respectively, for providing a human-computer interaction page and displaying the reaction chamber process parameters and real-time state variables, the physical field distribution in the reaction chamber and the predicted growth state of the film on the substrate surface and the correction parameters.

[0023] In the scheme, the digital twin model of the atomic layer deposition production system based on the multi-scale physical information neural network model utilizes the multi-scale PINN to explicitly decouple the macro flow field and the micro surface reaction, and is coupled through the physical boundary condition, ensuring that the digital twin model has high physical fidelity and excellent training stability, effectively overcoming the convergence problem caused by the scale difference of traditional single models; secondly, the trained digital twin model is used as a virtual training environment for the reinforcement learning agent, enabling the agent to learn complex nonlinear optimal control strategies in a low-cost and risk-free digital space, achieving adaptive optimization without explicitly solving inverse problems; finally, the architecture places time-consuming high-fidelity simulation calculations in the offline training stage, making online decision-making speed reach sub-second level, not only greatly improving optimization efficiency and robustness, avoiding the dilemma of traditional algorithms easily falling into local optimum, but also forming a real-time intelligent closed-loop control of perception-prediction-decision-execution, thereby realizing the autonomous optimization and high-quality production of the ALD process in the face of equipment aging and raw material fluctuations. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor based on the structures shown in the drawings.

[0025] Figure 1 is a flowchart of the atomic layer deposition process management method based on digital twinning provided by the embodiments of the present application;

[0026] Figure 2 is a structural block diagram of the atomic layer deposition process management system based on digital twinning provided by the embodiments of the present application.

[0027] BRIEF DESCRIPTION OF DRAWINGS

[0028] 201, entity module; 202, digital twin module; 203, decision control module; 204, interaction module. DETAILED DESCRIPTION

[0029] The terms "first", "second", "third", "fourth" and the like in the description and in the claims of the present application, and above-mentioned drawings, if any, are used to distinguish between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so-termed, data can be interchanged, where appropriate, so that the embodiments described herein can be carried out in orders other than those illustrated or described herein. Moreover, the terms "comprise" or "have" and any variations thereof, are intended to cover non-exclusive inclusions, for example, a process, method, system, product or apparatus comprising a list of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to such processes, methods, products or apparatuses.

[0030] For the sake of understanding, the specific flow of the embodiments of the present application is described below, please refer to Figure 1 In the embodiments of the present application, the atomic layer deposition process management method based on digital twinning includes:

[0031] S101, constructing a digital twin model of an atomic layer deposition production system based on a multi-scale physical information neural network model;

[0032] S102, acquiring real-time reaction cavity process parameters and real-time state variables, and inputting the real-time reaction cavity process parameters and real-time state variables into the digital twin model to obtain the physical field distribution in the reaction cavity and the predicted growth state of the substrate surface film;

[0033] S103, based on the physical field distribution in the reaction cavity and the predicted growth state of the substrate surface film, decision analysis is performed through a pre-trained reinforcement learning agent to obtain correction parameters for optimizing the film growth quality;

[0034] S104, generating a correction instruction based on the correction parameters, and updating the reaction cavity process parameters based on the correction instruction, so that the substrate surface film in the reaction cavity reaches the target film thickness and uniformity.

[0035] In this embodiment, step S101, in some embodiments, exemplarily, involves constructing a digital twin model of an atomic layer deposition production system based on a multi-scale physical information neural network model, including: constructing a macroscopic sub-network based on a fully connected neural network, where the macroscopic sub-network is used to solve fluid dynamics, mass transfer, and heat transfer problems at the macroscopic reaction chamber scale, and the physical constraints of the macroscopic sub-network include mass conservation, momentum conservation, energy conservation, and species transport equations; constructing a microscopic sub-network based on a fully connected neural network, where the microscopic sub-network is used to solve chemical adsorption and thin film growth kinetics problems at the microscopic scale of the substrate surface, and the physical constraints of the microscopic sub-network include surface reaction kinetic equations; coupling the macroscopic sub-network and the microscopic sub-network through physical boundary conditions, so that the predicted output of the macroscopic sub-network is used as the input of the microscopic sub-network, thereby obtaining a multi-scale physical information neural network model; acquiring training data and using the training data to train the multi-scale physical information neural network model, thereby obtaining a digital twin model. By constructing two sub-networks, one macroscopic and one microscopic, and embedding mass, momentum, and energy conservation equations and surface reaction kinetic equations as physical constraints, this method innovatively achieves effective decoupling and accurate modeling of multi-scale, strongly coupled physicochemical mechanisms in atomic layer deposition. Furthermore, by using physical boundary conditions to apply the predicted output of the macroscopic sub-network as the input of the microscopic sub-network, the physical consistency of cross-scale information transmission and the self-consistency of the model are ensured. Combined with a staged training strategy, this method can efficiently integrate CFD simulation data and sparse experimental data, significantly reducing the dependence on expensive measurement data. The resulting digital twin model not only possesses high physical fidelity and excellent data fitting ability but also exhibits outstanding generalization performance and stability, laying a solid foundation for achieving real-time high-precision prediction and intelligent optimization control of the process.

[0036] It should be noted that the species transport equation is one of the core equations in fields such as fluid mechanics, chemical engineering, and environmental science. It is used to describe the changes in the concentration of a certain species (such as molecules, ions, or particles) in a multi-component mixture over time and space. Essentially, it is a concrete manifestation of the "law of conservation of mass" in the context of multi-component flow. By quantifying the three core processes of "convection, diffusion, and generation / consumption," it reveals the evolutionary mechanism of species concentration.

[0037] In this embodiment, the overall loss function of the macroscopic subnetwork is L. macro Then L macro Represented as:

[0038] L macro =λ phys,macro L phys,macro +λ data,macro L data,macro +λ bc,macro L bc,macro

[0039] In the formula, L data,macro L represents the mean square error loss between macroscopic measurement data and network predictions. bc,macro For the initial and boundary condition constraint loss, L phys,macro λ represents the residual loss of the macroscopic physical equations. phys,macro , λ data , macro , λ bc,macro These are the hyperparameters for the Macro-PINN physical loss term, data loss term, and initial and boundary condition loss terms, respectively.

[0040] In this embodiment, the overall loss function of the micro-subnetwork is L. micro Then L micro Represented as:

[0041] L micro =λ phys,micro L phys,micro +λ data,micro L data,micro +λ bc,micro L bc,micro

[0042] In the formula, L data,micro L represents the mean square error loss between the microscopic measurement data and the network predictions. bc,micro For the initial and boundary condition constraint loss, L phys,micro λ represents the residual loss of the microscopic surface reaction kinetic equation. phys , micro , λ data,micro , λ bc,micro These are the hyperparameters for the Micro-PINN physical loss term, data loss term, and initial and boundary condition loss term, respectively.

[0043] The total loss function of the digital twin model is L total L total Represented as:

[0044] L total =L macro +L micro .

[0045] In this embodiment, in the thermal ALD growth process of alumina (Al2O3) thin films, the atomic layer deposition system (ALD system) uses trimethylaluminum (TMA) and deionized water (H2O) as precursors, and nitrogen (N2) as purging and carrier gas. One growth cycle of ALD includes four steps: First, a TMA pulse enters the reaction chamber and reacts with surface hydroxyl groups to generate methane and surface-bound aluminum species; second, an inert gas purging is performed to remove the TMA precursor and gas-phase reaction byproducts; then, a water vapor pulse is introduced to react with the aluminum species to complete the growth of the alumina layer and regenerate the surface hydroxyl groups; finally, an inert gas is introduced again to remove the H2O precursor and gas-phase reaction byproducts.

[0046] In this embodiment, training data is acquired and used to train a multi-scale physical information neural network model to obtain a digital twin model. Specifically, this includes: training the macroscopic sub-network using fluid dynamics simulation data combined with collected historical macroscopic physical field data, and optimizing the parameters of the macroscopic sub-network by minimizing its loss function; inputting historical reaction chamber process parameters and historical state variables into the trained macroscopic sub-model to obtain surface thin film growth state prediction data, and training the microscopic sub-network using experimentally measured surface thin film growth state data, and optimizing the parameters of the microscopic sub-network by minimizing its loss function; jointly fine-tuning the parameters of the macroscopic and microscopic sub-networks, and optimizing the parameters of both the macroscopic and microscopic sub-networks by minimizing the total loss function to obtain the digital twin model.

[0047] Specifically, the input vector of the macro-PINN subnetwork is X. macro = [x,y,z,t,q], where (x,y,z) are three-dimensional spatial coordinates, t is time, and q is a process parameter vector, which includes chamber temperature T, pressure P, and precursor pulse time t. pulse Inert gas purging time t purge etc.; the output is the macroscopic physical field Φ. macro (X macro )=[c(x,y,z,t),u(x,y,z,t),T(x,y,z,t)], where c is the precursor concentration and u is the flow field velocity vector. The physical constraints of Macro-PINN consist of the mass conservation equation, momentum conservation equation, energy conservation equation, and species transport equation describing continuous fluids, with the residual loss function L of the corresponding partial differential equation system. phys,macro Defined as:

[0048]

[0049] In the formula, F mc F ns F ecF st Representing the residual operators of the mass conservation equation, momentum conservation equation (i.e., the Navier-Stokes equation), energy conservation equation, and species transport equation, respectively, n θ It is a deep neural network with parameter θ, {X macro,i} is the location point sampled within the macroscopic computational domain, N macro This represents the total number of configuration points.

[0050] Furthermore, the loss function L of Macro-PINN macro It also includes data loss L data,macro and initial and boundary condition loss L bc,macro Data loss L data,macro The mean square error between the neural network predictions and the actual measurements, and the initial and boundary condition loss L. bc,macro The mean square error between the neural network output value and the true initial and boundary condition values ​​is...

[0051] L macro =λ phys,macro L phys,macro +λ data,macro L data,macro +λ bc,macro L bc,macro

[0052] In the formula, λ phys,macro , λ data,macro , λ bc,macro These are the hyperparameters for the Macro-PINN physical loss term, data loss term, and initial and boundary condition loss term, respectively, used to balance the contributions of each loss term.

[0053] Understandably, the initial and boundary condition loss L bc,macro This is the mean squared error between the neural network output value and the actual initial and boundary condition values. It can be set according to the actual situation. It can be the mean squared error between the neural network output value (predicted initial condition value) and the actual initial condition value, or the mean squared error between the neural network output value (predicted boundary condition value) and the actual boundary condition value, or it can be a weighted sum of the mean squared errors between the neural network output value (predicted initial condition value) and the actual initial condition value and the mean squared errors between the neural network output value (predicted boundary condition value) and the actual boundary condition value.

[0054] Specifically, the Micro-PINN subnetwork focuses on solving the chemisorption and thin film growth kinetics at the microscale on the substrate surface. Its input vector is X. micro =[x s ,y s ,t,q,c s (x s ,ys ,t),T s (x s ,y s ,t)], where (x s ,y s ) represents the coordinates of the substrate surface, c s (x s ,y s ,t)T s (x s ,y s ,t) represent the positions (x,t) of the Macro-PINN on the substrate surface. s ,y s The predicted concentration and temperature at the specified location are crucial for multi-scale coupling; the output is the predicted growth state Ψ of the thin film on the substrate surface. micro (X micro )=[d(x s ,y s ,t),β(x s ,y s ,t)], where d is the film thickness and β is the precursor surface coverage. The physical constraints of Micro-PINN are constituted by the surface reaction kinetic equation, and its residual loss function is defined as:

[0055]

[0056] In the formula, F sk For the residual operator of the surface reaction kinetic equation, m γ It is a deep neural network with parameter y, {X micro,j} is the configuration point sampled within the microscopic surface domain, N micro This represents the total number of configuration points.

[0057] Similarly, the loss function L of Micro-PINN micro It also includes data loss L data,micro Initial and boundary condition loss L bc,micro Two items, namely

[0058] L micro =λ phys,micro L phys,micro +λ data,micro L data,micro +λ bc,micro L bc,micro

[0059] In the formula, λ phys,micro , λ data,micro , λ bc,micro These are the hyperparameters for the Micro-PINN physical loss term, data loss term, and initial and boundary condition loss term, respectively.

[0060] The two sub-networks are coupled through physical boundary conditions, namely the output c(x) of Macro-PINN on the substrate surface. s ,y s ,t) and T(x s ,y s The input (t) is directly used as the input to the Micro-PINN. This coupling ensures the physical driving effect of the macroscopic flow field on the microscopic surface processes. The total loss function L of the multi-scale PINN is... total The sum of the loss functions of the two subnetworks:

[0061] L total =L macro +L micro .

[0062] In this embodiment, in step S102, in some embodiments, exemplaryly, in the thermal ALD growth of alumina (Al2O3) thin film, the reaction chamber process parameters include: reaction chamber temperature T, pressure P, and TMA pulse time t. pulse1 H2O pulse time t pulse2 Inert gas purging time t purge .

[0063] The state variables were obtained as follows: the instantaneous concentrations of TMA and H2O were obtained by a residual gas analysis mass spectrometer in the reaction chamber, and the film thickness d and uniformity δ were intermittently measured by an integrated in-situ ellipsometry.

[0064] In this embodiment, the physical field distribution within the reaction chamber is the precursor concentration distribution c(x,y,z,t), flow velocity vector distribution u(x,y,z,t), and temperature distribution T(x,y,z,t) in three-dimensional space output by the macroscopic sub-network.

[0065] The predicted growth state of the thin film on the substrate surface is the predicted film thickness d(x) at different locations on the substrate surface output by the micro-subnetwork. s ,y s ,t), Predicted value of precursor surface coverage β(x) s ,y s ,t), Predicted value of film thickness uniformity δ pred (t).

[0066] Furthermore, Macro-PINN is a 9-layer fully connected neural network, including an input layer, 7 hidden layers, and an output layer. The input parameters have a dimension of 9, including spatial coordinates (x, y, z), time t, and the aforementioned 5 process parameters q = [T, P, t]. pulse1 ,t pulse2 ,t purgeThe output parameters have four dimensions: TMA concentration c1, H2O concentration c2, flow rate u, and temperature T. Each hidden layer contains 386 neurons and uses the Swish activation function.

[0067] The macroscopic subnetwork is constrained by the conservation of mass, momentum, energy, and the species transport equation; the corresponding residual operator can be expressed as:

[0068]

[0069] In the formula, ρ is the gas density, μ is the gas viscosity, Γ is the external volume force, and C p κ is the specific heat capacity, D is the thermal conductivity. i Ω i Let represent the diffusion coefficient of the i gaseous components in the reaction chamber and the net flux from the surface chemical reaction or mass transfer boundary, respectively.

[0070] Substituting the above parameters into the overall loss function L of the macroscopic subnetwork macro .

[0071] Micro-PINN is a 7-layer fully connected neural network, including an input layer, 5 hidden layers, and an output layer. The input parameters have a 10-dimensional dimension, including the substrate surface coordinates (x, y, φ). s ,y s ), time t, process parameter q = [T, P, t pul se1 ,t pulse2 ,t purge ] and the TMA concentration c1(x) at the substrate surface provided by Macro-PINN prediction. s ,y s ,t) and H2O concentration c2(x s ,y s The output parameters have a dimension of 2, namely the film thickness d and the surface coverage β. Each hidden layer also contains 386 neurons and uses the Swish activation function.

[0072] Micro-PINN is constrained by surface reaction kinetics. Taking the classic condensation-hydrolysis reaction mechanism of Al2O3 AlD as an example, its physical residual can be expressed as:

[0073]

[0074] In the formula, α1 and α2 represent the reaction rates of the TMA and H2O half-reactions, respectively, exhibiting self-limiting characteristics. The film growth rate is proportional to α1, and β... OH The thickness d is the coverage of surface hydroxyl groups, and is obtained by integrating the growth rate over time.

[0075] Substituting the above parameters into the overall loss function L of the micro-subnetwork micro .

[0076] In this embodiment, in step S103, in some embodiments, exemplarily, based on the physical field distribution within the reaction chamber and the predicted growth state of the thin film on the substrate surface, a pre-trained reinforcement learning agent performs decision analysis to obtain correction parameters for optimizing the thin film growth quality. Prior to this, the method further includes: constructing an agent based on an Actor network and a Critic network; instantiating the digital twin model into a reinforcement learning environment, and defining the state space s(t), action space a(t), and reward function R(t). The state space s(t) includes the predicted thin film thickness d at the current moment. pred (t), Predicted film uniformity δ pred Given the controllable process parameters q(t) and q(t), the action space a(t) is defined, which includes the adjustment amount of the controllable process parameters. A reward function R(t) is designed to evaluate the effect of the control action and guide the agent to learn the optimal policy. The proximal policy optimization algorithm is adopted to generate experience tuples through the interaction between the agent and the reinforcement learning environment and store them in the experience replay buffer. The experience data is sampled to update the parameters of the Actor network and the Critic network until the agent converges, thus obtaining the pre-trained agent.

[0077] In this embodiment, a proximal policy optimization algorithm is used to train the agent's optimization strategy by having it interact with the environment, thereby maximizing long-term cumulative rewards. Using a reward function, after approximately 1000 iterations of training, the agent learns a strategy to stably approach the target thickness and maintain high uniformity by coordinating pulse / purge times and fine-tuning the temperature. Coordinating the pulse / purge times ensures sufficient reaction and removal of byproducts, while fine-tuning the temperature regulates the reaction rate.

[0078] Specifically, when the buffer accumulates enough data or reaches the update cycle, the policy updater samples a batch of empirical data from the buffer, uses the near-end policy optimization algorithm to calculate the policy gradient, and updates the parameters of the Actor network and the Critic network so that they can obtain higher cumulative rewards in future decisions.

[0079] In this embodiment, the reward function R(t) is expressed as:

[0080] R(t) = w1exp[-|d pred (t+1)-d target |ξ d -w2max(0,δ th -δ pred (t+1))

[0081] In the formula, d pred(t) represents the predicted value of the thin film on the substrate surface, d target ξ represents the target film thickness. d δ represents the thickness convergence adjustment factor. th To meet the minimum acceptable uniformity threshold, δ pred (t) represents the predicted film uniformity, δ pred (t+1) represents the predicted film uniformity at the next time step, and w1 and w2 are weighting coefficients used to balance the importance of thickness accuracy and uniformity. The first term of the reward function R(t) uses an exponential decay function to encourage the agent to quickly and smoothly approach the target thickness; the second term is a uniformity penalty term, which applies a negative reward when the uniformity is below a threshold, forcing the agent to ensure high quality at the same time.

[0082] In this embodiment, the real-time dynamic process optimization process based on reinforcement learning in the thermal ALD growth of alumina (Al2O3) thin films is as follows:

[0083] 1) Initialization: Set initial process parameters q0 and target thickness d. target Uniformity threshold δ th And reinforcement learning hyperparameters.

[0084] 2) State Observation: At the beginning of each control cycle, the system acquires the current process parameter q(t) and sensor data from the physical entity layer. This data is input into the digital twin model for forward inference to obtain the predicted growth state of the thin film on the substrate surface at the current moment: s(t) = [d...]. pred (t),δ pred (t)].

[0085] 3) Action Decision: The pre-trained reinforcement learning agent receives the state s(t) and makes a decision based on its currently learned policy. Output a control action a(t). This action defines the adjustment amount for one or more process parameters.

[0086] 4) Environment interaction and reward calculation: The environment (digital twin model) receives action a(t), updates the process parameters to q(t+1)=q(t)+a(t), and uses q(t+1) to perform forward inference again to predict the state s(t+1) at the next moment.

[0087] In this embodiment, in step S104, in some embodiments, exemplarily, the correction parameters include the adjustment amount ΔT of the reaction chamber temperature T and the adjustment amount Δt of the precursor pulse time. pulse Adjustment amount Δt for inert gas purging time purge .

[0088] The correction parameters are numerical adjustment quantities that need to be converted into control commands that the physical actuators can directly respond to. The core is to adapt to the actuator type and control logic. Based on the controlled object of the actuator, the correction parameters are mapped into specific operation commands and then transmitted to the actuator. The conversion of correction parameters into correction commands is shown in Table 1.

[0089] Table 1 Correction Parameters and Correction Commands

[0090]

[0091] After the instruction is generated, it is sent to the actuator using an industrial-grade communication interface, such as industrial Ethernet or RS485 serial port.

[0092] Before issuing a command, it must undergo a legality verification process to confirm whether the adjustment amount is within the actuator's safety threshold, such as ensuring that the heater power does not exceed the rated value and the pulse time is not less than the minimum safe duration, in order to avoid equipment damage.

[0093] After the actuator completes the parameter update, it returns a success / failure signal to ensure closed-loop controllability.

[0094] In this embodiment, a digital twin model of the atomic layer deposition (ALD) production system is constructed based on a multi-scale physical information neural network model. This model utilizes multi-scale PINN to explicitly decouple the macroscopic flow field and microscopic surface reaction, and couples them through physical boundary conditions. This ensures that the digital twin model possesses both high physical fidelity and excellent training stability, effectively overcoming the convergence problem caused by scale differences in traditional single models. Secondly, the trained digital twin model is used as a virtual training environment for the reinforcement learning agent, enabling the agent to autonomously learn complex nonlinear optimal control strategies in a low-cost, risk-free digital space, achieving adaptive optimization without explicitly solving inverse problems. Finally, this architecture places time-consuming high-fidelity simulation calculations in the offline training stage, enabling online decision-making speeds to reach sub-second levels. This not only significantly improves optimization efficiency and robustness and avoids the predicament of traditional algorithms easily getting trapped in local optima, but also forms a real-time intelligent closed-loop control of perception-prediction-decision-execution, thereby achieving autonomous optimization and high-quality production of the ALD process in the face of equipment aging and raw material fluctuations.

[0095] This invention also provides an atomic layer deposition process management system based on digital twins, including a physical module 201, a digital twin module 202, a decision control module 203, and an interaction module 204. The physical module 201 includes an ALD device, an integrated sensor network, and a programmable actuator. The integrated sensor network includes a chamber pressure sensor, a multi-point temperature sensor, and a residual gas analysis mass spectrometer. The programmable actuator is used to precisely control the precursor pulse valve, the purge gas valve, and the heater power. The digital twin module 202 includes a digital twin model, which is used to adjust the real-time reaction chamber process parameters and real-time state changes. The physical field distribution within the reaction chamber and the predicted growth state of the thin film on the substrate surface are obtained through quantity prediction. The decision control module 203 interacts with the digital twin model and performs decision analysis based on the physical field distribution within the reaction chamber and the predicted growth state of the thin film on the substrate surface obtained by the digital twin model to obtain correction parameters for optimizing the quality of thin film growth. The interaction module 204 communicates with the physical module 201, the digital twin module 202 and the decision control module 203 respectively to provide a human-computer interaction page and display the process parameters and real-time state variables of the reaction chamber, the physical field distribution within the reaction chamber and the predicted growth state of the thin film on the substrate surface and the correction parameters.

[0096] In this embodiment, through the deep integration and closed-loop linkage of the physical module 201, the digital twin module 202, the decision control module 203, and the interaction module 204, a complete intelligent system is constructed, from physical world perception to virtual space decision-making and then reverse control. First, by integrating a sensor network and a high-fidelity digital twin model, the system achieves accurate and real-time mapping and prediction of the macroscopic flow field and microscopic surface reaction process inside the reaction chamber, completely solving the inherent defects of slow calculation of traditional mechanism models and poor generalization of pure data models. Second, the decision control module 203, relying on a pre-trained reinforcement learning agent, can... Twin data autonomously and rapidly generates correction instructions for optimizing process parameters and issues adjustments in real time through programmable actuators. This transforms the traditional passive optimization that relies on offline trial and error into proactive control with online autonomous decision-making, greatly improving the efficiency and robustness of process control. Finally, the system achieves transparent visualization and interactive management of the entire process data through the interactive module 204. This not only provides operators with real-time insight into the process status and virtual metering capabilities, but also enables the atomic layer deposition process to adaptively respond to changes in equipment status and production disturbances, truly achieving the intelligent and autonomous goal of high-quality, high-uniformity thin film production.

[0097] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural transformations made using the contents of the present invention's specification and drawings under the inventive concept of the present invention, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. A digital-twin-based atomic layer deposition process management method, characterized by, The application relates to an atomic layer deposition production system digital twin model based on a multi-scale physical information neural network model. Real-time reaction cavity process parameters and real-time state variables are acquired, and the real-time reaction cavity process parameters and the real-time state variables are input into the digital twin model to obtain a physical field distribution in the reaction cavity and a predicted growth state of a substrate surface film. Based on the physical field distribution in the reaction cavity and the predicted growth state of the substrate surface film, a pre-trained reinforcement learning agent is used for decision analysis to obtain correction parameters for optimizing film growth quality. Based on the correction parameters, correction instructions are generated, and the reaction cavity process parameters are updated based on the correction instructions, so that the substrate surface film in the reaction cavity reaches a target film thickness and uniformity. The atomic layer deposition production system digital twin model based on the multi-scale physical information neural network model comprises the following steps:

2. The digital-twin-based atomic layer deposition process management method of claim 1, wherein, A macroscopic subnetwork is constructed based on a full connection neural network, the macroscopic subnetwork is used to solve fluid dynamics, mass transfer and heat transfer problems at a macroscopic reaction cavity scale, and physical constraints of the macroscopic subnetwork include mass conservation, momentum conservation, energy conservation and species transport equations; A microscopic subnetwork is constructed based on a full connection neural network, the microscopic subnetwork is used to solve chemical adsorption and film growth kinetics problems at a microscopic scale of a substrate surface, and physical constraints of the microscopic subnetwork include surface reaction kinetics equations; The macroscopic subnetwork and the microscopic subnetwork are coupled through physical boundary conditions, so that the predicted output of the macroscopic subnetwork is used as the input of the microscopic subnetwork, and a multi-scale physical information neural network model is obtained; Training data are acquired, and the multi-scale physical information neural network model is trained using the training data to obtain a digital twin model. The acquisition of the training data and the training of the multi-scale physical information neural network model using the training data to obtain the digital twin model specifically comprises the following steps:

3. The digital-twin-based atomic layer deposition process management method of claim 2, wherein, The total loss function of the digital twin model is L total , L total is expressed as: L total = L macro + L micro L macro = λ phys,macro L phys,macro + λ data,macro L data,macro + λ bc,macro L bc,macro L micro = λ phys,micro L phys,micro + λ data,micro L data,micro + λ bc,micro L bc,micro wherein L macro represents the overall loss function of the macroscopic subnetwork, L micro represents the overall loss function of the microscopic subnetwork, L data,macro is the mean square error loss between macroscopic measurement data and network predicted values, L bc,macro is the initial, boundary condition constraint loss, L phys,macro is the macroscopic physical equation residual loss, L data,micro is the mean square error loss between microscopic measurement data and network predicted values, L bc,micro is the initial, boundary condition constraint loss, L phys,micro is the microscopic surface reaction kinetics equation residual loss, λ phys,macro , λ data,macro , λ bc,macro are the hyperparameters of the physical loss term, data loss term and initial, boundary condition loss term of the macroscopic subnetwork, respectively. λ phys,micro , λ data,micro , λ bc,micro are the hyperparameters of the physical loss term, data loss term and initial, boundary condition loss term of the microscopic subnetwork, respectively.

4. The digital-twin-based atomic layer deposition process management method of claim 1, wherein, L phys,macro and L phys,micro are represented by: where F mc , F ns , F ec , and F st represent the residual operators of the mass conservation equation, the momentum conservation equation, the energy conservation equation, and the species transport equation, respectively, n θ is a deep neural network with parameters θ, {X macro,i} are the collocation points sampled in the macroscopic computational domain, N macro is the total number of collocation points, F sk is the residual operator of the surface reaction kinetics equation, m y is a deep neural network with parameters y, {X micro,j} are the collocation points sampled in the microscopic surface domain, N micro is the total number of collocation points. 5.The digital-twin-based atomic layer deposition process management method of claim 2, wherein, The macroscopic subnetwork is trained using fluid dynamics simulation data combined with collected historical macroscopic physical field data, and parameters of the macroscopic subnetwork are optimized by minimizing a loss function of the macroscopic subnetwork; Historical reaction cavity process parameters and historical state variables are input into the trained macroscopic scale submodel to obtain surface film growth state prediction data, the microscopic subnetwork is trained in combination with surface film growth state data measured through experiments, and parameters of the microscopic subnetwork are optimized by minimizing a loss function of the microscopic subnetwork; The parameters of the macroscopic subnetwork and the parameters of the microscopic subnetwork are jointly fine-tuned, and the parameters of the macroscopic subnetwork and the microscopic subnetwork are optimized by minimizing a total loss function, so that a digital twin model is obtained. The atomic layer deposition production system digital twin model based on the multi-scale physical information neural network model comprises the following steps:

6. The digital-twin-based atomic layer deposition process management method of claim 1, wherein, An agent is constructed based on an Actor network and a Critic network. ​ The digital twin model is instantiated as a reinforcement learning environment, and a state space, an action space and a reward function are defined, the state space includes a predicted film thickness, a predicted film uniformity and real-time process parameters at a current time, the defined action space includes an adjustment amount of a controllable process parameter, and the reward function R(t) is designed to evaluate the effect of the control action and guide the agent to learn an optimal strategy; A proximal policy optimization algorithm is used to generate experience tuples through interaction between the agent and the reinforcement learning environment and store them in an experience replay buffer, and experience data is sampled to update parameters of an actor network and parameters of a critic network until the agent converges, thereby obtaining a pre-trained agent.

7. The digital-twin-based atomic layer deposition process management method of claim 6, wherein, The reward function is defined as R(t), and R(t) is expressed as: R(t) = w1 exp[-|d pred (t+1)-d target | / ξ d ]-w2 max(0,δ th -δ pred (t+1)) where d pred (t) represents the predicted film thickness on the substrate surface, d target represents the target film thickness, ξ d represents the thickness convergence adjustment factor, δ th is the acceptable minimum uniformity threshold, δ pred (t) represents the predicted film uniformity, δ pred (t+1) represents the predicted film uniformity at the next time, w1 and w2 are weighting factors used to balance the importance of thickness accuracy and uniformity.

8. The digital-twin-based atomic layer deposition process management method of claim 1, wherein, The correction parameters include an adjustment amount of a reaction chamber temperature, an adjustment amount of a precursor pulse time and an adjustment amount of an inert gas purge time.

9. A digital-twin-based atomic layer deposition process management system, characterized by, The digital twin-based atomic layer deposition process management system is used to implement the digital twin-based atomic layer deposition process management method according to any one of claims 1-8, and the digital twin-based atomic layer deposition process management system includes an entity module, a digital twin module, a decision control module and an interaction module, the entity module includes an atomic layer deposition device, an integrated sensor network and a programmable actuator, the integrated sensor network includes a chamber pressure sensor, a multi-point temperature sensor and a residual gas analysis mass spectrometer, the programmable actuator is used to accurately control a precursor pulse valve, a purge gas valve and a heater power, the digital twin module includes a digital twin model, the digital twin model is used to predict a reaction chamber physical field distribution and a substrate surface film predicted growth state according to real-time reaction chamber process parameters and real-time state variables, the decision control module interacts with the digital twin model, and decision analysis is performed according to the reaction chamber physical field distribution and the substrate surface film predicted growth state predicted by the digital twin model, thereby obtaining correction parameters for optimizing film growth quality, and the interaction module communicates with the entity module, the digital twin module and the decision control module respectively, and is used to provide a man-machine interaction page and display reaction chamber process parameters and real-time state variables, reaction chamber physical field distribution and substrate surface film predicted growth state and correction parameters.

Images