AI drive control module for real-time process optimization in semiconductor process system

Abstract

The invention discloses a control module for a semiconductor process system. The control module utilizes a reinforcement learning (RL) algorithm to autonomously generate and adjust a process formula. The control module has comprehensive system digital twinning, including subsystem digital twinning, chamber plasma digital twinning and process digital twinning, and employs a neural network model to improve efficiency. By using a policy neural network and Monte Carlo Tree Search (MCTS), the present invention can make real-time adjustments according to calibration state data from various sensors, thereby improving the precision and adaptability of a manufacturing process.

Description

Cross Reference to Related Applications

[0001] This application claims priority to U.S. Patent Application No. 18 / 775,892, filed July 17, 2024. TECHNICAL FIELD

[0002] The present application relates to semiconductor process systems, and in particular, to a control module and an Artificial Intelligence (AI) machine that autonomously generates and adjusts process recipes using Reinforcement Learning (RL) algorithms. The present application is applicable to various semiconductor manufacturing processes including etching and deposition, and utilizes advanced AI techniques, digital twinning, and real-time monitoring to improve the precision, efficiency, and adaptability of these processes. BACKGROUND

[0003] In semiconductor manufacturing, precise control of process recipes is crucial to ensure that semiconductor devices meet the required quality and performance. Traditional process control methods often rely on pre-set process recipes and non-automatic adjustments, which not only consume time but also may not well adapt to changes in manufacturing environments or substrate conditions. As semiconductor devices become increasingly complex, there is an increasing demand for more advanced and flexible process control methods.

[0004] Digital twinning has become a powerful tool for simulating and optimizing various aspects of semiconductor manufacturing processes. By creating a virtual copy of the physical system, digital twinning enables detailed modeling and analysis of process dynamics, allowing for more accurate predictions and control adjustments. However, the computational complexity of traditional digital twinning models is relatively high, which can be a limiting factor, especially in real-time applications.

[0005] RL and AI techniques provide a promising solution for automation and process control optimization in semiconductor manufacturing processes. RL algorithms, through continuous learning and adaptation, can more effectively optimize process recipes than static process recipe settings. By combining AI with digital twinning, it is possible to develop an intelligent control system that can autonomously generate and adjust process recipes in real-time based on the current state of the system and real-time data from various sensors.

[0006] Despite the above possibilities, there is still a need for an efficient and reliable control system to cope with the complexity of the process and be able to adapt to changing conditions in real time. The present invention solves the above problems by utilizing a full system digital twin, including subsystem digital twins, chamber plasma and process digital twins, and using a more computationally efficient version of the neural network for the digital twin. In addition, the present invention uses a policy neural network and Monte Carlo tree search (MCTS) as part of the RL algorithm to autonomously generate and adjust process recipes during substrate processing, and to use calibrated state data from various sensors for precise real-time control. SUMMARY

[0007] The present invention relates to a control module and an AI machine designed to autonomously generate and adjust process recipes in a semiconductor process system using RL algorithms. The present invention uses advanced AI technology and real-time monitoring technology to improve the accuracy and efficiency of semiconductor processes.

[0008] The control module includes a subsystem controller, an AI engine, and a system controller. The subsystem controller manages the operational activities of various subsystems within the semiconductor process system. The AI engine trains a policy neural network using RL algorithms, including MCTS, to autonomously generate process recipes. The system controller uses the trained policy neural network and data provided by the RT monitor to adjust process recipes in real time during substrate processing.

[0009] The present invention uses a comprehensive system digital twin, which includes subsystem digital twins (RF, gas, temperature, etc.), chamber plasma digital twin, and process digital twin. This digital twin approach is computationally efficient, and using a neural network version of the digital twin can achieve faster processing speed and higher accuracy. In addition, chamber surface aging digital twin and edge ring digital twin are crucial for real-time control, as they simulate the effects of plasma exposure over time, providing the necessary inputs to the policy neural network to maintain optimal process conditions.

[0010] In some embodiments, the AI engine is part of a cloud-based AI machine and is connected to the system controller through a communication link. The AI engine controller within the AI engine uses an RL engine to generate process recipes based on the system digital twin. The policy neural network includes an input layer, a hidden layer, and an output layer, and uses softmax and / or logistic functions to describe the probability distribution of selected process recipe parameters at a discretized level.

[0011] The RT monitor uses various sensors to measure parameters of the RF, gas, and temperature subsystems, and uses optical sensors to collect plasma light radiation, and optical reflectometry sensors for determination of structure parameters during etching processes. The system controller adjusts the process recipe based on the difference between the calculated state and the calibrated state, and uses real-time data from these sensors to achieve precise control.

[0012] The real-time control method includes training a policy neural network using an AI engine, transmitting the trained network to a system controller, updating a digital twin, receiving a substrate specification, generating an initial state and process recipe, executing an action, calculating a state after action execution, calibrating the state using real-time data, and repeating these steps until a final state is reached. This continuous adjustment ensures process optimization and high precision in semiconductor manufacturing.

[0013] The present application ensures high precision, efficiency, and adaptability of manufacturing processes by integrating AI-driven autonomous recipe generation and real-time adjustment, which has important progress significance in the field of semiconductor processing. The combination of chamber surface aging digital twin and edge ring digital twin helps to make real-time adjustments according to the wear of components, thereby improving the accuracy of process control. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 is a schematic diagram of an exemplary process system of the present application.

[0015] Figure 2A is a schematic diagram of AI machine functions connected to multiple system controllers of a process system.

[0016] Figure 2B is a detailed functional module diagram of the AI engine.

[0017] Figure 3 is a schematic diagram of a system digital twin.

[0018] Figure 4 is a neural network representation diagram of a system digital twin.

[0019] Figure 5 is a process flow example using atomic layer etching (ALE) process, which is mapped to an RL process to automatically generate a process recipe.

[0020] Figure 6 is a schematic diagram of a policy neural network structure in an RL process.

[0021] Figure 7 is a schematic diagram of an exemplary RL algorithm that uses the MCTS procedure to autonomously train a policy neural network.

[0022] Figure 8A flowchart for training policy neural networks autonomously and generating process recipes through RL.

[0023] Figure 9A A flowchart for real-time control of a process system through dynamic adjustment of process recipes.

[0024] Figure 9B A schematic diagram for state calibration neural networks.

[0025] Figure 10 Table 1 summarizes the design parameters describing the structure and topology of the sub-systems.

[0026] Figure 11 Table 2 summarizes the parameters of the structure features before and after ALE process.

[0027] Figure 12 Table 3 shows the selected ALE process recipe parameters, which are discretized into multiple levels suitable for implementing RL. DETAILED DESCRIPTION

[0028] In order to provide a comprehensive understanding, the embodiments of the present application are described in detail below. Although specific details are set forth to provide a thorough understanding, modifications and variations are considered acceptable as long as they are within the scope of the present application. These detailed descriptions highlight the innovative features of the present application, which distinguish it from the prior art.

[0029] Figure 1 An embodiment of a process system 100 is shown. The system is suitable for a plasma enhanced etching or deposition process, such as for a reactive ion etching (RIE), atomic layer etching (ALE), plasma-enhanced chemical vapor deposition (PECVD), or atomic layer deposition (ALD). In some embodiments, the process system 100 can be converted to a thermal process system by removing the plasma generation related sub-systems. The inventive concept is generic and can be applied to any type of semiconductor process system, and the illustrated vacuum chamber plasma process system is merely an example and not intended to limit the scope of the present application.

[0030] The process system 100 also includes a control module 102. As shown, various components of the control module 102 are shown in dashed boxes. It should be understood that some of the components of the control module 102 can not be located locally at the process system 100, but deployed in the cloud. Figure 1

[0031] ​The process system 100 includes a plasma process chamber 104 configured to maintain a vacuum environment suitable for plasma processing. The process system 100 is also provided with a plasma source 106 that receives RF power from an RF power generator 108 via a resonator 110. The plasma source 106 can be implemented in a variety of configurations, such as an inductively coupled plasma (ICP) source or a transformer coupled plasma (TCP) source, among others.

[0032] The RF power generator 108 can operate at a single frequency or multiple frequencies (e.g., 13.56 MHz, 2.0 MHz, and 40 MHz). The resonator 110 is used to match the output impedance of the RF power generator 108 to the impedance of the plasma process chamber 104 according to the transmission line impedance characteristics. The resonator 110 typically includes inductors and capacitors, and can be configured with a mechanically adjustable capacitor. In some embodiments, the resonator 110 does not include a mechanically adjustable capacitor.

[0033] Impedance matching can be achieved by varying the operating frequency of the RF power generator 108 and the resonator 110. During processing, the plasma can exhibit variable states of impedance characteristics. To maintain efficient energy transfer and minimize the power reflected from the plasma process chamber 104 back to the resonator 110, the frequency is fine-tuned for each state of the plasma to ensure that the resonator 110 remains in a resonant state.

[0034] The plasma process chamber 104 is also configured with a chuck 112 that supports a substrate 114. Depending on the process requirements, the chuck 112 can be designed as an electrostatic chuck (ESC) or a vacuum chuck. When an ESC is used, the chuck 112 is electrically connected to an RF power generator 116 via a resonator 118. Similar to the resonator 110, the resonator 118 needs to be tuned to be in a resonant state by adjusting the operating frequency. The RF power generator 116 can operate at a different frequency than the RF power generator 108. For example, the RF power generator 116 can operate at a substantially lower frequency than the RF power generator 108.

[0035] RF power generator 116 provides a bias to the chuck 112. The bias is introduced through a blocking capacitor, although not shown, which is a conventional arrangement in the art. Additionally, a custom waveform generator 117 can also be used to provide a bias to the chuck 112. The custom waveform can significantly narrow the ion energy distribution of the plasma 128 generated by the plasma process chamber 104. Depending on the particular implementation, the custom waveform generator 117 can be connected to the chuck 112 alone or in conjunction with the RF power generator 116 and the resonator 118 to provide the desired bias.

[0036] The RF subsystem includes RF power generators, resonators, and plasma sources. The operation of the RF subsystem is managed by an RF controller 134. The RF controller 134 communicates with and is subordinate to the system controller 132. Both the RF controller 134 and the system controller 132 are components of the control module 102.

[0037] The plasma process chamber 104 is configured with a gas distribution unit 122 for delivering process gases from a gas source 120 into the plasma process chamber 104. The gas distribution unit 122 can take many forms, such as a gas injector or a showerhead, and can include lateral injection structures near the interior walls of the chamber. The gas source 120 typically obtains gases from a facility gas source through a gas box and in conjunction with valves, pressure regulators, and mass flow controllers (MFCs) to regulate the flow of gases into the chamber. In other embodiments, a precursor delivery system (not shown) can also be used to deliver gaseous, liquid, or even solid precursors.

[0038] Additionally, the plasma process chamber 104 is provided with a pump 124 (which can be a turbomolecular pump or other suitable type) for evacuating gases and byproducts from the plasma process chamber 104. A valve 126 is typically provided at the top of the pump 124 to regulate the rate of chamber evacuation. Chamber pressure is monitored by a pressure gauge (not shown) that triggers adjustments to the setpoint of the actuator of the valve 126 to maintain a suitable pressure for the vacuum process.

[0039] The gas distribution subsystem (also referred to as the gas subsystem) includes the gas distribution unit 122, the gas source 120, the pump 124, and the valve 126, and is monitored by a gas controller 136. The gas controller 136 is connected to the overall system controller 132 to ensure integrated management of the process system 100. The gas controller 136 and the system controller 132 are both components of the control module 102.

[0040] The plasma process chamber 104 is also equipped with a temperature control subsystem (also referred to as the temperature subsystem) for maintaining the thermal conditions required for the substrate and the chamber. In Figure 1In the illustrated embodiment, the temperature of the chuck 112 is regulated by a temperature controller 138, which controls the heater 128, the chiller 130, and temperature sensors (not shown). The chuck 112 can be designed to have multiple zones, each of which is maintained at a different temperature. In addition, temperature control of other components within the plasma processing chamber 104, such as the gas distribution unit 122 and various chamber surfaces, can be implemented in a manner conventional in the industry. The temperature subsystem is controlled by the temperature controller 138, which is connected to the system controller 132; both the system controller 132 and the temperature controller 138 are components of the control module 102.

[0041] The system controller 132 is connected to the AI engine 140. The AI engine 140 is designed to autonomously train a policy neural network and generate a process recipe based on system digital twin and RL, etc. learning algorithms. Generating a process recipe requires a large amount of computing resources; therefore, the AI engine 140 can be deployed in the cloud as part of the AI machine 200 (FIG. 2). The system controller 132 is set locally to the plasma processing chamber 104. The system controller 132 is connected to a real-time (RT) state corrector 142 and a trained policy neural network 144. The system controller 132 receives the trained policy neural network from the AI engine 140 and utilizes the trained neural network 144 to generate a process recipe according to the input parameters and output specifications of the substrate to be processed. The policy neural network 144 receives additional input parameters of the chamber inner surfaces and edge rings. The characteristics of the chamber inner surfaces and edge rings can be related to the plasma processing history and preventive maintenance (PM) procedures. The process recipe includes a plurality of consecutive actions, each of which causes the system to transition to a new state. The state is used to describe the structural features of the substrate, while the action represents a process step in the process recipe. Subsequently, the system controller 132 initiates the processing of the substrate by controlling the plasma processing chamber 104. The real-time (RT) state corrector 142 receives real-time measurements from the RT monitor 148 and calibrates the computed state at the current time to a state consistent with the measurements. The system controller 132 evaluates the difference between the computed state and the calibrated state and utilizes the trained policy neural network 144 to decide whether the process recipe needs to be adjusted. In some embodiments, the RT state corrector 142 and the trained policy neural network 144 can be implemented by a software program executed by the system controller 132. In other embodiments, the RT state corrector 142 and the trained policy neural network 144 can be implemented by firmware or hardware; in further embodiments, they can also be implemented by a combination of software, firmware, and hardware. For example, the trained policy neural network 144 can be implemented in the form of hardware of a neural network. The weights of the policy neural network 144 can be transferred by the AI engine 140. In one embodiment, the trained policy neural network 144 can be implemented as an analog computing unit.

[0042] The RT monitor 148 includes sensors for measuring RF, gas, and temperature subsystem states and performance. The RT monitor 148 can also include optical emission spectroscopy sensors for characterizing neutral particles in the plasma; further, it can also include optical reflectometry sensors for directly measuring changes in the structure of the substrate being processed.

[0043] Figure 2AAn embodiment of an AI machine 200 is illustrated. In one embodiment, the AI ​​machine is a dedicated AI computer optimized with advanced hardware and software modules. The hardware modules include advanced chips such as a Graphics Processing Unit (GPU) 240 and High Bandwidth Memory (HBM) 242. These components are integrated using advanced packaging techniques to achieve the ultra-high bandwidth required for AI applications. The software modules further include a Compute Unified Device Architecture (CUDA) 244. These hardware and software modules enable the AI ​​machine 200 to perform efficient parallel computing processing, such as algorithms for Reinforcement Learning (RL).

[0044] AI machine 200 also includes AI engine 140, which is capable of autonomous operation to train a policy neural network for generating process recipes. AI engine 140 further includes AI engine controller 202 for controlling the operation of the AI ​​engine. AI engine controller 202 can be implemented using GPU 240, HBM 242, and CUDA 244. AI engine 140 further includes RL engine 206, responsible for autonomously generating process recipes using system digital twin 204 and RL, which can replicate the operation of the process system in a virtual environment. System digital twin 204 includes various subsystem digital twins 208.

[0045] The AI ​​engine 140 in the AI ​​machine 200 can serve multiple process systems. For example... Figure 2A As shown, AI engine 140 can be exemplarily connected to system controllers 132A, 132B, and 132C via communication links 146A, 146B, and 146C. The communication links can take various forms, including but not limited to fiber optic, wireless, and wired communication channels known in the art.

[0046] Figure 2B The functional modules of AI engine 140 are shown in more detail. System digital twin 204 includes an RF digital twin 212 for simulating the operation of the RF subsystem, a gas digital twin 214 for simulating the gas subsystem, and a temperature digital twin 216 for simulating the temperature subsystem. System digital twin 204 further includes a chamber plasma digital twin 218, a surface flux digital twin 220, and a process digital twin 222.

[0047] The surface condition or dimensions of some chamber components can be affected by long-term exposure to harsh plasma. The surface condition can be improved by performing a PM procedure, such as surface cleaning. It is therefore important to capture these effects by incorporating the chamber surface aging digital twin 240 and the edge ring digital twin 242. Details of these subsystem digital twins will be discussed in the following sections.

[0048] The RL engine 206 also includes an RL agent 224, which is typically a software program stored in a storage medium of the AI engine controller 202 for performing the RL process. The RL agent 224 uses a policy neural network 226 and a MCTS procedure 228 to build a search tree and learns by evaluating the relationship between actions and rewards. A reward calculator 230 calculates a reward for each completed simulation case using the system digital twin 204.

[0049] Figure 3 A flowchart of the system digital twin 204 is schematically shown. The RF digital twin 212, the gas digital twin 214, and the temperature digital twin 216 take as inputs the relevant process recipe parameters as well as the design parameters of the subsystems and the system. The RF digital twin 212 is used to simulate the RF subsystem, which includes at least the RF power generator and the resonator. In some embodiments, the RF subsystem can further include a custom waveform generator for biasing, although the custom waveform generator typically does not operate in the RF range. In one embodiment, the RF digital twin 212 includes SPICE (Simulation Program with Integrated Circuit Emphasis) models for the RF circuitry, which determines the RF power injected into the plasma source in one time step. Subsequently, a Maxwell equation solver is employed to calculate the electromagnetic field distribution inside the chamber based on the chamber structure parameters.

[0050] The RF digital twin 212 receives the recipe parameters for this step (e.g., RF power and initial operating frequency). The set of system and subsystem design parameters, including the RF circuit topology, numerical values of the components, structure and parameters of the plasma source, and chamber structure parameters, are typically stored in a storage medium of the AI engine controller 202. Table 1 lists a set of exemplary design parameters for the RF subsystem. The RF digital twin 212 can be used to determine the resonant frequency of the RF subsystem. In another embodiment, multiple RF digital twins can be used. For example, the plasma source and the chuck bias can be modeled by different RF digital twins.

[0051] Similarly, the gas digital twin 214 simulates the functionality of the gas subsystem, including components such as the gas sources 120, the gas distribution unit 122, the pump 124, the valves 126, and the pressure gauges (not shown).

[0052] The gas digital twin 214 receives process recipe parameters, such as flow rates of process gases. For example, for an ALE process, the gas digital twin 214 receives the flow rates of the first and second process gases, respectively, as well as the chamber pressure for the surface modification step and the sputtering step. The design parameters of the gas delivery system include those of the gas distribution unit, which are exemplarily listed in Table 1. If a showerhead is employed, the design parameters would include its size, volume, distribution of gas injection channels / holes, and size of the gas injection channels / holes. The shape and size of the plasma process chamber are also important input parameters for the gas digital twin 214. The output of the gas digital twin 214 includes the three-dimensional (3D) gas distribution (e.g., density, partial pressure, velocity, and residence time) in the gas distribution unit 122 and the plasma process chamber 104. In some embodiments, the gas distribution in the gas line from the gas source 120 to the inlet of the gas distribution unit 122 is also modeled. The gas distribution can be simulated using a fluid dynamics-based approach, possibly with finite element techniques or other advanced computational techniques.

[0053] The temperature digital twin 216 models the temperature subsystem, which includes the heater 128, the chiller 130, and the temperature sensors (not shown). In addition to the chuck temperature control, the temperature digital twin 216 can also include the temperature regulation function of other chamber components, such as the gas distribution unit 122.

[0054] The temperature digital twin 216 receives process recipe parameters, such as the chuck temperature. In some implementations, the chuck 112 can be divided into multiple zones, each with a temperature set by the process recipe. The input parameters of the temperature digital twin 216 also include the design parameters of the heater and the chiller, as exemplarily listed in Table 1. For the heater 128, the design parameters include its location within the chuck or other chamber components, as well as its operating power range; further including the thermal conductivities of various materials and their interfaces. For the chiller, the design parameters can include the type of coolant, the flow rate of the coolant, and the number and location of the conduction channels. The temperature digital twin can apply numerical simulation methods, such as finite element method, to simulate the temperature distribution of the chuck, the substrate surface, and the inner surface of the plasma process chamber.

[0055] It is noted that treating the digital twins 212, 214, and 216 as independent objects can oversimplify the actual situation. For example, the RF power delivered to the chamber can affect the temperature of the substrate surface. Certain interactions between the digital twins of different subsystems should be carefully considered.

[0056] The dimensions of the chamber interior surfaces and certain components can change over time as they are exposed to plasma. Chamber surface aging digital twin 240 is used to model such“memory” effects of selected chamber surfaces, such as the surfaces of a window, a gas injector, or a showerhead. Input parameters include the surface material, the accumulated ion and neutral particle exposure, and the process history generated by the PM process. The PM process history plays an important role in the surface condition due to the effects of the cleaning procedures. Its output includes a set of surface parameters, such as the surface structure, composition, roughness, and adhesion coefficient. These parameters collectively have an impact on the distribution of neutral particles and ions in the chamber.

[0057] The plasma process chamber 104 includes some consumable components whose dimensions can decrease over time as they are exposed to plasma. Some of these changes can have a significant impact on process performance. For example, in modern etch chambers, an edge ring is often placed along the edge of the substrate being processed to improve plasma and temperature uniformity. The decrease in edge ring thickness after exposure to plasma for some time can significantly change the process performance at the edge of the substrate. Edge ring digital twin 242 is used to model such effects. The inputs to the edge ring digital twin 242 include the edge ring material and its structural parameters, such as the initial height of the edge ring. The input parameters also include the history of the edge ring exposure to ions and neutral particles in the plasma. The process history of the PM can also be a factor. The output of the edge ring digital twin 242 includes the height of the edge ring. In some implementations (not shown in the figure), the temperature and potential of the edge ring can be included as input parameters to determine the erosion rate of the edge ring.

[0058] The outputs of the subsystem digital twins are input into the chamber plasma digital twin 218. At a particular time step of the process, the chamber plasma digital twin 218 models the plasma within the chamber 104 and outputs the three-dimensional distribution of electrons, ions, and neutral particles. The distribution at a particular time is a function of the electromagnetic field, gas, and temperature at that time, as well as the distribution of electrons, ions, and neutral particles at that time. Therefore, the distribution of electrons, ions, and neutral particles needs to be determined in a looped manner. As shown in FIG. 2B, the output of the chamber plasma digital twin at the current time step can be input into the same digital twin at the next time step. The predetermined time step for each simulation event is set by the AI engine controller 202. Figure 3

[0059] ​After obtaining the three-dimensional distribution of ions and neutrals, the surface flux digital twin 220 calculates and outputs the ion flux and the neutral flux towards the substrate surface. In addition, the surface flux digital twin 220 can output the substrate surface temperature in coordination with the temperature digital twin 216. The plasma sheath above the substrate is crucial for determining the ion flux, which in turn greatly influences the etching behavior. The formation mechanism of the plasma sheath is well known in the art and can be accurately modeled using the chamber plasma digital twin 218.

[0060] The output of the surface flux digital twin 220 is inputted into the process digital twin 222 to simulate the process in the plasma process chamber 104. The updated substrate parameters or substrate state are inputted into the process digital twin 222. The process digital twin 222 utilizes the current state of the substrate parameters to determine its output.

[0061] Figure 3 The illustrated flow represents the process state of the plasma process chamber 104 at a certain time step. Therefore, the output of the process digital twin embodies the change of the structure in that time step.

[0062] In each time step, the cumulative ion and neutral flux should be calculated. The details of the distribution of ions and neutrals are crucial for the process in the plasma process chamber 104. For ions, their energy and angular distribution in that time step is crucial and varies depending on their location on the substrate surface. The output of the surface flux digital twin 220 should contain these key details. Similarly, for neutrals, their density, thermal energy, and activation energy are also important parameters for the substrate surface when it is undergoing a process.

[0063] It is noted that the design of the subsystem digital twin, the chamber plasma digital twin, and the process digital twin are exemplary embodiments. There can be multiple variations of the implementation strategy. In some embodiments, the chamber plasma digital twin and the surface flux digital twin can be combined into a single digital twin. In other embodiments, the surface flux digital twin can be combined with the process digital twin. In addition, the RF subsystem digital twin can be split into multiple digital twins, each representing a plasma source and a bias unit. Similarly, the temperature digital twin can be split into two or more digital twins, each dedicated to at least the chuck and the gas distribution unit. All the above variations are obvious equivalent variations within the inventive concept of the present disclosure and should be included in the scope of protection of the present disclosure.

[0064] The digital twin implemented by the neural network can follow the strategy of dividing the process system into multiple subsystems.

[0065] Figure 4An example process system is shown as system neural network 400. In this embodiment, the subsystem digital twin is reconstructed using various neural networks. The RF digital twin model 212 serves as the basis for training the RF neural network 402. Taking the plasma source 106 connected to the RF power generator 108 and resonator 110 as an example, a SPICE model can be first constructed to simulate the RF power generator 108 and resonator 110, including transmission line effects. The SPICE model outputs the initial AC current and voltage of the plasma source 106 coil, which requires an assumption of the initial impedance of the plasma 128. After that, a numerical simulator applies Maxwell’s equations to predict the electromagnetic field distribution inside the plasma process chamber 104.

[0066] The large amount of simulation data generated by the RF digital twin 212 will become the training set for the RF neural network 402. The inputs to the RF neural network 402 include the RF circuit topology and related parameters, such as the values of inductors, capacitors, resistors, and transistors inside the generator and resonator, as well as the detailed modeling of effects and transmission lines.

[0067] In addition, the RF neural network 402 also takes into account the chamber structure parameters, including the size specifications, the locations of the chuck and gas distribution units, and the material properties of these components, as shown in the example of Table 1.

[0068] Some parameters are measurable, so they have a greater weight in the training process of the RF neural network 402. For example, sensors can track the changes in current and voltage in the plasma source coil, or the reflected power at the resonator output node. Multiple small coil B-dot sensors can be placed inside the chamber to map the magnetic field distribution. The information collected from these sensors not only provides information support for the training process, but also ensures that the RF neural network 402 is highly consistent with actual observations.

[0069] The bias portion of the RF subsystem is modeled using a neural network, focusing on the initial electric field generated by the applied RF power. Unlike the magnetic field associated with plasma generation, the bias portion deals with the electric field that affects the surface of the substrate.

[0070] With respect to gas dynamics within the process system 100, a gas distribution neural network 404 is required, which is trained based on the gas digital twin 214. Numerical algorithms based on fluid dynamics are the basis for determining the gas distribution within the chamber 104. This complex interaction involves the inflow of gas from the gas distribution unit 122, as well as the outflow of gas controlled by the pump 124 and the valves 126, which is influenced by the chamber conductivity and volume parameters. While numerical simulations can provide high accuracy, they are computationally expensive and time-limited, making them less efficient for real-time applications. Therefore, the gas distribution neural network 404 is established.

[0071] The gas distribution neural network 404 is trained with simulation data reflecting various parameters, including the type and flow rate of the gas, the design of the gas distribution unit 122, the capacity of the pump 124, the set values of the valve 126 actuators, and the chamber size and conductivity. Some of the design parameters are listed in Table 1. The gas distribution unit 122 can be implemented as an injector, a showerhead, or a combination of both, and will affect the gas distribution within the process chamber 104. The size, number, and distribution of the internal channels / pores of the injector and the showerhead are important design parameters. The gas pressure within the process chamber is monitored by pressure gauges, and these measurement data enhance the training of the gas distribution neural network 404, typically with higher weights than simulation data, to ensure that the model is consistent with the actual situation.

[0072] At the same time, a temperature neural network 406 is also constructed based on the temperature digital twin 216, which is dedicated to mapping the thermal distribution within the plasma process chamber, especially on the substrate surface. The training of the temperature neural network 406 is derived from numerical models that simulate the thermal interaction and distribution. The inputs to the temperature neural network 406 include chuck and chamber parameters that affect heat generation and heat conduction. The thermal characteristics and heat conduction efficiency of the ESC, which can be influenced by the helium gas pressure as a medium, are crucial. Other chamber specifications, such as size and structural materials, also affect the model. Temperature readings from sensors within the chuck 112 and the process chamber 104 provide valuable actual data, and since these data directly measure the physical environment, they can have higher weights than simulation data when training the temperature neural network 406. The balance between simulation data and measurement data ensures that various neural networks can highly simulate the actual process, thereby achieving accurate prediction of the process system.

[0073] The chamber surface aging neural network 414 can be trained with data generated by the chamber surface aging digital twin 240. In addition, measurement data of specific chamber materials or surfaces can be generated using specially designed test equipment and used for training. The neural network 414 can simulate the digital twin 240, thereby significantly improving computational efficiency.

[0074] The edge ring neural network 416 can be trained with synthetic data generated by the edge ring digital twin 242. The erosion rate of the edge ring can be determined by measuring the height reduction as a function of plasma exposure time. The measured data can then be used to improve the accuracy of the training.

[0075] Figure 4 The complexity of the system neural network 400 is illustrated, where the outputs of the subsystem neural networks serve as inputs to the chamber plasma neural network 408. The chamber plasma digital twin 218 is the basis of the chamber plasma neural network 408, enabling a fine modeling of the plasma within the etch chamber.

[0076] To simulate the motion of particles within the plasma, a Monte Carlo or numerical plasma simulator can be used to visualize the three-dimensional distribution of electrons, ions, and neutral particles. This is crucial because electrons, which are significantly lighter in mass, move faster than ions, resulting in the formation of a sheath layer at the inner surfaces of the chamber. This sheath layer plays a critical role in the acceleration of ions towards the substrate, a process that is essential in sputtering but can have adverse effects in surface modification processes.

[0077] The training of the chamber plasma neural network 408 integrates simulated data to achieve faster computation speed and higher efficiency. However, to improve its predictive capabilities, the chamber plasma neural network 408 can also incorporate measured data collected from in-chamber sensors such as optical emission spectrometers and hairpin sensors that measure electron density. These measured data can be given a higher weight than the simulated data to ensure that the outputs of the plasma neural network 408 are as realistic as possible.

[0078] The dynamic nature of the plasma environment is captured by the recurrent neural network (RNN) design of the chamber plasma neural network 408. This means that it is capable of handling time-series data, taking snapshots of the plasma state at specific time points, and integrating this information into the model for subsequent predictions. This is a continuous cycle where the previous outputs of the neural network become part of the input data for the next time step, thereby simulating the continuous evolution of the plasma state.

[0079] Once the chamber plasma neural network 408 computes the three-dimensional distribution, the ion and neutral particle fluxes reaching the substrate surface can be determined based on the surface flux neural network 410. Subsequently, the ion and neutral particle fluxes, along with the substrate surface temperature, serve as inputs to the process neural network 412. The process neural network 412 can be trained based on data generated by the process digital twin. The outputs of the process neural network 412 also include the trends in the changes to the structures in the substrate.

[0080] Ultimately, the chamber plasma neural network 408 and the surface flux neural network 410 not only produce valuable flux outputs, but also work in concert with the temperature neural network 406 to provide critical information about the surface temperature. The accumulated flux over a time step should also contain important information about the ion energy and angular distribution, as well as the thermal and activation energy of the neutral particles. These parameters are critical to fine-tune the process within the plasma process chamber to achieve the desired etch precision and substrate surface quality.

[0081] Notably, Figure 4 An embodiment 400 of the full neural network of the system digital twin 204 is shown. In other embodiments or implementations, certain functional blocks can not be implemented in the form of a neural network. For example, the surface flux neural network 410 can be an analytical model. Thus, the embodiment 400 is merely an example. A variety of variants are possible for the selected building blocks of the system digital twin 204 by combining models, lookup tables, analytical models, numerical models, and Monte Carlo models. All of these variants are within the scope of the inventive concept.

[0082] The system and method for autonomously generating process recipes by applying RL algorithms are illustrated herein using an ALE process as an example. Figure 5 An ALE process flow 500 suitable for implementing the RL algorithms is shown. The exemplary ALE process alternates between a surface modification step A and a sputtering step B in a cyclical manner. It is noted that steps A and B here are commonly referred to as half cycles of the ALE process, which are different from the time steps described earlier for simulating the plasma behavior within the chamber. These time steps are significantly shorter than steps A and B of the ALE process.

[0083] In the surface modification step A, the surface of the substrate 114 is chemically modified using chemically active neutral particles formed in a plasma generated by a plasma source driven by an RF power generator. To this end, a halogen gas such as chlorine is typically introduced to generate the neutral particles. In this surface modification step, the bias of the chuck is typically set to zero to minimize the impact of ions on the substrate to maintain the integrity of the ALE process.

[0084] In contrast, in the sputtering step B, an inert gas such as argon is introduced to generate high-energy ions to physically remove the chemically modified layer from the substrate by sputtering. At this time, a bias is typically applied to the chuck by the RF power generator and resonator.

[0085] Between these steps, a purge step can be employed to transition the gas from step A (508) to step B (510), or vice versa, without mixing the two process gases. Figure 5 The purge step is not shown in the ALE process flow 500. Figure 5Step A(a) represents Step A at node a. Similarly, Step B(a) represents Step B at node a.

[0086] In some cases, especially when etching high aspect ratio structures, an additional deposition Step C (512) can be selectively added outside of Step A and Step B. Step C is strategically inserted into the ALE cycle sequence but at a lower frequency than Steps A and B, and its main role is to protect the sidewalls of the etched structure from possible lateral etching due to the angular distribution of the ions. Step C(b) represents Step C at node b.

[0087] The ALE process cycles, each cycle containing Step A and Step B. As Figure 5 illustrated, the ALE cycle starts with one state and ends with another state. State 502 describes the substrate being processed. State a represents the state at node a. Specifically, in the ALE process, the state describes one or more structures. The state description includes but is not limited to parameters describing the structure being etched, such as depth, critical dimension, profile, and loading, as exemplarily shown in Table 2. State 502 is associated with node 504. Thus, state a is associated with node a. The ALE cycle starts with one node with a state, performs one action 506 by selecting process recipe parameters using the policy neural network 226 and the MCTS procedure 228, and ends with another node with an updated state. In Figure 5 Action (a) represents the action triggered by the ALE recipe at node a.

[0088] It is noted that one node can lead to multiple nodes through different actions. If the recipe parameters are continuous, then the available new nodes will be infinite. Conversely, if the recipe parameters are discretized to a finite level, then the available new nodes will be finite.

[0089] Figure 5 In some embodiments, an ALE cycle is taken as one action (for example only). In other embodiments, half a cycle can be taken to separate nodes. At this time, the action can be a surface modification Step A, a sputtering Step B, or even a deposition Step C. All such variants shall fall within the protection scope of the present invention.

[0090] Figure 6An example policy neural network 226 is shown. The policy neural network 226 includes an input layer 602 for receiving as inputs the state of the current node and the required output specification. For real-time control, it is important to include as inputs to the policy neural network 226 those chamber parameters that vary with the exposure time of the chamber in the plasma. In the present disclosure, the outputs of the chamber surface aging digital twin 240 and the edge ring digital twin 242 that describe the real-time state of the chamber inner surface, and the edge ring height are included as inputs to the policy neural network 226. Thus, it can be effectively used to predict actions based on the state. It should be noted that the output specification here is the final output requirement after the completion of the entire process flow, not one step in the flow. Including the output specification as one of the inputs to the policy neural network 226 makes it more versatile and able to cope with changes in the output specification. In other implementations, the inputs include only the state.

[0091] The policy neural network 226 also includes one or more hidden layers 604 for processing the data received from the input layer 602. The policy neural network 226 also includes an output layer that can include multiple parts, each of which also includes multiple parameters that describe a Softmax or Logistic function. Figure 6 The parts of the output layer are schematically labeled as 606, 608, and 610, each of which outputs a probability distribution for a multi-level discretized process parameter. In addition, the output layer also includes a value predictor 612 for predicting the value of the state based on the current policy represented by the policy neural network 226 and its current weights. When using the policy neural network 226 to design a process system, the discretization levels for a recipe parameter should include the limits defined by the parameter range. The inherent capability of the process system needs to be evaluated based on the limits.

[0092] Figure 6 An example of a policy neural network designed for an ALE process is shown, in which three recipe parameters are selected. The first part 606 outputs a probability distribution for the 4 levels (D1, D2, D3, and D4) of the duration of step A, where P(D1), P(D2), P(D3), and P(D4) are the probabilities for each level, respectively. The probability distribution for the 4 levels has 4 output parameters, which can be described by a softmax function, from which the probabilities can be calculated accordingly. Similarly, the second part 608 outputs a probability distribution for the 3 levels of the chuck bias for step B. The third part 610 provides a probability distribution for the 2 possibilities of including or excluding step C after step B in the ALE cycle. The two-level probability distribution can be represented by a logistic function. The example ALE recipe parameters used in this implementation are shown in Table 3, and the example input parameters are shown in Table 2.

[0093] It is important to note that different sets of ALE formulation parameters can be selected, and each parameter can be selected at different levels. The parameter selection may differ if the process system 100 is used for different types of processes (e.g., deposition). The examples in this document are for illustrative purposes only and should not be considered as limiting the inventive concept. Furthermore, the selection of formulation parameters and levels can be dynamic. This means that the formulation parameters and levels can be adjusted during the execution of the RL algorithm. In one embodiment, after executing a predetermined number of simulation cases, the RL agent 224 may decide to narrow the parameter space and adjust the range and level of parameters to accelerate the convergence of the RL algorithm. In some embodiments, old parameters may be deleted and new parameters added. In other embodiments, the entire set of formulation parameters can be selected and determined by executing the RL algorithm. The parameter range is related to the capabilities and capacity of the subsystem and is stored in the storage medium of the AI ​​engine controller 202 of the AI ​​machine 200.

[0094] Figure 7 The diagram illustrates a network 700 generated by unfolding the RL process using the MCTS algorithm. (Example:) Figure 7 As shown, nodes (such as 702) are represented by circles. Each node is associated with a state, such as S. a1 A parent node can spawn multiple child nodes after performing an operation such as a 704 error. For example, a node in state S... a1 The node can be accessed through action A a1-b1 Transition to state S b1 The nodes. The RL agent 224 manages the selection process through a policy neural network 226 and an MCTS program 228. For the ALE process, each action exemplarily represents an ALE cycle with selected process recipe parameters, or half a cycle can be selected. Action selection continues until a termination state is reached that satisfies the calculation conditions of the reward calculator 230. For example, in the ALE process, a reward can be calculated when a specific etching depth is reached.

[0095] Incentives can be designed based on the cost function. The cost function of the ALE process is typically expressed as a squared function relating to the various output parameters of the structure after ALE treatment. The cost function can be defined as:

[0096] Where c is the cost, w i As the weight, p i To normalize the output parameters (e.g., critical dimensions in a selected vertical coordinate system), p itarget Let N be the normalized target value for this output parameter, and N be the sequence number of this parameter. If multiple structures are evaluated, the cost function can be further expressed as:

[0097] where C is the cumulative cost of multiple structures, W j is the weight, c j is the individual structure cost. This method can be applied to several or multiple structures on a substrate (e.g., a 300 mm wafer). This method can also be able to quantify various loading effects for different structures or different parts of a structure. The reward can be designed as: R = f(c), [3]

[0098] where R represents the reward, and f represents a function that determines the reward based on the cost c. In one embodiment, the reward can be designed as a number of discrete values according to the cost. For example, the cost range can be divided into 10 intervals. Each interval is represented by an integer.

[0099] The reward is calculated each time the RL process reaches a terminal node. Each state-action pair (e.g., S a1, A a1-b1 ) that is part of the state-action chain that obtains the reward for a test case is visited, and its visit count is updated synchronously. When a sufficient number of test cases are completed and the training period is over, the average reward for each state-action pair can be calculated by dividing the cumulative reward by the visit count.

[0100] The associated value of a node can be determined by the average reward of all state-action pairs that originate from the node. These data can be used to train the policy neural network 226 to favor the generation of high-reward actions.

[0101] In some embodiments, the RL algorithm can be designed to favor exploration over exploitation. For example, in a new training period of the RL, the initial weights of the policy neural network can be assigned randomly, which is an effective technique to avoid the RL process from falling into a local optimum of the process recipe parameter space.

[0102] In other embodiments, techniques similar to the ε-greedy algorithm can be employed to expand the search tree. This algorithm assigns a fraction of the probability distribution to be a completely random distribution, which is a well-known technique in the art.

[0103] The present disclosure is only illustrated with ALE as an example. In actual RL processes, the number of nodes can be very large. Through continuous updating of the weights, the action selection gradually narrows down until the policy neural network 226 reaches certainty. Subsequently, a process recipe for actual application can be generated.

[0104] Figure 8A flowchart illustrating the flow 800 of a self-starting process for autonomously generating process recipes through an RL process. The flow 800 starts at step 802, where the RL agent 224 initiates a training cycle of the RL process. A training cycle is represented as a network of nodes created by the MCTS procedure driven by the policy neural network. Each training cycle contains multiple episodes, each representing a complete simulation of a virtual flow based on the system digital twin 204. For example, an ALE episode represents a complete ALE process. The structure on the substrate has met a set of criteria, such as reaching a target etch depth. A completed training cycle should provide rewards associated with state-action pairs and node values.

[0105] At step 804, the policy neural network 226 is assigned initial weights. In one implementation, the weights are assigned randomly. In another implementation, the weights are based on previous RL events, enabling continuous improvement so that the policy neural network 226 generates more optimal actions to increase rewards.

[0106] At step 806, the initial node of the network is established. The initial state corresponding to the initial node describes several parameters of the incoming substrate, examples of which are given in Table 2. At this point, the RL agent 224 applies the policy neural network 226 to generate a probability distribution of the selected recipe parameters. Based on the probability distribution, the MCTS procedure 228 is used to generate an action with determined recipe parameters. Typically, a random number generator is used to generate the action based on the distribution. Subsequently, the RL agent 224 applies the action using the system digital twin 204 to generate the next node with a new state. The process is repeated until the episode is completed.

[0107] At step 808, the network is expanded step by step using the policy neural network 226 and the MCTS procedure 228. Each state-action pair in the network is associated with a visit count. Some state-action pairs are involved in multiple episodes (reflected by the visit count).

[0108] At step 810, the rewards of all completed episodes are calculated based on the reward calculator 230. If the state-action pair is involved in a particular episode, the episode will receive the corresponding reward at step 812. The reward is accumulated as the visit count increases. The average reward of a particular state-action pair is equal to the accumulated reward divided by the visit count of the state-action pair.

[0109] In step 814, the RL agent 224 determines whether the training cycle is complete. This can be determined by evaluating the nodes in the network and the completed cases, and according to the selected recipe parameters / discrete levels. If the result is no, the RL agent 224 continues to expand the network. Otherwise, the RL agent 224 determines the value of each state in step 816. For each node associated with a state, the RL agent 224 has established a relationship between the state-action pair and its associated reward. The node value based on the current policy neural network can be calculated as the average of the rewards from all state-action pairs originating from the node.

[0110] In step 818, the RL agent 224 updates the weights of the policy neural network 226 based on all available state-action pairs. At each node, the state is the input to the policy neural network 226, and a set of softmax / logistic function parameters is the output. The output also includes the predicted value. The updated weights should cause the policy neural network to be more inclined to generate actions with higher values, and to more accurately predict the values. As the policy neural network 226 improves, it should be able to more confidently select an action from a set of available actions to generate the highest reward. This becomes a typical classification problem, and therefore, the cost function used to update the policy neural network 226 should include a cross-entropy loss function and a squared error function of the value. The policy neural network 226 can be trained by utilizing the rewards associated with all actions of a node. In one embodiment, earlier nodes can be given more weight during training to comply with the discounting rule.

[0111] Different surface conditions and edge ring heights can be used as inputs to train the policy neural network 226. Thus, the policy neural network 226, after being transferred to the system controller of the process system, can be used for accurate predictions in actual applications.

[0112] In step 820, the RL agent 224 evaluates whether the weights have converged to a deterministic policy neural network. If the result is no, the RL agent 224 can initiate a new training cycle to repeat the process and generate more data through more exploration. In one embodiment, an ε-greedy algorithm can be employed to encourage exploration over exploitation. In another embodiment, a new set of initial weights can be applied to the policy neural network 226. In another embodiment, the weights generated from the previous training cycle can be used with the ε-greedy algorithm.

[0113] If the evaluation result in step 820 is yes, then the strategy neural network 226 is finally determined in step 822, thereby generating the process recipe. The finally determined strategy neural network 226 can be transmitted to the system controller 232 via communication link 146. In the actual processing, after the state is calibrated and determined, the strategy neural network can be used to generate actions in real time. This can be regarded as an inference operation using the trained strategy neural network 144.

[0114] The trained policy neural network 144 may be generated from multiple combinations of input and output specifications. Since training can be performed in the background, very large and deep neural networks can be applied to process large amounts of data. General ALE policy neural networks can be constructed for inputs with different types of stacks, key dimensions, and contour requirements. Implementations will vary widely, from application-specific policy neural networks to general-purpose policy neural networks applicable to multiple or more applications. All such variations are within the scope of this invention.

[0115] Figure 9A The real-time control flow of the process implemented in process system 100 is illustrated. Flow 900 begins at step 902, where AI engine 140 receives the input and output specifications of the substrate. In step 904, AI engine 140 of AI machine 200 trains a policy neural network. The trained policy neural network 144 is then transmitted from AI machine 200 to system controller 132 via communication link 146. The trained policy neural network 144 can be transmitted to multiple system controllers of a set of process systems. Upon receiving the trained policy neural network 144, system controller 132 can generate a process recipe for the substrate to be processed in plasma process chamber 104. In some embodiments, the trained neural network 144 may be stored in the storage medium of system controller 132 and retrieved to generate the process recipe. In step 906, the state of the inner surface and edge ring of the chamber is updated using chamber surface aging digital twins and edge ring digital twins, respectively. The aging condition of the surface and edge ring exposed to plasma, as well as other parameters (including parameters related to cleaning procedures during PM), are inputs to the digital twin model.

[0116] In step 908, system controller 132 receives the input and output specifications of the substrate to be processed. In step 910, an initial state of the substrate is generated, which uses a set of parameters to describe the structure of the incoming substrate. In step 912, system controller 132 generates a process recipe using a trained policy neural network 144. This process recipe consists of a series of actions presented in the form of state-action pairs. In step 914, system controller 132 executes the actions according to the process recipe. For example, the actions could be a loop including a surface modification step and a sputtering step in an ALE process. In step 916, the state is calculated based on the actions of the system controller.

[0117] Subsequently, the calculated states are calibrated using a state calibration neural network 916. The state calibration neural network 916 is a trained neural network, whose training can be based on simulated and measured data. For example, intermediate states can be calculated and compared with measured values ​​to train the state calibration neural network 916. Taking the ALE process as an example, the contours during the etching process can be predicted using a system digital twin 204. Then, transmission electron microscopy (TEM) technology is applied to the substrate removed from the plasma process chamber of this step to obtain the true contours of the structure. The difference between the actual data and the simulated data serves as a set of training data for the state calibration neural network 916. In some other embodiments, optical reflectance measurement techniques can be used to generate the actual data.

[0118] like Figure 9B As shown, the state calibration neural network 916 takes the calculated state as one input and the output of the RT monitor 148 as another input. The calibrated state is the output of the state calibration neural network 916. The RT monitor 148 includes... Figure 9BVarious sensors are exemplarily listed in the example. These sensors include, but are not limited to: IV probes for measuring RF current / voltage, RF power sensors for measuring reflected RF power, phase sensors for measuring the phase of RF current or voltage, optical emission spectroscopy sensors for measuring the neutral component within a cavity, pressure gauges for measuring cavity pressure, temperature sensors for measuring chuck temperature, and optical reflection measurement technology sensors for measuring substrate structure progression. In step 918, system controller 132 evaluates whether a termination state has been reached based on the calibrated state. The termination state represents the end point of the process flow. If the termination state is reached, process 900 is completed. Otherwise, in step 920, system controller 132 evaluates whether the calibration of the state meets the conditions for triggering step 922 to generate a new process recipe based on the calibrated state using a trained policy neural network 144. A squared error function can be constructed to measure the difference between the calculated state and the calibrated state. This error function contains a set of selected parameters for describing the state. If the normalized error is higher than a predetermined target, the process recipe will be regenerated for the remaining process steps.

[0119] In one implementation, all actions of the remaining process are generated at once. In another implementation, only the next action is generated. The state is calibrated sequentially in each step, and the corresponding action is generated step by step based on the calibration state. In step 920, if the calibration effect is insufficient, steps 914 and 918 are repeated until the termination state is reached.

Claims

1. A control module for a semiconductor process system, characterized in that, include: Multiple subsystem controllers are used to control the operation of multiple subsystems; The AI ​​engine is used to autonomously generate process recipes by training a policy neural network using RL algorithms. as well as A system controller that autonomously adjusts the process formulation during substrate processing using data provided by a trained policy neural network and an RT monitor.

2. The control module according to claim 1, wherein, The AI ​​engine is part of a cloud-based AI machine, which is connected to the system controller via a communication link.

3. The control module according to claim 1, wherein, The AI ​​engine also includes an AI engine controller, which generates the process recipe based on the system digital twin and using an RL engine.

4. The control module according to claim 3, wherein, The system digital twin also includes RF digital twin, gas digital twin, temperature digital twin, chamber plasma digital twin, chamber surface aging digital twin, edge ring digital twin, and process digital twin.

5. The control module according to claim 3, wherein, The RL engine also includes: an RL agent, a Monte Carlo Tree Search (MCTS) program, and a reward calculator.

6. The control module according to claim 1, wherein, The policy neural network includes an input layer, multiple hidden layers, and an output layer. The output layer comprises multiple parts, each part providing an output describing the probability distribution of the selected process formulation parameters at each discretized level through softmax and / or logistic functions.

7. The control module according to claim 1, wherein, The RT monitor also includes multiple sensors for measuring parameters of the RF subsystem, gas subsystem, and temperature subsystem.

8. The control module according to claim 1, wherein, The RT monitor also includes an optical emission spectrum sensor for monitoring plasma within the plasma process chamber.

9. The control module according to claim 1, wherein, The RT monitor also includes an optical reflection measurement sensor for determining the structural parameters of the etching process.

10. The control module according to claim 1, wherein, The system controller adjusts the process recipe based on a comparison between a calculated state and a calibration state, wherein the state represents the structure of the substrate being processed.

11. The control module according to claim 1, wherein, The control module is part of the etching or deposition process system.

12. An AI machine, comprising: Multiple hardware and software modules optimized for AI applications; as well as The AI ​​engine built on the hardware and software modules autonomously trains a policy neural network through a RL process. The trained policy neural network is then transmitted to the system controller of the process system to generate and adjust the process recipe in real time based on the data provided by the RT monitor.

13. The AI ​​machine according to claim 12, wherein, The AI ​​machine is connected to multiple process systems through multiple communication links, wherein the trained policy neural network is deployed in the multiple process systems.

14. The AI ​​machine according to claim 12, wherein, The system controller receives the trained strategy neural network and generates the process recipe in real time based on the input and output specifications of the substrate to be processed. The system controller also processes the outputs of the chamber surface aging digital twin and the edge ring digital twin.

15. The AI ​​machine according to claim 12, wherein, The AI ​​engine also includes an AI engine controller, which generates the process recipe based on the system digital twin and using an RL engine.

16. The AI ​​machine according to claim 15, wherein, The RL engine also includes an RL agent, a Monte Carlo Tree Search (MCTS) program, and a reward calculator.

17. A method for real-time control of a semiconductor process system, comprising: a) Train the policy neural network using the AI ​​engine of the AI ​​machine through the RL process; b) The trained policy neural network is transmitted to the system controller of the process system via a communication link; c) Update the chamber surface aging digital twin and the edge loop digital twin, wherein the trained policy neural network is provided with additional inputs from the chamber surface aging digital twin and the edge loop digital twin; d) Receive the input and output specifications of the substrate to be processed in the process system; e) Generate the initial state of the substrate based on the input; f) Generate a process recipe comprising a series of actions using the trained policy neural network; g) The system controller performs actions according to the process formula; h) The system controller calculates the state of the substrate after it performs the action based on the system digital twin; i) The status is calibrated based on the data provided by the RT monitor; j) If the difference between the calculated state and the calibrated state is higher than a predetermined target, then the process recipe is regenerated for the remaining process steps using the trained policy neural network; and k) Repeat steps g) to j) until the termination state is reached.

18. The method of claim 17, wherein the method further comprises data provided by the RT monitor using a plurality of sensors for measuring parameters of the RF subsystem, the gas subsystem, and the temperature subsystem.

19. The method of claim 17, wherein the method further comprises data provided by the RT monitor using an optical emission spectral sensor and / or an optical reflectance measurement sensor.

20. The method of claim 17, wherein, The chamber surface aging digital twin and the edge ring digital twin take into account the duration of plasma exposure to the chamber interior surface and the cleaning procedures during preventative maintenance.

Images