Training method of scheduling model, material scheduling method and semiconductor process equipment

Abstract

The embodiment of the present application provides a kind of training method of scheduling model, the method comprises: obtaining multiple groups of training data, each group of training data includes: multiple first state information and multiple first prediction action information, wherein the first state information is used to describe the time length required for material in different positions to complete multiple processes from the corresponding current position respectively and return to the work completion position, and the first prediction action information is used to describe the next process executed by the material;For each group of training data, determine the feedback information according to the execution result of the first prediction action information and the first state information executed by the semiconductor process equipment;According to the feedback information, target training data is selected from the multiple groups of training data;According to target training data, train the preset scheduling model until the preset scheduling model meets the training stop condition, and obtain the trained target scheduling model.

Description

Technical Field

[0001] This invention relates to the field of semiconductor technology, and in particular to a method for training a scheduling model, a material scheduling method, and semiconductor process equipment. Background Technology

[0002] Currently, with the development of automation technology, it is possible to process at least one material in semiconductor process equipment, with each material corresponding to multiple processes. The commonly used method is depth-first search (DFS) to plan the material processing. This involves taking the movable materials and the current state of the semiconductor process equipment as input, simulating a movement of one movable material to generate the state of another semiconductor process equipment and another set of movable materials, which are then used as input for the next simulated movement, and so on.

[0003] Based on existing technical solutions, efficiency can usually only be optimized for a certain step, and it is impossible to optimize the efficiency of the entire processing process. This will result in waste of resources and low equipment capacity. Summary of the Invention

[0004] The technical problem to be solved by the embodiments of the present invention is low equipment productivity.

[0005] To address the aforementioned problems, this invention discloses a training method for a scheduling model, applied to semiconductor process equipment. The semiconductor process equipment processes at least one material, and each material corresponds to multiple processes. The training method for the scheduling model includes:

[0006] Acquire multiple sets of training data. Each set of training data includes: multiple first state information and multiple first predicted action information. The first state information is used to describe the time required for materials at different positions to complete multiple processes from their corresponding current positions and return to the work completion position. The first predicted action information is used to describe the next process to be performed by the materials.

[0007] For each set of training data, feedback information is determined based on the execution result of the first predicted action information of the semiconductor process equipment and the first state information;

[0008] Based on the feedback information, target training data is selected from multiple sets of training data;

[0009] Based on the target training data, a preset scheduling model is trained until the preset scheduling model meets the training stopping condition, thus obtaining the trained target scheduling model.

[0010] Optionally, each set of training data also includes first state mask information, which describes whether the material meets the conditions for performing the next process. Multiple sets of training data are obtained, including:

[0011] The first predicted action information, the first state information, and the first state mask information are input into the preset scheduling model, and the updated first predicted action information is output. This updated first predicted action information is then used by the semiconductor process equipment to execute the updated first predicted action information until the material returns to the operation completion position, so as to obtain multiple first state information and multiple updated first predicted action information. The training data also includes the updated first predicted action information.

[0012] Optionally, for each set of training data, feedback information is determined based on the execution result of the first predicted action information performed by the semiconductor process equipment and the first state information, including:

[0013] If the execution result indicates that the material has completed the first predicted action, the feedback information is determined based on the first status information.

[0014] If the execution result is that the material returns to the operation completion position after completing all the first predicted action information, the feedback information is determined to be the first feedback value, and the first feedback value is a positive number.

[0015] If the first predicted action information is an action information that cannot be executed, the feedback information is determined to be the second feedback value, and the second feedback value is a negative number;

[0016] If the first predicted action information is 0, the feedback information is determined to be the third feedback value. The third feedback value is negative, and the second feedback value is less than the third feedback value.

[0017] Optionally, based on feedback information, target training data is selected from multiple sets of training data, including:

[0018] Calculate the evaluation function for each set of training data based on the multiple feedback messages corresponding to each set of training data.

[0019] Based on the evaluation function, target training data is selected from multiple sets of training data.

[0020] Optionally, each set of target training data also includes target preset action information. Based on the target training data, a preset scheduling model is trained until the preset scheduling model meets the training stopping condition, resulting in a trained target scheduling model, including:

[0021] The target training data is input into the preset scheduling model, and the target predicted action information is output.

[0022] Calculate the loss function based on the target predicted action information and the target preset action information;

[0023] The training parameters in the preset scheduling model are adjusted according to the loss function until the preset scheduling model meets the training stopping condition, thus obtaining the trained target scheduling model.

[0024] Optionally, the target predicted action information includes multiple action values. Based on the target predicted action information and the target preset action information, a loss function is calculated, including:

[0025] Adjust each action value in the target predicted action information to determine the adjusted target predicted action information;

[0026] The loss function is calculated based on the adjusted target predicted action information and the target preset action information.

[0027] Optionally, the action value in the target predicted action information is adjusted to determine the adjusted target predicted action information, including:

[0028] If the action value is greater than or equal to a preset threshold, the action value is updated to the first action value, which is greater than the preset threshold.

[0029] If the action value is less than a preset threshold, the action value is updated to a second action value, which is also less than the preset threshold.

[0030] Optionally, the first state information includes: multiple sets of processing positions and the time required to complete multiple processes. The time is used to describe the sum of the transportation time and the process time required to transfer the material from the current position to the operation completion position. Each set of processing positions includes a first processing position and a second processing position. Each set of processing positions is used to describe the material being transferred from the first processing position to the second processing position. The processing position is the position of the material in the semiconductor process equipment when performing each process.

[0031] Optionally, the semiconductor process equipment is a machine tool, the material is a wafer, and the processing positions include a wafer cassette, an atmospheric hand, a vacuum-atmosphere conversion module, a vacuum hand, a process chamber, and a cooling chamber.

[0032] This invention discloses a material scheduling method applied to semiconductor process equipment. The method involves processing at least one material, with each material corresponding to multiple processes.

[0033] Acquire second state information and first action information; the second state information is used to describe the time required for materials at different positions to complete multiple processes from their corresponding current positions and return to the work completion position, and the first action information is used to describe the process currently being performed by the material;

[0034] The second state information and the first action information are input into the target scheduling model as claimed in claim 1, and the second predicted action information is output for the semiconductor process equipment to schedule materials according to the second predicted action information.

[0035] This invention discloses a semiconductor process apparatus. The semiconductor process apparatus processes at least one material, and each material corresponds to multiple processes. The semiconductor process apparatus includes:

[0036] The controller is used to acquire multiple sets of training data. Each set of training data includes multiple first state information and multiple first predicted action information. The first state information is used to describe the time required for the material to complete multiple processes, and the first predicted action information is used to describe the next process to be executed by the material.

[0037] For each set of training data, feedback information is determined based on the execution result of the first predicted action information of the semiconductor process equipment and the first state information;

[0038] Based on the feedback information, target training data is selected from multiple sets of training data;

[0039] Based on the target training data, a preset scheduling model is trained until the preset training model meets the training stopping condition, thus obtaining the trained target scheduling model.

[0040] According to an embodiment of the present invention, multiple sets of training data are acquired, each set including: multiple sets of first state information describing the time required for materials at different locations to complete multiple processes from their corresponding current positions and return to the work completion position; and multiple sets of first predicted action information describing the next process to be performed by the materials. For each set of training data, feedback information is determined based on the execution result of the first predicted action information and the first state information of the semiconductor process equipment. Since the feedback data is determined based on the execution result of the first predicted action information and the first state information, the feedback data can reflect the effectiveness of the execution result corresponding to the training data and the time required to complete multiple processes. Based on the feedback information, target training data is selected from the multiple sets of training data. Here, high-quality target training data can be selected based on the execution result and the time required to complete multiple processes. Finally, a preset scheduling model is trained based on the target training data until the preset training model meets the training stopping condition, resulting in a trained target scheduling model. Thus, the trained target scheduling model can quickly and accurately output predicted action information based on the input state information, facilitating the efficient and accurate execution of the predicted action information output by the target scheduling model by the semiconductor process equipment, thereby improving the production efficiency of the semiconductor process equipment and increasing machine capacity. Attached Figure Description

[0041] Figure 1 This diagram illustrates the structure of a semiconductor process apparatus provided in an embodiment of the present invention.

[0042] Figure 2 A schematic diagram of a current scheduling model training method provided by an embodiment of the present invention is shown;

[0043] Figure 3 A flowchart of a training method for a scheduling model provided by an embodiment of the present invention is shown;

[0044] Figure 4 A schematic diagram of a decision-making unit provided in an embodiment of the present invention is shown;

[0045] Figure 5 This diagram illustrates the structure of a preset scheduling model provided in an embodiment of the present invention.

[0046] Figure 6 A schematic diagram of a semiconductor process equipment structure provided by an embodiment of the present invention is shown. Detailed Implementation

[0047] The features and exemplary embodiments of various aspects of the present invention will now be described in detail. To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only configured to explain the present invention and are not configured to limit the present invention. For those skilled in the art, the present invention can be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the present invention by illustrating examples of the invention.

[0048] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0049] First, combined Figure 1 A brief description of the semiconductor process equipment involved in the invention is provided.

[0050] Semiconductor process equipment includes wafer cassettes, atmospheric handpieces, vacuum-atmosphere conversion modules, vacuum handpieces, process chambers, and cooling chambers.

[0051] like Figure 1 As shown, semiconductor process equipment may specifically include:

[0052] The LoadPort (LP) is used to hold wafer cassettes. Wafers are removed from the LP and transferred to the PM (Processing Unit) for processing. After processing, the wafers are transferred out and returned to the LP. The LP includes: LP1, LP2, and LP3.

[0053] The ATMRobot is a double-layered, single-arm device that can handle one set of wafers at a time. Viewed from the ATMRobot, the vacuum-atmosphere conversion module includes LA, LB, LC, and LD.

[0054] The cooling chamber has a total of 6 slots, divided into 3 groups for use;

[0055] The vacuum-to-atmosphere conversion module (LoadLock) in semiconductor process equipment can include four LoadLocks: LA, LB, LC, and LD. Since the VACRobot needs to evacuate the LoadLocks to a vacuum state when handling wafers, and the ATMRobot needs to fill them with atmospheric pressure, the semiconductor process equipment includes four LoadLocks, each with two slots. These are typically used separately: the upper layer is for wafer entry, storing wafers retrieved from the LP (Layer of Process) to enter the chamber for processing; the lower layer is for wafer exit, storing wafers that have been processed and removed from the chamber. Because each VACRobot has two arms, LA and LB are used in combination, and LC and LD are used in combination.

[0056] The vacuum robot (VACRobot) has two layers and two arms, with a total of four wafer positions. When picking up and placing wafers in the LoadLock or PM chamber, the two arms move simultaneously, allowing two wafers to be picked up and placed at the same time.

[0057] A process chamber (PM) is a semiconductor process equipment that may include PMA, PMB, and PMC. Each PM chamber includes two slots and can process two wafers at a time. The semiconductor process equipment involved in this embodiment processes at least one material, and each material corresponds to multiple processes. For example, the material can be a wafer, and the wafer processing process includes multiple processes.

[0058] The task includes multiple specific process flows. Among them, the wafer transfer process involves the wafer moving inside the machine according to a set path, as shown below: LP->ATMRobot->LA / LB / LC / LD->VACRobot->PMA / B / C->VACRobot->LA / LB / LC / LD->ATMRobot->Cool->ATMRobot->LP. Continuous actions that generate displacement or consume time are abstracted into actions.

[0059] For details, please refer to Figure 1The multiple processes can be specifically as follows:

[0060] Step 1: Transferring the wafer from the wafer cassette to the atmospheric handpiece

[0061] Step 2: The wafer is transferred from the atmospheric hand to the vacuum atmospheric conversion module;

[0062] Step 3: The wafer is transferred from the vacuum atmosphere conversion module to the vacuum hand;

[0063] Step 4: The wafer is transferred from the vacuum handle to the process chamber;

[0064] Step 5: The wafer is transferred from the process chamber to the vacuum handle;

[0065] Step 6: Wafer transfer module from vacuum hand to vacuum atmosphere;

[0066] Step 7: The wafer is transferred from the vacuum-to-atmosphere conversion module to the atmospheric hand.

[0067] Step 8: The wafer is transferred from the atmospheric chamber to the cooling cavity;

[0068] Step 9: The wafer is transferred from the cooling chamber to atmospheric air.

[0069] Step 10: For the cooled wafer, return it to the wafer cassette from the atmospheric hand.

[0070] As shown in Table 1, columns represent time, rows represent various modules along the path, and actions within cells occupy relevant modules and have a sequential order.

[0071] Table 1

[0072]

[0073] Secondly, a brief explanation of deep reinforcement learning:

[0074] Reinforcement learning is a special type of machine learning algorithm that draws inspiration from behaviorist psychology. Unlike supervised and unsupervised learning, the goal of reinforcement learning algorithms is to determine how an agent—the entity running the algorithm—should perform actions in its environment to maximize cumulative rewards. It requires making decisions and taking actions based on current conditions to achieve a specific, desired objective.

[0075] Machine learning algorithms that solve these types of problems are called reinforcement learning (RL). For example, for an autonomous car, a reinforcement learning algorithm controls the car's actions to ensure it safely reaches its destination. For a Go algorithm, the algorithm decides how to move pieces based on the current board position to win the game. In the first problem, the environment is an abstraction of a system composed of parameters such as the vehicle's current driving state (e.g., speed) and road conditions; the reward is the expected outcome—that the car correctly travels on the road and reaches its destination without an accident.

[0076] Many control and decision-making problems can be abstracted into this type of model. Similar to supervised learning, reinforcement learning also has a training process, which requires continuously executing actions, observing the effects of the actions, and accumulating experience to form a model.

[0077] Unlike supervised learning, each action here generally doesn't have a directly assigned label as a supervisory signal. The system only provides feedback to the algorithm's actions, and this feedback is typically delayed. The full consequences of the current action won't be apparent until the future, and the future is also random. For example, what pedestrians or vehicles will be moving on the road in the next moment, and how the opponent will play their piece after the algorithm makes a move, are all random rather than deterministic. The effect of the current move is only apparent at the end of the game.

[0078] Reinforcement learning has a wide range of applications and is considered one of the core technologies leading to strong artificial intelligence / general artificial intelligence. It can be found wherever decision-making and control are required. Typical applications include games and strategic games, autonomous driving systems / driverless cars, and robot control. All these problems share a common characteristic: the intelligent agent needs to observe its environment and its own state, and then decide on the action to take to achieve the desired goal.

[0079] An agent is an action entity in reinforcement learning. For an autonomous car, the environment is the current road conditions; for Go, the state is the current game position. At each moment, the agent and the environment have their own states, such as the car's current position and speed, and the situation of vehicles and pedestrians on the road. The agent determines an action based on the current state and executes that action. Afterward, it and the environment enter the next state, and the system gives it a feedback value, rewarding or penalizing the action to compel the agent to perform the desired action.

[0080] Reinforcement learning is a class of methods for solving this type of decision-making problem. The algorithm learns a mapping function, called the policy function, from samples. Its input is the current environment information, and its output is the action to be performed.

[0081]

[0082] Here, 's' represents the state, and 'a' represents the action to be performed. The state and action are derived from the state set and action set, respectively. Actions and states can be discrete, such as turning 30 degrees left or right, or they can be continuous real numbers, such as turning 30 degrees left or right. For the former, the action and state sets are finite; for the latter, they are infinite. The goal of performing the action is to achieve a certain objective, such as the safe driving of an autonomous vehicle or winning a Go tournament. This objective is modeled using a reward function.

[0083] Although traditional reinforcement learning theory has been continuously improved over the past few decades, it still struggles to solve complex problems in the real world.

[0084] Deep reinforcement learning (DRL) is a product of the combination of deep learning and reinforcement learning. It integrates the powerful understanding capabilities of deep learning in perceptual problems such as vision with the decision-making capabilities of reinforcement learning, achieving end-to-end learning. The emergence of deep reinforcement learning has made reinforcement learning technology truly practical, enabling the solution of complex problems in real-world scenarios.

[0085] The training method for the scheduling model provided in this embodiment of the invention can be applied to at least the following application scenarios, which will be described below.

[0086] In related technologies, wafer path planning adopts a depth-first search method. The movable wafers of the current machine and the current machine state are used as inputs. After simulating a movement of the 0th movable wafer, a machine state and another set of movable wafers are generated as inputs for the next simulated movement.

[0087] Following this pattern, a search is performed at a depth of N (commonly 4). Each time a movement occurs, the time of the current action is recorded. When the depth reaches the preset N, the total time for N movements is recorded, and the process is recursively repeated until all possible N movements of all movable wafers are traversed. The N movements with the shortest total time are selected as the search result, and the state reached by the machine after N movements and the wafer position are used as the input for the next search at a depth of N.

[0088] Taking a search at depth 3 as an example, the three actions with the shortest time are taken as the output, and the machine state after simulating these three moves is used as the input for the next search. A search at depth N, taking the action with the shortest total time for N steps, is a local optimum, not a global optimum.

[0089] For example, such as Figure 2 As shown, t1=10, t2=9, t3=8, t4=11. The shortest time t3 is taken as the output of the first round of depth search, and state S43 is taken as the starting state of the second round of depth search. The search results t1~t4 of the second round are 5, 6, 7, and 8 respectively. Therefore, the shortest total search time for the two rounds is 8+5=13.

[0090] However, the inventors discovered that using state S42 as the starting state for the second round of deep search could result in t1~t4 of 3, 4, 5, and 6 respectively. Therefore, the shortest total search time for the two rounds would be 9+3=12. Since the total time for state S42 is even shorter, state S42 is generally superior and is not selected, thus resulting in wasted capacity.

[0091] Based on the above application scenarios, the training method of the scheduling model provided in the embodiments of the present invention will be described in detail below.

[0092] Figure 3 This is a flowchart of a training method for a scheduling model provided in an embodiment of the present invention.

[0093] like Figure 3 As shown, the training method for this scheduling model may include steps 310-340, as detailed below:

[0094] Step 310: Obtain multiple sets of training data. Each set of training data includes: multiple first state information and multiple first predicted action information. The first state information is used to describe the time required for materials at different positions to complete multiple processes from their corresponding current positions and return to the work completion position. The first predicted action information is used to describe the next process to be performed by the materials.

[0095] Step 320: For each set of training data, determine the feedback information based on the execution result of the first predicted action information of the semiconductor process equipment and the first state information;

[0096] Step 330: Based on the feedback information, select target training data from multiple sets of training data;

[0097] Step 340: Train the preset scheduling model based on the target training data until the preset training model meets the training stopping condition, and obtain the trained target scheduling model.

[0098] In the training method of the scheduling model provided by this invention, multiple sets of training data are acquired. Each set of training data includes: multiple sets of first state information describing the time required for materials at different locations to complete multiple processes from their corresponding current positions and return to the work completion position; and multiple sets of first predicted action information describing the next process to be executed by the materials. For each set of training data, feedback information is determined based on the execution result of the first predicted action information and the first state information of the semiconductor process equipment. Since the feedback data is determined based on the execution result of the first predicted action information and the first state information, the feedback data can reflect the effectiveness of the execution result corresponding to the training data and the time required to complete multiple processes. Based on the feedback information, target training data is selected from the multiple sets of training data. Here, high-quality target training data can be selected based on the execution result and the time required to complete multiple processes. Finally, based on the target training data, a preset scheduling model is trained until the preset training model meets the training stopping condition, thus obtaining the trained target scheduling model. Therefore, the trained target scheduling model can quickly and accurately output predicted action information based on the input state information, which facilitates the efficient and accurate execution of the predicted action information output by the target scheduling model by semiconductor process equipment, thereby improving the production efficiency of semiconductor process equipment and increasing machine capacity.

[0099] The following describes the contents of steps 310-340 respectively:

[0100] Step 310 is involved.

[0101] Acquire multiple sets of training data. Each set of training data includes: multiple first state information and multiple first predicted action information. The first state information is used to describe the time required for materials at different positions to complete multiple processes from their corresponding current positions and return to the work completion position. The first predicted action information is used to describe the next process to be performed by the materials.

[0102] The first state information includes: multiple processing positions and the time required to complete multiple processes. The time describes the sum of the transportation time and process time required to transfer the material from the current position to the completed position. Each processing position includes a first processing position and a second processing position, and each processing position describes the transfer of the material from the first processing position to the second processing position. The processing position is the location of the material in the semiconductor process equipment during each process. The semiconductor process equipment is a machine tool, the material is a wafer, and the processing positions include wafer cassettes, atmospheric handles, vacuum-atmosphere conversion modules, vacuum handles, process chambers, and cooling chambers.

[0103] During the manufacturing process, wafers are transferred between different slots. This transfer can be described by (starting module - slot, ending module - slot), which is denoted as a set of processing positions, also known as a set of description pairs. One set of slots includes two wafer storage locations.

[0104] For example, each wafer cassette has 25 wafer storage positions. The front-end atmospheric hand ATM picks up two wafers at a time, and the 25th wafer is picked up and placed separately, so a total of 39 sets of wafers can be stored ((25 / 2)*3). The atmospheric hand has 1 set of wafer storage positions (1 slot). The vacuum-atmosphere conversion module has two chambers (LA-LB, LC-LD) that independently control the conversion between vacuum and atmospheric environments, and each chamber has 2 sets of wafer storage positions, so a total of 4 slots are available. The vacuum hand has 2 sets of wafer storage positions for wafers entering and leaving the process chamber (2 slots). The process chamber has 3 independent chambers, each with 1 set of wafer storage positions, for a total of (1*3 slots). The cooling chamber has 3 sets of wafer storage positions (3 slots). In total, the semiconductor process equipment has 52 slots.

[0105] First, construct the first state information, which describes the time required for each wafer at different slot positions to reach the job completion point. This time is a known quantity.

[0106] For example, such as Figure 1 As shown, the time for a wafer in slot 1 of the LP1 wafer cassette to return to LP1-1 after completing the operation is: wafer retrieved from atmospheric hand + LoadLock atmospheric charging time + atmospheric hand placement from LoadLock to LoadLock time + LoadLock vacuuming time + vacuum hand wafer retrieval time + vacuum hand placement from PM time + PM process time + vacuum hand retrieval from PM time + LoadLock evacuation time + vacuum hand placement from LoadLock time + LoadLock atmospheric charging time + atmospheric hand retrieval from LoadLock time + atmospheric hand placement from Cool time + Cooling time + atmospheric hand retrieval from Cool time + atmospheric hand placement from LP1-1 time; the time for the wafer to return to LP from the atmospheric hand is: atmospheric hand placement from LP1 time. The dimension of this vector is affected by the number of slots and process actions.

[0107] Specifically, in the 52 slots of the semiconductor process equipment (39+1+4+2+3+3), there are 39 (13*1*3) description pairs from each slot group of the wafer cassette to the atmospheric hand.

[0108] There are 4 (1*4) description pairs in the atmospheric hand to vacuum atmospheric conversion module;

[0109] There are 8 (4*2) description pairs from the vacuum-atmosphere conversion module to the vacuum hand;

[0110] There are 6 (2*3) description pairs from the vacuum hand to the process chamber;

[0111] There are 6 (3*2) description pairs from the process chamber to the vacuum hand;

[0112] There are 8 (2*4) description pairs in the vacuum hand to vacuum atmosphere conversion module;

[0113] There are 4 (4*1) description pairs from the vacuum-atmosphere conversion module to the atmospheric hand;

[0114] There are 3 (1*3) description pairs from the atmospheric hand to the cooling cavity;

[0115] There are 3 (3*1) description pairs from the cooling cavity to the atmospheric hand;

[0116] Since the wafer needs to return to its original position in the wafer cassette after processing, there is only one slot in the cassette for a cooled wafer. Therefore, there is one valid description pair for returning from the atmospheric wafer to the cassette. This example has a total of 82 = 39 + 4 + 8 + 6 + 6 + 8 + 4 + 3 + 3 + 1 description pairs. Each description pair is the sum of the products of adjacent module slots along the path plus 1, as shown in Table 2.

[0117] Table 2

[0118]

[0119] Therefore, a first state information can be constructed, such that each dimension of the first state information corresponds one-to-one with the corresponding legal description.

[0120] Among them, 82-dimensional first state information .

[0121] in, This indicates that when a wafer is present at this location (LP1-1), without considering wafer conflicts and actual machine conditions, the time steps required for the wafer at slot 1 of the wafer cassette in LP1 to complete the process and return to the wafer cassette are defined. Here, a time step is the maximum time slice that accurately describes the machine's operating state. If no wafer is present at this location, The elements at other positions in the vector are similar.

[0122] For example, S0={500,500,0…0} indicates that there are two wafers on the current machine, located in LP1-1 and LP1-2 respectively, that is, located in slot1 and slot2 of the wafer cassette in LP1 respectively, and the time for both wafers to return to LP after completing the operation is 500 seconds.

[0123] In one possible embodiment, each set of training data further includes first state mask information, which describes whether the material meets the conditions for performing the next process. Step 310 includes:

[0124] The first predicted action information, the first state information, and the first state mask information are input into the preset scheduling model, and the updated first predicted action information is output. This updated first predicted action information is then used by the semiconductor process equipment to execute the updated first predicted action information until the material returns to the operation completion position, so as to obtain multiple first state information and multiple updated first predicted action information. The training data also includes the updated first predicted action information.

[0125] Each set of training data includes first state mask information, M0={1,1,0…0}, which represents the first state mask information corresponding to the state vector S that can be moved onto ATM-1 by both LP1-1 and LP1-2.

[0126] The first state mask information is composed of a binary vector. The number of dimensions of the first state mask information is the same as the number of dimensions of the first state information. It represents whether the process of the wafer on different slots is completed and whether it has the conditions for action response.

[0127] For example, when This indicates that slot 1 of wafer cassette LP1 contains a wafer awaiting processing, and the wafer picker is ready to perform the wafer removal operation; when This means that there is no wafer to be processed in slot 1 of LP1, or that the atmospheric hand slot is not empty.

[0128] The first predicted action information, the first state information, and the first state mask information are input into the preset scheduling model, and the updated first predicted action information is output. The updated first predicted action information is as follows:

[0129] Its dimensions are the same as the first state information, which is 82 dimensions.

[0130] The machine exists in many different states. By executing machine operations based on different initial predicted action information under different states, the machine can enter a new state. In the training samples, A is a known output action; for example, as described in the text, S0 may produce actions A0={1,0…0} or A0={0,1,0…0}.

[0131] The first state information S represents the time steps required for the wafer to return to the wafer cassette from its current position without considering wafer collisions or actual equipment conditions. The first state mask information M represents the actual constraints, indicating whether each wafer in the description pair is reachable. S and M together are needed to output the first predicted action information A.

[0132] For example, S0={500,500,0…0} indicates that there are two wafers on the current machine, located on LP1-1 and LP1-2, and the time for both wafers to return to LP after completing the operation is 500 seconds. M0={1,1,0…0} indicates that both LP1-1 and LP1-2 can be moved to ATM-1. The output A0={1,0,…0} indicates moving LP1-1 to ATM-1, or A0={0,1,0,…,0} indicates moving LP1-2 to ATM-1. The training sample A is sampled from the operation of the existing machine and can be determined to be a certain action, so it is 0 or 1. 0 is used to indicate that the first predicted action information is not executed, and 1 is used to indicate that the first predicted action information is executed.

[0133] Sample collection is the establishment of multiple groups Samples are provided for the decision-making unit to train. First, a certain state is... Input to the decision-making unit to obtain the first predicted action information. Update status to The current step is obtained through the Reward function. .

[0134] Then, continue with The state is used as input to obtain the updated first predicted action information. Update the state and calculate This involves repeatedly using updated first predicted action information for semiconductor process equipment until the material returns to the completed position, thereby obtaining multiple first state information and multiple updated first predicted action information. The training data also includes the updated first predicted action information; that is, the training data includes first predicted action information. and the updated first predicted action information .

[0135] At this point, samples have been collected. By collecting several sets of samples, multiple sets of first state information and multiple sets of first predicted action information are obtained. Here, R represents feedback information, which will be explained in detail in the following steps.

[0136] Step 320 is involved.

[0137] For each set of training data, feedback information is determined based on the execution result of the first predicted action information of the semiconductor process equipment and the first state information;

[0138] The decision-making unit is the core of the scheduling model, such as Figure 4 As shown, the decision-making unit obtains the status of the equipment and wafer by receiving first status information and first status mask information, and generates first predicted action information based on the first status information and first status mask information. The equipment executes actions according to the first predicted action information, and generates feedback information based on the execution result of the first predicted action information of the semiconductor process equipment and the first status information, and transmits the feedback information to the decision-making unit to assist the decision-making unit in updating its strategy.

[0139] In one possible embodiment, step 320 includes:

[0140] If the execution result indicates that the material has completed the first predicted action, the feedback information is determined based on the first status information.

[0141] If the execution result is that the material returns to the operation completion position after completing all the first predicted action information, the feedback information is determined to be the first feedback value, and the first feedback value is a positive number.

[0142] If the first predicted action information is an action information that cannot be executed, the feedback information is determined to be the second feedback value, and the second feedback value is a negative number;

[0143] If the first predicted action information is 0, the feedback information is determined to be the third feedback value. The third feedback value is negative, and the second feedback value is less than the third feedback value.

[0144] To evaluate the state of semiconductor process equipment under different execution strategies, feedback information is determined based on the execution result of the first predicted action information and the first state information of the semiconductor process equipment. The feedback information reflects the quality of how well the execution strategy affects the state.

[0145] If the execution result is that the material has completed the first predicted action, the feedback information is determined based on the first state information. That is, the feedback information of the normal action is related to the total time of the current state. Specifically, the longer the time, the smaller the reward value.

[0146] If the execution result is that the material returns to the operation completion position after completing all the first predicted actions, the feedback information is determined as the first feedback value. The first feedback value is a positive number, meaning that when the wafer completion process returns to the operation completion position, then... The first feedback value is

[0147] If the first predicted action information is an unexecutable action, the feedback information is determined to be a second feedback value, which is negative; that is, if the output of the first predicted action information results in an illegal action, then let... This assigns a large penalty value to the action and terminates the group of actions; the second feedback value is...

[0148] If the first predicted action information is 0, the feedback information is determined to be the third feedback value. The third feedback value is negative, and the second feedback value is less than the third feedback value. That is, if the output of the first predicted action information is a vector of all zeros, i.e., no action output, then let... The same action terminates, second feedback value Less than the third feedback value .

[0149] Specifically, feedback information can be determined based on the execution result of the first predicted action information and the first state information of the semiconductor process equipment using the following formula.

[0150]

[0151] Step 330 is involved.

[0152] Based on the feedback information, target training data is selected from multiple sets of training data;

[0153] Based on the above steps, multiple sets of training data (S, M, A, R) have been obtained. These sets of training data need to be filtered to obtain the target training data.

[0154] Each set of training data includes: .

[0155] In one possible embodiment, step 330 includes:

[0156] Calculate the evaluation function for each set of training data based on the multiple feedback messages corresponding to each set of training data.

[0157] Based on the evaluation function, target training data is selected from multiple sets of training data.

[0158] Based on the multiple feedback messages corresponding to each set of training data, calculate the evaluation function for each set of training data:

[0159]

[0160] Select training data with high evaluation function values ​​as target training data, such as the top 50% of high-reward samples.

[0161] Based on the evaluation function, target training data with higher evaluation function values ​​are selected from multiple sets of training data to form training data pairs. The neural network of the decision-making unit is trained.

[0162] Step 340 is involved.

[0163] Based on the target training data, a preset scheduling model is trained until the preset training model meets the training stopping condition, thus obtaining the trained target scheduling model.

[0164] The process of parameter fitting is equivalent to model training. This training process calculates the gradient values ​​of the parameters through backpropagation and adjusts the model parameters using gradient descent, ultimately completing the parameter fitting process.

[0165] Constructing an effective decision-making unit is crucial to the model's effectiveness. This decision-making unit can use traditional deep search methods to compute the first predicted action information, which then degenerates into a binary vector.

[0166] When the value of the first predicted action information is greater than 0.5, it indicates that the action is executed. This is because the operation of the neural network makes the value of the first predicted action information relatively continuous, distributed between 0 and 1, which represents the probability of the action being executed. Actions with a probability greater than 0.5 are recorded as executed actions.

[0167] Depth search only considers four actions, and these four actions are also actions that can be described in the 82-dimensional first predicted action information. Therefore, the result of depth search can also be transformed into the first predicted action information.

[0168] Since the result of depth search is deterministic, there are only two actions: {execute, do not execute}. Therefore, when converting into the first predicted action information, there are only two values: {0,1}. That is, the first predicted action information degenerates into a binary vector.

[0169] In one possible embodiment, each set of target training data includes target preset action information, and step 340 includes:

[0170] The target training data is input into the preset scheduling model, and the target predicted action information is output.

[0171] Calculate the loss function based on the target predicted action information and the target preset action information;

[0172] The training parameters in the preset scheduling model are adjusted according to the loss function until the preset training model meets the training stopping condition, thus obtaining the trained target scheduling model.

[0173] The neural network of the pre-defined scheduling model is a computational model consisting of a large number of directly interconnected nodes. Each node (except for the input node) represents a specific output function, called the activation function. The connection between any two nodes represents the proportion of that signal in the transmission, that is, the proportion of the node's "memory value" that is passed on, called the weight.

[0174] The network output varies depending on the activation function and weights, and is an approximation of a certain function or an approximate description of the mapping relationship. In some networks, there is a bias term, which is a correction to the summation of the weights. Therefore, the operation of each fully connected layer is: W = w(S) + b.

[0175] The target training data includes: target state information and target state mask information.

[0176] For example, such as Figure 5 For example, 82-dimensional target state information is input into a fully connected layer and subjected to matrix operations with the layer's parameters w and b. Parameter w is a two-dimensional matrix of size 82*n (n is generally taken as 512), and b is an 82-dimensional vector. According to mathematical operation rules, the output operation result is still an 82-dimensional vector, which is the hidden layer result.

[0177] After obtaining the hidden layer result, this result is used as input to perform a ReLU layer operation (i.e., y = max(0, x)) to obtain the output. This output is then used as input to perform matrix operations with the parameters w and b of the next fully connected layer.

[0178] Then, a vector dot product is performed on the result of the ReLU layer based on the target state information output and the target state mask information. Figure 5 The calculations are performed sequentially through the hierarchical levels until the target predicted action information is output.

[0179] The target predicted action information and the target preset action information corresponding to the target state information can be substituted into the loss function to obtain the loss value. That is, the loss function is calculated based on the target predicted action information and the target preset action information.

[0180] Then, by differentiating the above operations according to the chain rule, the gradient function corresponding to each parameter is obtained. The loss value is then substituted into the gradient function to obtain the gradient values ​​of the parameters of each fully connected layer. These gradient values ​​are then used to update the neural network parameters, thus completing one iteration. As described above, different target state information is used as input to continuously update and optimize the parameters of the network model.

[0181] During the model calculation process, through multiple rounds of alternating sample collection and parameter fitting, the model eventually acquires decision-making capabilities, obtains optimized target prediction action information for different state matrices of the machine, and completes the decision-making process.

[0182] In one possible embodiment, the target predicted action information includes multiple action values. Based on the target predicted action information and the target preset action information, a loss function is calculated, including:

[0183] Adjust each action value in the target predicted action information to determine the adjusted target predicted action information;

[0184] The loss function is calculated based on the adjusted target predicted action information and the target preset action information.

[0185] Since the action values ​​in the target prediction action information output by the target scheduling model are not binary but continuous values ​​between 0 and 1, the action values ​​in the target prediction action information are adjusted to determine definite and executable adjusted target prediction action information. This adjusted target prediction action information is then used to calculate the loss function based on the adjusted target prediction action information and the target preset action information.

[0186] Specifically, the steps mentioned above regarding adjusting the action values ​​in the target predicted action information and determining the adjusted target predicted action information may include the following steps:

[0187] If the action value is greater than or equal to a preset threshold, the action value is updated to the first action value, which is greater than the preset threshold.

[0188] If the action value is less than a preset threshold, the action value is updated to a second action value, which is also less than the preset threshold.

[0189] The preset threshold is typically 0.5.

[0190] If the action value is greater than or equal to a preset threshold, the action value is updated to the first action value, which is 1; if the action value is less than the preset threshold, the action value is updated to 0.

[0191] Therefore, the action value in the second predicted action information can be binarized to determine the specific third predicted action information that can be executed, making it easier for the device to execute.

[0192] In summary, in this embodiment of the invention, multiple sets of training data are acquired. Each set of training data includes: multiple sets of first state information describing the time required for materials at different locations to complete multiple processes from their corresponding current positions and return to the completed work position; and multiple sets of first predicted action information describing the next process to be performed by the materials. For each set of training data, feedback information is determined based on the execution result of the first predicted action information and the first state information of the semiconductor process equipment. Since the feedback data is determined based on the execution result of the first predicted action information and the first state information, the feedback data can reflect the effectiveness of the execution result corresponding to the training data and the time required to complete multiple processes. Based on the feedback information, target training data is selected from the multiple sets of training data. Here, high-quality target training data can be selected based on the execution result and the time required to complete multiple processes. Finally, a preset scheduling model is trained based on the target training data until the preset training model meets the training stopping condition, resulting in a trained target scheduling model. Thus, the trained target scheduling model can quickly and accurately output predicted action information based on the input state information, facilitating the efficient and accurate execution of the predicted action information output by the target scheduling model by the semiconductor process equipment. This enables the semiconductor process equipment to improve production efficiency and thereby increase machine capacity.

[0193] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0194] This invention also provides a material scheduling method applied to semiconductor process equipment, characterized in that the semiconductor process equipment processes at least one material, each material corresponding to multiple processes, and the method includes:

[0195] Acquire second state information and first action information; the second state information is used to describe the time required for materials at different positions to complete multiple processes from their corresponding current positions and return to the work completion position, and the first action information is used to describe the process currently being performed by the material;

[0196] Input the second state information and the first action information into the system. Figure 1 The target scheduling model shown in the embodiment outputs second predicted action information for semiconductor process equipment to schedule materials based on the second predicted action information.

[0197] Here, the target scheduling model trained based on the above embodiments can quickly and accurately output predicted action information according to the input state information. Therefore, the second state information describing the time required for materials at different positions to complete multiple processes from their corresponding current positions and return to the work completion position, and the first action information describing the process currently being performed by the materials, can be input into the target scheduling model as described in claim 1 to output the second predicted action information, which is then used by the semiconductor process equipment to schedule materials according to the second predicted action information. This enables the semiconductor process equipment to efficiently and accurately execute the second predicted action information output by the target scheduling model, thereby improving the production efficiency of the semiconductor process equipment and increasing machine capacity.

[0198] Reference Figure 6 The diagram illustrates a structural block diagram of a semiconductor process apparatus 610 according to an embodiment of the present invention. The semiconductor process apparatus 610 includes:

[0199] The controller 611 is used to acquire multiple sets of training data. Each set of training data includes multiple first state information and multiple first predicted action information. The first state information is used to describe the time required for materials at different positions to complete multiple processes from their corresponding current positions and return to the work completion position. The first predicted action information is used to describe the next process to be performed by the materials.

[0200] For each set of training data, feedback information is determined based on the execution result of the first predicted action information of the semiconductor process equipment and the first state information;

[0201] Based on the feedback information, target training data is selected from multiple sets of training data;

[0202] Based on the target training data, a preset scheduling model is trained until the preset scheduling model meets the training stopping condition, thus obtaining the trained target scheduling model.

[0203] In an optional embodiment of the present invention, each set of training data further includes first state mask information, which is used to describe whether the material meets the conditions for performing the next process. The controller 611 is further used for:

[0204] The first predicted action information, the first state information, and the first state mask information are input into the preset scheduling model, and the updated first predicted action information is output. This updated first predicted action information is then used by the semiconductor process equipment to execute the updated first predicted action information until the material returns to the operation completion position, so as to obtain multiple first state information and multiple updated first predicted action information. The training data also includes the updated first predicted action information.

[0205] In an optional embodiment of the invention, the controller 611 is further configured to:

[0206] If the execution result indicates that the material has completed the first predicted action, the feedback information is determined based on the first status information.

[0207] If the execution result is that the material returns to the operation completion position after completing all the first predicted action information, the feedback information is determined to be the first feedback value, and the first feedback value is a positive number.

[0208] If the first predicted action information is an action information that cannot be executed, the feedback information is determined to be the second feedback value, and the second feedback value is a negative number;

[0209] If the first predicted action information is 0, the feedback information is determined to be the third feedback value. The third feedback value is negative, and the second feedback value is less than the third feedback value.

[0210] In an optional embodiment of the invention, the controller 611 is further configured to:

[0211] Calculate the evaluation function for each set of training data based on the multiple feedback messages corresponding to each set of training data.

[0212] Based on the evaluation function, target training data is selected from multiple sets of training data.

[0213] In an optional embodiment of the present invention, each set of target training data further includes target preset action information, and the controller 611 is further configured to:

[0214] The target training data is input into the preset scheduling model, and the target predicted action information is output.

[0215] Calculate the loss function based on the target predicted action information and the target preset action information;

[0216] The training parameters in the preset scheduling model are adjusted according to the loss function until the preset scheduling model meets the training stopping condition, thus obtaining the trained target scheduling model.

[0217] In an optional embodiment of the present invention, the target predicted action information includes multiple action values, and the controller 611 is further configured to:

[0218] Adjust each action value in the target predicted action information to determine the adjusted target predicted action information;

[0219] The loss function is calculated based on the adjusted target predicted action information and the target preset action information.

[0220] In an optional embodiment of the invention, the controller 611 is further configured to:

[0221] If the action value is greater than or equal to a preset threshold, the action value is updated to the first action value, which is greater than the preset threshold.

[0222] If the action value is less than a preset threshold, the action value is updated to a second action value, which is also less than the preset threshold.

[0223] In an optional embodiment of the present invention, the first state information includes: multiple sets of processing positions and the time required to complete multiple processes. The time is used to describe the sum of the transportation time and the process time required for the material to be transferred from the current position to the operation completion position. Each set of processing positions includes a first processing position and a second processing position. Each set of processing positions is used to describe the material being transferred from the first processing position to the second processing position. The processing position is the position of the material in the semiconductor process equipment when each process is performed.

[0224] In an optional embodiment of the present invention, the semiconductor process equipment is a machine tool, the material is a wafer, and the processing position includes a wafer cassette, an atmospheric hand, a vacuum-atmosphere conversion module, a vacuum hand, a process chamber, and a cooling chamber.

[0225] In an optional embodiment of the present invention, the controller 611 is further configured to: acquire second state information and first action information; the second state information is used to describe the time required for materials at different positions to complete multiple processes from their corresponding current positions and return to the work completion position, and the first action information is used to describe the process currently being performed by the materials;

[0226] Input the second state information and the first action information into the system. Figure 1 The target scheduling model shown in the embodiment outputs second predicted action information for semiconductor process equipment to schedule materials based on the second predicted action information.

[0227] In summary, in this embodiment of the invention, multiple sets of training data are acquired. Each set of training data includes: multiple sets of first state information describing the time required for materials at different locations to complete multiple processes from their corresponding current positions and return to the completed work position; and multiple sets of first predicted action information describing the next process to be performed by the materials. For each set of training data, feedback information is determined based on the execution result of the first predicted action information and the first state information of the semiconductor process equipment. Since the feedback data is determined based on the execution result of the first predicted action information and the first state information, the feedback data can reflect the effectiveness of the execution result corresponding to the training data and the time required to complete multiple processes. Based on the feedback information, target training data is selected from the multiple sets of training data. Here, high-quality target training data can be selected based on the execution result and the time required to complete multiple processes. Finally, a preset scheduling model is trained based on the target training data until the preset training model meets the training stopping condition, resulting in a trained target scheduling model. Thus, the trained target scheduling model can quickly and accurately output predicted action information based on the input state information, facilitating the efficient and accurate execution of the predicted action information output by the target scheduling model by the semiconductor process equipment. This enables the semiconductor process equipment to improve production efficiency and thereby increase machine capacity.

[0228] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0229] This invention also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, it implements the various processes of the above-described training method embodiment for a scheduling model and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0230] This invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described training method embodiment for a scheduling model and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0231] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0232] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0233] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0234] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0235] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0236] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0237] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0238] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0239] The training method for a scheduling model and a semiconductor process equipment provided by the present invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A training method for a scheduling model, applied to semiconductor process equipment, characterized in that, The semiconductor process equipment processes at least one material, each of the materials corresponding to multiple processes, and the method includes: Multiple sets of training data are acquired. Each set of training data includes multiple sets of first state information and multiple sets of first predicted action information. The first state information describes the time required for the material at different locations to complete the multiple processes from its corresponding current location and return to the work completion location. The first predicted action information describes the next process to be performed by the material. The first state information includes multiple sets of processing locations and the time required to complete the multiple processes. The time is used to describe the sum of the transportation time and the process time required for the material to move from its current location to the work completion location. Each set of processing locations includes a first processing location and a second processing location. Each set of processing locations describes the material's position in the semiconductor process equipment when it moves from the first processing location to the second processing location. For each set of training data, feedback information is determined based on the execution result of the first predicted action information executed by the semiconductor process equipment and the first state information. Based on the feedback information, target training data is selected from the multiple sets of training data; Based on the target training data, a preset scheduling model is trained until the preset scheduling model meets the training stopping condition, thus obtaining the trained target scheduling model.

2. The method according to claim 1, characterized in that, Each set of training data further includes first state mask information, which describes whether the material meets the conditions for performing the next process. Acquiring multiple sets of training data includes: The first predicted action information, the first state information, and the first state mask information are input into the preset scheduling model, and the updated first predicted action information is output. The updated first predicted action information is then used by the semiconductor process equipment to execute the updated first predicted action information until the material returns to the job completion position, so as to obtain the plurality of first state information and the plurality of updated first predicted action information. The training data also includes the updated first predicted action information.

3. The method according to claim 1, characterized in that, For each set of training data, feedback information is determined based on the execution result of the first predicted action information executed by the semiconductor process equipment and the first state information, including: If the execution result is that the material has completed the first predicted action information, the feedback information is determined based on the first status information; If the execution result is that the material returns to the operation completion position after completing all the first predicted action information, the feedback information is determined to be a first feedback value, and the first feedback value is a positive number; If the first predicted action information is an action information that cannot be executed, the feedback information is determined to be a second feedback value, and the second feedback value is a negative number; When the first predicted action information is 0, the feedback information is determined to be a third feedback value, the third feedback value is a negative number, and the second feedback value is less than the third feedback value.

4. The method according to claim 3, characterized in that, The step of selecting target training data from the multiple sets of training data based on the feedback information includes: Calculate the evaluation function corresponding to each set of training data based on the multiple feedback messages corresponding to each set of training data; The target training data is selected from the multiple sets of training data according to the evaluation function.

5. The method according to claim 1, characterized in that, Each set of target training data also includes target preset action information. The step of training a preset scheduling model based on the target training data until the preset scheduling model meets the training stopping condition, thus obtaining a trained target scheduling model, includes: The target training data is input into the preset scheduling model, and the target predicted action information is output. Calculate the loss function based on the predicted action information of the target and the preset action information of the target; The training parameters in the preset scheduling model are adjusted according to the loss function until the preset scheduling model meets the training stopping condition, thereby obtaining the trained target scheduling model.

6. The method according to claim 5, characterized in that, The target predicted action information includes multiple action values. The step of calculating a loss function based on the target predicted action information and the target preset action information includes: Adjust each action value in the target predicted action information to determine the adjusted target predicted action information; The loss function is calculated based on the adjusted target predicted action information and the target preset action information.

7. The method according to claim 6, characterized in that, The step of adjusting the action value in the target predicted action information and determining the adjusted target predicted action information includes: If the action value is greater than or equal to a preset threshold, the action value is updated to a first action value, where the first action value is greater than the preset threshold. If the action value is less than the preset threshold, the action value is updated to a second action value, which is less than the preset threshold.

8. The method according to claim 1, characterized in that, The semiconductor process equipment is a machine tool, the material is a wafer, and the processing position includes a wafer cassette, an atmospheric hand, a vacuum-atmosphere conversion module, a vacuum hand, a process chamber, and a cooling chamber.

9. A material scheduling method applied to semiconductor process equipment, characterized in that, The semiconductor process equipment processes at least one material, each material corresponding to multiple processes, and the method includes: Acquire second state information and first action information; the second state information is used to describe the time required for the material at different positions to complete the multiple processes from its corresponding current position and return to the work completion position; the first action information is used to describe the process currently being performed by the material; the second state information includes: multiple sets of processing positions and the time required to complete the multiple processes; the time is used to describe the sum of the transportation time and the process time required for the material to be transferred from its current position to the work completion position; each set of processing positions includes a first processing position and a second processing position; each set of processing positions is used to describe the material being transferred from the first processing position to the second processing position; the processing position is the position of the material in the semiconductor process equipment when performing each of the processes. The second state information and the first action information are input into the target scheduling model as described in claim 1, and the second predicted action information is output for the semiconductor process equipment to schedule the materials according to the second predicted action information.

10. A semiconductor process apparatus, characterized in that, The semiconductor process equipment processes at least one material, each material corresponding to multiple processes, and the semiconductor process equipment includes: A controller is configured to acquire multiple sets of training data. Each set of training data includes multiple sets of first state information and multiple sets of first predicted action information. The first state information describes the time required for the material to complete the multiple processes, and the first predicted action information describes the next process to be performed by the material. The first state information includes multiple sets of processing positions and the time required to complete the multiple processes. The time is used to describe the sum of the transportation time and the process time required for the material to be transferred from the current position to the work completion position. Each set of processing positions includes a first processing position and a second processing position. Each set of processing positions describes the material being transferred from the first processing position to the second processing position. The processing position is the position of the material in the semiconductor process equipment when performing each of the processes. For each set of training data, feedback information is determined based on the execution result of the first predicted action information executed by the semiconductor process equipment and the first state information. Based on the feedback information, target training data is selected from the multiple sets of training data; Based on the target training data, a preset scheduling model is trained until the preset scheduling model meets the training stopping condition, thus obtaining the trained target scheduling model.

Images