Schedule creation program generation method, schedule creation program generation device, schedule creation device, recording medium, generation program, schedule creation program, substrate processing device, and substrate processing system

Abstract

This schedule creation program generation method includes a step for repeating an experience step through reinforcement learning. In each experience step, a plan element of a preceding substrate having first priority and a plan element of a following substrate having second priority are sequentially arranged on a timetable on the basis of a constraint condition, and a reward to be granted to the timetable on which all plan elements have been arranged is determined. The constraint condition indicates a condition for restricting, according to priority, a substrate that is processed by each of substrate processing units. The reward is granted when processing of at least one preceding substrate has been completed prior to the completion of processing of a lot having the second priority. Furthermore, the reward is granted when processing of the lot having the second priority has been completed prior to the completion of processing of a lot having the first priority.

Description

Schedule creation program generation method, schedule creation program generation device, schedule creation device, recording medium, generation program, schedule creation program, substrate processing apparatus, and substrate processing system

[0001] The present invention relates to a schedule creation program generating method, a schedule creation program generating device, a schedule creation device, a recording medium, a generating program, a schedule creation program, a substrate processing apparatus, and a substrate processing system.

[0002] A semiconductor integrated circuit manufacturing system is known that determines the order in which wafers are processed based on a priority assigned to each wafer and the order in which the wafers are reserved for processing (see, for example, Patent Document 1). The semiconductor integrated circuit manufacturing system disclosed in Patent Document 1 starts processing wafers in descending order of priority. The semiconductor integrated circuit manufacturing system also starts processing wafers with the same priority in the order in which they are reserved.

[0003] Special Publication No. 2003-533044

[0004] However, in a configuration in which the start order of substrate processing is determined in descending order of priority, if processing of a high-priority substrate is reserved after processing of a low-priority substrate has begun and an interruption occurs to the processing of the high-priority substrate, there is a possibility that variations in the quality of the low-priority substrates processed before the interruption occurs and those processed after the processing of the high-priority substrate is completed. For example, variations in the amount of native oxide film may occur between the substrates processed before the interruption occurs and the substrates processed after the processing of the high-priority substrate is completed, which may affect the electrical characteristics of the device.

[0005] An object of the present invention is to provide a schedule creation program generation method, a schedule creation program generation device, a schedule creation device, a recording medium, a generation program, a schedule creation program, a substrate processing apparatus, and a substrate processing system that, when an interruption occurs in processing of a lot having a higher priority than a lot being processed in advance, can complete processing of a subsequent lot before a preceding lot without stopping processing of the preceding lot.

[0006] According to one aspect of the present invention, a schedule creation program generation method uses reinforcement learning to generate a schedule creation program for creating a time schedule for processing multiple substrates using a substrate processing apparatus. The substrate processing apparatus includes a container mounting unit, multiple substrate processing units, and a transport unit. The container mounting unit mounts multiple substrate housing containers, each housing one or more substrates constituting a single lot. Each of the multiple substrate processing units processes the substrate one by one. The transport unit transports the substrates between the multiple substrate housing containers and the multiple substrate processing units. The substrate processing apparatus completes processing for the lot by performing the substrate processing on the substrates constituting the lot one by one. The schedule creation program generates an interruption schedule in response to a situation in which a subsequent container, a substrate housing container containing a lot with a second priority higher than the first priority, is placed on the container mounting unit when some of the substrates constituting a first-priority lot remain in a preceding container, a substrate housing container that is placed prior to the container mounting unit. The interrupt schedule indicates a time schedule for processing the leading substrates remaining in the leading container and the following substrates constituting the second-priority lot by the substrate processing apparatus. The schedule creation program generation method includes a process of increasing cumulative rewards by repeating an empirical step including a timetable acquisition process, an allocation process, and a reward determination process using reinforcement learning. The timetable acquisition process indicates a process of acquiring a timetable for defining the interrupt schedule. The allocation process indicates a process of sequentially arranging multiple planning elements assigned to each of the leading substrates remaining in the leading container and multiple planning elements assigned to each of the following substrates in the timetable based on constraint conditions, thereby sequentially changing the state of the timetable. The constraint conditions indicate conditions that restrict the substrates to be processed by each of the substrate processing apparatuses using the priority. The reward determination process indicates a process of determining a reward to be awarded for the final state of the timetable based on the final state of the timetable in which all of the planning elements have been arranged and conditions for awarding rewards.The final state of the timetable corresponds to the interrupt schedule. The cumulative reward indicates the total value of the rewards awarded for the final state of the timetable. The conditions for awarding the reward include a first reward condition and a second reward condition. The first reward condition indicates that a first reward is awarded when the final state of the timetable indicates a state in which processing for at least one of the preceding substrates remaining in the preceding container is completed before processing for the second-priority lot is completed. The second reward condition indicates that a second reward, the reward value of which is smaller than the first reward, is awarded when the final state of the timetable indicates a state in which processing for the second-priority lot is completed before processing for the first-priority lot is completed.

[0007] In one embodiment, the timetable acquisition step includes a step of acquiring a timetable in which plan elements assigned to each of the preceding substrates remaining in the substrate processing apparatus are arranged, and the schedule creation program creates the interrupt schedule without changing the time schedule of the preceding substrates remaining in the substrate processing apparatus.

[0008] In one embodiment, the substrate processing apparatus is capable of performing the substrate processing on two or more of the substrates in parallel, and the first remuneration condition indicates that the first remuneration is to be granted when the final state of the timetable indicates a state in which the substrate processing on the preceding substrate is being performed in parallel with the substrate processing on at least one of the following substrates.

[0009] In one embodiment, the conditions for granting the reward further include a third reward condition, which specifies that the earlier the time at which processing of the second-priority lot is completed, the greater the value of the third reward to be granted, and the maximum value of the third reward is smaller than the value of the first reward.

[0010] In one embodiment, the conditions for granting the reward further include a fourth reward condition, which specifies that the earlier the processing of the first-priority lot is completed, the greater the value of the fourth reward to be granted, and the maximum value of the fourth reward is smaller than the value of the first reward.

[0011] In one embodiment, the substrate processing apparatus is capable of performing the substrate processing on two or more of the substrates in parallel, and the conditions for providing the remuneration further include a fifth remuneration condition, which indicates that a fifth remuneration is to be provided when the final state of the timetable indicates a state in which the substrate processing on the preceding substrate is being performed in parallel with the substrate processing on at least one of the following substrates.

[0012] According to another aspect of the present invention, a schedule creation program generation device generates a schedule creation program for creating a time schedule for processing a plurality of substrates by a substrate processing apparatus using reinforcement learning. The schedule creation program generation device includes a storage unit and a processing unit. The storage unit stores a generation program that defines the schedule creation program generation method. The processing unit executes the generation program to generate the schedule creation program.

[0013] According to yet another aspect of the present invention, a schedule creation device creates a time schedule for processing a plurality of substrates using a substrate processing apparatus. The schedule creation device includes a storage unit and a processing unit. The storage unit stores a schedule creation program generated based on the schedule creation program generation method described above. The processing unit executes the schedule creation program to create the interrupt time schedule.

[0014] According to yet another aspect of the present invention, a computer-readable recording medium stores a generation program that defines the schedule creation program generation method described above.

[0015] According to yet another aspect of the present invention, a computer-readable recording medium is provided, which records a schedule creation program generated based on the above-described schedule creation program generation method.

[0016] According to yet another aspect of the present invention, a generating program is executable by a computer, and defines the above-described method for generating a schedule creation program.

[0017] According to yet another aspect of the present invention, there is provided a schedule creation program executable by a computer, the schedule creation program being generated based on the above-described schedule creation program generation method.

[0018] According to yet another aspect of the present invention, a substrate processing apparatus completes processing of one or more substrates constituting a lot by performing substrate processing on the substrates one by one. The substrate processing apparatus includes a container mounting unit, multiple substrate processing units, a transport unit, a memory unit, and a processing unit. Multiple substrate accommodation containers that accommodate the substrates constituting the lot are mounted on the container mounting unit. Each of the multiple substrate processing units performs the substrate processing on the substrate one by one. The transport unit transports the substrates between the multiple substrate accommodation containers and the multiple substrate processing units. The memory unit stores a generation program that defines the schedule creation program generation method. The processing unit executes the generation program to generate a schedule creation program for creating a time schedule for processing multiple substrates using the reinforcement learning. When some of the substrates constituting a first priority lot remain in a leading container, which is a substrate storage container that is placed prior to the container loading section, and a trailing container, which is a substrate storage container that stores a second priority lot that is higher than the first priority, is placed on the container loading section, the processing unit executes the schedule creation program in response to this, and creates the interrupt schedule for processing the leading substrates, which are the substrates remaining in the leading container, and the trailing substrates, which are substrates constituting the second priority lot.

[0019] According to yet another aspect of the present invention, a substrate processing apparatus completes processing of one or more substrates constituting a lot by performing substrate processing on the substrates one by one. The substrate processing apparatus includes a container mounting unit, multiple substrate processing units, a transport unit, a memory unit, and a processing unit. Multiple substrate accommodation containers for accommodating the substrates constituting the lot are mounted on the container mounting unit. Each of the multiple substrate processing units performs the substrate processing on the substrate one by one. The transport unit transports the substrates between the multiple substrate accommodation containers and the multiple substrate processing units. The memory unit stores a schedule creation program generated based on the schedule creation program generation method described above. When some of the substrates constituting a first priority lot remain in a leading container, which is a substrate storage container that is placed prior to the container loading section, and a trailing container, which is a substrate storage container that stores a second priority lot that is higher than the first priority, is placed on the container loading section, the processing unit executes the schedule creation program in response to this, and creates the interrupt schedule for processing the leading substrates, which are the substrates remaining in the leading container, and the trailing substrates, which are substrates constituting the second priority lot.

[0020] According to yet another aspect of the present invention, a substrate processing system includes a substrate processing apparatus and the schedule creation program generation device described above. The substrate processing apparatus performs substrate processing on one or more substrates constituting a lot, thereby completing processing for the lot. The schedule creation program generation device further includes a transmission unit that transmits the schedule creation program to the substrate processing apparatus. The substrate processing apparatus includes a container mounting unit, multiple substrate processing units, a transport unit, a receiving unit, and a processing unit. Multiple substrate accommodation containers that accommodate the substrates constituting the lot are mounted on the container mounting unit. The multiple substrate processing units each perform the substrate processing on the substrate one by one. The transport unit transports the substrates between the multiple substrate accommodation containers and the multiple substrate processing units. The receiving unit receives the schedule creation program transmitted from the transmission unit of the schedule creation program generation device. When some of the substrates constituting a first priority lot remain in a leading container, which is a substrate storage container that is placed prior to the container loading section, and a trailing container, which is a substrate storage container that stores a second priority lot that is higher than the first priority, is placed on the container loading section, the processing unit executes the schedule creation program in response to this, and creates the interrupt schedule for processing the leading substrates, which are the substrates remaining in the leading container, and the trailing substrates, which are substrates constituting the second priority lot.

[0021] According to yet another aspect of the present invention, a substrate processing system includes a substrate processing apparatus and the schedule creation device described above. The substrate processing apparatus performs substrate processing on one or more substrates constituting a lot, thereby completing processing for the lot. The schedule creation device further includes a transmission unit that transmits the interrupt schedule to the substrate processing apparatus. The substrate processing apparatus includes a container mounting unit, multiple substrate processing units, a transport unit, and a receiving unit. Multiple substrate accommodation containers that accommodate the substrates constituting the lot are mounted on the container mounting unit. The multiple substrate processing units each perform the substrate processing on the substrate one by one. The transport unit transports the substrates between the multiple substrate accommodation containers and the multiple substrate processing units. The receiving unit receives the interrupt schedule transmitted from the transmission unit of the schedule creation device.

[0022] According to at least one aspect of the present invention, the schedule creation program generation method, schedule creation program generation device, schedule creation device, recording medium, generation program, schedule creation program, substrate processing apparatus, and substrate processing system, processing of a subsequent lot can be completed before the preceding lot without stopping processing of the preceding lot.

[0023] 6A is a diagram showing a schedule creation program generation system including a schedule creation program generation device according to a first embodiment of the present invention. FIG. 6B is a diagram showing a schedule creation program generation system including a schedule creation program generation device according to a first embodiment of the present invention. FIG. 6C is a plan view schematically showing an example of a configuration of a substrate processing apparatus to which a schedule creation program according to a first embodiment of the present invention is applied. FIG. 6D is a diagram showing an example of a relationship between a preceding lot, a succeeding lot, and interrupt processing. FIG. 6E is a diagram showing another example of a relationship between a preceding lot, a succeeding lot, and interrupt processing. FIG. 6F is a diagram showing an example of a processing procedure, processing time, and planning elements. FIG. 6G is a diagram showing an example of a timetable corresponding to the substrate processing apparatus of FIG. 2. FIG. 6H is a diagram showing a continuation of the example timetable shown in FIG. 6A. FIG. 6I is a diagram showing an example of a processing flow for creating an interrupt schedule. FIG. 6I is a block diagram showing a reinforcement learning system. FIG. 6H is a flowchart showing a schedule creation program generation method according to a first embodiment of the present invention. FIG. 6I is a flowchart showing a timetable creation process. FIG. 6I is a flowchart showing a process for randomly selecting one of actions and arranging planning elements in the timetable. FIG. 6I is a flowchart showing a process for predicting an action that will maximize a reward and arranging planning elements in the timetable. FIG. 6I is a diagram showing a substrate processing system including the substrate processing apparatus according to a first embodiment of the present invention. FIG. 6I is a diagram showing a flow of processing executed by a control unit included in the substrate processing apparatus according to a second embodiment of the present invention. FIG. 6I is a diagram showing a substrate processing system according to a third embodiment of the present invention. FIG. 10 is a diagram showing a substrate processing system according to a fourth embodiment of the present invention.

[0024] Hereinafter, with reference to the drawings (FIGS. 1A to 17), embodiments of the schedule creation program generation method, schedule creation program generation device, schedule creation device, recording medium, generation program, schedule creation program, substrate processing apparatus, and substrate processing system of the present invention will be described. However, the present invention is not limited to the following embodiments, and can be implemented in various aspects without departing from the gist of the present invention. Note that duplicated explanations may be omitted as appropriate. Furthermore, in the drawings, the same or equivalent parts are designated by the same reference symbols, and explanations will not be repeated.

[0025] The "substrate" in the embodiments of the present invention can be any of various substrates, such as a semiconductor wafer, a glass substrate for a photomask, a glass substrate for a liquid crystal display, a glass substrate for a plasma display, a substrate for an FED (Field Emission Display), a substrate for an optical disk, a substrate for a magnetic disk, and a substrate for a magneto-optical disk. Hereinafter, the embodiments of the present invention will be described mainly using as examples a schedule creation program generation method, a schedule creation program generation device, a schedule creation device, a recording medium, a generation program, a schedule creation program, a substrate processing apparatus, and a substrate processing system used in processing a disc-shaped semiconductor wafer, but the present invention can also be applied to processing the various substrates exemplified above. Furthermore, various substrate shapes can also be applied.

[0026] [First Embodiment] Hereinafter, a first embodiment of the present invention will be described with reference to Figures 1A to 14. Figures 1A and 1B are diagrams showing a schedule creation program generation system 100A including a schedule creation program generation device 100 of this embodiment. In detail, Figure 1A shows the schedule creation program generation system 100A before a schedule creation program CP is created. Figure 1B shows the schedule creation program generation system 100A after the schedule creation program CP has been created.

[0027] As shown in FIG. 1A, a schedule creation program generation system 100A includes a schedule creation program generation device 100 and a recording medium 110.

[0028] The recording medium 110 is readable by a computer. A program (computer program) to be executed by a computer is non-temporarily recorded on the recording medium 110. The recording medium 110 stores a generating program GP. The generating program GP is a computer program that can be executed by a computer.

[0029] The recording medium 110 may be, for example, a medium including a semiconductor memory such as a secure digital (SD) memory card or a universal serial bus (USB) memory, or a medium including a magnetic disk such as a hard disk drive. Alternatively, the recording medium 110 may be an optical disk such as a compact disk (CD), a digital versatile disk (DVD), or a Blu-ray disk. Alternatively, the recording medium 110 may be a main storage device or an auxiliary storage device installed in another computer system.

[0030] The schedule creation program generation device 100 generates a schedule creation program CP for creating a time schedule for processing multiple substrates W by a substrate processing apparatus WP based on the generation program GP. More specifically, the generation program GP includes a program for reinforcement learning. The schedule creation program generation device 100 generates the schedule creation program CP by reinforcement learning. The schedule creation program generation device 100 may be, for example, a general-purpose computer system or a dedicated computer system.

[0031] In detail, the schedule creation program CP is a computer program that creates a time schedule for processing substrates W stored in a subsequent container, which is a substrate storage container CA that is docked to the substrate processing apparatus WP later, and substrates W remaining in the preceding container, by the substrate processing apparatus WP when an interrupt processing occurs when a substrate storage container CA that stores substrates W with a higher priority than the substrates W stored in the preceding container, which is a substrate storage container CA that is docked to the substrate processing apparatus WP earlier, is docked to the substrate processing apparatus WP.

[0032] Hereinafter, a substrate accommodation container CA that docks to the substrate processing apparatus WP before the preceding one may be referred to as a "preceding container," a lot accommodated in the preceding container may be referred to as a "preceding lot," and a substrate W constituting the preceding lot may be referred to as a "preceding substrate." Similarly, a substrate accommodation container CA that docks to the substrate processing apparatus WP after the preceding container docks may be referred to as a "following container," a lot accommodated in the following container may be referred to as a "following lot," and a substrate W constituting the following lot may be referred to as a "following substrate." Furthermore, among the following lots, a lot that has a higher priority than the preceding lot may be referred to as an "interrupt lot," a substrate W constituting the interrupt lot may be referred to as an "interrupt substrate," and a substrate accommodation container CA that accommodates the interrupt lot may be referred to as an "interrupt container." An interrupt lot (interrupt substrate) is a lot (substrate W) that causes interrupt processing.

[0033] In the following description, the time schedule for processing the preceding substrates remaining in the preceding container and the interrupt substrates by the substrate processing apparatus WP when interrupt processing occurs may be referred to as the “interrupt schedule.” Furthermore, the preceding substrates remaining in the preceding container when interrupt processing occurs may be referred to as the “container remaining substrates.”

[0034] As shown in FIG. 1A, the schedule creation program generation device 100 includes an input unit 101, a storage unit 102, an interface unit 103, a display unit 104, and a processing unit 105.

[0035] The interface unit 103 exchanges information, data, or signals with the recording medium 110. Specifically, the interface unit 103 reads the generation program GP from the recording medium 110 and inputs it to the calculation processing unit 105. As a result, the generation program GP is installed in the schedule creation program generation device 100. Furthermore, as shown in FIG. 1B , the interface unit 103 causes the recording medium 110 to carry the schedule creation program CP.

[0036] For example, the interface unit 103 may be electrically connected to the recording medium 110 to input and output information, data, or signals to and from the recording medium 110. For example, the interface unit 103 may include a slot and a USB terminal. A card-shaped information carrier, such as an SD memory card, may be inserted into the slot. For example, a USB memory may be inserted into the USB terminal, or the other end of a USB cable, one end of which is electrically connected to a hard disk drive, may be inserted into the USB terminal. Alternatively, the interface unit 103 may include an optical disk drive. The optical disk drive reads information (data) from a CD (compact disk), a DVD, and / or a Blu-ray disk. The optical disk drive also writes information (data) to a CD, a DVD, and / or a Blu-ray disk.

[0037] The interface unit 103 may receive the generation program GP from another computer system. The interface unit 103 may also transmit the schedule creation program CP to another computer system. For example, the interface unit 103 may be communicatively connected to the other computer system via a cable, or may be communicatively connected to the other computer system via a line network such as the Internet.

[0038] The input unit 101 includes a user interface device operated by an operator. The input unit 101 inputs a signal corresponding to an operation by the operator to the arithmetic processing unit 105. The input unit 101 includes, for example, a keyboard and a mouse. The input unit 101 may include a touch sensor superimposed on the display surface of the display unit 104. A graphical user interface may be configured by superimposing the touch sensor on the display surface of the display unit 104.

[0039] For example, the operator can operate the input unit 101 to instruct the installation of the generation program GP. The operator can operate the input unit 101 to instruct the start of reinforcement learning. The operator can operate the input unit 101 to set a condition for terminating reinforcement learning. For example, a threshold value for the number of times that an interrupt schedule (time schedule) is repeatedly created (tried) may be set as a condition for terminating reinforcement learning. In this case, reinforcement learning ends when the number of times that an interrupt schedule is created (number of trials) reaches the set threshold value. Hereinafter, the threshold value set as a condition for terminating reinforcement learning may be referred to as a "first threshold value."

[0040] In this embodiment, one interruption schedule (time schedule) is created in one reinforcement learning step. In other words, the creation of one interruption schedule completes one reinforcement learning episode. Therefore, the number of times the creation (trial) of the interruption schedule is repeated corresponds to the number of times the reinforcement learning step is repeated.

[0041] The display unit 104 is controlled by the arithmetic processing unit 105 to display various screens. Specifically, the display unit 104 may display a learning curve. The learning curve indicates the relationship between the number of reinforcement learning steps and the cumulative reward. The display unit 104 includes a display device such as a liquid crystal display device or an organic electroluminescence (EL) display device. Note that the display unit 104 may be omitted.

[0042] The storage unit 102 has a main storage device. The main storage device includes, for example, a semiconductor memory. The storage unit 102 may further include an auxiliary storage device. The auxiliary storage device includes, for example, at least one of a semiconductor memory and a hard disk drive. The storage unit 102 may include removable media. The storage unit 102 stores various computer programs and various data. Specifically, the storage unit 102 stores a generation program GP. The storage unit 102 also stores a condition for terminating reinforcement learning. For example, the storage unit 102 stores a first threshold set by an operator operating the input unit 101 as a condition for terminating reinforcement learning.

[0043] As already described, the generating program GP includes a program for reinforcement learning. The reinforcement learning algorithm is not particularly limited, but may be, for example, an algorithm conforming to Q-learning, SARSA, policy gradient method, Actor-Critic method, or Monte Carlo method.

[0044] The neural network for reinforcement learning may include a neural network that performs deep learning. Specifically, the neural network may include a deep neural network (DNN), a deep Q-network (DQN), a transformer, a recurrent neural network (RNN), a convolutional neural network (CNN), or a quantum neural network (QNN). For example, a deep neural network includes an input layer, multiple intermediate layers (hidden layers), and an output layer.

[0045] The arithmetic processing unit 105 includes a processor. The arithmetic processing unit 105 may include, for example, a central processing unit (CPU), a micro processing unit (MPU), a graphics processing unit (GPU), a neural network processing unit (NPU), or a quantum computer. Alternatively, the arithmetic processing unit 105 may include a general-purpose arithmetic device or a dedicated arithmetic device. For example, the arithmetic processing unit 105 may include a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC).

[0046] The processing unit 105 executes the generating program GP stored in the storage unit 102 to generate the schedule creating program CP. The processing unit 105 causes the recording medium 110 to record the schedule creating program CP.

[0047] Specifically, the arithmetic processing unit 105 creates a time schedule for processing multiple substrates W by the substrate processing apparatus WP by arranging multiple plan elements BL, which will be described with reference to Figure 5, in a time table TB, which will be described with reference to Figures 6A and 6B, at each step of reinforcement learning. More specifically, the arithmetic processing unit 105 repeatedly attempts to create an interrupt schedule through reinforcement learning. The multiple substrates W include preceding substrates (container remaining substrates) remaining in the preceding container and interrupt substrates (following substrates) contained in the interrupt container (following container).

[0048] For example, when the calculation processing unit 105 repeatedly attempts to create a time schedule for processing 25 substrates W using the substrate processing device WP under the condition that four planning elements BL are placed in the time table TB for each substrate W, it places 100 planning elements BL in the time table TB at each step of reinforcement learning.

[0049] Furthermore, the calculation processing unit 105 obtains a cumulative reward each time it repeats an attempt to create an interruption schedule, by referring to the final state of the timetable TB and the conditions for granting a reward. The conditions for granting a reward include a first reward condition and a second reward condition. The final state of the timetable TB indicates a timetable TB in which all plan elements BL are arranged. Here, all plan elements BL include multiple plan elements BL assigned to each of the container remaining substrates (preceding substrates) and multiple plan elements BL assigned to each of the interruption substrates (following substrates). In other words, the final state of the timetable TB corresponds to the interruption schedule. The cumulative reward indicates the total value of rewards assigned for the final state of the timetable TB.

[0050] The first remuneration condition indicates that the first remuneration is to be granted when the final state of the timetable TB indicates a state in which processing of at least one of the preceding substrates (container remaining substrates) remaining in the preceding container is completed before processing of the interruption lot (subsequent lot) is completed. The first remuneration is the remuneration with the largest remuneration value. By including the first remuneration condition in the conditions for granting the remuneration, it is possible to avoid stopping processing of the container remaining substrates (preceding substrates) until processing of the interruption lot (subsequent lot) is completed.

[0051] The completion of processing for a substrate W indicates a state in which the substrate W has been processed and has been loaded into the substrate accommodation container CA, and the completion of processing for a lot indicates a state in which substrate processing has been performed on all of the substrates W constituting the lot, and all of the substrates W (processed substrates W) constituting the lot have been loaded into the substrate accommodation container CA.

[0052] The second remuneration condition indicates that the second remuneration is to be granted when the final state of the timetable TB indicates that processing for an interruption lot (a subsequent lot) is completed before processing for a preceding lot is completed. The value of the second remuneration is smaller than the first remuneration. The second remuneration may be the remuneration with the second largest remuneration value. By including the second remuneration condition in the conditions for granting the remuneration, it is possible to create a time schedule (interruption schedule) that prioritizes processing of interruption lots.

[0053] In this embodiment, the conditions for awarding a reward further include a third reward condition and a fourth reward condition. The third reward condition indicates that the earlier the time at which processing for the interrupting lot is completed, the greater the reward value of the third reward is awarded. The fourth reward condition indicates that the earlier the time at which processing for the preceding lot is completed, the greater the reward value of the fourth reward is awarded. The maximum value of the third reward is smaller than the second reward. The maximum value of the fourth reward is smaller than the third reward.

[0054] By including the third reward condition in the conditions for granting rewards, it is possible to create a time schedule (interruption schedule) in which processing for interruption lots is completed in a shorter time. Furthermore, by including the fourth reward condition in the conditions for granting rewards, it is possible to create a time schedule (interruption schedule) in which processing for preceding lots is completed in a shorter time. Note that the third reward condition and the fourth reward condition may be specified by a reward function. The reward function is a function in which the reward value increases the earlier the time when processing for a lot is completed.

[0055] The arithmetic processing unit 105 adjusts the neural network parameters (weighting coefficients) in the process of repeating reinforcement learning steps (trials) so as to maximize the cumulative reward. In other words, the arithmetic processing unit 105 adjusts the neural network parameters so that the interrupt lot (subsequent lot) is processed with priority over the preceding lot without stopping the processing of the substrates remaining in the container (preceding substrates), and so that the processing of the interrupt lot (subsequent lot) and the preceding lot is completed in a shorter time. For example, the neural network parameters are adjusted until the number of trials reaches a first threshold. As a result, a schedule creation program CP (trained model) is generated.

[0056] By using the schedule creation program CP generated in this manner, it is possible to create an interruption schedule (time schedule) in which an interrupting lot (subsequent lot) is processed with priority over the preceding lot without stopping the processing of the substrates remaining in the container (preceding substrates), and in which the processing of the interrupting lot (subsequent lot) and the preceding lot is completed in a shorter time. Therefore, when an interruption occurs in the processing of a lot that has a higher priority than the lot being processed in advance, the processing of the interrupted subsequent lot can be completed before the preceding lot without stopping the processing of the preceding lot.

[0057] Next, an example of a substrate processing apparatus WP to which the schedule creation program CP of the present embodiment is applied will be described with reference to Fig. 2. Fig. 2 is a plan view schematically showing an example of the configuration of a substrate processing apparatus WP to which the schedule creation program CP of the present embodiment is applied.

[0058] The substrate processing apparatus WP is a single-wafer processing apparatus. The substrate processing apparatus WP performs substrate processing on one or more substrates W constituting one lot, one by one, to complete processing for one lot. The number of substrates W constituting one lot is, for example, 1 to 25. The substrate processing apparatus WP includes a container placement unit LPS, a plurality of substrate processing units PU, and a transport unit TR.

[0059] The container platform LPS is capable of holding a plurality of substrate accommodation containers CA. When a substrate accommodation container CA is placed on the container platform LPS, the substrate accommodation container CA is docked with the substrate processing apparatus WP.

[0060] The container mounting part LPS of the substrate processing apparatus WP shown in Fig. 2 includes four load ports LP (first load port LP1 to fourth load port LP4). A substrate accommodation container CA can be placed on each of the load ports LP. In the substrate processing apparatus WP shown in Fig. 2, four substrate accommodation containers CA can be placed on the container mounting part LPS. For example, the first substrate accommodation container CA1 to fourth substrate accommodation container CA4 may be placed on the container mounting part LPS.

[0061] Priorities are assigned in advance to the lots (substrates W) accommodated in the substrate accommodation container CA. The first substrate accommodation container CA1 accommodates lots (substrates W) with priority V. The second substrate accommodation container CA2 accommodates lots (substrates W) with priority X. The third substrate accommodation container CA3 accommodates lots (substrates W) with priority Y. The fourth substrate accommodation container CA4 accommodates lots (substrates W) with priority Z. Here, priority Z indicates the highest priority among the four priorities V, X, Y, and Z. Priority Y indicates the second highest priority among the four priorities V, X, Y, and Z. Priority X indicates the third highest priority among the four priorities V, X, Y, and Z. Priority V indicates the fourth highest priority among the four priorities V, X, Y, and Z. In other words, priority V indicates the lowest priority among the four priorities V, X, Y, and Z.

[0062] Hereinafter, a substrate W constituting a lot of priority V may be referred to as "substrate W1," a substrate W constituting a lot of priority X may be referred to as "substrate W2," a substrate W constituting a lot of priority Y may be referred to as "substrate W3," and a substrate W constituting a lot of priority Z may be referred to as "substrate W4."

[0063] In this embodiment, the processed substrate W is returned to the original substrate accommodation container CA. For example, the processed substrate W1 is accommodated in the first substrate accommodation container CA1. The processed substrate W refers to the substrate W after the substrate processing by the substrate processing unit PU is performed.

[0064] Each substrate storage container CA can store one or more substrates W that make up one lot. The substrate storage container CA may be, for example, a Front Opening Unified Pod (FOUP), a Standard Mechanical Interface (SMIF) pod, or an Open Cassette (OC).

[0065] Each substrate processing unit PU performs substrate processing on a single substrate W. The type of substrate processing is not particularly limited, and for example, the substrate processing unit PU may perform processing on the substrate W using a processing agent (processing liquid or processing gas, or processing liquid and processing gas), processing using electromagnetic waves such as ultraviolet rays, or physical cleaning processing (brush cleaning, spray nozzle cleaning, etc.). For example, the substrate processing unit PU may perform any of the following substrate processing on the substrate W: chemical cleaning processing, brush cleaning processing, wet etching processing, dry etching processing, photosensitive film coating processing, development processing, annealing processing, and drawing processing.

[0066] The transport part TR is capable of transporting substrates W between multiple substrate accommodation containers CA and multiple substrate processing units PU. In the example shown in Figure 2, the substrate processing apparatus WP includes 12 substrate processing units PU (first substrate processing unit PU1 to twelfth substrate processing unit PU12). Therefore, in the substrate processing apparatus WP shown in Figure 2, the transport part TR can transport substrates W between four substrate accommodation containers CA and 12 substrate processing units PU. The transport part TR includes, for example, an indexer robot IR, a transfer part PS, and a transport robot CR.

[0067] The indexer robot IR transports unprocessed substrates W from the substrate storage container CA to the passing part PS. The indexer robot IR also transports processed substrates W from the passing part PS to the substrate storage container CA. For example, the indexer robot IR transports an unprocessed substrate W1 from the first substrate storage container CA1 to the passing part PS, and transports the processed substrate W1 from the passing part PS to the first substrate storage container CA1. The unprocessed substrate W refers to a substrate W before substrate processing is performed by the substrate processing unit PU.

[0068] Specifically, the indexer robot IR has two hands (hands 8A and 8B). Each of the hands 8A and 8B holds one substrate W. More specifically, the hand 8A holds an unprocessed substrate W. The hand 8B holds a processed substrate W.

[0069] The passing part PS has a plurality of shelves for supporting substrates W. Specifically, the passing part PS has at least one shelf for supporting unprocessed substrates W and at least one shelf for supporting processed substrates W. In this embodiment, the passing part PS has one shelf for supporting unprocessed substrates W and one shelf for supporting processed substrates W. Hereinafter, the shelf for supporting unprocessed substrates W may be referred to as "shelf PS1." Furthermore, the shelf for supporting processed substrates W may be referred to as "shelf PS2."

[0070] The transport robot CR transports an unprocessed substrate W from the passing part PS to one of the substrate processing parts PU. The transport robot CR also transports a processed substrate W from the substrate processing part PU to the passing part PS. Substrate processing for one substrate W is performed by one of the substrate processing parts PU.

[0071] Specifically, the transport robot CR has two hands (hands 13A and 13B). Each of the hands 13A and 13B holds one substrate W. More specifically, the hand 13A holds an unprocessed substrate W. The hand 13B holds a processed substrate W.

[0072] As shown in Fig. 2, the multiple substrate processing units PU form multiple towers TW that are arranged to surround the transport robot CR in a plan view. In the example shown in Fig. 2, the substrate processing apparatus WP has four towers TW (first tower TW1 to fourth tower TW4). Each tower TW includes multiple substrate processing units PU stacked one above the other. In the example shown in Fig. 2, each tower TW includes three substrate processing units PU stacked one above the other.

[0073] Specifically, the first substrate processing unit PU1 to the third substrate processing unit PU3 form the first tower TW1, the fourth substrate processing unit PU4 to the sixth substrate processing unit PU6 form the second tower TW2, the seventh substrate processing unit PU7 to the ninth substrate processing unit PU9 form the third tower TW3, and the tenth substrate processing unit PU10 to the twelfth substrate processing unit PU12 form the fourth tower TW4.

[0074] Here, the relationship between a preceding lot (preceding substrate), a succeeding lot (successing substrate), and interrupt processing will be described with reference to Figures 2 to 4(c). Figures 3(a) and 3(b) are diagrams showing an example of the relationship between a preceding lot (preceding substrate), a succeeding lot (successing substrate), and interrupt processing. Figures 4(a), 4(b), and 4(c) are diagrams showing other examples of the relationship between a preceding lot (preceding substrate), a succeeding lot (successing substrate), and interrupt processing.

[0075] 3A, when the second substrate accommodation container CA2 is placed on the container platform LPS while the substrate processing apparatus WP is processing the preceding lot (substrate W1), an interruption occurs to process the following lot (substrate W2) because the priority (priority X) of the following lot (substrate W2) is higher than the priority (priority V) of the preceding lot (substrate W1). Therefore, the following lot (substrate W2) becomes an interrupt lot (interrupt substrate) for the preceding lot (substrate W1).

[0076] 3(b), if an interruption occurs to process substrate W2 and then the third substrate housing container CA3 is placed on the container platform LPS before the processing of the lot housed in the second substrate housing container CA2 is completed, i.e., before the processing of all substrates W2 is completed, substrate W2 becomes a preceding substrate (preceding lot) relative to substrate W3. Also, because the priority (priority Y) of the succeeding lot (substrate W3) is higher than the priority (priority X) of the preceding lot (substrate W2), an interruption occurs to process the succeeding lot (substrate W3). Therefore, the succeeding lot (substrate W3) becomes an interrupt lot (interrupt substrate) relative to the preceding lots (substrates W1 and W2).

[0077] 4A, when the third substrate accommodation container CA3 is placed on the container platform LPS while the substrate processing apparatus WP is processing the preceding lot (substrate W1), an interruption occurs to process the following lot (substrate W3) because the priority (priority Y) of the following lot (substrate W3) is higher than the priority (priority V) of the preceding lot (substrate W1). Therefore, the following lot (substrate W3) becomes an interrupt lot (interrupt substrate) for the preceding lot (substrate W1).

[0078] 4(b), if an interruption to process substrate W3 occurs and then the second substrate housing container CA2 is placed on the container rest part LPS before the processing of the lot contained in the third substrate housing container CA3 is completed, i.e., before the processing of all substrates W3 is completed, substrate W3 becomes the preceding substrate (preceding lot) with respect to substrate W2. On the other hand, because the priority (priority X) of the succeeding lot (substrate W2) is lower than the priority (priority Y) of the preceding lot (substrate W3), no interruption to process the succeeding lot (substrate W2) occurs.

[0079] 4(c), when processing of the lot accommodated in the third substrate accommodation container CA3 is completed, that is, when processing of all substrates W3 is completed, an interruption occurs to process the subsequent lot (substrate W2) because the priority (priority X) of the subsequent lot (substrate W2) is higher than the priority (priority V) of the preceding lot (substrate W1). Therefore, the subsequent lot (substrate W2) becomes an interrupt lot (interrupt substrate) for the preceding lot (substrate W1).

[0080] 2 to 4C, an interrupt process occurs when the priority of a lot accommodated in a substrate accommodation container CA that is subsequently placed on the container platform part LPS is higher than the priority of a lot accommodated in a substrate accommodation container CA that was previously placed on the container platform part LPS. On the other hand, when the priority of a lot accommodated in a substrate accommodation container CA that is subsequently placed on the container platform part LPS is lower than the priority of a lot accommodated in a substrate accommodation container CA that was previously placed on the container platform part LPS, an interrupt process does not occur. In this embodiment, when an interrupt process occurs, a time schedule (interrupt schedule) is created for processing at least one preceding substrate while processing a subsequent lot is being performed.

[0081] For example, as shown in Figure 3(b), if the first substrate storage container CA1, the second substrate storage container CA2, and the third substrate storage container CA3 are placed on the container loading section LPS in that order, and an interrupt occurs to process the following substrate W3, a time schedule (interrupt schedule) is created to process at least one of the preceding substrates W1 and at least one of the preceding substrates W2 before processing of all substrates W3 is completed.

[0082] Next, the processing procedure PD, processing time PT, and plan element BL will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of the processing procedure PD, processing time PT, and plan element BL. In detail, Fig. 5 shows the processing procedure PD, processing time PT, and plan element BL corresponding to the substrate processing apparatus WP of Fig. 2. Information indicating the processing procedure PD, information indicating the processing time PT, and information indicating the plan element BL are associated with each other and stored in the storage unit 102 described with reference to Figs. 1A and 1B.

[0083] The processing procedure PD indicates the procedure of processing performed by the substrate processing apparatus WP. As shown in Fig. 5, the processing procedure PD corresponding to the substrate processing apparatus WP in Fig. 2 includes processing pattern A to processing pattern M. Processing pattern A to processing pattern M are performed in this order on one substrate W. The processing procedure PD indicates the flow of processing (processing flow) performed on one substrate W. In a time table TB, which will be described with reference to Figs. 6A and 6B, processing pattern A to processing pattern M are arranged in this order along the time axis of the time table TB.

[0084] Processing pattern A shows a process in which the indexer robot IR takes out an unprocessed substrate W from a substrate storage container CA. Processing pattern B shows a process in which the indexer robot IR transports an unprocessed substrate W and loads the unprocessed substrate W into the transfer part PS. While processing pattern A and processing pattern B are being executed, the hand 8A of the indexer robot IR is occupied by one substrate W.

[0085] Processing pattern C indicates a process in which an unprocessed substrate W is carried into the transfer part PS. Processing pattern D indicates a process in which an unprocessed substrate W is carried out from the transfer part PS. While processing pattern C and processing pattern D are being executed, shelf PS1 of the transfer part PS is occupied by one substrate W.

[0086] Processing pattern E shows a process in which the transport robot CR takes out an unprocessed substrate W from the transfer part PS. Processing pattern F shows a process in which the transport robot CR transports an unprocessed substrate W and carries it into the substrate processing unit PU. While processing pattern E and processing pattern F are being executed, the hand 13A of the transport robot CR is occupied by one substrate W.

[0087] The processing pattern G indicates that the substrate processing unit PU performs substrate processing. While the processing pattern G is being performed, the substrate processing unit PU is occupied by one substrate W.

[0088] Processing pattern H indicates a process in which the transport robot CR unloads the processed substrate W from the substrate processing unit PU. Processing pattern I indicates a process in which the transport robot CR transports the processed substrate W and loads it into the transfer part PS. While processing pattern H and processing pattern I are being executed, the hand 13B of the transport robot CR is occupied by one substrate W.

[0089] Processing pattern J indicates a process in which a processed substrate W is carried into the passing part PS. Processing pattern K indicates a process in which a processed substrate W is carried out from the passing part PS. While processing pattern J and processing pattern K are being executed, shelf PS2 of the passing part PS is occupied by one substrate W.

[0090] Processing pattern L indicates a process in which the indexer robot IR takes out the processed substrate W from the transfer part PS. Processing pattern M indicates a process in which the indexer robot IR transports the processed substrate W and loads it into a substrate accommodation container CA. While processing pattern L and processing pattern M are being executed, the hand 8B of the indexer robot IR is occupied by one substrate W.

[0091] As explained above with reference to Fig. 5, one substrate W sequentially occupies a plurality of components included in the substrate processing apparatus WP. In this manner, processing patterns A to M indicate occupancy information for one substrate W. In the substrate processing apparatus WP of Fig. 2, one substrate W sequentially occupies any one of the hands 8A and 8B of the indexer robot IR, the shelves PS1 and PS2 of the transfer part PS, the hands 13A and 13B of the transport robot CR, and the substrate processing unit PU.

[0092] Next, the processing time PT will be described. The processing time PT indicates the duration of each process performed by the substrate processing apparatus WP. More specifically, the processing time PT indicates the duration of a process performed by each component included in the substrate processing apparatus WP. In other words, the processing time PT indicates the duration over which each component included in the substrate processing apparatus WP is occupied by one substrate W.

[0093] 5, the processing time PT corresponding to the substrate processing apparatus WP in FIG. 2 includes processing times X1 to X13. Processing times X1 to X13 indicate the time lengths of processing patterns A to M, respectively. Processing times X1 to X13 are associated with processing patterns A to M, respectively, and stored in the storage unit 102 described with reference to FIGS. 1A and 1B. Of processing times X1 to X13, processing times X1 to X6 and processing times X8 to X13 indicate the time lengths required for transporting the substrate W. Processing time X7 indicates the time length required for substrate processing. The time lengths of processing times X1 to X6 and processing times X8 to X13 are shorter than processing time X7.

[0094] Next, the plan elements BL will be described. As shown in Fig. 5, the processing procedure PD is divided into a plurality of plan elements BL. Each plan element BL includes at least one of the processing patterns included in the processing procedure PD. In the example shown in Fig. 5, processing patterns A to M are divided into four plan elements BL (first plan element BL1 to fourth plan element BL4). Each plan element BL indicates a component that is continuously occupied by one substrate W. The plurality of plan elements BL are given in advance for the processing procedure PD. In other words, a plurality of plan elements BL are given in advance for one substrate W.

[0095] The first plan element BL1 includes processing patterns A to C. That is, the first plan element BL1 indicates a plan for transporting an unprocessed substrate W from a substrate storage container CA to a shelf PS1 of the transfer part PS by the indexer robot IR. The first plan element BL1 also indicates that one substrate W successively occupies the hand 8A of the indexer robot IR and the shelf PS1 of the transfer part PS.

[0096] The second plan element BL2 includes processing patterns D to G. In other words, the second plan element BL2 indicates a plan in which an unprocessed substrate W is transported by the transport robot CR from the shelf PS1 of the transfer unit PS to the substrate processing unit PU, and then processed by the substrate processing unit PU. The second plan element BL2 also indicates that one substrate W consecutively occupies the shelf PS1 of the transfer unit PS, the hand 13A of the transport robot CR, and any one of the plurality of substrate processing units PU.

[0097] The third plan element BL3 includes processing patterns H to J. That is, the third plan element BL3 indicates a plan for transporting a processed substrate W from the substrate processing unit PU to the shelf PS2 of the transfer unit PS by the transport robot CR. In other words, the third plan element BL3 indicates that one substrate W successively occupies the hand 13B of the transport robot CR and the shelf PS2 of the transfer unit PS.

[0098] The fourth plan element BL4 includes processing patterns K to M. That is, the fourth plan element BL4 indicates a plan for transporting a processed substrate W from the shelf PS2 of the transfer part PS to a substrate accommodation container CA. The fourth plan element BL4 also indicates that one substrate W successively occupies the shelf PS2 of the transfer part PS and the hand 8B of the indexer robot IR.

[0099] 1A and 1B arranges a plurality of plan elements BL in a time table TB in accordance with the order of the processing procedures PD. More specifically, the calculation processing unit 105 arranges process patterns A to M in the time table TB for each plan element BL in accordance with the order of the processing procedures PD. By arranging process patterns A to M in the time table TB for each plan element BL, physically impossible actions can be prohibited in the substrate processing apparatus WP.

[0100] Next, the constraints will be explained. Table 1 below shows an example of the constraints corresponding to the substrate processing apparatus WP in FIG. 2. The constraints indicate conditions for arranging the plan elements BL in the timetable TB. The constraints include conditions that depend on the configuration of the substrate processing apparatus WP. For example, the constraints include a condition that prohibits physically impossible actions in the substrate processing apparatus WP.

[0101] 1A and 1B arranges multiple plan elements BL (processing pattern A to processing pattern M) in a timetable TB by referring to constraint conditions (constraint conditions 1 to 10). The constraint conditions are stored in the storage unit 102.

[0102] As shown in Table 1, the constraints include constraint 10. Constraint 10 indicates that each substrate processing unit PU restricts the substrates W to be processed by the unit using a priority. Constraint 10 allocates the number of substrate processing units PU assigned to an interrupt lot and the number of substrate processing units PU assigned to a preceding lot in accordance with the priority, the number of substrates W constituting the interrupt lot (interrupt substrates), and the number of substrates W remaining in the preceding container (container remaining substrates).

[0103] For example, when creating an interrupt schedule corresponding to the state in which two preceding containers and an interrupt container are placed on the container platform LPS as shown in Figure 3(b), the calculation processing unit 105 shown in Figures 1A and 1B creates an interrupt schedule in which the multiple substrate processing units PU are allocated to a substrate processing unit PU that processes substrates on the interrupt lot (substrates W3), a substrate processing unit PU that processes substrates on the first preceding lot (substrates W1), and a substrate processing unit PU that processes substrates on the second preceding lot (substrates W2) in accordance with constraint 10. In other words, the multiple substrate processing units PU are divided into three groups.

[0104] More specifically, the substrates W that each substrate processing unit PU processes may be restricted based on the priority of the substrate W that is first carried into each substrate processing unit PU after the occurrence of interrupt processing.

[0105] For example, when creating an interrupt schedule corresponding to a state in which two preceding containers and an interrupt container are placed on the container placement part LPS as shown in Figure 3(b), the calculation processing part 105 shown in Figures 1A and 1B may create an interrupt schedule in which the substrate processing part PU whose first substrate W to be loaded after the interrupt processing occurs is the interrupt substrate (substrate W3) is restricted to the substrate processing part PU that performs substrate processing on the interrupt substrate (substrate W3), the substrate processing part PU whose first substrate W to be loaded after the interrupt processing occurs is the first preceding substrate (substrate W1) is restricted to the substrate processing part PU that performs substrate processing on the first preceding substrate (substrate W1), and the substrate processing part PU whose first substrate W to be loaded after the interrupt processing occurs is the second preceding substrate (substrate W2) is restricted to the substrate processing part PU that performs substrate processing on the second preceding substrate (substrate W2).

[0106] 1A to 5 and Table 1, when arranging multiple plan elements BL in the timetable TB, the calculation processing unit 105 refers to information indicating the processing procedure PD, information indicating the processing time PT, information indicating the plan elements BL, and constraints. Furthermore, when arranging multiple plan elements BL in the timetable TB, the calculation processing unit 105 refers to number information indicating the number of substrates W to be processed and information related to substrate processing. The information related to substrate processing indicates the substrate processing procedure to be performed on the substrates W to be processed and the processing conditions for that substrate processing.

[0107] Specifically, when arranging multiple plan elements BL in the timetable TB, the calculation processing unit 105 refers to number information indicating the number of unprocessed substrates W remaining in the preceding container (container remaining substrates) and the number of unprocessed interrupt substrates (substrates W housed in the interrupt container). The number information indicates the number of unprocessed substrates W remaining in each preceding container (container remaining substrates) when creating an interrupt schedule corresponding to a state in which multiple preceding containers are placed on the container platform LPS. Note that the container remaining substrates correspond to the unprocessed substrates W remaining in the preceding container at the time of interrupt processing, among the preceding substrates.

[0108] Furthermore, the calculation processing unit 105 refers to information regarding substrate processing to be performed on remaining substrates in the container (preceding lot) and information regarding substrate processing to be performed on interrupt substrates (interrupt lot). When creating an interrupt schedule corresponding to a state in which multiple preceding containers are placed on the container placement part LPS, the information regarding substrate processing indicates information regarding each of the substrate processing to be performed on each of the preceding lots.

[0109] The information regarding the substrate processing to be performed on the container remaining substrates (previous lot) indicates the procedure of the substrate processing to be performed on the container remaining substrates and the processing conditions for the substrate processing. The information regarding the substrate processing to be performed on the interruption substrates (interruption lot) indicates the procedure of the substrate processing to be performed on the interruption substrates and the processing conditions for the substrate processing.

[0110] The substrate processing procedures performed for each lot may be the same or different. Furthermore, some of the substrate processing procedures performed for each lot may be the same, or some of the substrate processing procedures performed for each lot may be different. Similarly, the processing conditions for the substrate processing performed for each lot may be the same or different. Furthermore, some of the processing conditions for the substrate processing performed for each lot may be the same, or some of the processing conditions for the substrate processing performed for each lot may be different.

[0111] The calculation processing unit 105 refers to the information on the number of remaining substrates in the container and arranges in the time table TB the number of plan elements BL corresponding to the number of unprocessed substrates W remaining in the preceding container (substrates remaining in the container), and refers to the information on the number of interrupt substrates and arranges in the time table TB the number of plan elements BL corresponding to the number of interrupt substrates. When creating an interrupt schedule corresponding to a state in which a plurality of preceding containers are placed on the container mounting part LPS, the calculation processing unit 105 arranges in the time table TB the number of plan elements BL corresponding to the number of unprocessed substrates W remaining in each preceding container (substrates remaining in the container).

[0112] For example, under the condition that four planning elements BL are placed in the time table TB for each substrate W, if the total number of substrates remaining in the container and the number of interrupt substrates is 25, the calculation processing unit 105 places 100 planning elements BL in the time table TB to create an interrupt schedule (time schedule).

[0113] Next, the timetable TB will be described with reference to FIGS. 1A to 6B. FIGS. 6A and 6B are diagrams showing an example of the timetable TB corresponding to the substrate processing apparatus WP of FIG. 2. FIG. 6A shows a portion of the timetable TB, and FIG. 6B shows the remainder of the timetable TB. As shown in FIGS. 6A and 6B, a plurality of plan elements BL are arranged in the timetable TB. Each of the plan elements BL is arranged in the timetable TB and exclusively occupies at least one of a plurality of components included in the substrate processing apparatus WP. By arranging the plurality of plan elements BL in the timetable TB, a time schedule for processing a plurality of substrates W by the substrate processing apparatus WP is defined.

[0114] Specifically, the horizontal axis of the timetable TB represents time. The timetable TB shown in FIGS. 6A and 6B represents time t1 to time t16. In the timetable TB, processing patterns corresponding to the components of the substrate processing apparatus WP are arranged along the time axis of the timetable TB. In the timetable TB shown in FIGS. 6A and 6B, the first to fourth planning elements BL1 to BL4 (processing patterns A to M) described with reference to FIG. 5 are arranged based on constraints 1 to 10 shown in Table 1. The timetable TB is, for example, a Gantt chart. Note that processing patterns A to M arranged in the timetable TB may be numerical information or image information.

[0115] In detail, the calculation processing unit 105 adds information indicating the priority of the substrate W to the first plan element BL1 to the fourth plan element BL4 (processing pattern A to processing pattern M), and then arranges the first plan element BL1 to the fourth plan element BL4 in the timetable TB.

[0116] 6A and 6B show an example of a timetable TB in which the first plan element BL1 to the fourth plan element BL4 (processing pattern A to processing pattern M) assigned to the first substrate W4 (priority Z) and the first plan element BL1 to the fourth plan element BL4 (processing pattern A to processing pattern M) assigned to the second substrate W4 (priority Z) are arranged. Specifically, the examples shown in FIGS. 6A and 6B indicate that "W4" has "priority Z."

[0117] 6A and 6B, for ease of understanding, the order of the substrates W4 to be processed is indicated. Specifically, in the example shown in Figures 6A and 6B, "(1)" after "W4" indicates the first plan element BL1 to the fourth plan element BL4 (processing pattern A to processing pattern M) assigned to the first substrate W4, and "(2)" after "W4" indicates the first plan element BL1 to the fourth plan element BL4 (processing pattern A to processing pattern M) assigned to the second substrate W4.

[0118] As shown in Figures 6A and 6B, the substrate processing apparatus WP can perform substrate processing on two or more substrates W in parallel. Specifically, two or more substrate processing units PU can perform substrate processing in parallel. The example shown in Figures 6A and 6B shows a time schedule in which the second substrate processing unit PU2 and the fifth substrate processing unit PU5 perform substrate processing in parallel. Note that while the example shown in Figures 6A and 6B shows a time schedule in which substrate processing is performed in parallel on two substrates W of the same priority, the interrupt schedule may also show a time schedule in which substrate processing is performed in parallel on two or more substrates W of different priorities.

[0119] Next, the process of creating an interrupt schedule (time schedule) will be described with reference to Figures 1A to 7. Figure 7 is a diagram showing an example of the flow of the process of creating an interrupt schedule (time schedule). In more detail, Figure 7 shows the process of arranging multiple plan elements BL in a time table TB. Hereinafter, the process of arranging multiple plan elements BL in a time table TB may be referred to as the "time schedule creation process."

[0120] As shown in FIG. 7 , the calculation processing unit 105 refers to the current state of the timetable TB and the multiple plan elements BL to acquire plan elements BL that have not been placed in the timetable TB (step S1). Hereinafter, the unplaced plan elements BL may be referred to as "plan elements NBL." At the start of the time schedule creation process, all plan elements BL assigned to each substrate W to be processed are plan elements NBL. For example, under the condition that four plan elements BL are placed in the timetable TB for each substrate W, and the total number of substrates W to be processed is 25, the calculation processing unit 105 acquires 100 plan elements NBL at the start of the time schedule creation process. As the time schedule creation process progresses, the number of plan elements NBL decreases. The time schedule creation process continues until there are no more plan elements NBL.

[0121] For example, when creating an interrupt schedule for a state in which two preceding containers and one interrupt container are placed on the container placement unit LPS as illustrated in Figure 3 (b), the calculation processing unit 105, at the start of the time schedule creation process, acquires, as planning elements NBL, all planning elements BL assigned to each of all substrates W1 remaining in the first preceding container, all planning elements BL assigned to each of all substrates W2 remaining in the second preceding container, and all planning elements BL assigned to each of all substrates W contained in the interrupt container.

[0122] When the calculation processing unit 105 acquires the plan element NBL, it references the current state of the timetable TB, the plan element NBL, the constraints (constraints 1 to 10), the processing procedure PD, the processing time PT, information about the substrate processing to be performed on each lot, and information indicating the priority of each lot, and acquires a plan element BL that can be placed next from among the plan elements NBL. Hereinafter, the plan element BL that can be placed next is sometimes referred to as a "plan element ABL."

[0123] When the calculation processing unit 105 acquires the plan element ABL, it refers to the current state of the timetable TB, the plan element ABL, the constraints (constraints 1 to 10), the processing procedure PD, the processing time PT, information about the substrate processing to be performed on each lot, and information indicating the priority of each lot, and calculates the possible allocation time for each plan element ABL (step S2). The possible allocation time corresponds to the time indicated by the timetable TB. More specifically, the possible allocation time indicates the time on the timetable TB at which the processing included in the plan element ABL can be started.

[0124] After calculating the possible placement time, the calculation processing unit 105 generates an action AC for each plan element ABL (step S3). The action AC indicates an action to place the plan element ABL at the possible placement time in the timetable TB.

[0125] After generating the actions AC, the calculation processing unit 105 selects one of the action ACs (step S4), and arranges the plan element BL (plan element ABL) in the timetable TB based on the selected action AC (step S5). As a result, the timetable TB is updated. In other words, the timetable TB becomes the following state.

[0126] Here, a first example of the process of selecting an action AC will be described. For example, in the early stage of reinforcement learning, the calculation processing unit 105 randomly selects one of the action ACs. When the number of times that reinforcement learning steps (trials) are repeated reaches a second threshold, the calculation processing unit 105 predicts and selects, from among the action ACs, the action AC that will maximize the cumulative reward. Note that one step (one trial) indicates the process from the start of the time schedule creation process until all plan elements BL are placed in the time table TB and the cumulative reward is obtained.

[0127] Specifically, the generation program GP includes a behavior selection neural network 121 (see FIG. 8). The behavior selection neural network 121 includes a neural network that constructs a predictor. The predictor calculates an evaluation value (e.g., Q value) for each action AC. The evaluation value indicates the expected value of the cumulative reward. The calculation processing unit 105 (behavior selection neural network 121) selects the action AC with the largest evaluation value (expected value of the cumulative reward).

[0128] Note that the arithmetic processing unit 105 may randomly select one of the action ACs based on a predetermined selection condition after the number of times the reinforcement learning step (trial) has been repeated reaches or exceeds a second threshold. For example, the selection condition may indicate a period for randomly selecting one of the action ACs. In this case, the arithmetic processing unit 105 randomly selects one of the action ACs periodically.

[0129] Alternatively, the selection condition may indicate the timing for randomly selecting one of the actions AC by the number of steps (number of trials). Hereinafter, the number of steps that specifies the timing for randomly selecting one of the actions AC may be referred to as the "number of random selection steps."

[0130] The random selection step count may indicate a number of steps. In this case, the calculation processing unit 105 randomly selects one of the actions AC every time the number of times the reinforcement learning steps have been repeated (the number of trials) reaches the number of steps included in the random selection step count.

[0131] Next, a second example of the process of selecting an action AC will be described. For example, the calculation processing unit 105 acquires a random number and determines whether the value of the acquired random number is equal to or greater than a third threshold. If the value of the random number is equal to or greater than the third threshold, the calculation processing unit 105 predicts and selects, from among the action ACs, an action AC that will maximize the cumulative reward. If the value of the acquired random number is less than the third threshold, the calculation processing unit 105 randomly selects one of the action ACs. The calculation processing unit 105 decreases the third threshold as the number of steps (number of trials) increases.

[0132] When the timetable TB is updated, the calculation processing unit 105 executes the processes of steps S1 to S5 again. The calculation processing unit 105 repeats the processes of steps S1 to S5 until there are no more plan elements NBL. As a result, the time schedule creation process ends.

[0133] When the time schedule creation process is completed, the calculation processing unit 105 obtains the accumulated reward based on the final state of the timetable TB and the conditions for granting the reward (step S6). As already explained, the conditions for granting the reward include the first reward condition and the second reward condition. In this embodiment, the conditions for granting the reward further include the third reward condition and the fourth reward condition.

[0134] Next, the reinforcement learning system 120 constructed by the execution of the generation program GP by the arithmetic processing unit 105 will be described with reference to Fig. 8. Fig. 8 is a block diagram showing the reinforcement learning system 120.

[0135] As shown in FIG. 8 , the reinforcement learning system 120 selects an action for the agent AG. The action of the agent AG causes an interaction between the agent AG and the environment EB. Specifically, the action of the agent AG includes an action AC described with reference to FIG. 7 . The environment EB includes a timetable TB described with reference to FIGS. 6A and 6B . The reinforcement learning system 120 selects the action AC as described with reference to FIG. 7 . The agent AG places a plan element BL in the timetable TB based on the selected action AC.

[0136] The reinforcement learning system 120 includes an action selection neural network 121 and a training engine 122. The training engine 122 includes a replay memory 123. The replay memory 123 is included in the storage area of ​​the storage unit 102 described with reference to Figures 1A and 1B.

[0137] As already explained, the behavior selection neural network 121 calculates an evaluation value (e.g., Q value) for each action AC and selects the action AC with the largest evaluation value. The training engine 122 trains the behavior selection neural network 121 and adjusts multiple parameters (weighting coefficients) included in the behavior selection neural network 121.

[0138] More specifically, the training engine 122 stores the experience data in the replay memory 123. The experience data indicates the results of the interaction of the agent AG with the environment EB. The experience data is learning data (training data) for supervised learning, and the training engine 122 trains the action selection neural network 121 by having the action selection neural network 121 learn the experience data. As a result, multiple parameters (weighting coefficients) included in the action selection neural network 121 are adjusted, and the prediction accuracy of the action selection neural network 121 is improved.

[0139] Specifically, the training engine 122 (the calculation processing unit 105) generates experience data every time a plan element BL is placed in the timetable TB and stores the experience data in the replay memory 123. The experience data includes the current state of the timetable TB, the current plan element NBL (unassigned plan element BL), the current available time for placement, the selected action AC (selected plan element ABL), and the state of the next timetable TB. The experience data upon completion of creation of the time schedule (interrupt schedule) further includes the value of the accumulated reward.

[0140] The training engine 122 (the processing unit 105) trains the behavior selection neural network 121 based on predetermined training timing conditions. For example, the training timing conditions may indicate a period for training the behavior selection neural network 121. In this case, the training engine 122 (the processing unit 105) periodically trains the behavior selection neural network 121 by having the behavior selection neural network 121 learn experience data.

[0141] Alternatively, the training timing condition may indicate the timing for training the behavior selection neural network 121 in terms of the number of steps. Hereinafter, the number of steps that specifies the timing for training the behavior selection neural network 121 may be referred to as the "number of training steps." The number of training steps indicates a number of steps. In this case, the training engine 122 (arithmetic processing unit 105) trains the behavior selection neural network 121 by having the behavior selection neural network 121 learn empirical data each time the number of times the reinforcement learning steps have been repeated (the number of trials) reaches the number of steps included in the number of training steps.

[0142] Next, a schedule creation program generation method of this embodiment will be described with reference to Figures 1A to 12. Figure 9 is a flowchart showing the schedule creation program generation method of this embodiment. The schedule creation program generation method of this embodiment is executed by the schedule creation program generation device 100 described with reference to Figures 1A to 8. Specifically, the schedule creation program generation method of this embodiment is executed by the calculation processing unit 105 described with reference to Figures 1A to 8. Therefore, Figure 9 shows the flow of processing executed by the calculation processing unit 105.

[0143] 9, the schedule creation program generation method of this embodiment includes steps S11 to S13. For example, in a state where the generation program GP is installed in the schedule creation program generation device 100, an operator may operate the input unit 101 to instruct the start of reinforcement learning using the generation program GP, and the process shown in FIG.

[0144] 9 starts, the arithmetic processing unit 105 executes a time schedule creation process to create an interrupt schedule (time schedule) (step S11). The time schedule creation process (processing for creating an interrupt schedule) will be described later with reference to FIG.

[0145] After creating the interrupt schedule, the calculation processing unit 105 determines the reward to be awarded for the final state of the timetable TB based on the final state of the timetable TB and the conditions for awarding the reward, and obtains the accumulated reward (step S12). The final state of the timetable TB represents the created interrupt schedule. As already explained, the final state of the timetable TB represents the timetable TB in which all plan elements BL have been arranged. All plan elements BL include multiple plan elements BL (first plan element BL1 to fourth plan element BL4) assigned to each of the container remaining substrates and multiple plan elements BL (first plan element BL1 to fourth plan element BL4) assigned to each of the interrupt substrates. Step S12 is an example of a "reward determination step."

[0146] As already explained, the conditions for providing a reward include the first to fourth reward conditions. The first to fourth reward conditions may be input by an operator operating the input unit 101 before the start of reinforcement learning using the generation program GP, or may be included in the generation program GP.

[0147] After creating the interruption schedule and obtaining the cumulative reward, the calculation processing unit 105 determines whether to end the reinforcement learning (step S13). If the calculation processing unit 105 determines not to end the reinforcement learning (No in step S13), the process shown in Fig. 9 returns to step S11. On the other hand, if the calculation processing unit 105 determines to end the reinforcement learning (Yes in step S13), the process shown in Fig. 9 ends. As a result, a schedule creation program CP is created.

[0148] 1A and 1B, a first threshold may be set as a condition for terminating reinforcement learning. In this case, the arithmetic processing unit 105 determines to terminate reinforcement learning when the number of times the reinforcement learning steps have been repeated (the number of trials) becomes equal to or greater than the first threshold.

[0149] The schedule creation program CP is a computer program that creates an interrupt schedule (time schedule) in response to a substrate storage container CA (following container) containing a lot of a second priority, which is higher than the first priority, being placed on the container loading part LPS when some of the substrates W that make up a lot of a first priority remain in the substrate storage container CA (leading container) that is placed on the container loading part LPS in advance.

[0150] As described with reference to Figures 7 and 8, the calculation processing unit 105 adjusts the parameters (weighting coefficients) of the action selection neural network 121 in the process of repeating the time schedule creation process. Therefore, as the reinforcement learning steps (trials) are repeated, the value of the final reward increases. In this embodiment, one reinforcement learning step (one trial) includes steps S11 to S13 shown in Figure 9. Of steps S11 to S13, the step including steps S11 and S12 is an example of an "experience step."

[0151] Next, the time schedule creation process (step S11) shown in Fig. 9 will be described with reference to Fig. 10. Fig. 10 is a flowchart showing the time schedule creation process. As shown in Fig. 10, the time schedule creation process includes steps S21 to S25.

[0152] When the calculation processing unit 105 starts the time schedule creation process, it determines the preceding lot conditions (step S21). The preceding lot conditions include the number of preceding lots (preceding containers), the number of container-remaining substrates (substrates W remaining in the preceding container), the number of substrates remaining (positioned) inside the substrate processing apparatus WP, information about the substrate processing to be performed on the preceding lot (substrate processing procedure and processing conditions for that substrate processing), and the priority of the preceding lot. Hereinafter, substrates W remaining (positioned) inside the substrate processing apparatus WP may be referred to as "substrates remaining in the apparatus."

[0153] The respective numbers of container-remaining substrates and apparatus-remaining substrates may be randomly generated by the arithmetic processing unit 105 within a range in which the total number of container-remaining substrates and apparatus-remaining substrates for one preceding container is, for example, 25 or less. Alternatively, the respective numbers of container-remaining substrates and apparatus-remaining substrates may be randomly selected by the arithmetic processing unit 105 for each preceding container from among combination patterns of the numbers of container-remaining substrates and the numbers of apparatus-remaining substrates. Information indicating combination patterns of the numbers of container-remaining substrates and the numbers of apparatus-remaining substrates may be included in advance in the generation program GP, or may be input by the operator operating the input unit 101.

[0154] In addition, when the number of preceding containers indicates a plurality, the preceding lot conditions include the number of substrates remaining in each preceding container, the number of substrates remaining in the device for each preceding container, and the priority of each preceding container (preceding lot). Furthermore, the arithmetic processing unit 105 determines information regarding substrate processing for each preceding container (preceding lot) as the preceding lot conditions. The arithmetic processing unit 105 may randomly generate the number of preceding containers, the priority of each preceding lot, and information regarding the substrate processing to be performed on the preceding lot. In the case where the number of preceding containers indicates a plurality, the arithmetic processing unit 105 generates different priorities for each preceding container (preceding lot).

[0155] After determining the preceding lot conditions, the arithmetic processing unit 105 determines the succeeding lot conditions (step S22). The succeeding lot conditions indicate the number of interruption substrates, information regarding the substrate processing to be performed on the interruption lot (the procedure for the substrate processing and the processing conditions for the substrate processing), and the priority of the interruption lot. The arithmetic processing unit 105 may randomly generate the number of interruption substrates and the information regarding the substrate processing to be performed on the interruption lot. The priority of the interruption lot is determined to be higher than the priority of the preceding lot.

[0156] After determining the subsequent lot conditions, the calculation processing unit 105 acquires a timetable TB (step S23). Specifically, the calculation processing unit 105 creates a timetable TB in which the plan elements BL assigned to each of the substrates W remaining in the substrate processing apparatus WP (apparatus remaining substrates) are arranged. Note that information on a timetable TB in which no plan elements BL are arranged (an empty timetable TB) may be included in the generation program GP or may be stored in advance in the storage unit 102.

[0157] Specifically, the calculation processing unit 105 acquires multiple plan elements BL (first plan element BL1 to fourth plan element BL4) for each remaining substrate in the equipment by referring to the information indicating the number of remaining substrates in the equipment determined in step S21. Then, the calculation processing unit 105 randomly arranges the multiple plan elements BL assigned to each remaining substrate in the equipment on the time table TB by referring to the constraints (constraints 1 to 10) shown in Table 1 and information related to the substrate processing to be performed on the preceding lot.

[0158] More specifically, when some of the substrates W remain in a substrate storage container CA that has been placed prior to the container loading section LPS, and a substrate storage container CA that stores an interrupt lot is placed on the container loading section LPS, the other substrates W that make up the preceding lot and that do not remain in the preceding container are either located (remain) inside the substrate processing apparatus WP or are transported outside the substrate processing apparatus WP.

[0159] A substrate W (substrate remaining in the device) located (remaining) inside the substrate processing apparatus WP corresponds to either a processed preceding substrate before being transported outside the substrate processing apparatus WP, an unprocessed preceding substrate transported from a preceding container, or a preceding substrate being transported into the substrate processing unit PU.

[0160] Note that a processed substrate W before being unloaded outside the substrate processing apparatus WP refers to a preceding substrate that has undergone substrate processing and been unloaded from the substrate processing unit PU but has not yet been unloaded outside the substrate processing apparatus WP. An unprocessed substrate W unloaded from a preceding container refers to a preceding substrate that has been unloaded from the preceding container but has not yet been loaded into the substrate processing unit PU. A substrate W loaded into the substrate processing unit PU refers to a preceding substrate that is currently being processed by the substrate processing unit PU.

[0161] The calculation processing unit 105 randomly selects the planning elements BL assigned to each of the remaining substrates in the equipment and places them on the timetable TB, by referring to the constraints (constraints 1 to 10) shown in Table 1 and information regarding the substrate processing to be performed on the preceding lot, so that the state of the remaining substrates in the equipment becomes one of the three states described above.

[0162] Steps S21 to S23 are executed each time the reinforcement learning step (trial) is repeated. That is, each time the reinforcement learning step (trial) is repeated, the process of determining the preceding lot conditions, the process of determining the succeeding lot conditions, and the process of acquiring the time table TB are repeated.

[0163] After acquiring the timetable TB, the calculation processing unit 105 sequentially arranges a plurality of plan elements BL in the timetable TB based on the preceding lot conditions, the succeeding lot conditions, information indicating the processing procedure PD, information indicating the processing time PT, information indicating the plan elements BL, and the constraint conditions (constraint conditions 1 to 10) shown in Table 1, thereby sequentially changing the state of the timetable TB (steps S24 and S25). Steps S24 and S25 are examples of an "arrangement step."

[0164] Specifically, the calculation processing unit 105 references information on the number of remaining substrates in the container to acquire a plurality of plan elements BL to be assigned to each of the remaining substrates in the container, and references information on the number of interrupt substrates to acquire a plurality of plan elements BL to be assigned to each of the interrupt substrates. The calculation processing unit 105 references information on the substrate processing to be performed on the preceding lot, information on the substrate processing to be performed on the interrupt lot, information indicating the processing procedure PD, information indicating the processing time PT, the constraints (constraints 1 to 10) shown in Table 1, priority information for the preceding lot, and priority information for the interrupt lot, and places one of the plurality of plan elements BL on the time table TB to change the state of the time table TB (step S24).

[0165] After arranging one of the plurality of plan elements BL in the timetable TB, the calculation processing unit 105 determines whether or not all of the plan elements BL have been arranged in the timetable TB (step S25). In other words, the calculation processing unit 105 determines whether or not an interrupt schedule (time schedule) has been created.

[0166] When the calculation processing unit 105 determines that all the plan elements BL have been arranged in the timetable TB (Yes in step S25), the time schedule creation process ends, and the calculation processing unit 105 determines the reward (step S12 in FIG. 9). Steps S21 to S25 are executed each time a reinforcement learning step (trial) is repeated. In other words, an interrupt schedule (time schedule) is created each time a reinforcement learning step (trial) is repeated.

[0167] If the calculation processing unit 105 determines that some of the multiple plan elements BL are not arranged in the timetable TB (No in step S25), the calculation processing unit 105 returns to step S24, selects the next plan element BL, and arranges it in the timetable TB. In this way, the calculation processing unit 105 arranges the multiple plan elements BL in the timetable TB sequentially, and sequentially changes the state of the timetable TB.

[0168] Next, the process of arranging the plan elements BL in the timetable TB (step S24 in FIG. 10) will be described with reference to FIGS. 11 and 12. As described with reference to FIGS. 7 and 8, the processing unit 105 randomly selects one of the actions AC, thereby arranging one of the plan elements BL (plan elements ABL) that can be arranged next in the timetable TB. Alternatively, the processing unit 105 predicts and selects, from among the action ACs, the action AC that will maximize the reward, thereby arranging one of the plan elements BL (plan elements ABL) that can be arranged next in the timetable TB.

[0169] 11 is a flowchart showing a process of randomly selecting one of the actions AC and arranging the plan element BL in the timetable TB. As shown in FIG. 11, when randomly selecting one of the actions AC, the process of arranging the plan element BL in the timetable TB (step S24 in FIG. 10) includes steps S31 to S34.

[0170] When the calculation processing unit 105 starts the process of placing the plan element BL on the timetable TB, it acquires the plan element BL (plan element ABL) that can be placed next on the timetable TB from among the unplaced plan elements BL (plan element NBL) (step S31).

[0171] When the calculation processing unit 105 acquires the next plan element BL (plan element ABL) that can be placed, it acquires the placement possible time of each plan element ABL (step S32). After acquiring the placement possible time, the calculation processing unit 105 randomly selects one of the plan elements ABL (step S33). More specifically, the calculation processing unit 105 generates an action AC for each plan element ABL and randomly selects one of the action ACs.

[0172] The calculation processing unit 105 arranges the randomly selected plan element ABL in the timetable TB (step S34). As a result, the processing shown in Fig. 11 ends. In detail, the calculation processing unit 105 arranges the plan element BL (plan element ABL) in the timetable TB based on the randomly selected action AC.

[0173] 12 is a flowchart showing a process of predicting an action AC that will maximize the reward and arranging the plan element BL in the timetable TB. As shown in FIG. 12, when predicting an action AC that will maximize the reward, the process of arranging the plan element BL in the timetable TB (step S24 in FIG. 10) includes steps S41 to S44.

[0174] Steps S41, S42, and S44 shown in FIG. 12 are the same as steps S31, S32, and S34 shown in FIG. 11, and therefore their description will be omitted.

[0175] As shown in Fig. 12, after obtaining the available placement time, the processing unit 105 selects the plan element BL (plan element ABL) with the largest evaluation value (expected reward value) (step S43). More specifically, the processing unit 105 generates an action AC for each plan element ABL. Then, using the behavior selection neural network 121 (see Fig. 8), the processing unit 105 selects the action AC with the largest evaluation value from among the action ACs. The processing unit 105 places the plan element BL (plan element ABL) in the timetable TB based on the selected action AC.

[0176] Next, the substrate processing apparatus 200 of this embodiment will be described with reference to Fig. 13. Fig. 13 is a diagram showing a substrate processing system 200A including the substrate processing apparatus 200 of this embodiment.

[0177] 13, the substrate processing system 200A includes a recording medium 110 and a substrate processing apparatus 200. The substrate processing apparatus 200 includes an interface unit 201, a container placement unit LPS, a transport unit TR, a plurality of substrate processing units PU, a memory unit 202, and a control unit 203a.

[0178] In this embodiment, the container platform LPS has multiple load ports LP. For example, as described with reference to FIG. 2, the container platform LPS may have four load ports LP (first load port LP1 to fourth load port LP4). Furthermore, as described with reference to FIG. 2, the transport unit TR may have an indexer robot IR, a transfer unit PS (not shown), and a transport robot CR. The substrate processing apparatus 200 may also have 16 substrate processing units PU. The configurations of the container platform LPS, transport unit TR, and substrate processing unit PU are similar to those of the container platform LPS, transport unit TR, and substrate processing unit PU described with reference to FIG. 2, and therefore, description thereof will be omitted.

[0179] The interface unit 201 exchanges information, data, or signals with the recording medium 110. Specifically, the recording medium 110 stores the schedule creation program CP described with reference to Figures 1A to 12. The interface unit 201 reads the schedule creation program CP from the recording medium 110 and inputs it to the control unit 203a. The configuration of the interface unit 201 is similar to that of the interface unit 103 described with reference to Figures 1A and 1B, so a detailed description thereof will be omitted.

[0180] The storage unit 202 stores various types of information for controlling the operation of the substrate processing apparatus 200. For example, the storage unit 202 stores data and computer programs. The data includes various types of recipe data. The recipe data includes, for example, a process recipe. The process recipe is data that defines a procedure for substrate processing. The storage unit 202 also stores a schedule creation program CP read from the recording medium 110. The storage unit 202 further stores a processing procedure PD, a processing time PT, a planning element BL, and constraints.

[0181] The storage unit 202 includes a main storage device. The main storage device is, for example, a semiconductor memory. The storage unit 202 may further include an auxiliary storage device. The auxiliary storage device includes, for example, at least one of a semiconductor memory and a hard disk drive. The storage unit 202 may also include removable media.

[0182] The control unit 203a includes, for example, a processor. The control unit 203a may include a CPU or an MPU as the processor. Alternatively, the control unit 203a may include a general-purpose computing device or a dedicated computing device. The control unit 203a controls the operation of each unit of the substrate processing apparatus 200 based on various information stored in the memory unit 202. For example, the control unit 203a controls the interface unit 201, the load port LP, the indexer robot IR, the transport robot CR, the substrate processing unit PU, and the memory unit 202.

[0183] When the substrate accommodation container CA is placed on the container placement part LPS, the control part 203a acquires priority information, number information, recipe information, and processing condition information of the substrates W (lot) accommodated in the substrate accommodation container CA from a host computer (not shown), and stores the information in the memory part 202. In this embodiment, the priority information indicates one of four priorities V, X, Y, and Z. The number information indicates the number of substrates W accommodated in the substrate accommodation container CA. The recipe information indicates information for identifying (specifying) the recipe. The control part 203a acquires the procedure for substrate processing by referring to the recipe information. The processing condition information indicates the processing conditions for substrate processing.

[0184] Furthermore, when at least one substrate accommodation container CA is placed on the container platform LPS, the control unit 203a executes the schedule creation program CP in response to the placement on the container platform LPS of a substrate accommodation container CA (interrupt container) that contains a substrate W (lot) with a higher priority than the substrate W remaining in the substrate accommodation container CA (preceding container) that was placed on the container platform LPS earlier, thereby creating an interrupt schedule. Then, the control unit 203a controls the load port LP, the indexer robot IR, the transport robot CR, and the substrate processing unit PU based on the created interrupt schedule.

[0185] Next, the process executed by the control unit 203a will be described with reference to Fig. 13 and Fig. 14. Fig. 14 is a diagram showing the flow of the process executed by the control unit 203a included in the substrate processing apparatus 200 of this embodiment. In detail, Fig. 14 shows the flow of the process executed by the control unit 203a when creating an interrupt schedule.

[0186] The process shown in Figure 14 begins when at least one substrate storage container CA is placed on the container loading part LPS and a substrate storage container CA (interrupt container) containing a substrate W (lot) with a higher priority than the substrate W remaining in the substrate storage container CA (preceding container) that was placed on the container loading part LPS earlier is placed on the container loading part LPS.

[0187] 14 starts, the control unit 203a acquires a timetable TB (step S101). Specifically, the control unit 203a creates a timetable TB in which plan elements BL for substrates remaining in the device are arranged. For example, the control unit 203a may create the timetable TB by referring to the time schedule of a preceding lot. Note that information about a timetable TB in which no plan elements BL are arranged (an empty timetable TB) may be included in the schedule creation program CP or may be stored in advance in the storage unit 202.

[0188] When the control unit 203a acquires (creates) the time table TB, it executes the schedule creation program CP to create an interruption time schedule (step S103), thereby completing the process shown in FIG.

[0189] Specifically, the schedule creation program CP includes the behavior selection neural network 121 in which parameters (weighting coefficients) have been adjusted, as described with reference to Figures 7 and 8. The control unit 203a creates a time schedule (interrupt schedule) in accordance with steps S1 to S5 described with reference to Figure 7, based on the schedule creation program CP, preceding lot information, succeeding lot information, processing procedure PD, processing time PT, planning elements BL, and constraint conditions.

[0190] Here, the preceding lot information indicates the priority of the substrates W remaining in the preceding container (preceding lot), the number of substrates W remaining in the preceding container (number of substrates remaining in the container), and information regarding the substrate processing being performed on the preceding lot (the substrate processing procedure and the processing conditions for that substrate processing).The succeeding lot information indicates the priority of the substrates W contained in the interrupt container (interrupt lot), the number of substrates W contained in the interrupt container, and information regarding the substrate processing being performed on the interrupt lot (the substrate processing procedure and the processing conditions for that substrate processing).

[0191] 1A to 14, the first embodiment of the present invention has been described. According to the first embodiment, even when interrupt processing occurs, processing of the interrupt lot (subsequent lot) can be completed before the preceding lot without halting processing of the preceding lot. Furthermore, according to the first embodiment, the number of substrate processing units PU assigned to each lot during interrupt processing is allocated according to the priority of each lot, the number of substrates W remaining in the preceding container, and the number of substrates W constituting the interrupt lot, due to the constraint 10. This makes it possible to more efficiently process the subsequent lot that should be processed with priority and the lower-priority lot that is being processed in advance.

[0192] The constraints may further include constraint 11 shown in Table 2. Constraint 11 indicates that the substrate processing units PU included in each tower TW restrict the substrates W to be processed by the plurality of substrate processing units PU using priorities.

[0193] The number of tower TWs allocated to the interrupting lot and the number of tower TWs allocated to the preceding lot are allocated according to priority based on the constraint 11. Specifically, when processing conditions differ between lots, the calculation processing unit 105 and the control unit 203a may create an interrupting schedule by referring to the constraint 11 instead of the constraint 10.

[0194] For example, if the processing conditions differ between a preceding substrate and an interrupt substrate, and substrate processing for the preceding substrate and substrate processing for the interrupt substrate are performed in the same tower TW, various conditions (e.g., temperature and flow rate) must be changed to match the processing conditions each time the substrate processing target is changed between the interrupt substrate and the preceding substrate. As a result, throughput decreases. In contrast, if the processing conditions differ between lots, by referring to constraint 11 instead of constraint 10, substrate processing for the interrupt substrate and substrate processing for the preceding substrate are performed in different towers TW, and therefore various conditions (e.g., temperature and flow rate) do not need to be changed to match the processing conditions. Therefore, it is possible to more efficiently process a subsequent lot that should be processed with priority and a low-priority lot that is being processed in advance.

[0195] Furthermore, in this embodiment, the first remuneration condition indicates that the first remuneration is granted when the final state of the timetable TB indicates that processing of at least one of the preceding substrates (container-remaining substrates) remaining in the preceding container is completed before processing of the interrupt lot is completed. However, the first remuneration condition may also indicate that the first remuneration is granted when the final state of the timetable TB indicates that substrate processing of the preceding substrate is being performed in parallel with substrate processing of at least one interrupt substrate. According to this condition, a time schedule (interrupt schedule) can be created in which substrate processing of the interrupt substrate and the preceding substrate are performed in parallel. Therefore, a time schedule (interrupt schedule) can be more reliably created in which processing of the interrupt lot and processing of the preceding lot are completed in a shorter period of time.

[0196] Alternatively, the conditions for awarding the reward may include a fifth reward condition that awards the fifth reward when the final state of the timetable TB indicates a state in which substrate processing for a preceding substrate is being carried out in parallel with substrate processing for at least one interrupt substrate.

[0197] [Embodiment 2] Next, a second embodiment of the present invention will be described with reference to Fig. 15. However, differences from the first embodiment will be described, and a description of the same aspects as in the first embodiment will be omitted. In the second embodiment, unlike the first embodiment, the substrate processing apparatus 200 generates a schedule creation program CP.

[0198] Fig. 15 is a diagram showing a substrate processing system 200A including the substrate processing apparatus 200 of this embodiment. As shown in Fig. 15, the substrate processing system 200A includes a recording medium 110 and the substrate processing apparatus 200. The substrate processing apparatus 200 includes a container mounting part LPS, a transport part TR, a plurality of substrate processing units PU, a memory part 202, and a control part 203b.

[0199] The recording medium 110 stores the generation program GP described with reference to Figures 1A to 14. The interface unit 201 of the substrate processing apparatus 200 reads the generation program GP from the recording medium 110 and inputs it to the control unit 203b.

[0200] The storage unit 202 stores the generation program GP read from the recording medium 110. The storage unit 202 also stores recipe data, processing procedures PD, processing times PT, plan elements BL, and constraint conditions, as described with reference to FIG.

[0201] The control unit 203b has a processor. The processor of the control unit 203b may be a CPU, an MPU, a GPU, an NPU, or a quantum computer. Alternatively, the control unit 203b may have a general-purpose computing device or a dedicated computing device. The control unit 203b controls the operation of each unit of the substrate processing apparatus 200 based on various information stored in the memory unit 202. For example, the control unit 203b controls the interface unit 201, the load port LP, the indexer robot IR, the transport robot CR, the substrate processing unit PU, and the memory unit 202.

[0202] 1A to 12, the control unit 203b executes the generation program GP to generate the schedule creation program CP. The schedule creation program CP is stored in the storage unit 202.

[0203] In addition, similar to the control unit 203a described with reference to Figures 13 and 14, when the substrate storage container CA is placed on the container loading part LPS, the control unit 203b acquires priority information, number information, recipe information, and processing condition information of the substrates W (lot) stored in the substrate storage container CA from a host computer not shown, and stores the information in the memory unit 202.

[0204] 13 and 14, when at least one substrate accommodation container CA is placed on the container platform LPS, the control unit 203b executes the schedule creation program CP to create an interrupt schedule in response to the placement on the container platform LPS of a substrate accommodation container CA (interrupt container) that contains a substrate W (lot) with a higher priority than the substrate W remaining in the substrate accommodation container CA (preceding container) that has been placed on the container platform LPS earlier, thereby creating an interrupt schedule. Then, the control unit 203b controls the load port LP, the indexer robot IR, the transport robot CR, and the substrate processing unit PU based on the created interrupt schedule.

[0205] The second embodiment of the present invention has been described above with reference to Fig. 15. According to the second embodiment, similar to the first embodiment, it is possible to complete processing of an interruption lot (a subsequent lot) before a preceding lot without stopping processing of the preceding lot. Furthermore, according to the second embodiment, similar to the first embodiment, it is possible to more efficiently process a subsequent lot that should be processed with priority and a low-priority lot that is being processed in advance, due to constraint 10 or constraint 11.

[0206] [Embodiment 3] Next, a third embodiment of the present invention will be described with reference to Fig. 16. However, differences from the first and second embodiments will be described, and a description of the same aspects as the first and second embodiments will be omitted. Unlike the first and second embodiments, the third embodiment is configured such that a substrate processing system 200B includes a schedule creation device 300.

[0207] 16 is a diagram showing a substrate processing system 200B of this embodiment. As shown in FIG. 16, the substrate processing system 200B includes a schedule creation device 300 and a substrate processing apparatus 200.

[0208] The schedule creation device 300 creates an interrupt schedule based on a schedule creation program CP. Specifically, the schedule creation device 300 includes a storage unit 302, a communication unit 303, and a processing unit 304. The schedule creation device 300 is, for example, a server.

[0209] The storage unit 302 has a main storage device. The main storage device includes, for example, a semiconductor memory. The storage unit 302 may further have an auxiliary storage device. The auxiliary storage device includes, for example, at least one of a semiconductor memory and a hard disk drive. The storage unit 302 may include removable media. The storage unit 302 stores various computer programs and various data. Specifically, the storage unit 302 stores a schedule creation program CP. As described with reference to FIGS. 1A to 12, the schedule creation program CP is generated based on the generation program GP.

[0210] The arithmetic processing unit 304 includes a processor. The arithmetic processing unit 304 may include a CPU or an MPU as the processor. Alternatively, the arithmetic processing unit 304 may include a general-purpose arithmetic device or a dedicated arithmetic device. Similar to the control unit 203a described with reference to FIGS. 13 and 14 , the arithmetic processing unit 304 executes the schedule creation program CP stored in the storage unit 302 to create an interrupt schedule.

[0211] The communication unit 303 is connected to a network and performs communication with the substrate processing apparatus 200. The network includes, for example, the Internet, a local area network (LAN), a public telephone network, and a short-range wireless network. The communication unit 303 includes a communication device. The communication unit 303 is, for example, a network interface controller.

[0212] The communication unit 303 is controlled by the arithmetic processing unit 304 to receive preceding lot information and succeeding lot information from the substrate processing apparatus 200. The communication unit 303 is also controlled by the arithmetic processing unit 304 to transmit an interrupt schedule created by the arithmetic processing unit 304 to the substrate processing apparatus 200. The communication unit 303 is an example of a "transmitting unit."

[0213] The substrate processing apparatus 200 includes a container platform LPS, a transport unit TR, a plurality of substrate processing units PU, a storage unit 202, a control unit 203c, and a communication unit 204.

[0214] The communication unit 204 is connected to a network and communicates with the communication unit 303 of the schedule creation device 300. The communication unit 204 includes a communication device. The communication unit 204 is, for example, a network interface controller. The communication unit 204 is controlled by the control unit 203c to transmit preceding lot information and succeeding lot information to the schedule creation device 300. The communication unit 204 is also controlled by the control unit 203c to receive the interrupt schedule transmitted from the communication unit 303 of the schedule creation device 300. The communication unit 204 is an example of a "receiving unit."

[0215] The control unit 203c has, for example, a processor. The control unit 203c may have a CPU or an MPU as the processor. Alternatively, the control unit 203c may have a general-purpose computing device or a dedicated computing device. The control unit 203c controls the operation of each unit of the substrate processing apparatus 200 based on various information stored in the memory unit 202. For example, the control unit 203c controls the load port LP, the indexer robot IR, the transport robot CR, the substrate processing unit PU, the memory unit 202, and the communication unit 204.

[0216] Similar to the control unit 203a described with reference to Figures 13 and 14, when the substrate storage container CA is placed on the container loading part LPS, the control unit 203c acquires priority information, number information, recipe information, and processing condition information of the substrates W (lot) stored in the substrate storage container CA from a host computer (not shown) and stores the information in the memory unit 202.

[0217] Furthermore, when at least one substrate accommodation container CA is placed on the container platform LPS, the control unit 203c instructs the schedule creation device 300 to create an interrupt schedule in response to the placement on the container platform LPS of a substrate accommodation container CA (interrupt container) that contains a substrate W (lot) with a higher priority than the substrate W remaining in the substrate accommodation container CA (preceding container) that was placed on the container platform LPS earlier. As a result, the communication unit 204 receives the interrupt schedule from the schedule creation device 300. The control unit 203c controls the load port LP, the indexer robot IR, the transport robot CR, and the substrate processing unit PU based on the interrupt schedule received by the communication unit 204.

[0218] More specifically, the control unit 203c causes the communication unit 204 to transmit a command to create an interrupt schedule to the schedule creation device 300. As a result, the communication unit 204 receives the interrupt schedule from the schedule creation device 300. The command to create an interrupt schedule includes preceding lot information and succeeding lot information.

[0219] For example, when the schedule creation device 300 (communication unit 303) and the substrate processing apparatus 200 (communication unit 204) are communicatively connected, the control unit 203c may cause the communication unit 204 to transmit information indicating the processing procedure PD, information indicating the processing time PT, information indicating the plan elements BL, and the constraints to the schedule creation device 300. Alternatively, the information indicating the processing procedure PD, information indicating the processing time PT, information indicating the plan elements BL, and the constraints may be stored in advance in the storage unit 302 of the schedule creation device 300.

[0220] The third embodiment of the present invention has been described above with reference to Fig. 16. According to the third embodiment, similar to the first and second embodiments, it is possible to complete processing of an interruption lot (a subsequent lot) before a preceding lot without stopping processing of the preceding lot. Furthermore, according to the third embodiment, similar to the first and second embodiments, it is possible to more efficiently process a subsequent lot that should be processed with priority and a low-priority lot that is being processed in advance, due to constraint 10 or constraint 11.

[0221] [Fourth Embodiment] Next, a fourth embodiment of the present invention will be described with reference to Fig. 17. However, differences from the first to third embodiments will be described, and a description of the same aspects as the first to third embodiments will be omitted. Unlike the first to third embodiments, the fourth embodiment is configured such that a substrate processing system 200C includes a schedule creation program generation device 100.

[0222] Fig. 17 is a diagram showing a substrate processing system 200C of this embodiment. As shown in Fig. 17, the substrate processing system 200C includes a schedule creation program generating device 100 and a substrate processing apparatus 200. The schedule creation program generating device 100 is, for example, a server.

[0223] The schedule creation program generation device 100 includes an input unit 101, a storage unit 102, a calculation processing unit 105, and a communication unit 106. As described with reference to Figures 1A to 12, the calculation processing unit 105 executes the generation program GP stored in the storage unit 102 to generate the schedule creation program CP.

[0224] The communication unit 106 is connected to a network and communicates with the substrate processing apparatus 200. The communication unit 106 is controlled by the arithmetic processing unit 105 and transmits the schedule creation program CP created by the arithmetic processing unit 105 to the substrate processing apparatus 200. The communication unit 106 is an example of a "transmission unit." Note that the configuration of the communication unit 106 is similar to that of the communication unit 303 described with reference to FIG. 16 , and therefore a description thereof will be omitted.

[0225] The substrate processing apparatus 200 includes a container platform LPS, a transport unit TR, a plurality of substrate processing units PU, a storage unit 202 , a control unit 203 a , and a communication unit 204 .

[0226] The communication unit 204 is connected to a network and communicates with the communication unit 106 of the schedule creation program generation device 100. The communication unit 204 receives the schedule creation program CP transmitted from the communication unit 106 of the schedule creation program generation device 100. The schedule creation program CP received by the communication unit 204 is stored in the storage unit 202.

[0227] When at least one substrate accommodation container CA is placed on the container platform LPS, and a substrate accommodation container CA (interrupt container) containing a substrate W (lot) with a higher priority than the substrate W remaining in the substrate accommodation container CA (preceding container) placed on the container platform LPS first is placed on the container platform LPS, the control unit 203a executes the schedule creation program CP to create an interrupt schedule. Then, the control unit 203a controls the load port LP, the indexer robot IR, the transport robot CR, and the substrate processing unit PU based on the created interrupt schedule.

[0228] The fourth embodiment of the present invention has been described above with reference to Fig. 17. According to the fourth embodiment, similar to the first to third embodiments, it is possible to complete processing of an interruption lot (a subsequent lot) before a preceding lot without stopping processing of the preceding lot. Furthermore, according to the fourth embodiment, similar to the first to third embodiments, it is possible to more efficiently process a subsequent lot that should be processed with priority and a low-priority lot that is being processed in advance, due to constraint 10 or constraint 11.

[0229] The embodiments of the present invention have been described above with reference to the drawings (FIGS. 1A to 17). However, the present invention is not limited to the above embodiments and can be implemented in various forms without departing from the spirit of the present invention. Furthermore, the components disclosed in the above embodiments can be modified as appropriate. For example, some of the components shown in one embodiment may be added to the components of another embodiment, or some of the components shown in one embodiment may be deleted from the embodiment.

[0230] The drawings mainly show each component in a schematic manner to facilitate understanding of the invention, and the thickness, length, number, spacing, etc. of each component shown in the drawings may differ from the actual ones due to the convenience of creating the drawings. Furthermore, the configuration of each component shown in the above embodiment is merely an example and is not particularly limited, and it goes without saying that various modifications are possible within a range that does not substantially deviate from the effects of the present invention.

[0231] 1A to 17, the interruption schedule is a time schedule for completing processing for an interruption lot and processing for a preceding lot, but the interruption schedule may also indicate a time schedule for completing processing for an interruption lot. In this case, the fourth remuneration condition may be omitted.

[0232] Furthermore, in the embodiment described with reference to Figures 1A to 17, the processed substrate W was returned to the original substrate storage container CA, but the processed substrate W may also be stored in a substrate storage container CA different from the original substrate storage container CA.

[0233] Furthermore, in the embodiment described with reference to Figures 1A to 17, four priorities were used for the priority of the substrate W, but two, three, or five or more priorities may also be used for the priority of the substrate W.

[0234] 1A to 17, the container platform part LPS includes four load ports LP, but the number of load ports LP is not limited to four. The container platform part LPS may include two, three, five or more load ports LP.

[0235] Furthermore, the substrate processing apparatus 200 is not particularly limited as long as it is an apparatus for processing the substrate W. For example, the substrate processing apparatus 200 may be a chemical cleaning apparatus, a brush cleaning apparatus, a wet etching apparatus, a dry etching apparatus, a coating apparatus, a developing apparatus, an exposure apparatus, a coater developer, a baking apparatus, or a film forming apparatus.

[0236] The present invention is useful in methods and apparatus for processing substrates.

[0237] This application claims priority from Japanese Patent Application No. 2023-218110, filed December 25, 2023, the entire contents of which are incorporated herein by reference.

Claims

1. A schedule creation program generation method for generating a schedule creation program for creating a time schedule for processing a plurality of substrates by a substrate processing apparatus by reinforcement learning, wherein the substrate processing apparatus includes: a container placement unit on which a plurality of substrate storage containers for storing one or more substrates constituting one lot are placed; a plurality of substrate processing units for performing substrate processing on each substrate; and a transfer unit for transferring the substrate between the plurality of substrate storage containers and the plurality of substrate processing units, and the substrate processing apparatus completes the processing of the lot by performing the substrate processing on each of the substrates constituting the lot, and the schedule creation program creates an interrupt schedule in response to a subsequent container, which is a substrate storage container for storing a lot with a second priority higher than the first priority, being placed on the container placement unit when a part of the substrates constituting the lot with the first priority remains in a preceding container, which is a substrate storage container placed in front of the container placement unit, and the interrupt schedule indicates a time schedule for processing a preceding substrate, which is the substrate remaining in the preceding container, and a subsequent substrate, which is the substrate constituting the lot with the second priority, by the substrate processing apparatus, and the schedule creation program generation method includes a step of repeatedly performing an experience step including a time table acquisition step, an arrangement step, and a reward determination step by the reinforcement learning to increase the cumulative reward, the time table acquisition step indicates a step of acquiring a time table for defining the interrupt schedule, the arrangement step indicates a step of sequentially arranging a plurality of planned elements given to each of the preceding substrates remaining in the preceding container and a plurality of planned elements given to each of the subsequent substrates in the time table based on a constraint condition to sequentially change the state of the time table, the constraint condition indicates a condition for restricting the substrates to be processed by each of the substrate processing units by the priority, and the reward determination step indicates a step of determining a reward to be given to the final state of the time table based on the final state of the time table in which all the planned elements are arranged and a condition for giving a reward.The final state of the time table corresponds to the interrupt schedule, the cumulative reward indicates the total value of the rewards given for the final state of the time table, and the conditions for giving the rewards are as follows: a first reward condition for giving a first reward when the final state of the time table indicates that the processing for at least one of the preceding substrates remaining in the preceding container has been completed before the processing for the lot of the second priority is completed; and a second reward condition for giving a second reward whose value is smaller than that of the first reward when the final state of the time table indicates that the processing for the lot of the second priority has been completed before the processing for the lot of the first priority is completed. A method for generating a schedule creation program including these conditions.

2. The time table acquisition step includes a step of acquiring a time table in which the planned elements given to each of the preceding substrates remaining inside the substrate processing apparatus are arranged, and the schedule creation program creates the interrupt schedule without changing the time schedule of the preceding substrates remaining inside the substrate processing apparatus. The method for generating a schedule creation program according to claim 1.

3. The substrate processing apparatus is capable of executing the substrate processing in parallel for two or more of the substrates, and the first reward condition indicates that the first reward is given when the final state of the time table indicates a state in which the substrate processing for the preceding substrate is being executed in parallel with the substrate processing for at least one of the subsequent substrates. The method for generating a schedule creation program according to claim 1 or claim 2.

4. The condition for giving the reward further includes a third reward condition for giving a third reward in which the value of the reward increases as the time when the processing for the lot of the second priority is completed is earlier, and the maximum value of the third reward is smaller than the value of the first reward. The method for generating a schedule creation program according to claim 1 or claim 2.

5. The condition for giving the reward further includes a fourth reward condition for giving a fourth reward in which the value of the reward increases as the time when the processing for the lot of the first priority is completed is earlier, and the maximum value of the fourth reward is smaller than the value of the first reward. The method for generating a schedule creation program according to claim 1 or claim 2.

6. The substrate processing apparatus is capable of executing the substrate processing in parallel for two or more of the substrates, and the condition for giving the reward further includes a fifth reward condition for giving a fifth reward when the final state of the time table indicates a state in which the substrate processing for the preceding substrate is being executed in parallel with the substrate processing for at least one of the subsequent substrates. The method for generating a schedule creation program according to claim 1 or claim 2.

7. A schedule creation program generation device that generates a schedule creation program for creating a time schedule for processing a plurality of substrates by a substrate processing device by reinforcement learning, comprising: a storage unit that stores a generation program that defines the schedule creation program generation method according to claim 1 or claim 2; and a processing unit that executes the generation program to generate the schedule creation program.

8. A schedule creation device that creates a time schedule for processing a plurality of substrates by a substrate processing device, comprising: a storage unit that stores a schedule creation program generated based on the schedule creation program generation method according to claim 1 or claim 2; and a processing unit that executes the schedule creation program to create the interrupt time schedule.

9. A computer-readable recording medium that records a generation program that defines the schedule creation program generation method according to claim 1 or claim 2.

10. A computer-readable recording medium that records a schedule creation program generated based on the schedule creation program generation method according to claim 1 or claim 2.

11. A generation program that is executable by a computer and that defines the schedule creation program generation method according to claim 1 or claim 2.

12. A schedule creation program that is executable by a computer and that is generated based on the schedule creation program generation method according to claim 1 or claim 2.

13. A substrate processing apparatus that completes processing on a lot by performing substrate processing one by one on one or more substrates constituting the lot, the apparatus comprising: a container placement unit on which a plurality of substrate storage containers for storing the substrates constituting the lot are placed; a plurality of substrate processing units that perform the substrate processing on a per-substrate basis; a transfer unit that transfers the substrate between the plurality of substrate storage containers and the plurality of substrate processing units; a storage unit that stores a generation program that defines the schedule creation program generation method according to claim 1 or claim 2; a processing unit that executes the generation program to generate a schedule creation program for creating a time schedule for processing a plurality of substrates by the reinforcement learning; wherein, when a part of the substrates constituting a lot with a first priority remains in a preceding container which is a substrate storage container placed ahead of the container placement unit, the processing unit executes the schedule creation program in response to a succeeding container which is a substrate storage container for storing a lot with a second priority higher than the first priority being placed on the container placement unit, and creates the interrupt schedule for processing a preceding substrate which is the substrate remaining in the preceding container and a succeeding substrate which is the substrate constituting the lot with the second priority.

14. A substrate processing apparatus that completes processing on a lot by performing substrate processing one by one on one or more substrates constituting the lot, the apparatus comprising: a container placement unit on which a plurality of substrate storage containers for storing the substrates constituting the lot are placed; a plurality of substrate processing units that perform the substrate processing on a per-substrate basis; a transfer unit that transfers the substrate between the plurality of substrate storage containers and the plurality of substrate processing units; a storage unit that stores a schedule creation program generated based on the schedule creation program generation method according to claim 1 or claim 2; and a processing unit that creates the interrupt schedule for processing a preceding substrate, which is the substrate remaining in the preceding container, and a succeeding substrate, which is the substrate constituting the lot of the second priority, when a succeeding container, which is a substrate storage container that stores a lot of the second priority having a higher priority than the first priority, is placed on the container placement unit in a case where a part of the substrates constituting the lot of the first priority remains in a preceding container, which is a substrate storage container placed prior to the container placement unit.

15. A substrate processing apparatus that completes processing on a lot by performing substrate processing one by one on one or more substrates constituting the lot, and a schedule creation program generation apparatus according to claim 7, comprising: The schedule creation program generation apparatus further includes a transmission unit that transmits the schedule creation program to the substrate processing apparatus, and the substrate processing apparatus includes: a container placement unit on which a plurality of substrate storage containers for storing the substrates constituting the lot are placed; a plurality of substrate processing units that perform the substrate processing on a per-substrate basis; a transfer unit that transfers the substrate between the plurality of substrate storage containers and the plurality of substrate processing units; a reception unit that receives the schedule creation program transmitted from the transmission unit of the schedule creation program generation apparatus; and a processing unit that creates an interrupt schedule for processing a preceding substrate that is the substrate remaining in the preceding container and a succeeding substrate that is the substrate constituting the second-priority lot in response to a succeeding container, which is a substrate storage container for accommodating a second-priority lot having a priority higher than the first priority, being placed on the container placement unit when a part of the substrates constituting the first-priority lot remains in a preceding container that is a substrate storage container placed prior to the container placement unit. A substrate processing system.

16. A substrate processing apparatus that completes processing on a lot by performing substrate processing one by one on one or more substrates constituting the lot, and a schedule creation apparatus according to claim 8, comprising: The schedule creation apparatus further includes a transmission unit that transmits the interrupt schedule to the substrate processing apparatus, and the substrate processing apparatus includes: a container placement unit on which a plurality of substrate storage containers for storing the substrates constituting the lot are placed; a plurality of substrate processing units that perform the substrate processing on a per-substrate basis; a transfer unit that transfers the substrate between the plurality of substrate storage containers and the plurality of substrate processing units; and a reception unit that receives the interrupt schedule transmitted from the transmission unit of the schedule creation apparatus. A substrate processing system.

Images