Monitoring method in substrate processing apparatus

Abstract

Provided is a technology that makes it possible to monitor an object to be monitored with higher monitoring accuracy. This monitoring method, which is for monitoring an object to be monitored in a chamber of a substrate processing apparatus for processing a substrate, comprises an image generation step and a monitoring step. In the image generation step, a monitoring camera receives light from a monitoring region including at least one first object to be monitored among a plurality of objects to be monitored in the chamber, and generates monitoring image data. In the monitoring step, executed after the image generation step, a trained model for high resolution-achieving processing is used to generate high-resolution image data obtained by increasing the resolution of the monitoring image data, and the first object to be monitored is monitored on the basis of the high-resolution image data.

Description

Monitoring method for substrate processing apparatus

[0001] The present disclosure relates to a monitoring method for a substrate processing apparatus.

[0002] Conventionally, in manufacturing processes for semiconductor devices and the like, various processing liquids such as pure water, photoresist liquid, and etching liquid are supplied to substrates to perform various substrate processing processes such as cleaning processing and resist coating processing. As an apparatus for performing substrate processing using these processing liquids, a substrate processing apparatus is widely used in which a substrate holder rotates a substrate in a horizontal position while a processing liquid is ejected from a nozzle onto the surface of the substrate.

[0003] In such substrate processing apparatuses, whether the nozzle position is appropriate is monitored. For example, in Patent Document 1, an imaging means such as a camera is provided to monitor the nozzle position.

[0004] JP 2015-173148 A

[0005] In order to properly process a substrate, it is desirable to monitor more than just the nozzle.

[0006] For example, the substrate holder includes a plurality of chuck pins. The plurality of chuck pins are arranged at equal intervals along the periphery of the substrate. Each chuck pin is arranged to be displaceable between a holding position in which it abuts against the periphery of the substrate and a release position away from the substrate. When the chuck pin is positioned in the holding position, the substrate holder can hold the substrate. If the substrate holder cannot properly hold the substrate, the substrate cannot be processed properly, so the positions of the plurality of chuck pins of the substrate holder may be monitored.

[0007] When a camera receives light from a monitoring area including multiple chuck pins, the image data generated by the camera includes the multiple chuck pins. If the camera is an imaging camera, and the distance between the camera and the chuck pins in real space differs among the multiple chuck pins, the clarity of the contours of the multiple chuck pins in the captured image data may differ among the multiple chuck pins. For example, a chuck pin closer to the camera is included in the captured image data as a larger size, so the contour shape of the chuck pin is likely to be clear. On the other hand, a chuck pin farther from the camera is included in the captured image data as a smaller size, so the contour shape of the chuck pin is likely to be unclear. Furthermore, the farther a chuck pin is from the focus position of the camera lens in real space, the more likely the contour shape of the chuck pin is to be unclear in the captured image data.

[0008] Furthermore, if the camera is a depth camera, the image data generated by the camera contains the distance to the object in the monitored area for each pixel. Since the measurement accuracy of the depth camera is not very high, the contour shape of the object in the depth image data may be unclear.

[0009] If an object to be monitored is monitored based on image data that includes an object with an unclear contour shape, the monitoring accuracy may be reduced.

[0010] Therefore, an object of the present disclosure is to provide a technology that can monitor a monitoring target with higher monitoring accuracy.

[0011] A first aspect is a monitoring method for monitoring a monitored object within a chamber of a substrate processing apparatus for processing substrates, comprising an image generation step in which a monitoring camera receives light from a monitoring area including at least one first monitored object among a plurality of monitored objects within the chamber and generates monitored image data; and a monitoring step that is executed after the image generation step, in which high-resolution image data is generated by increasing the resolution of the monitored image data using a trained model for high-resolution processing, and the first monitored object is monitored based on the high-resolution image data.

[0012] A second aspect is a monitoring method for a substrate processing apparatus according to the first aspect, wherein in the monitoring process, the high-resolution image data is generated using a first trained model corresponding to the first monitored object from among a plurality of trained models of high-resolution processing corresponding to each of the plurality of monitored objects.

[0013] A third aspect is a monitoring method in a substrate processing apparatus according to the second aspect, wherein in the image generation process, the monitoring camera receives light from the monitoring area including at least the first and second monitoring objects among the plurality of monitoring objects to generate the monitoring image data, and the monitoring process includes a process of generating first high-resolution image data by increasing the resolution of a first judgment area of ​​the monitoring image data that does not include the second monitoring object but includes the first monitoring object using the first trained model, and generating second high-resolution image data by increasing the resolution of a second judgment area of ​​the monitoring image data that does not include the first monitoring object but includes the second monitoring object using a second trained model corresponding to the second monitoring object among the plurality of trained models, and a process of monitoring the first monitoring object based on the first high-resolution image data and monitoring the second monitoring object based on the second high-resolution image data.

[0014] A fourth aspect is a monitoring method for a substrate processing apparatus according to the second or third aspect, wherein in the monitoring process, the high-resolution image data is generated by increasing the resolution of the monitoring image data at a first resolution magnification set for the first monitored object out of multiple resolution magnifications set for each of the multiple monitored objects.

[0015] A fifth aspect is a monitoring method for a substrate processing apparatus according to the fourth aspect, further comprising a training step that is executed before the image generation step and generates the first trained model, and the training step comprises a training image generation step in which a high-performance camera having a higher resolution than the resolution of the monitoring camera images the first monitored object at multiple times while focusing on the first monitored object rather than on a second monitored object among the multiple monitored objects, thereby generating multiple high-resolution training data; a low-resolution reduction step in which the multiple high-resolution training data are downsampled by a factor that is equal to or less than the reciprocal of the first resolution factor and converted into multiple low-resolution training data; and a model generation step in which the first trained model is generated based on the multiple high-resolution training data and the multiple low-resolution training data.

[0016] A sixth aspect is the monitoring method for the substrate processing apparatus according to the fifth aspect, wherein in the learning image generating step, the high-performance camera captures an image of the area monitored by the monitoring camera.

[0017] A seventh aspect is a monitoring method for a substrate processing apparatus according to the fifth or sixth aspect, wherein in the learning image generation process, images of the first monitored object, which is displaceable, are taken at the plurality of times when the first monitored object is located at mutually different positions, and the plurality of high-resolution learning data are generated.

[0018] An eighth aspect is a monitoring method for a substrate processing apparatus according to the fourth aspect, further comprising a training process that is executed before the image generation process and generates the trained model, and the training process comprises a training image generation process that generates multiple sets of high-resolution training data and low-resolution training data based on three-dimensional data including information about the shape of the monitored object in the chamber, and a model generation process that generates the trained model based on the multiple sets of high-resolution training data and low-resolution training data.

[0019] A ninth aspect is a monitoring method for a substrate processing apparatus according to any one of the first to eighth aspects, wherein the monitoring camera includes a depth camera that generates depth image data indicating the distance to the monitored object as the monitoring image data.

[0020] A tenth aspect is the monitoring method for a substrate processing apparatus according to any one of the first to ninth aspects, wherein the monitoring camera includes an imaging camera that receives visible light.

[0021] An eleventh aspect is a monitoring method for a substrate processing apparatus according to any one of the first to tenth aspects, further comprising a reference image registration step that is executed before the image generation step and registers reference image data, and the reference image registration step includes a preliminary image generation step in which the monitoring camera receives light from the monitoring area and generates preliminary image data, and a reference image generation step after the preliminary image generation step in which the trained model is used to generate reference image data with a higher resolution than the preliminary image data, and a storage step in which the reference image data is written to a storage unit, and in the monitoring step, the first monitored object is monitored based on a comparison between the high-resolution image data and the reference image data.

[0022] According to the first, ninth, and tenth aspects, the resolution of the monitoring image data is increased using a trained model, so that it is possible to generate high-resolution image data with higher accuracy, thereby enabling the first monitoring object to be monitored with higher monitoring accuracy.

[0023] According to the second aspect, since the resolution of the monitoring image data is increased using the first trained model corresponding to the first monitoring object, it is possible to generate high-resolution image data with higher accuracy, thereby enabling the first monitoring object to be monitored with even higher monitoring accuracy.

[0024] According to the third aspect, there is no need to capture images of the first and second monitoring objects at different times, thereby reducing the number of times the monitoring camera operates. Moreover, since the resolution of the second judgment area is increased using the second trained model corresponding to the second monitoring object, second high-resolution image data with higher accuracy can be generated. Therefore, the second monitoring object can be monitored with higher accuracy.

[0025] According to the fourth aspect, it is possible to increase the resolution of monitoring image data with higher accuracy regardless of the state of the monitored object.

[0026] According to the fifth aspect, a plurality of high-resolution training data are generated while the first monitoring object is in focus, and then converted into a plurality of low-resolution training data at a magnification equal to or less than the reciprocal of the first resolution magnification, thereby generating a first trained model suitable for increasing the resolution of the first monitoring object at the first resolution magnification.

[0027] According to the sixth aspect, the high-performance camera captures the same monitoring area as the surveillance camera. Therefore, the optical characteristics of the lens group of the high-performance camera through which light passes during imaging are similar to the optical characteristics of the lens group of the surveillance camera through which light passes during imaging. Therefore, the high-resolution learning data from the high-performance camera can be made equivalent to high-resolution image data obtained by increasing the resolution of the surveillance image data from the surveillance camera. Consequently, the first surveillance object can be more appropriately monitored based on the high-resolution image data.

[0028] According to the seventh aspect, the plurality of high-resolution training data includes first monitored objects at different positions, and the first trained model is generated using the plurality of high-resolution training data, thereby enabling the first trained model to generate high-resolution image data with higher accuracy.

[0029] According to the eighth aspect, a high-performance camera is not required to generate training data, so a trained model can be generated at low cost.

[0030] According to an eleventh aspect, the high-resolution image data and the reference image data are generated by increasing the resolution of the preliminary image data captured by the monitoring camera using a trained model. That is, the high-resolution image data and the reference image data are generated using the same monitoring camera and the same trained model. Therefore, by comparing the high-resolution image data with the reference image data, the first monitoring object can be monitored with high accuracy.

[0031] 1 is a plan view schematically showing an example of the configuration of a substrate processing apparatus; FIG. 2 is a block diagram schematically showing an example of the configuration of a control unit; FIG. 3 is a longitudinal sectional view schematically showing an example of the configuration of a processing unit according to a first embodiment; FIG. 4 is a diagram schematically showing an example of monitoring image data generated by an imaging camera; FIG. 5 is a diagram schematically showing an example of monitoring image data generated by an imaging camera; FIG. 6 is a flowchart showing an example of a monitoring method using a processing unit; FIG. 7 is a flowchart showing a specific example of a monitoring step; FIG. 8 is a flowchart showing an example of the flow of each piece of data in the monitoring step; FIG. 9 is a flowchart showing a first example of a specific operation of a training step; FIG. 10 is a diagram schematically showing an example of a state in which a high-performance camera is provided in a chamber; FIG. 11 is a diagram schematically showing the flow of data in generating a trained model; FIG. 12 is a flowchart showing an example of a specific operation of a reference image registration step; FIG. 13 is a functional block diagram showing an example of the internal configuration of a control unit; FIG. 14 is a flowchart showing a second example of a specific operation of a training step; FIG. 15 is a diagram schematically showing an example of high-resolution training data; FIG. 16 is a longitudinal sectional view schematically showing an example of the configuration of a processing unit according to a second embodiment; FIG. 17 is a diagram schematically showing an example of monitoring image data; FIG. 18 is a diagram schematically showing an example of high-resolution training data; FIG. 19 is a diagram schematically showing an example of low-resolution training data. FIG. 10 is a diagram illustrating an example of an operation of a control unit.

[0032] Hereinafter, the embodiments will be described in detail with reference to the drawings. In the drawings, the dimensions and numbers of parts are exaggerated or simplified as necessary for ease of understanding. Parts having similar configurations and functions are designated by the same reference numerals, and duplicate explanations will be omitted below.

[0033] In the following description, the same components are denoted by the same reference numerals, and their names and functions are also the same. Therefore, detailed descriptions of them may be omitted to avoid duplication.

[0034] Furthermore, in the following description, even if ordinal numbers such as "first" or "second" are used, these terms are used for convenience to facilitate understanding of the contents of the embodiments, and are not limited to the order that may result from these ordinal numbers.

[0035] When expressions indicating relative or absolute positional relationships (e.g., "in one direction," "along one direction," "parallel," "orthogonal," "center," "concentric," "coaxial," etc.) are used, unless otherwise specified, the expressions not only strictly represent the positional relationship but also represent a state in which there is a relative displacement in terms of angle or distance within a range in which tolerance or equivalent functionality is obtained. When expressions indicating an equal state (e.g., "identical," "equal," "homogeneous," etc.) are used, the expressions not only represent a state in which there is strict quantitative equality but also represent a state in which there is a difference in which tolerance or equivalent functionality is obtained, unless otherwise specified. When expressions indicating a shape (e.g., "rectangular shape" or "cylindrical shape," etc.) are used, the expressions not only represent a geometrically strict shape but also represent a shape with, for example, irregularities or chamfers within a range in which equivalent effects are obtained. When the expressions "comprise," "include," "have," "includes," "includes," or "have" are used to describe one component, the expressions are not exclusive expressions that exclude the presence of other components. When the phrase "at least one of A, B, and C" is used, the phrase includes A only, B only, C only, any two of A, B, and C, and all of A, B, and C.

[0036] First Embodiment Overall Configuration of Substrate Processing Apparatus Fig. 1 is a plan view schematically showing an example of the configuration of a substrate processing apparatus 100. The substrate processing apparatus 100 is a single-wafer processing apparatus that processes substrates W one by one.

[0037] The substrate W is, for example, a semiconductor wafer, a liquid crystal display substrate, an organic electroluminescence (EL) substrate, an FPD (Flat Panel Display) substrate, an optical display substrate, a magnetic disk substrate, an optical disk substrate, a magneto-optical disk substrate, a photomask substrate, or a solar cell substrate. The substrate W has a thin, flat plate shape. In the following, the substrate W is assumed to be a semiconductor wafer. The substrate W has, for example, a disk shape. The diameter of the substrate W is, for example, about 300 mm, and the film thickness of the substrate W is, for example, about 0.5 mm or more and about 3 mm or less.

[0038] 1, the substrate processing apparatus 100 includes an indexer block 110, a processing block 120, and a control unit 90. The processing block 120 is a section that mainly processes substrates W, and the indexer block 110 is a section that mainly transports substrates W between the outside of the substrate processing apparatus 100 and the processing block 120.

[0039] The indexer block 110 includes a load port 111 and a first transport part 112. A substrate container (hereinafter referred to as a carrier) C that has been carried in from outside is placed on the load port 111. The carrier C accommodates a plurality of substrates W, for example, arranged at intervals in the vertical direction. In the example of Fig. 1, a plurality of load ports 111 are arranged.

[0040] The first transport unit 112 is a transport robot and can remove unprocessed substrates W from the carriers C placed on each load port 111. The first transport unit 112 may also be called an indexer robot. The first transport unit 112 transports the unprocessed substrates W removed from the carriers C to the processing block 120. The processing block 120 can process the unprocessed substrates W. The first transport unit 112 can also receive processed substrates W from the processing block 120 and transport the processed substrates W to the carriers C of the load port 111.

[0041] In the example of FIG. 1 , the processing block 120 includes a plurality of processing units 1 and a second transport part 122. The second transport part 122 is a transport robot and can transport substrates W between the first transport part 112 and the plurality of processing units 1. In the example of FIG. 1 , the processing block 120 also includes a receiver 123. The receiver 123 is, for example, a shelf on which a plurality of substrates W can be placed in a vertically arranged state. The first transport part 112 places an unprocessed substrate W on the receiver 123. The second transport part 122 removes the unprocessed substrate W from the receiver 123 and transports the substrate W to the processing unit 1. The processing unit 1 processes the substrate W. The configuration of the processing unit 1 will be described later. The second transport part 122 removes the processed substrate W from the processing unit 1 and transports the substrate W to the receiver 123. The first transport part 112 takes the substrate W from the platform 123 and transports the substrate W to a carrier C in the load port 111 .

[0042] 1 , a plurality of (e.g., four) processing units 1 are arranged to surround the second transport section 122 in a plan view. This second transport section 122 may also be called a center robot. At each position in a plan view, a plurality of processing units 1 may be stacked vertically. In other words, a plurality of (four in the figure) towers TW each made up of a plurality of processing units 1 stacked vertically may be arranged to surround the second transport section 122.

[0043] The control unit 90 comprehensively controls the substrate processing apparatus 100. Specifically, the control unit 90 controls the first transport unit 112, the second transport unit 122, and the processing unit 1. FIG. 2 is a block diagram schematically illustrating an example of the configuration of the control unit 90. The control unit 90 is an electronic circuit and includes, for example, a data processing unit 91 and a storage unit 92. In the example illustrated in FIG. 2, the data processing unit 91 and the storage unit 92 are connected to each other via a bus. The data processing unit 91 may be an arithmetic processing device such as a central processing unit (CPU). The storage unit 92 may include a non-transitory storage unit (e.g., a read-only memory (ROM)) 921 and a temporary storage unit (e.g., a random access memory (RAM)) 922. The non-transitory storage unit 921 may store, for example, a program that defines the processing to be performed by the control unit 90. The data processing unit 91 executes this program, allowing the control unit 90 to perform the processing defined in the program. Of course, some or all of the processing performed by the control unit 90 may be performed by hardware such as a dedicated logic circuit.

[0044] As shown in Fig. 2, the control unit 90 may be electrically connected to a storage unit 94. The storage unit 94 is a non-transitory storage unit, and may be, for example, a memory or a hard disk. In the example of Fig. 2, various data are stored in the storage unit 94. Each piece of data will be described in detail later.

[0045] <Processing Unit> Fig. 3 is a longitudinal cross-sectional view schematically showing an example of the configuration of a processing unit 1 according to the first embodiment. Note that it is not necessary for all processing units 1 belonging to the substrate processing apparatus 100 to have the configuration exemplified in Fig. 3. It is sufficient that at least one processing unit 1 has the configuration exemplified in Fig. 3.

[0046] The processing unit 1 includes a chamber 10. The chamber 10 forms a processing chamber for processing a substrate W. Specifically, the internal space of the chamber 10 corresponds to the processing chamber. The chamber 10 is provided with an openable / closable loading / unloading port (not shown). The second transport part 122 loads an unprocessed substrate W into the chamber 10 through the loading / unloading port, and loads a processed substrate W out of the chamber 10 through the loading / unloading port.

[0047] Various components used for processing the substrate W are provided within the chamber 10. For example, the chamber 10 includes a substrate holder 2, a nozzle 31, and a guard 7.

[0048] The substrate holder 2 holds the substrate W in a horizontal position and rotates the substrate W around a rotation axis Q1. The horizontal position here means that the thickness direction of the substrate W is aligned with the vertical direction. The rotation axis Q1 is an axis that passes through the center of the substrate W and is aligned with the vertical direction. Such a substrate holder 2 may also be called a spin chuck.

[0049] In the example of FIG. 3 , the substrate holder 2 includes a spin base 21, chuck pins 22, a rotation drive unit 23, and a pin drive unit 24. The spin base 21 has a plate-like shape (e.g., a disk shape) and is disposed with its thickness direction aligned vertically. A plurality of (e.g., three or more) chuck pins 22 are provided on the upper surface of the spin base 21. The chuck pins 22 are disposed at equal intervals along the circumferential direction about the rotation axis Q1. The chuck pins 22 are disposed so as to be displaceable between a holding position and a release position, which will be described below. The holding position is a position where the chuck pins 22 abut against the periphery of the substrate W. The chuck pins 22 hold the substrate W by stopping at their respective holding positions. FIG. 3 shows the chuck pins 22 stopped at the holding position. The release position is a position where each chuck pin 22 is separated from the substrate W. When the chuck pins 22 stop at their respective release positions, the substrate W is released from the holding by the chuck pins 22 .

[0050] The pin driver 24 displaces the chuck pin 22 between the holding position and the release position. In the example of FIG. 3 , the pin driver 24 includes a plurality of displacement magnets 241, a plurality of fixed magnets 242, a drive magnet 243, and a magnet lift driver 244. The displacement magnets 241 are provided at the lower ends of the chuck pins 22. The fixed magnets 242 are provided at the bottom of the spin base 21. The drive magnets 243 have, for example, an annular shape surrounding the rotation axis Q1. The magnet lift driver 244 lifts and lowers the drive magnet 243 between the magnet-up position and the magnet-down position. The magnet lift driver 244 includes a drive source such as a motor or an air cylinder and is controlled by the control unit 90. When the drive magnet 243 is located at the magnet-up position, the displacement magnet 241 and the chuck pin 22 move together to one of the holding position and the release position due to the magnetic force between the displacement magnet 241 and the drive magnet 243. When the drive magnet 243 is located in the magnet lower position, the magnetic force between the displacement magnet 241 and the fixed magnet 242 causes the displacement magnet 241 and the chuck pin 22 to move together to the other of the holding position and the release position.

[0051] The rotation drive unit 23 includes a shaft 231 and a motor 232. The upper end of the shaft 231 is connected to the lower surface of the spin base 21, and the shaft 231 extends from the lower surface of the spin base 21 along a rotation axis Q1. The motor 232 is controlled by the control unit 90 to rotate the shaft 231 about the rotation axis Q1. This causes the spin base 21, chuck pins 22, and substrate W to rotate integrally about the rotation axis Q1.

[0052] The nozzle 31 ejects the processing liquid toward the main surface of the substrate W held by the substrate holder 2. In the example of Fig. 3, the nozzle 31 is provided vertically above the substrate W held by the substrate holder 2, and ejects the processing liquid toward the upper surface of the substrate W. The nozzle 31 may be a nozzle that ejects the processing liquid in the form of a continuous flow, or may be a mist nozzle or spray nozzle that ejects the processing liquid in the form of droplets. Here, as an example, the nozzle 31 is a nozzle that ejects the processing liquid in the form of a continuous flow.

[0053] The processing liquid may be a chemical liquid, a rinse liquid, or a neutralization liquid. The chemical liquid may be a cleaning liquid for removing foreign matter from the substrate W or an etching liquid for removing a target film. Examples of chemical liquids that can be used include hydrofluoric nitric acid (hydrofluoric acid-nitric acid solution) obtained by mixing hydrofluoric acid, nitric acid, and water; hydrofluoric acid-hydrogen peroxide solution (FPM) obtained by mixing hydrofluoric acid, hydrogen peroxide, and water; tetramethylammonium hydroxide (TMAH); a mixture of sulfuric acid and hydrogen peroxide solution (SPM); ammonia water; a mixture of ammonia, hydrogen peroxide, and water (SC-1); and a mixture of hydrogen chloride, hydrogen peroxide, and water (SC-2). The chemical liquid may be a single liquid rather than a mixture. For example, a single liquid such as hydrofluoric acid (HF), hydrogen peroxide solution, or sulfuric acid can be used as the chemical liquid. The rinse liquid may be pure water (i.e., deionized water) or an organic solvent such as isopropyl alcohol, which is more volatile than pure water. The neutralization liquid is a liquid for removing electric charges from the substrate W and may be carbon dioxide water. Carbon dioxide water may be used as a rinse solution.

[0054] In the example of Fig. 3, the downstream end of a liquid supply pipe 32 is connected to the nozzle 31. The upstream end of the liquid supply pipe 32 is connected to a processing liquid supply source. The processing liquid supply source supplies the processing liquid to the upstream end of the liquid supply pipe 32. In the example of Fig. 3, a supply valve 33 and a flow rate adjustment valve 34 are inserted in the liquid supply pipe 32. The supply valve 33 switches the opening and closing of the liquid supply pipe 32, and the flow rate adjustment valve 34 adjusts the flow rate of the processing liquid flowing through the liquid supply pipe 32. The flow rate adjustment valve 34 may be a mass flow controller. These valves are controlled by a control unit 90.

[0055] In the example of Fig. 3, the processing unit 1 includes a nozzle movement drive unit 37 that moves the nozzle 31. The nozzle movement drive unit 37 moves the nozzle 31 between a processing position and a standby position, which will be described below. The processing position is a position where the nozzle 31 ejects the processing liquid toward the main surface of the substrate W held by the substrate holder 2, and is, for example, a position vertically opposite the center of the substrate W. The example of Fig. 3 shows the nozzle 31 positioned at the processing position. The standby position is a position where the nozzle 31 does not eject the processing liquid toward the main surface of the substrate W, and is, for example, a position radially outward from the substrate W.

[0056] In the example shown in FIG. 3 , the nozzle movement drive unit 37 includes an arm 371, a support column 372, and a drive source 373. The support column 372 is located radially outward of the guard 7 (described later) and extends vertically. The arm 371 extends horizontally, with its tip connected to the nozzle 31 and its base connected to the support column 372. The drive source 373 is controlled by the control unit 90 and rotates the support column 372 in forward and reverse directions within a predetermined angular range around its central axis Q2. The drive source 373 includes, for example, a motor. When the support column 372 rotates in forward and reverse directions within a predetermined angular range around the central axis Q2, the nozzle 31 reciprocates in the circumferential direction about the central axis Q2. The support column 372 is installed so that the processing position and the standby position are located on the movement trajectory of the nozzle 31. Note that the nozzle movement drive unit 37 is not necessarily limited to the embodiment shown in FIG. 3 and may include, for example, a linear motion mechanism such as a linear motor.

[0057] When the nozzle 31 is positioned at the processing position and the processing liquid is ejected toward the main surface of the rotating substrate W, the processing liquid lands on the main surface of the substrate W. The processing liquid flows radially outward due to centrifugal force caused by the rotation of the substrate W, and is scattered outside the periphery of the substrate W. At this time, the processing liquid acts on the main surface of the substrate W, and processing according to the type of processing liquid is performed on the substrate W.

[0058] The nozzle 31, the liquid supply pipe 32, the supply valve 33, and the flow rate adjustment valve 34 constitute the discharge unit 3. The discharge unit 3 may be configured to be able to sequentially discharge a plurality of types of processing liquid toward the main surface of the substrate W. For example, the discharge unit 3 may discharge a chemical liquid and a rinse liquid toward the main surface of the substrate W in this order. As a specific example, the nozzle 31 may be shared by both the chemical liquid and the rinse liquid. In this case, the liquid supply pipe 32 for the chemical liquid and the liquid supply pipe 32 for the rinse liquid join together and are connected to the nozzle 31. The liquid supply pipe 32 for the chemical liquid is provided with a supply valve 33 and a flow rate adjustment valve 34 for the chemical liquid, and the liquid supply pipe 32 for the rinse liquid is provided with a supply valve 33 and a flow rate adjustment valve 34 for the rinse liquid. Note that the nozzle 31 for the chemical liquid and the nozzle 31 for the rinse liquid may be provided separately.

[0059] The guard 7 catches the processing liquid splashed from the periphery of the substrate W. The guard 7 has a cylindrical shape that surrounds the substrate W held by the substrate holder 2. The guard lifting / lowering drive unit 8 raises and lowers the guard 7 between a guard processing position and a guard standby position, which will be described below. The guard processing position is a position where the upper end of the guard 7 is vertically above the upper surface of the substrate W held by the substrate holder 2. The example in FIG. 3 shows the guard 7 positioned at the guard processing position. When the guard 7 is positioned at the guard processing position, processing liquid splashed from the periphery of the substrate W is caught by the inner peripheral surface of the guard 7. The guard standby position is a position where the upper end of the guard 7 is vertically below the upper surface of the spin base 21. The guard lifting / lowering drive unit 8 includes, for example, a power transmission unit such as a rack and pinion mechanism and a ball screw mechanism, and a power source such as a motor. Alternatively, the guard lifting / lowering drive unit 8 may include an air cylinder.

[0060] In Figure 3, a cup 75 is provided corresponding to the guard 7. The cup 75 has an annular (e.g., annular) recess (groove) surrounding the rotation axis Q1. The cup 75 receives the processing liquid that has flowed down the inner circumferential surface of the guard 7. The upstream end of a drain pipe 76 is connected to, for example, the bottom of the cup 75. The processing liquid received in the cup 75 is discharged to the outside of the processing unit 1 through the drain pipe 76.

[0061] <Outline of an example of substrate processing> Such a processing unit 1 can process a substrate W, for example, as follows. First, the second transport part 122 transports the substrate W to the processing unit 1, and the substrate holding part 2 holds the substrate W received from the second transport part 122. As a specific example, the substrate holding part 2 displaces the multiple chuck pins 22 from their respective release positions to their holding positions. This causes the substrate holding part 2 to hold the substrate W. The substrate holding part 2 continues to hold the substrate W until processing on the substrate W is completed.

[0062] Next, the processing unit 1 performs chemical processing. Specifically, the guard lifting drive unit 8 raises the guard 7 to the guard processing position, the nozzle movement drive unit 37 moves the nozzle 31 to the processing position, and the substrate holder 2 rotates the substrate W around the rotation axis Q1. The substrate holder 2 may continue rotating the substrate W until processing of the substrate W is completed. Then, the control unit 90 opens the chemical supply valve 33. This causes the chemical to be discharged from the nozzle 31 toward the center of the main surface of the rotating substrate W. The chemical that has landed on the main surface of the substrate W flows radially outward as the substrate W rotates, and is scattered from the periphery of the substrate W. At this time, the chemical acts on the main surface of the substrate W, and processing according to the type of chemical is performed on the substrate W.

[0063] When the substrate W has been sufficiently processed, the processing unit 1 ends the chemical processing. For example, when a predetermined chemical time has elapsed since the start of chemical supply, the control unit 90 closes the chemical supply valve 33. The time is measured, for example, by a timer circuit (not shown) that belongs to the control unit 90.

[0064] Next, the processing unit 1 performs a rinse process. Specifically, the control unit 90 opens the rinse liquid supply valve 33. This causes the rinse liquid to be discharged from the nozzle 31 toward the center of the main surface of the rotating substrate W. The rinse liquid that has landed on the main surface of the substrate W flows radially outward as the substrate W rotates, and is scattered from the periphery of the substrate W. At this time, the rinse liquid washes away the chemical liquid on the main surface of the substrate W, so that the chemical liquid on the main surface of the substrate W can be replaced with the rinse liquid.

[0065] The processing unit 1 ends the rinsing process when the chemical liquid on the main surface of the substrate W has been sufficiently replaced with the rinsing liquid. For example, when a predetermined rinsing time has elapsed since the start of supply of the rinsing liquid, the control unit 90 closes the supply valve 33 for the rinsing liquid. Then, the nozzle movement drive unit 37 moves the nozzle 31 to the standby position.

[0066] Next, the processing unit 1 performs a drying process. Specifically, the substrate holder 2 increases the rotation speed of the substrate W (spin dry). When the substrate W is sufficiently dried, the processing unit 1 ends the drying process. Specifically, when a predetermined drying time has elapsed since the start of the drying process, the substrate holder 2 stops the rotation of the substrate W. This essentially ends the drying process, and the series of processes for the substrate W essentially ends.

[0067] Next, the guard lifting drive unit 8 lowers the guard 7 to the guard standby position, and the substrate holding unit 2 releases its hold on the substrate W. As a specific example, the substrate holding unit 2 displaces the multiple chuck pins 22 from their respective holding positions to the release position. This releases the substrate W from the multiple chuck pins 22. Then, the second transport unit 122 lifts the processed substrate W vertically upward from the substrate holding unit 2, and transports the substrate W out of the processing unit 1.

[0068] <Objects to be monitored> As described above, the substrates W are properly processed when each component in the chamber 10 operates properly. On the other hand, if each component does not operate properly, the substrates W may be over- or under-processed, or may not be able to be processed at all.

[0069] Therefore, in this embodiment, the processing unit 1 monitors each component of the chamber 10. As shown in Fig. 3, the processing unit 1 includes a monitoring camera 5. The camera 5 receives light from a monitoring area including at least one of the multiple monitoring objects in the chamber 10, and generates monitoring image data IM1. The camera 5 outputs the monitoring image data IM1 to the control unit 90. The control unit 90 also functions as an image processing unit that processes the monitoring image data IM1.

[0070] The monitoring camera 5 may be an imaging camera 52 that receives visible light. The imaging camera 52 includes a solid-state imaging element, such as a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), and an optical system, such as a lens. The monitoring image data IM1 generated by the imaging camera 52 may also be referred to as captured image data or two-dimensional image data. In the example of FIG. 3 , the camera 5 is disposed vertically above and radially outward of the substrate holder 2. The camera 5 is fixed to the chamber 10 via a fixing member (not shown). The imaging direction of the camera 5 is set diagonally downward and toward the rotation axis Q1. Here, the imaging area of ​​the camera 5 (i.e., the monitoring area) may be set to include, for example, the entire upper opening of the guard 7 located at the guard processing position (see also FIG. 5B ).

[0071] 4 and 5 are diagrams schematically showing an example of monitoring image data IM1 generated by the imaging camera 52. Fig. 4 shows monitoring image data IM1 generated by the imaging camera 52 capturing an image of the monitoring area under the following conditions: the imaging camera 52 captures an image of the monitoring area when the guard 7 is located at the guard standby position and the plurality of chuck pins 22 are in contact with the release position. In the example of Fig. 4, the substrate W has not yet been loaded, and the monitoring image data IM1 includes the entirety of each of the plurality of chuck pins 22.

[0072] 5A shows monitor image data IM1 generated by the imaging camera 52 imaging the monitoring area under the following conditions: the imaging camera 52 images the monitoring area when the guard 7 is positioned at the guard standby position and multiple chuck pins 22 are in contact with the peripheral edge of the substrate W. In FIG. 5A, the substrate W is held by multiple chuck pins 22, and each chuck pin 22 is partially hidden by the substrate W. FIG. 5B shows monitor image data IM1 generated by the imaging camera 52 imaging the monitoring area under the following conditions: the imaging camera 52 images the monitoring area when the guard 7 is positioned at the guard processing position, multiple chuck pins 22 are in contact with the peripheral edge of the substrate W, and the nozzle 31 is positioned at the processing position. In the example of FIG. 5B, the lower chuck pin 22 in the monitor image data IM1 is hidden by the guard 7.

[0073] Various objects such as the nozzle 31, the chuck pins 22, and the guard 7 may be used as the monitored objects in the chamber 10. For example, the processing unit 1 may monitor the states of the monitored objects, such as the position of the nozzle 31, the state of the processing liquid discharged from the nozzle 31 (discharge state, discharge stop state, etc.), the position of the chuck pins 22, and the position of the guard 7, based on the monitored image data IM1. In the following, a description will be given taking a plurality of chuck pins 22 as a specific example of the monitored objects.

[0074] 4, the substrate holder 2 does not hold a substrate W, and the substrate W has not yet been loaded into the chamber 10. Therefore, the monitoring image data IM1 includes all of the chuck pins 22 on the spin base 21. In the example of Fig. 4, the multiple chuck pins 22 include chuck pin 22a (e.g., corresponding to the first monitored object), chuck pin 22b (e.g., corresponding to the second monitored object), chuck pin 22c, and chuck pin 22d. In the example of Fig. 4, chuck pin 22a is located at the back as viewed from the imaging camera 52, chuck pin 22b is located at the right as viewed from the imaging camera 52, chuck pin 22c is located at the front as viewed from the imaging camera 52, and chuck pin 22d is located at the left as viewed from the imaging camera 52.

[0075] For example, when the substrate W is loaded, the second transport part 122 delivers the substrate W to the substrate holder 2 with the chuck pins 22 positioned at the release position. During this loading, it is not desirable for the chuck pins 22 to be positioned at the holding position. For this reason, the processing unit 1 may monitor the positions of the chuck pins 22 based on the monitoring image data IM1.

[0076] Here, the contour shape of the chuck pin 22 may become unclear in the monitoring image data IM1. For example, the focus position of the lens of the imaging camera 52 may be set approximately near the center of the spin base 21. In this case, the contour of an object that is far from the focus position in the imaging direction will be blurred in the monitoring image data IM1. In other words, the contour shape of the object may become unclear. This is because light from a single point on the object is incident across multiple pixels of the solid-state imaging device through the lens of the imaging camera 52. As a result, the pixel values ​​of multiple pixels are averaged, making the contour shape unclear. In other words, this averaging lowers the spatial frequency of the images representing the object in the monitoring image data IM1, making the contour shape unclear. In other words, the number of pixels required to represent the object is reduced. In the example of Figure 4, dashed lines adjacent to the contour lines are added to schematically indicate that the contour shape is unclear.

[0077] In the example of FIG. 4, since the chuck pins 22a and 22c are separated from the focus position in the imaging direction, the contour shapes of the chuck pins 22a and 22c may be less clear than the contour shapes of the chuck pins 22b and 22d.

[0078] Furthermore, the smaller the size of an object in the monitoring image data IM1, the less clear the shape of the object's contour may be. This is because the smaller the size of an object in the monitoring image data IM1, the fewer pixels are required to represent the object's contour. In the example of FIG. 4 , the distance between the chuck pin 22a and the imaging camera 52 is longer than the distance between the other chuck pins 22 and the imaging camera 52, so the chuck pin 22a is the smallest in the monitoring image data IM1. Therefore, the contour shape of the chuck pin 22a may be less clear than the contour shapes of the other chuck pins 22. For example, the contour shape of the chuck pin 22a may be less clear than the contour shape of the chuck pin 22c.

[0079] As described above, the clarity of the contour shapes of the chuck pins 22 in the monitoring image data IM1 may differ among the chuck pins 22a to 22d. If the monitoring image data IM1 including an object with such an unclear contour is used to monitor the object, the monitoring accuracy may be reduced.

[0080] Therefore, the control unit 90 generates high-resolution image data IM2 by increasing the resolution of the monitoring image data IM1 using the learned model LM for the high-resolution processing. The control unit 90 monitors the monitoring target based on the high-resolution image data IM2.

[0081] 2, the memory unit 94 stores a plurality of learned models LM corresponding to a plurality of monitored objects, respectively. More specifically, parameters used in the learned models LM are stored in the memory unit 94. Here, the plurality of learned models LM stored in the memory unit 94 include a learned model LMa (e.g., corresponding to a first learned model) corresponding to the chuck pin 22a, a learned model LMb (e.g., corresponding to a second learned model) corresponding to the chuck pin 22b, a learned model LMc corresponding to the chuck pin 22c, and a learned model LMd corresponding to the chuck pin 22d.

[0082] Each trained model LM is generated based on training image data including the corresponding monitored object. That is, trained model LMa is generated based on training image data including chuck pin 22a, trained model LMb is generated based on training image data including chuck pin 22b, trained model LMc is generated based on training image data including chuck pin 22c, and trained model LMd is generated based on training image data including chuck pin 22d. The generation method will be described later.

[0083] 2, the resolution correspondence data D1 is stored in the storage unit 94. Table 1 shows the resolution correspondence data D1.

[0084]

[0085] As shown in Table 1, the resolution correspondence data D1 indicates a plurality of resolution magnifications RM corresponding to a plurality of monitoring objects, respectively. In Table 1, a resolution magnification RMa (e.g., corresponding to the first resolution magnification) is set corresponding to the chuck pin 22a, a resolution magnification RMb (e.g., corresponding to the second resolution magnification) is set corresponding to the chuck pin 22b, a resolution magnification RMc is set corresponding to the chuck pin 22c, and a resolution magnification RMd is set corresponding to the chuck pin 22d. Each resolution magnification RM is set in advance, for example, so that the clarity of the outline of each monitoring object (chuck pin 22) is at least a value sufficient for monitoring. The resolution magnification RM can be set by experiment or simulation.

[0086] The control unit 90 generates high-resolution image data IM2 by increasing the resolution of the monitoring image data IM1 using a learned model LM corresponding to the monitored object. At this time, the control unit 90 increases the resolution of the monitoring image data IM1 by a resolution magnification RM corresponding to the monitored object. Specifically, the control unit 90 inputs the monitoring image data IM1 and the resolution magnification RM to the learned model LM, and generates high-resolution image data IM2 through a resolution increase process using the learned model LM.

[0087] The resolution enhancement process using the trained model LM may be, for example, a process using super-resolution technology. For example, the trained model LM may be a trained model using deep learning, and specific examples thereof include a trained model using VDSR, a trained model using ResNet, a trained model using SRResNet, or a trained model using an architecture in which the batch normalization layer is removed from SRResNet (Bee Lim, et. al., "Enhanced Deep Residual Networks for Single Image Super-Resolution," Department of ECE, ASRIM, Seoul National University, 08826, Seoul, Korea, arXiv:1707.02921v1 [cs.CV] 10 Jul 2017).

[0088] The control unit 90 also monitors each monitoring object based on the high-resolution image data IM2. For example, the control unit 90 monitors the position of each chuck pin 22 based on the high-resolution image data IM2.

[0089] FIG. 6 is a flowchart showing an example of a monitoring method using the processing unit 1. First, the imaging camera 52 captures an image of the monitoring area and generates monitoring image data IM1 (step S1: image generation process). The imaging camera 52 outputs the monitoring image data IM1 to the control unit 90. Here, as an example, the imaging camera 52 may capture an image of the monitoring area just before the second transport unit 122 loads the substrate W into the processing unit 1. Just before the second transport unit 122 loads the substrate W, the control unit 90 controls the pin drive unit 24 to stop the chuck pins 22 at the release position. Therefore, if no abnormality occurs, the chuck pins 22 are normally positioned at the release position. In other words, the monitoring image data IM1 shown in FIG. 4 is generated. On the other hand, if an abnormality occurs in the substrate holder 2, the chuck pins 22 may be positioned at a distance greater than the allowable distance from the release position.

[0090] The imaging camera 52 may capture an image of the monitoring area when the substrate holder 2 holds the substrate W received from the second transport part 122. In this case, if the chuck pins 22 move normally to the holding positions and the substrate holder 2 holds the substrate W normally, monitoring image data IM1 as exemplified in FIG.

[0091] Next, the control unit 90 monitors the monitoring object (e.g., the chuck pins 22) based on the monitoring image data IM1 (step S2: monitoring step). For example, the control unit 90 may determine whether or not each chuck pin 22 is located at the release position based on the monitoring image data IM1 of Fig. 4, or may determine whether or not each chuck pin 22 is located at the holding position based on the monitoring image data IM1 of Fig. 5(a).

[0092] Fig. 7 is a flowchart showing a specific example of the monitoring process, and Fig. 8 is a diagram showing an example of the flow of each data in the monitoring process. In the example of Fig. 7, the control unit 90 first cuts out multiple determination regions R1 each including multiple monitoring objects (e.g., chuck pins 22) from the monitoring image data IM1 to generate multiple determination image data IM11 (step S3: cutting out process). The determination image data IM11 is a part of the monitoring image data IM1. Each determination region R1 can be set so as not to include a monitoring object different from the corresponding monitoring object.

[0093] In the examples of FIGS. 4 and 5A, determination regions R1a to R1d are shown as the multiple determination regions R1. Determination regions R1a (e.g., corresponding to the first determination region) to R1d correspond to chuck pins 22a to 22d, respectively. The determination region R1 includes at least a portion of the corresponding chuck pin 22 but does not include other chuck pins 22. For example, determination region R1a includes at least a portion of chuck pin 22a but does not include all of chuck pins 22b to 22d. In the example of FIG. 4, determination region R1a is set to include all of the chuck pins 22a on the spin base 21. Determination regions R1b (e.g., corresponding to the second determination region) to R1d are set in a similar manner. The determination region R1 has, for example, a rectangular shape.

[0094] In the following, the determination image data IM11 obtained by cutting out the determination region R1a from the monitor image data IM1 will be referred to as determination image data IM11a. The determination image data IM11 corresponding to the determination regions R1b to R1d will be referred to as determination image data IM11b to IM11d, respectively. Note that FIG. 8 shows the flow of processing for the monitor image data IM1 in FIG. 4.

[0095] Next, the control unit 90 determines the resolution magnification RM corresponding to each monitored object (step S4: resolution determination step). Specifically, the control unit 90 reads out the resolution correspondence data D1 from the storage unit 94, and reads out the resolution magnifications RMa to RMd corresponding to the chuck pins 22a to 22d included in the resolution correspondence data D1.

[0096] Next, the control unit 90 determines a learned model LM corresponding to each monitored object (step S5: learned model determination step). Specifically, the control unit 90 reads out parameters of the learned models LMa to LMd corresponding to the chuck pins 22a to 22d, respectively, from the storage unit 94.

[0097] Next, the control unit 90 inputs each determination image data IM11 and the corresponding resolution magnification RM into the corresponding trained model LM and generates high-resolution image data IM2 according to the algorithm of the trained model LM (step S6: resolution increasing process). For example, the control unit 90 inputs the determination image data IM11a and the resolution magnification RMa into the trained model LMa to generate high-resolution image data IM2a (e.g., corresponding to the first high-resolution image data). Similarly, the control unit 90 inputs the determination image data IM11b and the resolution magnification RMb into the trained model LMb to generate high-resolution image data IM2b (e.g., corresponding to the second high-resolution image data), inputs the determination image data IM11c and the resolution magnification RMc into the trained model LMc to generate high-resolution image data IM2c, and inputs the determination image data IM11d and the resolution magnification RMd into the trained model LMd to generate high-resolution image data IM2d.

[0098] Next, the control unit 90 monitors the monitoring object based on the high-resolution image data IM2 (step S7). That is, the control unit 90 monitors the chuck pin 22a based on the high-resolution image data IM2a, monitors the chuck pin 22b based on the high-resolution image data IM2b, monitors the chuck pin 22c based on the high-resolution image data IM2c, and monitors the chuck pin 22d based on the high-resolution image data IM2d.

[0099] As a specific example, the control unit 90 monitors the monitoring target based on a comparison between the high-resolution image data IM2 and reference image data RIM stored in the storage unit 94. The reference image data RIM is, for example, an image including a normal monitoring target and can be stored in advance in the storage unit 94. The reference image data RIM is set for each monitoring target. For example, reference image data RIMa to RIMd corresponding to the chuck pins 22a to 22d, respectively, are stored in the storage unit 94 as the reference image data RIM. The reference image data RIMa is, for example, an image including the normal chuck pin 22a. In the example of FIG. 8 , the reference image data RIMa includes the chuck pin 22a positioned in the release position. Similarly, the reference image data RIMb includes the normal chuck pin 22b, the reference image data RIMc includes the normal chuck pin 22c, and the reference image data RIMd includes the normal chuck pin 22d.

[0100] The position, shape, and size of the reference image data RIM within the monitoring area are the same as those of the corresponding determination area R1. For example, the position, shape, and size of the reference image data RIMa within the monitoring area are the same as those of the determination area R1a. Furthermore, the clarity of the contours of the monitored object in the reference image data RIM is higher than that of the monitored object in the monitoring image data IM1 and is about the same as that of the monitored object in the high-resolution image data IM2. A method for generating such reference image data RIM will be described in detail later.

[0101] If the chuck pin 22a is positioned correctly at the release position, the high-resolution image data IM2a will generally match the reference image data RIMa. On the other hand, if the chuck pin 22a is not positioned at the release position due to an abnormality or the like, the high-resolution image data IM2a will not resemble the reference image data RIMa. Conversely, if the high-resolution image data IM2a is similar to the reference image data RIMa, it can be determined that the chuck pin 22a is positioned correctly at the release position, and if the high-resolution image data IM2a is not similar to the reference image data RIMa, it can be determined that the chuck pin 22a is positioned at a position other than the release position. The same is true for chuck pins 22b to 22d.

[0102] In the example of FIG. 5A, in the monitoring image data IM1, chuck pins 22a to 22d are positioned and stopped at the correct holding position. In this case, reference image data RIMa to RIMd include images similar to the determination regions R1a to R1d in FIG. 5A, respectively. However, the clarity of the contours of the monitored object in the reference image data RIMa to RIMd is comparable to the clarity of the contours of the monitored object in the high-resolution image data IM2. In this case, if the high-resolution image data IM2a is similar to the reference image data RIMa, it can be determined that chuck pin 22a is positioned correctly at the holding position. If the high-resolution image data IM2a is not similar to the reference image data RIMa, it can be determined that chuck pin 22a is positioned at a position different from the holding position. The same applies to chuck pins 22b to 22d.

[0103] Therefore, the control unit 90 compares the high-resolution image data IM2 with the corresponding reference image data RIM for each chuck pin 22, and determines the state of the chuck pin 22 based on the comparison result. As a specific example, the control unit 90 calculates the similarity between the high-resolution image data IM2 and the reference image data RIM. The similarity is not particularly limited, and may be, for example, a known similarity such as the sum of squared differences of pixel values, the sum of absolute differences of pixel values, normalized cross-correlation, or zero-mean normalized cross-correlation.

[0104] The control unit 90 determines for each chuck pin 22 whether the similarity is equal to or greater than a predetermined threshold value. The threshold value is a threshold value for determining whether or not an abnormality exists, and is set in advance, for example, by simulation or experiment, and stored in the storage unit 94. When the similarity is equal to or greater than the threshold value, the control unit 90 determines that the chuck pin 22 is located in a normal position, and when the similarity is less than the threshold value, the control unit 90 determines that the chuck pin 22 is located in a position different from the normal position. When the similarity is less than the threshold value, the control unit 90 may determine that an abnormality has occurred in the chuck pin 22. When the control unit 90 determines that an abnormality has occurred, the control unit 90 may interrupt the processing.

[0105] As described above, in this embodiment, the memory unit 94 stores the learned models LM corresponding to each of the multiple monitored objects. The control unit 90 then uses the learned models LM corresponding to the monitored objects to perform a resolution enhancement process on the monitored image data IM1 (specifically, the determination image data IM11) to generate high-resolution image data IM2. The learned models LM can more appropriately clarify the contours of the corresponding monitored objects, so the contours of the monitored objects are clear in the high-resolution image data IM2. For example, the high-resolution image data IM2 has a PSNR (Peak Signal to Noise Ratio) of 23 or greater, and a structural similarity (SSIM) of 0.75 or greater.

[0106] Therefore, the control unit 90 can monitor the monitoring object with higher accuracy based on the high-resolution image data IM2. In other words, the control unit 90 can determine the state of the monitoring object (here, the position of the chuck pin 22) with higher accuracy.

[0107] Furthermore, there is no need to use an expensive high-resolution imaging camera as the imaging camera 52. This allows the manufacturing cost of the substrate processing apparatus 100 to be reduced.

[0108] <First Method for Generating Trained Model> Next, a method for generating the trained model LM will be described. The training step for generating the trained model LM is performed before the image generation step (step S1). Specifically, the trained model LM is generated before the substrate processing apparatus 100 performs processing on the substrate W. FIG. 9 is a flowchart showing an example of a specific operation of the training step. First, the high-performance camera 51 captures an image of the monitoring area (step S11: training image generation step). FIG. 10 is a diagram schematically showing an example of the high-performance camera 51 provided in the chamber 10. The high-performance camera 51 may be installed by an operator in place of the imaging camera 52. In other words, the high-performance camera 51 may be attached in the same position as the imaging camera 52.

[0109] The high-performance camera 51 can generate captured image data with a higher resolution than the imaging camera 52. For example, the number of pixels of the high-performance camera 51 may be greater than the number of pixels of the imaging camera 52. Furthermore, the lens inside the high-performance camera 51 is provided so as to be movable in the optical axis direction. In other words, the high-performance camera 51 has an optical zoom function and a focus adjustment function. On the other hand, the imaging camera 52 does not necessarily have an optical zoom function, and does not necessarily have an optical focus adjustment function. In other words, the focus position of the imaging camera 52 may be fixed. Such an imaging camera 52 is inexpensive.

[0110] The imaging area of ​​the high-performance camera 51 may be the same as the imaging area of ​​the imaging camera 52. The imaging area of ​​the high-performance camera 51 also includes the monitored object (here, the chuck pin 22). The imaging area of ​​the high-performance camera 51 may include a plurality of chuck pins 22.

[0111] For example, the high-performance camera 51 first captures an image of the monitoring area with the chuck pin 22a in focus. That is, the high-performance camera 51 captures an image of the monitoring area with the chuck pin 22a in focus rather than at least one of the other monitored objects (e.g., the chuck pin 22d). The contour shape of the chuck pin 22a in the captured image data generated by the high-performance camera 51 is clear, the PSNR of the captured image data is 23 or higher, and the SSIM of the captured image data is 0.75 or higher. Hereinafter, the captured image data generated by the high-performance camera 51 will also be referred to as high-resolution training data IM3. Furthermore, the high-resolution training data IM3 captured with the chuck pin 22a in focus will be referred to as high-resolution training data IM3a.

[0112] The high-performance camera 51 may capture images of the monitoring area at multiple times to generate multiple high-resolution learning data IM3a. For example, the high-performance camera 51 may capture images of the monitoring area in different states of the chuck pins 22a to generate the high-resolution learning data IM3a. As a specific example, the high-performance camera 51 may capture images of the monitoring area when the chuck pins 22a are in the release position (see, for example, FIG. 4 ) and when the chuck pins 22a are in the holding position (see, for example, FIG. 5( a) ). The state in which the chuck pins 22a are in the holding position can be achieved by the substrate holder 2 holding the substrate W.

[0113] The high-performance camera 51 may capture an image of the monitoring area in a state where the monitored object is in various other states (for example, an abnormal state) and generate the high-resolution learning data IM3, without being limited to the above. For example, the high-performance camera 51 may capture an image of the monitoring area in a state where the chuck pin 22 is stopped at an intermediate position between the holding position and the release position.

[0114] 11 is a diagram schematically illustrating the flow of data in generating a trained model. As shown in FIG. 11, the control unit 90 may generate high-resolution training data IM31a by extracting a region including the chuck pin 22a from the high-resolution training data IM3a generated by the high-performance camera 51. This region may be the same region as the determination region R1a, for example. The high-resolution training data IM31a may have a PSNR of 23 or higher, and an SSIM of 0.75 or higher.

[0115] Next, the control unit 90 downsamples the multiple high-resolution training data IM3 by a factor equal to or less than the reciprocal of the corresponding resolution factor RM, and converts each of the multiple high-resolution training data IM3 into multiple low-resolution training data IM4 (step S12: resolution reduction step). For example, the control unit 90 downsamples the multiple high-resolution training data IM31a by a factor equal to or less than the reciprocal of the resolution factor RMa, and converts each of the multiple high-resolution training data IM31a into multiple low-resolution training data IM4a. The downsampling method is not particularly limited, and bicubic downsampling, for example, may be applied.

[0116] Next, the control unit 90 trains a learning model using multiple sets of high-resolution learning data IM3a (e.g., high-resolution learning data IM31a) and low-resolution learning data IM4a to generate a learned model LMa corresponding to the chuck pin 22a (step S13: model generation step). Specifically, the control unit 90 obtains parameters of the learned model LMa corresponding to the chuck pin 22a. Then, the control unit 90 stores the learned model LMa (parameters) in the storage unit 94.

[0117] Next, the control unit 90 determines whether learned models LM corresponding to all monitored objects have been generated (step S14). If all learned models LM have not yet been generated, the control unit 90 executes steps S11 to S13 for the next monitored object (e.g., the chuck pin 22b). For example, the high-performance camera 51 captures an image of the monitored area with the chuck pin 22b in focus to generate high-resolution learning data IM3 (step S11). The control unit 90 then extracts from the high-resolution learning data IM3 an area identical to the determination area R1b including the chuck pin to generate high-resolution learning data IM31, downsamples the high-resolution learning data IM31 by a factor equal to or less than the reciprocal of the resolution factor RMb, and generates low-resolution learning data IM4 (step S12). The control unit 90 then trains a learning model using the high-resolution learning data IM31 and the low-resolution learning data IM4 corresponding to the chuck pin 22b to generate a learned model LMb (step S13). The same applies to the chuck pins 22c and 22d.

[0118] In step S14, when it is determined that all the learned models LM have been generated, the control unit 90 ends the process. Then, the operator appropriately removes the high-performance camera 51 from the chamber 10 and installs the imaging camera 52 inside the chamber 10.

[0119] As described above, the control unit 90 generates a trained model LM based on a set of high-resolution training data IM3 that does not include other monitoring objects and clearly includes the monitoring object, and low-resolution training data IM4 generated from the high-resolution training data IM3. This allows the control unit 90 to generate a trained model LM that is suitable for the monitoring object. Therefore, the control unit 90 can increase the resolution of monitoring image data IM1 that includes the monitoring object with higher accuracy by using the trained model LM that corresponds to the monitoring object.

[0120] Furthermore, in the above example, the high-performance camera 51 captures the same monitoring area as the imaging camera 52. Therefore, the optical characteristics of the lens group of the high-performance camera 51 through which light passes during imaging are similar to the optical characteristics of the lens group of the imaging camera 52 through which light passes during imaging. This makes it possible to make the high-resolution learning data IM3 captured by the high-performance camera 51 more equivalent to high-resolution image data obtained by increasing the resolution of the monitoring image data IM1 captured by the imaging camera 52. Therefore, the control unit 90 can use the learned model LM to increase the resolution of the monitoring image data IM1 with even higher accuracy.

[0121] Furthermore, the high-performance camera 51 captures images of the chuck pin 22a at multiple times when the displaceable chuck pin 22a is positioned at different positions, thereby generating multiple high-resolution training data IM3a. Specifically, the high-performance camera 51 captures images of the monitoring area at each time when the chuck pin 22a is positioned at the release position and each time when the chuck pin 22a is positioned at the holding position. The high-performance camera 51 then generates a trained model LMa using the multiple high-resolution training data IM3. This allows the control unit 90 to increase the resolution of the monitoring image data IM1 with higher accuracy, regardless of the state (here, the position) of the chuck pin 22a in the image generation process (step S1).

[0122] <Reference Image Data> Next, a method for generating the reference image data RIM will be described. A reference image registration step for generating the reference image data RIM and storing the reference image data RIM in the storage unit 94 is performed before the image generation step (step S1). The reference image registration step is performed after the training step. That is, the reference image registration step is performed after the learned model LM is generated. Specifically, the reference image registration step may be performed before the substrate processing apparatus 100 performs processing on a substrate W, for example, when the substrate processing apparatus 100 is installed. FIG. 12 is a flowchart showing an example of a specific operation of the reference image registration step. FIG. 13 is a diagram showing an example of the flow of each data. Here, as an example, the processing unit 1 generates reference image data RIM corresponding to the monitored image data IM1 of FIG. 4.

[0123] First, the imaging camera 52 captures an image of the monitoring area to generate preliminary image data IM5 (step S21: preliminary image generation step). For example, the imaging camera 52 captures an image of the monitoring area when the substrate W has not been loaded and the chuck pins 22 are normally positioned at the release positions. Therefore, the preliminary image data IM5 includes the chuck pins 22 normally positioned at the release positions.

[0124] 5A. In this case, the imaging camera 52 captures an image of the monitoring area in a state where the substrate holder 2 normally holds the substrate W. In this case, the preliminary image data IM5 includes the chuck pins 22 normally positioned at the holding positions.

[0125] Next, the control unit 90 cuts out a determination region R1 for each monitored object from the preliminary image data IM5 to generate preliminary image data IM51 (step S22: cutting-out process). Specifically, the control unit 90 cuts out a determination region R1a from the preliminary image data IM5 to generate preliminary image data IM51a. The preliminary image data IM51a includes the chuck pin 22a in a normal state. In the example of FIG. 13, the preliminary image data IM51a includes the chuck pin 22a positioned in the released position. Furthermore, the control unit 90 cuts out determination regions R1b to R1d from the preliminary image data IM5 to generate preliminary image data IM51d from the preliminary image data IM5.

[0126] Next, the control unit 90 determines the resolution magnification RM corresponding to each monitored object (step S23: resolution determination step). Specifically, the control unit 90 reads out the resolution correspondence data D1 from the storage unit 94, and reads out the resolution magnifications RMa to RMd corresponding to the chuck pins 22a to 22d included in the resolution correspondence data D1.

[0127] Next, the control unit 90 determines a learned model LM corresponding to each monitored object (step S24: learned model determination step). Specifically, the control unit 90 reads out parameters of the learned models LMa to LMd corresponding to the chuck pins 22a to 22d, respectively, from the storage unit 94.

[0128] Next, the control unit 90 inputs each preliminary image data IM51 and the corresponding resolution magnification RM into the corresponding trained model LM, and increases the resolution of the preliminary image data IM51 according to the algorithm of the trained model LM to generate reference image data RIM (step S25: reference image generation process). For example, the control unit 90 inputs the preliminary image data IM51a and the resolution magnification RMa into the trained model LMa to generate reference image data RIMa. Similarly, the control unit 90 generates reference image data RIMd from the reference image data RIMb.

[0129] Next, the control unit 90 stores the reference image data RIMa and the reference image data RIMd in the storage unit 94 (step S26: storage step).

[0130] As described above, in the above example, the imaging camera 52 captures an image of the monitoring area and generates preliminary image data IM5 (step S21). This allows the position and size of the chuck pin 22a in the monitoring image data IM1 captured in the image generation process (step S1) to be equal to the position and size of the chuck pin 22a in the preliminary image data IM5. This increases the similarity between the determination image data IM11a, which includes the normal chuck pin 22a, and the reference image data RIMa, which includes the normal chuck pin 22a. This allows the control unit 90 to more accurately determine the position of the chuck pin 22a based on the similarity between the determination image data IM11a and the reference image data RIMa. The same applies to chuck pins 22b to 22d.

[0131] In the above example, the control unit 90 generates the reference image data RIMa using the learned model LMa used to generate the determination image data IM11a. That is, the determination image data IM11a and the reference image data RIMa are generated using the same algorithm. This increases the similarity between the determination image data IM11a, which includes the normal chuck pin 22a, and the reference image data RIMa, which includes the normal chuck pin 22a. Therefore, the control unit 90 can more accurately determine the position of the chuck pin 22a based on the similarity between the determination image data IM11a and the reference image data RIMa. The same applies to the chuck pins 22b to 22d.

[0132] In the specific example described above, the control unit 90 generates a learned model LM for each monitored object. However, this embodiment is not necessarily limited to this. The control unit 90 may generate one learned model LM and store the learned model LM in the storage unit 94. In other words, the control unit 90 may generate one learned model LM common to multiple monitored objects. For example, the control unit 90 may increase the resolution of monitored image data IM1 including multiple chuck pins 22 using the learned model LM to generate high-resolution image data IM2, and monitor each chuck pin 22 based on each determination region R1 in the high-resolution image data IM2.

[0133] <Second Method for Generating Trained Model> In the above example, the control unit 90 generated the trained model LM based on the high-resolution training data IM3 and the low-resolution training data IM4 generated by the high-performance camera 51. However, this embodiment is not necessarily limited to this. As described below, the control unit 90 may generate the trained model LM based on training data generated by simulation. For simplicity, the following describes a case where a trained model LM common to multiple monitored objects is generated.

[0134] FIG. 14 is a functional block diagram showing an example of the internal configuration of the control unit 90. The control unit 90 includes a training image generation unit 911 and a training unit 912. Three-dimensional data D2 is input to the training image generation unit 911. As shown in FIG. 14, the three-dimensional data D2 may be stored in the memory unit 94. The three-dimensional data D2 includes geometric information about the shape of each component within the chamber 10 of the processing unit 1. The three-dimensional data D2 may also be referred to as a three-dimensional model. The substrate holder 2, the nozzle 31, the nozzle movement driver 37, and the guard 7 are applied to each component within the chamber 10. In other words, the three-dimensional data D2 is three-dimensional data that models the monitored object within the chamber 10. The three-dimensional data D2 may be in any data format, such as CAD (Computer-Aided Design) data. The three-dimensional data D2 may further include information on at least one of the material and surface condition (e.g., pattern) of each component within the chamber 10. Hereinafter, the space inside the chamber 10 represented by the three-dimensional data D2 will be referred to as a virtual internal space.

[0135] The training image generation unit 911 generates simulation image data obtained by capturing an image of the virtual internal space using a virtual camera virtually placed in the virtual internal space. The training image generation unit 911 may be a three-dimensional simulator, and may be a functional unit based on a 3DCG (Dimension Computer Graphics) program such as Blender. For example, a 3DCG program is stored in the storage unit 921 (or the storage unit 94), and the data processing unit 91 reads and executes the program, thereby causing the control unit 90 to realize the function of the training image generation unit 911.

[0136] For example, the training image generation unit 911 may have a three-dimensional rendering function. For example, algorithms such as rasterization, ray tracing, and radiosity can be applied to three-dimensional rendering. For example, by applying either ray tracing or radiosity, the training image generation unit 911 generates simulation image data obtained by a virtual camera capturing an image of a virtual interior space illuminated by a virtual lighting device. Specifically, the training image generation unit 911 generates the simulation image data based on information about the virtual lighting device and information about the virtual camera. The information about the virtual lighting device may include information such as the type of virtual lighting device (e.g., point light source, sunlight, spotlight), the amount and position of the illumination light, etc. The information about the virtual camera may include optical information such as the focus of the lens, the number of pixels, and the position, etc. Here, the control unit 90 uses the simulation image data generated by the training image generation unit 911 as training data.

[0137] Fig. 15 is a flowchart showing a second example of the specific operation of the training process. In the example of Fig. 15, the training image generation unit 911 generates high-resolution simulation image data and low-resolution simulation image data as high-resolution training data IM3 and low-resolution training data IM4, respectively, through a three-dimensional simulation (step S31: training image generation process). Fig. 16 is a diagram schematically showing an example of the high-resolution training data IM3, and Fig. 17 is a diagram schematically showing an example of the low-resolution training data IM4.

[0138] Specifically, the training image generator 911 generates high-resolution training data IM3 obtained by imaging the virtual internal space with a high first resolution from a predetermined position using a virtual camera. The position of the virtual camera may be set, for example, in the virtual internal space above the substrate holder 2 and radially outward from the guard 7, similar to the real space. More specifically, the position of the virtual camera may be set at a position in the virtual internal space where the entire upper opening of the guard 7 is included in the high-resolution training data IM3. The resolution may be adjusted, for example, by the number of pixels of the virtual camera. Since the imaging area (monitoring area) of the virtual camera is the same, the greater the number of pixels, the higher the resolution.

[0139] The training image generation unit 911 also generates low-resolution training data IM4 obtained by capturing an image of the virtual internal space from the same position with a virtual camera at a lower second resolution. The first and second resolutions are, for example, set in advance so that the resolution magnification, which is the ratio of the first resolution to the second resolution, is equal to or greater than a predetermined value. As can be seen from FIGS. 16 and 17 , the high-resolution training data IM3 and the low-resolution training data IM4 differ in terms of resolution but are otherwise identical. In other words, the training image generation unit 911 generates the high-resolution training data IM3 and the low-resolution training data IM4, respectively, using different resolutions (number of pixels), the same virtual lighting conditions, the same virtual camera position, and the same virtual internal space.

[0140] This makes it possible to obtain a set of high-resolution training data IM3 and low-resolution training data IM4, which have different resolutions. Note that the term "set" here refers to two sets of simulation image data (i.e., high-resolution training data IM3 and low-resolution training data IM4) that have the same conditions except for conditions that affect the resolution (e.g., the number of pixels).

[0141] The training image generation unit 911 generates multiple sets of high-resolution training data IM3 and low-resolution training data IM4 by, for example, changing at least one of the position and resolution (number of pixels) of the virtual camera, the conditions of the virtual lighting device (e.g., type, position, and light intensity), and the state of each component in the virtual internal space. The state of each component in the virtual internal space can be, for example, the position of the nozzle 31, the position of the guard 7, and the presence or absence of the substrate W in the virtual internal space. The values ​​of these conditions may be set in advance, for example, or may be input through a user interface (not shown). The training image generation unit 911 generates sets of high-resolution training data IM3 and low-resolution training data IM4 under each condition while changing these conditions in various ways. This allows multiple sets of high-resolution training data IM3 and low-resolution training data IM4 to be obtained.

[0142] Next, the training unit 912 trains the learning model based on multiple sets of high-resolution learning data IM3 and low-resolution learning data IM4 to generate a learned model LM (step S32: training step).

[0143] As described above, if the control unit 90 generates the trained model LM based on simulation image data, an expensive high-performance camera 51 is not required. This allows the control unit 90 to generate the trained model LM at a lower cost. Furthermore, in a 3D simulation, the resolution can be changed over a wider range and more finely than the resolution of an actual high-performance camera 51. This allows the control unit 90 to easily obtain sets of high-resolution training data IM3 and low-resolution training data IM4 having various resolution magnifications.

[0144] As described above, the control unit 90 may generate a single trained model LM common to multiple monitored objects, or may generate a trained model LM for each monitored object. That is, the training image generation unit 911 may generate multiple sets of high-resolution training data IM3 and low-resolution training data IM4 for each monitored object. For example, the training image generation unit 911 generates multiple sets of high-resolution training data IM3 and low-resolution training data IM4 for each chuck pin 22. The resolution magnification of each of the multiple sets corresponding to each chuck pin 22 is equal to or greater than the resolution magnification RM set for the chuck pin 22. For example, the resolution magnification of each of the multiple sets corresponding to chuck pin 22c is equal to or greater than the resolution magnification RMc. Furthermore, the training image generation unit 911 may extract a determination region R1c from the high-resolution training data IM3 and low-resolution training data IM4 as training data for chuck pin 22c. The training unit 912 may generate a trained model LMc for the chuck pin 22c based on multiple pairs of the determination region R1c of the high-resolution training data IM3 and the low-resolution training data IM4. The same applies to the other chuck pins 22.

[0145] Second Embodiment FIG. 18 is a longitudinal cross-sectional view schematically illustrating an example of the configuration of a processing unit 1 according to a second embodiment. In the second embodiment, the camera 5 is a depth camera 53. The depth camera 53 may also be referred to as a distance camera, a 3D camera, a depth sensor, a distance sensor, or a 3D sensor. The depth camera 53 according to the second embodiment measures the depth to each object in the monitoring area and generates monitoring image data IM6 that indicates the depth for each pixel. The depth here refers to the distance between the depth camera 53 and each object. The monitoring image data IM6 may also be referred to as depth image data or distance image data. FIG. 19 is a diagram schematically illustrating an example of monitoring image data IM6. In the example of FIG. 19, each pixel is indicated by denser sandy hatching for objects closer to the depth camera 53. Note that the hatching for the guard 7 is omitted in the example of FIG. 19. Since the measurement accuracy of a depth camera is typically not that high, the contours of each object in the monitoring image data IM6 may be less clear. For example, the contour of the chuck pin 22 in the monitor image data IM6 may be less clear than in the example of FIG.

[0146] The depth camera 53 may be, for example, a time-of-flight (TOF) depth camera. In this case, the depth camera 53 includes a light-emitting unit, a light-receiving unit, and a calculation unit, all of which are not shown. The light-emitting unit may include a light source that emits measurement light toward the monitoring area and a lens positioned in front of the light source. The measurement light is not particularly limited, but may be, for example, invisible light such as infrared light. The measurement light from the light source is irradiated onto the monitoring area through a lens. The light-receiving unit may include multiple light-receiving elements that receive the measurement light reflected from the monitoring area and an imaging lens positioned in front of the multiple light-receiving elements that directs the measurement light to each light-receiving element. The calculation unit calculates the distance to an object corresponding to each light-receiving element based on the difference between the timing at which the light-emitting unit emits the measurement light and the timing at which the light-receiving unit receives the measurement light. The calculation unit may have a hardware configuration similar to that of the control unit 90. Note that the functions of the calculation unit may be included in the control unit 90. In this case, some of the functions of the control unit 90 function as the depth camera 53.

[0147] 18, a trained model LM1 is stored in the memory unit 94. The trained model LM1 is a trained model that performs high-resolution processing on the surveillance image data IM6. The trained model LM1 is a trained model that uses, for example, deep learning, and its specific algorithm example is the same as that of the trained model LM.

[0148] An example of a monitoring method using the processing unit 1 according to the second embodiment is the same as that shown in Fig. 6. However, in step S1, the light-emitting unit of the depth camera 53 emits measurement light toward the monitoring area, the light-receiving unit of the depth camera 53 receives the measurement light from the monitoring area, and the calculation unit of the depth camera 53 generates monitoring image data IM6 indicating the depth to each object in the monitoring area based on the difference between the timing of irradiation by the light-emitting unit and the timing of reception by the light-receiving unit.

[0149] Next, in step S2, the control unit 90 uses the learned model LM1 to increase the resolution of the monitoring image data IM6 to generate high-resolution image data IM7. Then, the control unit 90 monitors the monitoring object based on the high-resolution image data IM7. For example, the control unit 90 may identify the position of each chuck pin 22 in the high-resolution image data IM7 and determine whether the position of each chuck pin 22 is appropriate. For example, similar to the first embodiment, normal monitoring image data generated by the depth camera 53 may be increased in resolution using the learned model LM to generate reference image data and store it in the storage unit 94. The control unit 90 may determine whether the chuck pins 22 are appropriate based on a comparison between the high-resolution image data IM7 and the reference image data.

[0150] This allows the control unit 90 to monitor the monitoring target based on the high-resolution image data IM7 that more clearly includes the monitoring target, thereby enabling the control unit 90 to monitor the monitoring target with higher accuracy.

[0151] <Method for generating trained model> Next, an example of a method for generating the trained model LM1 will be described. As in the first embodiment, the training step for generating the trained model LM1 is performed before the image generation step (step S1).

[0152] However, a high-precision depth camera 53 is difficult to obtain and expensive. Therefore, as an example, simulation image data generated by a three-dimensional simulator is used as the training data. Here, the training image generation unit 911 includes the function of a three-dimensional simulator. Similar to the first embodiment, three-dimensional data D2 is input to the control unit 90.

[0153] The training image generation unit 911 has a function of generating simulation image data obtained by a virtual depth camera in a virtual internal space. For example, the training image generation unit 911 calculates the distance from the position of the virtual depth camera to each position of each component in the virtual internal space for each pixel of the virtual depth camera, and generates the simulation image data. The position of the virtual depth camera is set in advance, and each position of each component is set in the three-dimensional data D2, so the training image generation unit 911 can calculate the distance to each position. This simulation image data includes the distance from the position of the virtual depth camera to each component in the virtual internal space for each pixel.

[0154] The training image generation unit 911 can generate simulation image data at a specified resolution. The resolution can be adjusted, for example, by the number of pixels of the virtual depth camera. Since the measurement area (monitoring area) measured by the virtual depth camera is the same, the greater the number of pixels, the higher the resolution. For example, the training image generation unit 911 generates simulation image data at a high first resolution as high-resolution training data IM8. The training image generation unit 911 also generates simulation image data at a second resolution lower than the first resolution as low-resolution training data IM9. Alternatively, the training image generation unit 911 may downsample the high-resolution training data IM8 to generate the low-resolution training data IM9.

[0155] Fig. 20 is a diagram schematically illustrating an example of high-resolution training data IM8, and Fig. 21 is a diagram schematically illustrating an example of low-resolution training data IM9. In the examples of Fig. 20 and Fig. 21, the closer an object is to the virtual depth camera, the closer each pixel is to black.

[0156] An example of the specific operation of the training process for generating the trained model LM1 is the same as that shown in FIG. 15. First, in step S31, the training image generation unit 911 generates multiple sets of high-resolution training data IM8 and low-resolution training data IM9 through 3D simulation, as in the first embodiment. The resolution magnification for each set is set to, for example, a predetermined value or higher. The training image generation unit 911 generates multiple sets by changing at least one of the resolution (number of pixels) and position of the virtual depth camera and the state of each configuration in the virtual internal space.

[0157] Next, in step S32, similarly to the first embodiment, the control unit 90 trains the learning model using multiple sets of high-resolution learning data IM8 and low-resolution learning data IM9 to generate a learned model LM1.

[0158] As described above, the control unit 90 generates the trained model LM1 that increases the resolution of the monitored image data IM6 based on the simulation image data. This makes it possible to generate the trained model LM1 without requiring a high-performance depth camera 53.

[0159] In particular, since the measurement accuracy of the depth camera is not very high, the clarity of objects in the depth image data is low. For example, this clarity is lower than the clarity of objects in the captured image data generated by the imaging camera. Therefore, when the depth camera measures the monitored area and generates high-resolution training data IM8, the resolution (clarity) of the high-resolution training data IM8 is relatively low. Therefore, it is particularly preferable to apply simulation image data to the training data used for the trained model LM1 for the depth camera 53. This is because clearer high-resolution training data IM8 can be easily obtained by 3D simulation.

[0160] <Imaging Camera and Depth Camera> In the second embodiment, the processing unit 1 may include two cameras 5. The first camera 5 is an imaging camera 52 that receives visible light. The second camera 5 is a depth camera 53. The imaging camera 52 receives light (e.g., visible light) from the monitoring area and generates monitoring image data IM1. The monitoring image data IM1 is captured image data of the monitoring area. The depth camera 53 receives measurement light (e.g., invisible light) reflected from each object in the monitoring area and generates monitoring image data IM6. The monitoring image data IM6 is depth image data that indicates the depth to each object in the monitoring area.

[0161] The storage unit 94 may store both a learned model LM for the surveillance image data IM1 and a learned model LM1 for the surveillance image data IM6.

[0162] The control unit 90 may increase the resolution of the monitoring image data IM1 using the learned model LM to generate high-resolution image data IM2, and may increase the resolution of the monitoring image data IM6 using the learned model LM1 to generate high-resolution image data IM7. Figure 22 is a diagram schematically showing an example of the operation of the control unit 90.

[0163] The control unit 90 may detect each object in the high-resolution image data IM2 based on the high-resolution image data IM2 and the high-resolution image data IM7. For example, an algorithm such as deep learning can be applied as an object detection algorithm. In this case, the control unit 90 may generate composite image data by synthesizing the high-resolution image data IM2 and the high-resolution image data IM7. "Synthesis" here refers to a process of matching pixels in the high-resolution image data IM2 and the high-resolution image data IM7 that correspond to the same portion of the monitoring area, and synthesizing the high-resolution image data IM2, which is captured image data, with the high-resolution image data IM7, which is depth image data. In other words, each pixel of the composite image data includes visual light information at a corresponding position in the monitoring area and depth information to that position. The control unit 90 may perform object detection based on the composite image data. As a specific example, the control unit 90 may detect the chuck pin 22 based on the composite image data. The control unit 90 does not necessarily need to synthesize the high-resolution image data IM2 and the high-resolution image data IM7, and may execute an object detection algorithm separately for each of the high-resolution image data IM2 and the high-resolution image data IM7.

[0164] According to this embodiment, the control unit 90 combines the high-resolution image data IM2 and the high-resolution image data IM7, which allows for more accurate generation of combined image data than combining the monitoring image data IM1 and the monitoring image data IM6, thereby improving the object detection accuracy.

[0165] The clarity of the object in the monitoring image data IM1 captured by the imaging camera 52 is higher than the clarity of the object in the monitoring image data IM6 captured by the depth camera 53. Therefore, the control unit 90 does not necessarily need to increase the resolution of the monitoring image data IM1. For example, the control unit 90 may detect an object based on the monitoring image data IM1 and the high-resolution image data IM7. In this case, the learned model LM is not required.

[0166] <Multiple Trained Models> As in the first embodiment, multiple trained models LM1 corresponding to multiple monitored objects may be stored in the storage unit 94 as trained models LM1 for the monitoring image data IM6. Each trained model LM1 increases the resolution of the monitoring image data IM6 by a resolution magnification RM corresponding to the monitored object to generate high-resolution image data IM7. The resolution magnification RM for each monitored object is set in advance and, for example, stored in the storage unit 94. For example, a resolution magnification RMa is set for the chuck pin 22a, a resolution magnification RMb is set for the chuck pin 22b, a resolution magnification RMc is set for the chuck pin 22c, and a resolution magnification RMd is set for the chuck pin 22d.

[0167] As in the first embodiment, multiple trained models LM1 can be generated by training using simulation image data generated by a three-dimensional simulator as training data.

[0168] Although the substrate processing apparatus 100 and the substrate processing method have been described in detail above, the above description is merely an example in all respects, and this disclosure is not limited thereto. Furthermore, the various modifications described above can be combined and applied as long as they are not mutually inconsistent. It is understood that many modifications not illustrated can be envisioned without departing from the scope of this disclosure.

[0169] For example, in the above description, the high-resolution learning data IM3 used to generate the trained model LM includes multiple monitored objects (here, chuck pins 22a to 22d). The control unit 90 then extracts high-resolution learning data IM31 corresponding to each monitored object from the high-resolution learning data IM3. However, the high-performance camera 51 may also capture an image of a monitored area including a single monitored object. For example, the high-performance camera 51 is mounted inside the chamber 10 so that its imaging direction can be changed. Specifically, the high-performance camera 51 may be fixed to the ceiling of the chamber 10 so that it can rotate around a vertical axis of rotation and a horizontal axis of rotation. This allows the processing unit 1 to change the imaging direction of the high-performance camera 51. The high-performance camera 51 may then capture an image of a monitored area including a single monitored object using a zoom function and a function to change its imaging direction, thereby generating high-resolution learning data. For example, the high-performance camera 51 may capture an image of a monitoring area including the chuck pin 22a but not including the chuck pins 22b to 22d, and generate high-resolution learning data for the chuck pin 22a. The same applies to the chuck pins 22b to 22d.

[0170] In the above example, the control unit 90 downsamples the high-resolution training data to convert it into low-resolution training data. However, the camera 5 may capture an image of the monitoring area to generate low-resolution training data.

[0171] 10 Chamber 100 Substrate processing apparatus 22, 22a to 22d Monitoring object (chuck pin) 5 Monitoring camera 51 High-performance camera 52 Imaging camera 53 Depth camera D2 Three-dimensional data IM1, IM6 Monitoring image data IM2, IM2a to IM2d, IM7 High-resolution image data IM3, IM31, IM31a to IM31d, IM8 High-resolution learning data IM4, IM4a to IM4d, IM9 Low-resolution learning data IM5, IM51a to IM51d Preliminary image data RIM, RIMa to RIMd Reference image data LM, LM1, LMa to LMd Learned model RM, RMa to RMd Resolution magnification S1 Image generation process (step) S2 Monitoring process (step) S11, S31 Learning image generation step (step) S12 Resolution reduction step (step) S13 Model generation step (step) S21 Preliminary image generation step (step) S25 Reference image generation step (step) S26, S32 Storage step (steps)

Claims

1. A monitoring method for monitoring a monitored object in a chamber of a substrate processing apparatus which processes substrates, comprising: an image generation step in which a monitoring camera receives light from a monitoring area including at least one first monitored object among a plurality of monitored objects in the chamber, and generates monitored image data; and a monitoring step which is executed after the image generation step, and which generates high-resolution image data by increasing the resolution of the monitored image data using a trained model for high-resolution processing, and monitors the first monitored object based on the high-resolution image data.

2. A monitoring method in a substrate processing apparatus as described in claim 1, wherein in the monitoring process, the high-resolution image data is generated using a first trained model corresponding to the first monitored object among a plurality of trained models of high-resolution processing corresponding respectively to the plurality of monitored objects.

3. A monitoring method in a substrate processing apparatus as described in claim 2, wherein in the image generating step, the monitoring camera receives light from the monitoring area including at least the first and second monitored objects of the multiple monitored objects to generate the monitoring image data, and the monitoring step includes the steps of: generating first high-resolution image data as the high-resolution image data by increasing the resolution of a first judgment area of ​​the monitoring image data that does not include the second monitored object but includes the first monitored object using the first trained model; generating second high-resolution image data by increasing the resolution of a second judgment area of ​​the monitoring image data that does not include the first monitored object but includes the second monitored object using a second trained model corresponding to the second monitored object among the multiple trained models; monitoring the first monitored object based on the first high-resolution image data, and monitoring the second monitored object based on the second high-resolution image data.

4. A monitoring method in a substrate processing apparatus as described in claim 2 or 3, wherein in the monitoring process, the high-resolution image data is generated by increasing the resolution of the monitoring image data at a first resolution magnification set for the first monitored object among multiple resolution magnifications set for each of the multiple monitored objects.

5. A monitoring method in a substrate processing apparatus as described in claim 4, further comprising a training step executed before the image generation step to generate the first trained model, the training step comprising: a training image generation step in which a high-performance camera having a higher resolution than the resolution of the monitoring camera images the first monitored object at multiple timings with the first monitored object being focused on rather than a second monitored object among the multiple monitored objects, thereby generating multiple high-resolution training data; a low-resolution conversion step in which the multiple high-resolution training data are downsampled at a ratio equal to or less than the reciprocal of the first resolution ratio to convert them into multiple low-resolution training data; and a model generation step in which the first trained model is generated based on the multiple high-resolution training data and the multiple low-resolution training data.

6. A monitoring method for a substrate processing apparatus as described in claim 5, wherein in the learning image generating process, the high-performance camera images the area monitored by the monitoring camera.

7. A monitoring method in a substrate processing apparatus as described in claim 5, wherein in the learning image generation process, the first monitored object, which is displaceable, is imaged at the multiple times when the first monitored object is located at mutually different positions, and the multiple high-resolution learning data are generated.

8. A monitoring method in a substrate processing apparatus as described in claim 4, further comprising a training step executed before the image generation step to generate the trained model, the training step comprising: a training image generation step of generating a plurality of sets of high-resolution training data and low-resolution training data based on three-dimensional data including information about the shape of the monitored object in the chamber; and a model generation step of generating the trained model based on the plurality of sets of high-resolution training data and low-resolution training data.

9. A monitoring method in a substrate processing apparatus as described in any one of claims 1 to 3, wherein the monitoring camera includes a depth camera that generates depth image data indicating the distance to the monitored object as the monitoring image data.

10. A monitoring method for a substrate processing apparatus according to any one of claims 1 to 3, wherein the monitoring camera includes an imaging camera that receives visible light.

11. A monitoring method in a substrate processing apparatus as described in any one of claims 1 to 3, further comprising a reference image registration step that is executed before the image generation step and registers reference image data, the reference image registration step including: a preliminary image generation step in which the monitoring camera receives light from the monitoring area and generates preliminary image data; a reference image generation step in which, after the preliminary image generation step, the trained model is used to generate the reference image data by increasing the resolution of the preliminary image data; and a storage step in which the reference image data is stored in a storage unit, wherein, in the monitoring step, the first monitored object is monitored based on a comparison between the high-resolution image data and the reference image data.

Images