Monitoring method in substrate processing apparatus

Abstract

To provide technology with which an object to be monitored can be monitored with higher monitoring accuracy.SOLUTION: A monitoring method for monitoring an object to be monitored in a chamber of a substrate processing apparatus for processing a substrate includes an image generation step and a monitoring step. In the image generation step, a monitoring camera receives light from a monitoring region including at least one first object to be monitored among a plurality of objects to be monitored in the chamber, and generates monitoring image data. In the monitoring step that is executed after the image generation step, a trained model for high resolution-achieving processing is used to generate high-resolution image data obtained by increasing resolution of the monitoring image data, and the first object to be monitored is monitored on the basis of the high-resolution image data.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present disclosure relates to a monitoring method for a substrate processing apparatus. [Background technology]

[0002] Conventionally, in manufacturing processes for semiconductor devices, etc., various processing liquids such as pure water, photoresist liquid, and etching liquid are supplied to a substrate to perform various substrate processing such as cleaning processing and resist coating processing. As an apparatus for performing substrate processing using these processing liquids, a substrate processing apparatus in which a substrate holder rotates a substrate in a horizontal position while a processing liquid is discharged from a nozzle onto the surface of the substrate is widely used.

[0003] In such a substrate processing apparatus, whether or not the nozzle position is appropriate is monitored. For example, in Patent Document 1, an imaging means such as a camera is provided to monitor the nozzle position. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2015-173148 A Summary of the Invention [Problem to be solved by the invention]

[0005] In order to properly process a substrate, it is desirable to monitor more targets than just the nozzle.

[0006] For example, the substrate holding part includes a plurality of chuck pins. The plurality of chuck pins are provided at equal intervals along the periphery of the substrate. Each chuck pin is provided displaceable between a holding position in contact with the periphery of the substrate and a release position away from the substrate. The chuck pins are positioned at the holding position, allowing the substrate holding part to hold the substrate. If the substrate holding part cannot hold the substrate properly, the substrate cannot be processed properly, and therefore the positions of the plurality of chuck pins of the substrate holding part may be monitored.

[0007] When a camera receives light from a monitoring area including a plurality of chuck pins, the image data generated by the camera includes the plurality of chuck pins. If the camera is an imaging camera, and the distance between the camera and the chuck pins in real space is different among the plurality of chuck pins, the clarity of the contours of the plurality of chuck pins in the captured image data may differ from each other. For example, a chuck pin closer to the camera is included in the captured image data with a large size, so that the contour shape of the chuck pin is likely to be clear. On the other hand, a chuck pin farther from the camera is included in the captured image data with a small size, so that the contour shape of the chuck pin is likely to be unclear. In addition, the farther the chuck pin is from the focus position of the camera lens in real space, the more likely the contour shape of the chuck pin is to be unclear in the captured image data.

[0008] Also, if the camera is a depth camera, the image data generated by the camera contains the distance to the object in the monitoring area for each pixel. Since the measurement accuracy of the depth camera is not very high, the contour shape of the object in the depth image data may be unclear.

[0009] Furthermore, if an object to be monitored is monitored based on image data that includes an object with an unclear contour shape, the monitoring accuracy may decrease.

[0010] Therefore, an object of the present disclosure is to provide a technique capable of monitoring an object to be monitored with higher monitoring accuracy. [Means for solving the problem]

[0011] A first aspect is a monitoring method for monitoring a monitored object in a chamber of a substrate processing apparatus for processing substrates, comprising an image generation step in which a monitoring camera receives light from a monitoring area including at least one first monitored object among a plurality of monitored objects in the chamber and generates monitoring image data, and a monitoring step that is executed after the image generation step, in which high-resolution image data is generated by increasing the resolution of the monitoring image data using a trained model for high-resolution processing, and the first monitored object is monitored based on the high-resolution image data.

[0012] A second aspect is a monitoring method for a substrate processing apparatus according to the first aspect, wherein in the monitoring process, high-resolution image data is generated using a first trained model corresponding to the first monitored object among a plurality of trained models of high-resolution processing corresponding respectively to the plurality of monitored objects.

[0013] A third aspect is a monitoring method in a substrate processing apparatus according to the second aspect, wherein in the image generating process, the monitoring camera receives light from the monitoring area including at least the first and second monitored objects among the multiple monitored objects to generate the monitoring image data, and the monitoring process includes a process of generating first high-resolution image data as the high-resolution image data by increasing the resolution of a first judgment area of ​​the monitoring image data that does not include the second monitored object but includes the first monitored object using the first trained model, generating second high-resolution image data by increasing the resolution of a second judgment area of ​​the monitoring image data that does not include the first monitored object but includes the second monitored object using a second trained model corresponding to the second monitored object among the multiple trained models, monitoring the first monitored object based on the first high-resolution image data, and monitoring the second monitored object based on the second high-resolution image data.

[0014] A fourth aspect is a monitoring method in a substrate processing apparatus according to the second or third aspect, wherein in the monitoring process, the high-resolution image data is generated by increasing the resolution of the monitoring image data at a first resolution magnification set for the first monitored object among a plurality of resolution magnifications set for the plurality of monitored objects, respectively.

[0015] A fifth aspect is a monitoring method in a substrate processing apparatus according to the fourth aspect, further comprising a training step executed before the image generation step to generate the first trained model, the training step comprising a training image generation step of capturing images of the first monitored object at multiple timings with a high-performance camera having a higher resolution than the resolution of the monitoring camera in a state where the first monitored object is focused on more than a second monitored object among the multiple monitored objects, thereby generating multiple high-resolution training data, a low-resolution process of downsampling the multiple high-resolution training data at a ratio equal to or less than the reciprocal of the first resolution ratio to convert them into multiple low-resolution training data, and a model generation step of generating the first trained model based on the multiple high-resolution training data and the multiple low-resolution training data.

[0016] A sixth aspect is a monitoring method for a substrate processing apparatus according to the fifth aspect, wherein in the learning image generating step, the high performance camera captures an image of the area monitored by the monitoring camera.

[0017] A seventh aspect is a monitoring method for a substrate processing apparatus according to the fifth or sixth aspect, wherein in the learning image generation process, images of the first monitored object, which is displaceable, are captured at the multiple times when the first monitored object is located at mutually different positions, and the multiple high-resolution learning data are generated.

[0018] An eighth aspect is a monitoring method in a substrate processing apparatus according to the fourth aspect, further comprising a training step executed before the image generation step to generate the trained model, the training step comprising a training image generation step of generating multiple sets of high-resolution training data and low-resolution training data based on three-dimensional data including information about the shape of the monitored object in the chamber, and a model generation step of generating the trained model based on the multiple sets of high-resolution training data and low-resolution training data.

[0019] A ninth aspect is a monitoring method in a substrate processing apparatus according to any one of the first to eighth aspects, wherein the monitoring camera includes a depth camera that generates depth image data indicating a distance to the monitored object as the monitoring image data.

[0020] A tenth aspect is a monitoring method for a substrate processing apparatus according to any one of the first to ninth aspects, wherein the monitoring camera includes an imaging camera that receives visible light.

[0021] An eleventh aspect is a monitoring method in a substrate processing apparatus according to any one of the first to tenth aspects, further comprising a reference image registration step that is executed before the image generation step and registers reference image data, the reference image registration step including a preliminary image generation step in which the monitoring camera receives light from the monitoring area and generates preliminary image data, a reference image generation step that, after the preliminary image generation step, generates the reference image data with a high resolution from the preliminary image data using the trained model, and a storage step that records the reference image data in a storage unit, and in the monitoring step, the first monitored object is monitored based on a comparison between the high resolution image data and the reference image data. Effect of the Invention

[0022] According to the first, ninth and tenth aspects, since the resolution of the monitoring image data is increased by the trained model, it is possible to generate high-resolution image data with higher accuracy. Therefore, it is possible to monitor the first monitoring object with higher monitoring accuracy.

[0023] According to the second aspect, since the resolution of the monitoring image data is increased by the first trained model corresponding to the first monitoring object, it is possible to generate high-resolution image data with higher accuracy, and therefore it is possible to monitor the first monitoring object with even higher monitoring accuracy.

[0024] According to the third aspect, it is not necessary to capture the first and second monitoring objects at different times, and the number of times the monitoring camera operates can be reduced. Moreover, since the resolution of the second judgment area is increased by the second trained model corresponding to the second monitoring object, it is possible to generate second high-resolution image data with higher accuracy. Therefore, it is possible to monitor the second monitoring object with higher monitoring accuracy.

[0025] According to the fourth aspect, it is possible to increase the resolution of monitoring image data with higher accuracy regardless of the state of the monitored object.

[0026] According to the fifth aspect, a plurality of high-resolution learning data are generated in a state where the first monitored object is in focus, and are converted into a plurality of low-resolution learning data at a magnification equal to or smaller than the reciprocal of the first resolution magnification. This makes it possible to generate a first trained model suitable for increasing the resolution of the first monitored object at the first resolution magnification.

[0027] According to the sixth aspect, the high-performance camera captures the same monitoring area as the surveillance camera. Therefore, the optical characteristics of the lens group of the high-performance camera through which light passes when capturing images are similar to the optical characteristics of the lens group of the surveillance camera through which light passes when capturing images. Therefore, the high-resolution learning data from the high-performance camera can be made equivalent to the high-resolution image data obtained by increasing the resolution of the surveillance image data from the surveillance camera. As a result, the first surveillance object can be more appropriately monitored based on the high-resolution image data.

[0028] According to the seventh aspect, the plurality of high-resolution learning data includes first monitored objects at different positions, and the first trained model is generated using the plurality of high-resolution learning data. Therefore, the first trained model can generate high-resolution image data with higher accuracy.

[0029] According to the eighth aspect, a high-performance camera is not required to generate training data, so a trained model can be generated at low cost.

[0030] According to the eleventh aspect, the reference image data is generated by increasing the resolution of the preliminary image data captured by the surveillance camera using the trained model. That is, the high-resolution image data and the reference image data are generated using the same surveillance camera and the same trained model. Therefore, the first surveillance object can be monitored with high accuracy by comparing the high-resolution image data with the reference image data. [Brief description of the drawings]

[0031] [Figure 1] 1 is a plan view illustrating an example of a configuration of a substrate processing apparatus. [Diagram 2] FIG. 2 is a block diagram illustrating an example of the configuration of a control unit. [Diagram 3] FIG. 2 is a vertical cross-sectional view illustrating an example of the configuration of a processing unit according to the first embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of monitoring image data generated by an imaging camera. [Diagram 5] FIG. 2 is a diagram illustrating an example of monitoring image data generated by an imaging camera. [Figure 6] 13 is a flowchart illustrating an example of a monitoring method by a processing unit. [Figure 7] 13 is a flowchart showing a specific example of a monitoring process. [Figure 8] FIG. 11 is a diagram showing an example of the flow of each piece of data in a monitoring process. [Figure 9] 13 is a flowchart showing a first example of a specific operation of a training process. [Figure 10] FIG. 1 is a diagram illustrating an example of a state in which a high-performance camera is provided in a chamber. [Figure 11] FIG. 1 is a diagram illustrating the flow of data in generating a trained model. [Figure 12] 13 is a flowchart showing an example of a specific operation of a reference image registration step. [Figure 13] FIG. 2 is a diagram showing an example of the flow of each piece of data. [Figure 14] 4 is a functional block diagram showing an example of an internal configuration of a control unit. FIG. [Figure 15] 13 is a flowchart showing a second example of specific operations of the training process. [Figure 16] FIG. 1 is a diagram illustrating an example of high-resolution training data. [Figure 17] FIG. 10 is a diagram illustrating an example of low-resolution training data. [Figure 18] FIG. 11 is a vertical cross-sectional view illustrating an example of the configuration of a processing unit according to a second embodiment. [Figure 19] FIG. 2 is a diagram illustrating an example of monitor image data. [Figure 20] FIG. 1 is a diagram illustrating an example of high-resolution training data. [Figure 21] FIG. 10 is a diagram illustrating an example of low-resolution training data. [Figure 22] FIG. 11 is a diagram illustrating an example of an operation of a control unit. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0032] Hereinafter, the embodiments will be described in detail with reference to the drawings. In the drawings, the dimensions and numbers of each part are exaggerated or simplified as necessary for the purpose of easy understanding. Parts having similar configurations and functions are given the same reference numerals, and duplicated explanations will be omitted in the following description.

[0033] In the following description, the same components are denoted by the same reference numerals, and their names and functions are also the same. Therefore, detailed descriptions thereof may be omitted to avoid duplication.

[0034] In addition, in the description below, even if ordinal numbers such as "first" or "second" are used, these terms are used for convenience to facilitate understanding of the contents of the embodiments, and are not limited to the ordering that may result from these ordinal numbers.

[0035] When an expression indicating a relative or absolute positional relationship (e.g., "in one direction," "along one direction," "parallel," "orthogonal," "center," "concentric," "coaxial," etc.) is used, the expression not only strictly indicates the positional relationship, but also indicates a state in which the relative positional relationship is displaced with respect to angle or distance within a range in which a tolerance or similar function is obtained, unless otherwise specified. When an expression indicating an equal state (e.g., "same," "equal," "homogeneous," etc.) is used, the expression not only strictly indicates a quantitatively equal state, but also indicates a state in which a difference exists within a range in which a tolerance or similar function is obtained, unless otherwise specified. When an expression indicating a shape (e.g., "square shape" or "cylindrical shape," etc.) is used, the expression not only strictly indicates the shape geometrically, but also indicates a shape having, for example, unevenness or chamfering within a range in which a similar effect is obtained, unless otherwise specified. When an expression "comprises," "has," "includes," "includes," or "has" a component is used, the expression is not an exclusive expression that excludes the presence of other components. When the phrase "at least one of A, B, and C" is used, the phrase includes A only, B only, C only, any two of A, B, and C, and all of A, B, and C.

[0036] First Embodiment <Overall configuration of substrate processing apparatus> 1 is a plan view illustrating a schematic configuration of a substrate processing apparatus 100. The substrate processing apparatus 100 is a single-wafer processing apparatus that processes substrates W one by one.

[0037] The substrate W is, for example, a semiconductor wafer, a substrate for a liquid crystal display, a substrate for an organic EL (Electroluminescence), a substrate for an FPD (Flat Panel Display), a substrate for an optical display, a substrate for a magnetic disk, a substrate for an optical disk, a substrate for a magneto-optical disk, a substrate for a photomask, or a substrate for a solar cell. The substrate W has a thin flat plate shape. In the following, the substrate W is assumed to be a semiconductor wafer. The substrate W has, for example, a disk shape. The diameter of the substrate W is, for example, about 300 mm, and the thickness of the substrate W is, for example, about 0.5 mm or more and about 3 mm or less.

[0038] 1, the substrate processing apparatus 100 includes an indexer block 110, a processing block 120, and a control unit 90. The processing block 120 is a section that mainly processes the substrate W, and the indexer block 110 is a section that mainly transports the substrate W between the outside of the substrate processing apparatus 100 and the processing block 120.

[0039] The indexer block 110 includes a load port 111 and a first transport part 112. A substrate container (hereinafter referred to as a carrier) C that is carried in from the outside is placed on the load port 111. A plurality of substrates W are accommodated in the carrier C, for example, arranged at intervals from each other in the vertical direction. In the example of Fig. 1, a plurality of load ports 111 are arranged.

[0040] The first transport unit 112 is a transport robot and can take out an unprocessed substrate W from a carrier C placed on each load port 111. The first transport unit 112 can also be called an indexer robot. The first transport unit 112 transports the unprocessed substrate W taken out from the carrier C to the processing block 120. The processing block 120 can process the unprocessed substrate W. The first transport unit 112 can also receive a processed substrate W from the processing block 120 and transport the processed substrate W to a carrier C of the load port 111.

[0041] In the example of FIG. 1, the processing block 120 includes a plurality of processing units 1 and a second transport section 122. The second transport section 122 is a transport robot and can transport the substrate W between the first transport section 112 and the plurality of processing units 1. In the example of FIG. 1, the processing block 120 also includes a placement section 123. The placement section 123 is, for example, a shelf on which a plurality of substrates W can be placed in a vertically aligned state. The first transport section 112 places an unprocessed substrate W on the placement section 123. The second transport section 122 takes out the unprocessed substrate W from the placement section 123 and transports the substrate W to the processing unit 1. The processing unit 1 processes the substrate W. The configuration of the processing unit 1 will be described later. The second transport section 122 takes out the processed substrate W from the processing unit 1 and transports the substrate W to the placement section 123. The first transport section 112 takes out the substrate W from the platform 123 and transports the substrate W to the carrier C of the load port 111 .

[0042] 1, a plurality of (e.g., four) processing units 1 are provided so as to surround the second transport section 122 in a plan view. This second transport section 122 may also be called a center robot. At each position in a plan view, a plurality of processing units 1 may be stacked in the vertical direction. In other words, a plurality of towers TW (four in the figure) each of which is made up of a plurality of processing units 1 stacked in the vertical direction may be provided so as to surround the second transport section 122.

[0043] The control unit 90 performs overall control of the substrate processing apparatus 100. Specifically, the control unit 90 controls the first transport unit 112, the second transport unit 122, and the processing unit 1. FIG. 2 is a block diagram showing an example of the configuration of the control unit 90. The control unit 90 is an electronic circuit, and has, for example, a data processing unit 91 and a storage unit 92. In the specific example of FIG. 2, the data processing unit 91 and the storage unit 92 are connected to each other via a bus. The data processing unit 91 may be, for example, an arithmetic processing device such as a CPU (Central Processor Unit). The storage unit 92 may have a non-transient storage unit (for example, a ROM (Read Only Memory)) 921 and a temporary storage unit (for example, a RAM (Random Access Memory)) 922. The non-transient storage unit 921 may store, for example, a program that specifies the processing to be executed by the control unit 90. The data processing unit 91 executes this program, so that the control unit 90 can execute the processing specified in the program. Of course, a part or all of the processing executed by the control unit 90 may be executed by hardware such as a dedicated logic circuit.

[0044] As shown in Fig. 2, the control unit 90 may be electrically connected to a storage unit 94. The storage unit 94 is a non-transitory storage unit, and may be, for example, a memory or a hard disk. In the example of Fig. 2, various data are stored in the storage unit 94. Each piece of data will be described in detail later.

[0045] <Processing unit> Fig. 3 is a vertical cross-sectional view illustrating an example of the configuration of a processing unit 1 according to the first embodiment. It is not necessary that all processing units 1 in the substrate processing apparatus 100 have the configuration illustrated in Fig. 3. It is sufficient that at least one processing unit 1 has the configuration illustrated in Fig. 3.

[0046] The processing unit 1 includes a chamber 10. The chamber 10 forms a processing chamber for processing the substrate W. Specifically, the internal space of the chamber 10 corresponds to the processing chamber. The chamber 10 is provided with an openable / closable loading / unloading port (not shown). The second transport part 122 loads an unprocessed substrate W into the chamber 10 through the loading / unloading port, and loads a processed substrate W out of the chamber 10 through the loading / unloading port.

[0047] Various components used for processing the substrate W are provided within the chamber 10. For example, the chamber 10 includes a substrate holder 2, a nozzle 31, and a guard 7.

[0048] The substrate holding unit 2 holds the substrate W in a horizontal position and rotates the substrate W around a rotation axis Q1. The horizontal position here means that the thickness direction of the substrate W is aligned along the vertical direction. The rotation axis Q1 is an axis that passes through the center of the substrate W and is aligned along the vertical direction. Such a substrate holding unit 2 may also be called a spin chuck.

[0049] In the example of FIG. 3, the substrate holding unit 2 includes a spin base 21, chuck pins 22, a rotation drive unit 23, and a pin drive unit 24. The spin base 21 has a plate-like shape (for example, a disk shape) and is provided with its thickness direction aligned along the vertical direction. A plurality of chuck pins 22 (for example, three or more) are provided on the upper surface of the spin base 21. The plurality of chuck pins 22 are provided at equal intervals along the circumferential direction about the rotation axis Q1. The plurality of chuck pins 22 are provided so as to be displaceable between a holding position and a release position, which will be described next. The holding position is a position where the chuck pins 22 abut against the periphery of the substrate W. The plurality of chuck pins 22 hold the substrate W by stopping at the respective holding positions. FIG. 3 shows the chuck pins 22 stopped at the holding positions. The release position is a position where each chuck pin 22 is separated from the substrate W. As the plurality of chuck pins 22 stop at their respective release positions, the substrate W is released from the holding by the plurality of chuck pins 22.

[0050] The pin driver 24 displaces the chuck pin 22 between the holding position and the release position. In the example of FIG. 3, the pin driver 24 includes a plurality of displacement magnets 241, a plurality of fixed magnets 242, a driving magnet 243, and a magnet lift driver 244. The displacement magnets 241 are provided at the lower ends of the chuck pins 22. The fixed magnets 242 are provided at the lower part of the spin base 21. The driving magnets 243 have, for example, an annular shape surrounding the rotation axis Q1. The magnet lift driver 244 lifts and lowers the driving magnet 243 between the magnet upper position and the magnet lower position. The magnet lift driver 244 includes a driving source such as a motor or an air cylinder, and is controlled by the control unit 90. When the driving magnet 243 is located at the magnet upper position, the displacement magnet 241 and the chuck pin 22 move together to one of the holding position and the release position due to the magnetic force between the displacement magnet 241 and the driving magnet 243. When the drive magnet 243 is located at the magnet lower position, the magnetic force between the displacement magnet 241 and the fixed magnet 242 causes the displacement magnet 241 and the chuck pin 22 to move together to the other of the retained position and the released position.

[0051] The rotation drive unit 23 includes a shaft 231 and a motor 232. The upper end of the shaft 231 is connected to the lower surface of the spin base 21, and the shaft 231 extends from the lower surface of the spin base 21 along a rotation axis Q1. The motor 232 is controlled by the control unit 90 to rotate the shaft 231 about the rotation axis Q1. As a result, the spin base 21, the chuck pins 22, and the substrate W rotate together about the rotation axis Q1.

[0052] The nozzle 31 ejects the processing liquid toward the main surface of the substrate W held by the substrate holding part 2. In the example of Fig. 3, the nozzle 31 is provided vertically above the substrate W held by the substrate holding part 2, and ejects the processing liquid toward the upper surface of the substrate W. The nozzle 31 may be a nozzle that ejects the processing liquid in a continuous flow state, or may be a mist nozzle or a spray nozzle that ejects the processing liquid in the form of droplets. Here, as an example, the nozzle 31 is a nozzle that ejects the processing liquid in a continuous flow state.

[0053] The processing liquid may be a chemical liquid, a rinse liquid, or a charge removing liquid. The chemical liquid may be a cleaning liquid for removing foreign matter on the substrate W, or an etching liquid for removing a target film. As the chemical liquid, for example, hydrofluoric nitric acid obtained by mixing hydrofluoric acid, nitric acid, and water, hydrofluoric acid hydrogen peroxide aqueous solution (FPM) obtained by mixing hydrofluoric acid, hydrogen peroxide, and water, tetramethylammonium hydroxide (TMAH), a mixture of sulfuric acid and hydrogen peroxide (SPM), ammonia water, a mixture of ammonia, hydrogen peroxide, and water (SC-1), and a mixture of hydrogen chloride, hydrogen peroxide, and water (SC-2) can be used. The chemical liquid may be a single liquid instead of a mixture. For example, a single liquid such as hydrofluoric acid (HF), hydrogen peroxide, and sulfuric acid can be used as the chemical liquid. The rinse liquid may be pure water (i.e., deionized water), or an organic solvent such as isopropyl alcohol that is more volatile than pure water. The charge removing liquid is a liquid for removing electric charges from the substrate W, and may be carbon dioxide water. The carbon dioxide water may be used as a rinsing liquid.

[0054] In the example of FIG. 3, the downstream end of a liquid supply pipe 32 is connected to the nozzle 31. The upstream end of the liquid supply pipe 32 is connected to a processing liquid supply source. The processing liquid supply source supplies the processing liquid to the upstream end of the liquid supply pipe 32. In the example of FIG. 3, a supply valve 33 and a flow rate control valve 34 are interposed in the liquid supply pipe 32. The supply valve 33 switches between opening and closing the liquid supply pipe 32, and the flow rate control valve 34 adjusts the flow rate of the processing liquid flowing through the liquid supply pipe 32. The flow rate control valve 34 may be a mass flow controller. These valves are controlled by a control unit 90.

[0055] In the example of FIG. 3, the processing unit 1 includes a nozzle movement drive unit 37 that moves the nozzle 31. The nozzle movement drive unit 37 moves the nozzle 31 between a processing position and a standby position, which will be described below. The processing position is a position where the nozzle 31 ejects the processing liquid toward the main surface of the substrate W held by the substrate holder 2, and is, for example, a position facing the center of the substrate W in the vertical direction. In the example of FIG. 3, the nozzle 31 is shown positioned at the processing position. The standby position is a position where the nozzle 31 does not eject the processing liquid toward the main surface of the substrate W, and is, for example, a position radially outward from the substrate W.

[0056] In the example of FIG. 3, the nozzle movement drive unit 37 includes an arm 371, a support column 372, and a drive source 373. The support column 372 is provided radially outward from the guard 7 described later, and extends along the vertical direction. The arm 371 extends along the horizontal direction, its tip end is connected to the nozzle 31, and its base end is connected to the support column 372. The drive source 373 is controlled by the control unit 90, and rotates the support column 372 in a forward and reverse direction around its central axis Q2 within a predetermined angle range. The drive source 373 includes, for example, a motor. When the support column 372 rotates in a forward and reverse direction around the central axis Q2 within a predetermined angle range, the nozzle 31 reciprocates along the circumferential direction about the central axis Q2. The support column 372 is installed so that the processing position and the standby position are located on the movement trajectory of the nozzle 31. Note that the nozzle movement drive unit 37 is not necessarily limited to the embodiment of FIG. 3, and may include, for example, a linear motion mechanism such as a linear motor.

[0057] When the nozzle 31 is located at the processing position and the processing liquid is discharged toward the main surface of the rotating substrate W, the processing liquid lands on the main surface of the substrate W. The processing liquid flows radially outward due to centrifugal force accompanying the rotation of the substrate W, and is scattered outside the periphery of the substrate W. At this time, the processing liquid acts on the main surface of the substrate W, and processing according to the type of processing liquid is performed on the substrate W.

[0058] The nozzle 31, the liquid supply pipe 32, the supply valve 33, and the flow rate adjustment valve 34 constitute the discharge unit 3. The discharge unit 3 may have a configuration capable of sequentially discharging a plurality of types of processing liquid toward the main surface of the substrate W. For example, the discharge unit 3 may discharge the chemical liquid and the rinsing liquid toward the main surface of the substrate W in this order. As a specific example, the nozzle 31 may be shared by the chemical liquid and the rinsing liquid. In this case, the chemical liquid supply pipe 32 and the rinsing liquid supply pipe 32 join together and are connected to the nozzle 31. The chemical liquid supply pipe 32 is provided with the chemical liquid supply valve 33 and the flow rate adjustment valve 34, and the rinsing liquid supply pipe 32 is provided with the rinsing liquid supply valve 33 and the flow rate adjustment valve 34. The chemical liquid nozzle 31 and the rinsing liquid nozzle 31 may be provided separately.

[0059] The guard 7 receives the processing liquid splashed from the periphery of the substrate W. The guard 7 has a cylindrical shape surrounding the substrate W held by the substrate holder 2. The guard lifting drive unit 8 lifts and lowers the guard 7 between a guard processing position and a guard standby position, which will be described below. The guard processing position is a position where the upper end of the guard 7 is vertically above the upper surface of the substrate W held by the substrate holder 2. The example of FIG. 3 shows the guard 7 at the guard processing position. When the guard 7 is at the guard processing position, the processing liquid splashed from the periphery of the substrate W is received by the inner peripheral surface of the guard 7. The guard standby position is a position where the upper end of the guard 7 is vertically below the upper surface of the spin base 21. The guard lifting drive unit 8 includes, for example, a power transmission unit such as a rack and pinion mechanism and a ball screw mechanism, and a power source such as a motor. Alternatively, the guard lifting drive unit 8 may include an air cylinder.

[0060] In FIG. 3, a cup 75 is provided corresponding to the guard 7. The cup 75 has an annular (e.g., annular) recess (groove) surrounding the rotation axis Q1. The cup 75 receives the processing liquid that has flowed down the inner circumferential surface of the guard 7. The upstream end of a drain pipe 76 is connected to, for example, the bottom of the cup 75. The processing liquid received in the cup 75 is discharged to the outside of the processing unit 1 through the drain pipe 76.

[0061] <Outline of an example of substrate processing> Such a processing unit 1 can process the substrate W, for example, as follows. First, the second transport part 122 transports the substrate W to the processing unit 1, and the substrate holding part 2 holds the substrate W received from the second transport part 122. As a specific example, the substrate holding part 2 displaces the multiple chuck pins 22 from their respective release positions to their holding positions. This causes the substrate holding part 2 to hold the substrate W. The substrate holding part 2 continues to hold the substrate W until the processing of the substrate W is completed.

[0062] Next, the processing unit 1 performs chemical processing. Specifically, the guard lifting drive unit 8 raises the guard 7 to the guard processing position, the nozzle movement drive unit 37 moves the nozzle 31 to the processing position, and the substrate holder 2 rotates the substrate W around the rotation axis Q1. The substrate holder 2 may continue to rotate the substrate W until the processing of the substrate W is completed. Then, the control unit 90 opens the supply valve 33 for the chemical. This causes the chemical to be discharged from the nozzle 31 toward the center of the main surface of the rotating substrate W. The chemical that has landed on the main surface of the substrate W flows radially outward as the substrate W rotates, and is scattered from the periphery of the substrate W. At this time, the chemical acts on the main surface of the substrate W, and processing according to the type of chemical is performed on the substrate W.

[0063] When the processing of the substrate W has been sufficiently performed, the processing unit 1 ends the chemical processing. For example, when a predetermined chemical time has elapsed since the start of the supply of the chemical, the control unit 90 closes the chemical supply valve 33. The time is measured, for example, by a timer circuit (not shown) belonging to the control unit 90.

[0064] Next, the processing unit 1 executes a rinsing process. Specifically, the control unit 90 opens the supply valve 33 for the rinsing liquid. This causes the rinsing liquid to be discharged from the nozzle 31 toward the center of the main surface of the rotating substrate W. The rinsing liquid that has landed on the main surface of the substrate W flows radially outward as the substrate W rotates, and is scattered from the periphery of the substrate W. At this time, the rinsing liquid washes away the chemical liquid on the main surface of the substrate W, so that the chemical liquid on the main surface of the substrate W can be replaced with the rinsing liquid.

[0065] When the chemical liquid on the main surface of the substrate W is sufficiently replaced with the rinsing liquid, the processing unit 1 ends the rinsing process. For example, when a predetermined rinsing time has elapsed since the start of supply of the rinsing liquid, the control unit 90 closes the supply valve 33 for the rinsing liquid. Then, the nozzle movement driving unit 37 moves the nozzle 31 to the standby position.

[0066] Next, the processing unit 1 performs a drying process. Specifically, the substrate holder 2 increases the rotation speed of the substrate W (spin dry). When the substrate W is sufficiently dried, the processing unit 1 ends the drying process. Specifically, when a predetermined drying time has elapsed since the start of the drying process, the substrate holder 2 stops the rotation of the substrate W. This essentially ends the drying process, and the series of processes for the substrate W are essentially completed.

[0067] Next, the guard lift drive unit 8 lowers the guard 7 to the guard standby position, and the substrate holding unit 2 releases its hold on the substrate W. As a specific example, the substrate holding unit 2 displaces the multiple chuck pins 22 from their respective holding positions to the release position. This releases the substrate W from the multiple chuck pins 22. Then, the second transport unit 122 lifts the processed substrate W vertically upward from the substrate holding unit 2, and carries the substrate W out of the processing unit 1.

[0068] <Target of monitoring> As described above, the substrate W is appropriately processed by appropriately operating each component in the chamber 10. On the other hand, if each component cannot operate appropriately, the substrate W may be over- or under-processed, or the substrate W may not be processed at all.

[0069] Therefore, in this embodiment, the processing unit 1 monitors each component of the chamber 10. As shown in Fig. 3, the processing unit 1 includes a monitoring camera 5. The camera 5 receives light from a monitoring area including at least one of the multiple monitoring objects in the chamber 10, and generates monitoring image data IM1. The camera 5 outputs the monitoring image data IM1 to the control unit 90. The control unit 90 also functions as an image processing unit that processes the monitoring image data IM1.

[0070] The monitoring camera 5 may be an imaging camera 52 that receives visible light. The imaging camera 52 includes a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) and an optical system such as a lens. The monitoring image data IM1 generated by the imaging camera 52 may be called captured image data or two-dimensional image data. In the example of FIG. 3, the camera 5 is provided at a position vertically above and radially outward from the substrate holder 2. The camera 5 is fixed to the chamber 10 through a fixing member (not shown). The imaging direction of the camera 5 is set to the side of the rotation axis Q1 and in a vertically downward oblique direction. Here, the imaging area of ​​the camera 5 (i.e., the monitoring area) may be set to include, for example, the entire upper opening of the guard 7 located at the guard processing position (see also FIG. 5(b)).

[0071] 4 and 5 are diagrams that show an example of the monitoring image data IM1 generated by the imaging camera 52. Fig. 4 shows the monitoring image data IM1 generated by the imaging camera 52 capturing an image of the monitoring area under the following circumstances. That is, the imaging camera 52 captures an image of the monitoring area under the circumstances that the guard 7 is located at the guard standby position and the multiple chuck pins 22 are in contact with the release position. In the example of Fig. 4, the substrate W has not yet been carried in, and the monitoring image data IM1 includes the entirety of each of the multiple chuck pins 22.

[0072] FIG. 5(a) shows monitor image data IM1 generated by imaging the monitoring area by the imaging camera 52 under the following circumstances. That is, the imaging camera 52 images the monitoring area under the circumstances that the guard 7 is located at the guard standby position and the multiple chuck pins 22 are in contact with the periphery of the substrate W. In FIG. 5(a), the substrate W is held by the multiple chuck pins 22, and each chuck pin 22 is partially hidden by the substrate W. FIG. 5(b) shows monitor image data IM1 generated by imaging the monitoring area by the imaging camera 52 under the following circumstances. That is, the imaging camera 52 images the monitoring area under the circumstances that the guard 7 is located at the guard processing position, the multiple chuck pins 22 are in contact with the periphery of the substrate W, and the nozzle 31 is located at the processing position. In the example of FIG. 5(b), the chuck pins 22 on the lower side of the monitor image data IM1 are hidden by the guard 7.

[0073] As the monitored object in the chamber 10, various objects such as the nozzle 31, the chuck pin 22, the guard 7, etc. may be applied. For example, the processing unit 1 may monitor the state of the monitored objects such as the position of the nozzle 31, the state of the processing liquid discharged from the nozzle 31 (discharge state, discharge stop state, etc.), the position of the chuck pin 22, the position of the guard 7, etc. based on the monitoring image data IM1. In the following, a description will be given taking a plurality of chuck pins 22 as an example of a monitored object.

[0074] 4, the substrate holder 2 does not hold the substrate W, and the substrate W has not yet been loaded into the chamber 10. Therefore, all of the chuck pins 22 on the spin base 21 are included in the monitoring image data IM1. In the example of FIG. 4, the chuck pins 22 are provided as the multiple chuck pins 22, which are chuck pins 22a (e.g., corresponding to a first monitored object), chuck pin 22b (e.g., corresponding to a second monitored object), chuck pin 22c, and chuck pin 22d. In the example of FIG. 4, the chuck pin 22a is located at the back side as viewed from the imaging camera 52, the chuck pin 22b is located at the right side as viewed from the imaging camera 52, the chuck pin 22c is located at the front as viewed from the imaging camera 52, and the chuck pin 22d is located at the left side as viewed from the imaging camera 52.

[0075] For example, when the substrate W is loaded, the second transport part 122 delivers the substrate W to the substrate holding part 2 with the chuck pins 22 positioned at the release position. During this loading, it is not desirable for the chuck pins 22 to be positioned at the holding position. For this reason, the processing unit 1 may monitor the position of the chuck pins 22 based on the monitored image data IM1.

[0076] Here, in the monitoring image data IM1, the shape of the contour of the chuck pin 22 may become unclear. For example, the focus position of the lens of the imaging camera 52 may be set approximately near the center of the spin base 21. In this case, the contour of an object that is far from the focus position in the imaging direction becomes blurred in the monitoring image data IM1. That is, the shape of the contour of the object may become unclear. This is because light from one point on the object is incident across multiple pixels of the solid-state imaging element through the lens of the imaging camera 52. For this reason, the pixel values ​​of multiple pixels are averaged, and the shape of the contour becomes unclear. In other words, this averaging reduces the spatial frequency of the image group representing the object in the monitoring image data IM1, and the shape of the contour becomes unclear. In other words, the number of effective pixels for representing the object decreases. In the example of FIG. 4, a dashed line adjacent to the contour line is added to diagrammatically indicate that the shape of the contour is unclear.

[0077] In the example of FIG. 4, since the chuck pins 22a and 22c move away from the focus position in the imaging direction, the contour shapes of the chuck pins 22a and 22c may become less clear than the contour shapes of the chuck pins 22b and 22d.

[0078] Furthermore, in the monitoring image data IM1, the smaller the size of an object, the less clear the shape of the object's contour. This is because the smaller the size of an object in the monitoring image data IM1, the fewer the number of pixels required to represent the contour of the object. In the example of FIG. 4, the distance between the chuck pin 22a and the imaging camera 52 is longer than the distance between the other chuck pins 22 and the imaging camera 52, so the chuck pin 22a is the smallest in the monitoring image data IM1. For this reason, the shape of the contour of the chuck pin 22a may be less clear than the shapes of the contours of the other chuck pins 22. For example, the shape of the contour of the chuck pin 22a may be less clear than the shape of the contour of the chuck pin 22c.

[0079] As described above, the clarity of the contour shape of the chuck pin 22 in the monitoring image data IM1 may differ among the chuck pins 22a to 22d. If the monitoring image data IM1 including an object having such an unclear contour is used to monitor the object, the monitoring accuracy may decrease.

[0080] Therefore, the control unit 90 generates high-resolution image data IM2 by increasing the resolution of the monitoring image data IM1 using the learned model LM for the high-resolution processing. The control unit 90 monitors the monitoring object based on the high-resolution image data IM2.

[0081] 2, the memory unit 94 stores a plurality of learned models LM corresponding to a plurality of monitored objects, respectively. More specifically, the memory unit 94 stores parameters used in the learned models LM. Here, the memory unit 94 stores, as the plurality of learned models LM, a learned model LMa (e.g., corresponding to a first learned model) corresponding to the chuck pin 22a, a learned model LMb (e.g., corresponding to a second learned model) corresponding to the chuck pin 22b, a learned model LMc corresponding to the chuck pin 22c, and a learned model LMd corresponding to the chuck pin 22d.

[0082] Each trained model LM is generated based on training image data including the corresponding monitored object. That is, the trained model LMa is generated based on training image data including the chuck pin 22a, the trained model LMb is generated based on training image data including the chuck pin 22b, the trained model LMc is generated based on training image data including the chuck pin 22c, and the trained model LMd is generated based on training image data including the chuck pin 22d. The generation method will be described later.

[0083] 2, the resolution correspondence data D1 is stored in the storage unit 94. Table 1 shows the resolution correspondence data D1.

[0084] [Table 1]

[0085] As shown in Table 1, the resolution correspondence data D1 indicates a plurality of resolution magnifications RM corresponding to a plurality of monitoring objects, respectively. In Table 1, a resolution magnification RMa (e.g., corresponding to a first resolution magnification) corresponding to the chuck pin 22a is set, a resolution magnification RMb (e.g., corresponding to a second resolution magnification) corresponding to the chuck pin 22b is set, a resolution magnification RMc corresponding to the chuck pin 22c is set, and a resolution magnification RMd corresponding to the chuck pin 22d is set. Each resolution magnification RM is set, for example, in advance, so that the clarity of the contour of each monitoring object (chuck pin 22) is equal to or greater than a value sufficient for monitoring. The resolution magnification RM can be set by experiment or simulation.

[0086] The control unit 90 generates high-resolution image data IM2 by increasing the resolution of the monitoring image data IM1 using a learned model LM corresponding to the monitored object. At this time, the control unit 90 increases the resolution of the monitoring image data IM1 with a resolution magnification RM corresponding to the monitored object. Specifically, the control unit 90 inputs the monitoring image data IM1 and the resolution magnification RM to the learned model LM, and generates high-resolution image data IM2 by a high-resolution process using the learned model LM.

[0087] The high-resolution processing by the trained model LM may be, for example, a processing using a super-resolution technique. For example, the trained model LM is a trained model using deep learning, and as a specific example, it may be a trained model by VDSR, a trained model by ResNet, a trained model by SRResNet, or a trained model by an architecture in which the batch normalization layer is removed from SRResNet (Bee Lim, et. al., "Enhanced Deep Residual Networks for Single Image Super-Resolution", Department of ECE, ASRIM, Seoul National University, 08826, Seoul, Korea, arXiv:1707.02921v1 [cs.CV] 10 Jul 2017).

[0088] Furthermore, the control unit 90 monitors each monitoring object based on the high resolution image data IM2. For example, the control unit 90 monitors the position of each chuck pin 22 based on the high resolution image data IM2.

[0089] FIG. 6 is a flow chart showing an example of a monitoring method by the processing unit 1. First, the imaging camera 52 captures an image of the monitoring area to generate monitoring image data IM1 (step S1: image generating step). The imaging camera 52 outputs the monitoring image data IM1 to the control unit 90. Here, as an example, the imaging camera 52 may capture an image of the monitoring area just before the second transport unit 122 loads the substrate W into the processing unit 1. Just before the second transport unit 122 loads the substrate W, the control unit 90 controls the pin driving unit 24 to stop the chuck pin 22 at the release position. Therefore, if no abnormality occurs, the chuck pin 22 is normally located at the release position. That is, the monitoring image data IM1 exemplified in FIG. 4 is generated. On the other hand, if an abnormality occurs in the substrate holder 2, the chuck pin 22 may be located at a position away from the release position by a tolerable distance or more.

[0090] The imaging camera 52 may capture an image of the monitoring area when the substrate holding part 2 holds the substrate W received from the second transport part 122. In this case, if the chuck pins 22 move normally to the holding positions and the substrate holding part 2 holds the substrate W normally, monitoring image data IM1 as exemplified in FIG.

[0091] Next, the control unit 90 monitors the object to be monitored (e.g., the chuck pins 22) based on the monitoring image data IM1 (step S2: monitoring step). For example, the control unit 90 may determine whether or not each of the chuck pins 22 is located at the release position based on the monitoring image data IM1 of Fig. 4, and may determine whether or not each of the chuck pins 22 is located at the holding position based on the monitoring image data IM1 of Fig. 5(a).

[0092] Fig. 7 is a flow chart showing a specific example of the monitoring process, and Fig. 8 is a diagram showing an example of the flow of each piece of data in the monitoring process. In the example of Fig. 7, first, the control unit 90 cuts out a plurality of judgment regions R1 each including a plurality of monitored objects (e.g., chuck pins 22) from the monitoring image data IM1 to generate a plurality of judgment image data IM11 (step S3: cutting process). The judgment image data IM11 is a part of the monitoring image data IM1. Each judgment region R1 can be set so as not to include a monitored object different from the corresponding monitored object.

[0093] In the example of FIG. 4 and FIG. 5(a), the judgment region R1a to the judgment region R1d are shown as the multiple judgment regions R1. The judgment region R1a (e.g., corresponding to the first judgment region) to the judgment region R1d are regions corresponding to the chuck pins 22a to 22d, respectively. The judgment region R1 includes at least a part of the corresponding chuck pin 22, but does not include other chuck pins 22. For example, the judgment region R1a includes at least a part of the chuck pin 22a, but does not include all of the chuck pins 22b to 22d. In the example of FIG. 4, the judgment region R1a is set to include all of the chuck pins 22a on the spin base 21. The judgment region R1b (e.g., corresponding to the second judgment region) to the judgment region R1d are set in the same manner. The judgment region R1 has, for example, a rectangular shape.

[0094] In the following, the judgment image data IM11 obtained by cutting out the judgment region R1a from the monitor image data IM1 is called judgment image data IM11a. Also, the judgment image data IM11 corresponding to the judgment regions R1b to R1d are called judgment image data IM11b to IM11d, respectively. Note that FIG. 8 shows the flow of processing for the monitor image data IM1 in FIG. 4.

[0095] Next, the control unit 90 determines the resolution magnification RM corresponding to each monitored object (step S4: resolution determination step). Specifically, the control unit 90 reads the resolution correspondence data D1 from the storage unit 94, and reads out the resolution magnifications RMa to RMd corresponding to the chuck pins 22a to 22d included in the resolution correspondence data D1.

[0096] Next, the control unit 90 determines a learned model LM corresponding to each monitored object (step S5: learned model determination step). Specifically, the control unit 90 reads out parameters of the learned models LMa to LMd corresponding to the chuck pins 22a to 22d, respectively, from the storage unit 94.

[0097] Next, the control unit 90 inputs each judgment image data IM11 and the corresponding resolution magnification RM to the corresponding trained model LM, and generates high-resolution image data IM2 according to the algorithm of the trained model LM (step S6: high-resolution process). For example, the control unit 90 inputs the judgment image data IM11a and the resolution magnification RMa to the trained model LMa, and generates high-resolution image data IM2a (e.g., corresponding to the first high-resolution image data). Similarly, the control unit 90 inputs the judgment image data IM11b and the resolution magnification RMb to the trained model LMb to generate high-resolution image data IM2b (e.g., corresponding to the second high-resolution image data), inputs the judgment image data IM11c and the resolution magnification RMc to the trained model LMc to generate high-resolution image data IM2c, and inputs the judgment image data IM11d and the resolution magnification RMd to the trained model LMd to generate high-resolution image data IM2d.

[0098] Next, the control unit 90 monitors the monitoring object based on the high resolution image data IM2 (step S7). That is, the control unit 90 monitors the chuck pin 22a based on the high resolution image data IM2a, monitors the chuck pin 22b based on the high resolution image data IM2b, monitors the chuck pin 22c based on the high resolution image data IM2c, and monitors the chuck pin 22d based on the high resolution image data IM2d.

[0099] As a specific example, the control unit 90 monitors the monitoring target based on a comparison between the high resolution image data IM2 and the reference image data RIM stored in the storage unit 94. The reference image data RIM is, for example, an image including a normal monitoring target, and can be stored in the storage unit 94 in advance. The reference image data RIM is set for each monitoring target. For example, the reference image data RIM includes reference image data RIMa to reference image data RIMd corresponding to the chuck pins 22a to 22d, respectively, stored in the storage unit 94. The reference image data RIMa is, for example, an image including the normal chuck pin 22a. In the example of FIG. 8, the reference image data RIMa includes the chuck pin 22a located at the release position. Similarly, the reference image data RIMb includes the normal chuck pin 22b, the reference image data RIMc includes the normal chuck pin 22c, and the reference image data RIMd includes the normal chuck pin 22d.

[0100] The position, shape and size of the reference image data RIM in the monitoring area are the same as those of the corresponding determination area R1. For example, the position, shape and size of the reference image data RIMa in the monitoring area are the same as those of the determination area R1a. In addition, the clarity of the contour of the monitored object in the reference image data RIM is higher than that of the monitored object in the monitoring image data IM1 and is about the same as that of the monitored object in the high resolution image data IM2. A method for generating such reference image data RIM will be described in detail later.

[0101] If the chuck pin 22a is normally positioned at the release position, the high resolution image data IM2a will generally match the reference image data RIMa. On the other hand, if the chuck pin 22a is not at the release position due to an abnormality or the like, the high resolution image data IM2a will not resemble the reference image data RIMa. Conversely, if the high resolution image data IM2a is similar to the reference image data RIMa, it can be determined that the chuck pin 22a is normally positioned at the release position, and if the high resolution image data IM2a is not similar to the reference image data RIMa, it can be determined that the chuck pin 22a is positioned at a position different from the release position. The same is true for the chuck pins 22b to 22d.

[0102] In the example of FIG. 5(a), the chuck pins 22a to 22d are positioned in the monitoring image data IM1, which are normally stopped at the holding position. In this case, the reference image data RIMa to the reference image data RIMd include images similar to the judgment areas R1a to R1d in FIG. 5(a), respectively. However, the clarity of the contour of the monitoring object in the reference image data RIMa to the reference image data RIMd is about the same as the clarity of the contour of the monitoring object in the high-resolution image data IM2. In this case, if the high-resolution image data IM2a is similar to the reference image data RIMa, it can be determined that the chuck pin 22a is normally positioned at the holding position, and if the high-resolution image data IM2a is not similar to the reference image data RIMa, it can be determined that the chuck pin 22a is positioned at a position different from the holding position. The same applies to the chuck pins 22b to 22d.

[0103] Therefore, the control unit 90 compares the high-resolution image data IM2 with the corresponding reference image data RIM for each chuck pin 22, and determines the state of the chuck pin 22 based on the comparison result. As a specific example, the control unit 90 calculates the similarity between the high-resolution image data IM2 and the reference image data RIM. The similarity is not particularly limited, and may be any known similarity such as the sum of squared differences of pixel values, the sum of absolute differences of pixel values, normalized cross-correlation, and zero-mean normalized cross-correlation.

[0104] The control unit 90 judges for each chuck pin 22 whether the similarity is equal to or greater than a predetermined threshold value. The threshold value is a threshold value for judging the presence or absence of an abnormality, and is set in advance, for example, by simulation or experiment, and stored in the storage unit 94. When the similarity is equal to or greater than the threshold value, the control unit 90 judges that the chuck pin 22 is located in a normal position, and when the similarity is less than the threshold value, the control unit 90 judges that the chuck pin 22 is located in a position different from the normal position. When the similarity is less than the threshold value, the control unit 90 may judge that an abnormality has occurred in the chuck pin 22. When the control unit 90 judges that an abnormality has occurred, the control unit 90 may interrupt the process.

[0105] As described above, in this embodiment, the learned models LM corresponding to the multiple monitored objects are stored in the storage unit 94. Then, the control unit 90 performs a high-resolution process on the monitored image data IM1 (specifically, the determination image data IM11) using the learned models LM corresponding to the monitored objects to generate high-resolution image data IM2. Since the learned model LM can more appropriately clarify the contour of the corresponding monitored object, the contour of the monitored object becomes clear in the high-resolution image data IM2. For example, the PSNR (Peak Signal to Noise Ratio) of the high-resolution image data IM2 is 23 or more, and the SSIM (structural similarity) of the high-resolution image data IM2 is 0.75 or more.

[0106] Therefore, the control unit 90 can monitor the monitoring object with higher accuracy based on the high resolution image data IM2. In other words, the control unit 90 can determine the state of the monitoring object (here, the position of the chuck pin 22) with higher accuracy.

[0107] Furthermore, there is no need to use an expensive high-resolution imaging camera as the imaging camera 52. This allows the manufacturing cost of the substrate processing apparatus 100 to be reduced.

[0108] <First method for generating trained models> Next, a method for generating the trained model LM will be described. A training process for generating the trained model LM is performed before the image generating process (step S1). Specifically, the trained model LM is generated before the substrate processing apparatus 100 performs processing on the substrate W. FIG. 9 is a flowchart showing an example of a specific operation of the training process. First, the high-performance camera 51 captures an image of the monitoring area (step S11: training image generating process). FIG. 10 is a diagram showing an example of a state in which the high-performance camera 51 is provided in the chamber 10. The high-performance camera 51 may be installed by an operator in place of the imaging camera 52. In other words, the high-performance camera 51 may be attached at the same position as the imaging camera 52.

[0109] The high-performance camera 51 can generate captured image data having a higher resolution than that of the imaging camera 52. For example, the number of pixels of the high-performance camera 51 may be greater than that of the imaging camera 52. In addition, the lens inside the high-performance camera 51 is provided so as to be movable in the direction of the optical axis. In other words, the high-performance camera 51 has an optical zoom function and a focus adjustment function. On the other hand, the imaging camera 52 does not need to have an optical zoom function, and does not need to have an optical focus adjustment function. In other words, the focus position of the imaging camera 52 may be fixed. Such an imaging camera 52 is inexpensive.

[0110] The imaging area of ​​the high performance camera 51 may be the same as the imaging area of ​​the imaging camera 52. The imaging area of ​​the high performance camera 51 also includes a monitored object (here, the chuck pin 22). The imaging area of ​​the high performance camera 51 may include a plurality of chuck pins 22.

[0111] For example, the high-performance camera 51 first captures an image of the monitoring area in a state where the chuck pin 22a is in focus. That is, the high-performance camera 51 captures an image of the monitoring area in a state where the chuck pin 22a is in focus rather than at least one of the other monitoring objects (e.g., the chuck pin 22d). The shape of the outline of the chuck pin 22a in the captured image data generated by the high-performance camera 51 is clear, the PSNR of the captured image data is 23 or more, and the SSIM of the captured image data is 0.75 or more. Hereinafter, the captured image data generated by the high-performance camera 51 is also referred to as high-resolution learning data IM3. The high-resolution learning data IM3 captured in a state where the chuck pin 22a is in focus is referred to as high-resolution learning data IM3a.

[0112] The high-performance camera 51 may capture images of the monitoring area at multiple timings to generate multiple high-resolution learning data IM3a. For example, the high-performance camera 51 may capture images of the monitoring area in different states of the chuck pin 22a to generate the high-resolution learning data IM3a. As a specific example, the high-performance camera 51 may capture images of the monitoring area in a state where the chuck pin 22a is located at the release position (see FIG. 4, for example), and capture images of the monitoring area in a state where the chuck pin 22a is located at the holding position (see FIG. 5(a), for example). The state where the chuck pin 22a is located at the holding position can be realized by the substrate holding unit 2 holding the substrate W.

[0113] The high-performance camera 51 may capture an image of the monitoring area in a state where the monitored object is in various other states (e.g., an abnormal state) and generate high-resolution learning data IM3, without being limited to the above. For example, the high-performance camera 51 may capture an image of the monitoring area in a state where the chuck pin 22 is stopped at an intermediate position between the holding position and the release position.

[0114] Fig. 11 is a diagram showing a schematic diagram of a data flow in generating a trained model. As shown in Fig. 11, the control unit 90 may generate high-resolution training data IM31a by extracting a region including the chuck pin 22a from the high-resolution training data IM3a generated by the high-performance camera 51. The region is, for example, the same region as the judgment region R1a. The PSNR of the high-resolution training data IM31a may be 23 or more, and the SSIM of the high-resolution training data IM31a may be 0.75 or more.

[0115] Next, the control unit 90 downsamples the multiple high-resolution learning data IM3 at a magnification equal to or less than the reciprocal of the corresponding resolution magnification RM, and converts them into multiple low-resolution learning data IM4 (step S12: resolution reduction step). For example, the control unit 90 downsamples the multiple high-resolution learning data IM31a at a magnification equal to or less than the reciprocal of the resolution magnification RMa, and converts them into multiple low-resolution learning data IM4a. The downsampling method is not particularly limited, but for example, bicubic downsampling may be applied.

[0116] Next, the control unit 90 trains the learning model using multiple sets of high-resolution learning data IM3a (e.g., high-resolution learning data IM31a) and low-resolution learning data IM4a, and generates a learned model LMa corresponding to the chuck pin 22a (step S13: model generation step). Specifically, the control unit 90 obtains parameters of the learned model LMa corresponding to the chuck pin 22a. Then, the control unit 90 stores the learned model LMa (parameters) in the storage unit 94.

[0117] Next, the control unit 90 judges whether or not the learned models LM corresponding to all the monitored objects have been generated (step S14). If all the learned models LM have not yet been generated, the control unit 90 executes steps S11 to S13 for the next monitored object (for example, the chuck pin 22b). For example, the high-performance camera 51 captures an image of the monitored area with the chuck pin 22b in focus to generate high-resolution learning data IM3 (step S11), the control unit 90 cuts out an area the same as the judgment area R1b including the chuck pin from the high-resolution learning data IM3 to generate high-resolution learning data IM31, downsamples the high-resolution learning data IM31 at a magnification equal to or less than the reciprocal of the resolution magnification RMb to generate low-resolution learning data IM4 (step S12), and trains the learning model using the high-resolution learning data IM31 and the low-resolution learning data IM4 corresponding to the chuck pin 22b to generate the learned model LMb (step S13). The same applies to the chuck pin 22c and the chuck pin 22d.

[0118] When it is determined in step S14 that all the trained models LM have been generated, the control unit 90 ends the process. Then, the worker appropriately removes the high-performance camera 51 from the chamber 10 and installs the imaging camera 52 in the chamber 10.

[0119] As described above, the control unit 90 generates the trained model LM based on a set of high-resolution learning data IM3 that does not include other monitored objects and clearly includes the monitored object, and low-resolution learning data IM4 generated from the high-resolution learning data IM3. Therefore, the control unit 90 can generate a trained model LM suitable for the monitored object. Therefore, the control unit 90 can increase the resolution of the monitoring image data IM1 that includes the monitored object with higher accuracy by using the trained model LM that corresponds to the monitored object.

[0120] Furthermore, in the above example, the high-performance camera 51 captures the same monitoring area as the imaging camera 52. Therefore, the optical characteristics of the lens group of the high-performance camera 51 through which light passes when capturing images are similar to the optical characteristics of the lens group of the imaging camera 52 through which light passes when capturing images. Therefore, the high-resolution learning data IM3 from the high-performance camera 51 can be made more equivalent to high-resolution image data obtained by increasing the resolution of the surveillance image data IM1 from the imaging camera 52. Therefore, the control unit 90 can increase the resolution of the surveillance image data IM1 with even higher accuracy by using the learned model LM.

[0121] Moreover, the high-performance camera 51 captures images of the chuck pin 22a at multiple timings when the displaceable chuck pin 22a is positioned at different positions, to generate multiple high-resolution learning data IM3a. Specifically, the high-performance camera 51 captures images of the monitoring area at each timing when the chuck pin 22a is positioned at the release position and at each timing when the chuck pin 22a is positioned at the holding position. Then, the multiple high-resolution learning data IM3 are used to generate a trained model LMa. This allows the control unit 90 to increase the resolution of the monitoring image data IM1 with higher accuracy, regardless of the state (here, the position) of the chuck pin 22a in the image generation process (step S1).

[0122] <Reference image data> Next, a method for generating reference image data RIM will be described. A reference image registration step for generating reference image data RIM and storing the reference image data RIM in the storage unit 94 is executed before the image generation step (step S1). In addition, the reference image registration step is executed after the training step. That is, the reference image registration step is executed after the learned model LM is generated. Specifically, the reference image registration step may be executed before the substrate processing apparatus 100 processes the substrate W, for example, when the substrate processing apparatus 100 is installed. FIG. 12 is a flowchart showing an example of a specific operation of the reference image registration step. FIG. 13 is a diagram showing an example of the flow of each data. Here, as an example, the processing unit 1 generates reference image data RIM corresponding to the monitored image data IM1 of FIG. 4.

[0123] First, the imaging camera 52 captures an image of the monitoring area to generate preliminary image data IM5 (step S21: preliminary image generating step). For example, the imaging camera 52 captures an image of the monitoring area in a state where the substrate W has not been loaded and the chuck pin 22 is normally positioned at the release position. Therefore, the preliminary image data IM5 includes the chuck pin 22 normally positioned at the release position.

[0124] The processing unit 1 may generate reference image data RIM corresponding to the monitored image data IM1 of Fig. 5(a). In this case, the imaging camera 52 captures an image of the monitored area in a state where the substrate holding part 2 normally holds the substrate W. In this case, the preliminary image data IM5 includes the chuck pin 22 normally positioned at the holding position.

[0125] Next, the control unit 90 cuts out the determination region R1 for each monitored object from the preliminary image data IM5 to generate preliminary image data IM51 (step S22: cutting step). Specifically, the control unit 90 cuts out the determination region R1a from the preliminary image data IM5 to generate preliminary image data IM51a. The preliminary image data IM51a includes the chuck pin 22a in the normal state. In the example of FIG. 13, the preliminary image data IM51a includes the chuck pin 22a located at the release position. In addition, the control unit 90 cuts out the determination region R1d from the preliminary image data IM5, respectively, to generate preliminary image data IM51d from the preliminary image data IM51b.

[0126] Next, the control unit 90 determines the resolution magnification RM corresponding to each monitored object (step S23: resolution determination step). Specifically, the control unit 90 reads the resolution correspondence data D1 from the storage unit 94, and reads out the resolution magnifications RMa to RMd corresponding to the chuck pins 22a to 22d included in the resolution correspondence data D1.

[0127] Next, the control unit 90 determines a learned model LM corresponding to each monitored object (step S24: learned model determination step). Specifically, the control unit 90 reads out parameters of the learned models LMa to LMd corresponding to the chuck pins 22a to 22d, respectively, from the storage unit 94.

[0128] Next, the control unit 90 inputs each of the preliminary image data IM51 and the corresponding resolution magnification RM to the corresponding trained model LM, and increases the resolution of the preliminary image data IM51 according to the algorithm of the trained model LM to generate reference image data RIM (step S25: reference image generation step). For example, the control unit 90 inputs the preliminary image data IM51a and the resolution magnification RMa to the trained model LMa to generate reference image data RIMa. In the same manner, the control unit 90 generates reference image data RIMd from the reference image data RIMb.

[0129] Next, the control unit 90 stores the reference image data RIMa and the reference image data RIMd in the storage unit 94 (step S26: storage step).

[0130] As described above, in the above example, the imaging camera 52 captures an image of the monitoring area and generates preliminary image data IM5 (step S21). This allows the position and size of the chuck pin 22a in the monitoring image data IM1 captured in the image generating step (step S1) to be equal to the position and size of the chuck pin 22a in the preliminary image data IM5. This allows the degree of similarity between the determination image data IM11a including the normal chuck pin 22a and the reference image data RIMa including the normal chuck pin 22a to be increased. This allows the control unit 90 to perform the position determination of the chuck pin 22a based on the degree of similarity between the determination image data IM11a and the reference image data RIMa with higher accuracy. The same applies to the chuck pins 22b to 22d.

[0131] In the above example, the control unit 90 generates the reference image data RIMa using the learned model LMa used to generate the judgment image data IM11a. That is, the judgment image data IM11a and the reference image data RIMa are generated by the same algorithm. Therefore, the similarity between the judgment image data IM11a including the normal chuck pin 22a and the reference image data RIMa including the normal chuck pin 22a can be increased. Therefore, the control unit 90 can perform the position determination of the chuck pin 22a based on the similarity between the judgment image data IM11a and the reference image data RIMa with higher accuracy. The same applies to the chuck pins 22b to 22d.

[0132] In the above-mentioned specific example, the control unit 90 generates a trained model LM for each monitored object. However, this embodiment is not necessarily limited to this. The control unit 90 may generate one trained model LM and store the trained model LM in the storage unit 94. In other words, the control unit 90 may generate one trained model LM common to multiple monitored objects. For example, the control unit 90 may increase the resolution of monitored image data IM1 including multiple chuck pins 22 using the trained model LM to generate high-resolution image data IM2, and monitor each chuck pin 22 based on each judgment region R1 in the high-resolution image data IM2.

[0133] <Second method for generating trained models> In the above example, the control unit 90 generates the trained model LM based on the high-resolution training data IM3 and the low-resolution training data IM4 generated by the high-performance camera 51. However, this embodiment is not necessarily limited to this. As described later, the control unit 90 may generate the trained model LM based on training data generated by simulation. For simplicity, the following describes a case where a trained model LM common to multiple monitored objects is generated.

[0134] FIG. 14 is a functional block diagram showing an example of the internal configuration of the control unit 90. The control unit 90 includes a learning image generating unit 911 and a training unit 912. Three-dimensional data D2 is input to the learning image generating unit 911. As shown in FIG. 14, the three-dimensional data D2 can be stored in the storage unit 94. The three-dimensional data D2 includes geometric information about the shape of each component in the chamber 10 of the processing unit 1. The three-dimensional data D2 can also be called a three-dimensional model. The substrate holder 2, the nozzle 31, the nozzle movement driver 37, and the guard 7 are applied to each component in the chamber 10. In other words, the three-dimensional data D2 is three-dimensional data that models the object to be monitored in the chamber 10. The data format of the three-dimensional data D2 may be any three-dimensional data format such as CAD (Computer-Aided Design) data. The three-dimensional data D2 may further include at least one of information on the material and the surface state (e.g., pattern) of each component in the chamber 10. Hereinafter, the space within the chamber 10 represented by the three-dimensional data D2 will be referred to as a virtual internal space.

[0135] The learning image generating unit 911 generates simulation image data obtained by a virtual camera virtually placed in a virtual internal space capturing an image of the virtual internal space. The learning image generating unit 911 is a three-dimensional simulator, and may be a functional unit based on a 3DCG (Dimension Computer Graphics) program such as Blender. For example, a 3DCG program is stored in the storage unit 921 (or the storage unit 94), and the data processing unit 91 reads and executes the program, whereby the control unit 90 realizes the function of the learning image generating unit 911.

[0136] For example, the learning image generating unit 911 may have a function of three-dimensional rendering. For example, algorithms such as rasterization, ray tracing, and radiosity can be applied to the three-dimensional rendering. For example, if either ray tracing or radiosity is applied, the learning image generating unit 911 generates simulation image data obtained by a virtual camera capturing an image of a virtual internal space illuminated by a virtual lighting device. Specifically, the learning image generating unit 911 generates simulation image data based on information on the virtual lighting device and information on the virtual camera. The information on the virtual lighting device may include information such as the type of virtual lighting device, such as a point light source, sunlight, spot lighting, and the amount and position of the illumination light. The information on the virtual camera may include optical information such as the focus of the lens, and information such as the number of pixels and position. Here, the control unit 90 uses the simulation image data generated by the learning image generating unit 911 as learning data.

[0137] Fig. 15 is a flow chart showing a second example of a specific operation of the training process. In the example of Fig. 15, the learning image generating unit 911 generates high-resolution simulation image data and low-resolution simulation image data as high-resolution learning data IM3 and low-resolution learning data IM4, respectively, by a three-dimensional simulation (step S31: learning image generating process). Fig. 16 is a diagram showing an example of the high-resolution learning data IM3, and Fig. 17 is a diagram showing an example of the low-resolution learning data IM4.

[0138] Specifically, the training image generating unit 911 generates high-resolution training data IM3 obtained by imaging the virtual internal space with a high first resolution from a predetermined position with a virtual camera. The position of the virtual camera may be set, for example, in the virtual internal space above the substrate holding part 2 and radially outward from the guard 7, similar to the real space. More specifically, the position of the virtual camera may be set at a position in the virtual internal space where the entire upper opening of the guard 7 is included in the high-resolution training data IM3. The resolution may be adjusted, for example, by the number of pixels of the virtual camera. Since the imaging area (monitoring area) by the virtual camera is the same, the greater the number of pixels, the higher the resolution.

[0139] Further, the learning image generating unit 911 generates low-resolution learning data IM4 obtained by imaging the virtual internal space with a lower second resolution from the same position of the virtual camera. For example, the first resolution and the second resolution are set in advance so that the resolution magnification, which is the ratio of the first resolution to the second resolution, is equal to or greater than a predetermined value. As can be seen from FIG. 16 and FIG. 17, the high-resolution learning data IM3 and the low-resolution learning data IM4 are different in terms of resolution, but are otherwise identical. That is, the learning image generating unit 911 generates the high-resolution learning data IM3 and the low-resolution learning data IM4 with different resolutions (number of pixels), under the same virtual lighting conditions, at the same virtual camera position, and in the same virtual internal space.

[0140] This makes it possible to obtain a set of high-resolution learning data IM3 and low-resolution learning data IM4, which have different resolutions. Note that the set here refers to two simulation image data (i.e., high-resolution learning data IM3 and low-resolution learning data IM4) that have the same conditions except for conditions that affect the resolution (e.g., the number of pixels).

[0141] The learning image generating unit 911 generates multiple sets of high-resolution learning data IM3 and low-resolution learning data IM4 by changing at least one of the position and resolution (number of pixels) of the virtual camera, the conditions of the virtual lighting device (e.g., type, position, amount of light, etc.), and the state of each configuration in the virtual internal space. For example, the position of the nozzle 31, the position of the guard 7, and the presence or absence of the substrate W in the virtual internal space can be applied to the state of each configuration in the virtual internal space. The values ​​of these conditions may be set in advance, for example, or may be input through a user interface (not shown). The learning image generating unit 911 generates sets of high-resolution learning data IM3 and low-resolution learning data IM4 under each condition while changing these conditions in various ways. In this way, multiple sets of high-resolution learning data IM3 and low-resolution learning data IM4 can be obtained.

[0142] Next, the training unit 912 trains the learning model based on multiple sets of high-resolution learning data IM3 and low-resolution learning data IM4 to generate a learned model LM (step S32: training step).

[0143] As described above, if the control unit 90 generates the trained model LM based on the simulation image data, an expensive high-performance camera 51 is not required. Therefore, the control unit 90 can generate the trained model LM at a lower cost. Furthermore, in the three-dimensional simulation, it is possible to change the resolution in a wider range and more finely than the resolution of the actual high-performance camera 51. Therefore, the control unit 90 can easily obtain a set of high-resolution training data IM3 and low-resolution training data IM4 having various resolution magnifications.

[0144] As described above, the control unit 90 may generate one trained model LM common to a plurality of monitored objects, but may generate a trained model LM for each monitored object. That is, the training image generating unit 911 may generate a plurality of sets of high-resolution training data IM3 and low-resolution training data IM4 according to each monitored object. For example, the training image generating unit 911 generates a plurality of sets of high-resolution training data IM3 and low-resolution training data IM4 for each chuck pin 22. The resolution magnification in each of the plurality of sets corresponding to each chuck pin 22 is equal to or greater than the resolution magnification RM set for the chuck pin 22. For example, the resolution magnification in each of the plurality of sets corresponding to the chuck pin 22c is equal to or greater than the resolution magnification RMc. In addition, the training image generating unit 911 may cut out a determination region R1c from the high-resolution training data IM3 and the low-resolution training data IM4 as training data for the chuck pin 22c. The training unit 912 may generate a trained model LMc for the chuck pin 22c based on multiple sets of the determination regions R1c of the high-resolution training data IM3 and the low-resolution training data IM4. The same applies to the other chuck pins 22.

[0145] <Second embodiment> FIG. 18 is a vertical cross-sectional view showing an example of the configuration of the processing unit 1 according to the second embodiment. In the second embodiment, the camera 5 is a depth camera 53. The depth camera 53 may also be called a distance camera, a three-dimensional camera, a depth sensor, a distance sensor, or a three-dimensional sensor. The depth camera 53 according to the second embodiment measures the depth to each object in the monitoring area and generates monitoring image data IM6 showing the depth for each pixel. The depth here corresponds to the distance between the depth camera 53 and each object. The monitoring image data IM6 may also be called depth image data or distance image data. FIG. 19 is a diagram showing an example of the monitoring image data IM6. In the example of FIG. 19, each pixel is shown with denser sand hatching for objects closer to the depth camera 53. Note that in the example of FIG. 19, hatching for the guard 7 is omitted. Usually, the measurement accuracy of the depth camera is not that high, so the contour of each object may become more unclear in the monitoring image data IM6. For example, the contour of the chuck pin 22 in the monitoring image data IM6 may become less clear than in the example of FIG. 19.

[0146] The depth camera 53 may be, for example, a TOF (Time of Flight) depth camera. In this case, the depth camera 53 includes a light emitting unit, a light receiving unit, and a calculation unit, all of which are not shown. The light emitting unit may include a light source that emits measurement light toward the monitoring area, and a lens located in front of the light source. The measurement light is not particularly limited, but may be invisible light such as infrared light. The measurement light from the light source is irradiated to the monitoring area through the lens. The light receiving unit may include a plurality of light receiving elements that receive the measurement light reflected in the monitoring area, and an imaging lens located in front of the plurality of light receiving elements and directing the measurement light to each light receiving element. The calculation unit calculates the distance to the object corresponding to each light receiving element based on the difference between the timing when the light emitting unit emits the measurement light and the timing when the light receiving unit receives the measurement light. The calculation unit may have, for example, a hardware configuration similar to that of the control unit 90. Note that the function of the calculation unit may be included in the control unit 90. In this case, a part of the function of the control unit 90 functions as the depth camera 53.

[0147] 18, a trained model LM1 is stored in the storage unit 94. The trained model LM1 is a trained model that performs high-resolution processing on the surveillance image data IM6. The trained model LM1 is a trained model that uses, for example, deep learning, and a specific algorithm example thereof is the same as that of the trained model LM.

[0148] An example of a monitoring method by the processing unit 1 according to the second embodiment is similar to that shown in Fig. 6. However, in step S1, the light-emitting unit of the depth camera 53 irradiates measurement light toward the monitoring area, the light-receiving unit of the depth camera 53 receives the measurement light from the monitoring area, and the calculation unit of the depth camera 53 generates monitoring image data IM6 indicating the depth to each object in the monitoring area based on the difference between the irradiation timing by the light-emitting unit and the light-receiving timing by the light-receiving unit.

[0149] Next, in step S2, the control unit 90 uses the learned model LM1 to increase the resolution of the monitoring image data IM6 to generate high-resolution image data IM7. Then, the control unit 90 monitors the monitoring target based on the high-resolution image data IM7. For example, the control unit 90 may identify the position of each chuck pin 22 in the high-resolution image data IM7 and determine whether the position of each chuck pin 22 is appropriate. For example, similar to the first embodiment, normal monitoring image data generated by the depth camera 53 may be increased in resolution using the learned model LM to generate reference image data and store it in the storage unit 94. The control unit 90 may determine whether the chuck pin 22 is appropriate based on a comparison between the high-resolution image data IM7 and the reference image data.

[0150] This allows the control unit 90 to monitor the monitoring object based on the high resolution image data IM7 that includes the monitoring object more clearly, thereby allowing the control unit 90 to monitor the monitoring object with higher accuracy.

[0151] <How to generate a trained model> Next, an example of a method for generating the trained model LM1 will be described. As in the first embodiment, the training step for generating the trained model LM1 is executed before the image generation step (step S1).

[0152] However, a high-precision depth camera 53 is difficult to obtain and expensive. Therefore, as an example, simulation image data generated by a three-dimensional simulator is applied to the learning data. Here, the learning image generating unit 911 includes the function of a three-dimensional simulator. As in the first embodiment, three-dimensional data D2 is input to the control unit 90.

[0153] The learning image generating unit 911 has a function of generating simulation image data obtained by a virtual depth camera in a virtual internal space. For example, the learning image generating unit 911 obtains the distance from the position of the virtual depth camera in the virtual internal space to each position of each component for each pixel of the virtual depth camera, and generates simulation image data. Since the position of the virtual depth camera is set in advance and each position of each component is set by the three-dimensional data D2, the learning image generating unit 911 can calculate the distance to each position. This simulation image data includes the distance from the position of the virtual depth camera in the virtual internal space to each component for each pixel.

[0154] The training image generating unit 911 can generate simulation image data at a specified resolution. The resolution can be adjusted, for example, by the number of pixels of the virtual depth camera. Since the measurement area (monitoring area) by the virtual depth camera is the same, the greater the number of pixels, the higher the resolution. For example, the training image generating unit 911 generates simulation image data at a high first resolution as high-resolution training data IM8. Also, the training image generating unit 911 generates simulation image data at a second resolution lower than the first resolution as low-resolution training data IM9. Alternatively, the training image generating unit 911 may downsample the high-resolution training data IM8 to generate the low-resolution training data IM9.

[0155] Fig. 20 is a diagram showing an example of high-resolution learning data IM8, and Fig. 21 is a diagram showing an example of low-resolution learning data IM9. In the examples of Fig. 20 and Fig. 21, the closer an object is to the virtual depth camera, the closer the pixel is to black.

[0156] An example of a specific operation of the training process for generating the trained model LM1 is the same as that shown in FIG. 15. First, in step S31, the training image generating unit 911 generates a plurality of sets of high-resolution training data IM8 and low-resolution training data IM9 by 3D simulation, as in the first embodiment. The resolution magnification in each set is set to, for example, a predetermined value or more. The training image generating unit 911 generates a plurality of sets by changing at least one of the resolution (number of pixels) and position of the virtual depth camera, and the state of each configuration in the virtual internal space.

[0157] Next, in step S32, similarly to the first embodiment, the control unit 90 trains a learning model using a plurality of sets of high-resolution learning data IM8 and low-resolution learning data IM9 to generate a learned model LM1.

[0158] As described above, the control unit 90 generates the trained model LM1 that increases the resolution of the monitoring image data IM6 based on the simulation image data. This makes it possible to generate the trained model LM1 without requiring a high-performance depth camera 53.

[0159] In particular, since the measurement accuracy of the depth camera is not very high, the clarity of the object in the depth image data is low. For example, the clarity is lower than the clarity of the object in the captured image data generated by the imaging camera. Therefore, when the depth camera measures the monitored area and generates the high-resolution learning data IM8, the resolution (clarity) of the high-resolution learning data IM8 is relatively low. Therefore, it is particularly preferable to apply simulation image data to the learning data used for the trained model LM1 for the depth camera 53. This is because the clearer high-resolution learning data IM8 can be easily obtained by 3D simulation.

[0160] <Imaging camera and depth camera> In the second embodiment, the processing unit 1 may include two cameras 5. The first camera 5 is an imaging camera 52 that receives visible light. The second camera 5 is a depth camera 53. The imaging camera 52 receives light (e.g., visible light) from the monitoring area and generates monitoring image data IM1. The monitoring image data IM1 is captured image data of the monitoring area. The depth camera 53 receives measurement light (e.g., invisible light) reflected from each object in the monitoring area and generates monitoring image data IM6. The monitoring image data IM6 is depth image data indicating the depth to each object in the monitoring area.

[0161] The storage unit 94 may store both the trained model LM for the surveillance image data IM1 and the trained model LM1 for the surveillance image data IM6.

[0162] The control unit 90 may increase the resolution of the surveillance image data IM1 using the trained model LM to generate high-resolution image data IM2, and may increase the resolution of the surveillance image data IM6 using the trained model LM1 to generate high-resolution image data IM7. Fig. 22 is a diagram illustrating an example of the operation of the control unit 90.

[0163] The control unit 90 may detect each object in the high-resolution image data IM2 based on the high-resolution image data IM2 and the high-resolution image data IM7. For example, an algorithm such as deep learning can be applied as an object detection algorithm. At this time, the control unit 90 may generate composite image data by synthesizing the high-resolution image data IM2 and the high-resolution image data IM7. The synthesis here refers to a process of synthesizing the high-resolution image data IM2, which is the captured image data, and the high-resolution image data IM7, which is the depth image data, by matching pixels corresponding to the same part in the monitoring area among the high-resolution image data IM2 and the high-resolution image data IM7. In other words, each pixel of the synthetic image data includes visual light information at the corresponding position in the monitoring area and depth information to the corresponding position. The control unit 90 may perform object detection based on the synthetic image data. As a specific example, the control unit 90 may detect the chuck pin 22 based on the synthetic image data. It should be noted that the control unit 90 does not necessarily need to synthesize the high-resolution image data IM2 and the high-resolution image data IM7, and may execute an object detection algorithm for each of the high-resolution image data IM2 and the high-resolution image data IM7 individually.

[0164] According to this embodiment, the control unit 90 synthesizes the high resolution image data IM2 and the high resolution image data IM7, so that it is possible to generate synthetic image data with higher accuracy than when the monitoring image data IM1 and the monitoring image data IM6 are synthesized, and thus it is possible to improve the object detection accuracy.

[0165] The clarity of the object in the monitoring image data IM1 captured by the imaging camera 52 is higher than the clarity of the object in the monitoring image data IM6 captured by the depth camera 53. Therefore, the control unit 90 does not necessarily have to increase the resolution of the monitoring image data IM1. For example, the control unit 90 may detect an object based on the monitoring image data IM1 and the high-resolution image data IM7. In this case, the learned model LM is not necessary.

[0166] <Multiple pre-trained models> As in the first embodiment, a plurality of learned models LM1 corresponding to a plurality of monitored objects may be stored in the storage unit 94 as the learned model LM1 for the monitoring image data IM6. Each learned model LM1 converts the monitoring image data IM6 into a resolution using a resolution magnification RM corresponding to the monitored object to generate high-resolution image data IM7. The resolution magnification RM for each monitored object is set in advance and is stored in, for example, the storage unit 94. For example, a resolution magnification RMa is set for the chuck pin 22a, a resolution magnification RMb is set for the chuck pin 22b, a resolution magnification RMc is set for the chuck pin 22c, and a resolution magnification RMd is set for the chuck pin 22d.

[0167] Similar to the first embodiment, multiple trained models LM1 can be generated by training using simulation image data generated by a three-dimensional simulator as training data.

[0168] As described above, the substrate processing apparatus 100 and the substrate processing method have been described in detail, but the above description is merely an example in all respects, and this disclosure is not limited thereto. In addition, the above-mentioned various modified examples can be combined and applied as long as they are not mutually inconsistent. It is understood that many modified examples not exemplified can be envisioned without departing from the scope of this disclosure.

[0169] For example, in the above description, the high-resolution learning data IM3 used to generate the trained model LM includes multiple monitored objects (here, the chuck pin 22a to the chuck pin 22d). Then, the control unit 90 cuts out the high-resolution learning data IM31 corresponding to each monitored object from the high-resolution learning data IM3. However, the high-performance camera 51 may capture an image of a monitored area including a single monitored object. For example, the high-performance camera 51 is attached in the chamber 10 so that the imaging direction can be changed. Specifically, the high-performance camera 51 can be fixed to the ceiling of the chamber 10 so that it can rotate around a vertical rotation axis and can rotate around a horizontal rotation axis. This allows the processing unit 1 to change the imaging direction of the high-performance camera 51. Then, the high-performance camera 51 may capture an image of a monitored area including a single monitored object by using a zoom function and a function for changing the imaging direction to generate high-resolution learning data. For example, the high-performance camera 51 may capture an image of a monitoring area including the chuck pin 22a but not including the chuck pins 22b to 22d, and generate high-resolution learning data for the chuck pin 22a. The same applies to the chuck pins 22b to 22d.

[0170] In the above example, the control unit 90 downsamples the high-resolution learning data to convert it into low-resolution learning data. However, the camera 5 may capture an image of the monitored area to generate low-resolution learning data. [Explanation of symbols]

[0171] 10. Chamber 100 Substrate processing apparatus 22, 22a-22d Monitoring object (chuck pin) 5. Surveillance cameras 51 High-performance camera 52 Imaging camera 53 Depth Camera D2 3D data IM1, IM6 Surveillance image data IM2, IM2a-IM2d, IM7 high resolution image data IM3, IM31, IM31a~IM31d, IM8 high resolution training data IM4, IM4a-IM4d, IM9 low resolution training data IM5, IM51a~IM51d Advance image data RIM, RIMa~RIMd Reference image data LM, LM1, LMa~LMd trained models RM,RMa~RMd Resolution magnification S1 Image generation process (step) S2 Monitoring process (step) S11, S31 Learning image generation process (step) S12 Resolution reduction process (step) S13 Model generation process (step) S21 Pre-image generation process (step) S25 Reference image generation step S26, S32 Storage process (step)

Claims

1. 1. A monitoring method for monitoring an object to be monitored in a chamber of a substrate processing apparatus for processing a substrate, comprising: an image generating step of generating monitoring image data by a monitoring camera receiving light from a monitoring area including at least one first monitoring object among the multiple monitoring objects in the chamber; a monitoring step, which is executed after the image generating step, of generating high-resolution image data by increasing the resolution of the monitoring image data using a trained model of a resolution increasing process, and monitoring the first monitoring object based on the high-resolution image data; A monitoring method for a substrate processing apparatus comprising:

2. 2. A monitoring method for a substrate processing apparatus according to claim 1, comprising the steps of: A monitoring method for a substrate processing apparatus, in which, in the monitoring process, the high-resolution image data is generated using a first trained model corresponding to the first monitored object among multiple trained models of high-resolution processing corresponding respectively to the multiple monitored objects.

3. 3. A monitoring method for a substrate processing apparatus according to claim 2, comprising the steps of: In the image generating step, the monitoring camera receives light from the monitoring area including at least the first monitoring object and the second monitoring object among the plurality of monitoring objects, and generates the monitoring image data; The monitoring step includes: a step of generating, as the high-resolution image data, first high-resolution image data by increasing the resolution of a first determination area in the surveillance image data, which does not include the second surveillance object but includes the first surveillance object, using the first trained model, and generating second high-resolution image data by increasing the resolution of a second determination area in the surveillance image data, which does not include the first surveillance object but includes the second surveillance object, using a second trained model corresponding to the second surveillance object among the plurality of trained models; monitoring the first monitoring object based on the first high resolution image data and monitoring the second monitoring object based on the second high resolution image data; A monitoring method for a substrate processing apparatus comprising:

4. 4. A monitoring method for a substrate processing apparatus according to claim 2, further comprising the steps of: A monitoring method for a substrate processing apparatus, in which, in the monitoring process, the monitoring image data is made high-resolution at a first resolution magnification set for the first monitored object among multiple resolution magnifications set for the multiple monitored objects, thereby generating the high-resolution image data.

5. 5. A monitoring method for a substrate processing apparatus according to claim 4, comprising the steps of: A training step is further provided before the image generation step to generate the first trained model; The training step includes: a learning image generating process in which a high-performance camera having a higher resolution than the resolution of the monitoring camera captures an image of the first monitoring object at a plurality of timings in a state in which the first monitoring object is focused on more than a second monitoring object among the plurality of monitoring objects, thereby generating a plurality of high-resolution learning data; a resolution reduction step of downsampling the plurality of high-resolution training data by a magnification equal to or smaller than the reciprocal of the first resolution magnification to convert the plurality of high-resolution training data into a plurality of low-resolution training data; a model generation step of generating the first trained model based on the plurality of high-resolution training data and the plurality of low-resolution training data; A monitoring method for a substrate processing apparatus comprising:

6. 6. A monitoring method for a substrate processing apparatus according to claim 5, comprising the steps of: A monitoring method for a substrate processing apparatus, wherein in the learning image generating step, the high-performance camera images the area monitored by the monitoring camera.

7. 6. A monitoring method for a substrate processing apparatus according to claim 5, comprising the steps of: A monitoring method for a substrate processing apparatus, in which the learning image generation process images the first monitored object at a plurality of times when the displaceable first monitored object is located at mutually different positions, thereby generating a plurality of high-resolution learning data.

8. 5. A monitoring method for a substrate processing apparatus according to claim 4, comprising the steps of: A training step is further provided before the image generation step to generate the trained model; The training step includes: a training image generation step of generating a plurality of sets of high-resolution training data and low-resolution training data based on three-dimensional data including information on the shape of the monitored object in the chamber; a model generation step of generating the trained model based on the plurality of sets of the high-resolution training data and the low-resolution training data; A monitoring method for a substrate processing apparatus comprising:

9. A monitoring method for a substrate processing apparatus according to any one of claims 1 to 3, comprising the steps of: A monitoring method for a substrate processing apparatus, wherein the monitoring camera includes a depth camera that generates depth image data indicating a distance to the monitored object as the monitoring image data.

10. A monitoring method for a substrate processing apparatus according to any one of claims 1 to 3, comprising the steps of: The monitoring camera includes an imaging camera that receives visible light.

11. A monitoring method for a substrate processing apparatus according to any one of claims 1 to 3, comprising the steps of: A reference image registration step is performed before the image generation step, and registers reference image data. The reference image registration step includes: a preliminary image generating step of generating preliminary image data by the monitoring camera receiving light from the monitoring area; a reference image generation step of generating the reference image data by increasing the resolution of the preliminary image data using the trained model after the preliminary image generation step; a storage step of storing the reference image data in a storage unit; Including, The monitoring method for a substrate processing apparatus, wherein the monitoring step monitors the first monitored object based on a comparison between the high resolution image data and the reference image data.

Images