Substrate inspection method, substrate inspection program, and substrate inspection apparatus

Abstract

To detect abnormality on a substrate surface.SOLUTION: A substrate inspection method includes: acquiring a reference input image based on an image obtained by imaging a surface of a reference substrate; acquiring, when the reference input image is inputted into a neural network pre-configured to output a recognition result of the inputted image, reference intermediate information generated in an intermediate layer of the neural network; generating a reference feature image representing features of the reference input image on the basis of the reference intermediate information; acquiring an inspection input image based on an image obtained by imaging a surface of a substrate as a test object; acquiring inspection intermediate information generated by the intermediate layer of the neural network when the inspection input image is inputted into the neural network; generating an inspection feature image representing features of the inspection input image based on the inspection intermediate information; and determining whether or not abnormalities are present on the surface of the substrate as the test object on the basis of a result obtained by comparing the reference feature image with the inspection feature image.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present disclosure relates to a substrate inspection method, a substrate inspection program, and a substrate inspection apparatus.

Background Art

[0002] Patent Document 1 discloses an apparatus that classifies defects occurring on a substrate based on a captured image that is an inspection target obtained by imaging the substrate.

Prior Art Document

Patent Document

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The present disclosure provides a substrate inspection method, a substrate inspection program, and a substrate inspection apparatus that are useful for accurately detecting abnormalities on the surface of a substrate.

Means for Solving the Problems

[0005] A substrate inspection method according to an aspect of the present disclosure includes: obtaining a reference input image based on an image obtained by imaging the surface of a reference substrate; obtaining reference intermediate information generated in an intermediate layer of a neural network when the reference input image is input to the neural network that is pre-constructed to output a recognition result of the input image; generating a reference feature image indicating features of the reference input image based on the reference intermediate information; obtaining an inspection input image based on an image obtained by imaging the surface of a substrate to be inspected; obtaining inspection intermediate information generated in the intermediate layer of the neural network when the inspection input image is input to the neural network; generating an inspection feature image indicating features of the inspection input image based on the inspection intermediate information; and determining the presence or absence of an abnormality on the surface of the substrate to be inspected based on a result of comparing the reference feature image and the inspection feature image.

Advantages of the Invention

[0006] According to the present disclosure, there are provided a substrate inspection method, a substrate inspection program, and a substrate inspection apparatus that are useful for accurately detecting an abnormality on the substrate surface.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Embodiments for Carrying Out the Invention

[0008] Hereinafter, several embodiments will be described with reference to the drawings. In the description, the same reference numerals are given to the same elements or elements having the same function, and redundant descriptions are omitted. In some of the drawings, an orthogonal coordinate system defined by the X-axis, Y-axis, and Z-axis is shown. In the following description, the Z-axis corresponds to the vertical direction, and the X-axis and Y-axis correspond to the horizontal direction.

[0009] [First Embodiment] First, with reference to FIGS. 1 to 11, a substrate processing system according to a first embodiment will be described. The substrate processing system 1 shown in FIG. 1 is a system that performs formation of a photosensitive film, exposure of the photosensitive film, and development of the photosensitive film on a workpiece W. The workpiece W to be processed is, for example, a substrate or a substrate in a state where a film or a circuit or the like is formed by performing a predetermined process. The substrate included in the workpiece W is, as an example, a wafer containing silicon. The workpiece W (substrate) may be formed in a circular shape. The workpiece W to be processed may be a glass substrate, a mask substrate, an FPD (Flat Panel Display), etc., or may be an intermediate obtained by performing a predetermined process on these substrates or the like. The photosensitive film is, for example, a resist film.

[0010] The substrate processing system 1 includes a coating / developing apparatus 2 and an exposure apparatus 3. The exposure apparatus 3 performs an exposure process on the resist film (photosensitive film) formed on the workpiece W (substrate). Specifically, the exposure apparatus 3 irradiates an energy beam onto the portion of the resist film to be exposed by a method such as immersion exposure. The coating / developing apparatus 2 performs a process of forming a resist film on the surface of the workpiece W before the exposure process by the exposure apparatus 3, and performs a developing process of the resist film after the exposure process.

[0011] [Substrate Processing Apparatus] Hereinafter, as an example of the substrate processing apparatus, the configuration of the coating / developing apparatus 2 will be described. As shown in FIGS. 1 and 2, the coating / developing apparatus 2 includes a carrier block 4, a processing block 5, an interface block 6, and a control device 100.

[0012] The carrier block 4 conducts the introduction of the work W into the coating and developing apparatus 2 and the derivation of the work W from the coating and developing apparatus 2. For example, the carrier block 4 can support a plurality of carriers C (accommodation parts) for the work W and incorporates a transfer device A1 including a transfer arm. The carrier C accommodates, for example, a plurality of circular works W. The transfer device A1 takes out the work W from the carrier C and delivers it to the processing block 5, and receives the work W from the processing block 5 and returns it into the carrier C. The processing block 5 has a plurality of processing modules 11, 12, 13, 14.

[0013] The processing module 11 incorporates a liquid processing unit U1, a heat treatment unit U2, an inspection unit U3, and a transfer device A3 that transfers the work W to these units. The processing module 11 forms an underlayer film on the surface of the work W by the liquid processing unit U1 and the heat treatment unit U2. The liquid processing unit U1 of the processing module 11 applies a processing liquid for forming the underlayer film onto the work W. The heat treatment unit U2 of the processing module 11 performs various heat treatments associated with the formation of the underlayer film. The inspection unit U3 performs a process for inspecting the state of the surface of the work W before the formation of the underlayer film, after the formation of the underlayer film, or before the processing liquid for forming the underlayer film is applied and heat treatment is performed.

[0014] The processing module 12 incorporates a liquid processing unit U1, a heat treatment unit U2, an inspection unit U3, and a transfer device A3 that transfers the work W to these units. The processing module 12 forms a resist film on the underlayer film by the liquid processing unit U1 and the heat treatment unit U2. The liquid processing unit U1 of the processing module 12 applies a processing liquid (resist) for forming the resist film onto the underlayer film. The heat treatment unit U2 of the processing module 12 performs various heat treatments associated with the formation of the resist film. The inspection unit U3 performs a process for inspecting the state of the surface of the work W before the formation of the resist film, after the formation of the resist film, or before the resist is applied and heat treatment is performed.

[0015] The processing module 13 incorporates a liquid processing unit U1, a heat treatment unit U2, an inspection unit U3, and a transfer device A3 for transferring the workpiece W to these units. The processing module 13 forms an upper layer film on the resist film by means of the liquid processing unit U1 and the heat treatment unit U2. The liquid processing unit U1 of the processing module 13 applies a processing liquid for forming the upper layer film onto the resist film. The heat treatment unit U2 of the processing module 13 performs various heat treatments associated with the formation of the upper layer film. The inspection unit U3 performs a process for inspecting the surface state of the workpiece W before the formation of the upper layer film, after the formation of the upper layer film, or before the heat treatment is performed after the processing liquid for forming the upper layer film is applied.

[0016] The processing module 14 incorporates a liquid processing unit U1, a heat treatment unit U2, an inspection unit U3, and a transfer device A3 for transferring the workpiece W to these units. The processing module 14 performs a development process on the resist film after exposure by means of the liquid processing unit U1 and the heat treatment unit U2. The liquid processing unit U1 of the processing module 14, for example, supplies a developer onto the surface of the exposed workpiece W and then rinses it away with a rinse liquid to perform the development process of the resist film.

[0017] The heat treatment unit U2 of the processing module 14 performs various heat treatments associated with the development process. Specific examples of the heat treatment include a pre-development heat treatment (PEB: Post Exposure Bake), a post-development heat treatment (PB: Post Bake), etc. The inspection unit U3 performs a process for inspecting the surface state of the workpiece W before the development process and the PEB are performed, after the development process and the PB are performed, or before the PB is performed after the developer is supplied.

[0018] A shelf unit U10 is provided on the side of the carrier block 4 within the processing block 5. The shelf unit U10 is partitioned into a plurality of cells arranged in the vertical direction. A transfer device A7 including a lifting arm is provided in the vicinity of the shelf unit U10. The transfer device A7 raises and lowers the workpiece W between the cells of the shelf unit U10.

[0019] On the interface block 6 side within the processing block 5, a shelf unit U11 is provided. The shelf unit U11 is partitioned into a plurality of cells arranged in the vertical direction.

[0020] The interface block 6 performs the transfer of the work W with the exposure apparatus 3. For example, the interface block 6 incorporates a transfer device A8 including a transfer arm and is connected to the exposure apparatus 3. The transfer device A8 delivers the work W arranged in the shelf unit U11 to the exposure apparatus 3, receives the work W from the exposure apparatus 3, and returns it to the shelf unit U11.

[0021] The control device 100 controls each device included in the coating and developing apparatus 2 so as to execute coating and developing processing (substrate processing) according to, for example, the following procedure. First, the control device 100 controls the transfer device A1 to transfer the work W in the carrier C to the shelf unit U10, and controls the transfer device A7 to arrange this work W in the cell for the processing module 11.

[0022] Next, the control device 100 controls the transfer device A3 to transfer the work W in the shelf unit U10 to the liquid processing unit U1 within the processing module 11. The control device 100 controls the liquid processing unit U1 to form a film of the processing liquid for forming the lower layer film on the surface of this work W. The control device 100 controls the heat treatment unit U2 to heat the work W in a state where the film of the processing liquid for forming the lower layer film is formed to form the lower layer film. Thereafter, the control device 100 controls the transfer device A3 to return the work W with the lower layer film formed to the shelf unit U10, and controls the transfer device A7 to arrange this work W in the cell for the processing module 12. The control device 100 may control the inspection unit U3 to inspect the surface of the work W at any timing during the processing within the processing module 11.

[0023] Next, the control device 100 controls the transfer device A3 to transfer the workpiece W on the shelf unit U10 to the liquid processing unit U1 in the processing module 12. The control device 100 controls the liquid processing unit U1 to form a film of the processing liquid for resist film formation on the surface of this workpiece W. The control device 100 controls the heat treatment unit U2 to heat the workpiece W in a state where a film of the processing liquid for resist film formation is formed to form a resist film. Thereafter, the control device 100 controls the transfer device A3 to return the workpiece W to the shelf unit U10, and controls the transfer device A7 to place this workpiece W in the cell for the processing module 13. The control device 100 may control the inspection unit U3 to inspect the surface of the workpiece W at any timing during the processing in the processing module 12.

[0024] Next, the control device 100 controls the transfer device A3 to transfer the workpiece W on the shelf unit U10 to the liquid processing unit U1 in the processing module 13. Also, the control device 100 controls the liquid processing unit U1 to form a film of the processing liquid for upper layer film formation on the resist film of this workpiece W. The control device 100 controls the heat treatment unit U2 to heat the workpiece W in a state where a film of the processing liquid for upper layer film formation is formed to form an upper layer film. Thereafter, the control device 100 controls the transfer device A3 to transfer the workpiece W to the shelf unit U11. The control device 100 may control the inspection unit U3 to inspect the surface of the workpiece W at any timing during the processing in the processing module 13.

[0025] Next, the control device 100 controls the transfer device A8 to send out the workpiece W on the shelf unit U11 to the exposure device 3. Thereafter, the control device 100 controls the transfer device A8 to receive the workpiece W subjected to the exposure process from the exposure device 3 and place it in the cell for the processing module 14 in the shelf unit U11.

[0026] Next, the control device 100 controls the transfer device A3 to transfer the workpiece W of the shelf unit U11 to each unit in the processing module 14, and controls the liquid processing unit U1 and the heat treatment unit U2 to perform development processing on the resist film of this workpiece W. After that, the control device 100 controls the transfer device A3 to return the workpiece W to the shelf unit U10, and controls the transfer device A7 and the transfer device A1 to return this workpiece W into the carrier C. The control device 100 may control the inspection unit U3 to inspect the surface of the workpiece W at any timing during the processing in the processing module 14. The above completes the coating and development processing for one workpiece W. The control device 100 controls each device of the coating and development device 2 to execute the coating and development processing in the same manner as described above for each of the subsequent plurality of workpieces W.

[0027] The specific configuration of the substrate processing apparatus is not limited to the configuration of the coating and development device 2 exemplified above. The substrate processing apparatus may be any apparatus as long as it includes a unit for inspecting the surface of the workpiece W to be subjected to a predetermined process and a control device for controlling this unit.

[0028] (Inspection Unit) Subsequently, the inspection unit U3 included in the processing modules 11 to 14 will be described. The inspection unit U3 has a function of imaging the surface of the workpiece W (hereinafter referred to as "surface Wa") to acquire image data. The inspection unit U3 may acquire image data of the entire surface Wa by imaging the entire surface Wa of the workpiece W. As shown in FIG. 3, the inspection unit U3 includes, for example, a housing 30, a holding unit 31, a linear drive unit 32, an imaging unit 33, and a light projection / reflection unit 34.

[0029] The holding unit 31 holds the workpiece W horizontally with the surface Wa facing upward. The linear drive unit 32 includes a power source such as an electric motor and moves the holding unit 31 along a horizontal linear path. The imaging unit 33 has a camera 35 such as a CCD camera. The camera 35 is provided near one end within the inspection unit U3 in the moving direction of the holding unit 31 and is directed toward the other end in its moving direction. The light projecting and reflecting unit 34 projects light onto the imaging range and guides the reflected light from the imaging range to the camera 35. For example, the light projecting and reflecting unit 34 has a half mirror 36 and a light source 37. The half mirror 36 is provided at a position higher than the holding unit 31 and at the middle of the moving range of the linear drive unit 32, and reflects light from below to the camera 35. The light source 37 is provided above the half mirror 36 and irradiates illumination light downward through the half mirror 36.

[0030] The inspection unit U3 operates as follows to acquire image data of the surface Wa of the workpiece W. First, the linear drive unit 32 moves the holding unit 31. As a result, the workpiece W passes under the half mirror 36. During this passing process, the reflected light from each part of the surface Wa of the workpiece W is sequentially sent to the camera 35. The camera 35 forms an image of the reflected light from each part of the surface Wa of the workpiece W and acquires image data of an image of the surface Wa (the entire surface Wa) of the workpiece W. The captured image obtained by imaging the surface Wa of the workpiece W changes depending on the state of the surface Wa of the workpiece W. That is, acquiring the captured image (captured image data) of the surface Wa of the workpiece W corresponds to acquiring information indicating the state of the surface Wa of the workpiece W.

[0031] The captured image data acquired by the camera 35 is sent to the control device 100. In the control device 100, the state of the surface Wa of the workpiece W can be inspected based on the captured image data of the surface Wa. For example, it is possible to inspect the presence or absence of defects on the surface Wa of the workpiece W. In the present disclosure, image data in which pixel values are determined for each pixel may be simply referred to as an "image".

[0032] [Control device] As shown in FIG. 4, the control device 100 includes, as functional components (hereinafter referred to as "function modules"), a processing control unit 102 and an inspection control unit 110. The processes executed by the processing control unit 102 and the inspection control unit 110 correspond to the processes executed by the control device 100. The processing control unit 102 controls the liquid processing unit U1 and the heat treatment unit U2 so as to perform liquid processing and heat treatment in the above-described coating and developing process on the workpiece W.

[0033] The inspection control unit 110 (substrate inspection device) inspects the workpiece W based on the image data obtained from the inspection unit U3 at any stage when executing the coating and developing process. The inspection of the workpiece W includes determining the presence or absence of abnormalities (defects) on the surface Wa of the workpiece W. Defects on the surface Wa include, for example, scratches, foreign matter adhesion, uneven application of the processing liquid, and non-application of the processing liquid.

[0034] Before executing the inspection, the inspection control unit 110 prepares reference data for inspection from a reference workpiece W (reference substrate). The inspection control unit 110 executes the inspection of the workpiece W to be inspected based on the reference data. The reference workpiece W and the workpiece W to be inspected are workpieces (substrates) of the same type. The reference workpiece W and the workpiece W to be inspected are subjected to the coating and developing process under the same processing conditions, and the preparation of the reference data and the inspection of the workpiece W are performed at the same timing in the coating and developing process (for example, after resist coating and before heat treatment).

[0035] The inspection control unit 110 includes, as function modules, a first input image acquisition unit 112, a first intermediate information acquisition unit 114, a first feature image generation unit 116, a reference image holding unit 118, a model holding unit 132, a second input image acquisition unit 122, a second intermediate information acquisition unit 124, a second feature image generation unit 126, an abnormality determination unit 136, and a determination result output unit 138. The processes executed by each function module included in the inspection control unit 110 correspond to the processes executed by the inspection control unit 110 (control device 100).

[0036] The first input image acquisition unit 112 acquires a reference input image based on an image obtained by imaging a reference work W. The first intermediate information acquisition unit 114 acquires reference intermediate information generated in the intermediate layer of the neural network (hereinafter referred to as "image recognition model M") that is pre-constructed to output the recognition result of the input image when the reference input image is input thereto. The first feature image generation unit 116 generates a reference feature image indicating the features of the reference input image based on the reference intermediate information.

[0037] The reference image holding unit 118 holds (stores) the reference feature image generated by the first feature image generation unit 116. The reference feature image generated by the first feature image generation unit 116 is reference data used in the inspection of the work W to be inspected. The model holding unit 132 holds the above image recognition model M.

[0038] The second input image acquisition unit 122 acquires an inspection input image based on an image obtained by imaging the surface Wa of the work W to be inspected. The second intermediate information acquisition unit 124 acquires inspection intermediate information generated in the intermediate layer of the image recognition model M when the inspection input image is input to the image recognition model M. The second feature image generation unit 126 generates an inspection feature image indicating the features of the inspection input image based on the inspection intermediate information.

[0039] The abnormality determination unit 136 determines the presence or absence of an abnormality on the surface Wa of the work W to be inspected based on the result of comparing the reference feature image and the inspection feature image. The determination result output unit 138 outputs the determination result by the abnormality determination unit 136. The determination result output unit 138 may output an abnormality signal indicating that the work W to be inspected is abnormal when the abnormality determination unit 136 determines that there is an abnormality on the surface Wa of the work W. The determination result output unit 138 may output the abnormality signal to the process control unit 102, may output it to the upper controller, or may output it to an output device such as a monitor for notifying an operator or the like of information.

[0040] The control device 100 is composed of one or more computers. The control device 100 has, for example, a circuit 150 shown in FIG. 5. The circuit 150 has one or more processors 152, a memory 154, a storage 156, and an input / output port 158. The storage 156 has a computer-readable storage medium such as a hard disk. The storage medium stores a program (substrate inspection program) for causing the control device 100 to execute the substrate inspection method described later. The storage medium may be a removable medium such as a non-volatile semiconductor memory, a magnetic disk, and an optical disk.

[0041] The memory 154 temporarily stores the program loaded from the storage medium of the storage 156 and the calculation results by the processor 152. The processor 152 cooperates with the memory 154 to execute the above program, thereby constituting each of the functional modules described above. The input / output port 158 performs input / output of electrical signals with the liquid processing unit U1, the heat treatment unit U2, the inspection unit U3, etc. according to a command from the processor 152.

[0042] The hardware configuration of the control device 100 is not necessarily limited to one that constitutes each functional module by a program. For example, each functional module of the control device 100 may be constituted by a dedicated logic circuit or an ASIC (Application Specific Integrated Circuit) integrating the same. When the control device 100 is constituted by a plurality of computers (a plurality of circuits), a part of the above functional modules may be realized by one computer (circuit), and the remaining part of the above functional modules may be realized by other computers (circuits).

[0043] [Substrate Inspection Method] Next, as an example of a substrate inspection method, a series of processes executed by the control device 100 (inspection control unit 110) will be described. As shown in FIG. 6, for example, the control device 100 executes processes in a preparation phase and processes in a production phase. In the preparation phase, the control device 100 performs a coating and development process on a reference work W and then executes preparations for inspecting the work W in the production phase. In the production phase, the control device 100 sequentially performs a coating and development process on a plurality of works W and then inspects each work W on which the coating and development process has been performed. The work W inspected in the production phase corresponds to the work W to be inspected described above.

[0044] <Preparation phase> FIG. 7 is a flowchart showing a series of processes in the preparation phase shown in FIG. 6. The series of processes shown in FIG. 7 starts with the reference work W, for which it has been determined that the process is normal after the process before inspection execution in the coating and development process has been performed, being transported to the inspection unit U3.

[0045] The control device 100 first executes step Sa-1. In step Sa-1, for example, the first input image acquisition unit 112 of the inspection control unit 110 acquires a captured image PIr of the surface Wa of the reference work W by imaging the surface Wa of the reference work W with the inspection unit U3. The captured image PIr may be a color image. The captured image PIr may include the entire surface Wa, and the number of horizontal pixels and the number of vertical pixels in the captured image PIr may be the same as each other.

[0046] Next, the control device 100 executes step Sa-2. In step Sa-2, for example, the first input image acquisition unit 112 generates an enhanced image EIr by performing a process of enhancing the contrast on the captured image PIr obtained in step Sa-1. By executing the contrast enhancement process, for example, the difference (luminance difference) between the bright part and the dark part on the image is enhanced. The first input image acquisition unit 112 may execute the contrast enhancement process by various methods. The first input image acquisition unit 112 may execute the contrast enhancement process on the captured image PIr by deforming (adjusting) the tone curve. The first input image acquisition unit 112 may execute the contrast enhancement process by applying a known spatial filter to the captured image PIr.

[0047] Next, the control device 100 executes step Sa-3. In step Sa-3, for example, when the first intermediate information acquisition unit 114 inputs the enhanced image EIr (reference input image) obtained in step Sa-2 to the image recognition model M, the first intermediate information acquisition unit 114 acquires the reference intermediate information generated in the intermediate layer of the image recognition model M. The first intermediate information acquisition unit 114 acquires, as the reference intermediate information, for example, an extraction image group CIGr including a plurality of extraction images CIr (a plurality of reference extraction images) generated based on the enhanced image EIr and a plurality of filters that extract different features from each other in the intermediate layer of the image recognition model M.

[0048] Here, the image recognition model M used in step Sa-3 will be described. The image recognition model M is a model pre-constructed by machine learning so as to output a result (recognition result) of classifying the content included in the image into categories when the image is input. The image recognition model M may be a multi-layer neural network constructed by deep learning (deep neural network). The image recognition model M may be a CNN (Convolutional Neural Network).

[0049] The image recognition model M does not have to be a model constructed to classify the work W in the image into categories according to predetermined conditions. The image recognition model M may be a model for recognizing the type of an object (e.g., an animal, a fruit), a model for recognizing a human face, or a model for recognizing characters. The CNN may be composed of an input layer, a plurality of convolutional layers, a pooling layer, a fully connected layer, and an output layer.

[0050] In the convolutional layer (intermediate layer) included in the image recognition model M, a plurality of filters are used, and convolution is performed on the input image to the layer. A filter is also called a kernel, and each file is grid-shaped numerical data representing a specific shape (feature). The size of the filter is smaller than the size of the input image. The plurality of filters are set so that different shapes (features) can be obtained in the convolutional layer. In the convolution operation on the input image using one file, for example, the product is calculated pixel by pixel between a partial image (window) of the same size as the filter in the input image and the filter, and a conversion process is performed to calculate the sum of the calculation results of the products of all pixels. Then, the conversion process is repeated for the entire input image while moving the position of the partial image by a predetermined number of pixels.

[0051] By repeating the conversion process, an image in which the shape set by the filter is extracted (responsive to the shape) is obtained as the convolution result. The convolution result is called a feature map. The plurality of extracted images CIr obtained by the first intermediate information acquisition unit 114 are a plurality of images obtained by performing convolution using N filters in any one of the plurality of convolutional layers. N is a natural number of 2 or more. The first intermediate information acquisition unit 114 may input the enhanced image EIr obtained in step Sa-2 to the image recognition model M held by the model holding unit 132, and then acquire the plurality of extracted images CIr from the intermediate calculation results by the image recognition model M.

[0052] FIG. 8 schematically shows an extraction image group CIG obtained by inputting an image capturing the surface Wa of a workpiece W into an image recognition model M and performing convolution using N filters in any one of a plurality of convolutional layers. The extraction image group CIG includes a plurality of extraction images CI. When the input image to the image recognition model M is an enhanced image EIr, the plurality of extraction images CI obtained correspond to the plurality of extraction images CIr. The plurality of extraction images CI include extraction images CI1, CI2, ···, CIN. N is, for example, between 230 and 270. Hereinafter, a case where the number of vertical pixels of one extraction image CI is 255 and the number of horizontal pixels is 255 will be exemplified. The plurality of extraction images CI may be grayscale images. The image recognition model M may perform operations after converting a color image into grayscale when the color image is input. In FIG. 8, extraction images other than extraction images CI1, CI2, CI3, CIj, and CIN are shown simplified as simple circles, but these extraction images also have pixel values.

[0053] Returning to FIGS. 6 and 7, after the execution of step Sa-3, the control device 100 executes step Sa-0. In step Sa-0, for example, the control device 100 determines whether or not it has executed a series of processes in steps Sa-1 to Sa-3 for a predetermined number of reference workpieces W. The predetermined number is set to, for example, a number sufficient to eliminate individual differences in the reference workpieces W. When it is determined that the series of processes has not been executed for the predetermined number of reference workpieces W (step Sa-0: NO), the process executed by the control device 100 returns to step Sa-1. Then, the control device 100 executes a series of processes in steps Sa-1 to Sa-3 for reference workpieces W of different individuals.

[0054] In step Sa-0, when it is determined that a series of processes have been executed for a predetermined number of reference workpieces W (step Sa-0: YES), the process executed by the control device 100 proceeds to step Sa-4. In step Sa-4, for example, the first feature image generation unit 116 performs an operation for generating a reference feature image DIr based on the plurality of extracted images CIr acquired in step Sa-3. The first feature image generation unit 116 calculates the Mahalanobis distance (reference Mahalanobis distance) based on the data distribution of the plurality of extracted images CIr for the array data of pixel values (luminance values) for each pixel included in the plurality of extracted images CIr. The first feature image generation unit 116 may calculate the Mahalanobis distance for each of the plurality of reference workpieces W (for each reference workpiece W).

[0055] Here, with reference to FIGS. 9 and 10, an example of a method for calculating the Mahalanobis distance and a method for generating the reference feature image DIr will be described. Hereinafter, the vertical coordinate on the image is represented by "i", and the horizontal coordinate is represented by "j". The pixel (i, j) indicates the pixel located at the i-th row and j-th column, and each of i and j is a natural number from 1 to N. First, for the extracted images CIr1, CIr2, ···, CIrN (a plurality of extracted images) obtained from the first reference workpiece W, array data of pixel values is created. The pixel values of all the pixels included in the extracted image CIr1 can be represented by an array of data arranged vertically in one column. The pixel values included in the extracted images CIr2, ···, CIrN can also be represented by an array of data arranged vertically in one column each. The number of data in the array data arranged vertically obtained from one reference workpiece W is, for example, 255×255 = 65025.

[0056] Next, for the extracted images CIr1, CIr2, ···, CIrN obtained from the second reference workpiece W, create array data of pixel values and arrange them below the array data for the first reference workpiece W. Similarly, for the reference workpieces W from the third one onwards, create array data of pixel values and arrange them in order below the already created array data. In this case, if the number of reference workpieces W is A (A is a natural number of 2 or more), the number of data in a column of array data arranged vertically is, for example, 65025 × A.

[0057] In FIG. 9, the pixel values at all pixels in the extracted image CIr1 obtained from each of the plurality of reference workpieces W are shown as an array data arranged vertically as variable x1. The pixel values at all pixels in the extracted image CIr2 obtained from each of the plurality of reference workpieces W are shown as an array data arranged vertically as variable x2. Similarly, the pixel values at all pixels in the extracted images CIr3 to CIrN-1 obtained from each of the plurality of reference workpieces W are shown as array data arranged vertically as variables x3 to variable xN-1, respectively. The pixel values at all pixels in the extracted image CIrN obtained from each of the plurality of reference workpieces W are shown as an array data arranged vertically as variable xN. The N array data indicated by variables x1 to xN are arranged in order horizontally. The N array data indicated by variables x1 to xN are N-dimensional data because they contain N variables.

[0058] In each array data arranged vertically, the order in which the coordinates are arranged is the same. Therefore, in a plurality of array data arranged horizontally, the pixel values at the same coordinates (i, j) for variables x1 to xN are arranged horizontally. For example, regarding the first reference work W, the pixel value of variable x1 at pixel (1, 1), the pixel value of variable x2 at pixel (1, 1), and the pixel values of variables x3 to xN at pixel (1, 1) are arranged in the first row of the array data in FIG. 9. Here, any one of the variables x1 to xN is denoted as "xn", and n means any one natural number from 1 to N. The pixel value at a specific coordinate (i, j) of variable xn is denoted as "xn[i, j]". x1[i, j], x2[i, j], ···, and xN[i, j] are array data arranged horizontally in this order. As described above, the first feature image generation unit 116 executes a process of arranging the values (pixel values) included in each of the plurality of extracted images CIr in a vertical array with respect to the plurality of reference works W.

[0059] Next, the first feature image generation unit 116 calculates an average array data indicating the average for each variable and a covariance matrix in order to calculate the Mahalanobis distance. FIG. 10 shows a graph for explaining the concept of the Mahalanobis distance. Here, for the sake of simplicity of explanation, the concept of the Mahalanobis distance calculated from two-dimensional array data obtained from one reference work W and only including variables x1 and x2 is explained. In the graph shown in FIG. 10, the horizontal axis is variable x1 and the vertical axis is variable x2. In the graph of FIG. 10, the combination values (x1[i, j], x2[i, j]) of variables x1 and x2 are plotted for each pixel (coordinate).

[0060] The distance from the combined value of the mean value Mean(x1) of variable x1 and the mean value Mean(x2) of variable x2 is approximately the same between the value of the combination of pixel values indicated by [m1, n1] and the value of the combination of pixel values indicated by [m2, n2]. m1, m2, n1, and n2 are any one natural number from 1 to N. However, the value of the combination of pixel values indicated by [m2, n2] deviates from the distribution of the combined values of variable x1 and variable x2 compared to the value of the combination of pixel values indicated by [m1, n1]. The Mahalanobis distance can represent the degree of deviation (anomaly) from the data distribution of variable x1 and variable x2.

[0061] The first feature image generation unit 116 calculates the average μn of the pixel values for each variable xn (for each of variables x1 to xN), thereby calculating the average array data (average) shown in FIG. 9. The average μn is calculated from the array data of a column arranged vertically included in variable xn, and is the arithmetic mean of the pixel values of all pixels for all reference works W in that column. The first feature image generation unit 116 calculates the variance σn from the array data of pixel values arranged vertically for each variable xn (for each of variables x1 to xN). The first feature image generation unit 116 calculates the correlation coefficient Srs (covariance) for each combination of two variables among variables x1 to xN. Each of r and s in the correlation coefficient Srs satisfies r ≠ s and is a natural number from 1 to N. By calculating the variance σn and the correlation coefficient Srs, a covariance matrix is obtained.

[0062] The first feature image generation unit 116 calculates the Mahalanobis distance for the array data of pixel values arranged horizontally for each pixel (i, j) in variables x1 to xN regarding one reference work W, using the average array data and the covariance matrix. The first feature image generation unit 116 calculates the Mahalanobis distance from the array data of pixel values of each pixel for all pixels in one reference work W. In this case, for all pixels, one Mahalanobis distance is calculated for one pixel. Here, the Mahalanobis distance at pixel (i, j) is denoted as "distance MD(i, j)", and the set of distances MD(i, J) for all pixels is defined as "MD data".

[0063] The first feature image generation unit 116 similarly calculates the Mahalanobis distance from the array data of pixel values of each pixel for all pixels also for other (second and subsequent) reference works W. As a result, a plurality of MD data are generated for the plurality of reference works W. In the present disclosure, calculating the Mahalanobis distance based on the data distribution of the plurality of extracted images CIr obtained from one reference work W includes using the average and covariance matrices calculated using the data obtained from reference works W other than the work W.

[0064] Returning to FIGS. 6 and 7, next, the control device 100 executes step Sa-5. In step Sa-5, for example, the first feature image generation unit 116 generates a reference feature image DIr from the plurality of MD data obtained in step Sa-4. The first feature image generation unit 116 calculates the value (pixel value) of the pixel in the reference feature image DIr for each pixel (i, j) based on the plurality of distances MD(i, j) included in the plurality of MD data. The first feature image generation unit 116 may calculate the maximum value or the average value of the plurality of distances MD(i, j) for each pixel (i, j) as the pixel value in the reference feature image DIr.

[0065] For all pixels (i, j), after calculating the pixel values of the reference feature image DIr, the reference image holding unit 118 stores the reference feature image DIr. Thus, a series of processes in the preparation phase are completed, and the reference feature image DIr, which is the reference data used for inspection in the production phase, is generated. In the series of processes exemplified above, one reference feature image DIr is obtained from a plurality of enhanced images EIr obtained for at least two reference workpieces W. Note that one reference feature image DIr may be obtained from the enhanced image EIr of one reference workpiece W instead of two or more reference workpieces W. Instead of calculating the mean and covariance matrix from the array data obtained from all reference workpieces W, the mean and covariance matrix may be calculated for each reference workpiece W, and the Mahalanobis distance may be calculated.

[0066] When the two reference workpieces W used for generating the reference feature image DIr are the "reference workpiece Wr1" and the "reference workpiece Wr2", each functional module of the inspection control unit 110 executes the following processes. The first input image acquisition unit 112 acquires an enhanced image EIr1 based on an image obtained by imaging the surface Wa of the reference workpiece Wr1, and acquires an enhanced image EIr2 (second reference input image) based on an image obtained by imaging the surface Wa of the reference workpiece Wr2 (second reference substrate). The first intermediate information acquisition unit 114 acquires first intermediate information generated in the intermediate layer when the enhanced image EIr1 is input to the image recognition model M, and acquires second intermediate information (second reference intermediate information) generated in the intermediate layer when the enhanced image EIr2 is input to the image recognition model M. The first feature image generation unit 116 generates the reference feature image DIr based on the first intermediate information and the second intermediate information.

[0067] <Production phase> FIG. 11 is a flowchart showing a series of processes in the production phase shown in FIG. 6. The series of processes shown in FIG. 11 starts with a workpiece W to be inspected, whose inspection result is unknown, being conveyed to the inspection unit U3 after the processes before inspection execution in the coating and development process are performed.

[0068] The control device 100 first executes step Sb-1. Step Sb-1 is performed under the same conditions as the processing of step Sa-1 in the preparation phase. In step Sb-1, for example, the second input image acquisition unit 122 of the inspection control unit 110 captures the surface Wa of the work W to be inspected by the inspection unit U3, thereby acquiring a captured image PIs of the surface Wa of the work W to be inspected.

[0069] Next, the control device 100 executes step Sb-2. Step Sb-2 is performed under the same conditions as the processing of step Sa-2 in the preparation phase. In step Sb-2, for example, the second input image acquisition unit 122 generates an enhanced image EIs by performing a process of enhancing the contrast on the captured image PIs obtained in step Sb-1.

[0070] Next, the control device 100 executes step Sb-3. Step Sb-3 is performed under the same conditions as the processing of step Sa-3 in the preparation phase. In step Sb-3, for example, when the second intermediate information acquisition unit 124 inputs the enhanced image EIs (inspection input image) obtained in step Sb-2 into the image recognition model M, it acquires an inspection intermediate image generated in the intermediate layer of the image recognition model M. The second intermediate information acquisition unit 124 acquires, as inspection intermediate information, an extraction image group CIGs including a plurality of extraction images CIs (a plurality of inspection extraction images) generated based on the enhanced image EIs and the plurality of filters that extract different features from each other in the intermediate layer of the image recognition model M.

[0071] The plurality of filters used in the generation of the plurality of extraction images CIs are the same as the plurality of filters used in the generation of the plurality of extraction images CIr in the preparation phase. If there is an arc-shaped scratch on the surface Wa of the work W to be inspected, an extraction image CI (feature map) in which the filter reacts to the arc-shaped scratch can be generated in the intermediate layer of the image recognition model M, as shown in the "extraction image CIj" shown in FIG. 8.

[0072] Next, the control device 100 executes step Sb-4. Step Sb-4 is performed in a manner similar to step Sa-4 in the preparation phase. In step Sb-4, for example, the second feature image generation unit 126 performs an operation to generate an inspection feature image DIs based on the plurality of extracted images CIs acquired in step Sb-3. The second feature image generation unit 126 calculates the Mahalanobis distance (inspection Mahalanobis distance) with respect to the array data of pixel values (luminance values) for each pixel (i, j) included in the plurality of extracted images CIs, based on the data distribution of the plurality of extracted images CIr in any one of the extracted image groups CIGr obtained in the preparation phase.

[0073] Similar to step Sa-4 in the preparation phase, the second feature image generation unit 126 executes a process of arranging the pixel values included in each of the variables x1 to XN corresponding to the N extracted images CIs in a vertical array. The second feature image generation unit 126 calculates the Mahalanobis distance for the array data of the pixel values arranged horizontally for each pixel (i, j) included in the N extracted images CIs, using the average and the covariance matrix obtained from the plurality of extracted images CIr in step Sa-4. Thus, the calculation of the Mahalanobis distance in the present disclosure also includes calculating using the average and the covariance matrix used when generating the reference data, rather than the average and the covariance matrix obtained from the data for which the distance is to be calculated.

[0074] Next, the control device 100 executes step Sb-5. In step Sb-5, for example, the second feature image generation unit 126 generates an inspection feature image DIs based on the calculation result of the Mahalanobis distance in step Sb-4. The second feature image generation unit 126 may set the Mahalanobis distance calculated in step Sb-4 as the pixel value at each pixel (i, j).

[0075] Next, the control device 100 executes step Sb-6. In step Sb-6, for example, the abnormality determination unit 136 generates a comparison image DiI by comparing the inspection feature image DIs generated in step Sb-5 with the reference feature image DIr held by the reference image holding unit 118. The abnormality determination unit 136 may calculate the pixel value at the pixel of the comparison image DiI by calculating the difference between the pixel value of the inspection feature image DIs and the pixel value of the reference feature image DIr for each pixel (i, j).

[0076] Next, the control device 100 executes step Sb-7. In step Sb-7, for example, the abnormality determination unit 136 determines the presence or absence of an abnormality on the surface Wa of the work W to be inspected based on the result of comparing the inspection feature image DIs and the reference feature image DIr (the comparison image DiI generated in step Sb-6). The abnormality determination unit 136 executes a process of extracting pixels having a pixel value equal to or greater than a predetermined value in the inspection feature image DIs. The predetermined value is set to such an extent that the pixel value at the defective part can be extracted when a defect occurs on the surface Wa of the work W.

[0077] The abnormality determination unit 136 may determine that there is an abnormality on the surface Wa of the work W to be inspected when a region (or pixel) having a pixel value equal to or greater than a predetermined value is detected in the comparison image DiI. The abnormality determination unit 136 may determine that there is no abnormality on the surface Wa of the work W to be inspected when a region (or pixel) having a pixel value equal to or greater than a predetermined value is not detected in the comparison image DiI.

[0078] Next, the control device 100 executes step Sb-8. In step Sb-8, for example, the determination result output unit 138 outputs information indicating the determination result in step Sb-7 to the process control unit 102 or the host controller. When an abnormality signal indicating the presence of an abnormality is output to the process control unit 102 or the host controller, the work W determined to have an abnormality (defect) on the surface Wa may be excluded from the processing line after the inspection in the inspection unit U3.

[0079] [Effects of the First Embodiment] The substrate inspection method according to the first embodiment described above includes obtaining a reference input image based on an image obtained by imaging the surface Wa of the reference work W, obtaining reference intermediate information generated in the intermediate layer of the neural network (image recognition model M) that is pre-constructed to output the recognition result of the input image when the reference input image is input to the image recognition model M, and generating a reference feature image DIr indicating the features of the reference input image based on the reference intermediate information. The substrate inspection method further includes obtaining an inspection input image based on an image obtained by imaging the surface Wa of the work W to be inspected, obtaining inspection intermediate information generated in the intermediate layer of the image recognition model M when the inspection input image is input to the image recognition model M, and generating an inspection feature image DIs indicating the features of the inspection input image based on the inspection intermediate information. The substrate inspection method further includes determining the presence or absence of an abnormality on the surface Wa of the work W to be inspected based on the result of comparing the reference feature image DIr and the inspection feature image DIs.

[0080] In the intermediate layer of the image recognition model M, a process of extracting a specific shape from the image input to the image recognition model M is executed. Therefore, the reference feature image DIr generated from the information generated in the intermediate layer of the image recognition model M can represent the features of the entire surface Wa of the reference work W. Also, the inspection feature image DIs generated from the information generated in the intermediate layer of the image recognition model M can represent the features of the entire surface Wa of the work W to be inspected. By comparing the reference feature image DIr and the inspection feature image DIs, it is possible to cancel out the features of the work W itself and detect the features of the abnormal site. Therefore, the substrate inspection method is useful for accurately detecting abnormalities on the surface Wa of the work W.

[0081] In the substrate inspection method described above, the reference intermediate information may be a plurality of reference extraction images (a plurality of extraction images CIr) generated based on a reference input image and a plurality of filters that extract different features in an intermediate layer of the image recognition model M. Generating the reference feature image DIr may include generating the reference feature image DIr based on the plurality of extraction images CIr. The inspection intermediate information may be a plurality of inspection extraction images (a plurality of extraction images CIs) generated based on an inspection input image and the plurality of filters in an intermediate layer of the image recognition model M. Generating the inspection feature image DIs may include generating the inspection feature image DIs based on the plurality of extraction images CIs. In this case, various specific shapes are extracted by the plurality of filters in the intermediate layer of the image recognition model M. By extracting various specific shapes, when an abnormal site is included in the surface Wa of the workpiece W, the abnormal site can be extracted in response to the filter. Therefore, it is useful for highly accurate defect detection on the surface Wa of the workpiece W.

[0082] In the substrate inspection method described above, generating the reference feature image DIr may include calculating the pixel value for each pixel of the reference feature image DIr based on the result of calculating the reference Mahalanobis distance with respect to the array data of pixel values for each pixel (i, j) included in the plurality of extracted images CIr, based on the data distribution of the plurality of extracted images CIr. Generating the inspection feature image DIs may include calculating the pixel value for each pixel of the inspection feature image DIs based on the result of calculating the inspection Mahalanobis distance with respect to the array data of pixel values for each pixel (i, j) included in the plurality of extracted images CIs, based on the mean and covariance matrix used when calculating the above reference Mahalanobis distance. The Mahalanobis distance can represent the degree of deviation (anomaly) from the data distribution. Therefore, if there is an abnormal site on the surface Wa, the pixel value at a specific pixel fluctuates in response to the filter in the image recognition model M. As a result, the Mahalanobis distance at a specific pixel can be a large value. In the above configuration, since the feature images are compared with each other, it is possible to cancel out the regions where the Mahalanobis distance becomes large due to the features of the work W itself, and detect the regions where the Mahalanobis distance becomes large due to the abnormal site. Therefore, it is more useful for highly accurate defect detection on the surface Wa of the work W.

[0083] In the substrate processing method described above, the reference input image may be an enhanced image EIr generated by performing a process of enhancing the contrast on the captured image PIr of the surface Wa of the reference work W. The inspection input image may be an enhanced image EIs generated by performing a process of enhancing the contrast on the captured image PIs of the surface Wa of the work W to be inspected. In this case, the location corresponding to the abnormal site is emphasized, and the inspection feature image DIs reflecting the feature can be obtained. Even if the portions other than the abnormal site are emphasized as noise, the noise can be reduced by comparing the feature images. Therefore, it is more useful for highly accurate defect detection on the surface Wa of the work W.

[0084] The substrate processing method described above may include obtaining a second reference input image based on an image obtained by imaging the surface Wa of another work W for reference, and obtaining second reference intermediate information generated in the intermediate layer of the image recognition model M when the second reference input image is input to the image recognition model M. Generating the reference feature image DIr may include generating the reference feature image DIr based on the reference intermediate information and the second reference intermediate information. In this case, one reference feature image DIr is generated from the captured images of the surfaces Wa of a plurality of works W for reference. Therefore, it is possible to generate the reference feature image DIr while reducing the influence of the characteristics of an individual work W for reference. Accordingly, it is more useful for highly accurate abnormality detection on the surface Wa of the work W.

[0085] [Second Embodiment] Subsequently, with reference to FIGS. 12 to 16, the substrate processing system 1 according to the second embodiment will be described. In the substrate processing system 1 according to the second embodiment, a series of processes executed by the inspection control unit 110 are different from a series of processes executed by the inspection control unit 110 in the first embodiment. The coating and developing apparatus 2 may perform coating and developing processing (substrate processing) in lot units on a predetermined number of works W. The coating and developing apparatus 2 sequentially performs coating and developing processing on a predetermined number of works W in processing in lot units (lot processing). The predetermined number indicating the unit of the lot may be determined according to the number of works W that can be accommodated in the carrier C. The coating and developing apparatus 2 repeatedly performs coating and developing processing in lot units.

[0086] Unlike the inspection method in the first embodiment, the inspection control unit 110 generates a reference feature image, which is reference data used for inspection, while the production phase is being executed. The inspection control unit 110 generates the reference feature image using the work W (first substrate) on which the processing is executed first in the coating and development process in terms of lots. In this case, although the presence or absence of abnormalities on the surface Wa of the work W on which the processing is executed first is unknown, it serves as the reference work W. The inspection control unit 110 uses the reference feature image to perform an inspection on the work W (second substrate) on which the processing is executed second and later in the coating and development process in terms of lots as the work W to be inspected.

[0087] Unlike the inspection method in the first embodiment, the inspection control unit 110 executes two different inspection procedures and then determines the presence or absence of abnormalities on the surface Wa of the work W from the results obtained from these inspection procedures. FIG. 12 shows a series of processes executed in one inspection procedure, and FIG. 13 shows a series of processes executed in the other inspection procedure. In any of the inspection procedures, a reference feature image is generated from the first work W. FIG. 14 is a flowchart showing an example of a series of processes executed by the inspection control unit 110 in terms of lots.

[0088] In this series of processes, after the processing before the inspection execution in the coating and development process is performed, with the work W to be processed in a state of being transported to the inspection unit U3, the control device 100 executes step S41. In step S41, for example, the inspection control unit 110 determines whether the work W to be processed transported to the inspection unit U3 is the first work W in the processing in terms of lots. The inspection control unit 110 may determine whether it is the first work W in terms of lots by counting the number of works W inspected by the inspection unit U3 since the start of the production phase.

[0089] In step S41, when the workpiece W to be processed conveyed to the inspection unit U3 is the first workpiece W (step S41: YES), the process executed by the control device 100 proceeds to step S50. In step S50, for example, the inspection control unit 110 executes inspection preparation processing for inspecting the workpieces W to be processed from the second one. FIG. 15 is a flowchart showing an example of the inspection preparation processing in step S50. The inspection preparation processing in step S50 includes a series of processes for the first one shown in FIG. 12 and a series of processes for the first one shown in FIG. 13.

[0090] In the inspection preparation processing in step S50, the control device 100 first executes step Sc-1. Step Sc-1 is executed in the same manner as step Sa-1 in the substrate inspection method according to the first embodiment. In step Sc-1, for example, the first input image acquisition unit 112 of the inspection control unit 110 acquires an imaging image PIr of the surface Wa of the first workpiece W by imaging the surface Wa of the first workpiece W with the inspection unit U3.

[0091] Next, the control device 100 executes step Sc-3. Step Sc-3 is executed in the same manner as step Sa-3 in the substrate inspection method according to the first embodiment. In step Sc-3, for example, when the first intermediate information acquisition unit 114 inputs the imaging image PIr (reference input image) obtained in step Sc-1 to the image recognition model M, the reference intermediate information generated in the intermediate layer of the image recognition model M is acquired. The first intermediate information acquisition unit 114 acquires, as reference intermediate information, an extraction image group CIGr1 including a plurality of extraction images CIr1 (a plurality of reference extraction images) generated based on the imaging image PIr and a plurality of filters that extract different features from each other in the intermediate layer of the image recognition model M. The number of the plurality of extraction images CIr1 included in the extraction image group CIGr1 may be 30 to 60.

[0092] Next, the control device 100 executes step Sc-4. Step Sc-4 is executed in the same manner as step Sa-4 in the substrate inspection method according to the first embodiment. In step Sc-4, for example, the first feature image generation unit 116 calculates the Mahalanobis distance (reference Mahalanobis distance) based on the data distribution of the plurality of extracted images CIr1 for the array data of the pixel values (luminance values) for each pixel (i, j) included in the plurality of extracted images CIr1.

[0093] Next, the control device 100 executes step Sc-5. Step Sc-5 is executed in the same manner as step Sa-5 or step Sb-5 in the substrate inspection method according to the first embodiment. In step Sc-5, for example, the first feature image generation unit 116 generates a reference feature image DIr1 based on the calculation result of the Mahalanobis distance in step Sc-4. The first feature image generation unit 116 may set the Mahalanobis distance calculated in step Sc-4 as the pixel value at the corresponding pixel in the reference feature image DIr1 for each pixel (i, j). The reference image holding unit 118 stores the reference feature image DIr1.

[0094] In parallel with, or after the execution of, the series of processes including steps Sc-3, Sc-4, and Sc-5, the control device 100 executes step Se-2. Step Se-2 is executed in the same manner as step Sa-2 in the substrate inspection method according to the first embodiment. In step Se-2, for example, the first input image acquisition unit 112 generates an enhanced image EIr2 by performing a process of enhancing the contrast on the captured image PIr obtained in step Sc-1.

[0095] Next, the control device 100 executes step Se-3. Step Se-3 is executed in the same manner as step Sc-3. In step Se-3, for example, when the first intermediate information acquisition unit 114 inputs the enhanced image EIr2 (reference input image) obtained in step Se-2 into the image recognition model M, it acquires the reference intermediate information generated in the intermediate layer of the image recognition model M. The first intermediate information acquisition unit 114 acquires, as the reference intermediate information, for example, an extraction image group CIGr2 including a plurality of extraction images CIr2 (a plurality of second reference extraction images) generated based on the enhanced image EIr2 and a plurality of filters (a plurality of second filters) that extract different features in the intermediate layer of the image recognition model M. The number of the plurality of extraction images CIr2 included in the extraction image group CIGr2 may be different from the number of the plurality of extraction images CIr1 included in the extraction image group CIGr1 obtained in step Sc-3, or may be 180 to 220. That is, the number of the plurality of filters for generating the extraction image group may be different between step Sc-3 and step Se-3.

[0096] Next, the control device 100 executes step Se-4. Step Se-4 is executed in the same manner as step Sc-4. In step Se-4, for example, the first feature image generation unit 116 calculates the Mahalanobis distance (reference Mahalanobis distance) based on the data distribution of the plurality of extraction images CIr1 for the array data of the pixel values (luminance values) for each pixel (i, j) included in the plurality of extraction images CIr1.

[0097] Next, the control device 100 executes step Se-5. Step Se-5 is executed in the same manner as step Sc-5. In step Se-5, for example, the first feature image generation unit 116 generates a reference feature image DIr2 (second reference feature image) based on the calculation result of the Mahalanobis distance in step Se-4. The first feature image generation unit 116 may set, for each pixel (i, j), the Mahalanobis distance calculated in step Se-4 as the pixel value at the pixel in the reference feature image DIr2. The reference image holding unit 118 stores the reference feature image DIr2.

[0098] As described above, the inspection preparation process in step S50 is completed, and the reference feature images DIr1 and DIr2, which are the reference data used in the inspection of the second and subsequent workpieces W, are generated.

[0099] After the execution of step S50, as shown in FIG. 14, the control device 100 executes step S60. In step S60, for example, the abnormality determination unit 136 determines the presence or absence of an abnormality on the surface Wa of the first workpiece W based on at least one of the reference feature images DIr1 and DIr2 (for example, the reference feature image DIr2). The abnormality determination unit 136 executes a process of extracting pixels having a pixel value equal to or greater than a predetermined value in the reference feature image DIr2, and may determine that there is an abnormality (defect) on the surface Wa when a region (or pixel) having a pixel value equal to or greater than the predetermined value is detected. In step S60, although the sensitivity is low, it may be possible to detect an abnormality on the surface Wa of the first workpiece W.

[0100] On the other hand, in step S41, when the workpiece W to be processed conveyed to the inspection unit U3 is any of the second and subsequent workpieces W (step S41: NO), the process executed by the control device 100 proceeds to step S70. In step S70, for example, the inspection control unit 110 executes an inspection on the workpiece W to be processed from the second one. FIG. 16 is a flowchart showing an example of the inspection process in step S70. The inspection process in step S70 includes a series of processes after the second one shown in FIG. 12 and a series of processes after the second one shown in FIG. 13.

[0101] In the inspection process of step S70, the control device 100 first executes step Sd-1. Step Sd-1 is executed under the same conditions as step Sc-1 described above. In step Sd-1, for example, the second input image acquisition unit 122 of the inspection control unit 110 acquires an imaging image PIs of the workpiece W to be inspected by imaging the surface Wa of the workpiece W to be inspected processed second by the inspection unit U3.

[0102] Next, the control device 100 executes step Sd-3. Step Sd-3 is executed under the same conditions as step Sc-3. In step Sd-3, for example, when the second intermediate information acquisition unit 124 inputs the captured image PIs (reference input image) obtained in step Sd-1 into the image recognition model M, the inspection intermediate information generated in the intermediate layer of the image recognition model M is acquired. The second intermediate information acquisition unit 124 acquires, as inspection intermediate information, for example, an extraction image group CIGs1 including a plurality of extraction images CIs1 (a plurality of inspection extraction images) generated based on the captured image PIs and a plurality of filters that extract different features from each other in the intermediate layer of the image recognition model M. The plurality of filters used in step Sd-3 are the same as the plurality of filters used in step Sc-3.

[0103] Next, the control device 100 executes step Sd-4. Step Sd-4 is executed in the same manner as step Sb-4 in the substrate inspection method according to the first embodiment. In step Sd-4, for example, the second feature image generation unit 126 calculates the Mahalanobis distance (inspection Mahalanobis distance) based on the data distribution of the plurality of extraction images CIr1 obtained in step Sc-3 for the array data of the pixel values (luminance values) for each pixel (i, j) included in the plurality of extraction images CIs1 obtained in step Sd-3.

[0104] Next, the control device 100 executes step Sd-5. Step Sd-5 is executed in the same manner as step Sc-5. In step Sd-5, for example, the second feature image generation unit 126 generates an inspection feature image DIs1 based on the calculation result of the Mahalanobis distance in step Sd-4. The second feature image generation unit 126 may set the Mahalanobis distance calculated in step Sd-4 as the pixel value at the corresponding pixel in the inspection feature image DIs1 for each pixel (i, j).

[0105] Next, the control device 100 executes step Sd-6. Step Sd-6 is executed in the same manner as step Sb-6 in the substrate inspection method according to the first embodiment. In step Sd-6, for example, the abnormality determination unit 136 generates a comparison image DiI1 by comparing the inspection feature image DIs1 generated in step Sd-5 with the reference feature image DIr1 held by the reference image holding unit 118. The abnormality determination unit 136 may calculate the pixel value at the pixel of the comparison image DiI1 by calculating the difference between the pixel value of the inspection feature image DIs1 and the pixel value of the reference feature image DIr1 for each pixel (i, j).

[0106] In parallel with the series of processes including steps Sd-3 to Sd-6, or after the execution of the series of processes, the control device 100 executes step Sf-2. Step Sf-2 is executed under the same conditions as step Se-2 described above. In step Sf-2, for example, the second input image acquisition unit 122 generates an enhanced image EIs2 by performing a process of enhancing the contrast of the captured image PIs obtained in step Sd-1.

[0107] Next, the control device 100 executes step Sf-3. Step Sf-3 is executed under the same conditions as step Se-3 described above. In step Sf-3, for example, when the second intermediate information acquisition unit 124 inputs the enhanced image EIs2 (inspection input image) obtained in step Sf-2 into the image recognition model M, the second intermediate information acquisition unit 124 acquires the inspection intermediate information generated in the intermediate layer of the image recognition model M. The second intermediate information acquisition unit 124 acquires, as the inspection intermediate information, an extraction image group CIGs2 including a plurality of extraction images CIs2 (a plurality of second inspection extraction images) generated based on the enhanced image EIs2 and a plurality of filters (a plurality of second filters) that extract different features from each other in the intermediate layer of the image recognition model M. The plurality of filters used in step Se-3 and the plurality of filters used in step Sf-3 are the same ones.

[0108] Next, the control device 100 executes step Sf-4. Step Sf-4 is executed in the same manner as step Se-4. In step Sf-4, for example, the second feature image generation unit 126 calculates the Mahalanobis distance (inspection Mahalanobis distance) based on the data distribution of the plurality of extracted images CIs2 with respect to the array data of the pixel values (luminance values) for each pixel (i, j) included in the plurality of extracted images CIs2 obtained in step Sf-3. In step Sd-4, the average and covariance matrix used when calculating the Mahalanobis distance to generate the reference data are used, whereas in step Sf-4, the average and covariance matrix obtained from itself (the plurality of extracted images CIs2) are used.

[0109] Next, the control device 100 executes step Sf-5. Step Sf-5 is executed in the same manner as step Se-5. In step Sf-5, for example, the second feature image generation unit 126 generates an inspection feature image DIs2 (second inspection feature image) based on the calculation result of the Mahalanobis distance in step Sf-4. The second feature image generation unit 126 may set the Mahalanobis distance calculated in step Sf-4 as the pixel value at the corresponding pixel in the inspection feature image DIs2 for each pixel (i, j).

[0110] Next, the control device 100 executes step Sf-6. Step Sf-6 is executed in the same manner as step Sd-6. In step Sf-6, for example, the abnormality determination unit 136 generates a comparison image DiI2 by comparing the inspection feature image DIs2 generated in step Sf-5 with the reference feature image DIr2 held by the reference image holding unit 118. The abnormality determination unit 136 may calculate the pixel value at the corresponding pixel in the comparison image DiI2 by calculating the difference between the pixel value of the inspection feature image DIs2 and the pixel value of the reference feature image DIr2 for each pixel (i, j).

[0111] Next, the control device 100 executes step S47. In step S47, for example, the abnormality determination unit 136 determines the presence or absence of an abnormality on the surface Wa of the work W to be processed based on the result of comparing the reference feature image DIr1 with the inspection feature image DIs1 and the result of comparing the reference feature image DIr2 with the inspection feature image DIs2. The abnormality determination unit 136 determines the presence or absence of an abnormality on the surface Wa of the work W to be processed based on the comparison image DiI1 obtained in step Sd-6 and the comparison image DiI2 obtained in step Sf-6.

[0112] In one example, the abnormality determination unit 136 executes a process of extracting pixels having a pixel value equal to or greater than a predetermined value in each of the comparison image DiI1 and the comparison image DiI2. When a region (or pixel) having a pixel value equal to or greater than a predetermined value is detected in at least one of the comparison image DiI1 and the comparison image DiI2, the abnormality determination unit 136 may determine that there is an abnormality on the surface Wa of the work W to be inspected. When regions (or pixels) having pixel values equal to or greater than a predetermined value are not detected in both the comparison image DiI1 and the comparison image DiI2, the abnormality determination unit 136 may determine that there is no abnormality on the surface Wa of the work W to be inspected.

[0113] Returning to FIG. 14, after the execution of step S60 or step S70, the control device 100 executes step S48. In step S48, for example, the determination result output unit 138 outputs information indicating the determination results in steps S47 and S60 to the process control unit 102 or the upper controller. The work W determined to have an abnormality (defect) on the surface Wa may be excluded from the processing line after the inspection in the inspection unit U3. When an abnormality is detected in the first work W in step S60, the inspection control unit 110 may execute step S50 for the second work W to generate reference data from the captured image or the like of the second work W.

[0114] Next, the control device 100 executes step S49. In step S49, for example, the control device 100 determines whether the inspection of a predetermined number of works W that define the lot unit has been completed. In step S49, if it is determined that the inspection of the predetermined number of works W has not been completed (step S49: NO), the process executed by the control device 100 returns to step S41. In step S49, if it is determined that the inspection of the predetermined number of works W has been completed (step S49: YES), the board inspection for one lot ends. The control device 100 (inspection control unit 110) executes the same board inspection process for the next lot.

[0115] [Effects of the Second Embodiment] Also in the board inspection method executed by the board processing system 1 according to the second embodiment, the same effects as those of the first embodiment are achieved, so it is useful for accurately detecting abnormalities on the surface Wa of the work W.

[0116] In the board inspection method according to the second embodiment described above, generating the reference feature image DIr2 may include calculating the pixel value for each pixel of the reference feature image DIr2 based on the result of calculating the reference Mahalanobis distance with respect to the array data of the pixel values for each pixel included in a plurality of reference extraction images (a plurality of extraction images CIr2), based on the data distribution of the plurality of extraction images CIr2. Generating the inspection feature image DIs2 may include calculating the pixel value for each pixel of the inspection feature image DIs2 based on the result of calculating the inspection Mahalanobis distance with respect to the array data of the pixel values for each pixel included in a plurality of inspection extraction images (a plurality of extraction images CIs2), based on the data distribution of the plurality of extraction images CIs2. When calculating the inspection Mahalanobis distance using the mean and covariance matrix when calculating the reference Mahalanobis distance when creating the reference data, changes that are not desired to be determined as abnormalities between the reference work W and the work W to be inspected can be detected. In the above configuration, the inspection Mahalanobis distance is calculated from its own data distribution, and the difference from the reference work W is not reflected in the inspection Mahalanobis distance. Therefore, it is useful for adjusting the detection sensitivity according to the type of abnormality to be detected.

[0117] In the substrate inspection method according to the second embodiment described above, the reference work W may be the first work W to which the predetermined coating and development treatment is applied first in the lot process of sequentially applying the predetermined coating and development treatment to a predetermined number of works W to be processed. The work W to be inspected may be any of the second and subsequent works W to which the coating and development treatment is applied in any order after the second in the lot process. In this case, reference data for inspection is generated in lot units. Therefore, variations in the work W itself or in the coating and development treatment among a plurality of lots are less likely to affect the abnormality determination. Therefore, it is useful for highly accurate abnormality detection on the surface Wa of the work W.

[0118] The substrate inspection method according to the second embodiment described above includes, in addition to the generation of the reference feature image DIr1 and the generation of the inspection feature image DIs1, when a reference input image is input to the image recognition model M, in the intermediate layer of the image recognition model M, a plurality of second reference extraction images (a plurality of extraction images CIr2) generated based on the reference input image and a plurality of second filters that extract different features from each other are obtained, and a second reference feature image (reference feature image DIr2) is generated based on the plurality of extraction images CIr2. The substrate inspection method may further include, when an inspection input image is input to the image recognition model M, obtaining, in the intermediate layer of the image recognition model M, a plurality of second inspection extraction images (a plurality of extraction images CIs2) generated based on the inspection input image and the plurality of second filters, and generating a second inspection feature image (inspection feature image DIs2) based on the plurality of extraction images CIs2. Determining the presence or absence of an abnormality on the surface Wa of the work W to be inspected may include determining the presence or absence of an abnormality on the surface Wa of the work W to be inspected based on the result of comparing the reference feature image DIr1 and the inspection feature image DIs1 and the result of comparing the reference feature image DIr2 and the inspection feature image DIs2. In this case, since the feature images are compared in two different inspection procedures, it is possible to easily adjust the detection sensitivity according to the type of abnormality to be detected while reducing the possibility that the abnormality cannot be detected. Therefore, it is useful for achieving both the adjustment of the detection sensitivity of the abnormality and the detection accuracy.

[0119] The substrate inspection method according to the second embodiment described above may further include determining whether there is an abnormality on the surface Wa of the first workpiece W based on the reference feature image. In this case, if the surface Wa of the first workpiece W for generating the inspection reference data contains an obvious abnormality, that workpiece W can be excluded, and another workpiece W can be used to create the reference data. Therefore, it is useful for highly accurate abnormality detection on the surface Wa of the workpiece W.

[0120] [Modification Example] The present disclosure is not limited to the first and second embodiments described above. Some matters described in the first embodiment may be applied to the second embodiment, and some matters described in the second embodiment may be applied to the first embodiment. The series of processes described above can be changed as appropriate. In the above series of processes, the control device 100 (inspection control unit 110) may execute one step and the next step in parallel, or may execute each step in an order different from the above-described example. The control device 100 (inspection control unit 110) may omit any step, or may execute a process different from the above example in any step.

[0121] The inspection control unit 110 according to the first embodiment may omit steps Sa-2 and Sb-2. In this case, in step Sa-3, the inspection control unit 110 inputs the captured image PIr into the image recognition model M, and in step Sb-3, inputs the captured image PIs into the image recognition model M. The inspection control unit 110 according to the first embodiment may generate a reference feature image DIr, which is reference data, from one reference workpiece W.

[0122] The inspection control unit 110 according to the second embodiment executes two different inspection procedures, but it may execute only one of the inspection procedures. The inspection control unit 110 may determine the presence or absence of an abnormality on the surface Wa of the work W to be inspected based on the comparison image DiI1 obtained by executing the inspection procedure shown in FIG. 12 without executing the inspection procedure shown in FIG. 13. The inspection control unit 110 may determine the presence or absence of an abnormality on the surface Wa of the work W to be inspected based on the comparison image DiI2 obtained by executing the inspection procedure shown in FIG. 13 without executing the inspection procedure shown in FIG. 12.

[0123] In the inspection procedure according to the second embodiment, a series of processes performed on the first work W may be executed in the preparation phase before the production phase starts. Then, in the production phase, a series of processes performed on the second and subsequent works W may be executed for the work W to be processed regardless of the lot unit.

[0124] The external device of the control device 100 may store the image recognition model M without the inspection control unit 110 holding the image recognition model M. In this case, the inspection control unit 110 may transmit the captured image or the enhanced image to the external device and then acquire the extracted image group CIG from the external device.

[0125] The computer constituting the inspection control unit 110 may be provided outside the coating and developing apparatus 2. In this case, the control device 100 and the inspection control unit 110 may be communicably connected via wire, wirelessly, or a network. The control device 100 may acquire the captured image from the inspection unit U3 and then transmit the captured image to the inspection control unit 110. The inspection control unit 110 may transmit information indicating the determination result of the presence or absence of an abnormality to the control device 100. In one of the various examples described above, at least a part of the matters described in other examples may be applied.

Explanation of Signs

[0126] 1… Substrate processing system, 2… Coating and developing apparatus, U3… Inspection unit, W… Workpiece, 110… Inspection control unit, 112… First input image acquisition unit, 114… First intermediate information acquisition unit, 116… First feature image generation unit, 122… Second input image acquisition unit, 124… Second intermediate information acquisition unit, 126… Second feature image generation unit, 136… Abnormality determination unit, PIr, PIs… Captured images, EIr, EIs… Enhanced images, CIr, CIs… Extracted images, DIr… Reference feature image, DIs… Inspection feature image, DiI… Comparison image.

Claims

1. acquiring a reference input image based on an image of a surface of a reference substrate; generating a reference feature image indicative of features of the reference input image; acquiring an inspection input image based on an image of a surface of a substrate to be inspected; generating an inspection feature image indicative of features of the inspection input image; determining whether or not there is an abnormality on the surface of the substrate to be inspected based on a result of comparing the reference feature image with the inspection feature image; the reference substrate is a first substrate that is subjected to a predetermined substrate treatment first in a lot process in which a predetermined number of substrates to be processed are sequentially subjected to the predetermined substrate treatment, A substrate inspection method, wherein the substrate to be inspected is a second substrate to be subjected to the substrate processing in any order from the second onward in the lot processing.

2. A substrate inspection method as described in claim 1, further comprising determining whether or not there is an abnormality on the surface of the first substrate based on the reference feature image.

3. A substrate inspection method as described in claim 2, in which, when an abnormality is detected in the first substrate, the second substrate to be subjected to the substrate processing in the lot processing is used as the new reference substrate, and the acquisition of the reference input image and the generation of the reference feature image are performed again.

4. The method further includes, when the reference input image is input to a neural network that is pre-constructed to output a recognition result of the input image, obtaining, in an intermediate layer of the neural network, a plurality of reference extracted images that are generated based on the reference input image and a plurality of filters that extract different features from each other; 4. The substrate inspection method according to claim 1, wherein in generating the reference feature image, the reference feature image is generated based on the plurality of reference extracted images.

5. A substrate inspection method as described in Claim 4, wherein generating the reference feature image includes calculating pixel values ​​for each pixel of the reference feature image based on the results of calculating a reference Mahalanobis distance for array data of pixel values ​​for each pixel contained in the plurality of reference extracted images, using the data distribution of the plurality of reference extracted images as a reference.

6. Obtaining a second reference input image based on an image of a surface of another substrate for reference; and acquiring, in an intermediate layer of the neural network, a plurality of second reference extracted images generated based on the second reference input image and the plurality of filters when the second reference input image is input to the neural network, The substrate inspection method according to claim 4 , wherein generating the reference feature image includes generating the reference feature image based on the plurality of reference extracted images and the plurality of second reference extracted images.

7. A substrate inspection method described in any one of claims 1 to 3, wherein the reference input image is an image generated by applying a process to enhance contrast to an image taken of the surface of the reference substrate.

8. Obtaining a reference input image based on an image of a surface of a reference substrate; generating a reference feature image indicative of features of the reference input image; acquiring an inspection input image based on an image of a surface of a substrate to be inspected; generating an inspection feature image indicative of features of the inspection input image; and determining whether or not there is an abnormality on the surface of the substrate to be inspected based on a result of comparing the reference feature image with the inspection feature image, the reference substrate is a first substrate that is subjected to a predetermined substrate treatment first in a lot process in which a predetermined number of substrates to be processed are sequentially subjected to the predetermined substrate treatment, A substrate inspection program, wherein the substrate to be inspected is a second substrate to be subjected to the substrate processing in any order from the second onward in the lot processing.

9. a first input image acquisition unit that acquires a reference input image based on an image of a surface of a reference substrate; a first feature image generation unit that generates a reference feature image that indicates features of the reference input image; a second input image acquisition unit that acquires an inspection input image based on an image of a surface of a substrate to be inspected; a second feature image generating unit that generates an inspection feature image indicating a feature of the inspection input image; an abnormality determination unit that determines whether or not there is an abnormality on the surface of the substrate to be inspected based on a result of comparing the reference feature image with the inspection feature image, the reference substrate is a first substrate that is subjected to a predetermined substrate treatment first in a lot process in which a predetermined number of substrates to be processed are sequentially subjected to the predetermined substrate treatment, A substrate inspection apparatus, wherein the substrate to be inspected is a second substrate to be subjected to the substrate processing in any order from the second onward in the lot processing.

10. A substrate inspection device as described in Claim 9, wherein the abnormality determination unit further determines the presence or absence of an abnormality on the surface of the first substrate based on the reference feature image.

11. A substrate inspection device as described in Claim 10, wherein when an abnormality is detected in the first substrate, the second substrate to be subjected to the substrate processing in the lot processing is used as the new reference substrate, and the acquisition of the reference input image and the generation of the reference feature image are performed again.

12. The neural network further comprises a first intermediate information acquisition unit that, when the reference input image is input to a neural network that has been pre-constructed to output a recognition result of the input image, acquires, in an intermediate layer of the neural network, a plurality of reference extracted images that are generated based on the reference input image and a plurality of filters that extract different features from each other; 12. The substrate inspection device according to claim 9, wherein the first feature image generation unit generates the reference feature image based on the plurality of reference extracted images.

13. A substrate inspection device as described in Claim 12, wherein, in generating the reference feature image, the first feature image generation unit calculates the pixel value for each pixel of the reference feature image based on the result of calculating a reference Mahalanobis distance for array data of pixel values ​​for each pixel contained in the multiple reference extraction images, using the data distribution of the multiple reference extraction images as a reference.

14. The first input image acquisition unit further acquires a second reference input image based on an image of a surface of another substrate for reference; the first intermediate information acquisition unit, when inputting the second reference input image to the neural network, further acquires, in an intermediate layer of the neural network, a plurality of second reference extracted images generated based on the second reference input image and the plurality of filters; The substrate inspection device according to claim 12 , wherein the first feature image generating unit generates the reference feature image based on the plurality of reference extraction images and the plurality of second reference extraction images.

15. A substrate inspection device described in any one of claims 9 to 11, wherein the first input image acquisition unit generates the reference input image by performing a process to enhance contrast on an image captured of the surface of the reference substrate.

Images