Mark detection method and computer program

Abstract

To provide a technique that can lighten a burden of an operator who searches for an alignment mark.SOLUTION: An encoding step and an image selection step are performed by a computer 20. In the encoding step, a plurality of partial images D2 of an object are input to a learned VAE model M, and the plurality of partial images D2 are encoded to a plurality of latent variables in a multi-dimensional latent space. In the image selection step, one or more partial images D2 that are likely to include an alignment mark are selected on the basis of a distribution of the plurality of latent variables in the latent space. Thus, a partial image that is likely to include an alignment mark is automatically detected by the computer 20 with the use of the VAE model M. This can lighten a burden on an operator who searches for an alignment mark.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a technique for detecting an alignment mark from an object.

Background Art

[0002] Conventionally, in an apparatus that performs precise processing on an object, such as a semiconductor manufacturing apparatus, alignment of the object is performed based on an alignment mark attached to the object. A conventional technique for aligning an object using an alignment mark is described in, for example, Patent Document 1.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In this type of apparatus, when aligning an object, conventionally, an operator visually searches for an alignment mark while magnifying an image of the object. However, in such a conventional method, it takes time to set the alignment mark and the burden on the operator is also large.

[0005] The present invention has been made in view of such circumstances, and an object thereof is to provide a technique capable of reducing the burden on an operator who searches for an alignment mark.

Means for Solving the Problems

[0006] To solve the above problems, the first invention of this application provides a mark detection method for detecting alignment marks from an object, comprising: an encoding step of inputting a plurality of partial images of the object into a trained VAE model and encoding the plurality of partial images into a plurality of latent variables in a multidimensional latent space; and an image selection step of selecting one or more partial images that may contain the alignment marks based on the distribution of the plurality of latent variables in the latent space, all of which are performed by a computer.

[0007] The second invention of the present application is a mark detection method of the first invention, wherein the image selection step comprises: a) calculating the distance to the nearest latent variable for each latent variable in the latent space; and b) selecting one or a predetermined number of partial images corresponding to one or a predetermined number of latent variables whose distance is greater than that of other latent variables, as one or more partial images that may contain the alignment mark.

[0008] The third invention of this application is a mark detection method according to the first or second invention, wherein, prior to the encoding step, a partial image extraction step is performed to extract a partial image containing a characteristic shape from an image of the object, and in the encoding step, the partial image extracted by the partial image extraction step is input to the VAE model.

[0009] The fourth invention of this application is a mark detection method according to any one of the first to third inventions, further comprising an image removal step of removing a portion of the plurality of portion images having probe marks, scratches, or particles from the plurality of portion images using a classifier created by supervised learning.

[0010] The fifth invention of this application is a mark detection method according to any one of the first to fourth inventions, further comprising a learning step of creating or updating the VAE model by machine learning, wherein the learning step involves inputting learning input images of the object to the VAE model, and adjusting the parameters of the VAE model such that the difference between the input images and the reconstructed images decoded from the latent variables becomes small, and the distribution of the latent variables in the latent space becomes a predetermined probability distribution.

[0011] The sixth invention of this application is a mark detection method of the fifth invention, wherein the learning step comprises: a pre-adjustment learning step in which machine learning is performed with the latent space as a first number of dimensions; a dimensionality adjustment step in which, after the pre-adjustment learning step, the number of dimensions of the latent space is changed to a second number of dimensions which is less than the first number of dimensions, based on the distribution of the latent variables in each dimension of the latent space; and a post-adjustment learning step in which machine learning is performed with the latent space as a second number of dimensions.

[0012] The seventh invention of this application is a mark detection method of the sixth invention, wherein in the dimensional adjustment step, the number of dimensions of the latent space is reduced based on the variance or standard deviation calculated for each component of the plurality of latent variables in the latent space.

[0013] The eighth invention of this application is a mark detection method according to any one of the fifth to seventh inventions, wherein the learning step comprises: a first learning step of performing machine learning based on a first input image; an effective component determination step, after the first learning step, of determining a predetermined number of effective components with a large standard deviation or variance from among a plurality of components of the latent variable in the latent space; a corrected image generation step of generating a corrected image by decoding an effective vector having values ​​only in the effective components; an input image correction step of generating a second input image by blending the corrected image with the first input image; and a second learning step of performing machine learning based on the second input image.

[0014] The ninth invention of this application is a computer program that causes a computer to execute a mark detection method of any one of the first to eighth inventions. [Effects of the Invention]

[0015] According to the first to ninth inventions of this application, a VAE model is used to automatically detect partial images that may contain alignment marks using a computer. This reduces the burden on workers searching for alignment marks.

[0016] In particular, according to the second invention of this application, partial images with fewer similar features are selected based on the distance between latent variables. This allows for the appropriate selection of partial images that may contain alignment marks.

[0017] In particular, according to the third invention of this application, partial images containing characteristic shapes are extracted in advance, and the extracted partial images are input into the VAE model. This reduces the number of partial images input into the VAE model. As a result, the encoding process and the image selection process can be accelerated.

[0018] In particular, according to the fourth invention of this application, it is possible to suppress the selection of partial images having probe marks, scratches, or particles. This makes it possible to improve the accuracy of alignment mark detection.

[0019] In particular, according to the sixth invention of this application, the number of dimensions of the latent space can be adjusted to an appropriate number of dimensions. This makes it possible to speed up the encoding process and the image selection process.

[0020] In particular, according to the eighth invention of the present application, a corrected image is created based on an effective vector having a value only for the effective component capturing the features of the first input image, and the corrected image is blended with the first input image. Thereby, a second input image emphasizing the characteristic elements of the first input image can be generated. Then, by performing machine learning based on the second input image, a more accurate VAE model can be created. As a result, the detection of the alignment mark can be performed with higher accuracy.

Brief Description of Drawings

[0021] [Figure 1] It is a diagram showing the configuration of the inspection apparatus. [Figure 2] It is a block diagram conceptually showing the functions of the computer. [Figure 3] It is a diagram conceptually showing the VAE model. [Figure 4] It is a flowchart showing the flow of the learning process. [Figure 5] It is a flowchart showing the flow of the inspection process. [Figure 6] It is a diagram conceptually showing an example of the distribution of latent variables in the latent space. [Figure 7] It is a flowchart showing the flow of the inspection process according to the first modification example. [Figure 8] It is a flowchart showing the flow of the learning process according to the second modification example. [Figure 9] It is a graph showing an example of the result of calculating the variances of a plurality of latent variables for each component. [Figure 10] It is a flowchart showing the flow of the learning process according to the third modification example.

Embodiments for Carrying Out the Invention

[0022] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0023] <1. Configuration of Inspection Apparatus> Figure 1 shows the configuration of the inspection device 1. This inspection device 1 is a device that inspects an object 9 while performing a mark detection method according to one embodiment of the present invention. The object 9 is, for example, a semiconductor substrate. However, the object 9 may be a glass substrate used in flat panel displays such as liquid crystal displays. The object 9 may also be other precision electronic components such as printed circuit boards or battery components. Furthermore, the object 9 may be a sheet-like substrate such as printing paper.

[0024] As shown in Figure 1, the inspection device 1 comprises an imaging unit 10, a computer 20, and a display unit 30.

[0025] The imaging unit 10 has a camera 11 for photographing the object 9. The camera 11 has an image sensor such as a CCD or CMOS, and an optical system that forms an image on the image sensor of light incident from the object 9. The imaging unit 10 may also have an illumination device that irradiates the object 9 with light during shooting. The imaging unit 10 is electrically connected to the computer 20.

[0026] As shown in Figure 1, a pattern to be inspected 91, such as a wiring pattern, is formed on the surface of the object 9. Alignment marks 92 are also marked on the surface of the object 9. The alignment marks 92 are marks used to align the object 9 in the inspection device 1 in order to inspect multiple objects 9 at the same position. The imaging unit 10 captures images of the pattern to be inspected 91 and the alignment marks 92. This yields an image D1 containing the pattern to be inspected 91 and the alignment marks 92. The image D1 is data in which a large number of pixels are arranged on a two-dimensional coordinate system, and a brightness value is defined for each pixel. The imaging unit 10 inputs the image D1 obtained by the imaging into the computer 20.

[0027] The computer 20 is an information processing device that performs alignment of the object 9 and inspects the object 9 based on the captured image D1 input from the imaging unit 10. As shown in Figure 1, the computer 20 has a processor 21 such as a CPU, memory 22 such as RAM, and a storage unit 23 such as a hard disk drive.

[0028] The memory unit 23 stores a computer program 24 that causes the computer 20 to perform the learning process, alignment mark 92 detection process, alignment process, and inspection process, which will be described later. The computer program 24 is read from a storage medium readable by the computer 20, such as a CD or DVD, and stored in the memory unit 23. However, the computer program 24 may also be downloaded to the computer 20 via a network.

[0029] The display unit 30 is a device that displays various information related to the processing of the inspection device 1. For example, a liquid crystal display device is used for the display unit 30. The display unit 30 is electrically connected to the computer 20. The display unit 30 displays information output from the computer 20 on its screen. The information displayed on the display unit 30 includes, for example, the detection results of the alignment marks 92, which will be described later, and the inspection results of the object 9.

[0030] Figure 2 is a block diagram conceptually illustrating the functions of the computer 20 described above. As shown in Figure 2, the computer 20 has a learning unit 41, a partial image extraction unit 42, an encoding unit 43, a nearest neighbor distance calculation unit 44, an image selection unit 45, an alignment unit 46, and an inspection unit 47. The functions of the learning unit 41, partial image extraction unit 42, encoding unit 43, nearest neighbor distance calculation unit 44, image selection unit 45, alignment unit 46, and inspection unit 47 are realized by the computer 20's processor 21 operating according to the computer program 24. The functions of each of these units will be explained below along with the processing flow.

[0031] <2. About the learning process> First, we will explain the learning process that is performed in the inspection device 1 before the inspection process.

[0032] The learning process involves creating a Variational Autoencoder (VAE) model M using machine learning. Figure 3 is a conceptual diagram of the VAE model M. The VAE model M is a learning model that outputs a reconstructed image Do by encoding and decoding an input image Di.

[0033] As shown in Figure 3, the VAE model M has an encoder unit Me and a decoder unit Md. The encoder unit Me is a processing unit that converts (encodes) the input image Di into latent variables in a multidimensional latent space (feature space) LS. The latent variables are represented as vector values ​​having components in each dimension of the latent space LS. The decoder unit Md is a process that reconstructs (decodes) the image from the latent variables and outputs a reconstructed image Do that approximates the input image Di. A multilayer neural network is used for both the encoder unit Me and the decoder unit Md.

[0034] In the latent space LS, one latent variable represents a feature of one input image Di. In the VAE model M, a large number of input images Di are encoded such that the distribution of latent variables in the latent space LS follows a predetermined probability distribution. For example, a normal distribution is used as the probability distribution.

[0035] Figure 4 is a flowchart showing the flow of the learning process. The computer 20 inputs a number of pre-prepared input images Di (images corresponding to the partial images D2 described later) of training objects 9 into the VAE model M (step S11). The VAE model M then encodes and decodes each input image Di and outputs a reconstructed image Do (step S12).

[0036] The learning unit 41 of the computer 20 adjusts the parameters of the VAE model M so that the difference between the input image Di and the reconstructed image Do is small, and the distribution of latent variables in the latent space LS is the predetermined probability distribution described above (step S13). Specifically, the evaluation function representing the difference between the input image Di and the reconstructed image Do is MSE, and the evaluation function representing the amount of deviation of the distribution of latent variables in the latent space LS from the predetermined probability distribution is KLD, and the loss function Loss is expressed by the following equation (1). The learning unit 41 then adjusts the parameters of the VAE model M so that this loss function Loss is minimized. Loss = MSE + β × KLD ... (1)

[0037] In equation (1) above, β is a coefficient representing the weight of the evaluation function KLD in the loss function. By changing the coefficient β, the weighting of the two evaluation functions MSE and KLD can be adjusted. This allows for a more optimized loss function and a further improvement in the accuracy of the VAE model M.

[0038] The learning unit 41 repeats the processing in steps S11 to S13 until a predetermined termination condition is met (step S14). This generates a VAE model M that can accurately output a reconstructed image Do based on the input image Di of the object 9.

[0039] Furthermore, the learning unit 41 may perform the above learning process again in order to update the VAE model M after it has been created.

[0040] <3. Regarding the inspection process> Next, we will explain the inspection process for object 9. Figure 5 is a flowchart showing the flow of the inspection process.

[0041] When inspecting the object 9, first the imaging unit 10 photographs the object 9 (step S21). In this embodiment, the imaging unit 10 photographs a portion of the object 9 including the alignment mark 92 (the search range of the alignment mark 92). This yields an image D1 that includes the alignment mark 92 and a portion of the inspection pattern 91 surrounding the alignment mark 92. The imaging unit 10 inputs the obtained image D1 to the computer 20.

[0042] Next, the partial image extraction unit 42 of the computer 20 extracts a plurality of partial images D2 smaller than the captured image D1 from the captured image D1 (step S22). Step S22 is an example of the "partial image extraction process" in the present invention. The partial images D2 are candidate images that include the alignment marks 92.

[0043] Alignment marks 92 are typically cross-shaped or rectangular and include right-angle corners. Therefore, in this embodiment, the partial image extraction unit 42 first detects corners in the captured image D1. Then, the partial image extraction unit 42 extracts an image of a predetermined size centered on the corner as a partial image D2.

[0044] The partial image extraction unit 42 detects corners in the captured image D1, for example, using the Harris corner detection algorithm. However, the partial image extraction unit 42 may extract a partial image D2 that includes characteristic shapes other than corners, such as circular shapes or wiring patterns. In addition, the partial image extraction unit 42 may use other feature detection algorithms such as SIFT, SURF, or ORB instead of the Harris corner detection algorithm.

[0045] In this embodiment, instead of extracting a partial image D2 from the entire region of the captured image D1, only the partial image D2 containing the corners is extracted. This reduces the number of candidate partial images D2 containing the alignment marks 92. As a result, the number of partial images D2 input to the VAE model M in the next step S23 can be reduced. Consequently, the processing in steps S23 to S26 can be sped up.

[0046] The encoding unit 43 of the computer 20 inputs the extracted partial images D2 into the VAE model M (step S23). The VAE model M then encodes the partial images D2 (step S24). Steps S23 to S24 are an example of the "encoding process" in the present invention. As a result, the multiple partial images D2 are transformed into multiple latent variables in a multidimensional latent space LS. As described above, in the latent space LS, the multiple latent variables are distributed according to a predetermined probability distribution (e.g., a normal distribution). Partial images D2 with similar features become latent variables that are close to each other in the latent space LS after encoding.

[0047] Next, the nearest neighbor distance calculation unit 44 of the computer 20 calculates the distance to the nearest latent variable (hereinafter referred to as "nearest neighbor distance Lm") for each latent variable in the latent space LS (step S25). Figure 6 is a conceptual diagram showing an example of the distribution of latent variables in the latent space LS. In the example in Figure 6, the nearest neighbor distance Lm for one latent variable in the latent space LS (the latent variable enclosed by the dashed line) is shown. The nearest neighbor distance calculation unit 44 calculates such nearest neighbor distance Lm for all latent variables.

[0048] A small nearest neighbor distance Lm for a latent variable means that in the latent space LS, other latent variables exist in the vicinity of that latent variable. This indicates that there is a sub-image D2 with features similar to those of the sub-image D2 corresponding to that latent variable. Conversely, a large nearest neighbor distance Lm for a latent variable means that in the latent space LS, no other latent variables exist in the vicinity of that latent variable. This indicates that there is no sub-image D2 with features similar to those of the sub-image D2 corresponding to that latent variable.

[0049] The image selection unit 45 of the computer 20 selects a partial image D2 from among a plurality of partial images D2 that may contain the alignment mark 92, based on the nearest neighbor distance Lm calculated in step S25 (step S26). Steps S25 to S26 are an example of the "image selection process" in the present invention.

[0050] The image selection unit 45 selects, for example, a predetermined number of partial images D2 corresponding to a predetermined number of latent variables whose nearest neighbor distance Lm is greater than that of other latent variables, as partial images D2 that may contain alignment marks 92. The number of partial images D2 selected may be, for example, five. The image selection unit 45 then displays the selected predetermined number of partial images D2 on the display unit 30. At this time, the predetermined number of partial images D2 may be displayed on the display unit 30 sorted in descending order of nearest neighbor distance Lm.

[0051] The operator selects a partial image D2 containing the alignment mark 92 from a predetermined number of partial images D2 displayed on the display unit 30 and inputs it into the computer 20. This makes it easier for the operator to select the alignment mark 92 than if they had to search for it from scratch.

[0052] The image selection unit 45 may also select only one partial image D2 corresponding to one latent variable whose nearest neighbor distance Lm is greater than that of the other latent variables, as the partial image D2 that may contain the alignment marks 92. In this case, the operator may confirm the selected partial image D2 on the display unit 30 and input to the computer 20 whether the selection of partial image D2 by the image selection unit 45 is correct.

[0053] When alignment marks are detected in two or more areas on the surface of object 9, the computer 20 performs the processing described in steps S21 to S26 for each area. This allows the computer 20 to select a partial image D2 containing the alignment marks 92 in each area.

[0054] As described above, the inspection device 1 inputs multiple partial images D2 into the VAE model M and selects partial images D2 that may contain alignment marks 92 based on the distribution of multiple latent variables in the latent space LS. In this way, the computer 20 can automatically detect partial images D2 containing alignment marks 92. Therefore, the burden on the operator searching for alignment marks 92 can be reduced.

[0055] In particular, in this embodiment, a partial image D2 is selected based on the nearest neighbor distance Lm of each latent variable in the latent space LS. This makes it possible to select a partial image D2 with few similar features. Therefore, alignment marks 92 having different features from the pattern under inspection 91 can be appropriately detected.

[0056] Subsequently, the alignment unit 46 of the computer 20 aligns the object 9 based on the selected partial image D2 (step S27). In step S27, the alignment unit 46 identifies the positions of alignment marks 92 included in the selected partial image D2 through image processing. Then, the alignment unit 46 aligns the object 9 based on the identified positions of the alignment marks 92.

[0057] The positioning of object 9 is performed, for example, by fine-tuning the relative position of object 9 with respect to camera 11. In this process, object 9 may be moved, or camera 11 may be moved.

[0058] Once the alignment is complete, the inspection unit 47 of the computer 20 inspects the object 9 (step S28). Specifically, the camera 11 photographs a predetermined inspection area on the surface of the object 9, which is set based on the alignment marks 92. Then, based on the obtained images, it inspects whether there are any defects in the inspection pattern 91 of the object 9.

[0059] The inspection unit 47 detects defects, for example, by comparing the captured image with a pre-prepared reference image. However, the inspection unit 47 may also input the captured image into the VAE model M described above and detect defects in the pattern under inspection 91 based on the difference between the captured image and the reconstructed image. The inspection unit 47 displays the inspection results on the display unit 30.

[0060] <4. Variation> Although one embodiment of the present invention has been described above, the present invention is not limited to the above embodiment. Below, various modifications will be described, focusing on the differences from the above embodiment.

[0061] <4-1. First variation> In the above embodiment, based on the nearest neighbor distance Lm of the latent variables in the latent space LS of the VAE model M, the partial image D2 that does not have any other partial image D2 with similar features was estimated to be the partial image D2 containing the alignment mark 92. However, with this method, if the object 9 has inherent defects such as probe marks, scratches, or particles, there is a possibility that the partial image D2 containing such defects will be selected in step S25 above.

[0062] To suppress such false detections, a step (step S26A) may be added to remove partial images D2 containing probe marks, scratches, or particles, as shown in Figure 7. In step S26A, for example, a classifier created by supervised machine learning can be used to remove partial images D2 containing probe marks, scratches, or particles from multiple partial images D2.

[0063] The classifier is a pre-trained model that takes a partial image D2 as input and outputs a classification result of the partial image D2 according to the presence or absence of probe marks, scratches, or particles. The classifier can be trained in advance by using a supervised machine learning model with known partial images D2 that have probe marks, scratches, or particles, and known partial images D2 that do not have probe marks, scratches, or particles.

[0064] This method prevents the false detection of a partial image D2 containing probe marks, scratches, or particles as a partial image D2 containing alignment marks 92. Therefore, the detection accuracy of alignment marks 92 can be further improved.

[0065] In the example shown in Figure 7, step S26A is executed after step S26. However, step S26A may also be executed before step S26.

[0066] <4-2. Second variation> In the above embodiment, a partial image D2 containing the alignment mark 92 was selected based on the distribution of latent variables in the multidimensional latent space LS. In such processing, if the number of dimensions of the latent space LS is too large, the processing time will increase. Therefore, the number of dimensions of the latent space LS may be adjusted using the procedure shown below.

[0067] Figure 8 is a flowchart showing the procedure for performing learning processing while adjusting the number of dimensions of the latent space LS. In the example in Figure 8, first, the learning unit 41 performs machine learning with the number of dimensions of the latent space LS as the first number of dimensions (step S31, pre-adjustment learning process). In step S31, the same processing as steps S11 to S14 in Figure 4 described above is performed. This creates a VAE model M. The first number of dimensions is set to a number of dimensions that allows for image reconstruction with sufficient accuracy, for example, 512 dimensions.

[0068] Next, the learning unit 41 changes the number of dimensions of the latent space LS from the first number of dimensions to a second number of dimensions which is less than the first number of dimensions, based on the distribution of latent variables in each dimension of the latent space LS (step S32, dimension adjustment step). As described above, latent variables are represented as vector values ​​having components in each dimension of the latent space LS. In this step S32, the variance of multiple latent variables in the latent space LS is calculated for each component.

[0069] Figure 9 is a graph showing an example of the results of calculating the variance of multiple latent variables component by component. The horizontal axis of Figure 9 represents the components corresponding to each dimension of the latent space LS. The vertical axis of Figure 9 represents the variance value. The graph in Figure 9 is shown sorted in descending order of variance value. In the example in Figure 9, the number of components with a variance value greater than 0 is 64 or less. Therefore, it can be considered that the number of components that effectively capture the features of the partial image D2 is 64 or less.

[0070] In step S32, the number of dimensions of the latent space LS is reduced based on the calculation results of the variance for each component. For example, in the example in Figure 9, the second number of dimensions of the latent space LS should be 64, which includes components that substantially capture the features of the partial image D2 and is less than the first number of dimensions. This optimizes the number of dimensions of the latent space LS. Note that in the processing of step S32, the standard deviation may be used instead of the variance mentioned above.

[0071] Subsequently, the learning unit 41 performs machine learning using the second dimension of the latent space LS (step S33, adjusted learning process). In step S33, the same processing as steps S11 to S14 in Figure 4 described above is performed. This recreates the VAE model M. In this way, a VAE model M with an optimized number of dimensions can be created. Therefore, the processing in steps S23 to S26 described above can be made faster.

[0072] <4-3. Third Variation> Furthermore, in order to improve the accuracy of feature extraction in the VAE model M, the input image Di during training may be processed as follows.

[0073] Figure 10 is a flowchart showing the learning procedure when the input image Di is processed. In the example in Figure 10, first, the learning unit 41 performs machine learning based on the unprocessed input image Di (hereinafter referred to as "first input image Di1") (step S41, first learning process). In step S41, the same processing as steps S11 to S14 in Figure 4 described above is performed. This creates the VAE model M.

[0074] Next, the learning unit 41 calculates the variance or standard deviation of multiple latent variables in the latent space LS for each component in the created VAE model M, similar to step S32 described above. Then, it determines a predetermined number of components with large variances or standard deviations (hereinafter referred to as "effective components") from among the multiple components of the latent variables (step S42, effective component determination step). The effective components may be, for example, the top 10% of components with large variance or standard deviation values.

[0075] Next, the learning unit 41 creates a latent space vector LS (hereinafter referred to as the "effective vector") in which only the effective components have values. The effective vector is created, for example, by multiplying the basis vector of the effective components by a predetermined value of 1 or more. Then, the learning unit 41 generates a corrected image by decoding the effective vector with the decoder unit Md of the VAE model M (step S43, corrected image generation step).

[0076] Next, the learning unit 41 blends the corrected image generated in step S43 with the first input image Di1. This generates a corrected input image Di (hereinafter referred to as "second input image Di2") (step S44, input image correction step). The learning unit 41 blends the corrected image with the first input image Di1 using, for example, the alpha blending method. However, the learning unit 41 may also blend the corrected image with the first input image Di1 using a method other than alpha blending. The generated second input image Di2 is an image in which the characteristic elements of the first input image Di1 are emphasized.

[0077] Subsequently, the learning unit 41 performs machine learning based on the second input image Di2 (step S45, second learning process). In step S45, the same processing as in steps S11 to S14 in Figure 4 described above is performed. This recreates the VAE model M. Thus, in step S45, machine learning is performed based on the second input image Di2, which emphasizes the characteristic elements of the first input image Di1. This makes it possible to create a more accurate VAE model M. Therefore, the detection of alignment marks 92 by steps S23 to S26 described above can be performed with greater accuracy.

[0078] <4-4. Other variations> In the above embodiment, the image selection unit 45 selected a partial image D2 that may contain the alignment marks 92 based on the nearest neighbor distance Lm. However, the image selection unit 45 may select a partial image D2 that may contain the alignment marks 92 in a manner different from the nearest neighbor distance Lm. For example, a partial image D2 that may contain the alignment marks 92 may be selected based on the density of latent variables around each latent variable.

[0079] Furthermore, in the above embodiment, an example was described in which alignment marks 92, which are attached to the surface of the object 9 separately from the wiring pattern, are detected. However, the mark detection method of the present invention may also detect a pattern that can be used as an alignment mark from among a number of patterns such as wiring patterns attached to the object 9.

[0080] Furthermore, in the above embodiment, alignment in the inspection device 1 was described. However, the mark detection method of the present invention may also be used to detect alignment marks in devices other than inspection devices. For example, the mark detection method of the present invention may be used to detect alignment marks from an object in a device that performs exposure or drawing on an object.

[0081] Furthermore, the details of the methods and apparatus described above can be modified or omitted as appropriate, without departing from the spirit of the present invention. In addition, the elements that appear in the above embodiments and modifications can be combined as appropriate, without creating any inconsistencies.

[0082] <5. Inventions from other perspectives> <5-1. First Perspective> Furthermore, if the first objective is set as "detecting partial images that may contain alignment marks while removing partial images that contain probe marks, scratches, or particles," then an invention can be derived from the above embodiments and modifications in which the use of a VAE model is not a mandatory requirement, but rather an "image removal process" is the mandatory requirement.

[0083] The invention can be described, for example, as "a mark detection method for detecting alignment marks from an object, comprising: an image selection step of selecting one or more partial images that may contain alignment marks from a plurality of partial images of the object; and an image removal step of removing partial images having probe marks, scratches, or particles from the plurality of partial images using a classifier created by supervised learning, all of which are performed by a computer."

[0084] According to this invention, it is possible to suppress the selection of partial images having probe marks, scratches, or particles. This improves the accuracy of alignment mark detection. Furthermore, it is possible to combine the elements that appeared in the above embodiments and modifications with this invention.

[0085] <5-2. Second Perspective> Furthermore, if we set "creating a VAE model capable of high-speed processing" as the primary objective, we can extract from the above embodiments and modifications a learning method invention that requires only the "pre-adjustment learning process," the "dimensionality adjustment process," and the "post-adjustment learning process" as essential requirements.

[0086] The invention can be described, for example, as "a method for learning a VAE model, comprising: a pre-adjustment learning step in which machine learning is performed with the latent space of the VAE model having a first number of dimensions; a dimensionality adjustment step, after the pre-adjustment learning step, in which the number of dimensions of the latent space is changed to a second number of dimensions which is less than the first number of dimensions, based on the distribution of the latent variables in each dimension of the latent space; and a post-adjustment learning step in which machine learning is performed with the latent space having the second number of dimensions."

[0087] According to this invention, the number of dimensions of the latent space can be adjusted to an appropriate number of dimensions. This makes it possible to create a VAE model that can be processed at high speed. Furthermore, it is possible to combine the elements that appeared in the above embodiments and modifications with this invention.

[0088] <5-3. Third Perspective> Furthermore, if "creating a highly accurate VAE model" is set as the primary objective, then an invention of a learning method that requires only the "first learning step," "effective component determination step," "corrected image generation step," "input image correction step," and "second learning step" as essential requirements can be extracted from the above embodiments and modifications.

[0089] The invention can be described, for example, as "a method for learning a VAE model, comprising: a first learning step of performing machine learning based on a first input image; an effective component determination step, after the first learning step, of determining a predetermined number of effective components with a large standard deviation or variance from among a plurality of components of latent variables in the latent space; a corrected image generation step of generating a corrected image by decoding an effective vector having values ​​only for the effective components; an input image correction step of generating a second input image by blending the corrected image with the first input image; and a second learning step of performing machine learning based on the second input image."

[0090] According to this invention, a second input image can be generated that emphasizes characteristic elements of the first input image. Then, by performing machine learning based on the second input image, a highly accurate VAE model can be created. Furthermore, it is possible to combine the elements that appeared in the above embodiments and modifications with this invention. [Explanation of Symbols]

[0091] 1. Inspection device 9. Object 10 Imaging Unit 11 Cameras 20 Computers 21 processors 22 memory 23 Memory section 24 Computer Programs 30 Display section 41 Learning Department 42 Partial image extraction section 43 Encoding section 44 Nearest neighbor distance calculation section 45 Image Selection Section 46 Alignment Section 47. Inspection Department 91 Tested Patterns 92 Alignment Marks D1 shooting images D2 Partial Image Di Input Image Reconstructed image Lm nearest neighbor distance M VAE Model Me Encoder Section Md Decoder Section

Claims

1. A mark detection method for detecting alignment marks from an object, The process involves inputting multiple partial images of the object into a pre-trained VAE model and encoding the multiple partial images into multiple latent variables in a multidimensional latent space; An image selection step of selecting one or more partial images that may contain the alignment marks based on the distribution of the plurality of latent variables in the latent space, A mark detection method that is performed by a computer.

2. A mark detection method according to claim 1, The aforementioned image selection step is, a) A step of calculating the distance to the nearest latent variable for each latent variable in the latent space, b) A step of selecting one or a predetermined number of partial images that correspond to one or a predetermined number of latent variables whose distance is greater than that of other latent variables, as one or more partial images that may contain the alignment marks, A mark detection method having the following characteristics.

3. A mark detection method according to claim 1 or claim 2, Prior to the encoding process, A partial image extraction step, which extracts a partial image containing a characteristic shape from an image of the aforementioned object. Further execution, A mark detection method in which, in the encoding step, the partial image extracted in the partial image extraction step is input to the VAE model.

4. A mark detection method according to claim 1 or claim 2, Image removal process: Using a classifier created through supervised learning, removes partial images containing probe marks, scratches, or particles from the plurality of partial images. A mark detection method that further performs this task.

5. A mark detection method according to claim 1 or claim 2, A learning process for creating or updating the VAE model using machine learning. Further execution, A mark detection method comprising the following steps: in the learning step, input learning images of the target object into the VAE model, and adjusting the parameters of the VAE model so that the difference between the input image and the reconstructed image decoded from the latent variables becomes small, and the distribution of the latent variables in the latent space becomes a predetermined probability distribution.

6. A mark detection method according to claim 5, The aforementioned learning process is, A pre-adjustment learning process in which machine learning is performed with the aforementioned latent space as the first dimension, After the pre-adjustment learning step, a dimensionality adjustment step is performed to change the number of dimensions of the latent space to a second number of dimensions which is less than the first number of dimensions, based on the distribution of the latent variables in each dimension of the latent space. A modified learning process in which machine learning is performed with the latent space as the second dimension, A mark detection method having the following characteristics.

7. A mark detection method according to claim 6, A mark detection method comprising the dimensionality adjustment step, which reduces the number of dimensions of the latent space based on the variance or standard deviation calculated for each component of the plurality of latent variables in the latent space.

8. A mark detection method according to claim 5, The aforementioned learning process is, The first learning process involves performing machine learning based on the first input image, After the first learning step, an effective component determination step is performed to determine a predetermined number of effective components with large standard deviations or variances from among the multiple components of the latent variables in the latent space. A correction image generation step, which generates a corrected image by decoding an effective vector having values ​​only for the aforementioned effective component, An input image correction step that generates a second input image by blending the corrected image with the first input image, A second learning process in which machine learning is performed based on the second input image, A mark detection method having the following characteristics.

9. A computer program that causes the computer to execute the mark detection method described in claim 1 or claim 2.

Images