Defect inspection system and defect inspection method

Abstract

The invention provides a defect inspection system and a defect inspection method, which can obtain inspection results with good reproducibility. This defect inspection system (1) is provided with an imaging device (100) for acquiring an observation image of a sample (107), and is provided with: a learning unit (330a) for acquiring a learning image by imaging the sample (107), and learning, using the learning image, an estimation model (401) for estimating a multi-quality product image in which a plurality of quality product images of the sample (107) are estimated for one input image; and a defect inspection unit (330b) that acquires an image to be inspected (1000) by capturing an image of a sample to be inspected by means of an imaging device (100), inputs the image to be inspected (1000) to the learned estimation model (401) of the multi-quality product image, outputs a plurality of estimated quality product images (440) corresponding to the image to be inspected (1000), and extracts a defect site using the plurality of estimated quality product images (440).

Description

Technical Field

[0001] The present invention relates to a defect inspection system and a defect inspection method. Background Art

[0002] As background technology in this technical field, there is Patent Document 1. In Patent Document 1, the following content is recorded: "Causing a computer to perform the following processing: accepting a selection of either a learning mode or an operation mode, acquiring a learning image including an inspection object when the learning mode is selected, accepting the setting of hyperparameters of a first learning model generated by unsupervised learning, generating the first learning model that outputs a reconstructed image corresponding to the learning image when the acquired learning image is input based on the accepted hyperparameters, acquiring an inspection image including an inspection object, accepting the setting of a threshold value for verifying the abnormality of the inspection image, calculating the abnormality of the acquired inspection image and the reconstructed image output by the learned first learning model, and displaying the detected abnormal image based on the calculated abnormality and the accepted threshold value of the abnormality".

[0003] Patent Document 1: Japanese Patent Application Publication No. 2021-140739 Summary of the invention

[0004] However, in Patent Document 1, a model that outputs a reconstructed image corresponding to an input image is generated by unsupervised learning, and an image inspection is performed using the inspection image and the reconstructed image. The deep learning model including VAE (Variational Auto Encoder) described in the embodiment of Patent Document 1 includes random processing in the process of learning internal parameters. Therefore, even when learning is performed using the same image data set, the internal parameters obtained after learning are different for each model. Due to this influence, when the inspection is performed using the learned model, the inspection result may be biased due to the learning result of each model.

[0005] Therefore, the present invention provides a defect inspection system and a defect inspection processing method capable of obtaining inspection results with good reproducibility.

[0006] In order to solve the above-mentioned problem, the defect inspection system of the present invention has a photographing device for obtaining an observation image of a sample, and the defect inspection system comprises: a learning unit, which photographs the sample to obtain a learning image, and uses the learning image to learn an estimation model of multiple qualified product images for estimating multiple qualified product images of the sample from one input image; and a defect inspection unit, which photographs the inspection object sample through the photographing device to obtain an inspected image, inputs the inspected image into the learned estimation model of the multiple qualified product images, outputs multiple estimated qualified product images corresponding to the inspected image, and uses the multiple estimated qualified product images to extract defective parts.

[0007] In addition, the defect inspection method of the defect inspection system of the present invention, the defect inspection system has a photographing device for obtaining an observation image of a sample, and the defect inspection method has the following steps: a learning unit photographs the sample to obtain a learning image, and uses the learning image to learn an estimation model of multiple qualified product images for estimating multiple qualified product images of the sample from one input image; and the defect inspection unit photographs the inspection object sample by the photographing device to obtain an inspected image, inputs the inspected image into the learned estimation model of the multiple qualified product images, outputs multiple estimated qualified product images corresponding to the inspected image, and uses the multiple estimated qualified product images to extract defective parts.

[0008] According to the present invention, it is possible to provide a defect inspection system and a defect inspection processing method capable of obtaining inspection results with good reproducibility.

[0009] The other problems, structures, and effects than those described above will become clear from the following description of the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is a schematic structural diagram of a defect inspection system according to Example 1 of the present invention.

[0011] Figure 2 Yes means Figure 1 A flowchart of the process flow of the defect inspection system is shown.

[0012] Figure 3 This is an example of the structure of the non-defective product image estimation model constituting the defect inspection system of Example 1.

[0013] Figure 4 This is an example of the structure of an estimation model of multiple non-defective product images constituting the defect inspection system of Example 1.

[0014] Figure 5 Yes means Figure 1 FIG. 1 is a flowchart of a learning process of an estimation model of multiple conforming product images by a learning unit shown in FIG.

[0015] Figure 6 Yes means Figure 1 FIG. 1 is a flowchart of a learning process of a conforming product image estimation model performed by a learning unit.

[0016] Figure 7 yes Figure 1 An explanatory diagram of the data expansion process performed by the learning unit shown.

[0017] Figure 8 yes Figure 1 An explanatory diagram of the noise addition process performed by the learning unit shown.

[0018] Fig. 9 Yes means Figure 1 A flowchart of the inspection process flow performed by the defect inspection unit shown.

[0019] Fig.10 yes Figure 1 The figure is an explanatory diagram of the integration of the inspection results of the defect image by the defect inspection section shown.

[0020] Fig.11 yes Figure 1 An explanatory diagram of the false alarm identification processing performed by the defect inspection unit shown.

[0021] Fig. 12A Yes means Figure 1 The diagram of an example of a GUI of a display device shown is a diagram showing the first half of a display screen.

[0022] Fig. 12B Yes means Figure 1 The diagram of an example of a GUI of a display device shown is a diagram showing the latter half of a display screen.

[0023] Fig.13 This is an example of the structure of a multi-head model constituting the defect inspection system of the second embodiment of the present invention.

[0024] Fig.14 Yes means Figure 1 Flowchart of the learning process of the multi-head model performed by the learning unit shown.

[0025] Fig.15 Yes Description Figure 1 Flowchart of the learning process of the multi-head model performed by the learning unit shown.

[0026] Fig.16 This is a flowchart showing the inspection process using a multi-head model of the defect inspection system constituting the third embodiment of the present invention.

[0027] Fig.17 Yes means Figure 1FIG. 1 is a diagram showing an example of defect image inspection using integration processing of estimated non-defective product images performed by the defect inspection section.

[0028] Fig.18 This is a diagram showing an example of learning of a non-defective product image estimation model using maximization of the non-defective product conversion amount performed by a learning unit of the defect inspection system constituting the fourth embodiment of the present invention.

[0029] Fig.19 This is a diagram showing an example of inspection processing using selection of estimated non-defective product images performed by the defect inspection unit of the defect inspection system constituting the fifth embodiment of the present invention.

[0030] Fig. 20 Yes means Figure 1 The diagram of an example of a GUI of the display device shown is a diagram showing a display screen of the GUI for selecting an estimated non-defective product image. DETAILED DESCRIPTION

[0031] In this specification, the imaging device includes a charged particle beam device, an optical inspection device, and an imaging device using a camera, and the charged particle beam device includes an SEM, a review SEM, and a FIB, etc. In the following, an SEM is used as an example of a charged particle beam device.

[0032] The following describes embodiments of the present invention using the accompanying drawings. The embodiments described below do not limit the invention to the scope of the claimed patent protection, and the elements and their combinations described in the embodiments are not necessarily all essential to the solution means of the invention.

[0033] [Example 1]

[0034] Figure 1 FIG. 1 is a schematic diagram of the defect inspection system of Example 1 of the present invention. Figure 1 As shown, the defect inspection system 1 of the present embodiment is configured to include a charged particle beam device 2 and a computer 3. In the present embodiment, the following situation is taken as an example for explanation: the defect inspection system 1 estimates a plurality of corresponding qualified product images based on the inspected image of the sample projected in the charged particle beam device 2, and extracts defective parts through inspection processing using statistical processing.

[0035] <Structure of defect inspection system>

[0036] like Figure 1As shown, the defect inspection system 1 has a charged particle beam device 2 and a computer 3. The charged particle beam device 1 has a photographing device 100 and a controller 200. The photographing device 100 has components such as a workbench as a sample table and a driving circuit in a shell, in other words, a column, and a sample chamber. The controller 200 is a control system that drives and controls the photographing device 100 and can be installed through a computer or a circuit. In other words, the computer 3 is a computer system. The defect inspection system 1 has structural elements required to generate a signal waveform and an image based on a detection signal of the photographing device 100 constituting the charged particle beam device 2. An example of the charged particle beam device 2 is a scanning electron microscope (SEM: Scanning Electron Microscope).

[0037] The imaging device 100 of the charged particle beam device 2 outputs the obtained detection signal s1 according to the irradiation of the sample 107 on the workbench 108 by the charged particle beam 103. The controller 200 inputs or receives the detection signal s1, processes it, generates an image or other signal as the detection signal of the charged particle beam device 2, and stores it. The controller 200 outputs the signal s2 of the image or other signal. The computer (computer system) 3 inputs or receives the signal s2 of the image or other signal, and processes it.

[0038] In the photographing device 100 of the charged particle beam device 2, the charged particle beam 103 extracted from the electron source 101 by the extraction electrode 102 is accelerated by the acceleration electrode not shown. The accelerated charged particle beam 103 is concentrated by the focusing lens 104 as a mode of focusing lens. The charged particle beam 103 is deflected by the scanning deflector 105, thereby scanning one-dimensionally or two-dimensionally on the surface of the sample 107 on the workbench 108 as the sample stand. The charged particle beam 103 is decelerated by the negative voltage (recovery voltage) applied to the electrode built into the workbench 108 as the sample stand, and is converged by the lens action of the objective lens 106 and irradiated onto the sample 107. The workbench 108 can be a mechanism that can move in the Z direction perpendicular to the X direction and the Y direction corresponding to the vertical direction, or a mechanism that can rotate or tilt in each axial direction. In addition, the details of the installation of the charged particle beam device 2 are not limited, but it can also be a structure that irradiates multiple beams. In addition, the charged particle beam device 2 is shown to have a structure with one detector 110, but it is not limited to this, and it can also be a structure with multiple detectors. For example, it can also be a structure with an SE detector for detecting SE (Secondary Electron) and a BSE detector for detecting BSE (Backscattered Electron). In addition, for example, it can also be a structure in which multiple detectors are arranged at multiple positions. That is, as a structure for capturing images, it can also be a structure with multiple channels and multiple detection systems. In addition, when generating an image, it is also possible to accumulate multiple images based on multiple signals repeatedly detected from the same area, so as to generate one image.

[0039] The sample 107 is, for example, a semiconductor wafer. When the charged particle beam 103 irradiates the sample 107, electrons 109 such as secondary electrons (SE) and backscattered electrons (BSE) are released from the inside of the sample 107. The released electrons 109 are accelerated by the acceleration effect based on the negative voltage (recovery voltage) applied to the sample 107 and captured by the detector 110. The detection signal s1 output from the detector 110 is sent to the controller 200. The controller 200 receives the detection signal s1 via the communication device 201.

[0040] The controller 200 includes a communication device 201, a processor 202, a memory 203 such as a RAM or a non-volatile storage device, a display device 204, and an input / output device 205. These components are connected to each other through a bus or other architecture. The controller 200 controls the shooting of the shooting device 100 according to the set shooting process. The processor 202 generates an observation image (inspected image) based on the detection signal s1 obtained via the communication device 201, as an image in which the amount of electrons captured by the detector 110 becomes brightness. The processor 202 stores the generated observation image (inspected image) and other data in the memory 203. The processor 202 sends a signal s2 corresponding to the generated observation image (inspected image) and other data to the computer 3 via the communication device 201. The computer 3 receives the signal s2 through the communication device 310. In addition, the user U1 can also operate the display device 204 and the input / output device 205 to use the controller 200.

[0041] The communication device 201 is a device equipped with a communication interface with the camera 100 and a communication interface with the computer 3. The communication interface may be, for example, a LAN, but is not limited thereto. The input / output device 205 is an input device and an output device. The input device and the output device may be built-in or externally connected. Examples of input devices are a keyboard, a mouse, or a microphone. Examples of output devices are a display, a printer, or a speaker.

[0042] The computer 3 is provided with a communication device 310, a storage device 320, a learning unit 330a, a defect inspection unit 330b, a memory 331 such as a RAM or a non-volatile storage device, a display device 332, and an input / output device 333. These components are interconnected through a bus or other architecture. The storage device 320 is a memory having a relatively large storage area. The storage device 320 stores a learning image DB 321, an inspected image DB 322, a learned parameter DB 323, and a program 324. The memory 331 of the computer 3 also stores necessary management information and a database (DB). In addition, an external storage device, a server, etc. may be connected to the computer 3, necessary data and information may be stored in the external storage device, the server, etc., and the data and information may be read and written appropriately. In addition, the learning unit 330a and the defect inspection unit 330b are implemented by, for example, a processor such as a CPU not shown, a ROM storing various programs, a RAM temporarily storing data of the operation process, an external storage device, and other storage devices, and the processor such as the CPU reads out and executes the various programs stored in the ROM, and stores the operation results as the execution results in a cloud storage device via the RAM, an external storage device or a network connection. Alternatively, it can also be configured to store the above-mentioned various programs in the program 324 in the storage device 320. In addition, it can also be configured to appropriately store various data and information processed by the learning unit 330a and the defect inspection unit 330b in the memory 331. The memory 331 stores, for example, the execution module of the program, image data, processing result information, etc. In addition to the DB shown in the figure, the storage device 320 can also store processing result information, historical information, screen data, etc.

[0043] The communication device 310 is a device equipped with a communication interface with the controller 200. External devices and the like can also be connected to the computer (computer system) 3 via a network such as a LAN. The input / output device 333 is an input device or an output device. The input device and the output device can be built-in or externally connected. Examples of input devices are keyboards, mice, microphones, etc. Examples of output devices are displays, printers, speakers, etc.

[0044] The user U1 of the defect inspection system 1 operates the display device 332 and the input / output device 333, etc., and uses the computer (computer system) 3. Thus, the user U1 uses the defect inspection system 1. On the display screen of the display device 332, a screen that serves as a user interface of the defect inspection system 1, in other words, a screen accompanied by a graphical user interface (GUI) is displayed. In addition, the computer 3 may also be composed of a client server system. In this case, the user U1 operates a PC or the like that serves as a client terminal, and the PC or the like accesses the computer (computer system) 3 that serves as a server through communication.

[0045] The computer (computer system) 3 is a part that performs the characteristic processing in this embodiment. The computer (computer system) 3 is connected to the controller 200. In addition, the controller 200 can be configured as a part of the computer (computer system) 3, and the computer 3 can also be configured as a part of the controller 200. The controller 200 and the computer (computer system) 3 can also be composed of an integrated computer system. The controller 200 can perform at least a part of the characteristic processing in this embodiment. The entirety including the computer (computer system) 3 can also be used as the charged particle beam device 2.

[0046] In addition, regarding the computer 3 (computer system), when it is set in the form of a client / server system, for example, the following actions are performed. User U1 accesses the computer (computer system) 3 as a server from a client terminal such as a PC via a network. The server provides a screen with a graphical user interface (GUI) to the client terminal. The server sends the GUI screen data used for this (for example, it can also be a web page) to the client terminal. The client terminal displays the GUI screen on the display based on the received screen data. User U1 views the GUI screen and inputs instructions, settings, etc. The client terminal sends the input information to the server. The server performs processing corresponding to the received input information. For example, the server performs evaluation of the captured image and estimation processing of the device status, stores the processing results, and sends the GUI screen data used to display the processing results (it can also be only update information) to the client terminal. The client terminal updates the display of the GUI screen based on the received screen data. User U1 can observe the GUI screen and confirm the processing results, such as the estimation results of the device status.

[0047] <Processing flow of defect inspection system>

[0048] Figure 2 Yes means Figure 1 FIG. 1 is a flowchart of the processing flow of the defect inspection system 1 shown in FIG. Figure 2 As shown, in step S1, the electron beam (charged particle beam) 103 is scanned in the observation area of ​​the sample 107 and the detection signal s1 captured by the detector 110 is sent to the controller 200 via the communication device 201. The observation image generated by the processor 202 based on the sent detection signal s1 is sent to the computer (computer system) 3 via the communication device 310. The learning unit 330a stores the sent observation image in the learning image DB 321.

[0049] In step S2 , the learning unit 330 a determines internal parameters of a multi-conforming product image estimation model for generating a plurality of conforming product images corresponding to an input image by learning using the learning image DB 321 , and stores the determined internal parameters in the learned parameter DB 323 .

[0050] In step S3, the detection signal s1 captured by the detector 110 by scanning the electron beam (charged particle beam) 103 in the defect inspection area on the sample 107 is sent to the controller 200 via the communication device 201. The observation image generated by the processor 202 based on the sent detection signal s1 is sent to the computer 300 via the communication device 310. The defect inspection unit 330b stores the signal s2 corresponding to the data such as the sent observation image (inspected image) in the inspected image DB 322.

[0051] In step S4, the defect inspection unit 330b inputs the image to be inspected among the images stored in the inspected image DB 322 as an input image to the estimation model 401 (in the case of the image of the multiple qualified products) whose internal parameters have been read from the learned parameter DB 323. Figure 4 The inspection result is outputted using the input image and the outputted multiple qualified product images, and the processing is terminated.

[0052] <Example of Inspection Using Estimation Model of Multiple Conformable Product Images>

[0053] Below, in Figures 3 to 12A as well as Fig. 12B In the following, a specific example of defect inspection processing is described assuming that defects on a semiconductor wafer serving as a sample 107 are inspected in the defect inspection system 1 of the present embodiment.

[0054] Figure 3 FIG. 1 is an example of a structure of a good product image estimation model constituting the defect inspection system 1 of this embodiment. Figure 3 As shown, the qualified product image estimation model 400 is constructed based on a machine learning model, such as an autoencoder, U-Net, GAN (Generative Adversairal Networks), etc. In the initial stage of the operation of the defect inspection system 1, when the amount of defect images used for model learning is insufficient, these models are made to learn in a manner of estimating the qualified product image corresponding to the input image 430 using only the qualified product image as learning data, thereby enabling defect inspection based on the comparison between the input image 430 and the estimated qualified product image (estimated qualified product image 440). As an example of a learning method, the internal parameters are repeatedly updated in a manner that minimizes the error between the input image 430 and the output image of the model, thereby learning the feature expression of the qualified product image and the reconstruction of the qualified product image from the partial space constructed by the feature expression.

[0055] In this embodiment, a qualified product image estimation model 400 based on U-net is used as an example for explanation. U-net is composed of a feature extraction unit 410 in the first half and an image generation unit 420 in the second half. The feature extraction unit 410 is configured to repeatedly perform processing based on a convolution layer for extracting image features and a pooling layer for reducing the size of the output vector of the convolution layer to obtain local feature quantities and global feature quantities of the input image 430. In the image generation unit 420 in the second half, the feature quantity obtained by the feature extraction unit 410 in the first half is repeatedly processed based on a convolution layer and an upsampling layer for increasing the size of the output vector of the convolution layer to obtain an estimated qualified product image 440 of the same size as the input image 430. In addition, a jump connection that combines the output of the feature extraction unit 410 with the feature quantity obtained by upsampling can be used to obtain a clearer output image. By learning the qualified product image estimation model 400 using the qualified product image as learning data, internal parameters that can generate the qualified product image based on the feature expression of the qualified product image and the extracted feature quantity are learned.

[0056] Figure 4 This is an example of the structure of the estimation model 401 of multiple qualified product images constituting the defect inspection system 1 of this embodiment. The estimation model 401 of multiple qualified product images is composed of multiple qualified product image estimation models 400, and by inputting one input image 430, the estimated qualified product images 440 of the same number as the number of the constituted qualified product image estimation models 400 are output. The structures of the respective qualified product image estimation models 400 are the same, but the internal parameters determined by the learning process including randomness are different, so different estimated qualified product images 440 are output even for the same input image. The qualified product image estimation models 400 constituting the estimation model 401 of multiple qualified product images only need to be at least two or more.

[0057] Figure 5 It means by Figure 1 Flowchart of the learning process of the estimation model 401 for multiple qualified product images by the learning unit 330a shown in FIG. The case of learning the estimation model 401 for multiple qualified product images including the structure of n qualified product image estimation models 400 is described as an example. First, in step S201, the learning unit 330a reads images used for learning the estimation model 401 for multiple qualified product images from the learning image DB 321 to create a learning data set.

[0058] Afterwards, in step S202, the learning unit 330a learns a plurality of (i=n, 1, i+1) qualified product image estimation models 400 respectively in order to generate estimation models 401 for multiple qualified product images. Before the learning of each qualified product image estimation model 400 begins, a non-repeated random initial seed is set for each qualified product image estimation model 400 (step S203). As an example of a method for determining the initial seed, it can be determined based on the calling time of the function of setting the initial seed of the program. Next, in step S204, the learning unit 330a sets the initial values ​​of the internal parameters of the qualified product image estimation model 400 (i). Since the initial value is determined by referring to the random table, different values ​​are set when the initial seeds are different (step S205).

[0059] Next, in step S206, the learning unit 330a performs a learning process for determining internal parameters of the qualified product image estimation model 400 (i) with the initial value set using the learning data set, thereby producing a learned qualified product image estimation model 400 (i) (step S207). By repeatedly performing the process from the setting of the initial seed to the learning process at least twice (step S208), an estimation model 401 for learned multiple qualified product images is produced (step S209). After producing an arbitrary number of estimation models 401 for learned multiple qualified product images, all internal parameters are used as internal parameters of one estimation model 401 for multiple qualified product images and stored in the learned parameter DB 323.

[0060] Figure 6 Yes means Figure 1 Flowchart of the learning process of the qualified product image estimation model of the learning unit shown in FIG. Figure 6As shown, in step S301, learning loop 1 (i=n, 1, i+1) is started. First, in step S302, the learning unit 330a performs learning mini-batch segmentation of the learning data set. Learning mini-batch segmentation divides the images contained in the learning data set into an arbitrary number of images and sets them as learning mini-batches respectively. Data expansion (step S305), noise addition (step S306), learning loss calculation (step S307) and internal parameter update (step S308) are performed on each divided learning mini-batch. In the data expansion process (step S305), the learning unit 330a processes brightness conversion, contrast conversion, and deformation addition for each image contained in the learning mini-batch with a random intensity within a constant range based on the reference value of the random number table, thereby producing an input image for the qualified product image estimation model 400. In order to make the qualified product image estimation model 400 learn the estimation process from the unqualified product image to the qualified product image, noise is added to the input image as a unqualified product feature to produce an unqualified product image. By inputting the produced non-conforming product image into the conforming product image estimation model 400, the estimated conforming product image 440 is obtained as an output. At this time, as a learning loss, the average value of the square error of the brightness of each pixel of the estimated conforming product image and the image before adding noise, that is, the input image 430, is calculated. In the learning loss, SSIM (Structural Similarity) as an index value related to image quality, a regularization term that imposes a penalty on internal parameters to prevent the subject matching of the learning data set, etc. can also be added. Based on the calculated learning loss, an optimization method such as a gradient descent method is used to search for the value of the internal parameter that reduces the learning loss, and the internal parameters of the conforming product image estimation model 400 are updated with the search results. After the processing from data expansion processing (step S305) to internal parameter update (step S308) is repeated the same number of times as the number of learning mini-batches, the learning mini-batch division of the learning data set is performed again, and the internal parameters are updated for each learning mini-batch. By repeating this learning cycle any number of times, the internal parameters of the conforming product image estimation model are determined.

[0061] Figure 7 Is Figure 1 FIG. 3 is an explanatory diagram of the data expansion process performed by the learning unit 330a shown in FIG. Figure 7As shown, in the data expansion process 700, the learning unit 330a processes contrast conversion 701, brightness conversion 702 and deformation addition 703 for each learning image included in the learning mini-batch. The input image 430 is thus produced. In contrast conversion 701, the contrast of the entire image to be processed is randomly changed within a set range. For example, a value obtained by multiplying a random value selected from the range of 0.8 to 1.2 by the brightness value of each pixel is updated to randomly change the contrast. In the case of less than 1.0, the contrast becomes lower, and in the case of greater than 1.0, the contrast becomes larger. In brightness conversion process 702, a randomly determined brightness value is added to each pixel of the image, or a random value is determined for each pixel and added to the brightness value. The added value uses a value randomly selected from a set range. The range is, for example, ±10% of the average value of the pixel value of the entire image to be processed. In deformation addition 703, deformation processing is used to randomly deform the image to be processed. In order to deform the image, for example, it can be achieved by applying affine transformation in the longitudinal and transverse directions of the image. By learning images with brightness changes, contrast and image deformation within a constant range as qualified products, the robustness of the inspection process for changes in image quality such as brightness and contrast caused by deformation caused by the shooting device 100 and changes in the state of the shooting device 100 during operation can be improved.

[0062] Figure 8 yes Figure 1: is an explanatory diagram of the noise addition process 800 of the learning unit 330a shown. In the noise addition process 800, the learning unit 330a takes the input image 430 generated by the data expansion process as the object, and adds noise as the image feature of the non-conforming product. The noise addition process 800 is performed probabilistically using random numbers. For example, a random value is obtained from the range of 0 to 1 as the random value, and the noise addition process (802) is performed only when the random value is less than the set value (801). When the set value is 0.8, the noise addition process 802 is performed with a probability of 80%. By performing the noise addition process 800 probabilistically, images without noise addition and images with noise addition are included in the learning mini-batch, and it is possible to perform learning to keep the qualified product image as the qualified product image and convert the non-conforming product image into the qualified product. The generation method of the noise applied as the image feature of the non-conforming product includes, for example, Perlin noise and wavelet noise. Regardless of the noise generation method, an image of a random situation can be generated, and a value obtained by multiplying the generated image by a random coefficient in a constant range is added to the qualified product image to produce a noise image 803 having the characteristics of a non-qualified product. In addition to the above-mentioned noise generation method, as a feature of a non-qualified product, an image obtained by cutting out a part of another image in the image or the learning data set can be pasted to the input image 430 or a line with a random position, angle, length, thickness, brightness, etc. can be drawn.

[0063] Fig. 9 Yes means Figure 1 Flow chart of the inspection process flow of the defect inspection unit 330b shown in FIG. Fig. 9 As shown, first, in step S401, the inspection area of ​​the sample 107 to be inspected is scanned with the charged particle beam 103, and the obtained inspected image is stored in the inspected image DB322. Next, in step S402, the defect inspection unit 330b reads the image to be inspected from the inspected image DB322. Thereafter, in step S403, the defect inspection unit 330b reads the internal parameters of the estimation model 401 of multiple qualified product images from the learned parameter DB323. In step S404, the defect inspection unit 330b inputs the inspected image 1000 ( Fig.10 ), and output multiple images of estimated qualified products 440 ( Fig.10). When an inspected image 1000 containing defects is input into the estimation model 401 of multiple qualified product images, an image in which the defective part is converted like a qualified product is output. Therefore, in the comparison process (step S405), the defect inspection unit 330b compares the inspected image 1000 with each estimated qualified product image 440 in order to extract the defective part, thereby producing a differential image in which each pixel represents the difference in pixel value between the inspected image 1000 and the estimated qualified product image 440. At this time, multiple estimated qualified product images 440 are output for one inspected image 1000, so the same number of differential images 1002 as the number of estimated qualified product images 440 are produced ( Fig.10 In the false alarm identification process (step S406), the defect inspection unit 330b excludes the false alarms that are generated in the difference but are not actually defects based on the image features of the differential image 1002, and outputs a defect map 1004 ( Fig.10 The number of defect maps 1004 outputted is the same as the number of estimated qualified product images 440. In the integration process (step S407), the defect inspection unit 330b performs statistical processing on the plurality of defect maps 1004, thereby outputting an integrated defect map 1006 ( Fig.10 ) as the inspection result (step S408).

[0064] Fig.10 yes Figure 1 FIG. 3 is an explanatory diagram of the integration of the inspection results of the defect image by the defect inspection unit 330b shown in FIG. Fig.10As shown, the estimation model 401 of multiple qualified product images includes three qualified product image estimation models not shown in the figure, and the case where three estimated qualified product images 440 are output for one inspected image 1000 is described as an example. The inspected image 1000 is an example of an image of a circuit pattern of a semiconductor projected using a charged particle beam device 2. The defect inspection unit 330b obtains three estimated qualified product images 440 by inputting the inspected image 1000 into the estimation model 401 of multiple qualified product images whose internal parameters have been read from the learned parameter DB 323. The internal parameters of the three qualified product image estimation models 400 (not shown) are different, so different estimated qualified product images 440 are output even when the same inspected image 1000 is input. Next, the differential image 1002 is produced by comparing the three estimated qualified product images 440 with the inspected image 1000 respectively. The area where the difference is generated in the comparison process 1001 is the area where the inspected image 1000 is converted into a different shape, and therefore is an area with a high probability of being a defect. Next, by performing a false alarm identification process 1003 on the difference image 1002, the area where the difference is generated but is not actually a defect is excluded, and a defect map 1004 representing the defective part is produced. Next, as a statistical process, an integrated defect map 1006 representing the central value of the pixel value at the same position of the three defect maps 1004 is produced for the three defect maps 1004. By using the central value of a plurality of defect maps 1004 (central value process 1005), it is possible to exclude false alarms generated when the estimation model 401 of multiple qualified product images includes a qualified product image estimation model 400 in which the estimation error of a specific qualified product pattern is large due to the influence of the randomness of learning. In addition, by using the defective part extracted from the plurality of defect maps 1004 as the final inspection result, it is possible to suppress the deviation of the inspection that can be extracted in a certain model but cannot be extracted in another model, and the reproducibility of the inspection result can be improved. The statistical processing when integrating the defect map 1004 may also be an average value, a mode value, or a maximum value.

[0065] Fig.11 Is Figure 1 FIG. 1 is an explanatory diagram of the false alarm identification process performed by the defect inspection unit 330b shown in FIG. The object of the false alarm identification process is the inspected image 1000 ( Fig.10 ) and the estimated qualified product image 440 ( Fig.10 ) comparison processing 1001 ( Fig.10 ) produced in the differential image 1002 ( Fig.10 ).like Fig.11As shown, first, the defect inspection unit 330b performs a defect candidate grouping process 1101 on the difference image 1002. The pixel value of the defect part in the difference image 1002 becomes locally high, so in the grouping process 1101, a defect candidate map 1102 is created in which the area with a certain pixel value or more is set to 1 and the area other than this is set to 0. Using a labeling process that assigns the same identification number to the connected pixels, the connected area is set as 1 defect candidate and the corresponding defect candidate id is assigned. Next, the defect inspection unit 330b uses the difference image 1002 and the defect candidate map 1102 to calculate the feature quantity 1103 of each defect candidate. The feature quantity is, for example, the center of gravity coordinates, the area on the defect candidate map 1102, and the maximum value, average value, and variance of the difference value of the brightness. The feature quantity of the defect candidate can also use the minimum value, circularity, edge strength, etc. Next, the defect inspection unit 330b determines whether each defect candidate is a false alarm or a real alarm through a false alarm determination process 1105. The conditions for determining a false alarm are, for example, that the area is below a threshold, that the maximum value of the brightness difference value is below a threshold, that the average value of the brightness difference is below a threshold, and that the variance of the brightness difference is below a certain value. Here, the variance and maximum value of the brightness are values ​​for each pixel, and the average value is the average value of the brightness of all pixels in the area. In addition, a combination of multiple conditions is also possible. An image in which the pixel values ​​of the defect candidates on the defect candidate map corresponding to the defect candidate id determined to be a false alarm are changed to 0 is output as a defect map 1004.

[0066] Fig. 12A and Fig. 12B They are respectively Figure 1 The figure is a diagram showing an example of a GUI of a display device shown in FIG. Figure 1 The figure of an example of a GUI of a display device shown in FIG. 1 is a figure showing the second half of a display screen. Fig. 12AAs shown, the GUI of this embodiment has an input column 1210 for learning parameters of the estimation model 401 for multiple qualified product images and a display column 1220 for the learning status. In the input column 1201 for learning parameters, as learning parameters 1211, input columns for parameters related to data expansion processing 1212, noise addition 1213, number of learning times 1214, number of image generation units 1215, and learning images 1216 are included. In the input column for data expansion processing 1212, a check box is used to select the expansion processing to be performed during learning. In the input column for noise addition 1213, a value of 0.0 to 1.0 is input as the probability of performing noise addition during learning. In the input column for number of image generation units 1215, the number of estimated qualified product images output by the estimation model 401 for multiple qualified product images for one input image is input. According to the number of image generation units 1215, it is determined how many qualified product image estimation models 400 constitute the estimation model 401 for multiple qualified product images. In the input column of the learning image 1216, the file path of the image used for learning the estimation model 401 for multiple qualified product images is input. When learning, the image corresponding to the specified file path is used. After inputting each parameter, by pressing (clicking with a mouse, etc.) the learning start button 1217, the learning of the estimation model 401 for multiple qualified product images is performed. In the display column 1220 of the learning status, there are a display column for the learning loss 1222 of the estimation model 401 for multiple qualified product images and a display column for the estimated qualified product image 1223 that can confirm the learning loss for each image generation unit 1221. The learning loss of the internal parameters of the estimation model 401 for multiple qualified product images at each learning number is displayed in the display column of the learning loss 1222. The display column of the estimated qualified product image 1223 has an input column for the number of learning times. After inputting any number of learning times in the input column of learning times, press the display button (click with a mouse, etc.) to display the learning images of the corresponding learning times of the internal parameters of the estimation model 401 of multiple qualified product images and the corresponding estimated qualified product images 440.

[0067] like Fig. 12BAs shown, the GUI of this embodiment has an input column 1230 for inspection parameters of the estimation model 401 for multiple qualified product images and a display column 1240 for inspection results. The input column 1230 for inspection parameters includes an input column 1232 for integration of estimated qualified products, an input column 1233 for learned parameters of the estimation model 401 for multiple qualified product images, and an input column 1234 for images to be inspected. In the input column 1232 for integration of estimated qualified products, a check box is used to select a statistical processing method for integration processing. In the input column 1233 for learned parameters, the path of a file storing the internal parameters of the estimation model 401 for multiple qualified product images is input. In the input column 1234 for images to be inspected, the file path of the images to be inspected as the inspection object is input. The display column 1240 for inspection results has an image display column 1241 and a defect distribution display column 1242. In the image display column 1241, the images to be inspected, the defect map, and each estimated qualified product image are displayed as the inspection object. The defect distribution display field 1242 displays a wafer map 1243 showing the defect density of each region on the semiconductor wafer as the sample 107, and the number of defects and the defect density (color bar display) in each region. The region is, for example, a semiconductor chip manufactured on the semiconductor wafer.

[0068] As described above, according to the present embodiment, it is possible to provide a defect inspection system and a defect inspection processing method capable of obtaining inspection results with good reproducibility.

[0069] [Example 2]

[0070] use Figures 13 to 15 , Example 2 is described. The basic structure of Example 2 is the same as / shared with Example 1. Hereinafter, the structural parts of Example 2 that are different from Example 1 are mainly described. In this embodiment, the estimation model 401 of the qualified product image is not composed of multiple qualified product image estimation models, but is composed of a shared feature extraction unit 410 and multiple image generation units. Hereinafter, a specific example is described.

[0071] Fig.13 FIG. 1 is an example of a structure of a multi-head model constituting a defect inspection system of Example 2. Fig.13As shown, the multi-head model is an example of a qualified product image estimation model 400a, which is composed of a feature extraction unit 410 in the front half and multiple image generation units in the back half. Specifically, the multiple image generation units in the back half have a first image generation unit 420a and a second image generation unit 420b. The feature extraction unit 410 is a structure that repeatedly performs the processing of a convolution layer for extracting image features and a pooling layer for reducing the size of the output vector of the convolution layer, and inputs the extracted feature quantity to each image generation unit in the back half (the first image generation unit 420a and the second image generation unit 420b). The first image generation unit 420a and the second image generation unit 420b obtain an output of the same size as the input image by repeatedly performing the processing of a convolution layer and an upsampling layer for increasing the size of the output vector of the convolution layer on the feature quantity obtained from the feature extraction unit 410. Through this model structure, when one input image 430 is input to the multi-head model, the same number of estimated qualified product images as the image generation unit are output. Specifically, the first image generating unit 420a outputs the first estimated conforming product image 1301, and the second image generating unit 420b outputs the second estimated conforming product image 1302. In this way, the input image multi-head model is composed of at least two image generating units.

[0072] Fig.14 Yes means Figure 1 Flowchart of the learning process of the multi-head model performed by the learning unit shown in FIG. The following is a description of the learning process of the multi-head model including n image generation units. Fig.14 As shown, first, the learning unit 330a reads the image used in the learning of the multi-head model from the learning image DB 321 to create a learning data set (step S501). Next, the learning unit 330a randomly sets an initial seed (step S502). Next, in step S503, the learning unit 330a randomly initializes the internal parameters of the multi-head model. The learning unit 330a saves all the internal parameters of the learned multi-head model obtained by executing the learning process of the initialized multi-head model (steps S504 and S505) in the learned parameter DB 323 (step S506), and ends the process.

[0073] Fig.15 Yes Description Figure 1 The flowchart of the learning process of the multi-head model performed by the learning unit 330a shown in FIG. First, the learning unit 330a divides the learning data set into learning small batches (step S603). Data expansion (step S606), noise addition (step S607), and learning loss calculation (step S608) are performed for each learning small batch after division. Fig.15In the flowchart of , when i of learning loop 1 is 1, that is, when it is the first processing that complies with learning loop 1 (step S609), the internal parameter update of the feature extraction unit 410, the first image generation unit 420a and the second image generation unit 420bi is performed. In the case where i is not 1 (step S609), the internal parameters of the first image generation unit 420a and the second image generation unit 420bi are updated. In learning loop 3, after the internal parameters are updated using all of 1 learning mini-batch, the learning mini-batch segmentation of the learning data set is performed again, and learning loop 3 is executed (step S604, step S612). In learning loop 2, learning loop 3 is executed m times (step S602, step S613). m can be any number greater than 1. In learning loop 1, learning loop 2 is executed the same number of times as the number of image generation units, that is, n (step S601, step S614).

[0074] As described above, according to this embodiment, in addition to the effects of embodiment 1, for the image feature extraction unit shared by multiple head models, by having multiple image generation units, the number of model parameters can be reduced, and the processing time and resource consumption of learning and inspection can be reduced.

[0075] [Example 3]

[0076] use Fig.16 and Fig.17 Embodiment 3 is described. The basic structure of embodiment 3 is the same as / shared with embodiment 1. Hereinafter, the structure of embodiment 3 that is different from embodiment 1 is mainly described. In embodiment 3, in the inspection process, the defective part is extracted by processing the integrated process into an estimated qualified product image instead of processing the integrated process into a defect map. A specific example is described below.

[0077] Fig.16 1 is a flowchart showing the inspection process of the multi-head model of the defect inspection system constituting the third embodiment of the present invention. First, the inspection area of ​​the sample 107 to be inspected is scanned with the charged particle beam 103, and the obtained inspection image is stored in the inspection image DB 322 ( Figure 1 ) (step S701). Next, in step S702, the defect inspection unit 330b obtains the defect inspection image from the inspection image DB 322 ( Figure 1 ) reads the image to be inspected. Then, in step S703, the defect inspection unit 330b reads the image to be inspected from the learned parameter DB 323 ( Figure 1) reads the internal parameters of the estimation model 401 of multiple qualified product images. In step S704, the defect inspection unit 330b inputs the inspected image to the estimation model 401 of multiple qualified product images that has read the internal parameters, and outputs multiple estimated qualified product images. In step S705, the defect inspection unit 330b performs statistical processing on the multiple estimated qualified product images 440 to produce one integrated estimated qualified product image. In step S706, in order to extract the defective part, the defect inspection unit 330b generates a differential image by performing a comparison process on the inspected image and the integrated estimated qualified product image. In step S707, the defect inspection unit 330b performs a false alarm identification process on the differential image and outputs a defect map representing the defective part (step S708).

[0078] Fig.17 It means by Figure 1 FIG. 1 is a diagram showing an example of defect image inspection using integration processing of estimated conforming product images performed by the defect inspection unit 330b shown in FIG. The case where the estimation model 401 for multiple conforming product images includes three conforming product image estimation models 400 (not shown) and outputs three estimated conforming product images 440 for one inspected image 1000 is described as an example. The defect inspection unit 330b inputs the inspected image 1000 to the estimation model 401 for multiple conforming product images that has read the internal parameters from the learned parameter DB 323, thereby obtaining three estimated conforming product images 440. The internal parameters of the three conforming product image estimation models 400 (not shown) are different, so three different estimated conforming product images 440 are output. Next, as a statistical process, an integrated estimated conforming product image 1702 representing the central value (central value process 1701) of the pixel values ​​at the same position of the three estimated conforming product images 440 is produced for the three estimated conforming product images 440. Next, by comparing the inspected image 1000 and the integrated estimated conforming product image 1702 (1703), a difference image 1002 is created. By performing false positive identification processing 1704 on the difference image 1002, a defect map 1004 indicating defective parts is output. The statistical processing when integrating the estimated conforming product image may be an average value, a mode value, or a maximum value.

[0079] As described above, according to the present embodiment, in addition to the effects of the first embodiment, by using the integrated image to perform comparison processing and false alarm identification processing, it is possible to perform false alarm identification processing using reproducible results, thereby achieving higher accuracy.

[0080] [Example 4]

[0081] use Fig.18Embodiment 4 is described. The basic structure of Embodiment 4 is the same as / shared with that of Embodiment 1. Hereinafter, the structural parts of Embodiment 4 that are different from those of Embodiment 1 will be mainly described. In Embodiment 4, the learning loss 1801 calculation in the learning process of the qualified product image estimation model 400 uses not only the reconstruction error but also the qualified product conversion amount. A specific example will be described below.

[0082] Fig.18 1 is a diagram showing an example of learning of the non-conforming product image estimation model 400 that maximizes the non-conforming product conversion amount by the learning unit 330a constituting the defect inspection system 1 of the fourth embodiment of the present invention. Fig.18 As shown, the internal parameters of the estimation model 401 of multiple qualified product images are learned by minimizing the square error between the input image 430 and the estimated qualified product image 440 and maximizing the square error between the noise image and the estimated qualified product image. The reconstruction error 1803 is learned in a manner of outputting the estimated qualified product image 440 like the input image 430. However, when the defect of the inspected image is minute during inspection, even if the qualified product image can be correctly estimated, it will be mistakenly judged as a false alarm in the false alarm identification process due to the small difference value with the inspected image. If the qualified product image estimation model 400 is learned in a manner of maximizing the square error between the noise image 803 and the estimated qualified product image 440, an image in which the noise image is converted as much as possible is output. However, if learning is performed only by maximizing the qualified product conversion amount 1802, the noise image 803 must be converted as much as possible, so the learning is performed in a manner of outputting the estimated qualified product image 440 that is greatly different from the input image 430. By combining the minimization of the reconstruction error 1803 and the maximization of the qualified product conversion amount 1802 for learning, it is possible to learn the output of the following image: the qualified product part is correctly maintained as a qualified product, and the defective part becomes an image with a difference value greater than the difference value with the corresponding original qualified product pattern. As a result, the difference value of the defective part becomes larger in the comparison process, the stability of the extraction of the defective part is improved, and the reproducibility of the final inspection result is improved.

[0083] As described above, according to the present embodiment, in addition to the effects of the first embodiment, the difference value of the defective part becomes larger in the comparison process, the stability of the defective part extraction is improved, and the reproducibility of the final inspection result can be improved.

[0084] [Example 5]

[0085] use Fig.19 and Fig. 20Embodiment 5 is described. The basic structure of Embodiment 5 is the same as / shared with that of Embodiment 1. Hereinafter, the structural parts in Embodiment 5 that are different from those in Embodiment 1 will be mainly described. In Embodiment 5, in the inspection process of the qualified product image estimation model 400, not all of the multiple estimated qualified product images 440 are used, but only a part of the estimated qualified product images 440 is selectively used. A specific example will be described below.

[0086] Fig.19 1 is a diagram showing an example of inspection processing using the selection of the estimated conforming product image 440 by the defect inspection unit 330b of the defect inspection system 1 constituting the fifth embodiment of the present invention. The case where the estimation model 410 for multiple conforming product images includes three conforming product image estimation models 400 not shown in the figure and outputs three estimated conforming product images 440 for one inspected image 1000 is described as an example. The defect inspection unit 330b inputs the inspected image 1000 to the estimation model 401 for multiple conforming product images whose internal parameters have been read from the learned parameter DB 323, thereby obtaining three estimated conforming product images 440. The internal parameters of the three conforming product image estimation models 400 not shown in the figure are different, so three different estimated conforming product images 440 are output. Next, for the comparison processing 1902, the inspected image 1000, all the estimated conforming product images 440, and the parameters 1900 representing the image generation unit used are input. In the comparison process 1902, the estimated conforming product image 440 outputted from the image generating unit other than the designated image generating unit is excluded from the processing object, and the difference image 1002 between the inspected image 1000 and the estimated conforming product image 440 is generated. In the case where the parameters representing 1 and 2 are inputted as the used image generating unit, the estimated conforming product image 440 outputted from the image generating unit 420 is excluded from the processing object. With the difference image 1002 generated by the comparison process 1902 as the object, as a statistical process, an integrated difference image 1904 representing the average value of the pixel values ​​at the same position of the difference image 1002 is generated. Then, by performing the false alarm identification process 1003 on the integrated difference image 1904, a defect map 1004 representing the defective part is output. The used image generating unit 1900 may also be switched for each inspection area. In the case where a specific image generating unit included in the estimation model 401 of multiple conforming product images cannot output an appropriate estimated conforming product image 440 for the pattern of a part of the inspection area, false alarms in the inspection can be suppressed.

[0087] Fig. 20 Yes means Figure 1 The figure of an example of a GUI of a display device shown in FIG. 1 is a figure showing a display screen of a GUI for selecting an image of an estimated non-defective product. Fig. 20As shown, the GUI of this embodiment has an input column 2001 for inspection parameters. The input column 2001 for inspection parameters includes an input column 1232 for estimated conforming product integration, an input column 1233 for learned parameters of the estimation model 401 for multiple conforming product images, an input column 2002 for the image generator to be used, and an input column 1234 for the image to be inspected. The number of the image generator to be used in the comparison process of the inspection process flow is input in the input column 2002 for the image generator to be used.

[0088] As described above, according to this embodiment, in addition to the effects of embodiment 1, if the learning of some models included in the estimation model of multiple qualified product images fails, the accuracy of the final inspection result decreases. Therefore, by using only the output of the well-learned model, the accuracy can be further improved.

[0089] In addition, the present invention is not limited to the above-mentioned embodiments, and includes various modified examples. For example, the above-mentioned embodiments are examples described in detail in order to easily explain the present invention, and are not necessarily limited to having all the structures described. In addition, a part of the structure of a certain embodiment can be replaced with the structure of other embodiments, and in addition, the structure of another embodiment can be added to the structure of a certain embodiment.

[0090] In addition, the above-mentioned structures, functions, processing parts, processing units, etc. can also be implemented in part or in whole by hardware, for example, by designing with integrated circuits. In addition, the above-mentioned structures, functions, etc. can also be implemented by software by interpreting and executing programs that implement each function through a processor. Information such as programs, tables, files, etc. that implement each function can be placed in a recording device such as a memory, a hard disk, an SSD (Solid State Drive), or a recording medium such as an IC card, an SD card, or a DVD.

[0091] In addition, the control lines and information lines are those considered necessary for explanation, and not all control lines and information lines are necessarily shown in the product. In fact, it can be considered that almost all the structures are connected to each other.

[0092] Explanation of symbols

[0093] 1…Defect Inspection System

[0094] 2… Charged particle beam device

[0095] 3…Computer (computer system)

[0096] 100…Photography device

[0097] 101…Electron Source

[0098] 102…Leader electrode

[0099] 103…Charged particle beam

[0100] 104…Focusing lens

[0101] 105…Scanning Deflector

[0102] 106…Objective lens

[0103] 107…Sample

[0104] 108…Workbench

[0105] 109…Electronics

[0106] 110…Detector

[0107] 200…Controller

[0108] 201, 310…communication device

[0109] 202…Processor

[0110] 203, 331…Memory

[0111] 204, 332…Display device

[0112] 205, 333…Input and output devices

[0113] 320… Storage device

[0114] 321…Learning Image DB

[0115] 322…Inspected image DB

[0116] 323…Learned parameter DB

[0117] 324…Procedure

[0118] 330a…Learning Department

[0119] 330b…Defect Inspection Department

[0120] 400, 400a…Qualified product image estimation model

[0121] 401…Estimation model of multiple qualified product images

[0122] 410…Feature Extraction Unit

[0123] 420…Image generation unit

[0124] 420a ...first image generating unit

[0125] 420b ...second image generating unit

[0126] 430…Input image

[0127] 440…Presumed conforming product image

[0128] 700…Data expansion processing

[0129] 701…Contrast conversion processing

[0130] 702…Brightness conversion processing

[0131] 703…Deformation additional processing

[0132] 800…Noise additional processing

[0133] 803...Noise Image

[0134] 1000...Inspected images

[0135] 1001…Comparison processing

[0136] 1002…Difference Image

[0137] 1003, 1704…False reporting identification and processing

[0138] 1004…Defect Mapping

[0139] 1005, 1701…Central value processing

[0140] 1006···Integrated Defect Mapping

[0141] 1102…Defect Candidate Mapping

[0142] 1105…False Report Judgment

[0143] 1301…First estimated qualified product image

[0144] 1302 ...Second estimated qualified product image

[0145] 1702…Integrate images of presumed qualified products

[0146] 1902…Selective Comparative Processing

[0147] 1903…Averaging

[0148] 1904…Integrated difference images.

Claims

1. A defect inspection system having an imaging device for obtaining an observation image of a sample, characterized in that: The defect inspection system comprises: a learning unit that captures the sample to obtain a learning image, and uses the learning image to learn a multiple conforming product image estimation model for estimating a plurality of conforming product images of the sample from one input image; as well as The defect inspection unit obtains an inspected image by photographing the inspected sample through the photographing device, inputs the inspected image into the learned estimation model of the multiple qualified product images, outputs multiple estimated qualified product images corresponding to the inspected image, and uses the multiple estimated qualified product images to extract defective parts.

2. The defect inspection system according to claim 1, characterized in that: The estimation model of multiple conforming product images includes a plurality of conforming product image estimation models.

3. The defect inspection system according to claim 1, characterized in that: The estimation model of the plurality of qualified product images comprises: a common image feature extraction unit; and a plurality of image generating units for generating one image of a conforming product based on the output of the image feature extracting unit.

4. The defect inspection system according to claim 2, characterized in that: The defect inspection unit creates one integrated estimated non-defective product image from the plurality of estimated non-defective product images using statistical processing, compares the inspected image with the integrated estimated non-defective product image, and extracts defective parts.

5. The defect inspection system according to claim 2, characterized in that: The defect inspection unit compares the inspected image with the plurality of estimated qualified product images, respectively, creates an image representing a defect location, namely, a defect map, performs statistical processing on the plurality of defect maps, and outputs an integrated defect map.

6. The defect inspection system according to claim 4 or 5, characterized in that: The statistical processing creates an image indicating at least one of an average value, a median value, a mode value, and a maximum value of pixel values ​​at the same position on a plurality of images to be processed.

7. The defect inspection system according to claim 1, characterized in that: The learning unit repeatedly updates the internal parameters of the estimation model of the multiple qualified product images to minimize the reconstruction error, wherein the reconstruction error is the difference between the learning image and the estimated qualified product image estimated based on the noise image after the unqualified product features are added to the learning image, and the learning image is a qualified product image.

8. The defect inspection system according to claim 7, characterized in that: The learning unit repeatedly updates the internal parameters of the estimation model of the plurality of conforming product images so as to minimize the reconstruction error and maximize the amount of conforming product conversion from the noise image to the estimated conforming product image.

9. The defect inspection system according to claim 1, characterized in that: The learning unit performs at least one of brightness conversion, contrast conversion, and deformation addition on the learning image before inputting the image into the estimation model of the plurality of conforming product images.

10. The defect inspection system according to claim 1, characterized in that: The defect inspection unit extracts defective parts using a part of the estimated non-defective product images among the plurality of estimated non-defective product images.

11. The defect inspection system according to claim 6, characterized in that: The defect inspection system comprises a display device, An input field for outputting the number of estimated non-defective product images of the estimation model for the multiple non-defective product images is provided on the screen of the display device.

12. A defect inspection method of a defect inspection system, the defect inspection system having a photographing device for obtaining an observation image of a sample, characterized in that: The defect inspection method comprises the following steps: The learning unit photographs the sample to obtain a learning image, and uses the learning image to learn a multiple conforming product image estimation model for estimating a plurality of conforming product images of the sample from one input image; as well as The defect inspection unit obtains an inspected image by photographing the inspection object sample through the photographing device, inputs the inspected image into the learned estimation model of the multiple qualified product images, outputs multiple estimated qualified product images corresponding to the inspected image, and uses the multiple estimated qualified product images to extract defective parts.

13. The defect inspection method according to claim 12, characterized in that: The estimation model of multiple conforming product images includes a plurality of conforming product image estimation models.

14. The defect inspection method according to claim 13, characterized in that: The defect inspection method comprises the following steps: The defect inspection unit creates one integrated estimated conforming product image from the plurality of estimated conforming product images using statistical processing; and The defect inspection unit compares the inspected image with the integrated estimated conforming product image to extract defective parts.

15. The defect inspection method according to claim 13, characterized in that: The defect inspection method comprises the following steps: The defect inspection unit compares the inspected image with the plurality of estimated conforming product images, respectively, and creates a defect map, which is an image showing defect locations; and The defect inspection unit performs statistical processing on a plurality of defect maps and outputs an integrated defect map.

Images