Defect inspection system, defect inspection method, and computer

Abstract

The purpose of the present invention is to efficiently obtain an image to be used for training by a machine learning model when inspecting an image of an object by means of a machine learning apparatus. A defect inspection system according to the present invention divides a layout image representing the structure of a sample into small area images, groups the small area images, and captures the small area images in each group obtained by the grouping with a resolution equal to or higher than that of the layout image (see FIG. 2).

Description

Defect inspection system, defect inspection method, computer 【0001】 The present invention relates to a defect inspection system. 【0002】 As background art in this technical field, there is Patent Document 1. Patent Document 1 describes "a teacher data generation device that generates teacher data, comprising: an image reception unit that receives, from an inspection device that images an object and detects defects, a defect image of a predetermined size including a defect detection region and defect information indicating the range of the detection region in the defect image; a cutout image generation unit that cuts out, based on the defect information, a region including the detection region from the defect image as a cutout image; a display control unit that displays at least a part of the defect image on a display; a determination result reception unit that receives an input of a determination result of a defect type by an operator for the defect image displayed on the display; and a teacher data generation unit that labels the cutout image with the determination result to generate teacher data." (Claim 1). 【0003】 Japanese Unexamined Patent Application Publication No. 2023-122140 【0004】 Patent Document 1 generates learning data for a model used in inspection by cutting out an image so as to include a defect detection region from an inspection image and generating teacher data. Since the model used in inspection needs to be learned with learning images acquired under imaging conditions similar to those during inspection, in order to create learning data by cutting out an image, it is necessary to image the image before cutting out under imaging conditions similar to those during inspection. Therefore, in order to train an inspection model for detecting minute defects, it is necessary to acquire learning images under high-resolution imaging conditions, which requires a great deal of imaging time for acquiring the learning images. As a result, a great deal of preparation time is required until the inspection can be started. 【0005】 The present invention has been made in view of the above problems, and an object thereof is to efficiently obtain an image used by a machine learning model for learning when inspecting an image of an object by a machine learning device. 【0006】The defect inspection system according to the present invention divides a layout image representing the structure of a sample into small region images, groups the small region images, and captures the small region images within each group obtained by the grouping with a resolution higher than that of the layout image. 【0007】 According to the defect inspection system of the present invention, when inspecting images of an object using a machine learning model, it becomes possible to efficiently obtain images that the machine learning model can use for training. Other issues, configurations, and effects will be clarified by the following description of embodiments. 【0008】 This is a schematic diagram of the defect inspection system 1 according to Embodiment 1. This is a flowchart of the processing flow of the defect inspection system 1. This is a diagram showing an example of a layout image acquired in step S1. This is a flowchart of the grouping process targeting the layout image 5 in step S2. This is a diagram showing an example of sub-region division of the layout image 5 in step S21. This is a diagram showing an example of classifying process using image features. This is a diagram showing an example of the result of the grouping process of the layout image. This is a diagram showing an example of setting the second region 14 using the result of the grouping process. This is a diagram showing an example of setting the imaging coordinates of the learning image based on the second region 14. This is a diagram showing an example of a learning image acquired under the second imaging conditions. This is a flowchart of the process of learning a good product image estimation model in step S5. This is a flowchart of the inspection process using the learned good product image estimation model in step S6. This is a diagram showing an example of the inspection result using the good product image estimation model. This is an example of the GUI of the defect inspection system. This is an example of the GUI of the defect inspection system. This shows how a layout image is acquired in Embodiment 2. This shows how a layout image is acquired in Embodiment 3. This is a diagram showing an example of setting imaging coordinates that take into account differences in image quality within the imaging field of view. This is a diagram showing an example of a GUI that sets the inspection area using the grouping result. This is a flowchart for training a defect extraction model. This is a flowchart for defect inspection using a defect extraction model. This figure shows an example of a method for selecting the second domain. This is a schematic diagram of the defect inspection system 1 according to embodiment 8. 【0009】In this specification, imaging devices include charged particle beam systems and optical microscope systems. Charged particle beam systems include Scanning Electron Microscopes (SEMs), Review SEMs, and Focused Ion Beams (FIBs). Below, an example of a charged particle beam system will be described using an SEM. 【0010】 Embodiments of the present invention will be described below with reference to the drawings. Note that the embodiments described below are not intended to limit the invention as defined in the claims, and not all of the elements and combinations described in the embodiments are necessarily essential to the solution of the invention. 【0011】 <Embodiment 1> Figure 1 is a schematic diagram of a defect inspection system 1 according to Embodiment 1 of the present invention. The defect inspection system 1 includes a charged particle beam device 2 and a computer 3. In Embodiment 1, a system comprising the charged particle beam device 2 and the computer 3 is referred to as the defect inspection system 1. Embodiment 1 will be explained using as an example a case in which the charged particle beam device 2 sets a region for acquiring a learning image using an image of a sample captured with an optical microscope. 【0012】 <Configuration of the Defect Inspection System> The defect inspection system 1 comprises a charged particle beam apparatus 2 and a computer 3. The charged particle beam apparatus 2 comprises an imaging device 100 and a controller 200. The imaging device 100 comprises components such as a stage which is a sample holder and a drive circuit in its housing (in other words, a column and a sample chamber). The controller 200 is a control system that drives and controls the imaging device 100 and can be implemented as a computer or circuit. The computer 3 is, in other words, a computer system. The defect inspection system 1 comprises the components necessary to generate signal waveforms and images based on the detection signals of the imaging device 100 which constitutes the charged particle beam apparatus 2. An example of the charged particle beam apparatus 2 is a scanning electron microscope (SEM). 【0013】The imaging device 100 outputs a detection signal obtained by the detector 112 based on the irradiation of the charged particle beam 103 onto the sample 109 on the stage 110, and also outputs the reflected or scattered light of the laser irradiated by the optical microscope 113 as a detection signal. The controller 200 inputs or receives these detection signals and processes them to generate and store signals such as images as detection signals for the charged particle beam device 2. The controller 200 outputs these signals such as images. The computer 3 (computer system) inputs or receives these signals such as images and processes them. 【0014】 In the imaging device 100, the charged particle beam 103 extracted from the electron source 101 by the extraction electrode 102 is focused by a condenser lens 104, which is a type of focusing lens. The charged particle beam 103 is deflected by a scanning deflector 105 and scanned one-dimensionally or two-dimensionally over the surface of the sample 109 on the stage 110. The charged particle beam 103 is decelerated by a negative voltage (retarding voltage) applied to electrodes built into the stage 110, which is the sample holder, and is focused by the lens action of the objective lens 108 and irradiated onto the sample 109. The stage 110 may be a mechanism that can move in the vertical direction (Z direction perpendicular to the X and Y directions), or a mechanism that can rotate and tilt in each axial direction. 【0015】 The details of the implementation of the charged particle beam device 2 are not limited, but a multi-beam irradiation configuration is also possible. Furthermore, although the charged particle beam device 2 is shown with one detector 112, it is not limited to this and may have multiple detectors. For example, it may have a configuration that includes an SE detector for detecting Secondary Electrons (SE) and a BSE detector for detecting Backscattered Electrons (BSE). Alternatively, for example, it may have a configuration in which multiple detectors are installed at multiple locations. In other words, the configuration for image acquisition may include multiple channels and multiple detection systems. Also, when generating an image, it may generate a single image by processing multiple images, such as by integrating them, based on multiple signals repeatedly detected from the same region. 【0016】 The sample 109 is, for example, a semiconductor wafer. When the charged particle beam 103 is irradiated onto the sample 109, electrons 111 such as secondary electrons (SE) and backscattered electrons (BSE) are emitted from within the sample 109. These emitted electrons 111 are accelerated by an acceleration effect based on a negative voltage (retarding voltage) applied to the sample 109 and are detected by the detector 112. The detection signal output from the detector 112 is sent to the controller 200. The controller 200 receives the detection signal via the communication device 201. 【0017】 The controller 200 includes a communication device 201, a processor 202, a memory 203 such as RAM or a non-volatile storage device, a display device 204, and an input / output device 205. These components are interconnected by an architecture such as a bus. The controller 200 controls imaging by the imaging device 100 according to a set imaging recipe. The processor 202 generates an observation image (image under inspection) as an image where the amount of electrons captured by the detector 112 becomes brightness, based on the detection signal obtained via the communication device 201. The processor 202 stores the generated observation image (image under inspection) and other data in the memory 203. The processor 202 transmits a signal corresponding to the generated observation image (image under inspection) and other data to the computer 3 via the communication device 201. The computer 3 receives the signal via the communication device 310. User U1 may also use the controller 200 by operating the display device 204 and the input / output device 205. 【0018】 The communication device 201 is a device that implements communication interfaces with the imaging device 100 and the computer 3. The communication interface may be, for example, a LAN, but is not limited to this. The input / output device 205 is an input device or an output device. The input and output devices may be built-in or externally connected. Examples of input devices include a keyboard, mouse, or microphone. Examples of output devices include a display, printer, or speaker. 【0019】Computer 3 includes a communication device 310, storage 320, a processor 330, memory 331 such as RAM or a non-volatile storage device, a display device 332, and an input / output device 333. These components are interconnected by an architecture such as a bus. Storage 320 is a memory with a relatively large storage capacity. Storage 320 stores a layout image DB 321, a training image DB 322, an image under test DB 323, a trained parameter DB 324, and a program 325. The memory 331 of computer 3 also stores other necessary management information and databases (DBs). External storage devices or servers may be connected to computer 3, and necessary data and information may be stored in such external storage devices or servers, and this data and information may be read and written as appropriate. Alternatively, the various programs mentioned above may be stored in the program 325 within storage 320. Furthermore, the memory 331 may be configured to store various data and information processed by the processor 330 as appropriate. Memory 331 stores, for example, program execution modules, image data, and processing result information. In addition to the illustrated database, storage 320 may also store processing result information, history information, screen data, and the like. 【0020】 The communication device 310 is a device that implements a communication interface with the controller 200. External devices may be connected to the computer (computer system) 3 via a network such as a LAN. The input / output device 333 is an input device or an output device. The input and output devices may be built-in or connected externally. Examples of input devices include a keyboard, mouse, or microphone. Examples of output devices include a display, printer, or speaker. 【0021】User U1 of the defect inspection system 1 uses the computer (computer system) 3 by operating the display device 332 and input / output device 333, etc. This allows user U1 to use the defect inspection system 1. The display screen of the display device 332 shows the user interface of the defect inspection system 1, in other words, a screen with a graphical user interface (GUI). Note that computer 3 may be configured as a client-server system. In that case, user U1 operates a PC or similar device acting as a client terminal, and the PC or similar device accesses computer (computer system) 3, which acts as a server, via communication. 【0022】 Computer (computer system) 3 is the part that performs characteristic processing in this embodiment. Computer (computer system) 3 is connected to controller 200. Note that controller 200 may be configured as part of computer (computer system) 3, or computer 3 may be configured as part of controller 200. Controller 200 and computer (computer system) 3 may be configured as an integrated computer system. Controller 200 may be configured to perform at least a part of the characteristic processing in this embodiment. The entire system, including computer (computer system) 3, may be called the charged particle beam apparatus 2. 【0023】Regarding computer 3 (computer system), if it is configured as a client-server system, the operation will be as follows: User U1 accesses computer 3 (computer system), which is the server, from a client terminal such as a PC via the network. The server provides the client terminal with a screen with a graphical user interface (GUI). The server sends GUI screen data (for example, a web page) to the client terminal. The client terminal displays the GUI screen on its display based on the received screen data. User U1 looks at the GUI screen and inputs instructions, settings, etc. The client terminal sends the input information to the server. The server executes processing according to the received input information. For example, the server performs processing to evaluate the captured image and estimate the device status, stores the processing results, and sends GUI screen data (even just update information) to the client terminal to display the processing results. The client terminal updates the GUI screen display based on the received screen data. User U1 can look at the GUI screen and confirm the processing results, such as the estimated device status. 【0024】 <Processing Flow of the Defect Inspection System> Figure 2 is a flowchart of the processing flow of the defect inspection system 1. Each step in Figure 2 is explained below. 【0025】 Step S1: The imaging device 100 acquires an image of the first region 4 (described later) on the sample 109 based on the first imaging conditions, and saves it in the layout image DB 321 as a layout image 5 (described later) showing the structure of the first region 4, along with supplementary information such as imaging coordinates. In this embodiment, an example will be described in which, based on the first imaging conditions including the setting value of the imaging magnification, a laser is irradiated onto the first region 4 on the sample 109 with an optical microscope 113, and an Optical Microscope (OM) image is generated based on the detection signal obtained from the reflected or scattered light, and this is used as the layout image 5. 【0026】Step S2: The processor 330 divides the layout image 5 acquired in step S1 into small region images 6 (described later), calculates image features for each of the small region images 6, and performs a clumping process to output a pattern map 12 showing the result of dividing the small regions in the first region 4 where similar circuit patterns are formed into the same cluster 13. 【0027】 Step S3: The processor 330 selects at least one representative sub-region from each of the clusters 13 of the pattern map 12 as the second region 14 (described later). 【0028】 Step S4: The processor 330 acquires an image of the second region 14 on the sample 109 using the imaging device 100 based on the second imaging conditions, and saves it as a training image in the training image DB 322. In this embodiment, an example will be described in which an SEM image is generated by irradiating the second region 14 on the sample 109 with a charged particle beam 103 based on the second imaging conditions, which include setting values ​​such as imaging magnification and pixel resolution, and this image is saved in the training image DB 322. 【0029】 Step S5: The processor 330 uses the images stored in the training image DB 322 to train an inspection model to be used for inspecting defective areas of the sample, and stores the internal parameters of the trained inspection model in the trained parameter DB 324. 【0030】 Step S6: The imaging device 100 acquires an image of the sample to be inspected by irradiating the area to be inspected on the sample 109 with a charged particle beam 103 and saves it to the image to be inspected DB 323. The processor 330 reads the internal parameters of the inspection model to be used for inspection from the learned parameter DB 324, reads the image to be inspected from the image to be inspected DB 323, and inputs it to the inspection model, thereby outputting a good product image corresponding to the input image. The processor 330 extracts defective areas by comparing the input image to be inspected with the output good product image. 【0031】 <Example of Defect Inspection> Using Figures 3 to 13, we will explain a specific example of the defect inspection process, assuming that a defect inspection system is used to inspect defects on a semiconductor wafer. 【0032】 Figure 3 shows an example of a layout image acquired in step S1. In semiconductor device design, structures with repeating similar circuit patterns are sometimes used. The inspection model needs to comprehensively learn the circuit patterns within a specified region, but if the region contains similar circuit patterns, the acquisition of training images will involve repeatedly and redundantly capturing the same circuit patterns, which increases the time required to acquire training images. Therefore, in this embodiment, a layout image 5 is acquired using an optical microscope, which has a larger imaging area per unit time than a SEM, to understand the layout of the circuit pattern. The layout image 5 is acquired in order to quickly understand the layout of the circuit pattern formed within the first region 4. The layout image 5 is acquired by irradiating the first region 4 (for example, a single die formed on a semiconductor wafer) designated as the inspection region on the sample 109 with a laser using an optical microscope 113 based on first imaging conditions including the setting value of the imaging magnification, and generating an Optical Microscope (OM) image based on the detection signal obtained from the reflected or scattered light. Semiconductor wafers have a structure in which circuit patterns are stacked, and in OM images, information from not only the circuit patterns formed in the upper layers but also the circuit patterns formed in the lower layers is included in the detection signal. Since SEM images show the circuit patterns of the lower layers depending on the imaging conditions, using OM images as layout images allows for consideration of variations in training images, including the reflection of the lower layers in SEM images. 【0033】 Figure 4 is a flowchart of the grouping process for the layout image 5 in step S2. The grouping process is the process of recognizing regions within the first region 4 where similar circuit patterns are formed, using the layout image 5. 【0034】 Step S21: The processor 330 divides the layout image 5 into small regions and generates small region images 6. 【0035】Step S22: The processor 330 calculates image features for each of the small region images 6. In this embodiment, the small region images 6 are input to a trained deep learning model (e.g., VGG-16 or ResNet-50), and the feature maps output from the intermediate layers are calculated as image features. A deep learning model trained in classification can acquire a general-purpose feature representation to appropriately classify images. Training may include training an autoencoding process that processes the input so that the output is the same as the input, or training a segmentation process that divides the input image into regions according to the objects contained in it. The intermediate layer feature representation of this trained deep learning model is used as a feature that represents the structure of the circuit pattern contained in the small region image 6. 【0036】 Step S23: The processor 330 clusters the small region images 6 using the image features calculated in step S22. 【0037】 An example of clustering processing is explained using Figures 5 to 7. 【0038】 Figure 5 shows an example of sub-region division of the layout image 5 in step S21. In this embodiment, the size of the sub-regions is the same as the field of view size of the second imaging condition, and the layout image 5 is comprehensively divided without overlapping regions. When the size of the sub-regions is the same as the field of view size of the second imaging condition, subsequent processing can be performed computationally efficient. However, the size of the sub-regions may be different from the field of view size of the second imaging condition, and regions may be divided with overlap. 【0039】Figure 6 shows an example of clustering using image features. In a feature space 7 with image features calculated from small region images 6 as axes, each image feature 8 of the small region image 6 is plotted. In the feature space 7, the distance between image feature 8 of small region images 6 with similar circuit pattern layouts is small. Therefore, by applying clustering processing based on the distance between image feature 8 in the feature space 7, clusters 9 can be separated according to the circuit pattern layout that depicts the small region image 6. For clustering processing, for example, the g-means algorithm can be used. The g-means algorithm dynamically determines the appropriate number of clusters using statistical testing, so it can prevent small region images 6 with different circuit pattern layouts from being mixed in cluster 9. Clustering processing may also be done using the k-means algorithm or DBSCAN (Density-Based Spatial Clustering of Applications with Noise), etc. 【0040】 Figure 7 shows an example of the results of grouping a layout image. The pattern map 12 is the same size as the layout image 5, and each of the small region images 6 is assigned a cluster index 13 corresponding to the cluster 9 to which it belongs, and these are assigned to the corresponding regions of the layout image 5. By referring to this pattern map 12, it is possible to identify regions within the first region 4 where similar circuit pattern layouts are formed. 【0041】Figure 8 shows an example of setting the second region 14 using the results of the grouping process. In step S3, based on the pattern map 12 corresponding to the first region 4, the small region image 6 closest to the centroid in each of the clusters 9 is selected as the second region 14. This is because the small region image 6 closest to the centroid in each of the clusters 9 is an image of the average image features in that cluster and can be said to be a pattern that represents the cluster. However, it is also possible to select multiple small region images 6 as the second region 14, not just the small region image 6 closest to the centroid in each of the clusters 9. For example, if the small region image 6 furthest from the centroid is also selected as the second region, the variations within each of the clusters 9 can be increased. 【0042】 Figure 9 shows an example of setting the imaging coordinates for a training image based on the second region 14. The imaging coordinates 15, which are the center of the imaging field of view, are determined so that the imaging field of view 16 under the second imaging condition covers the second region 14, and these are set as the imaging coordinates for the training image. Since the second region 14 is a region that represents the layout of each circuit pattern, by comprehensively imaging the second region 14, a training image that shows all variations of the circuit pattern layout can be obtained without imaging everything within the first region 4. 【0043】 Figure 10 shows an example of a training image acquired under the second imaging conditions. The second imaging conditions for acquiring the training image are the same as those used during inspection. In this embodiment, the second imaging conditions are SEM imaging conditions, and are imaging conditions that yield a higher resolution image than the OM image acquired under the first imaging conditions. Training image 17 is an example of imaging the imaging coordinates set based on the second region 14 with the SEM under the second imaging conditions. The detailed shape of the circuit pattern, which could not be captured with the resolution of the first imaging conditions, can be captured with the higher resolution of the second imaging conditions, and can be used to train an inspection model because the imaging conditions are the same as those used during inspection. 【0044】Figure 11 is a flowchart of a process for training a non-defective product image estimation model in step S5. In the present embodiment, a case where a non-defective product image estimation model is used as an inspection model will be described as an example. The non-defective product image estimation model is configured based on a machine learning model, such as an auto encoder, U-Net, GAN (Generative Adversarial Networks), etc. Further, the non-defective product image estimation model can be trained only with non-defective product images, and is a model that does not require the collection of defective product images, which is considered to have a high collection cost. Each step is implemented by the processor 330. 【0045】 S502: First, create a training dataset. The training dataset is created by reading the training images stored in the training image DB 322. 【0046】 S503: Repeatedly execute the following training process by means of a loop of iterative training. 【0047】 S504: Divide the images included in the training dataset into an arbitrary number of pieces, and divide the training dataset into training mini-batches by using each of them as a training mini-batch. 【0048】 S505: Skip S506 to S510 for each training mini-batch. 【0049】 S506: Obtain the training mini-batch l. 【0050】 S507 to S510: For each of the divided training mini-batches, execute data augmentation (S507), training loss calculation (S508 to S509), and internal parameter update (S510). 【0051】 S507: In data augmentation, for each image included in the training mini-batch, perform luminance conversion, contrast conversion, and distortion addition on each of them with random intensities within a certain range based on the reference values of a random number table, thereby creating input images for the non-defective product image estimation model. 【0052】S508-S509: To train the good image estimation model to estimate good images from poor images, noise is added to the input image as poor features to create poor images. By inputting the created poor images into the good image estimation model, an estimated good image is obtained as output. At this time, the average of the squared errors of the brightness of each pixel of the estimated good image and the input image (the image before noise was added) is calculated as the learning loss. The learning loss may also include SSIM (Structural Similarity), which is an index value related to image quality, or a regularization term that penalizes internal parameters to prevent overfitting to the training dataset. 【0053】 S510: Based on the calculated learning loss, the values ​​of the learner's internal parameters that minimize the learning loss are searched for using optimization methods such as gradient descent, and the internal parameters of the good image estimation model are updated according to the search results. 【0054】 S511-S512: The process from data augmentation (S507) to internal parameter update (S510) is repeated the same number of times as the number of training minibatches (S511). Then, the training dataset is divided into training minibatches again, and the internal parameters are updated for each training minibatch (S512->S505). By repeating this training loop any number of times, the internal parameters of the good image estimation model are determined. 【0055】Figure 12 is a flowchart of the inspection process using the trained good product image estimation model in step S6. The imaging device 100 scans the area to be inspected of the sample 109 with the charged particle beam 103 and saves the acquired image to be inspected in the image database 323. The processor 330 reads the image to be inspected from the image database 323. The processor 330 reads the internal parameters of the good product image estimation model from the trained parameter database 324. The processor 330 inputs the image to be inspected to the good product image estimation model from which the internal parameters have been read and outputs an estimated good product image. If the image to be inspected contains defects, an image in which the defective area is transformed to look like a good product is output. The processor 330 compares the image to be inspected and the estimated good product image to extract the defective area, thereby creating a difference image in which each pixel shows the difference in pixel value between the image to be inspected and the estimated good product image. The processor 330 excludes false information—areas where a difference has occurred but which are not actually defects—based on the image features of the difference image. The processor 330 outputs information about the defective areas extracted as a result of the inspection. 【0056】Figure 13 shows an example of the inspection results using a good image estimation model. The inspection uses the image under inspection 18 and the estimated good image 19. The estimated good image 19 is obtained by inputting the image under inspection 18 into a good image estimation model that has been loaded with trained parameters, and outputting it as an output. In the comparison process, the difference value of each pixel between the image under inspection 18 and the estimated good image 19 is calculated, and a difference image 20 is output. Regions where a difference occurs in the comparison process are regions where the image under inspection has been transformed into a different shape, and are therefore regions that are highly likely to be defective. Next, by performing a false reporting discrimination process on the difference image 20, regions where a difference has occurred but are not actually defective are excluded, and an inspection result 21 indicating the defective area is output. In the false reporting discrimination process, connected regions in the difference image are considered as one defect candidate, and for each defect candidate, the mean, variance, maximum value of the difference value, circularity, area, etc. are calculated as features, and it is determined whether it is a defect or a false report by determining whether each feature is outside the normal range. The criteria for judgment might be, for example, that the area is less than or equal to a threshold, the maximum difference value is less than or equal to a threshold, or the average difference value is less than or equal to a threshold. 【0057】 Figures 14A and 14B show examples of GUIs for a defect inspection system. Figure 14A shows the first half of the screen, and Figure 14B shows the second half of the screen. As shown in Figure 14A, the GUI according to this embodiment has input fields 1400 for learning parameters of a good product image estimation model and a display field 1410 for learning images. The learning parameter input field 1400 includes a learning area input field 1401 and a learning parameter input field 1406. The learning area input field 1401 includes a display field 1405 for the arrangement and coordinate information of chips manufactured on the sample, an input field 1402 for the chip number to be used as the learning area, an input field 1403 for the chip coordinate of the top-left vertex of a rectangle to specify the learning area within the specified chip, and an input field 1404 for the chip coordinate of the bottom-right vertex of the rectangle. The learning parameter input field 1406 includes an input field 1407 for the number of learning iterations and an input field 1408 for the batch size to determine the size of the learning mini-batch, in order to set the parameters used for learning the good product image estimation model. After entering each parameter, pressing the learning start button 1409 will execute the processes corresponding to steps S1 to S5. 【0058】 The learning image display area 1410 includes a display area 1411 for the acquired layout image, a pattern map 1412 output as a result of grouping the layout image, and a display area 1413 for the acquired learning image and supplementary information such as coordinates and cluster names. In the layout image display area 1411, the areas in the layout image where the learning image was acquired are displayed as rectangles or the like. 【0059】 As shown in Figure 14B, the GUI according to this embodiment has an input field 1420 for inspection parameters and an input field 1430 for displaying inspection results. The input field 1420 for inspection parameters includes an input field 1421 for the inspection area and an input field 1426 for trained parameters that specify the parameters of the model used for inspection. The input field 1421 for the inspection area includes an input field 1425 for displaying the arrangement and coordinate information of the chips manufactured on the sample, an input field 1422 for the chip number that specifies the chip to be inspected as a number or ALL to indicate all chips, an input field 1423 for the chip coordinate of the top-left vertex of the rectangle to specify the inspection area within the specified chip as a rectangle, and an input field 1424 for the chip coordinate of the bottom-right vertex of the rectangle. The input field 1426 for trained parameters specifies the trained parameters to be used for inspection by inputting the path to the data file in which the trained parameters of the model used for inspection are stored. After entering each parameter, pressing the test start button 1427 executes the process corresponding to step S6, and information regarding the test results is displayed in the test results display area 1430 during and after execution. 【0060】 The inspection result display area 1430 includes an image display area 1431 related to the inspection and a defect distribution display area 1432 on the sample. The image display area 1431 displays the inspected image, the estimated good product image which is the output of the good product image estimation model, and an image showing the extracted defect location, along with supplementary information such as chip number and coordinate information. The defect distribution display area 1432 displays the number of defects and defect density detected for each region on the sample, for example, for each chip manufactured on a semiconductor wafer, and a wafer map 1433 that represents the defect density with a color bar. 【0061】<Embodiment 1: Summary> According to the defect inspection system 1 of Embodiment 1, it is possible to reduce redundant imaging of training images with similar circuit pattern layouts, and to provide an imaging method and defect inspection system that can efficiently obtain training images for the model used for inspection. 【0062】 <Embodiment 2> Figure 15 shows how a layout image is acquired in Embodiment 2 of the present invention. The basic configuration of Embodiment 2 is the same as and common to Embodiment 1, and below, the components of Embodiment 2 that differ from Embodiment 1 will be mainly described. In Embodiment 2, instead of an optical microscope, a SEM with a lower magnification than the second imaging condition is used as the first imaging condition for acquiring the layout image. 【0063】 Figure 15 shows an example of a layout image acquired by SEM using low-magnification imaging conditions. When the imaging magnification of an SEM is reduced, it becomes more difficult to resolve the shape of fine circuit patterns, but the imaging area per unit time can be increased. The purpose of acquiring the layout image is not inspection, but rather to grasp the general structure within a specified area, so low-magnification SEM images can be used. Therefore, in this embodiment, a layout image is acquired using an SEM with a lower magnification than the second imaging condition as the first imaging condition. The layout image 22 acquired at low magnification is an image in which the circuit pattern appears blurred compared to the image 23 acquired under the second imaging condition, which is the same imaging condition as for inspection, but it is still possible to grasp the general structure. Even with SEM imaging conditions at low magnification, the resolution is higher than that of an optical microscope and the shape of the circuit pattern is captured, so it is possible to determine the area for acquiring training images based on finer structural information than when using images acquired with an optical microscope as layout images. 【0064】 According to the defect inspection system 1 of Embodiment 2, in addition to the effects of Embodiment 1, it is possible to further significantly reduce the number of times training images are captured. 【0065】<Embodiment 3> Figure 16 shows how a layout image is acquired in Embodiment 3 of the present invention. The basic configuration of Embodiment 3 is the same as and common to Embodiment 1, and below, the components of Embodiment 3 that differ from Embodiment 1 will be mainly described. In Embodiment 3, an SEM with a lower pixel resolution than the second imaging condition is used as the first imaging condition for acquiring the layout image, instead of an optical microscope. 【0066】 Figure 16 shows an example of a layout image acquired by SEM using imaging conditions with low pixel resolution. SEM can increase the imaging area per unit time by lowering the pixel resolution (resolution per pixel) without changing the imaging field of view. The layout image 24, acquired with the same imaging field as the second imaging condition but with lower pixel resolution, shows a blurred circuit pattern, but the general structure can still be understood. Even with SEM imaging conditions that reduce pixel resolution, the resolution is higher than that of an optical microscope, and the shape of the circuit pattern is visible. Therefore, it is possible to determine the area for acquiring training images based on finer structural information than when using images acquired with an optical microscope as layout images. Furthermore, by lowering the pixel resolution, the total number of pixels in the acquired layout image decreases, reducing the amount of computation required for subsequent processing and shortening processing time. 【0067】 According to the defect inspection system 1 of Embodiment 2, in addition to the effects of Embodiment 1, it is possible to shorten the grouping processing time and improve the effect of reducing the number of times training images are captured. 【0068】 <Embodiment 4> Embodiment 4 of the present invention will be described with reference to Figure 17. The basic configuration of Embodiment 4 is the same as and common to Embodiment 1, and below, the components of Embodiment 4 that differ from Embodiment 1 will be mainly described. In Embodiment 4, the setting of the imaging coordinates of the learning image based on the second region in step S4 is carried out taking into consideration the difference in image quality within the imaging field of view of the second imaging conditions. 【0069】Figure 17 shows an example of setting imaging coordinates that take into account differences in image quality within the imaging field of view. In imaging systems such as optical microscopes and SEMs, aberrations can occur, where the actual imaging position deviates from the ideal imaging position due to differences in the incident position of light rays or electron beams on the lens. For example, an electron beam passing near the center of the lens will have a different imaging position than an electron beam passing far from the center of the lens, and the image plane will be curved, resulting in images with different amounts of blurring near the center and at the periphery. Since the inspection model processes based on learned image features, if the relationship between the circuit pattern and its position within the imaging field of view differs between the learning and inspection periods, false detection of good parts or failure to detect defective parts may occur. As a countermeasure, in this embodiment, when setting imaging coordinates based on the second region, the imaging coordinates are set to cover the second region while overlapping the imaging fields of view, so that when imaging the circuit patterns contained in each of the second regions, learning images are obtained that are captured near the center of the imaging field of view and near the periphery of the imaging field of view. 【0070】 According to the defect inspection system 1 of Embodiment 4, in addition to the effects of Embodiment 1, it is possible to reduce false detection of good parts and failure to detect defective parts caused by changes in how they appear due to the positional relationship of circuit patterns within the imaging field of view, thereby enabling more accurate defect inspection. 【0071】 <Embodiment 5> Embodiment 5 will be described using Figure 18. The basic configuration of Embodiment 5 is the same as and common to Embodiment 1, and below, the components that differ in Embodiment 5 from those of Embodiment 1 will be mainly described. In Embodiment 5, the defect inspection in step S6 is performed on the area determined based on the pattern map, which is the grouping result. 【0072】Circuit patterns formed on a semiconductor wafer include those that should be inspected and those that do not. For example, one type of circuit pattern that may not need to be inspected is a dummy pattern, which is formed to ensure uniform wiring density within the chip and does not affect the operation of the chip. In some cases, it may be desirable to focus on inspecting circuit patterns that have a significant impact on manufacturing yield. Therefore, the importance of inspection differs for each circuit pattern. Inspecting circuit patterns of low importance leads to increased inspection time, so the inspection area needs to be set according to the user's inspection objectives. In this embodiment, a pattern map visualizing the layout of the circuit patterns is presented, and the user is allowed to select the circuit patterns and areas to be inspected. 【0073】 Figure 18 shows an example of a GUI for setting inspection areas using grouping results. The inspection area input field 1800 has a layout image display field 1801, a training image display field 1801, and an area selection input field 1803. The area selection input field 1803 includes a checkbox to select either manual setting 1804 or pattern specification 1805 as the method for selecting the inspection area. When using manual setting, the cursor 1806 is operated with a mouse or the like to select the areas to be inspected on the pattern map by clicking, etc. If all areas of a specified pattern are to be inspected, the cluster name of the pattern to be inspected is entered in the input field. 【0074】 According to the defect inspection system 1 of Embodiment 5, in addition to the effects of Embodiment 1, by narrowing the inspection area to the region where the circuit pattern that the user of the defect inspection system wants to focus on is formed, it becomes possible to speed up defect inspection and improve the efficiency of defect analysis. 【0075】 <Embodiment 6> Embodiment 6 of the present invention will be described using Figures 19 and 20. The basic configuration of Embodiment 6 is the same as and common to Embodiment 1, and below, the components of Embodiment 6 that differ from Embodiment 1 will be mainly described. In Embodiment 6, a defect extraction model that learns the characteristics of defective parts is used as the inspection model, rather than a good product image estimation model. 【0076】Figure 19 is a flowchart of the defect detection model training process. The defect detection model is constructed based on a machine learning model, such as a CNN (Convolutional Neural Network) or U-Net. First, defect training labels are created that indicate the location of defects in the training images (S1902). Defect training labels are, for example, images in which defective areas have a pixel value of 1 and other areas have a pixel value of 0. The creation of defect training labels may be done manually, such as by visual inspection, or based on the results of inspection methods such as image processing. The defect detection model is trained using a training dataset that associates the training images with the created defect training labels (S1903-S1912). Training is performed by minimizing the error between the defect confidence score of each pixel output by the defect detection model and the defect training label. Other processing is the same as in Embodiment 1. 【0077】 Figure 20 is a flowchart of defect inspection using a defect extraction model. When using a pre-trained defect extraction model for inspection, an inspection result showing the defect confidence level of each pixel in the image to be inspected can be obtained by inputting the image to be inspected. Other processing is the same as in Embodiment 1. 【0078】 In Embodiment 1, the good product image estimation model is trained only on good product images and therefore cannot learn the image features of defects. However, the defect extraction model can learn the image features of defects, allowing for more sensitive defect inspection. 【0079】 According to the defect inspection system 1 of Embodiment 6, in addition to the effects of Embodiment 1, it is possible to improve the sensitivity of defect inspection by learning the image characteristics of the defective part. 【0080】 <Embodiment 7> Embodiment 7 will be described using Figure 21. The basic configuration of Embodiment 7 is the same as and common to Embodiment 1, and below, the components of Embodiment 7 that differ from Embodiment 1 will be mainly described. In Embodiment 7, in step S3, not only is the one closest to the centroid of each cluster selected as the second region, but also those that are at a distance of more than a threshold from the centroid are selected. 【0081】Figure 21 shows an example of a method for selecting the second region. Clustering processing forms a cluster for each small region image that shows a similar circuit pattern layout. However, even within the same cluster, if the image features are different, the circuit pattern layout shown in the image is not exactly the same. For example, there may be differences in the position where the circuit pattern is shown in the image, or noise and minute distortions caused by the imaging device. If these differences are large, even if they are the same cluster, they may actually be small region images with different layouts. In this case, if only the small region image closest to the cluster centroid is selected, there is a possibility that the learning of the circuit pattern layout will be missed. To prevent this, in this embodiment, in addition to selecting the one closest to the centroid of each cluster as the second region, images that are at a distance greater than a threshold from the centroid are also selected. The threshold is set, for example, using the mean and standard deviation of the distance from the cluster centroid. By setting a threshold, an outlier boundary 26 centered on the cluster centroid can be set, and the small region images 27 outside the outlier boundary can be determined to have a different trend from the small region images that occupy a large proportion of the cluster, and are therefore selected as the second region. 【0082】 According to the defect inspection system 1 of Embodiment 7, in addition to the effects of Embodiment 1, it is possible to prevent omissions in learning circuit patterns and stabilize the inspection performance of the inspection model. 【0083】 <Embodiment 8> Embodiment 8 will be described using Figure 22. The basic configuration of Embodiment 6 is the same as and common to Embodiment 1, and below, the components of Embodiment 8 that differ from Embodiment 1 will be mainly described. Embodiment 8 uses a configuration in which a computer and multiple devices are interconnected by a network. 【0084】Figure 22 is a schematic diagram of the defect inspection system 1 according to Embodiment 8. The computer 3, multiple charged particle beam devices 2, multiple optical inspection devices 1901, multiple review SEMs 1902, and multiple manufacturing devices 1903 are interconnected via a network 1900. The computer 3 collects data output from each device via a communication device 310 and stores it in storage 320 as needed. Each device connected to the computer 3 can receive operation instructions and data via its own communication device. Multiple units of multiple types of devices may be operating within a manufacturing plant. 【0085】 According to the defect inspection system 1 of Embodiment 8, in addition to the effects of Embodiment 1, by connecting multiple devices and computers, it becomes possible to improve the performance of the inspection model by accumulating a wide variety of training images and to perform more detailed defect analysis by linking inspection results from multiple manufacturing processes. 【0086】 <Regarding Variations of the Invention> The present invention is not limited to the embodiments described above, and various variations are included. For example, the embodiments described above are described in detail to make the present invention easy to understand, and are not necessarily limited to those having all the configurations described. Furthermore, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. 【0087】 Furthermore, each of the above configurations, functions, processing units, and processing means may be implemented in hardware, either partially or entirely, by designing them as integrated circuits, for example. Alternatively, each of the above configurations and functions may be implemented in software by having the processor interpret and execute programs that implement each function. Information such as programs, tables, and files that implement each function can be stored in memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD. 【0088】Furthermore, the control lines and information lines shown are those deemed necessary for explanatory purposes, and not all control lines and information lines are necessarily shown in the actual product. In reality, it is safe to assume that almost all components are interconnected. 【0089】 In the embodiments described above, the following are examples of models (feature extraction models) that extract image features in the intermediate layer, and any of them may be used: (a) a model that classifies the input image into classes based on its features; (b) a model that divides the input image into sub-images based on its features; a model that extracts the features of the input image and uses them to regenerate the input image. 【0090】 100...Imaging device 101...Electron source 102...Drawer electrode 103...Charged particle beam 104...Condenser lens 105...Scanning deflector 108...Objective lens 109...Sample 110...Stage 111...Electron 112...Detector 113...Optical microscope 200...Controller 201, 310...Communication device 202, 330...Processor 203, 331...Memory 204, 332...Display device 205, 333...Input / output device 320...Storage 321...Layout image DB 322...Training image DB 323...Image under inspection DB 324...Trained parameter DB 325...Program

Claims

1. A defect inspection system for inspecting defects contained in a sample using an observation image of the sample, comprising: an imaging device for acquiring the observation image; a computer for inspecting the defects using the observation image; wherein the imaging device acquires a layout image showing the structure of a first region of the sample under first imaging conditions; the computer creates small region images by dividing the layout image into small regions; the computer calculates image features of the small region images; the computer groups the small region images based on the image features; the computer selects a small region image from each group obtained by the grouping; the computer sets the small region images selected from the groups as a second region; the imaging device acquires an image of the second region under second imaging conditions having a resolution equal to or greater than the first imaging conditions; the computer learns an inspection model for inspecting the defects using the image of the second region; and the computer inspects the defects using the inspection model.

2. The defect inspection system according to claim 1, wherein the computer implements a feature extraction model for extracting feature quantities from the observed image, and the computer inputs the small region image to the feature extraction model to obtain the feature quantities that can be obtained from the intermediate layer of the feature extraction model as image features of the small region image.

3. The defect inspection system according to claim 1, characterized in that the computer selects from each of the groups the small region image closest to the centroid of the group in the feature space represented by the image features.

4. The defect inspection system according to claim 1, characterized in that the imaging device images all planar areas within the area set as the second area.

5. The defect inspection system according to claim 1, wherein the inspection model is configured as a good product image estimation model that estimates a good product image representing the appearance that the input image is presumed to have when there are no defects in the input image, and the computer extracts the defects contained in the input image by comparing the good product image obtained by inputting the input image to the good product image estimation model with the input image.

6. The defect inspection system according to claim 1, characterized in that the computer provides a user interface that presents at least one of the observation image, the layout image, the small area image, the grouping result, and the result of the defect inspection.

7. The defect inspection system according to claim 1, characterized in that the imaging device comprises an optical microscope and a charged particle beam device, the first imaging condition is the imaging condition of the optical microscope, and the second imaging condition is the imaging condition of the charged particle beam device.

8. The defect inspection system according to claim 1, characterized in that the imaging device is composed of a charged particle beam device, the first imaging condition and the second imaging condition are the imaging conditions of the charged particle beam device, and the first imaging condition has an imaging magnification less than or equal to that of the second imaging condition.

9. The defect inspection system according to claim 1, characterized in that the imaging device is composed of a charged particle beam device, the first imaging condition and the second imaging condition are imaging conditions of the charged particle beam device, and the first imaging condition has an image resolution less than or equal to that of the second imaging condition.

10. The defect inspection system according to claim 1, characterized in that the computer sets the second region such that it acquires the region including the center of the small region image selected from the group multiple times while shifting the imaging range.

11. The defect inspection system according to claim 1, characterized in that the computer presents a pattern map showing the positions of the small region images together with the groups obtained by the grouping, and the computer sets the second region on the pattern map according to the results specified by the user for the small region images to be inspected.

12. The defect inspection system according to claim 1, wherein the inspection model is configured as a defect extraction model that estimates the location of the defect in the input image, and the computer extracts the defect contained in the input image by inputting the input image to the defect extraction model.

13. The defect inspection system according to claim 1, characterized in that the computer selects from the group the small region image closest to the centroid of the group and the small region image having a distance of a threshold or more from the centroid in the feature space represented by the image features.

14. A defect inspection method for inspecting defects contained in a sample using an observation image of the sample, comprising: a step of acquiring a layout image showing the structure of a first region of the sample under first imaging conditions; a step of creating small region images by dividing the layout image into small regions; a step of calculating image features of the small region images; a step of grouping the small region images based on the image features; a step of selecting the small region images from each group obtained by the grouping; a step of setting the small region images selected from the groups as a second region; a step of acquiring an image of the second region under second imaging conditions having a resolution equal to or greater than that of the first imaging conditions; a step of learning an inspection model for inspecting defects using the image of the second region; and a step of inspecting the defects using the inspection model.

15. A computer for inspecting defects contained in a sample using an observation image of the sample, comprising a processor configured to perform the following steps: acquiring a layout image of the structure of a first region of the sample using a first imaging condition; creating sub-region images by dividing the layout image into sub-regions; calculating image features of the sub-region images; grouping the sub-region images based on the image features; selecting a sub-region image from each group obtained by the grouping; setting the sub-region images selected from the groups as a second region; acquiring an image of the second region using a second imaging condition having a resolution equal to or greater than that of the first imaging condition; learning an inspection model for inspecting defects using the image of the second region; and inspecting the defects using the inspection model.

Images