Defect classification apparatus and defect classification method

Abstract

An appropriate defect classification can be achieved. Therefore, a defect classification device (100) includes a defect region image generation section (120) that generates a defect region image based on a difference between a learning image including one or more circuit pattern regions and a defect region and a first reference image, a defect type determination section (130) that determines a positional relationship between the circuit pattern region and the defect region, and an image composition section (143) that composes the defect region image and a second reference image based on the positional relationship, thereby generating a composition image.

Description

Technical Field

[0001] This invention relates to a defect classification device and a defect classification method. Background Technology

[0002] As background technology in this field, the abstract of the following patent document 1 describes a "judgment system (100) comprising: a judgment device (1) for judging whether an object is qualified or not using a judgment model (15); a data collection device (3) for acquiring an image when the judgment device (1) mistakenly judges it as qualified; an image generation device (4) for generating multiple learning images based on the unqualified parts reflected in the image; and a learning device (5) for relearning the judgment model (15) using the multiple learning images as training data."

[0003] Furthermore, the abstract of Patent Document 2 below describes "a learning data generation apparatus that can generate learning data suitable for learning a recognition model. The learning data generation apparatus has the following functions: cutting out a portion of first image data as second image data; generating a two-dimensional graphic representing a suspected defect corresponding to the area of ​​the second image data; combining the second image data and the two-dimensional graphic to generate third image data; and assigning a label to the third image data corresponding to the two-dimensional graphic. By using the third image data for learning a recognition model, a high-precision recognition model can be generated."

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2022-142555

[0007] Patent Document 2: International Publication No. 2021 / 152416 Summary of the Invention

[0008] The problem that the invention aims to solve

[0009] However, it is desirable to achieve more appropriate defect classification using the techniques described above.

[0010] The present invention was made in view of the above circumstances, and its object is to provide a defect classification device and a defect classification method capable of achieving appropriate defect classification.

[0011] Methods for solving problems

[0012] To address the aforementioned issues, the defect classification apparatus of the present invention comprises: a defect region image generation unit that generates a defect region image containing defect regions within a defect region image corresponding to the defect region based on a difference between a learning image obtained by photographing an inspection target region including one or more circuit pattern regions and a defect region and a first reference image containing circuit pattern regions within a first reference image; a defect type determination unit that determines the positional relationship between the circuit pattern region and the defect region based on a circuit pattern recognition image containing circuit pattern regions within a circuit pattern recognition image corresponding to the circuit pattern region in the first reference image and the defect region image; and an image synthesis unit that synthesizes the defect region image with a second reference image based on the positional relationship to generate a synthesized image, wherein the first reference image is an image obtained by photographing a first semiconductor wafer, and the second reference image is an image obtained by photographing a second semiconductor wafer different from the first semiconductor wafer.

[0013] Invention Effects

[0014] According to the present invention, appropriate defect classification can be achieved. Attached Figure Description

[0015] Figure 1 These are diagrams illustrating specific examples of various images applied to different implementation methods.

[0016] Figure 2 This is a diagram illustrating an example of the structure of an image management table.

[0017] Figure 3 It is an explanatory diagram showing the positional relationship between the circuit pattern area and the defect area.

[0018] Figure 4 This is an explanatory diagram illustrating the definition of the location relationship of defect types.

[0019] Figure 5 This is a diagram illustrating the structure of a table showing the results of determining the location relationship of defect types.

[0020] Figure 6 This is a diagram illustrating a structural example of image synthesis setting information.

[0021] Figure 7 This is an example of learning to select a screen using images.

[0022] Figure 8 This is an example of a screen showing the location and relationship of defect types.

[0023] Figure 9 This is an example of a composite image confirmation screen.

[0024] Figure 10This is a block diagram illustrating a structural example of the defect classification device according to the first embodiment.

[0025] Figure 11 This is a block diagram illustrating an example of the hardware structure of a defect classification server.

[0026] Figure 12 This is a flowchart representing an example of a processing routine executed by the defect classification server.

[0027] Figure 13 This is a diagram showing specific examples of various images applied to the second embodiment.

[0028] Figure 14 This is a flowchart illustrating an example of a processing routine executed by the defect classification server in the second embodiment.

[0029] Figure 15 This is a flowchart illustrating an example of a processing routine executed by the defect classification server in the third embodiment.

[0030] Figure 16 This is a flowchart illustrating an example of a processing routine executed by the defect classification server in the fourth embodiment. Detailed Implementation

[0031] [Summary of Implementation Methods]

[0032] In the semiconductor manufacturing process, ensuring high product yield is crucial. Therefore, yield management utilizes early detection of yield reductions and yield variation analysis techniques. Yield management allows for the early detection of various defects on semiconductor wafers generated during the manufacturing process, enabling corrective actions. This typically involves the following three steps:

[0033] (1) Use wafer inspection equipment to inspect semiconductor wafers and detect the location of defects and foreign objects.

[0034] (2) Take images of the defects that appear at the detected locations, and classify the types of defects based on their appearance.

[0035] (3) Take countermeasures based on the classification results of defects.

[0036] In (2) above, SEM (Scanning Electron Microscope) is used to photograph defects at high magnification. However, when the number of defects in the object being inspected is very large, photographing and classifying the types of defects in (2) requires labor. Therefore, sometimes the ADR (Automatic Defect Review) function, which automatically photographs and collects images of defect locations, and the ADC (Automatic Defect Classification) function, which automatically classifies the defects reflected in the collected images, are used.

[0037] Regarding ADR and ADC, with the recent development of machine learning technologies, represented by deep learning, the application of machine learning techniques to these functions has been continuously evolving. The implementation method described later, for the ADC as the object, utilizes machine learning to automatically classify the types of defects reflected in the collected images.

[0038] To automatically classify the types of defects reflected in images using machine learning, machine learning programs and learned models are needed. These programs and models are typically developed by data scientists (operators). The developed programs and models are then installed as software, and the machine learning program is executed when the user uses the device, using the learned model to automatically classify the types of defects.

[0039] A machine learning program that automatically classifies defect types reads in an input image, processes the image using machine learning algorithms, and outputs the defect type (hereinafter, this series of processes is referred to as classification processing). When performing machine learning algorithm processing, a learned model is utilized. Machine learning algorithms are broadly classified into supervised and unsupervised methods; however, when using supervised algorithms, a learned model needs to be prepared beforehand before performing classification processing.

[0040] Examples of methods for preparing learned models include reusing existing learned models that have been developed separately, relearning existing learned models, and developing new learned models. In the case of relearning an existing learned model or developing a new learned model, training images and training data for those images are required. To develop a learned model for automatically classifying defect types, images reflecting the defects and the types of defects are provided as training data. Regarding training data, data scientists typically visually identify the types of defects reflected in the images and create training data for each image (hereinafter, the set of training images and training data will be referred to as the learning dataset).

[0041] In cases where the accuracy of automatic classification of defect types deteriorates, it is preferable to relearn the model using a training dataset. Collecting a training dataset is a labor-intensive task, but preparing one can improve classification accuracy. In improving accuracy, consider applying manually designed techniques similar to those used in conventional image processing algorithms. In this case, the algorithm needs to be redesigned after analyzing the causes of accuracy degradation. On the other hand, in the case of machine learning, if a training dataset can be prepared, accuracy can be improved; therefore, it is considered superior in handling immediate adverse situations when accuracy deteriorates.

[0042] However, in actual semiconductor product manufacturing processes, it is sometimes difficult to prepare sufficient training datasets for model learning. For example, in a multi-variety, low-volume production line, it is impossible to capture a large number of images required for machine learning. Therefore, in model learning, it is required to go beyond relying solely on captured images and consider processing the already captured images to increase the number of images, using these additional images as training images. Therefore, the object of this embodiment is to increase the number of images that can be used as training images, taking the images collected in the semiconductor product manufacturing process as the target.

[0043] If the technology of Patent Document 1 is applied, it is believed that it is possible to acquire an image that is mistakenly judged as qualified, and to generate a defective part image based on the defective part reflected in the image. It is also believed that, in addition to generating the defective part image, a mask image can be generated simultaneously, and transformation processing is applied to the defective part image and the mask image to generate a composite image. Then, it is believed that the generated composite image is combined with the qualified image to obtain a learning image. As for the transformation processing, it is believed that various processing methods such as rotation transformation, shape transformation, and color transformation can be applied.

[0044] However, the images collected during the semiconductor product manufacturing process reflect various defects on the semiconductor wafers produced during the manufacturing process. When images of the semiconductor wafer are captured, the appearance of various defects and the circuit patterns drawn on the semiconductor wafer are captured in the images. Defects produced during the semiconductor product manufacturing process include defects generated on the circuit patterns as well as defects generated between circuit patterns. From the perspective of product yield management in the manufacturing process, it is preferable to classify defects not only according to the type of defect appearance but also consider where the defects occur in the circuit pattern. That is, it is preferable to distinguish and classify the location of defects on the circuit pattern even if the types of defects appearance are the same.

[0045] When using machine learning, the above points are considered for defect classification. Therefore, the learning images are prepared according to the type of defect appearance and the positional relationship of the defects on the circuit pattern. For example, when considering "foreign object" as a type of defect, it is preferable to distinguish whether the foreign object is generated on the circuit pattern or between circuit patterns, and machine learning is used to classify the defects.

[0046] Therefore, the learning images required for learning are preferably prepared separately, taking into account where defects occur in the circuit pattern. In the technology of Patent Document 1, images of defective parts are combined with images of acceptable parts to obtain learning images. However, since this technology does not take an image reflecting the circuit pattern as its object, it is impossible to generate learning images that take into account the positional relationship between the circuit pattern and the defects.

[0047] Furthermore, if the technology in Patent Document 2 is applied, it is believed that a two-dimensional graphic representing an analog defect can be generated using an image that reflects a circuit pattern as the object, and the image of areas where the defect is not reflected can be pasted. That is, it is believed that by specifying the shape and color to generate a two-dimensional graphic, it is possible to synthesize a two-dimensional graphic from an image that only shows the normal area.

[0048] Therefore, it is possible to generate multiple learning images that reflect two-dimensional graphics and circuit patterns representing simulation defects. However, the technology applied in Patent Document 2 does not consider the positional relationship between the circuit pattern and the defects, and therefore it is impossible to generate learning images according to the type of defect appearance and the positional relationship of the defects on the circuit pattern.

[0049] If learning images can be generated based on the type of defect appearance and the location relationship of defects on the circuit pattern, then in the event of deterioration in the accuracy of automatic classification of defect types, learning images required for model relearning can be generated efficiently, which helps to quickly respond to immediate adverse situations when classification accuracy deteriorates.

[0050] Therefore, in the implementation described later, when the accuracy of the automatic classification of defect types deteriorates, learning images are automatically generated based on images collected during the semiconductor product manufacturing process, according to the type of defect appearance and the positional relationship of defects on the circuit pattern. This assists data scientists in creating the learning dataset required for model learning. According to the implementation described later, data scientists can reduce the workload of generating learning datasets and shorten the processing time when accuracy deteriorates. Even in situations where it is not possible to capture a large number of images required for machine learning, such as on a multi-variety, low-volume production line, they can quickly address and handle the accuracy deterioration of machine learning.

[0051] The defect classification apparatus described later assists, for example, in the automatic generation of a training dataset. Therefore, an example of automatically generating training images based on images collected during the semiconductor product manufacturing process will be described in detail. Examples of images collected during the semiconductor product manufacturing process include images mapping various defects on a semiconductor wafer produced during the manufacturing process. When the surface of the semiconductor wafer is photographed, the appearance of various defects and circuit patterns drawn on the semiconductor wafer are reflected in the image.

[0052] When using machine learning to automatically classify the types of defects reflected in such images, the machine learning program takes an image and a learned model as input, uses the machine learning algorithm and the learned model to classify the types of defects reflected in the image, and outputs the classification results through text, etc. Specifically, if a foreign object is mapped in an image, and the text output is "foreign object" as the input image, then the machine learning algorithm and the learned model can correctly classify it.

[0053] Machine learning models sometimes suffer from accuracy degradation based on images, necessitating model relearning. Relearning involves re-collecting images and generating training data. However, in actual semiconductor manufacturing processes, it is sometimes difficult to prepare sufficient training datasets for model learning. Therefore, in the implementation described later, the number of images is increased by performing the processing described later on the already captured images, thereby automatically generating the training images required for model relearning.

[0054] In the following description, an image containing defects obtained by photographing a chip (die) with defects on a semiconductor wafer is called a learning image. An image without defects obtained by photographing a chip without defects (reference chip) is called a "first reference image." Furthermore, an image representing a region of a circuit pattern, extracted from a reference image using general image region segmentation methods, or an image representing design information of a circuit pattern manufactured during the semiconductor product design process, is called a "circuit pattern recognition image." Additionally, the types of defects that are predetermined and should be classified as defects are called "defect types."

[0055] [Specific examples of images]

[0056] Figure 1 These are illustrations showing specific examples of various images applied to the embodiments described later.

[0057] Generally, a semiconductor wafer (not shown) is formed in the shape of a circular plate. On the surface of the semiconductor wafer, multiple rectangular chips are arranged along the X direction (lateral) and Y direction (vertical). Multiple chips formed on a single semiconductor wafer generally have the same circuit structure. Figure 1 The learning image 20-A is an example of an image obtained by taking a partial segment of the inspection area of ​​a semiconductor wafer, which becomes a chip, using SEM. The learning image 20-A includes circuit pattern areas 22-A1 and 22-A2 corresponding to a pair of elongated circuit patterns, and a defect area 26-A corresponding to a defect that occurs between them.

[0058] Furthermore, "circuit pattern" refers to an area containing semiconductors. Additionally, in the following description, multiple components or information having the same or similar function or meaning may sometimes be referred to using the same reference numerals "-" and alphanumeric characters, such as in "circuit pattern areas 22-A1, 22-A2". However, when it is not necessary to distinguish these multiple components, such as in "circuit pattern area 22", "-" and alphanumeric characters may sometimes be omitted.

[0059] Learning images 20-A and similar images are collected during the semiconductor product manufacturing process. They can maintain pixel values ​​in a two-dimensional arrangement, both vertically and horizontally, and can hold multiple channels. The data type of each pixel value is, for example, an unsigned integer. The file format of the learning images 20 can be PNG. Furthermore, as illustrated in learning image 20-A, the learning images 20 include the appearance of the defect region 26 and the appearance of the circuit pattern region 22. Moreover, the learning images 20 may also include the appearance of other elements. To uniquely identify each learning image 20, a learning image ID is assigned.

[0060] Furthermore, referring to image 30-A (the first reference image), is an example of an image obtained by SEM imaging of a segmented region corresponding to the learning image 20-A on another chip located near the chip associated with the learning image 20-A. Image 30-A includes a pair of circuit pattern regions 32-A1 and 32-A2, identical to the circuit pattern regions 22-A1 and 22-A2 in the learning image 20-A. However, image 30-A does not include defective regions such as defective region 26-A.

[0061] Thus, reference images 30-A, etc., are images collected during the semiconductor product manufacturing process. They can maintain pixel values ​​in a two-dimensional arrangement (vertical and horizontal) and hold multiple channels. The data type of each pixel value is, for example, an unsigned integer. The file format of reference images 30 can be PNG. To uniquely identify each reference image 30, a reference image ID is assigned. Furthermore, reference images 30 are not necessarily limited to images obtained by photographing an actual chip. For example, they can also be images that visualize the design information of the circuit pattern corresponding to the portion of the learning image 20, based on the design information of the circuit pattern created during the semiconductor product design process.

[0062] The circuit pattern recognition image 40-A represents the result of recognizing the circuit pattern regions 32-A1 and 32-A2 (circuit pattern regions within the first reference image) in the reference image 30-A. In the illustrated example, the circuit pattern recognition image 40-A has circuit pattern regions 42-A1 and 42-A2 (circuit pattern regions within the circuit pattern recognition image) that correspond to the circuit pattern regions 32-A1 and 32-A2, respectively.

[0063] Thus, the circuit pattern recognition image 40-A, etc., is an image representing a region of a circuit pattern. It is an image that can be extracted from the reference image 30 and visualized using a general image region segmentation method. The circuit pattern recognition image 40 can also maintain pixel values ​​in a two-dimensional arrangement (vertical and horizontal) and hold multiple channels. The data type of each pixel value is, for example, an unsigned integer. Furthermore, the circuit pattern recognition image 40 is not necessarily limited to an image extracted from the reference image 30; for example, it can be an image that visualizes the design information of the circuit pattern corresponding to the portion of the learning image 20 based on design information of the circuit pattern produced in the design process of a semiconductor product.

[0064] The file format of the circuit pattern recognition image 40 is, for example, PNG. Furthermore, in the circuit pattern recognition image 40, the pixel value of the circuit pattern region 42 can be set to, for example, "255", and the pixel value of the portion outside the circuit pattern region 42 can be set to, for example, "0". Thus, the region of the circuit pattern region 42 can be determined based on the pixel values ​​from the circuit pattern recognition image 40. To uniquely identify each circuit pattern recognition image 40, a circuit pattern recognition image ID is assigned to it.

[0065] Defect region image 50-A represents the result of identifying defect region 26-A in learning image 20-A. In the illustrated example, defect region image 50-A has defect region 56-A (the defective region within the defect region image) and background region 58-A. The pixel value of background region 58-A is "0", and the pixel value of defect region 56-A is "1" or higher. The shape of defect region 56-A corresponds to defect region 26-A in learning image 20-A.

[0066] Thus, defect region image 50-A, representing the defective region, is generated by learning the differences between image 20 and reference image 30. Defect region image 50 can maintain pixel values ​​in a two-dimensional arrangement (vertical and horizontal) and retain multiple channels. The data type of each pixel value is, for example, an unsigned integer. The file format of defect region image 50 can be PNG.

[0067] Referring to image 30-B (the second reference image), it is an example of an image obtained by SEM imaging of a segmented region of a chip in a semiconductor wafer different from the learning image 20-A and reference image 30-A. However, reference image 30-B does not include defective regions such as defective region 26. In the illustrated example, reference image 30-B includes a pair of circuit pattern regions 32-B1 and 32-B2 (circuit pattern regions in the second reference image). Furthermore, the shapes of these circuit pattern regions 32-B1 and 32-B2 are different from those of the circuit pattern regions 32-A1 and 32-A2 in reference image 30-A.

[0068] Circuit pattern recognition image 40-B represents the result of recognizing circuit pattern regions 32-B1 and 32-B2 in reference image 30-B. In the illustrated example, circuit pattern recognition image 40-B has circuit pattern regions 42-B1 and 42-B2 that correspond to circuit pattern regions 32-B1 and 32-B2, respectively.

[0069] Composite image 70-B is the result of combining reference image 30-B with defect area image 50-A. In the illustrated example, composite image 70-B includes a pair of circuit pattern regions 72-B1 and 72-B2 (circuit pattern regions within the composite image) and a defect region 76-B (defect region within the composite image). Circuit pattern regions 72-B1 and 72-B2 have the same shape as circuit pattern regions 32-B1 and 32-B2, respectively, and defect region 76-B has the same shape as defect region 56-A. Composite image 70-B and other composite images 70 can also adopt the same data format as learning image 20.

[0070] [Data Structures]

[0071] Next, examples of data structures used in the embodiments described below will be explained. In the following description of data structures, data is represented in tabular form and graphically. In tabular form, each column in the table is called a field, and each row is called a record.

[0072] Figure 2 This represents an example of the structure of image management table 111.

[0073] In addition, Figure 2 In this table, each row is called a record, and each column is called a field. Image management table 111 has fields 111a to 111e. Field 111a stores the device ID used in the semiconductor product manufacturing process. However, field 111a may contain not only the device ID but also the device name. The data type of field 111a is, for example, string.

[0074] Field 111b stores learning images 20 collected during the semiconductor product manufacturing process (e.g., Figure 1 The identification information assigned to the learning image 20-A shown is the learning image ID. However, field 111b may contain not only the learning image ID but also the filename of the learning image 20. The data type of field 111b may be, for example, a string.

[0075] Field 111c stores reference images 30 collected during the manufacturing process of semiconductor products (e.g., ...). Figure 1 The identification information assigned to the reference image 30-A is the reference image ID. However, field 111c may also contain not only the reference image ID but also the filename of the reference image 30. The data type of field 111c is, for example, a string.

[0076] Field 111d stores the defect types for defect regions 26 in the learning image 20. Defect types include, for example, "foreign object" or "scratch." Defect types can be selected from a pre-defined pool of candidates, or data scientists can input any defect type. The data type of field 111d is, for example, a string.

[0077] The circuit pattern recognition image 40 is stored in field 111e (e.g., Figure 1 The circuit pattern recognition image 40-A shown is assigned the recognition information, namely the circuit pattern recognition image ID. However, field 111e may also contain not only the circuit pattern recognition image ID but also the filename of the circuit pattern recognition image 40. The data type of field 111c is, for example, a string. However, the fields described above are just examples and are not limited to these examples; any information can be stored in any field that may be generated for image management.

[0078] In each record (row) of the image management table 111, the learning image 20, specified by field 111b of the learning image ID, and the reference image 30, specified by field 111c of the reference image ID, are images obtained from the interior of each chip on the same semiconductor wafer from relatively identical positional relationships. For example, in Figure 2 In the record with device ID “AAA” in field 111a, the learning image 20 with “defect_img_1” is specified in field 111b of the learning image ID.

[0079] The learning image 20 is an image obtained from the coordinates (X, Y) of a chip DA (not shown). Here, "coordinates (X, Y)" refers to the coordinates within the chip. On the other hand, a reference image 30 is specified as "ref_img_1" in the reference image ID field 111c. This reference image 30 is, for example, an image obtained from the same coordinates (X, Y) in another chip DB (not shown). Here, when the chips on the semiconductor wafer depict the same circuit pattern, the circuit pattern of the learning image 20 is the same as the circuit pattern of the reference image 30.

[0080] In this way, these images are obtained so that the circuit patterns in learning image 20 and reference image 30 are identical. Furthermore, the aforementioned coordinates (X, Y) can also be applied by the defect classification system 1 (referencing...). Figure 10 The coordinates detected by images such as ) are used. That is, the learning image 20 and the reference image 30 can also be images extracted from images with relatively wide regions using general image region segmentation methods. Thus, as Figure 1 As shown, circuit pattern regions 22, 32, and 42 of the same shape are depicted in learning image 20, reference image 30, and circuit pattern recognition image 40.

[0081] Figure 3 This is an explanatory diagram showing the positional relationship between the circuit pattern area 22 and the defect area 26.

[0082] The positional relationship between the circuit pattern area 22 and the defect area 26 can be broadly categorized into "contact" and "non-contact." "Contact" means that the defect area 26 is in contact with the circuit pattern area 22, while "non-contact" means that the defect area 26 is not in contact with the circuit pattern area 22. Figure 3 In the example, learning image 20-A was classified as "non-contact", while the other learning images 20-C, 20-D, 20-E, and 20-F were classified as "contact".

[0083] Furthermore, "contact" is categorized into "contact with inclusion" and "contact without inclusion." "Inclusion" means that the defective region 26 is contained within the circuit pattern region 22. On the other hand, "contact without inclusion" means that a portion of the defective region 26 is located outside the circuit pattern region 22. Figure 3 In the example, learning image 20-F was classified as "contact, containment", while learning images 20-C, 20-D, and 20-E were classified as "contact, non-contact".

[0084] Additionally, in the case of "contact, non-inclusion," the "circuit direction" and "number of lines" are also considered. "Circuit direction" refers to the direction of the long side of the circuit pattern area 22. Figure 3 In the example, the "circuit direction" for learning images 20-C and 20-D is "vertical," while the "circuit direction" for learning image 20-E is "horizontal." Additionally, "number of lines" refers to the number of circuit pattern areas 22 that intersect with defect area 26. Figure 3 In the example, the number of "strips" for learning images 20-C and 20-E is "1", and the number of "strips" for learning image 20-D is "2".

[0085] Figure 4 This is an explanatory diagram of the definition information 131 regarding the location relationship of defect types.

[0086] The defect type location relationship definition information 131 defines the location relationship between the circuit pattern area 22 and the defect area 26 in the learning image 20, and has fields 131a, 131b, and 131c. Field 131a represents the location relationship between the circuit pattern area 22 and the defect area 26. In the three records of the defect type location relationship definition information 131, field 131a respectively... Figure 3 The text describes "contact / inclusion," "contact / non-inclusion," and "non-contact." Field 131b indicates whether the circuit includes the defect area 26 in each positional relationship. The data type of field 131b is, for example, Boolean. Figure 4 As shown, this information is "True" when the positional relationship is "contact" or "includes", and "False" otherwise.

[0087] Field 131c indicates the number of contacting circuits in each positional relationship, i.e., the range of the number of circuit pattern areas 22 in contact with defect area 26. This information is "1" when the positional relationship is "contact, contain", "0" when the positional relationship is "non-contact", and "more than 1" when the positional relationship is "contact, non-contact". The data type of field 131c is, for example, Integer or String. Furthermore, the defect type positional relationship definition information 131 is not limited to... Figure 4 As shown in the example, any information that may arise in the relationship between the circuit pattern area 22 and the defect area 26 can also be stored in any field.

[0088] Figure 5 This is a diagram illustrating the structural example of Table 136, which shows the results of the determination of the location relationship of defect types.

[0089] The Defect Type Location Relationship Determination Result Table 136 describes the relationship between defect area 26 and circuit pattern area 22, and has fields 136a to 136i. Field 136a stores the ID of each record in the Defect Type Location Relationship Determination Result Table 136, i.e., the table ID. The data type of field 136a is, for example, Integer.

[0090] Field 136b stores the learning image ID used to identify the learning image 20 associated with this record. This learning image ID is related to image management table 111 (see reference). Figure 2 The content of field 111b corresponds to the data of field 136b. The data type of field 136b is, for example, string. Field 136c stores the circuit pattern recognition image ID used to identify the circuit pattern recognition image 40. This circuit pattern recognition image ID corresponds to the image management table 111 (see reference...). Figure 2 The content of field 111e corresponds to this. The data type of field 136c is, for example, string.

[0091] like Figure 1 As shown, a circuit pattern recognition image 40 contains one or more circuit pattern regions 42. The tag IDs assigned to each of these circuit pattern regions 42 are stored in field 136d. That is, multiple tag IDs can be stored for a single circuit pattern recognition image 40. The data type of field 136d is, for example, Integer.

[0092] Field 136e stores the aspect ratio of the circuit pattern corresponding to each tag ID. An aspect ratio of "1" indicates a square circuit pattern, an aspect ratio greater than "1" indicates a horizontally elongated circuit pattern, and an aspect ratio less than "1" indicates a vertically elongated circuit pattern. The data type of field 136e is, for example, Float.

[0093] Field 136f stores the roundness of the circuit pattern shape corresponding to each tag ID. A roundness value of "1" indicates a circular circuit pattern, meaning that as the roundness value decreases from "1", the shape of the circuit pattern deviates further from a circle. The data type of field 136f is, for example, Float.

[0094] In field 136g, regarding the learning image 20 corresponding to field 136b of the learning image ID, the positional relationship determination result is stored. The positional relationship determination result is the defect type positional relationship definition information 131 (refer to...). Figure 4 The positional relationship can be determined using any one of the fields 131a in the image, specifically "contact, contain", "contact, non-contact", or "non-contact". In determining these positional relationships, the positional relationship between the defect region 56 in the defect region image 50 and the circuit pattern region 42 in the circuit pattern recognition image 40 can be used. The data type of field 136g is, for example, a string.

[0095] If the positional relationship determination result is "non-contact", the distance "distance from the circuit center of gravity" is stored in field 136h. "Distance from the circuit center of gravity" refers to defect area 26 (refer to...). Figure 1 The distance in the X and Y directions between the centroid of the nearest circuit pattern region 22 and the centroid of the nearest circuit pattern region 22. The data type of field 136h is, for example, an arrangement or list.

[0096] Additionally, when the positional relationship determination result is "contact" or "inclusion", the "distance to the outside of the circuit pattern" is stored in field 136i. "Distance to the outside of the circuit pattern" refers to the distance in the X and Y directions between the closest points on the outer edge of the defect area 26 and the outer edge of the circuit pattern area 22. The data type of field 136i can be, for example, an arrangement or a list.

[0097] Figure 6 This is a diagram illustrating a structural example of image synthesis setting information 141.

[0098] Image compositing settings 141 include reference image retrieval conditions 141a and compositing position conditions 141b. Reference image retrieval conditions 141a are used to determine the reference image 30-B (reference image 30-B). Figure 1 The search criteria are set for the image. In addition, the composite position condition 141b is a setting that determines the positional relationship between the reference image 30-B and the defect area image 50 when they are composited.

[0099] Referring to image search criteria 141a, which includes "circuit num: threshold", "aspect ratio: threshold", and "circularity: threshold",... Figure 1In the example, the number of circuit pattern regions 32 in reference image 30-B is consistent with the number of circuit pattern regions 42 in circuit pattern recognition image 40, both being "2". However, the case where the two numbers differ is also considered. "circuit num:threshold" is a threshold used to determine the maximum value of the difference (absolute value) between the number of circuit pattern regions 32 and the number of circuit pattern regions 42.

[0100] Additionally, "aspect ratio: threshold" is a threshold used to determine the maximum absolute value of the difference between the aspect ratio of the circuit pattern recognition image 40 and the aspect ratio of the reference image 30-B being retrieved. Similarly, "circularity: threshold" is a threshold used to determine the maximum absolute value of the difference between the circularity of the circuit pattern in the circuit pattern recognition image 40 and the circularity of the circuit pattern in the reference image 30-B being retrieved. The data type of "circuit num: threshold" is, for example, Integer. The data types of both "aspect ratio: threshold" and "circularity: threshold" are, for example, Float.

[0101] Additionally, the synthetic position condition 141b includes "distance pixel: threshold", "overlay curcuit: same flag", and "overlay curcuit: threshold". As mentioned above, in the defect type position relationship determination result table 136 (refer to...) Figure 5 The image contains a field 136h for "distance from the center of the circuit" and a field 136i for "distance to the outside of the circuit pattern". "distance pixel: threshold" is a threshold value that determines the permissible difference (absolute value) between the composite position of the composite defect region image 50-A within the image range of image 30-B, based on either the "distance from the center of the circuit" or the "distance to the outside of the circuit pattern" in the learning image 20-A. In the illustrated example, "distance pixel: threshold" is represented by the number of pixels, and its data type is, for example, Integer.

[0102] In addition, as mentioned above, the result of the determination of the location relationship of defect types is shown in Table 136 (refer to...). Figure 5The field 136g contains the positional relationship determination result. Furthermore, when the positional relationship is "contact, non-containment," the number of circuit pattern regions 22 connected to the defect region 26 is determined. The "overlay curcuit: same flag" is a flag that, when the positional relationship determination result in the learning image 20-A is "contact, non-containment," becomes "True" if the synthesized image 70-B should be generated such that the "number of circuit pattern regions connected to the defect region" is the same as in the learning image 20-A; otherwise, it becomes "False." The data type of "overlay curcuit: same flag" is, for example, Boolean.

[0103] Here, we will explain a specific example of the case where "overlay curcuit: same flag" is "True". See Table 136 for the defect type location relationship determination results (refer to...). Figure 5 In the reference table, if the record with field 136a of ID is "5" to "7", then the record with field 136c of ID of circuit pattern recognition image is "design_img_3". Furthermore, in the record with field 136a as "6", the record with field 136d of ID is "2" and the record with field 136g of position relationship determination result is "contact, non-containment".

[0104] This means that among the multiple circuit patterns contained in the circuit pattern recognition image 40 of "design_img_3", only the circuit pattern with tag ID "2" is "contact, non-containment". Additionally, the number of contact circuits, i.e., the number of circuit pattern regions 22 that contact the defect region 26 in the circuit pattern, is "1". Then, since "overlay curcuit: same flag" is "True", the comparison is performed with reference image 30-B and defect region image 50-A (reference). Figure 1 Image synthesis results in "the number of circuit pattern regions connected to the defect region" being "1".

[0105] In "overlay curcuit:threshold", when the defect is determined to be "contact" or "included" in field 136g of the positional relationship determination result table 136, the upper limit number of circuit patterns connected to the defect area is stored. When combining the defect area image 50 with the reference image 30-B, the value of "overlay curcuit:threshold" is used as the upper limit for the number of circuit patterns connected to it, and candidate image areas for image synthesis are retrieved. The data type of "overlay curcuit:threshold" is, for example, Integer. Furthermore, Figure 6 The text format shown is an example of setting information, but it is not limited to this example. Any information can be stored in any text format that can be generated as setting information.

[0106] [Example of screen display]

[0107] Next, the various display screens shown in each embodiment will be described.

[0108] Figure 7 This represents an example of learning to select screen 201 using an image.

[0109] The learning image selection screen 201 includes a learning image selection unit 201a and a positional relationship determination execution button 201c. The learning image selection unit 201a contains multiple checkboxes 201b corresponding to the learning image IDs.

[0110] The learning image IDs corresponding to these checkboxes 201b are obtained from image management table 111 (see reference). Figure 2 The information is obtained from field 111b. When the user selects any checkbox 201b and presses the positional relationship determination execution button 201c, the corresponding learning image 20 is selected as the learning image 20 to be processed.

[0111] Figure 8 This is an example of screen 202 showing the location relationship of defect types.

[0112] The defect type location relationship confirmation screen 202 includes a learning image display bar 202a, a reference image display bar 202b, a judgment result display bar 202c, and an image synthesis execution button 202d. In the learning image display bar 202a and the reference image display bar 202b, the learning image 20 and the reference image 30, respectively, are displayed, showing the location relationship of the defect types determined by the defect type judgment unit 130 (details will be described later).

[0113] The judgment result display column 202c displays the judgment result table 136 (refer to the judgment result table 136) showing the judgment result relationship of defect type and location. Figure 5 The contents of fields 136d (label ID), 136e (aspect ratio), 136f (circularity), 136g (positional relationship determination result), 136h (distance from the center of gravity of the circuit), and 136i (distance from outside the circuit pattern) in the label ID.

[0114] To display reference image 30, refer to image management table 111 (see...) Figure 2 The reference image 30, corresponding to reference image ID111c in row ID111b of the learning image, is retrieved and displayed. The user can confirm the location relationship of the defect type on screen 202, and can also confirm the location of the image on screen 201 (see reference 201). Figure 7 The table 136 shows the result of determining the location relationship of the defect type corresponding to the learning image 20 specified in the table. This allows the user to determine whether to perform the generation process of the synthetic image 70.

[0115] By including a defect type location relationship confirmation screen 202 in the user interaction, it is possible to prevent the generation of a composite image 70 that differs from the user's intention. That is, the execution of the composite image 70 generation process can be canceled after the user confirms the defect type location relationship confirmation screen 202.

[0116] Figure 9 This is an example of a composite image confirmation screen 203.

[0117] The composite image confirmation screen 203 includes a composite image display bar 203a, which displays the composite image 70 described above. This allows the user to confirm the appearance of the composite image 70.

[0118] [First Implementation Method]

[0119] <Structure of the First Embodiment>

[0120] Figure 10 This is a block diagram illustrating a structural example of the defect classification system 1 according to the first embodiment.

[0121] exist Figure 10 In this system, the defect classification system 1 includes a defect classification server 100 (defect classification device, computer) and a display unit 200. The display unit 200 outputs various information by communicating with the defect classification server 100.

[0122] The defect classification server 100 includes a data management unit 110, a defect area image generation unit 120 (defect area image generation process), a defect type determination unit 130 (defect type determination process), an image acquisition and synthesis unit 140, a data acquisition unit 150, a positional relationship acquisition unit 160 (positional relationship acquisition process), and a synthesized image acquisition unit 170. The data management unit 110 stores an image management table 111, a learning image 20, a reference image 30, and a circuit pattern recognition image 40.

[0123] The data acquisition unit 150 includes a data input unit 151, through which various types of data are input. The various types of data stored in the aforementioned data management unit 110 are data collected during the semiconductor product manufacturing process or data input by data scientists, and are input from the data input unit 151.

[0124] More specifically, the image 20 used for learning to recognize images 20, 30, and 40 based on circuit patterns is data collected during the semiconductor product manufacturing process, and the image management table 111 is data entered by a data scientist. The image 20 used for learning to recognize images 30, 30, and 40 based on circuit patterns can also be images generated during the manufacturing processes of various semiconductor products. Therefore, in the aforementioned image management table 111 (refer to...) Figure 2 In this process, the IDs of the newly collected learning image 20, reference image 30, and circuit pattern recognition image 40 are stored in fields 111b, 111c, and 111e. Data scientists can store these IDs.

[0125] The defect region image generation unit 120 includes an image acquisition unit 121 and a defect region extraction unit 122. The image acquisition unit 121 acquires a learning image 20 defined in the image management table 111 and a corresponding reference image 30 from the data management unit 110. The defect region extraction unit 122 calculates the image differences between the learning image 20 and the reference image 30, and generates an image representing the defect region 56 (reference image 30). Figure 1 The defect region image 50 is a background region image 58 that covers the defect region 56. In other words, the defect region image 50 becomes an image that covers the background region 58 other than the defect region 56. In the calculation of the defect region extraction unit 122, the difference in pixel values ​​between the learning image 20 and the reference image 30 can also be used.

[0126] The defect type determination unit 130 determines the defect region 56 (refer to) of the defect region image 50 based on the circuit pattern recognition image 40 and the defect type positional relationship definition information 131. Figure 4 The positional relationship between the circuit pattern region 42 in the circuit pattern recognition image 40 and the circuit pattern recognition image 40 (refer to...) Figure 3 ).

[0127] The circuit pattern recognition image acquisition unit 132 acquires the circuit pattern recognition image 40 and the defect type location relationship definition information 131. When acquiring the circuit pattern recognition image 40, it refers to the image management table 111 (refer to...). Figure 2 ), obtain the circuit pattern recognition image 40 corresponding to field 111b of the learning image ID and field 111e of the circuit pattern recognition image ID (refer to Figure 1 ).

[0128] The tag generation unit 133 recognizes the circuit pattern region 42 of the circuit pattern recognition image 40 (refer to...). Figure 1 ) Perform markup appending processing. For example, when appending to Figure 1 When the circuit pattern recognition image 40-A shown was subjected to additional labeling processing, “1” was assigned as the label ID to the circuit pattern area 42-A1, and “2” was assigned as the label ID to the circuit pattern area 42-A2.

[0129] Considering the case where the circuit pattern recognition image 40 contains three or more circuit pattern regions, the tag generation unit 133 performs tagging and additional processing to assign different tag IDs to each circuit pattern region 42. The tag attribute information calculation unit 134 calculates the aspect ratio and circularity of each circuit pattern region 42 that has been assigned a tag ID.

[0130] The defect type location relationship determination unit 135 determines the location relationship between the defect region 26 and the circuit pattern region 22 in the learning image 20 based on the location relationship between the defect region 56 and each circuit pattern region 42. That is, firstly, the defect type location relationship determination unit 135 obtains a first determination result as to whether the circuit pattern region 22 is adjacent to the defect region 26. Then, if the first determination result is positive, the defect type location relationship determination unit 135 obtains a second determination result as to whether the defect region 26 is included in the circuit pattern region 22.

[0131] Then, the defect type location relationship determination unit 135 calculates the distance (X-direction and Y-direction distance) from the defect area 56 to the outside of the circuit pattern when the location relationship between the two is "contact" or "inclusion". This calculation result corresponds to the defect type location relationship determination result table 136 (see...). Figure 5 The content of field 136i. Furthermore, when the positional relationship between the two is "non-contact," the defect type positional relationship determination unit 135 calculates the distance (X-direction and Y-direction distance) from the centroid of the defect region 56 to the centroid of each circuit pattern region 42. This calculation result corresponds to the defect type positional relationship determination result table 136 (refer to...). Figure 5The content of field 136h is as follows. The defect type location relationship determination unit 135 outputs the calculated location relationship and distance as the defect type location relationship determination result table 136.

[0132] The image acquisition and compositing unit 140 includes an image acquisition unit 142 for compositing and an image compositing unit 143 (image compositing process). The image acquisition unit 142 for compositing determines the result based on the defect type and location relationship table 136 (refer to...). Figure 5 ) and image compositing settings 141 (refer to Figure 6 ), obtain reference image 30-B (refer to Figure 1 The image compositing unit 143 combines the defect region 56 of the defect region image 50 with the reference image 30-B to generate a composite image 70.

[0133] The reference image 30-B acquired by the image acquisition unit 142 for compositing can be an image whose circuit pattern count, aspect ratio, and circularity differ from the circuit pattern area 32 of the reference image 30. When the image acquisition unit 142 searches for the reference image 30-B, it refers to the defect type and location relationship determination result table 136 (see reference). Figure 5 The label ID field 136d, aspect ratio field 136e, and roundness field 136f in the label ID field.

[0134] After the image acquisition unit 142 retrieves and acquires the reference image 30-B, the image is retrieved from the image management table 111 (reference). Figure 2 The image compositing unit 143 retrieves the reference image 30-B (referring to field 111c of the reference image ID) and the circuit pattern recognition image 40 (field 111e of the circuit pattern recognition image ID) corresponding to the row of the reference image 30-B. The image compositing unit 143 prioritizes the defect region image 50 (placing the defect region image 50 on top) and composites the reference image 30-B and the defect region image 50. During image compositing, the result table 136 (referring to the defect type positional relationship determination table) is used. Figure 5 The positional relationship determination result field 136g is used to perform a pixel search on the circuit pattern recognition image 40 to retrieve the pixel coordinates that meet the determination conditions.

[0135] The determination criteria may include the positional relationship determination result (field 136g) of the defect type positional relationship determination result table 136, the centroid distance (field 136h) from the circuit centroid, and the distance to the outside of the circuit (field 136i). If a pixel coordinate that meets the determination criteria can be found, the image synthesis unit 143 projects the pixel coordinate onto the reference image 30-B and performs image synthesis of the defect area 56 on the reference image 30-B.

[0136] The position relationship acquisition unit 160 causes the display unit 200 to display a defect type position relationship confirmation screen 202 (see reference) that shows the defect type position relationship determination result table 136 determined by the defect type determination unit 130. Figure 8 ).

[0137] The composite image acquisition unit 170, in order to display the composite image confirmation screen 203 (see reference) via the display unit 200, Figure 9 The image 70 generated by the image acquisition and compositing unit 140 is displayed, and the image selection screen 201 (see reference) is used to select the image for learning. Figure 7 The composite image 70 is generated from the learning image 20 specified in the image acquisition and synthesis unit 140. The user can confirm the appearance of the composite image 70, which is the output result of the image acquisition and synthesis unit 140, by using the composite image confirmation screen 203.

[0138] As described above, the display unit 200 displays a learning image selection screen 201 (see reference). Figure 7 ), Defect type and location relationship confirmation screen 202 (refer to) Figure 8 ) and the composite image confirmation screen 203 (see Figure 9 In the learning image selection screen 201, the user selects the learning image 20 that has already been captured as the object for image compositing. In the defect type location relationship confirmation screen 202, the user can view the defect type location relationship determination result table 136 of the learning image 20 specified in the learning image selection screen 201 on the screen. In the composite image confirmation screen 203, the user can confirm the appearance of the composite image 70, which is the output result of the image acquisition and compositing unit 140.

[0139] By constructing the defect classification system 1 as described above, data scientists can automatically generate training datasets using various circuit patterns collected in the past during the semiconductor product manufacturing process, based on the type of defect appearance and the location relationship of defects on the circuit patterns. This reduces the workload of generating training datasets required for machine learning model relearning and shortens the processing time for accuracy degradation.

[0140] Next, an example of the hardware structure of the defect classification server 100 will be described.

[0141] Figure 11 This is a block diagram illustrating an example of the hardware structure of the defect classification server 100.

[0142] like Figure 11 As shown, the defect classification server 100 includes a storage device 301, a computing device 303, a memory 304, and a communication device 305, which are interconnected via a bus 308.

[0143] Storage device 301 is preferably composed of non-volatile storage elements such as SSD (Solid State Drive) or hard disk drive. Storage device 301 stores a program 302 that specifies the operation of computing device 303 and various information used or generated by computing device 303. The information stored in storage device 301 includes the aforementioned image management table 111, learning image 20, reference image 30, circuit pattern recognition image 40, defect area image 50, defect type location relationship definition information 131, defect type location relationship determination result table 136, image synthesis setting information 141, and synthesized image 70. Memory 304 is preferably composed of volatile storage elements such as RAM (Random Access Memory).

[0144] The arithmetic unit 303 is preferably configured with a processor such as a CPU (Central Processing Unit). The arithmetic unit 303 reads the program 302 stored in the storage device 301 into the memory 304 for execution. Thus, it is possible to achieve... Figure 10 The data management unit 110, defect area image generation unit 120, defect type determination unit 130, image acquisition and synthesis unit 140, data acquisition unit 150, positional relationship acquisition unit 160, and synthesized image acquisition unit 170 each have their respective functions. The communication device 305 can communicate with the network 310 via the network. Figure 10 It communicates with external devices such as the display unit 200 shown.

[0145] <Operation of the First Embodiment>

[0146] Next, the operation of the first embodiment will be explained.

[0147] Figure 12 This is a flowchart representing an example of a processing routine executed by the defect classification server 100.

[0148] exist Figure 12 In step S12, image acquisition processing is performed. Specifically, the image acquisition unit 121, referring to the image management table 111, acquires the image from the learning image selection screen 201 (see reference). Figure 7 The learning image 20 and its corresponding reference image 30 are specified in the learning image ID.

[0149] For example, such as Figure 7 As shown in the learning image selection screen 201, suppose the learning image ID "defect_img_1" is selected in the checkbox 201b of the learning image selection unit 201a. In this case, the image acquisition unit 121 retrieves the image from the image management table 111 (see...). Figure 2 The record corresponding to “defect_img_1” in the image management table 111 is extracted from field 111b of the image ID. Figure 2 “defect_img_1” corresponds to “ref_img_1” as the reference image ID, “foreign object” as the defect type, and “design_img_1” as the circuit pattern identification image ID.

[0150] As a method for extracting records, there are methods for retrieving records whose strings of the learning image IDs are completely identical, methods for retrieving records whose strings of the image IDs are partially identical, and any retrieval method can be used. In the following description, the learning image 20 and the reference image 30 obtained in step S12 will be referred to as learning image 20-A and reference image 30-A, respectively. These images may also be considered, for example, as... Figure 1 The image shown.

[0151] Next, when the process proceeds to step S14, the defect region extraction unit 122 calculates the difference between the learning image 20-A and the reference image 30-A, generating a defect region image 50. The calculation of the difference refers, for example, calculating the difference between the pixel values ​​of the learning image 20-A and the reference image 30-A. In the following description, the defect region image 50 generated by the calculation will be referred to as the defect region image 50-A. This defect region image 50-A can also be considered, for example, as... Figure 1 The image shown.

[0152] Next, when the process proceeds to step S16, the circuit pattern recognition image acquisition unit 132 acquires the circuit pattern recognition image 40 and the defect type location relationship definition information 131 (see reference). Figure 4 The circuit pattern recognition image 40 is the image corresponding to the learning image 20-A and the reference image 30-A, and will be referred to as the circuit pattern recognition image 40-A in the following description. This circuit pattern recognition image 40-A can also be considered, for example, as... Figure 1 The image shown.

[0153] Next, when the process proceeds to step S18, the tag generation unit 133 performs tag number generation processing. That is, it searches the pixel values ​​of the circuit pattern recognition image 40-A and assigns tag numbers to the pixels representing the circuit pattern region 42. As a pixel-specific tag number assignment process, for example, an image processing algorithm can be used to group pixels with the same pixel value by performing neighbor-to-neighbor processing starting from a certain pixel. Figure 1 In the example of the circuit pattern recognition image 40-A shown, for example, the circuit pattern area 42-A1 can be assigned the label number "1" and the circuit pattern area 42-A2 can be assigned the label number "2".

[0154] Next, when processing proceeds to step S20, the tag attribute information calculation unit 134 performs calculation processing of attribute information, etc. That is, the tag attribute information calculation unit 134 calculates the number of circuit pattern areas 42 (the number of tags assigned), attribute information of pixel sets assigned the same tag number, etc., based on the tag numbers previously assigned in step S18. Figure 1 In the example shown, “1” and “2” are assigned as label numbers, so the number of circuit pattern areas 42 is “2”.

[0155] In addition, the label attribute information calculation unit 134 calculates the aspect ratio and roundness as attribute information of the pixel set assigned the same label number. Regarding the calculation of the aspect ratio, taking a label number as the object, the minimum and maximum values ​​of the X and Y coordinates of the region of pixels assigned the same label number are calculated. When the minimum value of the X coordinate is set to Xmin, the maximum value of the X coordinate is set to Xmax, the minimum value of the Y coordinate is set to Ymin, and the maximum value of the Y coordinate is set to Ymax, the calculation can be performed by the following formula (1).

[0156] [Formula 1]

[0157]

[0158] The label attribute information calculation unit 134 stores the aspect ratio calculation results in the defect type location relationship determination result table 136 (see reference). Figure 5 In field 136e of ) . In addition, regarding the calculation of circularity, it is possible to calculate the perimeter and area of ​​the region of pixels assigned the same label number, using a label number as the object, by the following formula (2).

[0159] [Formula 2]

[0160]

[0161] The label attribute information calculation unit 134 stores the roundness calculation result in field 136f of the defect type location relationship determination result table 136. Furthermore, in step S20, the label attribute information calculation unit 134 is not limited to aspect ratio and roundness; it can calculate any attribute information.

[0162] Next, when the process enters step S22, the defect type location relationship determination unit 135 performs the location relationship determination process of the defect area. That is, the defect type location relationship determination unit 135 determines the location relationship between the circuit pattern area 42 and the defect area 56 based on the defect area image 50-A and the circuit pattern recognition image 40-A which has been assigned a label number.

[0163] like Figure 4 As shown, the result of determining the positional relationship is one of "contact, inclusion", "contact, non-inclusion", and "non-contact". Furthermore, in the case of "contact, non-inclusion", the defect type positional relationship determination unit 135 also calculates the number of contact circuits (the number of circuit pattern areas 42 contacted by the defect area 56).

[0164] To determine the positional relationship, the defect type positional relationship determination unit 135 compares each pixel of the defect region 56-A in the defect region image 50-A with each pixel located at the same position in the circuit pattern recognition image 40-A, which has been assigned a label number. Furthermore, if all pixels of the defect region 56-A are not included in the circuit pattern region 42, which has been assigned a label number, the positional relationship determination result of the defect type positional relationship determination unit 135 is "non-contact".

[0165] Furthermore, if all pixels of the defect area 56-A are contained within the circuit pattern area 42 which has been assigned a label number, the positional relationship determination result of the defect type positional relationship determination unit 135 is "contact, contain".

[0166] Furthermore, if a portion of the pixels in defect area 56-A are contained within a circuit pattern area 42 that has been assigned a tag number, the positional relationship determination result of defect type positional relationship determination unit 135 is "contact, non-inclusion". In this case, defect type positional relationship determination unit 135 extracts the tag numbers corresponding to the pixels in circuit pattern area 42 that overlap with the pixels of defect area 56-A. The number of extracted tag numbers is called the number of contact circuits. Defect type positional relationship determination unit 135 stores these positional relationship determination results in defect type positional relationship determination result table 136 (see reference). Figure 5 In field 136g.

[0167] Furthermore, when the positional relationship determination result is "non-contact," the defect type positional relationship determination unit 135 calculates the pixel centroid coordinates of the defect region 56-A in the defect region image 50-A. Moreover, in the circuit pattern recognition image 40-A, which has been assigned tagged numbers, the defect type positional relationship determination unit 135 calculates the pixel centroid coordinates of each circuit pattern region 42. Then, the defect type positional relationship determination unit 135 calculates the pixel distance (X-direction and Y-direction distance) between the pixel centroid coordinates of the defect region 56-A and the pixel centroid coordinates of each circuit pattern region 42.

[0168] The pixel centroid coordinates of the defect region 56-A in the defect region image 50-A and the pixel centroid coordinates of each circuit pattern region 42 in the circuit pattern recognition image 40-A which has been assigned a label number can be calculated using the following formulas (3) and (4).

[0169] [Formula 3]

[0170]

[0171] [Formula 4]

[0172]

[0173] In equation (3), x represents the X coordinate value and y represents the Y coordinate value. Additionally, f(x, y) is a pixel value, which is the pixel value of defect region 56-A in defect region image 50-A, or the pixel value of each circuit pattern region 42 in circuit pattern recognition image 40-A, which has been assigned a label number. The pixel value f(x, y) is "0" in the background and "1" in the object. Variables i and j are values ​​that are either "1" or "0".

[0174] However, any calculation method other than equations (3) and (4) can be used to calculate the pixel centroid coordinates. The pixel distance between the pixel centroid coordinates of each circuit pattern area 42 and the pixel centroid coordinates of the defect area 56-A can be obtained by calculating the difference between the X and Y coordinate values ​​shown in equation (4). The pixel distance between the pixel centroid coordinates of each circuit pattern area 42 and the pixel centroid coordinates of the defect area 56-A is stored in the defect type position relationship determination result table 136 (refer to...). Figure 5 In the field 136h of “distance from the center of gravity of the circuit”.

[0175] On the other hand, when the positional relationship determination result is determined to be "contact or inclusion", the defect type positional relationship determination unit 135 confirms the pixel values ​​f(x, y) in the up, down, left, and right directions of the defect area image 50 based on the pixel centroid coordinates of the defect area 56-A in the defect area image 50-A, and calculates the distance to the pixel coordinates outside the circuit pattern. The pixel values ​​f(x, y) are confirmed by comparing the image with the circuit pattern recognition image 40-A, which has been assigned a label number. The defect type positional relationship determination unit 135 calculates the distance to the pixel coordinates outside the circuit pattern for both X and Y coordinates. Then, the defect type positional relationship determination unit 135 stores the calculation result in the "distance to outside the circuit pattern" field 136i in the defect type positional relationship determination result table 136.

[0176] exist Figure 12 Next, when the processing enters step S24, the position relationship acquisition unit 160 performs the determination result display processing. That is, the position relationship acquisition unit 160 acquires the defect type position relationship determination result table 136 and displays its contents as the defect type position relationship confirmation screen 202 (see reference). Figure 8 The image is displayed on display unit 200.

[0177] like Figure 8 As shown, the defect type location relationship confirmation screen 202 displays a learning image 20 and a reference image 30. In the example above, learning image 20-A is displayed as learning image 20, and reference image 30-A is displayed as reference image 30. Additionally, the determination result display column 202c displays the contents of the defect type location relationship determination result table 136, which is based on the defect area image 50-A and the circuit pattern recognition image 40-A assigned with a label number. According to the defect type location relationship determination result table 136, the label ID, aspect ratio, roundness, location relationship determination result, distance from the circuit center of gravity, and distance from outside the circuit are displayed.

[0178] exist Figure 8 In the example shown, the learning uses the defect region 26 contained in image 20 (refer to...) Figure 1 The defect area 26 is not included in the circuit pattern area 22. Therefore, the "distance from the center of gravity of the circuit" is calculated, but the "distance to the outside of the circuit" is not calculated, hence the hyphen is displayed. By confirming the results of this defect type positional relationship determination, the user can understand the relationship between the defect area 26 and the circuit pattern area 22 reflected in the learning image 20.

[0179] The above description of steps S12 to S24 focuses on a learning image 20-A, its corresponding reference image 30-A, and a circuit pattern recognition image 40-A. However, in the processing of steps S12 to S24, the defect classification server 100 also performs the same processing on all learning images 20, reference images 30, and circuit pattern recognition images 40 stored in the data management unit 110. Therefore, the defect classification server 100 generates... Figure 5 The results of the determination of the location relationship of the defect types are shown in Table 136.

[0180] exist Figure 12 In the next step, when processing proceeds to step S26, the image acquisition unit 142 for compositing acquires a reference image 30-B for compositing based on the defect region image 50-A, the image compositing setting information 141, and the defect type location relationship determination result table 136. Reference image 30-B is an image having a circuit pattern region 32 different from reference image 30-A, and is an image taken from a semiconductor wafer different from reference image 30-A. Regarding reference image 30-B, it can also be considered, for example, as... Figure 1 The image shown.

[0181] As described above, in image composition setting information 141 (refer to...) Figure 6The image acquisition unit 142 for compositing includes reference image retrieval conditions 141a for retrieving reference image 30-B. Therefore, the image acquisition unit 142 compares the number of circuit pattern regions 42, aspect ratio, and roundness calculated from the circuit pattern recognition image 40-A with the number of circuit pattern regions 32, aspect ratio, and roundness in each reference image 30, and retrieves reference image 30-B that satisfies reference image retrieval conditions 141a.

[0182] by Figure 6 Taking the image retrieval condition 141a as an example, since "circuit num: threshold" is set to "1", the image 30-B needs to include "±1" circuit pattern regions 32 relative to the number of circuit patterns calculated from the circuit pattern recognition image 40-A. Furthermore, since "aspect ratio: threshold" is set to "0.1", the image 30-B needs to include circuit pattern regions 32 with an aspect ratio within the range of "±0.1" relative to the aspect ratio calculated from the circuit pattern recognition image 40-A. Additionally, since "circularity: threshold" is set to "0.1", the image 30-B needs to include circuit pattern regions 32 with a circularity within the range of "±0.1" relative to the circularity calculated from the circuit pattern recognition image 40-A.

[0183] Next, when the process proceeds to step S28, the image compositing unit 143, based on the defect type location relationship determination result table 136, overlays the defect region image 50-A onto the reference image 30-B to generate a composite image 70. The generated composite image 70 is, for example, Figure 1 The composite image 70-B is shown. More specifically, the image compositing unit 143 first acquires a circuit pattern recognition image 40-B corresponding to the reference image 30-B. Next, in order to compare the defect region image 50-A with the circuit pattern recognition image 40-B, the image compositing unit 143 calculates the pixel centroid coordinates of the defect region 56 in the defect region image 50-A and the pixel centroid coordinates of each circuit pattern in the circuit pattern recognition image 40-B.

[0184] Next, the image synthesis unit 143 obtains the positional relationship determination result for the defect region image 50-A, the distance to the circuit center of gravity, and the distance to the outside of the circuit from the defect type positional relationship determination result table 136. Then, using the threshold information set in the synthesis position condition 141b of the image synthesis setting information 141 and the defect region image 50-A, it searches for image regions that meet the threshold conditions in the circuit pattern recognition image 40-B. When an image region that meets the conditions is found, a synthesized image 70 is generated by superimposing the defect region image 50-A with the reference image 30-B.

[0185] Next, when the process enters step S30, the composite image acquisition unit 170 acquires the composite image 70 generated in step S28, and displays it on the composite image confirmation screen 203 of the display unit 200 (see reference). Figure 9 The synthesized image 70 is displayed. Users (e.g., data scientists) can view the generated synthesized image 70 on the screen. Thus, the defect classification server 100 can automatically generate multiple training datasets based on the type of appearance of the defect region 26 and the positional relationship between the circuit pattern regions 22 and 26. Furthermore, it can reduce the user's workload and shorten the processing time when the accuracy of the machine learning model deteriorates.

[0186] [Second Implementation]

[0187] Next, the defect classification device of the second embodiment will be described. The structure of the defect classification device of the second embodiment is similar to that of the defect classification system 1 of the first embodiment (see below) except for the points described below. Figure 10 The same applies. Furthermore, in the description of this embodiment, sometimes the same reference numerals are used for parts corresponding to those in other embodiments described above, and their descriptions are omitted.

[0188] Figure 13 Specific examples of various images applicable to the second embodiment are shown.

[0189] Figure 13 Learning with images 20-A and Figure 1 The image shown also has circuit pattern areas 22-A1 and 22-A2. However, in Figure 13 In the circuit pattern area 22-A1, a portion is missing, and the missing area becomes defect area 26-A. Thus, the state of a partial defect in the circuit pattern area is referred to as a "broken line". That is, the concept of "broken line" in this specification is not limited to the complete interruption of the circuit pattern area 22, but also includes situations such as... Figure 13 That kind of partial loss.

[0190] In the second embodiment, the reference images 30-A and 30-B and the circuit pattern recognition images 40-A and 40-B are... Figure 1 The images shown are the same. Furthermore, Figure 13 The identification results of defect region 26-A in the image are as follows: Figure 13 The defect area shown is similar to image 50-A. Image 50-A of the defect area is similar to... Figure 1 The image shown also has a defect region 56-A and a background region 58-A. Here, the region of a predetermined width that includes the defect region 56-A and its surroundings is referred to as the buffer region 57-A.

[0191] In the second embodiment, a region corresponding to the buffer region 57-A is generated by cutting out the defect region 26-A. Figure 13 The cut-out image 60-A is shown. The cut-out image 60-A has a buffer region 66-A as the cut-out result of the defect region 26-A and a surrounding background region 68-A.

[0192] In this embodiment, the composite image 70-B is based on the reference image 30-B (reference image 30-B). Figure 1 The image of cut-out image 60-A is superimposed on the composite image 70-B. In the illustrated example, the composite image 70-B contains a pair of circuit pattern regions 72-B1 and 72-B2. Moreover, region 77-B corresponds to the buffer region 66-A of cut-out image 60-A. Thus, a composite image 70-B with a portion of circuit pattern region 72-B1 missing is obtained.

[0193] Figure 14 This is a flowchart illustrating an example of a processing routine executed by the defect classification server 100 in the second embodiment.

[0194] exist Figure 14 In the process, the processing in steps S42 to S50 is the same as that in the first embodiment (refer to...). Figure 12 The processing in steps S12 to S20 is the same. That is, in step S42, the image acquisition unit 121 acquires the learning image 20-A and the corresponding reference image 30-A. Then, in step S44, the defect region extraction unit 122 generates a defect region image 50-A based on these acquired images.

[0195] Next, in step S46, the circuit pattern recognition image acquisition unit 132 acquires the circuit pattern recognition image 40-A and the defect type location relationship definition information 131 (refer to...). Figure 4 Next, in step S48, the label generation unit 133 performs label number generation processing. Next, in step S50, the label attribute information calculation unit 134 performs attribute information calculation processing.

[0196] Next, when processing proceeds to step S52, the defect type location relationship determination unit 135 obtains the image management table 111 (refer to) for the learning image 20-A. Figure 2 The value of field 111d for the defect type is then determined. Next, when processing proceeds to step S54, the defect type location relationship determination unit 135 determines whether the defect type of the defect region 26 in the learning image 20-A is "broken line". Here, if the determination is "no", the processing of this routine ends. On the other hand, if the determination is "yes", the processing proceeds to step S56.

[0197] In step S56, the defect type location relationship determination unit 135 determines the location of the broken line in the circuit pattern area 42, i.e., the location of the defect area 56-A, based on the defect area image 50-A and the circuit pattern recognition image 40-A which has been assigned a tag number. The method for obtaining the defect area 56-A can be the same as the method in step S22 of the first embodiment.

[0198] Next, when processing proceeds to step S58, the position relationship acquisition unit 160 displays the determination result of the defect type position relationship determination unit 135, that is, the position of the defect area 56-A, which is the break location. In other words, the position relationship acquisition unit 160 acquires the defect type position relationship determination result table 136 and displays its contents as the defect type position relationship confirmation screen 202 (see reference). Figure 8 The image is displayed on display unit 200.

[0199] Next, when the processing reaches step S60, the image acquisition unit 142 for compositing acquires the reference image 30-B for compositing using the same method as step S26 in the first embodiment. Then, when the processing reaches step S62, the image acquisition unit 142 for compositing generates the cut-out image 60-A (see...). Figure 13 ).

[0200] Next, when the processing reaches step S64, the image synthesis unit 143 extracts candidate positions for synthesizing the cut-out image 60-A from the reference image 30-B based on the defect type position relationship determination result table 136. This extraction process is similar to step S28 of the first embodiment, and can utilize an image comparison method for the defect region image 50-A and the circuit pattern recognition image 40-B. The image comparison conditions can use the conditions set in the synthesis position condition 141b of the image synthesis setting information 141.

[0201] Next, when the processing reaches step S66, the image compositing unit 143 overlays the cut-out image 60-A onto the reference image 30-B to generate a composite image 70 (e.g., composite image 70-B). That is, the image compositing unit 143 uses the positional candidates extracted in step S64 as objects and performs image correlation pattern matching on the cut-out image 60-A onto the reference image 30-B. Then, the cut-out image 60-A is overlaid onto the reference image 30-B, which has a certain degree of image correlation, to generate the composite image 70-B.

[0202] In the image correlation pattern matching described above, pattern matching processing is performed on the shape of the circuit pattern region 62 in the cut-out image 60-A and the shape of the circuit pattern region 32 in the reference image 30-B. This allows image synthesis to be performed at the location of the circuit pattern region 32, which has a shape similar to the original learning image 20-A. Specific methods for image correlation pattern matching include, for example, pre-setting the image template size and image search region, and extracting the pixels with the highest image correlation. Furthermore, any algorithm can be used for the process of extracting pixel regions with similar shapes from two images.

[0203] Next, when the process proceeds to step S68, the composite image acquisition unit 170 acquires the composite image 70 generated in step S66, and displays the composite image confirmation screen 203 on the display unit 200 (see reference). Figure 9 The composite image 70 is displayed in the image. Thus, the defect classification server 100 can automatically generate multiple training datasets even when the defect type in defect region 26 is "broken line".

[0204] [Third Implementation Method]

[0205] Next, the defect classification device of the third embodiment will be described. The structure of the defect classification device of the third embodiment is similar to that of the defect classification system 1 of the first embodiment (see below) except for the points described below. Figure 10 The same applies. Furthermore, in the description of this embodiment, sometimes the same reference numerals are used for parts corresponding to those in other embodiments described above, and their descriptions are omitted.

[0206] The images in this embodiment are the same as those in the first embodiment (see reference). Figure 1 The same applies. However, in this embodiment, the periphery of the circuit pattern region 22 in the learning image 20 is blurred. As a result, the periphery of the circuit pattern region 42 in the corresponding circuit pattern recognition image 40 also becomes blurred.

[0207] Figure 15 This is a flowchart illustrating an example of a processing routine executed by the defect classification server 100 in the third embodiment.

[0208] exist Figure 15 In the process, the processing in steps S82 to S90 is the same as that in the first embodiment (refer to...). Figure 12 The processing in steps S12 to S20 is the same. That is, in step S82, the image acquisition unit 121 acquires the learning image 20-A and the corresponding reference image 30-A. Then, in step S84, the defect region extraction unit 122 generates a defect region image 50-A based on the acquired image.

[0209] Next, in step S86, the circuit pattern recognition image acquisition unit 132 acquires the circuit pattern recognition image 40-A and the defect type location relationship definition information 131 (refer to...). Figure 4 Next, in step S88, the label generation unit 133 performs label number generation processing. Next, in step S90, the label attribute information calculation unit 134 performs attribute information calculation processing.

[0210] Next, when the processing proceeds to step S92, the image synthesis unit 143 applies an edge extraction filter to the circuit pattern recognition image 40-A to generate an edge image of the circuit pattern reflected in the circuit pattern recognition image 40-A. The edge extraction filter can be applied to algorithms used in general image processing. The edge image extracted from the circuit pattern recognition image 40-A is called "edge image 80-A (not shown)". In the edge image 80-A, there is an edge region 82-A (not shown) representing the shape of the circuit pattern region 42. The edge region 82-A is along... Figure 1 The narrow area at the outer edge of the circuit pattern areas 42-A1 and 42-A2 in the diagram.

[0211] Next, when the processing reaches step S94, the image synthesis unit 143 performs image correlation pattern matching between the edge region 82-A of the edge image 80-A and the defect region image 50-A, and calculates the image correlation value in the edge region 82-A. As a specific method for image correlation pattern matching, for example, considering pre-setting the image template size, image search area, etc., the pixels with the highest image correlation are extracted. Furthermore, any algorithm can be used for any process that extracts pixel regions with similar shapes from two images.

[0212] As described above, the defect region image 50-A is generated by learning the image difference between image 20-A and reference image 30-A. At this time, if the circuit pattern in image 20-A is blurred in the image, and the image difference with reference image 30-A is calculated, the pixel values ​​of the circuit pattern are not eliminated, and it is assumed that information about the pixels of the circuit pattern region 22 remains in the defect region image 50-A. Therefore, in step S94, in order to determine whether information about the pixels of the circuit pattern region 22 remains in the defect region image 50-A, image correlation pattern matching is performed between the edge region 82-A and the defect region image 50-A, and an image correlation value is calculated.

[0213] Next, when the process proceeds to step S96, the image compositing unit 143 determines whether the calculated image correlation value is above a predetermined threshold. If the determination is "yes," the process proceeds to step S98, where the image compositing unit 143 performs morphological processing. That is, if the image correlation value is above the threshold, it is assumed that information of the pixels of the circuit pattern region 22 remains in the defect region image 50-A, and therefore processing is performed to remove this pixel information.

[0214] That is, the image synthesis unit 143 removes information about pixels corresponding to the blurred circuit pattern contained in the defect region image 50-A. As a morphological process, it can be applied to the repeated shrinking and dilation processes used in general image processing algorithms.

[0215] Next, when the process proceeds to step S100, the image synthesis unit 143 determines a new defect region image 50-A from the original defect region image 50-A after removing the information of the circuit pattern pixels. If the processing of step S100 ends, or if it is determined to be "no" in step S96, then the processing of steps S102 to S110 is executed. The processing of these steps S102 to S110 is the same as the processing of steps S22 to S30 in the first embodiment.

[0216] That is, in step S102, the defect type location relationship determination unit 135 performs location relationship determination processing for the defect area, and in step S104, the location relationship acquisition unit 160 performs determination result display processing. Next, in step S106, the image acquisition unit 142 acquires the reference image 30-B for compositing, and in step S108, the image compositing unit 143 generates the composite image 70. Then, in step S110, the image acquisition unit 170 displays the composite image confirmation screen 203 (see reference) on the display unit 200. Figure 9 The composite image 70 is displayed in the image.

[0217] [Fourth Implementation Method]

[0218] Next, the defect classification device of the fourth embodiment will be described. The structure of the defect classification device of the fourth embodiment is similar to that of the defect classification system 1 of the first embodiment (see below) except for the points described below. Figure 10 The same applies. Furthermore, in the description of this embodiment, sometimes the same reference numerals are used for parts corresponding to those in other embodiments described above, and their descriptions are omitted.

[0219] The fourth embodiment envisions a scenario where, by effectively utilizing the frequency of defect types collected in past semiconductor inspection lines, the number of images for data expansion is suggested based on the generation patterns of subsequently generated defect types.

[0220] Figure 16 This is a flowchart illustrating an example of a processing routine executed by the defect classification server 100 in the fourth embodiment.

[0221] exist Figure 16 In the process, the processing in steps S122 to S134 is the same as that in the first embodiment (refer to...). Figure 12 The processing in steps S12 to S24 is the same. That is, in step S122, the image acquisition unit 121 acquires the learning image 20-A and the corresponding reference image 30-A. Then, in step S124, the defect region extraction unit 122 generates a defect region image 50-A based on these acquired images. Next, in step S126, the circuit pattern recognition image acquisition unit 132 acquires the circuit pattern recognition image 40-A and the defect type location relationship definition information 131 (refer to...). Figure 4 ).

[0222] Next, in step S128, the label generation unit 133 performs label number generation processing. Next, in step S130, the label attribute information calculation unit 134 performs attribute information calculation processing. Next, when the process proceeds to step S132, the defect type location relationship determination unit 135 performs location relationship determination processing for the defect area. Next, when the process proceeds to step S134, the location relationship acquisition unit 160 performs determination result display processing. Next, when the process proceeds to step S136, the defect type location relationship determination unit 135 acquires the image management table 111 (see reference) for the learning image 20-A. Figure 2 The value of field 111d for the defect type.

[0223] Next, when the process proceeds to step S138, the defect type location relationship determination unit 135 refers to the defect type generation frequency information to obtain the generation frequency of the defect type in the learning image 20-A. Here, regarding the defect type generation frequency information, it is envisioned that the information is obtained by statistically analyzing the generation frequency of defect types collected in the semiconductor inspection line in the past. If this information can be used, the generation frequency of the defect type generated in the past can be obtained according to this information for each defect type.

[0224] Next, when the processing reaches step S140, the image acquisition unit 142 acquires the image compositing setting information 141. Then, the image acquisition unit 142 changes the threshold stored in the reference image retrieval condition 141a in the image compositing setting information 141 based on the frequency of occurrence of the defect type obtained in step S138. Here, we will use "circuit num:threshold", which represents the threshold for the number of circuit patterns in reference image 30-B, as an example for explanation.

[0225] exist Figure 6In the example shown, "circuit num: threshold" is set to "1". However, if the frequency of defect types in the learning image 20-A is high, it is preferable to enrich the learning dataset with as many reference images 30 as possible that map the circuit pattern regions 32 of each type. Therefore, for example, consider changing "circuit num: threshold" to "2". If "circuit num: threshold" is changed to "2", reference images 30 with a range of "±2" circuit pattern regions 32 can be retrieved relative to the number of circuit pattern regions 22 in the learning image 20-A. Therefore, the candidates for the reference image 30-B to be searched can be further broadened.

[0226] When learning a machine learning model, it is envisioned that the number of images reflecting frequently occurring defect types be increased, and the distribution of defect types be considered when preparing the images required for learning. Therefore, by extensively obtaining retrieval candidates with reference to image 30, a large number of synthetic images 70 can be generated. In addition to the aforementioned "circuit num: threshold," which is the threshold for the number of circuit pattern regions 32, the same modification process can be applied to the "aspect ratio: threshold," which represents the aspect ratio of the circuit pattern, and the "circularity: threshold," which represents the circularity of the circuit pattern.

[0227] Next, when the processing proceeds to step S142, the image acquisition unit 142 for compositing acquires the reference image 30-B for compositing based on the modified threshold, using the same method as step S26 in the first embodiment. Then, in steps S144 and S146, steps S28 and S30 in the first embodiment are executed (see reference...). Figure 12 The same process applies. That is, in step S144, the image compositing unit 143 generates a composite image 70. Then, in step S146, the composite image acquisition unit 170 displays the composite image confirmation screen 203 on the display unit 200 (see reference). Figure 9 The composite image 70 is displayed in the image.

[0228] [Effects of the Implementation Method]

[0229] As described above, according to the various embodiments described above, the defect classification device (100) is characterized by comprising: a defect type determination unit 130, which determines the positional relationship between the circuit pattern area 22 and the defect area 26; and an image synthesis unit 143, which synthesizes the defect area image 50 and a second reference image (30-B) based on the positional relationship to generate a synthesized image 70, wherein the first reference image (30-A) is an image obtained by photographing a first semiconductor wafer, and the second reference image (30-B) is an image obtained by photographing a second semiconductor wafer different from the first semiconductor wafer.

[0230] Therefore, appropriate defect classification can be achieved. That is, a synthetic image 70 can be automatically generated based on the relationship between the appearance of the defect region 26 in the learning image 20 and the position of the circuit pattern region 22, thereby automatically generating an appropriate learning dataset. With this automatically generated learning dataset, appropriate defect classification can be achieved. Moreover, this reduces workload and shortens the processing time when the accuracy of the learning dataset deteriorates. In other words, even in situations where it is impossible to take a large number of images required for machine learning, such as on a multi-variety, small-batch production line, each embodiment can quickly handle the deterioration of machine learning accuracy.

[0231] Furthermore, more preferably, the defect type determination unit 130 obtains a first determination result based on the circuit pattern recognition image 40 and the defect region image 50 to determine whether the circuit pattern region 22 is connected to the defect region 26. If the first determination result is positive, it obtains a second determination result to determine whether the defect region 26 is included in the circuit pattern region 22. As a result, the positional relationship between the circuit pattern region 22 and the defect region 26 can be classified more appropriately.

[0232] Furthermore, the defect type determination unit 130 more preferably obtains the direction of the circuit pattern region 22 and the number of circuit pattern regions 22 that are in contact with the defect region 26. Therefore, the positional relationship between the circuit pattern region 22 and the defect region 26 can be more appropriately classified based on the direction of the circuit pattern region 22 and the number of circuit pattern regions 22 that are in contact with the defect region 26.

[0233] Furthermore, the defect type determination unit 130 preferably obtains the distance between the center of gravity of each circuit pattern region 22 and the center of gravity of the defect region 26 based on the shape of each circuit pattern region 22. Therefore, the positional relationship between the circuit pattern region 22 and the defect region 26 can be more appropriately classified based on the distance between the center of gravity of the circuit pattern region 22 and the center of gravity of the defect region 26.

[0234] Furthermore, a more preferable feature is a positional relationship acquisition unit 160, which causes the display unit 200 to display a defect type positional relationship confirmation screen 202 for indicating positional relationships. This allows the user to visually grasp the positional relationship between the circuit pattern area 22 and the defect area 26.

[0235] Furthermore, more preferably, the second reference image (30-B) includes one or more circuit pattern regions (32-B1, 32-B2) within the second reference image, and the composite image 70 includes a composite image circuit pattern region (72-B1, 72-B2) corresponding to the circuit pattern regions (32-B1, 32-B2) within the second reference image and a composite image defect region (76-B) corresponding to the defect region 26. The image synthesis unit 143 determines the positional relationship between the circuit pattern regions (72-B1, 72-B2) within the composite image and the defect region (76-B) within the composite image 70 based on the positional relationship between the circuit pattern region 22 and the defect region 26 in the learning image 20.

[0236] Therefore, the positional relationship between the circuit pattern region (72-B1, 72-B2) and the defect region (76-B) in the synthetic image 70 can be approximated as the positional relationship between the circuit pattern region 22 and the defect region 26 in the learning image 20.

[0237] [Variation Example]

[0238] This invention is not limited to the embodiments described above, and various modifications are possible. The embodiments described above are examples exemplified for ease of understanding and explanation of the invention, and are not necessarily limited to having all the structures described. Furthermore, a portion of the structure of one embodiment can be replaced with the structure of another embodiment, or the structure of another embodiment can be added to the structure of one embodiment. Additionally, a portion of the structure of each embodiment can be deleted, or other structures can be added or replaced. Furthermore, the control lines and information lines shown in the figures represent lines deemed necessary for explanation, and are not limited to representing all control lines and information lines necessary for the product. In practice, it can be considered that almost all structures are interconnected. For example, modifications that can be made relative to the above embodiments are as follows.

[0239] (1) The hardware of the defect classification server 100 in the above embodiments can be implemented by a general computer. Therefore, the programs for performing the processes corresponding to the above block diagrams, flowcharts and other processes can be stored in a storage medium (a computer-readable recording medium containing the program) or distributed via a transmission path.

[0240] (2) Regarding the processing corresponding to each block diagram and flowchart above, and other processing described above, the above embodiments have been described as software processing using a program, but some or all of them may be replaced by hardware processing using ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

[0241] (3) The various processes performed in the above embodiments can be performed by the server computer via a network not shown, and the various data stored in the above embodiments can also be stored on the server computer.

[0242] Explanation of reference numerals in the attached figures

[0243] 20 Learning with Images

[0244] 22 Circuit Pattern Area

[0245] 26 Defect Areas

[0246] 30-A Reference Image (First Reference Image)

[0247] 30-B Reference Image (Second Reference Image)

[0248] Circuit pattern areas 32-A1 and 32-A2 (circuit pattern areas within the first reference image)

[0249] Circuit pattern areas 32-B1 and 32-B2 (circuit pattern areas within the second reference image)

[0250] 40 Circuit Pattern Recognition Images

[0251] 42-A1 and 42-A2 circuit pattern areas (circuit pattern areas within the circuit pattern recognition image)

[0252] 50 defect area images

[0253] 56. Defect Area (Defect area within the image)

[0254] 70 composite images

[0255] 72-B1 and 72-B2 circuit pattern areas (circuit pattern areas within the composite image)

[0256] 76-B Defect Region (Defect Region within Synthetic Image)

[0257] 100 Defect Classification Server (Defect Classification Device, Computer)

[0258] 120 Defect Region Image Generation Unit (Defect Region Image Generation Process)

[0259] 130 Defect Type Determination Department (Defect Type Determination Process)

[0260] 143 Image Compositing Unit (Image Compositing Process)

[0261] 160 Position Relationship Acquisition Section (Position Relationship Acquisition Process)

[0262] 200 display unit

[0263] Screen for confirming the location relationship of defect type 202.

Claims

1. A defect classification device, characterized in that, have: The defect region image generation unit generates a defect region image that includes the defect region within the defect region image corresponding to the defect region, based on the difference between a learning image obtained by taking a picture of an inspection object region containing one or more circuit pattern regions and a defect region and a first reference image containing the circuit pattern regions in a first reference image. The defect type determination unit determines the positional relationship between the circuit pattern region and the defect region based on a circuit pattern recognition image that includes a circuit pattern region in a circuit pattern recognition image corresponding to the circuit pattern region in the first reference image and the defect region image. as well as The image synthesis unit, based on the aforementioned positional relationship, synthesizes the defect region image with a second reference image, thereby generating a synthesized image. The first reference image is an image obtained by photographing the first semiconductor wafer. The second reference image is an image obtained by photographing a second semiconductor wafer that is different from the first semiconductor wafer.

2. The defect classification device according to claim 1, characterized in that, The defect type determination unit obtains a first determination result based on the circuit pattern recognition image and the defect area image to determine whether the circuit pattern area is connected to the defect area. If the first determination result is positive, it obtains a second determination result to determine whether the defect area is included in the circuit pattern area.

3. The defect classification device according to claim 1, characterized in that, The defect type determination unit obtains the direction of the circuit pattern area and the number of circuit pattern areas that are in contact with the defect area.

4. The defect classification device according to claim 1, characterized in that, The defect type determination unit obtains the distance between the centroid of each circuit pattern area and the centroid of the defect area based on the shape of each circuit pattern area.

5. The defect classification device according to claim 1, characterized in that, The defect classification device further includes a position relationship acquisition unit, which causes the display unit to display a defect type position relationship confirmation screen indicating the position relationship.

6. The defect classification device according to claim 2, characterized in that, The second reference image includes one or more circuit pattern areas within the second reference image. The synthesized image includes a synthesized image circuit pattern region corresponding to the circuit pattern region in the second reference image and a synthesized image defect region corresponding to the defect region. The image synthesis unit determines the positional relationship between the circuit pattern region and the defect region in the synthesized image based on the positional relationship between the circuit pattern region and the defect region in the learning image.

7. A defect classification method, characterized in that, Make the computer perform the following procedures: The defect region image generation process generates a defect region image based on the difference between a learning image obtained by taking a picture of an inspection object area containing one or more circuit pattern areas and a defect region and a first reference image containing the circuit pattern areas in a first reference image. The defect type determination process is based on the circuit pattern recognition image containing the circuit pattern region in the circuit pattern recognition image corresponding to the circuit pattern region in the first reference image and the defect region image, to determine the positional relationship between the circuit pattern region and the defect region. as well as In the image synthesis process, based on the aforementioned positional relationship, the image of the defective region is synthesized with a second reference image to generate a composite image. The first reference image is an image obtained by photographing the first semiconductor wafer. The second reference image is an image obtained by photographing a second semiconductor wafer that is different from the first semiconductor wafer.

8. The defect classification method according to claim 7, characterized in that, The defect type determination process is as follows: based on the circuit pattern recognition image and the defect area image, a first determination result is obtained as to whether the circuit pattern area is connected to the defect area. If the first determination result is positive, a second determination result is obtained as to whether the defect area is included in the circuit pattern area.

9. The defect classification method according to claim 7, characterized in that, The defect type determination process is still the process of obtaining the direction of the circuit pattern area and the number of circuit pattern areas in contact with the defect area.

10. The defect classification method according to claim 7, characterized in that, The defect type determination process is still based on the shape of each circuit pattern area, and the process of obtaining the distance between the centroid of each circuit pattern area and the centroid of the defect area.

11. The defect classification method according to claim 7, characterized in that, The defect classification method also causes the computer to perform a process of obtaining the positional relationship by displaying a defect type positional relationship confirmation screen to represent the positional relationship.

12. The defect classification method according to claim 8, characterized in that, The second reference image includes one or more circuit pattern areas within the second reference image. The synthesized image includes a synthesized image circuit pattern region corresponding to the circuit pattern region in the second reference image and a synthesized image defect region corresponding to the defect region. The image synthesis process is a process of determining the positional relationship between the circuit pattern region and the defect region in the synthesized image based on the positional relationship between the circuit pattern region and the defect region in the learning image.

Images