Image learning method and apparatus, image inspection method and apparatus, program, and semiconductor device manufacturing method.

Abstract

We provide technologies that can improve efficiency and accuracy in image inspection techniques for semiconductors and other materials. [Solution] The image learning method includes the steps of acquiring multiple types of images (101) taken of a sample using multiple different imaging principles, as steps performed by a computer system; generating a composite image (102) by aligning the acquired multiple types of images, unifying their specifications, and assigning them to multiple channels using a predetermined assignment method; and learning a model (104) using the composite image (103) as input and defect classification information regarding defects in the sample as output.

Description

Technical Field

[0001] This disclosure relates to technologies such as learning and inference using images. In particular, it relates to technologies such as imaging, measurement, inspection, and evaluation for classifying, extracting, and determining abnormalities and defects from images of samples such as semiconductor wafers. In this specification, imaging, measurement, inspection, evaluation, etc. may be collectively referred to as inspection.

Background Art

[0002] For example, in the manufacture of semiconductor wafers using semiconductor materials such as SiC, inspections are performed on defects that may occur on the semiconductor wafers.

[0003] As prior art examples, JP-A-2023-138330 (Patent Document 1) and JP-A-2023-114827 (Patent Document 2) can be cited.

[0004] Patent Document 1 describes that in the manufacture of products by soldering, "using a machine learning model that outputs inspection results in an inspection after reflow from the input of image data based on an image before reflow, determine good or bad in the inspection scheduled to be performed after reflow from the image data based on the image before reflow acquired in real time."

[0005] Patent Document 2 describes that "an image storage unit 21 that stores three inspection images having different transmission characteristics with respect to the inspection object W obtained by imaging the inspection object W as pseudo RGB images, and for the pseudo RGB images stored in the image storage unit 21, processes each pixel based on a learning model 22 created in advance by learning using an image in the same format as the pseudo RGB image to obtain a quality defect degree, and a determination unit 24 that determines the quality state of the inspection object W by comparing the quality defect degree with a preset threshold value."

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

[0007] To improve the efficiency and accuracy of the above-mentioned inspections, it is conceivable to apply image inspection using machine learning / artificial intelligence (AI). In this image inspection, a sample such as a semiconductor wafer is imaged using an imaging device, the image obtained is input into a pre-trained model, and the results of the model's inference are output, such as defect classification.

[0008] As an example of prior art, defect inspection of SiC epitaxial wafers employs, for example, three different inspection methods and corresponding imaging means (in other words, image generation means) with varying sensitivities to defects. For example, these three inspection methods and imaging means are optical microscopes, photoluminescence (PL), and X-ray topography (XRT).

[0009] An optical microscope works by shining light onto a sample and using lenses to form an image of the transmitted or reflected light for observation.

[0010] Photoluminescence (PL) is the process by which a substance absorbs light and then re-emits it. A PL device irradiates a sample with high-energy light and detects the light re-emitted from the sample.

[0011] X-ray topography (XRT) is a method of visualization that involves irradiating a sample with X-rays and measuring the intensity of the diffracted X-rays.

[0012] When using the three inspection methods described above for defect inspection of SiC epitaxial wafers, each inspection method requires the acquisition of inspection images and the classification and extraction of defects from those images, which is time-consuming and laborious. Furthermore, when applying machine learning using images to each inspection method, each method requires a separate learning phase and inference phase, which is even more time-consuming and laborious. In the learning phase, it is necessary to acquire a large number of training images and build and train a machine learning model, which can take an enormous amount of time. In addition, it is necessary to separately construct algorithms to combine the defect classification results from each inspection method's model to obtain a single final defect classification result (in other words, the inspection result).

[0013] The purpose of this disclosure is to provide a technology that can improve efficiency and accuracy in image inspection technology for semiconductors and the like. [Means for solving the problem]

[0014] A representative embodiment (embodiment) of this disclosure has the following configuration. One embodiment is an image learning method for learning images and constructing a model, the method comprising the steps of: acquiring multiple types of images taken of a sample using multiple different imaging principles, as performed by a computer system; generating a composite image, which is an image obtained by aligning the acquired multiple types of images, unifying their specifications, and assigning them to multiple channels using a predetermined assignment method; and learning the model using the composite image as input and defect classification information regarding defects in the sample as output.

[0015] Furthermore, one embodiment is an image inspection method for classifying defects from an image based on a model that has learned images, and the steps performed by a computer system include: acquiring multiple types of images taken of a sample using multiple different imaging principles; generating a composite image, which is a single image obtained by aligning the acquired multiple types of images, unifying their specifications, and assigning them to multiple channels using a predetermined assignment method; and inputting the composite image into the model and obtaining defect classification information regarding defects in the sample as the output of the inference result by the model. [Effects of the Invention]

[0016] According to a representative embodiment of this disclosure, it is possible to improve efficiency and accuracy in the image inspection technology for semiconductors and the like. Other issues, configurations, and effects will be shown in the embodiments for carrying out the invention. [Brief explanation of the drawing]

[0017] [Figure 1] This figure shows an overview of a system and method for performing image learning and inference (inspection) in defect inspection of semiconductor wafers, as an embodiment (Embodiment 1). [Figure 2] This figure shows examples of three types of images obtained using three different inspection methods and imaging means for defects in a semiconductor wafer in this embodiment. [Figure 3] This figure shows an example of assigning and combining R, G, and B layers for three types of images in this embodiment. [Figure 4] This figure shows the efficiency of image learning according to the assignment method in this embodiment. [Figure 5] A diagram showing the learning process flow in this embodiment. [Figure 6] A diagram showing the processing flow of inference (checking) in this embodiment. [Figure 7] This figure shows an example of the data structure for managing the allocation method, etc., in this embodiment. [Figure 8] A diagram showing a first system configuration example in this embodiment. [Figure 9] A diagram showing a configuration example of a computer system in this embodiment. [Figure 10] A diagram showing processing examples during learning and inference in this embodiment. [Figure 11] A diagram showing an example of adjustment such as alignment of multiple images in this embodiment. [Figure 12] A diagram showing a configuration example of a machine learning model in this embodiment. [Figure 13] A diagram showing a configuration example of an input stage to a model (CNN) in the case of a 3-channel image in this embodiment. [Figure 14] A diagram showing specific examples of defects and images in this embodiment. [Figure 15] A diagram showing specific examples of synthetic images with different allocation methods in this embodiment. [Figure 16] A diagram showing a configuration example of synthesis and input stage in the case of 2 channels in a modification of this embodiment. [Figure 17] A diagram showing a configuration example of synthesis and input stage in the case of 4 channels in a modification of this embodiment. [Figure 18] A diagram showing a configuration example of synthesis and input stage in the case of 5 channels in a modification of this embodiment. [Figure 19] A diagram showing a second system configuration example in a modification of this embodiment. [Figure 20] A diagram showing a third system configuration example in a modification of this embodiment.

Embodiments for Carrying Out the Invention

[0018] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the drawings, the same components are generally denoted by the same reference numerals, and repeated descriptions are omitted. In the drawings, the representation of components may not represent the actual position, size, shape, range, etc. for the purpose of facilitating understanding of the invention.

[0019] In explanations, when describing program-based processing, the focus may sometimes be on the program, functions, or processing units. However, the core hardware component is the processor, or a controller, device, computer, or system composed of such a processor. The computer, using its processor, executes processing according to the program read into memory, utilizing resources such as memory and communication interfaces as appropriate. This realizes the specified functions and processing units. The processor is composed of semiconductor devices such as CPUs / MPUs and GPUs. Processing is not limited to software program processing; it can also be implemented using dedicated circuits. FPGAs, ASICs, CPLDs, etc., can be used as dedicated circuits.

[0020] The program may be pre-installed as data on the target computer, or it may be distributed as data to the target computer from the program source. The program source may be a program distribution server on a communication network, or a non-transient computer-readable storage medium, such as a memory card or disk. The program may consist of multiple modules. The computer system may consist of multiple devices. The computer system may consist of a client-server system, a cloud computing system, an IoT system, etc. Various types of data and information are composed of structures such as tables and lists, but are not limited to these. Representations such as identification information, identifiers, IDs, names, and numbers are interchangeable.

[0021] [Solution, etc.] This embodiment relates to a method and apparatus for performing defect inspection on semiconductors such as SiC epitaxial wafers as samples. This embodiment specifically targets crystal defects in SiC. This embodiment applies machine learning using images, in other words, image inspection, to this defect inspection. This embodiment uses, for example, three different inspection methods and corresponding imaging means (imaging methods, imaging devices, etc.) that have different sensitivities to defects and imaging principles for the sample. In this embodiment, an optical microscope, a photoluminescence device, and an X-ray topography device are used as the three types of inspection methods and imaging devices. In this embodiment, for example, the wafer surface of the same sample is imaged using the three types of imaging devices (in other words, inspection devices, etc.) and three types of images (sometimes referred to as inspection images) are obtained.

[0022] This embodiment uses three acquired images as input to a machine learning model. This embodiment assigns the three acquired images (for example, three images as a set) to three channels corresponding to the three primary colors R (red), G (green), and B (blue) in a color image. This embodiment combines these three channel images to obtain a single pseudo-color image as a composite image. This embodiment inputs this single pseudo-color image, the composite image, into a single machine learning model and obtains an automatic defect classification result (in other words, a defect inspection result) as the output of the model's inference results.

[0023] Machine learning generally consists of a training phase and an inference phase. In the training phase, a machine learning model is constructed and trained using the pseudo-color images mentioned above as training images. For example, deep learning can be applied to the machine learning model. For example, an object detection CNN model can be applied to the machine learning model. After sufficient training, the machine learning model becomes a defect classifier. In the inference phase (in other words, the inspection phase), during actual manufacturing inspection, the trained model (defect classifier) ​​is used to acquire multiple images (inspection target images) of the sample / defect to be inspected, captured using three different inspection methods / imaging means, including those used during training. In this embodiment, multiple images are combined into a single image, similar to the training phase. In this embodiment, the combined image is input to the model (defect classifier), and an automatic defect classification result (in other words, defect inspection result) is obtained as the output of the inference result.

[0024] This embodiment assigns multiple types of images corresponding to multiple inspection methods to multiple channels (3 channels when using pseudo-color images of R, G, and B) and synthesizes them into a single image. This embodiment constructs a single machine learning model using this single synthesized image as input. This embodiment allows for the construction and management of a single model, rather than three separate models for each inspection method, through synthesis. As a result of training this model with the synthesized image, this embodiment can be used to create a single defect classifier and an inspection device / inspection system equipped with it.

[0025] According to this embodiment, when using multiple types of inspection methods, the effort and time required can be reduced. According to this embodiment, compared to training multiple machine learning models corresponding to multiple types of inspection methods, the learning efficiency is increased and the accuracy of the inspection is improved.

[0026] [Regarding image assignment] This embodiment assigns multiple types of images to multiple channels (in other words, associates them, etc.) and combines the assigned images into a single image. The method of this assignment has the following characteristics.

[0027] When using multiple inspection and imaging methods, such as an optical microscope, photoluminescence, and X-ray topography, the three images acquired / generated by these methods are referred to as the first image, the second image, and the third image. These multiple images are then assigned to three channels (1st ch, 2nd ch, 3rd ch) corresponding to the layers of each color, for example, {R, G, B}. There are a total of 3 × 2 × 1 = 6 combinations (patterns) of assignment methods. The following are representations of these 6 patterns.

[0028] Pattern 1: {R, G, B} Pattern 2: {R,B,G} Pattern 3: {G, R, B} Pattern 4: {G, B, R} Pattern 5: {B, R, G} Pattern 6: {B, G, R}

[0029] For example, the notation in Pattern 1 above represents a method of assigning {optical microscope, photoluminescence, X-ray topography} to {R (1st channel), G (2nd channel), B (3rd channel)}. The order of the elements corresponds to the assignment.

[0030] Here, the inventors' investigations revealed that the effectiveness of defect classification, such as accuracy, differs depending on the assignment method. It was found that when assigning multiple {R, G, B} channels to multiple types of images and synthesizing them into a single image, changing the assignment pattern results in differences in accuracy, etc. Therefore, this embodiment can enhance its effectiveness by selecting a suitable assignment method that yields higher accuracy, etc.

[0031] Furthermore, the following embodiments of the above allocation method can be cited. Any of these embodiments are possible.

[0032] (1) In the first embodiment, the design involves experiments and studies to determine which assignment is preferable, and based on that, for example, one assignment method that yields the highest accuracy is decided, and the inspection method and inspection device are configured to implement that one assignment method in a fixed manner. In the learning and inference phases, processing is carried out according to that one assignment method.

[0033] (2) In the second embodiment, instead of fixing on one assignment method, the inspection method and inspection device are configured such that, in the learning phase, multiple patterns of assignment methods are comprehensively tried, and based on the results of the trials and evaluation of each pattern, the most suitable assignment method or an assignment method with accuracy higher than a predetermined threshold is selected and determined. The procedures and functions for the trials are implemented in this inspection method and inspection device. In the actual inspection (inference phase), the model (defect classifier) ​​trained with the most suitable assignment method selected as a result of the trials is used. The second embodiment is useful when it is unclear which assignment method is best for a sample or type of defect.

[0034] [Learning and Inference] This embodiment applies a machine learning model to defect inspection of, for example, SiC epitaxial wafers to perform image inspection, and includes a learning phase and an inference phase in machine learning technology. The categories of this embodiment broadly include inspection methods, inspection systems, inspection equipment, inspection programs, etc., related to defect inspection (especially image inspection), and in terms of machine learning technology, they are expressed as learning methods, learning equipment, inference methods, inference equipment, etc. The learning phase is another way of saying training phase. The inference phase corresponds to the phase in which a sample is inspected during actual manufacturing using the trained model, and the inference equipment corresponds to the actual inspection equipment, etc. In the following, the learning phase and inference phase will be explained separately as appropriate.

[0035] <Embodiment 1> An embodiment of this disclosure (referred to as Embodiment 1) will be described using Figures 1 and subsequent figures.

[0036] In this embodiment, for the semiconductor wafer sample, images are first captured for each of three inspection devices (corresponding imaging devices, etc.) corresponding to, for example, three types of inspection methods and imaging means, and three types of images (corresponding image data, etc.) are acquired. In this embodiment, instead of individually training the image data of the three acquired images using three different machine learning models, the images are assigned to multiple channels to form a single composite image, and then trained using a single machine learning model. In this embodiment, the three images are assigned to three channels {R, G, B} and combined into a single pseudo-RGB color image. Similarly, training is repeated using multiple composite images.

[0037] [Overview of Learning and Inference] Figure 1 is an explanatory diagram illustrating the overview and basic configuration of the system and method for learning and inferring images in semiconductor wafer defect inspection according to this embodiment. Figure 1 shows an example configuration in which three types of images are combined into one and input into a single machine learning model (e.g., a deep learning model). The process shown in Figure 1 is executed by a predetermined computer system (in other words, a learning system and an inspection system). In this embodiment, machine learning is performed on inspection images to construct a model (defect classifier) ​​in order to generate a defect classifier for inspection.

[0038] In the learning phase, the computer system acquires and inputs inspection images as training data 101. These inspection images (in other words, training images) are a set of multiple images taken of a semiconductor wafer (particularly a SiC epitaxial wafer), which is the sample, corresponding to multiple types of inspection methods and imaging means with different imaging principles. In this example, these multiple images are three types of images taken using an optical microscope, photoluminescence, and X-ray topography. The three corresponding imaging / inspection devices are an optical microscope, a photoluminescence device, and an X-ray topography device.

[0039] The computer system takes this examination image (training data 101) as input and performs the processing of assigning it to multiple (three) channels, namely the R, G, and B layers, and then synthesizing them 102. This results in one synthesized image 103 for each set.

[0040] The computer system takes the synthesized image 103 as input and performs machine learning (deep learning in this example) to construct a machine learning model 104. The learning process is repeated until sufficient accuracy is achieved.

[0041] In the inference phase, or in other words, the inspection phase in which the actual inspection is performed, the computer system acquires and inputs inspection images as inspection target data 106. These inspection images (or inspection target images) are, like the inspection images used during training, data consisting of multiple images taken of the semiconductor wafer (especially a SiC wafer) sample, corresponding to multiple types of inspection methods and imaging means, as a single set. The inspection methods and imaging means used during inference are assumed to be the same as those used during training.

[0042] The computer system takes the test image (test data 106) as input and performs the assignment and synthesis processing 107 to multiple (three) channels: R, G, and B layers. This results in one synthesized image 108 (in other words, a synthesized image for inference) for each set. The synthesis processing 107 during inference is the same as the assignment processing 102 during training.

[0043] The computer system inputs the synthesized image 108 into the model 105 and obtains the output of the inference and judgment results from the model 105, i.e., the automatic defect classification result as the defect inspection result. The computer system then records and outputs the inspection results 110 to the user or other relevant parties.

[0044] [Defective images and composites] Figures 2 and 3 are explanatory diagrams illustrating the synthesis of three types of images in this embodiment (102 and 107 in Figure 1). In this embodiment, synthesis does not involve arranging multiple images on a plane, but rather overlapping them by aligning their positions and regions.

[0045] Figure 2 is a schematic diagram illustrating three types of images (inspection image / defect image) obtained by imaging the same target area on a semiconductor wafer surface using three different inspection methods and imaging means. The target area contains a certain type of defect (referred to as defect A). This inspection image is also referred to as the defect image. The target area is, for example, a rectangular region on the wafer surface (corresponding coordinate system) from coordinates (x1, y1) to (x2, y2). Figure 2 and other diagrams schematically illustrate the contents of the images.

[0046] The first defect image 201 (in other words, the first image, the first type of image) is an image captured by the first inspection method / imaging means (e.g., an optical microscope). Defect A (in other words, the defect region) 211 is visible within the target region.

[0047] The second defect image 202 (in other words, the second image, the second type of image) is an image captured by the second inspection method / imaging means (e.g., photoluminescence). Defect A212 is visible within the target area.

[0048] The third defect image 203 (in other words, the third image, the third type of image) is an image acquired by the third inspection method / imaging means (e.g., X-ray topography). Defect A213 is visible within the target area.

[0049] For subsequent image synthesis, the positional coordinates of each inspection image, for example, the coordinates of the bottom left and top right corners on the wafer (bottom left coordinate (x1, y1), top right coordinate (x2, y2)), are adjusted during imaging or after imaging (e.g., by cropping from previously acquired images) so that they are identical. In Figure 2, defect A is shown as being located in the center of each image, but this is not the only way; as long as the coordinates are the same in each image, the defect can be located anywhere within the image. Also, in the example image in Figure 2, only one type of defect, defect A, is captured, but multiple types of defects, or multiple defects of the same type, may be captured in the same image.

[0050] In this embodiment, the three types of images captured by each imaging device are all monochrome grayscale images. In this embodiment, these images are assigned to the three channels of the R, G, and B layers. By combining the image data of the three channels of the R, G, and B layers, a pseudo-RGB color image is obtained as a composite image. This composite image reflects and retains the characteristic information (such as how defects are depicted) of the various inspection images. Therefore, appropriate defect classification and inspection can be performed from this composite image through the input and output of a machine learning model.

[0051] The computer system verifies that the parameter values, such as position and size, of multiple images of the sample (which may be the wafer surface or a wafer cross-section) are identical in order to achieve image synthesis. If the parameters are not identical (if they differ), the computer system adjusts them to become identical. This adjustment may be performed during imaging by the imaging device, or through image processing after imaging. For example, after imaging, adjustments such as alignment, cropping, and scaling may be performed. Since the size of the images may differ depending on the imaging device, such adjustments are made as needed.

[0052] Furthermore, to achieve image synthesis, the computer system adjusts the color and contrast of each image from multiple image types as needed, and adjusts them to a unified format and specification for synthesis, such as monochrome grayscale. In other words, the computer system performs scaling of the pixel values. Since the scaling of the pixel values ​​may differ depending on the imaging method, such adjustments are made as needed.

[0053] In this example, the composite image format is assumed to be a standard 24-bit image (8 bits each for R, G, and B). In this case, the pre-compositing adjustments are made so that each image is scaled to 8 bits (monochromatic grayscale).

[0054] The input and output images (composite images) are shown as being square in size, for example, but the image size is not limited to this and can be any shape.

[0055] Figure 3 is a schematic diagram illustrating the assignment and synthesis of the three types of images in Figure 2 to the R, G, and B layers. In this example, we show an example of assignment (or mapping) to three types of channels, namely the R, G, and B layers. In the example in Figure 3, the first channel is the R layer, the second channel is the G layer, and the third channel is the B layer. The computer system assigns the first defective image 201 obtained by the first inspection method to the R layer, which is the first channel, the second defective image 202 obtained by the second inspection method to the G layer, which is the second channel, and the third defective image 203 obtained by the third inspection method to the B layer, which is the third channel (corresponding to the "RGB" pattern described later).

[0056] The computer system combines the R, G, and B layers from three assigned images into a single image. The computer system overwrites the R, G, and B data layers with three monochrome grayscale images (201, 202, 203) to obtain a single 24-bit pseudo-RGB color image. This combined pseudo-RGB color image is referred to as the combined image 300 (or, in other words, the combined defect image). The region of coordinates (x1, y1) to (x2, y2) in the combined image 300 contains defect A310, which is a composite of defect A from the three images.

[0057] This single composite image 300 reflects and integrates the characteristic information of three types of defects A obtained by three different inspection methods and imaging means. From this information, including defects A310, it is possible to classify and determine defects using a machine learning model.

[0058] In the learning phase, the computer system uses the synthesized image 300 in Figure 3, obtained based on the training images, as the synthesized image 103 in Figure 1, and performs machine learning 104. During learning, the computer system adjusts the parameter values ​​of the machine learning model 105 using the input synthesized image 103 and the output label information (defect classification information). Through learning that achieves sufficient accuracy, the model 105 becomes a defect classifier.

[0059] In the inference phase (in other words, the inspection phase), the computer system (in other words, the inspection device) similarly creates the composite image 300 shown in Figure 3 based on the image to be inspected, which becomes the composite image 108 shown in Figure 1. The computer system inputs the composite image 108 into the model 105 and obtains the defect inspection result (defect classification result) as the output of the inference result.

[0060] In this embodiment, by combining three types of images into a single composite image, the number of training cycles and the required processing time can be reduced to approximately one-third of those in the conventional method.

[0061] [Accuracy and efficiency of learning] Figure 4 is an explanatory diagram regarding the accuracy and efficiency of image learning. Figure 4 shows the results of evaluating the defect detection accuracy using various metrics when learning defects (micropipes, threading screw dislocations; TSDs, polytype inclusions) on SiC epitaxial wafers, while varying the method of assigning R, G, and B layers to each type of image obtained using different inspection methods. Three inspection methods / imaging techniques were selected: optical microscopy, photoluminescence, and X-ray topography.

[0062] The line 400 represents throughput. The bar graph shows four evaluation values ​​(evaluation metrics) in sets, from left to right: Precision, Recall, F-score, and Accuracy. The evaluation metrics used here are Precision, Recall, F-Score, and Accuracy, which are commonly used in the evaluation of object detection CNNs. For these evaluation metrics, a value closer to 1 is better.

[0063] The "individual" training data in the graph represents the conventional method, for example, when training is performed individually on three different test methods, and the results of training by independently annotating three different test images are shown.

[0064] In contrast to "Individual," the six bar graphs on the horizontal axis, from "BGR" to "GBR," represent the training data for the "simultaneous learning" method of this embodiment, showing six combinations (patterns) of assigning three types of images from three different inspection methods to the three layers R, G, and B. For example, the "RGB" pattern represents assigning optical microscope images to the R layer, photoluminescence images to the G layer, and X-ray topography images to the B layer (Figure 3). In Figure 4, the six patterns are arranged in descending order of evaluation value. The "BGR" pattern has the lowest evaluation value. In particular, the three patterns "BRG," "RGB," and "GBR" have the highest evaluation values ​​and throughput.

[0065] Figure 4 shows that all six assignment methods for "simultaneous learning" yield higher evaluation values ​​and throughput than the conventional "individual" method. In this embodiment, since three types of images are combined into one, the effort and processing time involved in learning are reduced to about one-third. Compared to the throughput value of approximately 3 in the case of "individual" learning, this embodiment allows for a throughput value of up to approximately 7 depending on the pattern, achieving more than double the efficiency. The inference phase can also achieve the same throughput as the learning phase.

[0066] Since the R, G, and B colors are not symmetrical, it was found that the efficiency and accuracy of the learning results differ depending on the assignment method (pattern) used to assign the three types of images to each layer, as shown in Figure 4. In this embodiment, based on the learning experiments and studies shown in Figure 4, the design was to select and use the assignment method (pattern) that yielded the highest evaluation value (accuracy) and throughput (efficiency) from among the six patterns. In the case of Figure 4, when the target sample is a SiC epitaxial wafer and the type of defect to be inspected is a micropipe (described later), the patterns with the highest accuracy are the three patterns "BRG", "RGB", and "GBR". In this embodiment, for example, the "RGB" pattern (pattern 1 mentioned above) was selected and used (Figure 3). However, other patterns may also be used, and they may be more effective than conventional methods.

[0067] Furthermore, when deciding which patterns to use in the implementation, you may set a threshold for the evaluation value / throughput value and select from patterns that are greater than or equal to the threshold. For example, if you set the evaluation value threshold of 401 as 0.95 as shown in the figure, you can select from the four patterns "GRB", "BRG", "RGB", and "GBR", which are greater than the threshold of 401.

[0068] [Processing flow of the learning phase] Figure 5 shows the processing flow of the learning phase. In step S1, the computer system collects and acquires images of the defects to be detected, which are images of the defects to be detected, as training images. For each type of defect, the collected images are set into three types, corresponding to, for example, three different inspection methods and imaging means used, as shown in the example in Figure 2.

[0069] In the case of SiC epitaxial wafers, the defects to be detected are various types of defects that could potentially cause malfunctions in the operation of power devices, such as those described in Reference 1.

[0070] Reference 1: JEITA Standard EDR-4712 / 100, "Non-destructive testing method for crystal defects in SiC wafers (Part 1: Classification of crystal defects)"

[0071] For these defects to be inspected, this system generates and acquires defect images using multiple inspection methods and imaging means with different physical factors for image grayscale generation (in other words, grayscale formation principles and imaging principles). Examples of these inspection methods for SiC epitaxial wafers include optical microscopy, photoluminescence, and X-ray topography.

[0072] The density of images captured by optical microscopy is the distribution of light intensity in the visible light region scattered from the irradiated light on the wafer surface. The density of images captured by photoluminescence is the distribution of photoluminescent light intensity generated when ultraviolet light is irradiated. The density of images captured by X-ray topography is the distribution of X-ray intensity that increases or decreases due to deviations from Bragg reflection conditions caused by the presence of defects when X-rays with approximately a single wavelength are irradiated in line with the crystal orientation of the wafer.

[0073] Therefore, depending on the grayscale formation principle of the image, the inspection image / defect image reflects the characteristics of various defects, and by combining these (i.e., image synthesis), as in this embodiment, more accurate detection and classification of defects becomes possible.

[0074] The image data obtained from each inspection method is two-dimensional intensity data. If the intensity of each pixel is P, it can be expressed as a function: P = P(px, py), where px and py are the vertical and horizontal coordinates of the pixel.

[0075] Step S2 is the synthesis of each type of defect image. As shown in Figure 2, in the case of the three types of inspection methods, each image data can be associated with the R, G, and B primary color data in a color image. In step S2, the computer system assigns the three types of detection target defect images to the R, G, and B layers for each set, as shown in the example in Figure 3, and synthesizes them into a single composite image.

[0076] During assignment and synthesis, the computer system adjusts each image as needed, such as by resizing, to ensure that the number of pixels in the width and height (image size) of each defective image matches (see below).

[0077] Furthermore, the pixel intensity values ​​of each inspection image reflect the sensitivity to defects and the strength of the image signal intensity depending on the inspection method, and the relative magnitudes of the pixel values ​​of each inspection image may be adjusted as appropriate. For example, if the average value of the image values ​​from the first inspection method and the average value of the image values ​​from the second inspection method differ significantly, the weight of information from each inspection method will differ greatly in the composite image, which is undesirable. Therefore, it is effective to standardize the intensity data (in other words, unify the image specifications) so that the intensity values ​​are suitable for defect detection for various types of images.

[0078] The image data of the composite image is 3-channel 2D data (see Figure 13 below), and if the pixel values ​​of the R, G, and B layers are PR, PG, and PB respectively, it can be expressed in functional form by the following equation.

[0079] PR=PR(px,py), PG = PG(px, py), PB = PB(px, py),

[0080] Alternatively, the three equations above can be combined and expressed as P3 = P3(i, px, py), where i is the index corresponding to R, G, and B.

[0081] In step S3, the computer system and the user performing the learning task using the computer system perform annotation, or in other words, labeling of the training data for supervised machine learning, on the composite image obtained in step S2. The user labels what type of defect is present in a given composite image. The annotation / labeling task includes, for example, the user determining where and what kind of defect is in the image, enclosing the determined defect with a rectangle, and inputting the name of the defect type (or defect type index) along with the position data of this rectangle (for example, the pixel coordinate value of the upper left corner of the rectangle and the pixel values ​​of the width and height of the rectangle) for each image.

[0082] For training the model (defect classifier), multiple (usually dozens or more) annotation data sets are prepared for each type of defect to be trained.

[0083] In step S4, the computer system automatically performs machine learning processing using the multiple composite image and annotation (labeled) data prepared up to step S3, and a machine learning model (e.g., a deep learning model).

[0084] Various types of machine learning algorithms can be applied as models (classifiers). A suitable example is an object detection algorithm using a Convolutional Neural Network (CNN). Object detection CNNs are CNNs developed to automatically detect what objects are captured in an image, and well-known examples include Faster-RCNN, SSD, and YOLO. These object detection CNNs are open source and can be used on various frameworks (PyTorch, TensorFlow, Keras, etc.). In these object detection CNNs, the input image is a color image. If there are three types of inspection images, the object detection CNN can be used immediately by assigning the images prepared in step S2 to three channels (R, G, B layers). The annotation data format (COCO format, PascalVOC format, etc.) is prepared according to the input conditions of the object detection CNN to be used.

[0085] The learning process is no different from learning a model for general images. The computer system learns the model by adjusting the learning parameter values ​​until the desired accuracy is achieved.

[0086] In step S5, the computer system and the user evaluate the machine learning model obtained as a result of the learning in step S4, checking whether the accuracy of inference (defect detection) is sufficient, and obtain a model with sufficient accuracy that satisfies the user. If it is not sufficient, the learning process is repeated. If necessary, the process returns to step S1, and the accuracy is improved by adding or replacing training images, etc. As a means of improving accuracy, selecting the assignment method to the R, G, and B layers, as shown in Figure 4, is effective.

[0087] In step S6, the computer system determines a model with sufficient accuracy to be used in the inference phase (inspection phase), i.e., as a defect classifier, and sets up the data for that model in the computer system / inspection device to be used in the inference phase (inspection phase).

[0088] [Flow of the inference (testing) phase] Figure 6 shows the processing flow of the inference phase (in other words, the inspection phase). In step S21, the computer system captures and acquires three types of inspection target images of the semiconductor wafer sample using three types of inspection methods and imaging means. In step S22, the computer system assigns the three inspection target images to the three channels of the R, G, and B layers to obtain a single composite image. In step S23, the computer system inputs the composite image into a model (defect classifier) ​​and obtains information on the defect classification result (inspection result) as the output of the inference and judgment result by the model. In step S24, the computer system records the inspection result and outputs it to the user (e.g., on the screen).

[0089] This system provides support information for learning and inference tasks through a user interface, such as a GUI. The screen may display multiple types of images, such as those in Figures 2 and 14, composite images, such as those in Figure 3, graphs, such as those in Figure 4, management information, such as those in Figure 7, defect candidate location information, image adjustment information, model information, such as those in Figures 12 and 13, and images and information of defect classification results, such as those in Figure 14.

[0090] [Assignment Management] Figure 7 shows an example of the configuration of data and information for which the computer system manages and controls multiple inspection methods / imaging means and assignment methods (patterns, etc.) for assigning multiple types of images to multiple channels in this embodiment. In this example, the computer system creates, manages, and stores data and information (e.g., tables) as shown in the figure in its memory resources, and makes it available for use by referencing it as needed.

[0091] Table 701 in (A) contains management information about inspection methods and imaging means. Table 701 contains information about multiple types of inspection methods and imaging means used. In this example, Table 701 has the following column items: management number (#), "inspection method / imaging means", "device ID", "imaging conditions (parameter values)", etc. The "inspection method / imaging means" includes the three types mentioned above. The "device ID" is the identification information / address of the device (imaging device / inspection device) that has the implementation corresponding to the "inspection method / imaging means". The "imaging conditions (parameter values)" indicates the setting information regarding the imaging conditions / parameter values ​​used when capturing images (inspection images) with that device.

[0092] It is also possible to set and use multiple imaging conditions for the same device (see the modified examples below). For example, it is possible to designate an image captured with imaging condition #1 in device A as type 1 image, and an image captured with a different imaging condition #2 in device A as type 2 image.

[0093] Table 702 in (B) contains management information regarding the channels to be used and how they are assigned. Table 702 includes information such as the channels (ch) to be used, the assignment of the corresponding RGB layers, and attribute information such as the image size of each ch (corresponding layer). This example shows the settings when applying the aforementioned "RGB" pattern ("Pattern 1").

[0094] Table 703 in (C) contains management information for all assignment methods. Table 703 contains information that covers assignment patterns based on combinations of multiple types of inspection methods / imaging means used and the number of channels used. For example, Table 703 includes information such as pattern ID, correspondence with RGB layers (ch), pattern evaluation value, and whether or not it is applied (1 / 0).

[0095] [Example of System Configuration 1] Figure 8 shows a first system configuration example as an example of the system configuration of this embodiment. In the system shown in Figure 8, the first system configuration example includes three types of imaging devices (in other words, inspection devices) and a server device that is independent of them, and the server device is configured to perform learning and inference (inspection) processing as its main processing.

[0096] In Figure 8, the system is configured such that a server device 800 is connected to a first imaging device, an optical microscope 801, a second imaging device, a photoluminescence device 802, and a third imaging device, an X-ray topography device 803, via a communication network (e.g., LAN) 809. The user operates the server device 800 and each imaging device to perform learning and inference tasks. The server device 800 functions as both a learning device and an inference device. Note that the tasks of imaging and learning may be shared among multiple people.

[0097] The first imaging device, an optical microscope 801, acquires an image (the first image) of the sample 810. The second imaging device, a photoluminescence apparatus 802, acquires an image (the second image) of the sample 810. The third imaging device, an X-ray topography apparatus 803, acquires an image (the third image) of the sample 810. The sample 810 is a semiconductor wafer, for example, a SiC epitaxial wafer. The technology of each imaging device is publicly known and is not limited in detail. Furthermore, the order in which the sample 810 is imaged by the multiple imaging devices is not limited.

[0098] The server device 800 receives and acquires image data and other data from each imaging device (801-803) via the communication network 809 and stores it in memory.

[0099] The server device 800 has, as functions, at least a learning function 800A and an inference function (in other words, a testing function) 800B. The learning function 800A is a function that performs the processing of the learning phase described above. The inference function 800B is a function that performs the processing of the inference phase (testing phase) described above.

[0100] During the learning phase, the server device 800 uses the images acquired from each imaging device as training images and performs the aforementioned learning process. This results in an inference model (defect classifier).

[0101] During the inference phase, the server device 800 acquires images to be inspected from each imaging device, inputs them into the model, and obtains defect inspection results (defect classification results) as the output of the inference results from the model.

[0102] The server device 800 also provides a user interface for learning and inference. For example, the server device 800 provides a screen with a graphical user interface (GUI) (which may be a web page, for example) to the user's PC or other terminal. The user can input instructions and settings on the screen and check various information displayed on the screen (such as inspection images and inspection results). On the screen, for example, images from each inspection method in Figure 3 (201, 202, 203) and the composite image 300 can also be displayed and checked.

[0103] [Computer System] Figure 9 shows an example of the hardware and software configuration of the server device 800 in Figure 8 as a computer system. The computer 1000, which corresponds to the server device 800, has a processor 1001, memory 1002, communication interface device 1003, input / output interface device 1004, etc., and these are interconnected via an architecture such as a bus.

[0104] The processor 1001 executes processing according to the control program 1002A for the memory 1002. This enables the implementation of a predetermined function 1001A (such as the learning function 800A in Figure 8) as an execution module. The processor 1001 may also be an NPU (Neural Processing Unit) suitable for machine learning / AI calculations.

[0105] Memory 1002 is composed of non-volatile storage devices, etc. Memory 1002 stores, for example, control program 1002A, setting information 1002B, management information 1002C, training data 1002D, inference data 1002E, etc. Control program 1002A is a training program and inference program, etc., for realizing the training function 800A and the inference function 800B, etc. Setting information 1002B is system setting information and user setting information related to control program 1002A. Management information 1002C is data like that shown in Figure 7, for example. Training data 1002D is various data used in the training phase, such as image data, machine learning models, and training data. Inference data 1002E is various data used in the inference phase, such as image data, machine learning models, and test result information.

[0106] The communication interface device 1003 is a device that implements a communication interface with the communication network 809, etc. Input / output interface device 1004 is externally connected to input device 1005 and output device 1006. The computer 1000 may also have built-in input / output devices. Examples of input devices 1005 include a mouse, keyboard, microphone, etc. Examples of output devices 1006 include a display, printer, speaker, etc.

[0107] [Processing example in the first system configuration example] Figure 10 shows a schematic example of common processing during training and inference in the first system configuration example in Figure 8. First, in step S31, the system acquires the sample to be inspected (semiconductor wafer), and the server device 800 sets the imaging target area, imaging conditions, etc. for each imaging device (801 to 803) corresponding to the first to third inspection methods.

[0108] In step S32, the server device 800 transmits instructions to each imaging device (801-803) to perform inspection / imaging of the sample. For example, the first imaging device 801 of the first inspection method images the sample, thereby obtaining a first image (first inspection image data) and information related to the imaging, such as first image position information data. The image position information data is data / information that represents the position, region, etc. of the captured image in the coordinate system of the sample (e.g., semiconductor wafer). Instructions for inspection / imaging to each imaging device (801-803) may also be input by the user to each imaging device.

[0109] As a result of step S32, the first to third inspection image data and the first to third image position information data, which includes information representing the imaging position and imaging range within the wafer, are output.

[0110] In step S33, the server device 800 acquires inspection image data and image position information data from each imaging device (801-803) and stores them in memory. This data may be accumulated sequentially during each inspection operation, or it may be saved all at once after each inspection operation is completed.

[0111] In step S34, the server device 800 (especially the processor) performs adjustments such as cropping, aligning, resizing, and scaling of image regions based on each acquired image data (for example, three types of images) for image synthesis. In other words, the server device 800 performs processing to unify the format and specifications of the three types of images for synthesis. Alignment and other adjustments can be achieved using image position information data obtained in step S33.

[0112] Each imaging device (801-803) of this system, or any device that manages and controls imaging by the imaging devices (e.g., server device 800), has image position information data (related information at the time of imaging), such as which position coordinates or range on the surface of the semiconductor wafer sample were targeted for imaging. The computer system, for example, server device 800, can perform the image adjustment in step S34 by using such image position information data along with the image data.

[0113] During adjustment, the server device 800 uses image position data based on the inspection image to extract an image of a corresponding area (target region) on the wafer, or generates an image of a corresponding area by cutting and pasting multiple inspection images (Figure 11, described later). These pre-set areas are, for example, areas obtained by dividing the wafer into a grid pattern with a set size and number. These areas may be the same as or different from the imaging area of ​​the image output of any imaging device / inspection device.

[0114] In step S35, the server device 800 assigns the three adjusted images to multiple channels (e.g., R, G, B layers) using the configured assignment method, and combines the assigned image data into a single image for each set. The server device 800 stores the resulting combined image in memory.

[0115] The three types of inspection images reconstructed for each area through the above adjustments are combined into a single pseudo-RGB color image for each area through image synthesis. The method of assigning the R, G, and B layers of the inspection images obtained by each inspection method is the same as the configuration used during model training. The intensity values ​​of each inspection image may be normalized to ensure optimal accuracy.

[0116] In step S36, the server device 800 inputs the synthesized image into a machine learning model and performs training or inference.

[0117] In step S37, the server device 800 obtains the output of the machine learning model. During training, the model's parameter values ​​are adjusted using the input and output. During inference, the images of each area are input to the defect classifier, which detects and classifies defects captured in the images, and outputs defect classification information. The defect classification information includes defect type, location / area, size, and other characteristic data, and this data is compiled into a defect list. The output defect classification information is stored in the server device 800 and displayed on the screen of the display device viewed by the user.

[0118] [Adjustments such as alignment] Figure 11 is an explanatory diagram showing an example of the image adjustment process in step S34. An inspection image of the entire surface of the wafer 810 is captured using three types of imaging devices (801-803). An example of the imaging area with the optical microscope (801) is shown in grid 1101. Each square in grid 1101 corresponds to one image. For illustrative purposes, the number of squares in grid 1101 is shown sparingly. Also, since the entire circular wafer is imaged, at the wafer edge, the wafer may only be a part of the image. Similarly, an example of the imaging area with the photoluminescence device (802) is shown in grid 1102, and an example of the imaging area with the X-ray topography device (803) is shown in grid 1103.

[0119] To explain the image adjustment process in step S34, we focus on the images in which the defect 1104 present on the wafer 810 is captured. Image 1111, in which the defect 1104 is captured by the optical microscope (801), is an image of the region from coordinates (X1, Y1) to coordinates (X2, Y2), and has a pixel size of SX1 × SX1. Image 1112, in which the defect 1104 is captured by the photoluminescence apparatus (802), is an image of the region from coordinates (X3, Y3) to coordinates (X4, Y4), and has a pixel size of SX2 × SX2. Image 1113, in which the defect 1104 is captured by the X-ray topography apparatus (803), is an image of the region from coordinates (X5, Y5) to coordinates (X6, Y6), and has a pixel size of SX3 × SX3. Thus, because the inspection devices are different, simply overlaying the images from each device in which the defect 1104 of interest was captured will not result in the defect location matching. Furthermore, while the image area is square in this example, it is not limited to this.

[0120] The computer system (server device 800) adjusts the alignment and cropping of the three types of images acquired from various imaging devices, based on the inspection image acquisition position and inspection image size information. Here, as an example, we will explain the adjustment to match the range of the photoluminescence inspection image and the X-ray topography inspection image to the inspection image range of the optical microscope. In adjusting the photoluminescence inspection image, the same region 1114 is cropped, including the image adjacent to image 1112, so that it has the same imaging range as the region from coordinates (X1,Y1) to coordinates (X2,Y2), which is the imaging range of the optical microscope inspection image. Similarly, in adjusting the X-ray topography inspection image, the same region 1115 is cropped, including the image adjacent to image 1113, if necessary, so that it has the same imaging range as the region from coordinates (X1,Y1) to coordinates (X2,Y2), which is the imaging range of the optical microscope inspection image. Note that this adjustment can be performed on all inspection images using the image acquisition position and size information, regardless of the presence of defects. Furthermore, after adjusting the alignment of the image, the pixel size, image signal intensity scaling, and image value bit depth matching are performed to synthesize three types of inspection images. The details of the above adjustments can be handled using publicly known techniques.

[0121] The computer system may also acquire adjusted images from various imaging devices by setting and instructing imaging conditions, etc., to achieve alignment and other adjustments.

[0122] [Machine learning models] Figure 12 shows an example of a machine learning model configuration (model 105 in Figure 1) using a deep learning neural network. In Figure 12, the input data 1201 to the machine learning model 1200 is a synthesized image 1201, and the output data 1202 is defect classification information 1201. During training, the computer system and the user repeatedly perform calculations on the machine learning model 1200 using the input data 1201 and output data 1202 to obtain the desired relationship between the input (synthesized image 1201) and the output (defect classification information 1202).

[0123] The lower part shows the case where the machine learning model 1200 is a deep learning neural network. Each pixel value in the input image 1211 (synthesized image 1201) is assigned to a node in the input layer. The nodes in the output layer are associated with truth values ​​or probabilities of whether or not there is a defect in the defect classification information 1202, which is the inference result 1212.

[0124] Figure 13 is an explanatory diagram showing an example of the configuration of the input stage for learning when using a convolutional neural network (CNN) as a deep learning neural network as an example of machine learning model configuration. In this example, Faster-RCNN is used. Figure 13 shows the case where a pseudo-RGB color image with 3 channels (R, G, B layers) is used. The input data (Pcolor) 1301 is image data with 3 channels R, G, and B, and each channel has an image size of 224 × 224 pixels (vertical pixels × horizontal pixels). The input data 1301 is subjected to a convolution operation 1302 with a filter 1303. The filter 1303 has multiple (64) filters, F1, ..., F64, each with an image size of 7 × 7 for each channel. As a result of the operation, a feature map 1304 is obtained. The feature map 1304 has multiple (64) feature maps, C1, ..., C64, each with an image size of 112 × 112.

[0125] The input data 1301 is a 2D image of 224x224 pixels, and by adding three channels (R, G, B), it becomes 3D data. One filter is 3D data with 7x7 pixels and 3 channels. After performing a convolution operation on each channel in 2-pixel steps and then summing up the results of the three channels, a single 112x112 feature map is obtained.

[0126] When using Faster-RCNN on PyTorch as an object detection CNN, the input color image (composite image) data is subjected to a predefined normalization process unless otherwise specified. This normalization converts the R, G, and B layers (the three channels [R, G, B] shown in the diagram) into numerical data between 0 and 1, with mean values ​​of [0.485, 0.456, 0.406] and standard deviations of [0.229, 0.224, 0.225].

[0127] [Specific examples of defects and images] Figure 14 shows images of defects and a specific example of image synthesis. Micropipes, which are crystal defects, are known as a type of defect that can occur in SiC epitaxial wafers. Figure 14 shows an example of synthesizing three types of defect images of micropipe defects on a SiC epitaxial wafer into a pseudo-RGB color image. In this example, the first defect image 1401 obtained by optical microscopy is assigned to the B layer, the second defect image 1402 obtained by photoluminescence is assigned to the R layer, and the third defect image 1403 obtained by X-ray topography is assigned to the G layer (in the case of the "BRG" pattern mentioned above).

[0128] In the first defect image 1401, a micropipe is visible as defect A near the center of the region. In the second defect image 1402, a micropipe is visible as defect A near the center of the region, but it is difficult to discern from this image alone. In the third defect image 1403, a micropipe is visible as defect A in a black area that extends outward from near the center of the region.

[0129] In the composite image (composite defect image) 1404, the images of defect A = micropipe from the three original images are combined. The composite image 1404 reflects and retains the characteristic information of defect A = micropipe that was present in the three original images. In the learning phase, when the user performs annotation (labeling), they look at the composite image 1404 and determine that the portion of defect A within the region is a defect type = micropipe. Then, as shown in the defect classification result image 1405 (training image), the user adds, for example, a frame 1406 (e.g., a red square) to specify the defect region / location and a label 1407 to specify the defect type ("micropipe").

[0130] In the inference phase, the computer system inputs the synthesized image 1404 into the model and outputs the defect classification result image 1405 (inspection result image). In this image, defect A = micropipe, located near the center of the region, is recognized and detected, and is displayed with a frame 1406 representing the defect region and location, and a label 1407 representing the defect type ("micropipe").

[0131] In conventional individual inspection methods, for example, when viewing the first defect image 1401 using an optical microscope, the micropipe area is small, but when viewing the third defect image 1403 using X-ray topography, the micropipe area is larger. In reality, strain has occurred in the semiconductor not only in the location of the micropipe in the first defect image 1401, but also in the black, extended area in the third defect image 1403.

[0132] In contrast, according to this embodiment, such distorted areas can be detected as defect regions from the composite image 1404. In the three types of inspection images, which use different imaging principles, the same defect will appear in different ways. As shown in the figure, the characteristic information of the three types of images is reflected and preserved in the composite image, so defects can be classified and detected from the composite image with high accuracy.

[0133] Furthermore, as mentioned above (Figure 4), by selecting and applying a more suitable assignment method (in this case, the "BRG" pattern), it is possible to create a higher-quality composite image that reflects the characteristic information of various inspection images. This improves the accuracy of inference and inspection.

[0134] As mentioned above, the accuracy of image recognition / inspection may vary depending on the assignment method. Figure 15 shows examples of composite defective images when the assignment method differs from that in Figure 14. (A) is composite image 1501 for the same "BRG" pattern as in Figure 14, for comparison. (B) is composite image 1502 when the "GRB" pattern is used.

[0135] In composite image 1502, the micropipe defect is less visible compared to composite image 1501. For example, in composite image 1502, it is difficult to determine how far the strain extends from the center of the region outwards. When each composite image is input into its respective model for defect inference, the accuracy will differ.

[0136] Micropipe defects are one of the typical types of defects that can occur in SiC epitaxial wafers, but they can also occur in other types of samples. The method of this embodiment is similarly applicable and effective for other samples and other types of defects.

[0137] In the above embodiment, assuming the use of a deep learning CNN, a preferred example of using 3 channels is shown, where a pseudo-color image is constructed using R, G, and B layers. The number of channels is not limited to this.

[0138] [Different version: 2channel] As a variation, it is also possible to configure the system to synthesize two types of images obtained from two different inspection methods and imaging means, assigning them to two channels, and then performing learning and inference.

[0139] Figure 16 shows a modified example in which a first image 1601 obtained by optical microscopy and a second image 1602 obtained by X-ray topography are assigned to two channels (1st ch, 2nd ch) and combined to obtain a combined image 1603. When using two types of inspection methods / imaging means, for example, two types selected from optical microscopy, photoluminescence, and X-ray topography may be used. For example, a combination of optical microscopy and X-ray topography may be used.

[0140] As an example of the configuration of the image input stage to the model (CNN: Faster-RCNN), input data (Pmulti) 1604 corresponding to the synthesized image 1603 is subjected to a filter 1606 (F1, ..., F64) and a convolution operation 1605, resulting in a feature map 1607 (C1, ..., C64).

[0141] [Modified version: 4 channels or more] As a variation, it is also possible to configure the system to synthesize four or more images obtained from four or more inspection methods / imaging means, assigning them to four or more channels, and then performing training and inference. In this case, the image data for training and inference is multi-channeled beyond three channels. Note that when using four or more channels, a 3-channel R, G, B layer model cannot be used as is. When using four or more channels, the image data of those multiple channels should be assigned to the corresponding multiple channels on the model (CNN).

[0142] Figure 17 shows a modified example of the synthesis of four types of images in the case of 4 channels as multi-channel data, and an example of the configuration of the image input stage to CNN (Faster-RCNN). It has the first image 1701 to the fourth image 1704. The first image 1701 to the third image 1703 are images obtained by the first to third inspection methods as described above. The fourth image 1704 may be an image obtained by the fourth inspection method, or an image obtained by changing the imaging conditions of any of the first to third inspection methods described above. In this example, the third image 1703 is an image taken with an X-ray topography device under the first imaging conditions (first parameter value), and the fourth image 1704 is an image taken with an X-ray topography device under the second imaging conditions (second parameter value).

[0143] In Faster-RCNN, regardless of the pixel size of the input image data, each channel is first reshaped to a predetermined pixel size, for example, 224 x 224 pixels. The reshaped input data (Pmulti) 1706 corresponding to the composite image 1705 is 4-channel image data with an image size of 224 x 224. The input data 1706 is subjected to a convolution operation 1707 with the filter 1708 (F1, ..., F64), and as a result of the operation, a feature map 1709 (C1, ..., C64) is obtained.

[0144] As described above, when the input data consists of four or more types of image data, the input data becomes Pmulti(x,224,224), which is obtained by changing the three channels mentioned earlier to four or more x channels. The filter is also adjusted to match the number of channels in the input data. The convolution operation is the same as in the 3-channel case, with only the number of channels increased. The generation of 64 feature maps as output is also the same. The calculations within the CNN after the input stage are also the same as in the 3-channel case. In this way, when increasing the number of channels, it is only necessary to change the configuration of the convolution operation section in the input stage.

[0145] Figure 18 shows a modified example, similarly with 5 channels. It has the first image 1801 to the fifth image 1805. The fourth image 1804 and the fifth image 1805 may be images obtained by the fourth or fifth inspection method, or images obtained by the first to third inspection methods described above with different imaging conditions. In this example, the third image 1803 is an image taken with an X-ray topography device under the first imaging conditions (first parameter value), the fourth image 1804 is an image taken with a photoluminescence device under the second imaging conditions (second parameter value), and the fifth image 1805 is an image taken with an X-ray topography device under the second imaging conditions (second parameter value). The input data (Pmulti) 1807 to the CNN (Faster-RCNN) is 5-channel image data with an image size of 224 × 224. The input data 1807 is subjected to a convolution operation 1808 with the filter 1809 (F1, ..., F64), and the result of the operation is a feature map 1810 (C1, ..., C64).

[0146] When using four or more types of inspection methods and imaging means, for example, in addition to optical microscopes, photoluminescence, and X-ray topography, a polarizing microscope may be used as a fourth type of imaging device. A microwave photoconductivity attenuation (μ-PCD) measuring device may be used as a fifth type of imaging device, and a mirror electron microscope may be used as a sixth type of imaging device.

[0147] [Second system configuration example] In the system of the above embodiment (Figure 8), a separate server device is provided in addition to the three types of imaging / inspection devices, and the main processing performed by this server device is learning and inference. However, the system is not limited to this, and various system configurations are possible. Modifications are shown below. The server device may be a computer installed in one of the imaging / inspection devices.

[0148] Figure 19 shows a second system configuration example as a modification. The second system configuration example shows a case where an integrated device 1900 (optical-photoluminescence integrated device) is used, which integrates an optical microscope and a photoluminescence device into one. This system has an integrated device 1900 and an X-ray topography device 803 as imaging devices, which are connected by communication as appropriate or data is transferred using a recording medium. Furthermore, the second system configuration example shows a case where learning and inference are performed in the integrated device 1900, rather than a server device. In this case, the integrated device 1900 corresponds to both a learning device and an inference device. In this system, the integrated device 1900 (corresponding computer) acquires images from the X-ray topography device 803 and performs the main processing on the three types of acquired images.

[0149] In Figure 19, the integrated device 1900 is a device that includes an optical microscope 801 and a photoluminescence device 802 as functional units. The controller 1901 of the integrated device 1900 controls the optical microscope 801 and the photoluminescence device 802. In addition to its control function, the controller 1901 also has a learning function 1900A and an inference function 1900B.

[0150] When inspecting the sample 810, the integrated device 1900 uses the optical microscope 801 and the photoluminescence device 802 to capture a first image and a second image, and also acquires a third image from the X-ray topography device 803 via communication or recording medium. The controller 1901 uses the three acquired images to perform learning processing by the learning function 1900A, or inference (inspection) processing by the inference function 1900B.

[0151] Another variation is that the X-ray topography apparatus 803 (corresponding computer system) acquires first and second images from the integration apparatus 1900, and performs learning and inference as the main processing on the three acquired images.

[0152] Beyond the examples above, various integrated device configurations are possible for various imaging / inspection devices. Furthermore, beyond the examples above, configurations where the learning and inference functions are handled by other devices are also possible. Additionally, a configuration where the learning device and the inference (inspection) device are separate may be adopted.

[0153] Figure 20 shows a third system configuration example as a modified example. In this system, there are two server devices, 2001 of the first operator and 2002 of the second operator, connected to various imaging devices (801-803) on the communication network 809. The first operator's server device 2001 performs learning processing as a learning device equipped with a learning function 2000A. The second operator's server device 2002 performs inference (inspection) processing as an inference device (inspection device) equipped with an inference function 2000B. The server device 2001 acquires various images (training images) from various imaging devices (801-803) and trains a model. The server device 2002 sets up the trained model, acquires various images (inspection target images) from the imaging devices (801-803), inputs them into the model, and obtains inspection results.

[0154] The computers performing the various image capture, acquisition, storage, image adjustment, image synthesis, model training, and inference processes described above may be the same computer, or they may be multiple different computers connected by communication or capable of data transfer using recording media.

[0155] [Example: Input color image] Other possible modifications include the following. In the above embodiment, as shown in Figure 2, the image captured by the inspection method and imaging means before synthesis was a monochrome grayscale image, but it is not limited to this and may be a color image. In the case of a color image, the following method can be applied. For example, suppose the first image captured with an optical microscope is a color image.

[0156] Method 1: Convert a color image to a monochrome grayscale image and assign it to a channel for compositing.

[0157] Method 2: A method in which color images, for example, images with R, G, and B values, are assigned to multiple channels (e.g., 3 channels) for synthesis, and other grayscale images are assigned to other channels. During assignment, conversion and adjustment to the aforementioned unified format and specifications are performed as appropriate. For example, color images from an optical microscope are assigned to channels 1 to 3, and other grayscale images such as a photoluminescent image are assigned to channel 4, and an X-ray topography image is assigned to channel 5. A total of 5 channels are used.

[0158] Method 3: A method in which one or two color values ​​are selected from a color image, for example, an image with R, G, and B values, to be used for synthesis, and the selected color values ​​are assigned to the corresponding number of channels. For example, only the R value is selected from a color image obtained by an optical microscope and assigned to the 1st channel. The image obtained by photoluminescence is assigned to the 2nd channel, and the image obtained by X-ray topography is assigned to the 3rd channel. A total of 3 channels are used.

[0159] [Modified form: Inspection method, imaging means] The above embodiment shows a configuration using three types of inspection methods (optical microscope, photoluminescence, and X-ray topography), but additional inspection methods may be added to further improve accuracy. For example, while we have assumed one type of image using the photoluminescence method, the contrast differs depending on the type of defect if the wavelength band of the light used for imaging is different. Therefore, using photoluminescence light of multiple wavelength bands (corresponding imaging conditions) can be expected to improve accuracy. Similarly, in the X-ray topography method, the crystal planes that are diffracted differ by changing the energy and angle of the incident X-rays, and the contrast differs depending on the type of defect.

[0160] Alternatively, completely different inspection methods other than the three types of inspection methods described above may be used. For example, in a method that visualizes the reflection intensity of an incident electron beam dependent on the wafer surface potential, such as a mirror electron microscope, a contrast corresponding to the electrical properties of each type of defect can be obtained. Other methods such as polarized light microscopy and μ-PCD may also be used. In polarized light microscopy, polarized light is incident on the wafer, and the contrast of defects is obtained by detecting the birefringence characteristics of the transmitted light. In the μ-PCD method, a laser is locally irradiated to excite carriers, and the local carrier lifetime is measured from the attenuation rate of the reflection intensity of the simultaneously irradiated microwaves. The lifetime distribution is then imaged by scanning the laser on the wafer. This provides a contrast between normal areas and defects with different lifetimes.

[0161] [Variation: Sample] The applicable samples are not limited to SiC epitaxial wafers, but also include other semiconductors such as SiC bulk wafers, GaN, and diamond. Furthermore, it is applicable not only to semiconductors but also to compounds other than SiC.

[0162] [Variation: Control according to sample and defect type] The above-described embodiment (the first embodiment) designs multiple types of inspection methods and imaging devices with different imaging principles, as well as multiple channel assignment methods, corresponding to a certain target sample (SiC epitaxial wafer) and defect type (micropipe, etc.). However, the applicable inspection methods, imaging devices, and assignment methods may be changed depending on the target sample and defect type. For example, the correspondence may be managed by using the first imaging device set for the first sample and the second imaging device set for the second sample. Alternatively, the correspondence may be managed by using the first assignment method for the first defect type of the first sample and the second assignment method for the second defect type of the first sample. The management can be similarly applied to data as shown in Figure 7. Since the imaging principle and assignment method that yield high accuracy may change depending on the sample and defect type, it is advisable to select a suitable imaging principle and assignment method.

[0163] [Modified version: Imaging conditions] In the above-described embodiment, the three types of imaging devices were shown to capture images under predetermined imaging conditions (corresponding parameter values). However, as a variation, the same imaging device may be used to capture and use multiple images under different imaging conditions. The obtained multiple images can then be similarly assigned to multiple channels and combined.

[0164] For example, in an X-ray topography system, a first image is acquired under first imaging conditions, and simultaneously or sequentially, a second image is acquired under second imaging conditions. These two images are assigned to two channels. This configuration corresponds to the third image 1703 and the fourth image 1704 in Figure 17 mentioned above.

[0165] [Note] This embodiment includes the following configuration.

[0166] (1) An image learning method for learning images and constructing a model, comprising the steps of: acquiring multiple types of images taken of a sample using multiple different imaging principles, as steps performed by a computer system; generating a composite image, which is an image obtained by aligning the acquired multiple types of images, unifying their specifications, and assigning them to multiple channels using a predetermined assignment method; and learning the model using the composite image as input and defect classification information relating to defects in the sample as output.

[0167] (2) The image learning method described in (1), wherein the plurality of types of images are captured by a plurality of different types of imaging devices.

[0168] (3) An image learning method according to (1) or (2), wherein the assignment method is selected from all combinations of assignment patterns to have the highest accuracy with respect to the output of the model or the highest accuracy is higher than a threshold.

[0169] (4) An image learning method according to (1) or (2), comprising the step of comprehensively learning all combinations of assignment patterns in the assignment method, evaluating the accuracy of the model's output for each pattern, and selecting the one pattern with the highest accuracy or the one pattern with accuracy above a threshold as a result of the evaluation, and deciding to use it in the inference phase.

[0170] (5) An image learning method according to any of (1) to (4), wherein three types of images captured as multiple types of images are assigned to three channels corresponding to the R (red), G (green), and B (blue) layers to generate the composite image.

[0171] (6) An image learning method according to any of (1) to (4), wherein two types of images captured as multiple types of images are assigned to two channels to generate the composite image.

[0172] (7) An image learning method according to any of (1) to (4), wherein four or more images captured as multiple types of images are assigned to four or more channels to generate the composite image.

[0173] (8) An image learning method according to any of (2) to (7), wherein the plurality of types of imaging devices include an optical microscope, photoluminescence, and X-ray topography.

[0174] (9) An image learning method according to any of (2) to (8), wherein the sample is a SiC wafer, the plurality of imaging devices include three types: an optical microscope, a photoluminescence device, and an X-ray topography device, and the three types of images captured by the three types of imaging devices are assigned to three channels corresponding to the R (red), G (green), and B (blue) layers to generate the composite image, and the assignment method is one of the following patterns: a pattern assigning {a first image from an optical microscope, a second image from photoluminescence, and a third image from X-ray topography} to {layer B, layer R, and layer G}, a pattern assigning to {layer R, layer G, and layer B}, and a pattern assigning to {layer G, layer B, and layer R}.

[0175] (10) An image learning device that performs each step of the image learning method described in any of (1) to (9).

[0176] (11) An image inspection method for classifying defects from an image based on a model that has learned images, comprising the steps of: acquiring multiple types of images taken of a sample using multiple different imaging principles, each of which images are acquired from the sample; generating a composite image, which is a single image obtained by aligning the acquired multiple types of images, unifying their specifications, and assigning them to multiple channels using a predetermined assignment method; and inputting the composite image into the model and obtaining defect classification information regarding defects in the sample as the output of the inference result by the model.

[0177] (12) An image inspection method according to (11), wherein the plurality of types of images are captured by a plurality of different types of imaging devices.

[0178] (13) An image inspection method according to (11) or (12), wherein three types of images captured as multiple types of images are assigned to three channels corresponding to the R (red), G (green), and B (blue) layers to generate the composite image.

[0179] (14) An image inspection device that performs each step of the image inspection method described in any of (11) to (13).

[0180] (15) A program (learning program) for causing a computer to perform each step of the image learning method described in any of (1) to (9). Or, a recording medium storing said program.

[0181] (16) A program (inference program / inspection program) for causing a computer to perform each step of the image inspection method described in any of (11) to (13). Or, a recording medium storing said program.

[0182] (17) A semiconductor device manufacturing method comprising: an inspection step of performing an image inspection that classifies defects from images based on a model that has learned images; and a step of shipping semiconductor devices that have passed the inspection step and been determined to be good products, wherein the inspection step is a step of performing each step of the image inspection method described in any of (11) to (13) by a computer system.

[0183] (18) An image learning method according to any of (2) to (9), wherein the plurality of types of images by the plurality of types of imaging devices include a first image captured by a first imaging device under first imaging conditions and a second image captured by the first imaging device under second imaging conditions, and the plurality of types of images including the first image and the second image are assigned to the plurality of channels to generate the composite image.

[0184] (19) For example, an image learning device for learning images and constructing a model, wherein the device acquires multiple types of images taken of a sample using multiple different imaging principles, aligns the positions of the acquired multiple types of images, unifies their specifications, assigns them to multiple channels using a predetermined assignment method, generates a composite image which is a single composite image, takes the composite image as input and outputs defect classification information regarding defects in the sample, and learns the model.

[0185] (20) For example, the image learning device includes a processor and memory, the memory which stores a program for executing each step of any of the image learning methods (1) to (10), and data related to learning, and the processor performs processing according to the program to acquire the plurality of types of images, generate the composite image, and learn the model.

[0186] (21) For example, an image inspection method for classifying defects from an image based on a model that has learned images, the method comprising the steps of: acquiring multiple types of images taken of a sample using multiple different imaging principles, each acquired by a computer system; generating a composite image by aligning the acquired multiple types of images, unifying their specifications, and assigning them to multiple channels using a predetermined assignment method to create a single composite image; and inputting the composite image into the model and obtaining defect classification information regarding defects in the sample as the output of the inference result by the model.

[0187] (22) For example, an image inspection device that classifies defects from an image based on a model that has learned images, wherein it acquires multiple types of images taken of a sample using multiple different imaging principles, aligns the positions of the acquired multiple types of images, unifies their specifications, assigns them to multiple channels using a predetermined assignment method, generates a composite image which is a single composite image, inputs the composite image into the model, and obtains defect classification information regarding defects in the sample as the output of the inference result by the model.

[0188] (23) For example, the image inspection apparatus includes a processor and memory, the memory which stores a program for executing each step of any of the image inspection methods (11) to (13), and data relating to the inspection, and the processor performs processing according to the program to acquire the plurality of types of images, generate the composite image, and obtain the defect classification information from the model.

[0189] For example, in the system shown in Figure 8, the server device (the computer in Figure 9) stores the learning program and inference program in its memory resources. The server device's processor executes learning processes according to the learning program read into memory, or inference processes according to the inference program. This realizes the learning and inference functions. The computer may also read these programs from an external storage medium.

[0190] (Model) A model / recording medium that causes a computer to perform the processing of classifying defects from an image based on a model that has learned images, wherein the computer performs the steps of: acquiring multiple types of images taken of a sample using multiple different imaging principles; generating a composite image which is an image obtained by aligning the acquired multiple types of images, unifying their specifications, and assigning them to multiple channels using a predetermined assignment method; and inputting the composite image into the model and obtaining defect classification information regarding defects in the sample as the output of the inference result by the model.

[0191] For example, in the system shown in Figure 8, the server device (the computer in Figure 9) stores the machine learning model data in memory resources. The server device's processor performs learning or inference processing based on the machine learning model data read into memory. This enables the learning and inference functions. The computer may also read the machine learning model data from an external storage medium.

[0192] Although embodiments of this disclosure have been described in detail above, the invention is not limited to the embodiments described above and can be modified in various ways without departing from the gist of the invention. Each embodiment can be modified by adding, deleting, or replacing components, except for essential components. Unless otherwise specified, each component may be singular or plural. Combinations of each embodiment and its variations are also possible. [Explanation of Symbols]

[0193] 101...Inspection image (training data), 102...Assignment and synthesis, 103...Synthesized image, 104...Machine learning, 105...Model, 106...Inspection image (data to be inspected), 107...Assignment and synthesis, 108...Synthesized image, 109...Inference and judgment, 110...Inspection result recording and output.

Claims

1. This is an image learning method that builds a model by learning from images, As a step performed by a computer system, The steps include acquiring multiple types of images of a sample, each captured using multiple different imaging principles, and The steps include: generating a composite image by aligning the positions of the acquired multiple types of images, unifying their specifications, assigning them to multiple channels using a predetermined assignment method, and combining them into a single image; The steps include: training the model using the synthesized image as input and defect classification information regarding defects in the sample as output; An image learning method having the following characteristics.

2. In the image learning method described in claim 1, The aforementioned multiple types of images were captured by multiple different types of imaging devices. Image learning methods.

3. In the image learning method described in claim 1, The aforementioned assignment method involves selecting one pattern from all possible combinations of assignments that has the highest accuracy regarding the output of the model, or one pattern whose accuracy is higher than a threshold. Image learning methods.

4. In the image learning method described in claim 1, The assignment method includes a step of comprehensively learning all possible combinations of assignment patterns, evaluating the accuracy of the model's output for each pattern, and selecting the pattern with the highest accuracy or the pattern with accuracy above a threshold as a result of the evaluation, and deciding to use it in the inference phase. Image learning methods.

5. In the image learning method described in claim 1, The three images captured as multiple types of images are assigned to three channels corresponding to the R (red), G (green), and B (blue) layers to generate the composite image. Image learning methods.

6. In the image learning method described in claim 1, Two types of images captured as multiple types of images are assigned to two channels to generate the composite image. Image learning methods.

7. In the image learning method described in claim 1, The four or more images captured as multiple types of images are assigned to four or more channels to generate the composite image. Image learning methods.

8. In the image learning method described in claim 2, The aforementioned plurality of imaging devices include optical microscopes, photoluminescence, and X-ray topography. Image learning methods.

9. In the image learning method described in claim 2, The aforementioned sample is a SiC wafer, The aforementioned multiple types of imaging devices include three types: optical microscopes, photoluminescence, and X-ray topography. The three images captured by the three types of imaging devices are assigned to three channels corresponding to the R (red), G (green), and B (blue) layers to generate the composite image. The aforementioned assignment method is one of the following patterns: assigning {the first image obtained by optical microscopy, the second image obtained by photoluminescence, and the third image obtained by X-ray topography} to {layer B, layer R, and layer G}, assigning them to {layer R, layer G, and layer B}, or assigning them to {layer G, layer B, and layer R}. Image learning methods.

10. An image learning device that performs each step of the image learning method according to any one of claims 1 to 9.

11. An image inspection method that classifies defects from images based on a model that has learned images, As a step performed by a computer system, The steps include acquiring multiple types of images of a sample, each captured using multiple different imaging principles, and The steps include: generating a composite image by aligning the positions of the acquired multiple types of images, unifying their specifications, assigning them to multiple channels using a predetermined assignment method, and combining them into a single image; The steps include inputting the synthesized image into the model and obtaining defect classification information regarding defects in the sample as the output of the inference result by the model, An image inspection method having the following characteristics.

12. In the image inspection method according to claim 11, The aforementioned multiple types of images were captured by multiple different types of imaging devices. Image inspection methods.

13. In the image inspection method according to claim 11, The three images captured as multiple types of images are assigned to three channels corresponding to the R (red), G (green), and B (blue) layers to generate the composite image. Image inspection methods.

14. An image inspection apparatus that performs each step of the image inspection method according to any one of claims 11 to 13.

15. A program for causing a computer to perform each step of the image learning method described in any one of claims 1 to 9.

16. A program for causing a computer to perform each step of the image inspection method described in any one of claims 11 to 13.

17. A semiconductor device manufacturing method comprising: an inspection step of performing image inspection to classify defects from images based on a model that has learned images; and a step of shipping semiconductor devices that have passed the inspection step and been determined to be good products, The inspection step is a step in which each step of the image inspection method described in any one of claims 11 to 13 is performed by a computer system. Semiconductor device manufacturing method.

Images