Image learning method and apparatus, image inspection method and apparatus, program, and semiconductor device manufacturing method

Abstract

Provided is technology that makes it possible to achieve improvements in efficiency and accuracy, and the like, with respect to an image inspection technique for semiconductors and the like. This image learning method comprises, as steps to be executed by a computer system: a step for acquiring multiple types of images (101) of a sample that have been respectively captured using multiple different imaging principles; a step (102) for aligning the acquired multiple types of images, unifying specifications of the images, allocating the images to multiple channels using a prescribed allocation method, and generating a composite image, which is an image synthesized from the images; and a step (104) for training a model using the composite image (103) as an input and using defect classification information relating to defects in the sample as an output.

Description

Image learning method and apparatus, image inspection method and apparatus, program, and semiconductor device manufacturing method

[0001] The present disclosure relates to technologies such as learning and inference using images. In particular, it relates to technologies such as imaging, measurement, inspection, and evaluation for classifying, extracting, and determining abnormalities and defects from images of samples such as semiconductor wafers. In this specification, imaging, measurement, inspection, evaluation, etc. may be collectively referred to as inspection.

[0002] For example, in the manufacture of semiconductor wafers using semiconductor materials such as SiC, inspections are performed on defects that may occur on the semiconductor wafers.

[0003] As prior art examples, JP-A-2023-138330 (Patent Document 1) and JP-A-2023-114827 (Patent Document 2) can be cited.

[0004] Patent Document 1 describes that in the manufacture of products by soldering, "using a machine learning model that outputs inspection results in an inspection after reflow from the input of image data based on an image before reflow, the quality (good or bad) in an inspection scheduled to be performed after reflow is determined from the image data based on the image before reflow acquired in real time."

[0005] Patent Document 2 describes that "it includes an image storage unit 21 that stores three inspection images having different transmission characteristics for the inspection object W obtained by imaging the inspection object W as pseudo-RGB images, and for the pseudo-RGB images stored in the image storage unit 21, a pixel-by-pixel processing is performed based on a learning model 22 created in advance by learning using an image in the same format as the pseudo-RGB images to obtain a quality defect degree, and a determination unit 24 that determines the quality state of the inspection object W by comparing the quality defect degree with a preset threshold value."

[0006] Japanese Patent Application Laid-Open No. 2023-138330, Japanese Patent Application Laid-Open No. 2023-114827

[0007] To improve the efficiency and accuracy of the above-mentioned inspections, it is conceivable to apply machine learning / artificial intelligence (AI)-based image inspection. In this image inspection, a sample such as a semiconductor wafer is imaged using an imaging device, the image obtained is input into a pre-trained model, and the model's inference results are output, such as defect classification.

[0008] As an example of prior art, defect inspection of SiC epitaxial wafers employs, for example, three different inspection methods and corresponding imaging means (in other words, image generation means) with varying sensitivities to defects. For example, these three inspection methods and imaging means are optical microscopes, photoluminescence (PL), and X-ray topography (XRT).

[0009] An optical microscope works by shining light onto a sample and using lenses to form an image of the transmitted or reflected light for observation.

[0010] Photoluminescence (PL) is the process by which a substance absorbs light and then re-emits it. A PL device irradiates a sample with high-energy light and detects the light re-emitted from the sample.

[0011] X-ray topography (XRT) is a method of visualization that involves irradiating a sample with X-rays and measuring the intensity of the diffracted X-rays.

[0012] When using the three inspection methods described above for defect inspection of SiC epitaxial wafers, each inspection method requires the acquisition of inspection images and the classification and extraction of defects from those images, which is time-consuming and laborious. Furthermore, when applying machine learning using images to each inspection method, each method requires a separate learning phase and inference phase, which is even more time-consuming and laborious. In the learning phase, it is necessary to acquire a large number of training images and build and train a machine learning model, which can take an enormous amount of time. In addition, it is necessary to separately construct algorithms to combine the defect classification results from each inspection method's model to obtain a single final defect classification result (in other words, the inspection result).

[0013] The purpose of this disclosure is to provide a technology that can improve efficiency and accuracy in image inspection technology for semiconductors and the like.

[0014] A representative embodiment (embodiment) of this disclosure has the following configuration. One embodiment is an image learning method for learning images and constructing a model, the steps of which are performed by a computer system include: acquiring multiple types of images taken of a sample using multiple different imaging principles; generating a composite image, which is an image obtained by aligning the acquired multiple types of images, unifying their specifications, and assigning them to multiple channels using a predetermined assignment method; and learning the model using the composite image as input and defect classification information regarding defects in the sample as output.

[0015] Furthermore, one embodiment is an image inspection method for classifying defects from an image based on a model that has learned images, and the steps performed by a computer system include: acquiring multiple types of images taken of a sample using multiple different imaging principles; generating a composite image, which is a single image obtained by aligning the acquired multiple types of images, unifying their specifications, and assigning them to multiple channels using a predetermined assignment method; and inputting the composite image into the model and obtaining defect classification information regarding defects in the sample as the output of the inference result by the model.

[0016] According to a representative embodiment of this disclosure, it is possible to improve efficiency and accuracy in the image inspection technology for semiconductors and the like. Other issues, configurations, and effects will be shown in the embodiments for carrying out the invention.

[0017] This embodiment (Embodiment 1) shows an overview of a system and method for learning and inferring (inspecting) images in semiconductor wafer defect inspection. This embodiment shows examples of three types of images obtained by three types of inspection methods and imaging means targeting defects in a semiconductor wafer. This embodiment shows examples of assigning and combining the R, G, and B layers for the three types of images. This embodiment shows efficiency, etc., according to the assignment method for image learning. This embodiment shows the learning processing flow. This embodiment shows the inferring (inspection) processing flow. This embodiment shows an example of the data configuration for managing the assignment method, etc. This embodiment shows an example of the first system configuration. This embodiment shows an example of the computer system configuration. This embodiment shows examples of processing during learning and inferring. This embodiment shows an example of adjustment such as alignment of multiple images. This embodiment shows an example of the machine learning model configuration. This embodiment shows an example of the input stage configuration to the model (CNN) in the case of a 3-channel image. This embodiment shows specific examples of defects and images. This embodiment shows specific examples of composite images with different assignment methods. A diagram showing an example of the synthesis and input stage configuration for a 2-channel system in a modified version of this embodiment. A diagram showing an example of the synthesis and input stage configuration for a 4-channel system in a modified version of this embodiment. A diagram showing an example of the synthesis and input stage configuration for a 5-channel system in a modified version of this embodiment. A diagram showing a second system configuration example in a modified version of this embodiment. A diagram showing a third system configuration example in a modified version of this embodiment.

[0018] Embodiments of this disclosure will be described in detail below with reference to the drawings. In the drawings, the same parts are generally denoted by the same reference numerals, and repeated descriptions are omitted. In the drawings, the representation of components may not show their actual location, size, shape, extent, etc., in order to facilitate understanding of the invention.

[0019] In explanations, when describing program-based processing, the focus may sometimes be on the program, functions, or processing units. However, the main hardware component is the processor, or a controller, device, computer, or system composed of such a processor. The computer, using its processor, executes processing according to the program read into memory, utilizing resources such as memory and communication interfaces as appropriate. This realizes the specified functions and processing units. The processor is composed of semiconductor devices such as a CPU / MPU or GPU. Processing is not limited to software program processing; it can also be implemented using dedicated circuits. Applicable dedicated circuits include FPGAs, ASICs, CPLDs, etc.

[0020] The program may be pre-installed as data on the target computer, or it may be distributed as data to the target computer from the program source. The program source may be a program distribution server on a communication network, or a non-transient computer-readable storage medium, such as a memory card or disk. The program may consist of multiple modules. The computer system may consist of multiple devices. The computer system may consist of a client-server system, a cloud computing system, an IoT system, etc. Various types of data and information are composed of structures such as tables and lists, but are not limited to these. Representations such as identification information, identifiers, IDs, names, and numbers are interchangeable.

[0021] [Solution, etc.] This embodiment relates to a method and apparatus for performing defect inspection on semiconductors such as SiC epitaxial wafers as samples. This embodiment particularly targets crystal defects in SiC. This embodiment applies machine learning using images, in other words, image inspection, to this defect inspection. This embodiment uses, for example, three types of inspection methods and corresponding imaging means (imaging methods, imaging devices, etc.) that have different sensitivities to defects and imaging principles for the sample. In this embodiment, an optical microscope, a photoluminescence device, and an X-ray topography device are used as the three types of inspection methods and imaging devices. In this embodiment, for example, the wafer surface of the same sample is imaged using three types of imaging devices (in other words, inspection devices, etc.) and three types of images (sometimes referred to as inspection images) are obtained.

[0022] This embodiment uses three acquired images as input to a machine learning model. This embodiment assigns the three acquired images (for example, three images as a set) to three channels corresponding to the three primary colors R (red), G (green), and B (blue) in a color image. This embodiment combines these three channel images to obtain a single pseudo-color image as a composite image. This embodiment inputs this single pseudo-color image, the composite image, into a single machine learning model and obtains an automatic defect classification result (in other words, a defect inspection result) as the output of the model's inference results.

[0023] Machine learning generally consists of a training phase and an inference phase. In the training phase, a machine learning model is constructed and trained using the pseudo-color images mentioned above as training images. For example, deep learning can be applied to the machine learning model. For example, an object detection CNN model can be applied to the machine learning model. After sufficient training, the machine learning model becomes a defect classifier. In the inference phase (in other words, the inspection phase), during actual manufacturing inspection, the trained model (defect classifier) ​​is used to acquire multiple images (inspection target images) of the sample / defect to be inspected, captured using three different inspection methods and imaging means, including those used during training. In this embodiment, multiple images are combined into a single image, similar to the training phase. In this embodiment, the combined image is input to the model (defect classifier), and an automatic defect classification result (in other words, defect inspection result) is obtained as the output of the inference result.

[0024] This embodiment assigns multiple types of images corresponding to multiple inspection methods to multiple channels (three channels when using pseudo-color images of R, G, and B) and synthesizes them into a single image. This embodiment uses this single synthesized image as input to construct a single machine learning model. This embodiment allows for the construction and management of a single model, rather than three separate models for each inspection method, through synthesis. As a result of training this model with the synthesized image, this embodiment can provide a single defect classifier and an inspection device / inspection system equipped with it.

[0025] According to this embodiment, when using multiple types of inspection methods, the effort and time required can be reduced. According to this embodiment, compared to training multiple machine learning models corresponding to multiple types of inspection methods, the learning efficiency is increased and the accuracy of the inspection is improved.

[0026] [Regarding Image Assignment] This embodiment assigns multiple types of images to multiple channels (in other words, associates them, etc.) and combines the assigned images into a single image. The method of this assignment has the following characteristics.

[0027] When using multiple inspection methods and imaging means, such as an optical microscope, photoluminescence, and X-ray topography, the three types of images acquired / generated by these methods are referred to as the first image, the second image, and the third image. These multiple images are then assigned to three channels (1st ch, 2nd ch, 3rd ch) corresponding to the layers of each color, for example, {R, G, B}. There are a total of 3 × 2 × 1 = 6 combinations (patterns) of assignment methods. The following are representations of these six patterns.

[0028] Pattern 1: {R, G, B} Pattern 2: {R, B, G} Pattern 3: {G, R, B} Pattern 4: {G, B, R} Pattern 5: {B, R, G} Pattern 6: {B, G, R}

[0029] For example, the notation in Pattern 1 above means a method of assigning {optical microscope, photoluminescence, X-ray topography} to {R (1st channel), G (2nd channel), B (3rd channel)}. The order of the elements corresponds to the assignment.

[0030] Here, the inventors' investigations revealed that the effectiveness, such as the accuracy of defect classification, differs depending on the assignment method. It was found that when assigning multiple channels {R, G, B} to multiple types of images and synthesizing them into a single image, changing the assignment pattern results in differences in accuracy, etc. Therefore, this embodiment can enhance its effectiveness by selecting a suitable assignment method that yields higher accuracy, etc.

[0031] Furthermore, the following embodiments of the above allocation method can be cited. Any of these embodiments are possible.

[0032] (1) In the first embodiment, the design involves experiments and studies to determine which assignment is preferable, and based on that, for example, one assignment method that yields the highest accuracy is decided, and the inspection method and inspection device are configured to implement that one assignment method in a fixed manner. In the learning and inference phases, processing is carried out according to that one assignment method.

[0033] (2) In the second embodiment, instead of fixing on one assignment method, the inspection method and inspection device are configured such that, in the learning phase, multiple patterns of assignment methods are comprehensively tried, and based on the results of the trials and evaluation of each pattern, the most suitable assignment method or an assignment method with accuracy higher than a predetermined threshold is selected and determined. The procedures and functions for the trials are implemented in this inspection method and inspection device. In the actual inspection (inference phase), the model (defect classifier) ​​trained with the most suitable assignment method selected as a result of the trials is used. The second embodiment is useful when it is unclear which assignment method is best for a sample or type of defect.

[0034] [Learning and Inference] This embodiment applies a machine learning model to defect inspection of, for example, SiC epitaxial wafers to perform image inspection, and has a learning phase and an inference phase in machine learning technology. The categories of this embodiment are broadly inspection methods, inspection systems, inspection equipment, inspection programs, etc. related to defect inspection (especially image inspection), and in terms of machine learning technology, they are learning methods, learning equipment, inference methods, inference equipment, etc. The learning phase is another way of saying training phase. The inference phase corresponds to the phase in which a sample is inspected during actual manufacturing using the trained model, and the inference equipment corresponds to the actual inspection equipment, etc. In the following, the learning phase and the inference phase will be explained separately as appropriate.

[0035] <Embodiment 1> An embodiment of the present disclosure (referred to as Embodiment 1) will be described using Figure 1 and subsequent figures.

[0036] In this embodiment, for the semiconductor wafer sample, images are first captured for each of three inspection devices (corresponding imaging devices, etc.) corresponding to, for example, three types of inspection methods and imaging means, and three types of images (corresponding image data, etc.) are acquired. In this embodiment, instead of individually training the image data of the three acquired images using three different machine learning models, the images are assigned to multiple channels to form a single composite image, and then trained using a single machine learning model. In this embodiment, the three images are assigned to three channels {R, G, B} and combined into a single pseudo-RGB color image. Similarly, training is repeated using multiple composite images.

[0037] [Overview of Learning and Inference] Figure 1 is an explanatory diagram showing the overview and basic configuration of the system and method for learning and inferring images in semiconductor wafer defect inspection according to this embodiment. Figure 1 shows an example configuration in which three types of images are combined into one and input into a single machine learning model (e.g., a deep learning model). The process shown in Figure 1 is executed by a predetermined computer system (in other words, a learning system and an inspection system). In this embodiment, in order to generate a defect classifier for inspection, machine learning is performed on the inspection images to construct a model (defect classifier).

[0038] In the learning phase, the computer system acquires and inputs inspection images as training data 101. These inspection images (in other words, training images) are a set of multiple images taken of a semiconductor wafer (particularly a SiC epitaxial wafer), which is the sample, corresponding to multiple types of inspection methods and imaging means with different imaging principles. In this example, these multiple images are three types of images taken using an optical microscope, photoluminescence, and X-ray topography. The three corresponding imaging / inspection devices are an optical microscope, a photoluminescence device, and an X-ray topography device.

[0039] The computer system takes this inspection image (training data 101) as input and performs the processing of assigning it to multiple (three) channels, namely the R, G, and B layers, and then synthesizing them 102. As a result, one synthesized image 103 is obtained for each set.

[0040] The computer system takes the synthetic image 103 as input, performs machine learning (deep learning in this example), and constructs a machine learning model 104. The learning is repeated until sufficient accuracy is obtained.

[0041] In the inference phase, or rather, in the inspection phase where the actual inspection is carried out, the computer system acquires and inputs an inspection image as the inspection target data 106. This inspection image (or rather, the inspection target image) is data that consists of a set of multiple images taken for a semiconductor wafer (especially a SiC wafer) as a sample corresponding to multiple types of inspection methods and imaging means, similar to the inspection images during learning. The inspection methods and imaging means used during inference are the same as those used during learning.

[0042] The computer system takes this inspection image (inspection target data 106) as input and performs processing of allocation and synthesis 107 to the R, G, and B layers, which are multiple (three) channels. As a result, one synthetic image 108 (or rather, a synthetic image for inference) is obtained for each set. The processing of this synthesis 107 during inference is also the same as the allocation in the processing of synthesis 102 during learning.

[0043] The computer system inputs the synthetic image 108 into the model 105 and obtains, as the defect inspection result, the output of the inference and determination result by the model 105, that is, the automatic defect classification result. The computer system records and outputs 110 the inspection result to the user or the like.

[0044] [Defect Images and Synthesis] Figures 2 and 3 are explanatory diagrams showing the synthesis (102, 107 in Figure 1) of three types of images in this embodiment. The synthesis in this embodiment is not to arrange multiple images on a plane, but to overlap them by aligning the positions and regions.

[0045] Figure 2 is a schematic explanatory diagram of three types of images (inspection images / defect images) obtained by imaging the same target area on the semiconductor wafer surface using three types of inspection methods and imaging means. A certain type of defect (referred to as defect A) is included in the target area. This inspection image is also described as a defect image. The target area is, for example, a rectangular area from coordinates (x1, y1) to (x2, y2) on the wafer surface (corresponding coordinate system). In Figure 2 and the like, the content of the image is illustrated schematically.

[0046] The first defect image 201 (or the first image, the first type of image) is an image captured by the first inspection method and imaging means (e.g., an optical microscope). Defect A (or the defect area) 211 is shown in the target area.

[0047] The second defect image 202 (or the second image, the second type of image) is an image captured by the second inspection method and imaging means (e.g., photoluminescence). Defect A 212 is shown in the target area.

[0048] The third defect image 203 (or the third image, the third type of image) is an image captured by the third inspection method and imaging means (e.g., X-ray topography). Defect A 213 is shown in the target area.

[0049] For subsequent image synthesis, the position coordinates of each inspection image, for example, the coordinates on the wafer at the lower left and upper right (lower left coordinates (x1, y1), upper right coordinates (x2, y2)), are adjusted during imaging or after imaging (such as cutting out from the captured image) so that they are the same. In Figure 2, defect A is shown to be located at the center of each image, but this is not limited to this case. As long as the coordinates are the same in each image, the defect can be located anywhere in the image. Also, in the example of the images in Figure 2, only one type of defect, defect A, is captured, but multiple types of defects or multiple defects of the same type may be captured in the same image.

[0050] In this embodiment, the three types of images captured by each imaging device are all monochrome grayscale images. In this embodiment, these images are assigned to three channels: R, G, and B. By combining the image data from the three channels of the R, G, and B layers, a pseudo-RGB color image is obtained as a composite image. This composite image reflects and retains characteristic information (such as how defects are depicted) from various inspection images. Therefore, suitable defect classification and inspection can be performed from this composite image through the input and output of a machine learning model.

[0051] The computer system verifies that the parameter values, such as position and size, of multiple images of the sample (which may be the wafer surface or a wafer cross-section) are identical in order to achieve image synthesis. If the images are not identical (if the parameter values ​​differ), the computer system adjusts them to become identical. This adjustment may be performed during imaging by the imaging device, or it may be performed through image processing on the image after imaging. For example, this could include alignment, cropping, and scaling of the image after imaging. Since the size of the image may differ depending on the imaging device, such adjustments are made as needed.

[0052] Furthermore, to achieve image synthesis, the computer system adjusts the color and contrast of each image from multiple image types as needed, and adjusts them to a unified format and specification for synthesis, such as monochrome grayscale. In other words, the computer system performs scaling of the pixel values. Since the scaling of the pixel values ​​may differ depending on the imaging method, such adjustments are made as needed.

[0053] In this example, the composite image format is assumed to be a standard 24-bit image (8 bits each for R, G, and B). In this case, the pre-compositing adjustments are made so that each image is on an 8-bit scale (monochromatic grayscale).

[0054] The input and output images (composite images) are shown as being square in size, for example, but the image size is not limited to this and can be any shape.

[0055] Figure 3 is a schematic diagram illustrating the assignment and synthesis of the three types of images in Figure 2 to the R, G, and B layers. In this example, we show an example of assignment (or mapping) to three types of channels, namely the R, G, and B layers. In the example in Figure 3, the first channel is the R layer, the second channel is the G layer, and the third channel is the B layer. The computer system assigns the first defective image 201 obtained by the first inspection method to the R layer, which is the first channel, the second defective image 202 obtained by the second inspection method to the G layer, which is the second channel, and the third defective image 203 obtained by the third inspection method to the B layer, which is the third channel (corresponding to the "RGB" pattern described later).

[0056] The computer system combines the R, G, and B layers from three assigned images into a single image. The computer system overwrites the R, G, and B data layers with three monochrome grayscale images (201, 202, 203) to obtain a single 24-bit pseudo-RGB color image. This combined pseudo-RGB color image is referred to as the combined image 300 (or in other words, the combined defect image). The region of coordinates (x1, y1) to (x2, y2) in the combined image 300 contains defect A310, which is a composite of defect A from the three images.

[0057] This single composite image 300 reflects and integrates characteristic information of three types of defects A obtained by three different inspection methods and imaging means. From this information, including defects A 310, it is possible to classify and determine defects using a machine learning model.

[0058] In the learning phase, the computer system uses the synthesized image 300 in Figure 3, obtained based on the training images, as the synthesized image 103 in Figure 1, and performs machine learning 104. During learning, the computer system adjusts the parameter values ​​of the machine learning model 105 using the input synthesized image 103 and the output label information (defect classification information). Through learning that achieves sufficient accuracy, the model 105 becomes a defect classifier.

[0059] In the inference phase (in other words, the inspection phase), the computer system (in other words, the inspection device) similarly creates the composite image 300 shown in Figure 3 based on the image to be inspected, which becomes the composite image 108 shown in Figure 1. The computer system inputs the composite image 108 into the model 105 and obtains the defect inspection result (defect classification result) as the output of the inference result.

[0060] In this embodiment, by combining three types of images into a single composite image, the number of training cycles and the required processing time can be reduced to approximately one-third of those in the conventional method.

[0061] [Accuracy and Efficiency of Learning] Figure 4 is an explanatory diagram regarding the accuracy and efficiency of image learning. Figure 4 shows the results of evaluating the defect detection accuracy using various indicators when learning defects (micropipes, threading screw dislocations; TSDs, polytype inclusions) on SiC epitaxial wafers, while changing the method of assigning R, G, and B layers to each type of image with different inspection methods. Optical microscopy, photoluminescence, and X-ray topography were selected as the three types of inspection methods / imaging means.

[0062] Line 400 represents throughput. The bar graph shows four evaluation values ​​(evaluation metrics) in sets, from left to right: Precision, Recall, F-score, and Accuracy. The evaluation metrics used here are Precision, Recall, F-Score, and Accuracy, which are commonly used in the evaluation of object detection CNNs. For these evaluation metrics, a value closer to 1 is better.

[0063] The "individual" training data in the graph represents the conventional method, for example, when training is performed individually on three different test methods, and the results of training by independently annotating three different test images are shown.

[0064] In contrast to "Individual," the six sets of bar graphs on the horizontal axis, from "BGR" to "GBR," represent the training data for the "simultaneous learning" method of this embodiment, showing six combinations (patterns) of assigning three types of images from three different inspection methods to the three layers R, G, and B. For example, the "RGB" pattern represents assigning the optical microscope image to the R layer, the photoluminescence image to the G layer, and the X-ray topography image to the B layer (Figure 3). In Figure 4, the six patterns are arranged in descending order of evaluation value. The "BGR" pattern has the lowest evaluation value. In particular, the three patterns "BRG," "RGB," and "GBR" have the highest evaluation values ​​and throughput.

[0065] Figure 4 shows that all six assignment methods for "simultaneous learning" yield higher evaluation values ​​and throughput than the conventional "individual" method. In this embodiment, since three types of images are combined into one, the effort and processing time involved in learning are reduced to about one-third. Compared to the throughput value of approximately 3 in the case of "individual" learning, this embodiment allows for a maximum throughput value of approximately 7 depending on the pattern, achieving more than double the efficiency. The inference phase can also achieve the same throughput as the learning phase.

[0066] Since the R, G, and B colors are not symmetrical, it was found that the efficiency and accuracy of the learning results differ depending on the assignment method (pattern) used to assign the three types of images to which layer, as shown in Figure 4. In this embodiment, based on the learning experiments and studies shown in Figure 4, the design was to select and use the assignment method (pattern) that yielded the highest evaluation value (accuracy) and throughput (efficiency) from among the six patterns. In the case of Figure 4, when the target sample is a SiC epitaxial wafer and the type of defect to be inspected is a micropipe (described later), the patterns with the highest accuracy are the three patterns "BRG", "RGB", and "GBR". In this embodiment, for example, the "RGB" pattern (pattern 1 mentioned above) was selected and used (Figure 3). However, other patterns may also be used, and they may be more effective than conventional methods.

[0067] Furthermore, when deciding which patterns to use in the implementation, it is also possible to set thresholds for evaluation value / throughput values ​​and select from patterns that are greater than or equal to these thresholds. For example, if the evaluation value threshold 401 is set to 0.95 as shown in the figure, then one can select from the four patterns "GRB", "BRG", "RGB", and "GBR", which are greater than the threshold 401.

[0068] [Processing Flow of the Learning Phase] Figure 5 shows the processing flow of the learning phase. In step S1, the computer system collects and acquires images of the defects to be detected, which are images of the defects to be detected, as training images. For each type of defect, the collected images are set into three types, corresponding to, for example, three different inspection methods and imaging means used, as shown in the example in Figure 2.

[0069] In the case of SiC epitaxial wafers, the defects to be detected are various types of defects that pose a risk of causing some kind of malfunction in the operation of power devices, such as those described in Reference 1.

[0070] Reference 1: JEITA Standard EDR-4712 / 100, "Non-destructive testing method for crystal defects in SiC wafers (Part 1: Classification of crystal defects)"

[0071] For these target defects, this system generates and acquires defect images using multiple inspection methods and imaging means with different physical factors for image grayscale generation (in other words, grayscale formation principles and imaging principles). Examples of these inspection methods for SiC epitaxial wafers include optical microscopy, photoluminescence, and X-ray topography.

[0072] The density of images captured by optical microscopy is the distribution of light intensity in the visible light region scattered from the irradiated light on the wafer surface. The density of images captured by photoluminescence is the distribution of photoluminescent light intensity generated when ultraviolet light is irradiated. The density of images captured by X-ray topography is the distribution of X-ray intensity that increases or decreases due to deviations from Bragg reflection conditions caused by the presence of defects when X-rays with approximately a single wavelength are irradiated in line with the crystal orientation of the wafer.

[0073] Therefore, depending on the grayscale formation principle of the image, the inspection image / defect image reflects the characteristics of various defects, and by combining these (i.e., image synthesis), as in this embodiment, more accurate detection and classification of defects becomes possible.

[0074] The image data obtained by each inspection method is two-dimensional intensity data. If the intensity of each pixel is P, then it can be expressed in functional form as P = P(px, py), where px and py are the vertical and horizontal coordinates of the pixel.

[0075] Step S2 is the synthesis of each type of defect image. As shown in Figure 2, in the case of the three types of inspection methods, each image data can be associated with the R, G, and B primary color data in a color image. In step S2, the computer system assigns the three types of detection target defect images to the R, G, and B layers for each set, as shown in the example in Figure 3, and synthesizes them into a single composite image.

[0076] During the assignment and synthesis process, the computer system adjusts each image as needed, such as by resizing it, to ensure that the number of pixels in the width and height (image size) of each defective image matches (see below).

[0077] Furthermore, the pixel intensity values ​​of each inspection image reflect the sensitivity to defects and the strength of the image signal intensity depending on the inspection method, and the relative magnitudes of the pixel values ​​of each inspection image may be adjusted as appropriate. For example, if the average value of the image values ​​from the first inspection method and the average value of the image values ​​from the second inspection method differ significantly, the weight of information from each inspection method will differ greatly in the composite image, which is undesirable. Therefore, it is effective to standardize the intensity data (in other words, unify the image specifications) so that the intensity values ​​are suitable for defect detection for various types of images.

[0078] The image data of the composite image is 3-channel 2D data (see Figure 13 below), and if the pixel values ​​of the R, G, and B layers are PR, PG, and PB, it can be expressed in functional form by the following equation.

[0079] PR=PR(px,py), PG=PG(px,py), PB=PB(px,py),

[0080] Alternatively, the three equations above can be combined and expressed as P3 = P3(i, px, py), where i is the index corresponding to R, G, and B.

[0081] In step S3, the computer system and the user performing the learning task using the computer system perform annotation, or in other words, labeling of the training data for supervised machine learning, on the composite image obtained in step S2. The user labels the composite image, for example, what type of defect it is. The annotation / labeling task involves, for example, the user determining where and what kind of defect is in the image, enclosing the determined defect with a rectangle, and inputting the name of the defect type (or defect type index) along with the position data of this rectangle (for example, the pixel coordinate value of the upper left corner of the rectangle and the pixel values ​​of the width and height of the rectangle) for each image.

[0082] For training the model (defect classifier), multiple (usually dozens or more) annotation data sets are prepared for each type of defect to be trained.

[0083] In step S4, the computer system automatically performs machine learning processing using the multiple composite image and annotation (labeled) data prepared up to step S3, and a machine learning model (e.g., a deep learning model).

[0084] Various types of machine learning algorithms can be applied as models (classifiers). A suitable example is an object detection algorithm using a Convolutional Neural Network (CNN). Object detection CNNs are CNNs developed to automatically detect what objects are captured in an image, and Faster-RCNN, SSD, and YOLO are well known examples. These object detection CNNs are open source and can be used on various frameworks (PyTorch, TensorFlow, Keras, etc.). In these object detection CNNs, the input image is a color image. If there are three types of inspection images, the object detection CNN can be used immediately by assigning the images prepared in step S2 to three channels (R, G, B layers). The annotation data format (COCO format, PascalVOC format, etc.) is prepared according to the input conditions of the object detection CNN to be used.

[0085] The learning process is no different from learning a model for general images. The computer system learns the model by adjusting the learning parameter values ​​until the desired accuracy is achieved.

[0086] In step S5, the computer system and the user evaluate the machine learning model obtained as a result of the learning in step S4, checking whether the accuracy of inference (defect detection) is sufficient, and obtain a model with sufficient accuracy that satisfies the user. If it is not sufficient, the learning process is repeated. If necessary, the process returns to step S1, and the accuracy is improved by adding or replacing training images, etc. As a means of improving accuracy, selecting the assignment method to the R, G, and B layers, as shown in Figure 4, is effective.

[0087] In step S6, the computer system determines a model with sufficient accuracy to be used in the inference phase (inspection phase), i.e., as a defect classifier, and sets up the data of that model in the computer system / inspection device to be used in the inference phase (inspection phase).

[0088] [Inference (Inspection) Phase Flow] Figure 6 shows the processing flow of the inference phase (in other words, the inspection phase). In step S21, the computer system uses three types of inspection methods and imaging means to capture and acquire three types of inspection target images of the semiconductor wafer, which is the sample. In step S22, the computer system assigns the three types of inspection target images to the three channels of the R, G, and B layers to obtain a single composite image. In step S23, the computer system inputs the composite image into a model (defect classifier) ​​and obtains information on the defect classification result (inspection result) as the output of the inference and judgment result by the model. In step S24, the computer system records the inspection result and outputs it to the user (for example, on the screen).

[0089] This system provides support information for learning and inference tasks through a user interface, such as a GUI. The screen may display multiple types of images, such as those in Figures 2 and 14, composite images, such as those in Figure 3, graphs, such as those in Figure 4, management information, such as those in Figure 7, defect candidate location information, image adjustment information, model information, such as those in Figures 12 and 13, and images and information of defect classification results, such as those in Figure 14.

[0090] [Assignment Management] Figure 7 shows an example of the configuration of data and information for which the computer system manages and controls the assignment method (pattern, etc.) for assigning multiple inspection methods / imaging means and multiple types of images to multiple channels in this embodiment. In this example, the computer system creates, manages, and stores data and information (e.g., tables) as shown in the figure in memory resources, and makes it available for use by referencing it as needed.

[0091] Table 701 in (A) contains management information about inspection methods and imaging means. Table 701 contains information about multiple types of inspection methods and imaging means used. In this example, Table 701 has the following column items: management number (#), "inspection method / imaging means", "device ID", "imaging conditions (parameter values)", etc. The "inspection method / imaging means" includes the three types mentioned above. The "device ID" is the identification information / address of the device (imaging device / inspection device) that has the implementation corresponding to the "inspection method / imaging means". The "imaging conditions (parameter values)" indicates the setting information regarding the imaging conditions / parameter values ​​used when capturing images (inspection images) with that device.

[0092] It is also possible to set and use multiple imaging conditions for the same device (see the modified examples below). For example, it is possible to designate an image captured with imaging condition #1 in device A as type 1 image, and an image captured with a different imaging condition #2 in device A as type 2 image.

[0093] Table 702 in (B) contains management information regarding the channels to be used and how they are assigned. Table 702 includes the channels (ch) to be used, the assignment of the corresponding RGB layers, attribute information such as the image size of each ch (corresponding layer), etc. This example shows the settings when applying the aforementioned "RGB" pattern ("Pattern 1").

[0094] Table 703 in (C) contains management information for all allocation methods. Table 703 contains information that covers allocation patterns based on combinations of multiple types of inspection methods / imaging means used and the number of channels used. For example, Table 703 contains information such as pattern ID, correspondence with RGB layer (ch), evaluation value of the pattern, and whether or not it is applied (1 / 0).

[0095] [First System Configuration Example] Figure 8 shows a first system configuration example as an example of the system configuration of this embodiment. In the system shown in Figure 8, the first system configuration example includes three types of imaging devices (in other words, inspection devices) and a server device independent of them, and the server device is configured to perform learning and inference (inspection) processing as its main processing.

[0096] In Figure 8, the system is configured such that a server device 800 is connected to a first imaging device, an optical microscope 801, a second imaging device, a photoluminescence device 802, and a third imaging device, an X-ray topography device 803, via a communication network (e.g., LAN) 809. The user operates the server device 800 and each imaging device to perform learning and inference tasks. The server device 800 acts as both a learning device and an inference device. Note that the tasks of imaging and learning may be shared among multiple people.

[0097] The first imaging device, an optical microscope 801, captures an image (the first image) of the sample 810. The second imaging device, a photoluminescence apparatus 802, captures an image (the second image) of the sample 810. The third imaging device, an X-ray topography apparatus 803, captures an image (the third image) of the sample 810. The sample 810 is a semiconductor wafer, for example, a SiC epitaxial wafer. The technology of each imaging device is publicly known and is not limited in detail. Furthermore, the order in which the sample 810 is imaged by the multiple imaging devices is not limited.

[0098] The server device 800 receives and acquires image data and other data of images captured from each imaging device (801 to 803) via the communication network 809 and stores it in memory.

[0099] The server device 800 includes, as functions, at least a learning function 800A and an inference function (in other words, a testing function) 800B. The learning function 800A is a function that executes the processing of the learning phase described above. The inference function 800B is a function that executes the processing of the inference phase (testing phase) described above.

[0100] During the learning phase, the server device 800 uses the images acquired from each imaging device as training images and performs the aforementioned learning process. This results in an inference model (defect classifier).

[0101] During the inference phase, the server device 800 acquires images to be inspected from each imaging device, inputs them into the model, and obtains defect inspection results (defect classification results) as the output of the inference results from the model.

[0102] The server device 800 also provides a user interface for learning and inference. For example, the server device 800 provides a screen with a graphical user interface (GUI) (for example, a web page) to the user's terminal, such as a PC. The user can input instructions and settings on the screen and check various information displayed on the screen (such as inspection images and inspection results). On the screen, for example, images from each inspection method in Figure 3 (201, 202, 203) and the composite image 300 can also be displayed and checked.

[0103] [Computer System] Figure 9 shows an example of the hardware and software configuration of the server device 800 in Figure 8 as a computer system. The computer 1000, which corresponds to the server device 800, has a processor 1001, memory 1002, communication interface device 1003, input / output interface device 1004, etc., and these are interconnected via an architecture such as a bus.

[0104] The processor 1001 executes processing according to the control program 1002A for the memory 1002. This enables the implementation of a predetermined function 1001A (such as the learning function 800A in Figure 8) as an execution module. The processor 1001 may also have an NPU (Neural Processing Unit) suitable for machine learning / AI calculations.

[0105] Memory 1002 is composed of a non-volatile storage device, etc. Memory 1002 stores, for example, a control program 1002A, setting information 1002B, management information 1002C, learning data 1002D, inference data 1002E, etc. Control program 1002A is a learning program and inference program, etc., for realizing the learning function 800A and the inference function 800B, etc. Setting information 1002B is system setting information and user setting information related to control program 1002A. Management information 1002C is, for example, data like that shown in Figure 7. Learning data 1002D is various data used in the learning phase, such as image data, machine learning models, and training data. Inference data 1002E is various data used in the inference phase, such as image data, machine learning models, and test result information.

[0106] The communication interface device 1003 is a device that implements a communication interface with the communication network 809, etc. Input / output interface device 1004 has an input device 1005 and an output device 1006 externally connected to it. The computer 1000 may also have built-in input / output devices. Examples of input devices 1005 are a mouse, keyboard, microphone, etc. Examples of output devices 1006 are a display, printer, speaker, etc.

[0107] [Processing Example in the First System Configuration Example] Figure 10 shows a schematic example of processing that is common during learning and inference in the first system configuration example in Figure 8. First, in step S31, the system acquires the sample to be inspected (semiconductor wafer), and the server device 800 sets the imaging target area, imaging conditions, etc. for each imaging device (801 to 803) corresponding to the first to third inspection methods.

[0108] In step S32, the server device 800 transmits instructions to each imaging device (801-803) to perform inspection / imaging of the sample. For example, the first imaging device 801 of the first inspection method images the sample, thereby obtaining a first image (first inspection image data) and first image position information data as information related to the imaging. The image position information data is data and information that represents the position and region of the image captured in the coordinate system of the sample (e.g., semiconductor wafer). Instructions for inspection / imaging to each imaging device (801-803) may also be input by the user to each imaging device.

[0109] As a result of step S32, the first to third inspection image data and the first to third image position information data, which include information representing the imaging position and imaging range within the wafer, are output.

[0110] In step S33, the server device 800 acquires inspection image data and image position information data from each imaging device (801-803) and stores them in memory. This data may be accumulated sequentially during each inspection operation, or it may be saved all at once after each inspection operation is completed.

[0111] In step S34, the server device 800 (especially the processor) performs adjustments such as cropping, aligning, resizing, and scaling of the grayscale based on each acquired image data (for example, three types of images) for image synthesis. In other words, the server device 800 performs processing to unify the format and specifications of the three types of images for synthesis. Alignment and the like can be achieved using the image position information data obtained in step S33.

[0112] Each imaging device (801-803) of this system, or any device that manages and controls imaging by the imaging devices (e.g., server device 800), has image position information data (related information at the time of imaging), such as which position coordinates or range of the surface of the semiconductor wafer sample was targeted for imaging. The computer system, for example, the server device 800, can perform the image adjustment in step S34 by using such image position information data along with the image data.

[0113] During adjustment, the server device 800 uses image position data based on the inspection image to extract an image of a corresponding area (target region) on the wafer, or generates an image of a corresponding area by cutting and pasting multiple inspection images (Figure 11, described later). These pre-set areas are, for example, areas obtained by dividing the wafer into a grid pattern with a set size and number. These areas may be the same as or different from the imaging area of ​​the image output of any imaging device / inspection device.

[0114] In step S35, the server device 800 assigns the three adjusted images to multiple channels (e.g., R, G, and B layers) using a set assignment method, and combines the assigned image data into a single image for each set. The server device 800 stores the resulting combined image in memory.

[0115] The three types of inspection images reconstructed for each area through the above adjustments are combined into a single pseudo-RGB color image for each area by image synthesis. The method of assigning the R, G, and B layers of the inspection images obtained by each inspection method is the same as the configuration used during model training. The intensity values ​​of each inspection image may be normalized to ensure optimal accuracy.

[0116] In step S36, the server device 800 inputs the synthesized image into a machine learning model and performs training or inference.

[0117] In step S37, the server device 800 obtains the output of the machine learning model. During training, the model's parameter values ​​are adjusted using the input and output. During inference, the images of each area are input to the defect classifier, which detects and classifies defects captured in the images, and outputs defect classification information. The defect classification information includes defect type, location / area, size, and other characteristic data, and this data is compiled into a defect list. The output defect classification information is stored in the server device 800 and displayed on the screen of the display device viewed by the user.

[0118] [Adjustment such as alignment] Figure 11 is an explanatory diagram showing an example of the image adjustment process in step S34. An inspection image of the entire surface of the wafer 810 is captured using three types of imaging devices (801 to 803). An example of the imaging area with the optical microscope (801) is shown in grid 1101. Each square in grid 1101 corresponds to one image. For explanatory purposes, the number of squares in grid 1101 is shown to be small. Also, since the entire circular wafer is imaged, at the wafer edge, the wafer may only be a part of the image. Similarly, an example of the imaging area with the photoluminescence device (802) is shown in grid 1102, and an example of the imaging area with the X-ray topography device (803) is shown in grid 1103.

[0119] To explain the image adjustment process in step S34, we focus on the images in which the defect 1104 present on the wafer 810 is captured. Image 1111, in which the defect 1104 is captured by the optical microscope (801), is an image of the region from coordinates (X1, Y1) to coordinates (X2, Y2), and has a pixel size of SX1 × SX1. Image 1112, in which the defect 1104 is captured by the photoluminescence apparatus (802), is an image of the region from coordinates (X3, Y3) to coordinates (X4, Y4), and has a pixel size of SX2 × SX2. Image 1113, in which the defect 1104 is captured by the X-ray topography apparatus (803), is an image of the region from coordinates (X5, Y5) to coordinates (X6, Y6), and has a pixel size of SX3 × SX3. Thus, because the inspection devices are different, simply overlaying the images from each device in which the defect 1104 of interest was captured will not result in the defect location matching. Furthermore, while the image area is square in this example, it is not limited to this.

[0120] The computer system (server device 800) performs alignment and cropping adjustments based on the inspection image acquisition position and inspection image size information from the three types of images acquired from various imaging devices. Here, as an example, we will explain the adjustment to match the range of the photoluminescence inspection image and the X-ray topography inspection image to the inspection image range of the optical microscope. In adjusting the photoluminescence inspection image, the same region 1114 is cropped, including the image adjacent to image 1112, so that it has the same imaging range as the region from coordinates (X1, Y1) to coordinates (X2, Y2), which is the imaging range of the optical microscope inspection image. Similarly, in adjusting the X-ray topography inspection image, the same region 1115 is cropped, including the image adjacent to image 1113, if necessary, so that it has the same imaging range as the region from coordinates (X1, Y1) to coordinates (X2, Y2), which is the imaging range of the optical microscope inspection image. Note that this adjustment can be performed on all inspection images using the image acquisition position and size information, regardless of the presence of defects. Furthermore, after adjusting the alignment of the image, the pixel size, image signal intensity scaling, and image value bit depth matching are performed to synthesize three types of inspection images. The details of the above adjustments can be handled using publicly known techniques.

[0121] The computer system may also acquire adjusted images from various imaging devices by setting and instructing imaging conditions, etc., to achieve alignment and other adjustments.

[0122] [Machine Learning Model] Figure 12 shows an example of a machine learning model (model 105 in Figure 1) using a deep learning neural network. In Figure 12, the input data 1201 to the machine learning model 1200 is a synthesized image 1201, and the output data 1202 is defect classification information 1202. During training, the computer system and the user repeatedly perform calculations on the machine learning model 1200 using the input data 1201 and output data 1202 to obtain the desired relationship between the input (synthesized image 1201) and the output (defect classification information 1202).

[0123] The lower part shows the case where the machine learning model 1200 is a deep learning neural network. Each pixel value in the input image 1211 (synthesized image 1201) is assigned to a node in the input layer. The nodes in the output layer are associated with truth values ​​or probabilities of whether or not there is a defect in the defect classification information 1202, which is the inference result 1212.

[0124] Figure 13 is an explanatory diagram showing an example of the configuration of the input stage for learning when using a convolutional neural network (CNN) as a deep learning neural network as an example of machine learning model configuration. In this example, Faster-RCNN is used. Figure 13 shows the case where a pseudo-RGB color image with 3 channels (R, G, B layers) is used. The input data (Pcolor) 1301 is image data with 3 channels R, G, and B, and each channel has an image size of 224 × 224 pixels (vertical pixels × horizontal pixels). The input data 1301 is subjected to a convolution operation 1302 with a filter 1303. The filter 1303 has multiple (64) filters, F1, ..., F64, each having an image size of 7 × 7 for each channel. As a result of the operation, a feature map 1304 is obtained. Feature map 1304 has multiple (64) feature maps, C1, ..., C64, each with an image size of 112 × 112.

[0125] The input data 1301 is a 224x224 image, and by adding three channels (R, G, B), it becomes 3D data. One filter is 3D data with 7x7 and 3 channels. After performing a convolution operation on each channel in 2-pixel steps and then summing up the results of the three channels, a single 112x112 feature map is obtained.

[0126] When using Faster-RCNN on PyTorch as an object detection CNN, for example, the input color image (composite image) data is subjected to a pre-configured normalization unless otherwise specified. In this normalization, the R, G, and B layers (the three channels [R, G, B] shown in the diagram) are converted into numerical data between 0 and 1, with mean values ​​of [0.485, 0.456, 0.406] and standard deviations of [0.229, 0.224, 0.225].

[0127] [Specific Examples of Defects and Images] Figure 14 shows specific examples of images showing defects and their synthesis. Micropipes, which are crystal defects, are known as a type of defect that can occur in SiC epitaxial wafers. Figure 14 shows an example of synthesizing three types of defect images taken of micropipe defects in a SiC epitaxial wafer into a pseudo-RGB color image. In this example, the first defect image 1401 obtained by optical microscopy is assigned to the B layer, the second defect image 1402 obtained by photoluminescence is assigned to the R layer, and the third defect image 1403 obtained by X-ray topography is assigned to the G layer (in the case of the "BRG" pattern mentioned above).

[0128] In the first defect image 1401, a micropipe is visible as defect A near the center of the region. In the second defect image 1402, a micropipe is visible as defect A near the center of the region, but it is difficult to discern from this image alone. In the third defect image 1403, a micropipe is visible as defect A in a black area that extends outward from near the center of the region.

[0129] In the composite image (composite defect image) 1404, the images of defect A = micropipe from the three original images are combined. The composite image 1404 reflects and retains the characteristic information of defect A = micropipe that was present in the three original images. In the learning phase, when the user performs annotation (labeling), they look at the composite image 1404 and determine that the portion of defect A within the region is a defect type = micropipe. Then, as shown in the defect classification result image 1405 (training image), the user adds, for example, a frame 1406 (for example, a red square) to specify the defect region / location and a label 1407 to specify the defect type ("micropipe").

[0130] In the inference phase, the computer system inputs the synthesized image 1404 into the model and outputs an image of the defect classification result 1405 (inspection result image). In this image, the defect A = micropipe located near the center of the region is recognized and detected, and a frame 1406 representing the defect region and location, and a label 1407 representing the defect type ("micropipe") are added and displayed.

[0131] In conventional individual inspection methods, for example, when viewing the first defect image 1401 using an optical microscope, the micropipe area is small, but when viewing the third defect image 1403 using X-ray topography, the micropipe area is larger. In reality, strain has occurred in the semiconductor not only in the area of ​​the micropipe in the first defect image 1401, but also in the black, extended area in the third defect image 1403.

[0132] In contrast, according to this embodiment, such distorted areas can be detected as defect regions from the composite image 1404. In the three types of inspection images, which use different imaging principles, the same defect will appear in different ways depending on the type of image. As shown in the figure, the characteristic information of the three types of images is reflected and retained in the composite image, so defects can be classified and detected from the composite image with high accuracy.

[0133] Furthermore, as mentioned above (Figure 4), by selecting and applying a more suitable assignment method (in this case, the "BRG" pattern), it is possible to create a higher-quality composite image that reflects the characteristic information of various examination images. This improves the accuracy of inference and examination.

[0134] As mentioned above, the accuracy of image recognition / inspection may vary depending on the assignment method. Figure 15 shows an example of a composite defective image when the assignment method is different from that in Figure 14. (A) is a composite image 1501 in the case of the same "BRG" pattern as in Figure 14 for comparison. (B) is a composite image 1502 in the case of the "GRB" pattern.

[0135] In composite image 1502, the micropipe defect is less visible compared to composite image 1501. For example, in composite image 1502, it is difficult to determine how far the strain extends from the center of the region outwards. When each composite image is input into its respective model for defect inference, the accuracy will differ.

[0136] Micropipe defects are one of the typical types of defects that can occur in SiC epitaxial wafers, but they can also occur in other types of samples. The method of this embodiment can be similarly applied and is effective for other samples and other types of defects.

[0137] In the above embodiment, assuming the use of a deep learning CNN, a preferred example of using three channels is shown, where a pseudo-color image is constructed using R, G, and B layers. The number of channels is not limited to this.

[0138] [Modified Version: 2-Channel] As a modified version, it is also possible to configure a system in which two types of images obtained from two different inspection methods and imaging means are assigned to two channels, combined, and then used for learning and inference.

[0139] Figure 16 shows a modified example in which a first image 1601 obtained by optical microscopy and a second image 1602 obtained by X-ray topography are assigned to two channels (first channel and second channel) and combined to obtain a combined image 1603. When using two types of inspection methods / imaging means, for example, two types selected from optical microscopy, photoluminescence, and X-ray topography may be used. For example, a combination of optical microscopy and X-ray topography may be used.

[0140] As an example of the configuration of the image input stage to the model (CNN: Faster-RCNN), input data (Pmulti) 1604 corresponding to the synthesized image 1603 is subjected to a filter 1606 (F1, ..., F64) and a convolution operation 1605, and as a result of the operation, a feature map 1607 (C1, ..., C64) is obtained.

[0141] [Modification: 4 or more channels] As a modification, it is also possible to configure a system in which four or more images obtained from four or more inspection methods / imaging means are assigned to four or more channels, synthesized, and then used for learning and inference. In this case, the image data for learning and inference is multi-channeled beyond three channels. Note that when using four or more channels, a 3-channel R, G, and B layer model cannot be used as is. When using four or more channels, the image data of those multiple channels should be assigned to the corresponding multiple channels on the model (CNN).

[0142] Figure 17 shows a modified example of the synthesis of four types of images in the case of 4 channels as multi-channel data, and an example of the configuration of the image input stage to the CNN (Faster-RCNN). It has the first image 1701 to the fourth image 1704. The first image 1701 to the third image 1703 are images obtained by the first to third inspection methods as described above. The fourth image 1704 may be an image obtained by the fourth inspection method, or an image obtained by changing the imaging conditions of any of the first to third inspection methods described above. In this example, the third image 1703 is an image taken with an X-ray topography device under the first imaging conditions (first parameter value), and the fourth image 1704 is an image taken with an X-ray topography device under the second imaging conditions (second parameter value).

[0143] In Faster-RCNN, regardless of the pixel size of the input image data, each channel is first reshaped to a predetermined pixel size, for example, 224 x 224 pixels. The reshaped input data (Pmulti) 1706 corresponding to the composite image 1705 is a 4-channel image data with an image size of 224 x 224. The input data 1706 is subjected to a convolution operation 1707 with a filter 1708 (F1, ..., F64), and as a result of the operation, a feature map 1709 (C1, ..., C64) is obtained.

[0144] As described above, when the input data consists of four or more types of image data, the input data becomes Pmulti(x,224,224), which is obtained by changing the three channels mentioned earlier to four or more x channels. The filter is also adjusted to match the number of channels in the input data. The convolution operation is the same as in the three-channel case, with only the number of channels increased. The generation of 64 feature maps as output is also the same. The calculations within the CNN after the input stage are also the same as in the three-channel case. In this way, when implementing multi-channel processing, it is only necessary to change the configuration of the convolution operation unit in the input stage.

[0145] Figure 18 shows a modified example, similarly in the case of 5 channels. It has the first image 1801 to the fifth image 1805. The fourth image 1804 and the fifth image 1805 may be images obtained by the fourth or fifth inspection method, or images obtained by the first to third inspection methods described above with different imaging conditions. In this example, the third image 1803 is an image taken with an X-ray topography apparatus under the first imaging conditions (first parameter value), the fourth image 1804 is an image taken with a photoluminescence apparatus under the second imaging conditions (second parameter value), and the fifth image 1805 is an image taken with an X-ray topography apparatus under the second imaging conditions (second parameter value). The input data (Pmulti) 1807 to the CNN (Faster-RCNN) is 5-channel image data with an image size of 224 × 224. The input data 1807 is subjected to a convolution operation 1808 with the filter 1809 (F1, ..., F64), and as a result of the operation, a feature map 1810 (C1, ..., C64) is obtained.

[0146] When using four or more types of inspection methods and imaging means, for example, in addition to optical microscopes, photoluminescence, and X-ray topography, a polarizing microscope may be used as a fourth type of imaging device. A microwave photoconductivity attenuation (μ-PCD) measuring device may be used as a fifth type of imaging device, and a mirror electron microscope may be used as a sixth type of imaging device.

[0147] [Second System Configuration Example] In the system of the above embodiment (Figure 8), a separate server device is provided in addition to the three types of imaging / inspection devices, and the main processing performed by this server device is learning and inference. However, the system is not limited to this, and various system configurations are possible. Modifications are shown below. The server device may be a computer provided in any of the imaging / inspection devices.

[0148] Figure 19 shows a second system configuration example as a modification. The second system configuration example shows a case where an integrated device 1900 (optical-photoluminescence integrated device) is used, in which an optical microscope and a photoluminescence device are integrated into one. This system has an integrated device 1900 and an X-ray topography device 803 as imaging devices, which are connected by communication as appropriate or data is transferred using a recording medium. Furthermore, the second system configuration example shows a case where learning and inference are performed in the integrated device 1900, rather than a server device. In this case, the integrated device 1900 corresponds to both a learning device and an inference device. In this system, the integrated device 1900 (corresponding computer) acquires images from the X-ray topography device 803 and performs the main processing on the three types of acquired images.

[0149] In Figure 19, the integrated device 1900 is a device that includes an optical microscope 801 and a photoluminescence device 802 as functional units. The controller 1901 of the integrated device 1900 controls the optical microscope 801 and the photoluminescence device 802. In addition to the control function, the controller 1901 is equipped with a learning function 1900A and an inference function 1900B.

[0150] When inspecting the sample 810, the integrated device 1900 uses the optical microscope 801 and the photoluminescence device 802 to capture a first image and a second image, and also acquires a third image captured by the X-ray topography device 803 via communication or recording medium. The controller 1901 uses the three acquired images to perform learning processing by the learning function 1900A, or inference (inspection) processing by the inference function 1900B.

[0151] In another variation, the X-ray topography apparatus 803 (corresponding computer system) may acquire first and second images from the integration apparatus 1900 and perform learning and inference as the main processing for the three acquired images.

[0152] Beyond the examples above, various integrated device configurations are possible for various imaging / inspection devices. Furthermore, beyond the examples above, configurations where the learning and inference functions are handled by other devices are also possible. Additionally, a configuration where the learning device and the inference (inspection) device are separate may be adopted.

[0153] Figure 20 shows a third system configuration example as a modification. In this system, a server device 2001 of the first operator and a server device 2002 of the second operator are connected to various imaging devices (801-803) on the communication network 809. The server device 2001 of the first operator performs learning processing as a learning device equipped with a learning function 2000A. Another device, the server device 2002 of the second operator performs inference (inspection) processing as an inference device (inspection device) equipped with an inference function 2000B. The server device 2001 acquires various images (training images) from various imaging devices (801-803) and trains a model. The server device 2002 sets up the trained model, acquires various images (inspection target images) from the imaging devices (801-803), inputs them into the model, and obtains inspection results.

[0154] The computers performing the various image capture, acquisition, storage, image adjustment, image synthesis, model training, and inference processes described above may be the same computer, or they may be multiple different computers connected by communication or capable of data transfer using recording media.

[0155] [Modification: Input Color Image] The following are also possible modifications. In the above embodiment, as shown in Figure 2, the image before synthesis, which is captured by the inspection method and imaging means, is a monochrome grayscale image, but it is not limited to this and may be a color image. In the case of a color image, the following method can be applied. For example, suppose the first image captured with an optical microscope is a color image.

[0156] Method 1: Convert a color image to a monochrome grayscale image and assign it to a channel for compositing.

[0157] Method 2: A method in which color images, for example, images with R, G, and B values, are assigned to multiple channels (e.g., 3 channels) for synthesis, and other grayscale images are assigned to other channels. During the assignment, conversion and adjustment to the aforementioned unified format and specifications are performed as appropriate. For example, color images from an optical microscope are assigned to channels 1 to 3, and other grayscale images such as a photoluminescent image are assigned to channel 4, and an X-ray topography image is assigned to channel 5. A total of 5 channels are used.

[0158] Method 3: A method in which one or two color values ​​are selected from a color image, for example, an image having R, G, and B values, to be used for synthesis, and the selected color values ​​are assigned to the corresponding number of channels. For example, only the R value is selected from a color image obtained by an optical microscope and assigned to the first channel. The image obtained by photoluminescence is assigned to the second channel, and the image obtained by X-ray topography is assigned to the third channel. A total of three channels are used.

[0159] [Modifications: Inspection Methods and Imaging Means] The above embodiment shows a configuration using three types of inspection methods (optical microscope, photoluminescence, and X-ray topography), but additional inspection methods may be added to improve accuracy. For example, we have assumed one type of image using the photoluminescence method, but since the contrast differs depending on the type of defect when the wavelength band of the light used for imaging is different, accuracy can be improved by using photoluminescence light of various wavelength bands (corresponding imaging conditions). Also, in the X-ray topography method, the crystal planes that are diffracted differ by changing the energy and angle of the incident X-rays, and the contrast differs depending on the type of defect.

[0160] Alternatively, completely different inspection methods other than the three types of inspection methods described above may be used. For example, in a method that visualizes the reflection intensity of an incident electron beam dependent on the wafer surface potential, such as a mirror electron microscope, a contrast corresponding to the electrical properties of each type of defect can be obtained. Other methods such as polarized light microscopy and μ-PCD may also be used. In polarized light microscopy, polarized light is incident on the wafer, and the contrast of defects is obtained by detecting the birefringence characteristics of the transmitted light. In μ-PCD, a laser is locally irradiated to excite carriers, and the local carrier lifetime is measured from the attenuation rate of the reflection intensity of the simultaneously irradiated microwaves. The lifetime distribution is then imaged by scanning the laser on the wafer. This provides a contrast between normal areas and defects with different lifetimes.

[0161] [Variation: Sample] The applicable samples are not limited to SiC epitaxial wafers, but can also be other semiconductors, such as SiC bulk wafers, GaN, diamond, etc. Furthermore, it can be applied to compounds other than SiC, not just semiconductors.

[0162] [Modification: Management according to sample and defect type] The above-described embodiment (first embodiment) designs multiple types of inspection methods and imaging devices with different imaging principles, as well as multiple channel assignment methods, corresponding to a certain target sample (SiC epitaxial wafer) and defect type (micropipe, etc.). However, the inspection method, imaging device, and assignment method to be applied may be changed depending on the target sample and defect type. For example, the correspondence may be managed by using the first imaging device set for the first sample and the second imaging device set for the second sample. Alternatively, the correspondence may be managed by using the first assignment method for the first defect type of the first sample and the second assignment method for the second defect type of the first sample. The management can be similarly applied to data as shown in Figure 7. The imaging principle and assignment method that provide high accuracy may change depending on the sample and defect type, so it is advisable to select a suitable imaging principle and assignment method.

[0163] [Modification: Imaging Conditions] In the above embodiment, the three types of imaging devices were shown to capture images under predetermined imaging conditions (corresponding parameter values). However, as a modification, multiple images may be captured and used using the same imaging device under different imaging conditions. The obtained multiple images can then be similarly assigned to multiple channels and combined.

[0164] For example, in an X-ray topography system, a first image is acquired under first imaging conditions, and simultaneously or sequentially, a second image is acquired under second imaging conditions. These two images are assigned to two channels. This configuration corresponds to the third image 1703 and the fourth image 1704 in Figure 17 mentioned above.

[0165] [Note] This embodiment includes the following configuration.

[0166] (1) An image learning method for learning images and constructing a model, comprising the steps of: acquiring multiple types of images taken of a sample using multiple different imaging principles, as steps performed by a computer system; generating a composite image, which is an image obtained by aligning the acquired multiple types of images, unifying their specifications, and assigning them to multiple channels using a predetermined assignment method; and learning the model using the composite image as input and defect classification information relating to defects in the sample as output.

[0167] (2) An image learning method according to (1), wherein the plurality of types of images are captured by a plurality of different types of imaging devices.

[0168] (3) An image learning method according to (1) or (2), wherein the assignment method is selected from all combinations of assignment patterns to have the highest accuracy with respect to the output of the model or the highest accuracy with respect to a threshold.

[0169] (4) An image learning method according to (1) or (2), comprising the step of comprehensively learning all combinations of assignment patterns in the assignment method, evaluating the accuracy of the model's output for each pattern, and selecting the one pattern with the highest accuracy or the one pattern with accuracy above a threshold as a result of the evaluation, and deciding to use it in the inference phase.

[0170] (5) An image learning method according to any one of (1) to (4), wherein three types of images captured as multiple types of images are assigned to three channels corresponding to the R (red), G (green), and B (blue) layers to generate the composite image.

[0171] (6) An image learning method according to any one of (1) to (4), wherein two types of images captured as multiple types of images are assigned to two channels to generate the composite image.

[0172] (7) An image learning method according to any of (1) to (4), wherein four or more images captured as multiple types of images are assigned to four or more channels to generate the composite image.

[0173] (8) An image learning method according to any one of (2) to (7), wherein the plurality of types of imaging devices include an optical microscope, a photoluminescent device, and an X-ray topography device.

[0174] (9) An image learning method according to any one of (2) to (8), wherein the sample is a SiC wafer, the plurality of imaging devices include three types: an optical microscope, a photoluminescence device, and an X-ray topography device, the three types of images captured by the three types of imaging devices are assigned to three channels corresponding to the R (red), G (green), and B (blue) layers to generate the composite image, and the assignment method is one of the following patterns: assigning {a first image taken by an optical microscope, a second image taken by photoluminescence, and a third image taken by X-ray topography} to {layer B, layer R, and G}, assigning {layer R, layer G, and B}, or assigning {layer G, layer B, and R}.

[0175] (10) An image learning device that performs each step of the image learning method described in any of (1) to (9).

[0176] (11) An image inspection method for classifying defects from an image based on a model that has learned images, comprising the steps of: acquiring multiple types of images taken of a sample using multiple different imaging principles, each acquired by a computer system; generating a composite image by aligning the acquired multiple types of images, unifying their specifications, and assigning them to multiple channels using a predetermined assignment method to create a single composite image; and inputting the composite image into the model and obtaining defect classification information regarding defects in the sample as the output of the inference result by the model.

[0177] (12) An image inspection method according to (11), wherein the plurality of types of images are captured by a plurality of different types of imaging devices.

[0178] (13) An image inspection method according to (11) or (12), wherein three types of images captured as multiple types of images are assigned to three channels corresponding to the R (red), G (green), and B (blue) layers to generate the composite image.

[0179] (14) An image inspection device that performs each step of the image inspection method described in any of (11) to (13).

[0180] (15) A program (learning program) for causing a computer to perform each step of the image learning method described in any of (1) to (9). Or, a recording medium storing said program.

[0181] (16) A program (inference program / inspection program) for causing a computer to perform each step of the image inspection method described in any of (11) to (13). Or, a recording medium storing said program.

[0182] (17) A semiconductor device manufacturing method comprising: an inspection step of performing an image inspection that classifies defects from images based on a model that has learned images; and a step of shipping semiconductor devices that have passed the inspection step and been determined to be good products, wherein the inspection step is a step of performing each step of the image inspection method described in any of (11) to (13) by a computer system.

[0183] (18) An image learning method according to any of (2) to (9), wherein the plurality of types of images by the plurality of types of imaging devices include a first image captured by a first imaging device under first imaging conditions and a second image captured by the first imaging device under second imaging conditions, and the plurality of types of images including the first image and the second image are assigned to the plurality of channels to generate the composite image.

[0184] (19) For example, an image learning device for learning images and constructing a model, wherein the device acquires multiple types of images taken of a sample using multiple different imaging principles, aligns the positions of the acquired multiple types of images, unifies their specifications, assigns them to multiple channels using a predetermined assignment method, generates a composite image which is a single composite image, takes the composite image as input and outputs defect classification information regarding defects in the sample, and learns the model.

[0185] (20) For example, the image learning device includes a processor and memory, the memory storing a program for executing each step of any of the image learning methods (1) to (10), and data related to learning, and the processor performs processing according to the program to acquire the plurality of types of images, generate the composite image, and learn the model.

[0186] (21) For example, an image inspection method for classifying defects from an image based on a model that has learned images, the method comprising the steps of: acquiring multiple types of images taken of a sample using multiple different imaging principles, each of which images are acquired from the sample; generating a composite image, which is an image obtained by aligning the acquired multiple types of images, unifying their specifications, and assigning them to multiple channels using a predetermined assignment method; and inputting the composite image into the model and obtaining defect classification information regarding defects in the sample as the output of the inference result by the model.

[0187] (22) For example, an image inspection device that classifies defects from an image based on a model that has learned images, wherein it acquires multiple types of images taken of a sample using multiple different imaging principles, aligns the positions of the acquired multiple types of images, unifies their specifications, assigns them to multiple channels using a predetermined assignment method, generates a composite image which is a single composite image, inputs the composite image into the model, and obtains defect classification information regarding defects in the sample as the output of the inference result by the model.

[0188] (23) For example, the image inspection apparatus includes a processor and memory, the memory storing a program for executing each step of any of the image inspection methods (11) to (13), and data relating to the inspection, and the processor performs processing according to the program to acquire the plurality of types of images, generate the composite image, and obtain the defect classification information from the model.

[0189] For example, in the system shown in Figure 8, the server device (the computer in Figure 9) stores learning programs and inference programs in its memory resources. The server device's processor executes learning processes according to the learning programs read into memory, or inference processes according to the inference programs. This realizes the learning and inference functions. The computer may also read these programs from an external storage medium.

[0190] (Model) A model / recording medium that causes a computer to perform the processing of classifying defects from an image based on a model that has learned images, wherein the computer performs the steps of: acquiring multiple types of images taken of a sample using multiple different imaging principles; generating a composite image which is an image obtained by aligning the acquired multiple types of images, unifying their specifications, and assigning them to multiple channels using a predetermined assignment method; and inputting the composite image into the model and obtaining defect classification information regarding defects in the sample as the output of the inference result by the model.

[0191] For example, in the system shown in Figure 8, the server device (the computer in Figure 9) stores the machine learning model data in memory resources. The server device's processor performs learning or inference processing based on the machine learning model data read into memory. This enables the learning and inference functions. The computer may also read the machine learning model data from an external storage medium.

[0192] Although embodiments of this disclosure have been described in detail above, the invention is not limited to the embodiments described above, and various modifications are possible without departing from the gist of the invention. Each embodiment allows for the addition, deletion, and replacement of components, except for essential components. Unless otherwise specified, each component may be singular or plural. Combinations of each embodiment and its variations are also possible.

[0193] 101...Inspection image (training data), 102...Assignment and synthesis, 103...Synthesized image, 104...Machine learning, 105...Model, 106...Inspection image (data to be inspected), 107...Assignment and synthesis, 108...Synthesized image, 109...Inference and judgment, 110...Inspection result recording and output.

Claims

1. An image learning method for learning images and constructing a model, comprising the steps of: acquiring multiple types of images taken of a sample using multiple different imaging principles, each taking such images; generating a composite image by aligning the acquired multiple types of images, unifying their specifications, and assigning them to multiple channels using a predetermined assignment method; and learning the model using the composite image as input and defect classification information relating to defects in the sample as output.

2. The image learning method according to claim 1, wherein the plurality of types of images are captured by a plurality of different types of imaging devices.

3. An image learning method according to claim 1, wherein the assignment method is characterized in that, from among all combinations of assignment patterns, one pattern is selected that has the highest accuracy with respect to the output of the model, or one pattern whose accuracy is higher than a threshold.

4. An image learning method according to claim 1, comprising the steps of comprehensively learning all combinations of assignment patterns in the assignment method, evaluating the accuracy of the model's output for each pattern, and selecting the pattern with the highest accuracy or the pattern with accuracy above a threshold as a result of the evaluation, and deciding to use it in the inference phase.

5. An image learning method according to claim 1, wherein three types of images captured as a plurality of types are assigned to three channels corresponding to the R (red), G (green), and B (blue) layers to generate the composite image.

6. An image learning method according to claim 1, wherein two types of images captured as a plurality of types of images are assigned to two channels to generate the composite image.

7. An image learning method according to claim 1, wherein four or more images captured as multiple types of images are assigned to four or more channels to generate the composite image.

8. The image learning method according to claim 2, wherein the plurality of types of imaging devices include an optical microscope, a photoluminescent device, and an X-ray topography device.

9. The image learning method according to claim 2, wherein the sample is a SiC wafer, the plurality of imaging devices include three types: an optical microscope, a photoluminescence device, and an X-ray topography device, the three types of images captured by the three types of imaging devices are assigned to three channels corresponding to the R (red), G (green), and B (blue) layers to generate the composite image, and the assignment method is one of the following patterns: assigning {a first image from an optical microscope, a second image from photoluminescence, and a third image from X-ray topography} to {layer B, layer R, and layer G}, assigning {layer R, layer G, and layer B}, or assigning {layer G, layer B, and layer R}.

10. An image learning device that performs each step of the image learning method according to any one of claims 1 to 9.

11. An image inspection method for classifying defects from an image based on a model that has learned images, comprising the steps of: acquiring multiple types of images of a sample, each captured by multiple different imaging principles; generating a composite image by aligning the acquired multiple types of images, unifying their specifications, and assigning them to multiple channels in a predetermined assignment method; and inputting the composite image into the model and obtaining defect classification information regarding defects in the sample as the output of the inference result by the model.

12. An image inspection method according to claim 11, wherein the plurality of types of images are captured by a plurality of different types of imaging devices.

13. An image inspection method according to claim 11, wherein three types of images captured as a plurality of types are assigned to three channels corresponding to the R (red), G (green), and B (blue) layers to generate the composite image.

14. An image inspection apparatus that performs each step of the image inspection method according to any one of claims 11 to 13.

15. A program for causing a computer to perform each step of the image learning method described in any one of claims 1 to 9.

16. A program for causing a computer to perform each step of the image inspection method according to any one of claims 11 to 13.

17. A semiconductor device manufacturing method comprising: an inspection step of performing image inspection to classify defects from images based on a model that has learned images; and a step of shipping semiconductor devices that have passed the inspection step and been determined to be good products, wherein the inspection step is a step of performing each step of the image inspection method described in any one of claims 11 to 13 by a computer system.

Images