Interpretable deep learning based defect detection and classification

Abstract

A description of the detection / classification algorithm, implemented using a deep learning neural network, clarifies the results produced and helps users identify the root cause of defect detection / classification model performance issues. A relevance map is determined based on a layer-by-layer relevance propagation algorithm. An average intersection-union ratio score between the relevance map and the ground truth is determined. The portion of one of the semiconductor images that contributed to classification using the deep learning model is identified based on the relevance map and the average intersection-union ratio score.

Description

[Technical field]

[0001] This disclosure relates to process control for semiconductor manufacturing. [Background technology]

[0002] As the semiconductor manufacturing industry evolves, there is an increasing demand for yield management, especially metrology and inspection systems. As critical dimensions continue to shrink, the industry is under pressure to achieve higher yields and higher value production in shorter times. Reducing the total time between detecting a yield problem and correcting it is critical to the return on investment for semiconductor manufacturers.

[0003] In the fabrication of semiconductor devices, such as logic and memory devices, semiconductor wafers are typically processed using a number of manufacturing processes to form the various features and multiple tiers of the semiconductor devices. For example, in a semiconductor fabrication process known as lithography, a pattern is transferred from a reticle onto a photoresist disposed on the semiconductor wafer. Further examples of semiconductor fabrication processes include, but are not limited to, chemical mechanical polishing (CMP), etching, deposition, and ion implantation. An array of semiconductor devices fabricated on a semiconductor wafer can be separated into individual semiconductor devices.

[0004] Inspection processes are used at various steps during semiconductor manufacturing, whereby detection of wafer-side defects can help increase yields and therefore profits in the manufacturing process. Inspection has always been an important part of the manufacturing of semiconductor devices, such as integrated circuits (ICs). However, as the dimensions of semiconductor devices shrink, inspection has become ever more important for the successful manufacture of acceptable semiconductor devices, because even smaller defects can cause device malfunctions. For example, as the dimensions of semiconductor devices shrink, detection of smaller sized defects becomes necessary, because even relatively small defects can cause unwanted errors in those semiconductor devices.

[0005] As design rules shrink, semiconductor manufacturing processes may operate closer to the limits of their performance capabilities. In addition, as design rules shrink, smaller defects may affect the electrical parameters of the devices, thus driving the inspection to become more sensitive. As design rules shrink, the population of potential yield-related defects detected by inspection grows dramatically, and the population of nuisance defects detected by inspection also grows dramatically. Thus, more defects may be detected on the wafers, and thus eliminating all defects by correcting the process may become difficult and expensive. By determining which defects actually affect the electrical parameters and yield of the devices, process control methods may focus on those defects while generally ignoring other defects. Furthermore, with smaller design rules, process-induced upsets tend to be systematic, in some cases. That is, process-induced upsets tend to be upset in a given design pattern that is often repeated many times in the design. Elimination of spatially systemic electrical related defects can have a large impact on yield.

[0006] Deep learning models can be used to identify and classify defects in inspection images. However, the classification results are not subject to step-by-step validation throughout the network architecture. Because the inner workings of the deep learning model are "hidden," the final decisions made by the deep learning model cannot be retraced for each neuron in the network and are not known to the user. Providing more confidence is necessary to enhance quality.

[0007] Currently, the performance of deep learning models is evaluated by observing output metrics and inferring possible causes of performance issues. This can incorporate observing training loss curves. For detection, the receiver operating characteristic (ROC) can be observed along with the detection map to infer why defects were missed. For classification, the ROC can be observed along with the confusion matrix to infer why classification failed. With these current techniques, the user can only evaluate the output of the neural network. The user is not told why the neural network made the decision it did; the user must infer the root cause of the problem, which requires extensive domain knowledge. In the context of semiconductor manufacturing, it is difficult to explain performance issues without an explanation of why the neural network made the decision it did. [Prior art documents] [Non-patent literature]

[0008] [Non-Patent Document 1] Bach et al., “On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation,” PloS one 10 (2015) [Non-Patent Document 2] Zintraf et al, “Visualizing Deep Neural Network Decisions: Prediction Difference Analysis” (2017) Summary of the Invention [Problem to be solved by the invention]

[0009] Therefore, new inspection and classification systems and methods are needed. [Means for solving the problem]

[0010] In a first embodiment, a method is provided, comprising: performing defect detection and classification on a plurality of semiconductor images using a processor in conjunction with a deep learning model; applying a layer-wise relevance propagation algorithm to the semiconductor images derived from the deep learning model using the processor; determining a relevance map based on the layer-wise relevance propagation algorithm using the processor; determining a mean intersection over union score between the relevance map and a ground truth using the processor; and identifying a portion of one of the semiconductor images that contributed to classification using the deep learning model based on the relevance map and the mean intersection over union score.

[0011] The relevance map can be based on the output layer relevance and the early layer relevance. In one example, the relevance map is normalized. The early layer relevance is calculated using the equation

number

[0012] The method may further include smoothing the multiple semiconductor images using the processor.

[0013] The method can also vary the annotations of the training images for the deep learning model.

[0014] The method can also vary the class code of the training images for the deep learning model.

[0015] The method further includes the step of generating the semiconductor images using a wafer inspection tool that uses an electron beam.

[0016] In a second embodiment, a wafer inspection tool is provided. The wafer inspection tool includes an energy beam source generating an energy beam, a stage configured to hold a wafer, a detector receiving the energy beam returning from the wafer, and a processor in electronic communication with the detector. The processor is configured to generate a plurality of semiconductor images using data received from the detector, perform defect detection and classification on the plurality of semiconductor images using a deep learning model, apply a layer-by-layer relevance propagation algorithm to the semiconductor images derived from the deep learning model, determine a relevance map based on the layer-by-layer relevance propagation algorithm, determine an average intersection-union ratio score between the relevance map and a ground truth, and identify a portion of one of the semiconductor images that contributed to the classification using the deep learning model based on the relevance map and the average intersection-union ratio score. The energy beam can be an electron beam, and the energy beam source can be an electron beam source.

[0017] The relevance map can be based on the output layer relevance and the early layer relevance. The relevance map can also be normalized. The early layer relevance can be calculated by the equation

number

[0018] In a third embodiment, a non-transitory computer-readable storage medium is provided, the non-transitory computer-readable storage medium having one or more programs for executing the following steps on one or more information processing devices, including: performing defect detection and classification on a plurality of semiconductor images using a deep learning model, applying a layer-by-layer relevance propagation algorithm to the semiconductor images derived from the deep learning model, determining a relevance map based on the layer-by-layer relevance propagation algorithm, determining an average intersection-union ratio score between the relevance map and a ground truth, and identifying a portion of one of the semiconductor images that contributed to classification using the deep learning model based on the relevance map and the average intersection-union ratio score.

[0019] The relevance map can be based on the output layer relevance and the early layer relevance. The relevance map can also be normalized. The early layer relevance can be calculated by the equation

number

[0020] The steps may further include generating the semiconductor images using data from a wafer inspection tool that uses an electron beam.

[0021] For a more complete understanding of the nature and objects of the present disclosure, reference should be made to the following detailed description taken in conjunction with the accompanying drawings, in which: [Brief description of the drawings]

[0022] [Figure 1]FIG. 1 shows example corresponding semiconductor images, with a scanning electron microscope (SEM) image on the left, a layer-by-layer relation propagation (LRP) image in the middle, and an overlay of the SEM and LRP on the right. [Diagram 2] 1 is a flow chart of a method embodiment according to the present disclosure. [Diagram 3] FIG. 13 is a diagram showing the results of a comparison of visualization methods. [Figure 4] 1 is a flow chart of one embodiment of a potential user interface according to the present disclosure. [Diagram 5] FIG. 1 illustrates the use of LRP to address inaccurately classified sites. [Figure 6] FIG. 1 illustrates the use of LRP with class-based accurate labels. [Figure 7] FIG. 1 is a diagram of a system embodiment according to the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Although the claimed subject matter is described in terms of certain embodiments, other embodiments exist within the scope of this disclosure, including embodiments that do not provide all of the benefits and features described herein. Various structural, logical, process step, and electronic changes may be made without departing from the scope of the disclosure. Accordingly, the scope of the disclosure is to be determined solely by reference to the appended claims.

[0024] In the embodiments disclosed herein, an explanation of the results of the detection / classification algorithm using neural networks is provided. The explanation can assist users, e.g., semiconductor manufacturers, in understanding the deep learning results and how to diagnose and / or improve the performance. The root cause of performance issues of the defect detection / classification model can be identified, thereby assisting the user to improve the performance of the model. A quantitative score (mean intersection-union ratio (mIoU)) can be determined between the LRP map and the ground truth annotations. This can provide annotation feedback to the user in case the annotation is inaccurate. This can also guide the user to improve the detection and classification performance. The results can be used to interpret the deep learning-based defect detection and classification using LRP.

[0025] FIG. 1 shows example corresponding semiconductor images, with an SEM image on the left, an LRP image in the middle, and an overlay of the SEM and LRP on the right. As shown in the SEM image, the trench 100 is not located in the center of the structure (shown in black outline). When making a decision to detect or classify a defect because of the location of the trench 100, the deep learning model looks at region 101 (shown in gray) in the overlay image in this example. In FIG. 1, region 101 may lead the deep learning model to detect / classify the defect. Embodiments of the present disclosure may help identify region 101, which may aid a user in understanding the operation of the deep learning model.

[0026] 2 is a flow chart of method 200. Some or all of the steps of method 200 may be executed on a processor. At 201, defect detection and classification is performed on a semiconductor image using a deep learning model. In one example, the deep learning model is a convolutional neural network (CNN). The semiconductor image may be generated, for example, using a wafer inspection tool, such as a SEM, that uses an electron beam.

[0027] The deep learning model can be trained with training images, and the training images can be annotated and categorized by class codes.

[0028] At 202, the LRP algorithm is applied to the semiconductor images derived from the deep learning model. LRP is a framework that decomposes neural network predictions computed over an example (e.g., an image) into relevance scores for a single input dimension of the example, e.g., sub-pixels of an image. The LRP algorithm is disclosed in Non-Patent Document 1, which is incorporated by reference in its entirety.

[0029] In the forward pass, the trained neural network propagates the input through the network and a prediction is made at the output layer. The relevance of the output layer is the predicted probability of the class the user chooses to investigate, which is usually the one with the maximum value. The maximum value is the one with the highest probability. In other words, it means that the network is most confident about that class. In the backward pass, the relevance of the early layers is calculated using the equation

number

[0030] By way of example, the neural network may have approximately 50 layers, although more or fewer layers may be used.

[0031] In 203, a relevance map based on the layer-by-layer relevance propagation algorithm is determined. j ) can be calculated backwards from output layer j to input layer i. Iterations can be performed until the image layer is reached. The relevance map can be based on the relevance of the output layer and the relevance of the earlier layers. The relevance map can be normalized, for example, between [0,255]. For example, an example relevance map is shown in Figure 3.

[0032] Returning to FIG. 2, at 204, the average intersection-union ratio (mIoU) score between the relevance map and the ground truth is determined. The average intersection-union ratio is the area of ​​overlap divided by the area of ​​union. Thus, it is obtained by dividing the true positives by the sum of the true positives, false positives, and false negatives. The mIoU can be used to determine the overlap between the network predictions and the ground truth. The mIoU can be used to find out which training images or annotations are causing the user's results.

[0033] For accurate training images, running LRP should yield a high mIoU along with the image label, while for inaccurate training images, it should yield a low mIoU. Highlighted are the relevance scores. mIoU is determined using the relevance map along with the image's original label.

[0034] At 205, one of the semiconductor images that contributed to the classification using the deep learning model is identified based on the relevance map and the mIoU score. The mIoU threshold can be set for all images run and to find examples with mIoU below the threshold to identify problematic SEM images with incorrect annotations.

[0035] For example, an example semiconductor image is shown in FIG. 1. A visual display on the relevance map, as shown in area 101, can provide information about the portions of the image that were used by the neural network. The LRP relevance map and mIoU scores can be used to display the portions of the input image that contributed to the neural network decision. Overlay of the threshold mIoU on the raw SEM image can be used to highlight the contributing portions of the image. The ground truth used by the neural network can be quantitatively measured. The user can receive guidance on next steps and how to correct performance issues with the neural network. This area 101 can guide the user in determining why an image is or is not classified.

[0036] In one example, the images can be smoothed using a Gaussian kernel or other techniques before making decisions or identifications. Pre-processing the images to remove noise can improve the mIoU performance.

[0037] The method 200 may further vary the annotations of the training images or the class codes of the training images. For example, the region 101 in FIG. 1 may be used as a basis to vary the annotations or class codes. The annotations may be varied if non-defect areas contribute to the detection. The class codes may be varied if the images contribute to an inaccurate class code. More training images may be used for a particular structure or defect class if this would improve the defect detection or classification. For example, more training images of a structure may be provided to the neural network if the structure is not used by the neural network or if the relevance map value is low, almost zero.

[0038] For example, a class A vs. class B misclassification may be caused by an annotation of class C in a training image. Using the LRP relevance map, we can identify parts of the image that are likely to contribute to class C.

[0039] Although LRP is disclosed, other deep learning visualization methods can also use the embodiments of the present disclosure. Thus, GradCam, deconvolution, and Guided-Bp can all benefit from the embodiments of the present disclosure. LRP exhibits improved focus and tends to capture useful information. As shown in FIG. 3, more detailed information is captured in LRP compared to GradCam, which can contribute to the results. Diff in FIG. 3 is a predictive difference analysis described in Non-Patent Document 2, which is incorporated by reference in its entirety. LRP provides improved results compared to Diff. LRP shows a more continuous relevance map compared to Guided-Bp, while Guided-Bp has breaks due to discontinuous gradients.

[0040] Figure 4 depicts the user interface flowchart. The user can review the classification (cls) and detection (detect) results in Smart Example Annotation (SEA). The SMARTS Training Station (STS) performs layer-wise relevance propagation. The overlay or relevance image can be displayed in SEA. The user can decide whether the relevance is likely to be acceptable or not. If yes, the patch can be saved as reviewed. If no, the mIoU score can be used to identify any problematic annotations.

[0041] For detection, the relevance map provided by the LRP algorithm can be used to search for potentially erroneous annotations in the image by calculating the similarity between the annotations and the relevance provided by the LRP. For example, the image shown in Figure 5 is incorrectly classified. The highlighted area in Figure 5 shows the image section used for the neural network detection and classification. There is strong overlap with high mIoU. The LRP image can be preprocessed to determine the mIoU of the annotations and the LRP. In this case, the root causes include several shifted in-place defects that were incorrectly annotated as twist defects.

[0042] Furthermore, if the model performs well, the site will be correctly annotated and its IoU will be higher than other annotations, as shown in Figure 6. The highlighted area in Figure 6 shows the image segments used for detection and classification by the neural network.

[0043] Currently, it is difficult to explain deep learning results and guide next steps during recipe setup. Users use trial and error to debug performance issues, resulting in long time-to-recipe (TTR). The disclosed embodiments can provide clear guidance for performance diagnosis, thereby reducing TTR. For example, TTR can be reduced by about 20%. The disclosed embodiments also improve ease of use and reduce the need for support at the manufacturing facility. Deep learning-based detection and classification is more usable because of direct feedback in the neural network. Deep learning-based detection and classification can be used more intuitively at the manufacturing facility.

[0044] 7 is a block diagram of one embodiment of a system 300. The system 300 includes a wafer inspection tool (that includes an electron column 301) configured to generate an image of a wafer 304.

[0045] The wafer inspection tool includes a power acquisition subsystem that includes at least an energy source and a detector. The power acquisition subsystem can be an electron beam power acquisition subsystem. For example, in one embodiment, the energy directed to the wafer 304 is electrons, and the energy detected from the wafer 304 is electrons. In this way, the energy source can be an electron beam source. One such embodiment is shown in FIG. 7, where the power acquisition subsystem includes an electron column 301 coupled to a computer subsystem 302. A stage 310 can hold the wafer 304.

[0046] 7, electron column 301 includes an electron beam source 303 configured to generate electrons that are focused onto wafer 304 by one or more elements 305. Electron beam source 303 may include, for example, a cathode source or an emitter tip. The one or more elements 305 may include, for example, a gun lens, an anode, a beam limiting aperture, a gate valve, a beam current selection aperture, an objective lens, and a scanning subsystem, all of which may include any type of suitable element known in the art.

[0047] Electrons returning from wafer 304 (e.g., secondary electrons) can be focused by one or more elements 306 onto detector 307. One or more elements 306 can include, for example, a scanning subsystem, which can be the same scanning subsystem included in element 305.

[0048] Any other suitable elements known in the art may also be included within electron column 301 .

[0049] 7, electron column 301 is configured such that electrons are directed at wafer 304 at one oblique angle of incidence and scattered from wafer 304 at another oblique angle, although the electron beam may be directed at wafer 304 and scattered from wafer 304 at any suitable angle. In addition, the electron beam subsystem may be configured to generate images of wafer 304 using a number of different modes (e.g., with various illumination angles, collection angles, etc.). The different modes of the electron beam subsystem may be determined by different imaging parameters of the output acquisition subsystem.

[0050] The computer subsystem 302 may be coupled to the detector 307 as described above. At the detector 307, electrons returning from the surface of the wafer 304 may be detected to form electron beam images of the wafer 304. The electron beam images may include any suitable electron beam images. The computer subsystem 302 may be configured to perform any of the functions described herein using the output of the detector 307 and / or the electron beam images. The computer subsystem 302 may be configured to perform any additional step(s) described herein. The system 300 having the output acquisition subsystem shown in FIG. 7 may be further configured as described herein.

[0051] It is noted that FIG. 7 is provided herein to provide a general depiction of an e-beam power acquisition subsystem configuration that may be used in the embodiments described herein. As is commonly done when designing a commercial power acquisition system, the e-beam power acquisition subsystem configuration described herein may be modified to optimize the performance of the power acquisition subsystem. In addition, existing systems may be used to implement the systems described herein (e.g., by adding the features described herein to the existing system). Some of these systems may provide the methods described herein as optional features of the system (e.g., in addition to other features of the system). Alternatively, the systems described herein may be designed as entirely new systems.

[0052] Although the power acquisition subsystem is described above as an electron beam power acquisition subsystem, the power acquisition subsystem can also be an ion beam power acquisition subsystem. Such a power acquisition subsystem can be configured as shown in FIG. 7, except that the electron beam source is replaced with any suitable ion beam source known in the art. In addition, the power acquisition subsystem can be any other suitable ion beam power acquisition subsystem, such as those found in commercially available focused ion beam (FIB) systems, helium ion microscope (HIM) systems, and secondary ion mass spectrometry (SIMS) systems.

[0053] The computer subsystem 302 includes a processor 308 and an electronic data storage unit 309. The processor 308 may include a microprocessor, microcontroller or other device.

[0054] The computer subsystem 302 may be coupled to the components of the system 300 in any suitable manner (e.g., via one or more transmission media, including, for example, wired and / or wireless transmission media) such that its output may be received by its processor 308. The processor 308 may be configured to use its output to perform a number of functions. A wafer inspection tool may receive instructions and other information from the processor 308. The processor 308 and / or electronic data storage unit 309 may optionally be in electronic communication with another wafer inspection tool, wafer metrology tool, or wafer review tool (not shown) to receive additional information or send instructions.

[0055] The processor 308 is in electronic communication with a wafer inspection tool, such as the detector 307. The processor 308 may be configured to process images generated using metrology results from the detector 307. For example, embodiments of the method 200 may be performed in the processor.

[0056] The computer subsystem 302 or other system(s) or other subsystem(s) described herein can be part of a variety of systems, including personal computer systems, image computers, mainframe computer systems, workstations, network appliances, Internet appliances, and other devices. The subsystem(s) or system(s) can include any suitable processor, such as a parallel processor, known in the art. Additionally, the subsystem(s) or system(s) can include platforms with high speed processing and software, whether standalone or networked tools.

[0057] The processor 308 and electronic data storage unit 309 may be part of, e.g., located within, the system 300 or other device. According to one example, the processor 308 and electronic data storage unit 309 may be part of a stand-alone control unit or may be provided within a centralized quality control unit. Multiple processors 308 or electronic data storage units 309 may also be used.

[0058] Processor 308 may be implemented in any combination of hardware, software, and firmware, and its functions, as described herein, may be performed in a single unit or may be distributed among several different components, each of which may in turn be implemented in any combination of hardware, software, and firmware. Program codes or instructions for causing processor 308 to perform and implement various methods and functions may be stored in a readable storage medium, such as a memory in electronic data storage unit 309 or other memory.

[0059] If system 300 includes multiple computer subsystems 302, the separate subsystems may be coupled together to allow images, data, information, instructions, etc. to be sent between the subsystems. For example, a subsystem may be coupled to additional subsystem(s) by any suitable transmission medium, which may include any suitable wired and / or wireless transmission medium known in the art. Two or more of such subsystems may also be coupled together substantially by a shared computer-readable storage medium (not shown).

[0060] Processor 308 may be configured to perform a number of functions using the output of system 300 or other outputs. For example, processor 308 may be configured to send its output to an electronic data storage unit 309 or other storage medium. Processor 308 may be further configured as described herein.

[0061] The processor 308 or computer subsystem 302 may be part of a defect review system, an inspection system, a metrology system, or some other system, such that the several configurations described in accordance with the embodiments of the present disclosure may be tailored in various ways for systems having different capabilities and more or less suited to different applications.

[0062] Processor 308 may be communicatively coupled to any of the various components or subsystems of system 300 in any manner known in the art. Additionally, processor 308 may be configured to receive and / or acquire data or information from other systems (e.g., inspection results from an inspection system, such as a review tool, design data in a remote database, etc.) and to do so through a transmission medium, such as having wired and / or wireless legs, that may act as a data link between processor 308 and other subsystems of system 300 or systems outside of system 300.

[0063] The various steps, functions and / or operations of the system 300 and methods disclosed herein may be performed by one or more of electronic circuits, logic gates, multiplexers, programmable logic devices, ASICs, analog or digital controllers / switches, microcontrollers, or information processing systems. Program instructions implementing the methods, such as those described herein, may be transmitted or stored on a carrier medium. The carrier medium may include a storage medium, such as a read-only memory, a random access memory, a magnetic or optical disk, a non-volatile memory, a solid-state memory, a magnetic tape, or the like. The carrier medium may include a transmission medium, such as a wire, a cable, or a wireless transmission link. For example, the various steps described throughout this disclosure may be performed by a single processor 308 (or computer subsystem 302) or, alternatively, by multiple processors 308 (or multiple computer subsystems 302). Additionally, the various subsystems included in the system 300 may include one or more information processing or logic systems. Therefore, the above description should be construed as merely illustrative and not limiting on the present disclosure.

[0064] In one example, the processor 308 may be a GPU, and in another example, the processor 308 may be a CPU.

[0065] The processor 308 can perform any of the embodiments disclosed herein, including the embodiment of FIG. 2. The processor 308 can also be configured to perform other functions or additional steps using the output of the system 300 or using images or data from other sources. In one example, the processor 308 is configured to generate a plurality of semiconductor images from data received from a detector. The processor 308 can perform defect detection and classification on the plurality of semiconductor images using a deep learning model. The processor can apply an LRP algorithm to the semiconductor images derived from the deep learning model. The processor 308 can determine a relevance map based on the LRP propagation algorithm. The processor 308 can determine an average intersection union ratio score between the relevance map and a ground truth. The processor 308 can identify a portion of one of the semiconductor images that contributed to the classification using the deep learning model based on the relevance map and the average intersection union ratio score.

[0066] An additional embodiment relates to a non-transitory computer readable medium having program instructions executable on the controller. Specifically, as shown in Figure 7, a non-transitory computer readable medium having program instructions executable on the processor 308 may be included in an electronic data storage unit 309 or other storage medium. The computer implemented method may include any step(s) of any method(s) described herein.

[0067] In one example, the electronic data storage unit 309 may include one or more programs for executing steps on one or more information processing devices, including performing defect detection and classification on a plurality of semiconductor images using a deep learning model. An LRP algorithm may be applied to the semiconductor images derived from the deep learning model. A relevance map may be determined based on the LRP algorithm. An average intersection-union ratio score between the relevance map and a ground truth may be determined. A portion of one of the semiconductor images that contributed to classification using the deep learning model may be identified based on the relevance map and the average intersection-union ratio score. The relevance map may be based on output layer relevance and early layer relevance. In one example, the relevance map is normalized. Steps may include generating the semiconductor images using data from an electron beam wafer inspection tool or sending a command to obtain the semiconductor images.

[0068] Although described in the context of an image formed from an electron beam, the embodiments described herein can also be implemented using an image formed from a photon beam. That is, the energy beam can be an electron beam or a photon beam. The photon beam can be generated by a laser or a broadband plasma source. The wafer can be held on a chuck in the path of the photon beam. A detector can capture the photon beam reflected from the wafer.

[0069] The embodiments described herein may use deep learning models, which may be CNNs or other types of applications. Such approaches may be referred to as end-to-end learning strategies. For example, in one embodiment, the generative model is a deep generative model. In another embodiment, the generative model is a machine learning model. For example, the generative model may be configured to have a deep learning architecture with multiple layers within the generative model that perform multiple algorithms or transformations. The generative model may also incorporate one or more encoder-side layers and / or one or more decoder-side layers. The number of layers on either or both sides of the generative model is variable and generally example-dependent. The deep generative models and machine learning models may include any suitable such models known in the art that may be configured to perform the functions described herein.

[0070] In one embodiment, the generative model is a CNN. The generative model may have any CNN configuration or architecture known in the art. In general, a wide variety of CNN architectures may be employed and are known in the art.

[0071] The steps of the methods may be performed as described herein. The methods may include any other step(s) that may be performed by a processor and / or computer subsystem(s) or system(s) described herein. The steps may be performed by one or more computer systems, which may be configured according to any of the embodiments described herein. In addition, the methods described above may be performed by any of the system embodiments described herein.

[0072] Although the present disclosure has been described in connection with one or more specific embodiments, other embodiments of the present disclosure may be made without departing from the scope of the present disclosure, that is, the present disclosure is limited only by the appended claims and the reasonable interpretation thereof.

Claims

1. 1. A method comprising: Using a processor in conjunction with a deep learning model to perform defect detection and classification on multiple semiconductor images; applying, using the processor, a layer-by-layer relevance propagation algorithm to the semiconductor images derived from the deep learning model; determining, using the processor, a relevance map based on the layer-by-layer relevance propagation algorithm; determining, with the processor, an average intersection-union ratio score between the relevance map and a ground truth; and using the processor to identify portions of one of the semiconductor images that contributed to the classification using the deep learning model based on the relevance map and the average intersection union ratio score. method.

2. The method of claim 1 , wherein the relevance map is based on output layer relevance and early layer relevance.

3. The method of claim 2 , wherein the relevance map is normalized.

4. The method of claim 2, wherein the relevance of the early tier is R ｉ Let R be the relevance of neuron i in the first layer. ｊ is the relevance of neuron j in the second layer, and a ｉ is the activation value of the neuron i, and w ｉｊ is the weight between the neuron i and the neuron j. [0010] A method in which the layer-by-layer structure is determined using the

5. 2. The method of claim 1, further comprising smoothing the plurality of semiconductor images using the processor.

6. 10. The method of claim 1, further comprising: varying annotations of training images for the deep learning model.

7. 10. The method of claim 1, further comprising varying class codes of training images for the deep learning model.

8. 10. The method of claim 1, further comprising generating the semiconductor image with a wafer inspection tool that uses an electron beam.

9. 1. A wafer inspection tool comprising: an energy beam source generating an energy beam; a stage configured to hold a wafer; a detector for receiving the energy beam returning from the wafer; a processor in electronic communication with the detector; wherein the processor: generating a plurality of semiconductor images using data received from the detector; performing defect detection and classification on the plurality of semiconductor images using a deep learning model; applying a layer-by-layer relevance propagation algorithm to the semiconductor images derived from the deep learning model; determining a relevance map based on the layer-by-layer relevance propagation algorithm; determining an average intersection-union ratio score between the relevance map and a ground truth; and identifying a portion of one of the semiconductor images that contributed to the classification using the deep learning model based on the relevance map and the average intersection union ratio score; A configured wafer inspection tool.

10. 10. The wafer inspection tool of claim 9, wherein the relevance map is based on an output layer relevance and an early layer relevance.

11. The wafer inspection tool of claim 10 , wherein the relevance map is normalized.

12. 11. The wafer inspection tool of claim 10, wherein the relevance of the early layer is R ｉ Let R be the relevance of neuron i in the first layer. ｊ is the relevance of neuron j in the second layer, and a ｉ is the activation value of the neuron i, and w ｉｊ is the weight between the neuron i and the neuron j. [0025] A wafer inspection tool that uses a layer-by-layer determination method.

13. 10. The wafer inspection tool of claim 9, wherein the energy beam is an electron beam and the energy beam source is an electron beam source.

14. A non-transitory computer readable storage medium, the non-transitory computer readable storage medium comprising: performing defect detection and classification on a plurality of semiconductor images using a deep learning model; applying a layer-by-layer relevance propagation algorithm to the semiconductor images derived from the deep learning model; determining a relevance map based on the layer-by-layer relevance propagation algorithm; determining an average intersection-union ratio score between the relevance map and ground truth; and identifying portions of one of the semiconductor images that contributed to the classification using the deep learning model based on the relevance map and the average intersection union ratio score; A non-transitory computer-readable storage medium having one or more programs thereon for executing the method.

15. 15. The non-transitory computer-readable storage medium of claim 14, wherein the relevance map is based on output layer relevance and early layer relevance.

16. 16. The non-transitory computer readable storage medium of claim 15, wherein the relevance map is normalized.

17. 16. The non-transitory computer-readable storage medium of claim 15, wherein the relevance of the early tier is R ｉ Let R be the relevance of neuron i in the first layer. ｊ is the relevance of neuron j in the second layer, and a ｉ is the activation value of the neuron i, and w ｉｊ is the weight between the neuron i and the neuron j. [0030] A non-transitory computer-readable storage medium that is determined for each layer using the above.

18. 15. The non-transitory computer readable storage medium of claim 14, further comprising generating the semiconductor image using data from a wafer inspection tool that uses an electron beam.

Images