Semiconductor inspection system with a deep neural network

Abstract

In a system, such as an optical inspection system, a light source generates a beam of light; a stage holds a workpiece in a path of the beam of light; a detector receives the beam of light reflected from the workpiece; and a processor is in electronic communication with the detector. The processor is configured to operate a deep neural network that includes a backbone network and a plurality of head networks connected to a top layer. Each of the head networks is configured for use during inspection of an image of the workpiece generated using data from the detector.

Description

SEMICONDUCTOR INSPECTION SYSTEM WITH A DEEP NEURAL NETWORKCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to the provisional patent application filed December 7, 2023 and assigned U.S. App. No. 63 / 607,106, the disclosure of which is hereby incorporated by reference.FIELD OF THE DISCLOSURE

[0002] This disclosure relates to semiconductor inspection.BACKGROUND OF THE DISCLOSURE

[0003] Evolution of the semiconductor manufacturing industry is placing greater demands on yield management and, in particular, on metrology and inspection systems. Critical dimensions continue to shrink, yet the industry needs to decrease time for achieving high-yield, high-value production. Minimizing the total time from detecting a yield problem to fixing it determines the return-on-investment for a semiconductor manufacturer.

[0004] Fabricating semiconductor devices, such as logic and memory devices, typically includes processing a workpiece, such as a semiconductor wafer, using a large number of fabrication processes to form various features and multiple levels of the semiconductor devices. For example, lithography is a semiconductor fabrication process that involves transferring a pattern from a reticle to a photoresist arranged on a semiconductor wafer. Additional examples of semiconductor fabrication processes include, but are not limited to, chemical-mechanical polishing (CMP), etch, deposition, and ion implantation. An arrangement of multiple semiconductor devices fabricated on a single semiconductor wafer may be separated into individual semiconductor devices.

[0005] Inspection processes are used at various steps during semiconductor manufacturing to detect defects on wafers to promote higher yield in the manufacturing process and, thus, higher profits. Inspection has always been an important part of fabricating semiconductor devices such as integrated circuits (ICs). However, as the dimensions of semiconductor devices decrease, inspection becomes even more important to the successful manufacture of acceptable semiconductor devices because smaller defects can cause the devices to fail. For instance, as the dimensions ofsemiconductor devices decrease, detection of defects of decreasing size has become necessary because even relatively small defects may cause unwanted aberrations in the semiconductor devices.

[0006] As design rules shrink, however, semiconductor manufacturing processes may be operating closer to the limitation on the performance capability of the processes. In addition, smaller defects can have an impact on the electrical parameters of the device as the design rules shrink, which drives more sensitive inspections. As design rules shrink, the population of potentially yield-relevant defects detected by inspection grows dramatically, and the population of nuisance defects detected by inspection also increases dramatically. Therefore, more defects may be detected on the wafers, and correcting the processes to eliminate all of the defects may be difficult and expensive. Determining which of the defects actually have an effect on the electrical parameters of the devices and the yield may allow process control methods to be focused on those defects while largely ignoring others. Furthermore, at smaller design rules, process-induced failures, in some cases, tend to be systematic. That is, process-induced failures tend to fail at predetermined design patterns often repeated many times within the design. Elimination of spatially-systematic, electrically-relevant defects can have an impact on yield.

[0007] Deep neural networks (DNNs) can be used to identify and classify defects in inspection images, but DNNs have limitations. Previously, an image-based DNN was operated by saving the model to a binary file ready for inference, such as TensorFlow or Torch. At inference time, the binary model was loaded into the CPU or GPU memory using a dedicated C++ library or using Python. If the model did not fit into the available GPU memory, the model was not loaded into memory. For example, a 20 GB model cannot fit into a 12 GB GPU device. Loading the model into memory may have taken few seconds because the GPU device needed to allocate hundreds of compute unified device architecture (CUD A) buffers and load the large set of network weights into memory. After loading, the model could be used. However, if a different model was needed for a different inspection task, then the model needed to be unloaded and a new model was loaded in its place. This unloading and loading process took time, which slowed inspection throughput. New systems and techniques are needed.BRIEF SUMMARY OF THE DISCLOSURE

[0008] A system is provided in a first embodiment. The system includes a light source that generates a beam of light; a stage configured to hold a workpiece (e.g., a semiconductor wafer) in a path of the beam of light; a detector that receives the beam of light reflected from the workpiece; a processor (e.g., CPU or GPU) in electronic communication with the detector; and an electronic data storage unit in electronic communication with the processor. The processor is configured to operate a deep neural network that includes a backbone network and a plurality of head networks connected to a top layer. Each of the head networks is configured for use during inspection of an image of the workpiece generated using data from the detector. The deep neural network is stored on the electronic data storage unit.

[0009] Each of the head networks can be configured to perform image classification, image regression, object detection, dense segmentation, instance segmentation, or depth estimation.

[0010] One or more of the head networks can be configured to operate simultaneously.

[0011] The deep neural network can be configured to change between the plurality of head networks during the inspection of the image.

[0012] A method is provided in a second embodiment. The method includes receiving, at a processor (e.g., CPU or GPU), an image of a workpiece (e.g., a semiconductor wafer). The image is inspected using the processor. The inspecting uses a deep neural network that includes a backbone network and a plurality of head networks connected to a top layer. Each of the head networks is configured for use during inspection of the image.

[0013] The method can include directing a beam of light at the workpiece, receiving the beam of light reflected from the workpiece at a detector, and communicating data from the detector to the processor to generate the image.

[0014] Each of the head networks can be configured to perform image classification, image regression, object detection, dense segmentation, instance segmentation, or depth estimation.

[0015] One or more of the head networks can be configured to operate simultaneously.

[0016] The deep neural network can be configured to change between the plurality of head networks during the inspection of the image.

[0017] A non-transitory computer-readable storage medium is provided in a third embodiment. The non-transitory computer-readable storage medium includes one or more programs for executing the following steps on one or more computing devices. The steps include receiving an image of a workpiece (e.g., a semiconductor waferO and inspecting the image using a deep neural network that includes a backbone network and a plurality of head networks connected to a top layer. Each of the head networks is configured for use during inspection of the image.

[0018] Each of the head networks can be configured to perform image classification, image regression, object detection, dense segmentation, instance segmentation, or depth estimation.

[0019] One or more of the head networks can be configured to operate simultaneously.

[0020] The deep neural network can be configured to change between the plurality of head networks during the inspection of the image.DESCRIPTION OF THE DRAWINGS

[0021] For a fuller understanding of the nature and objects of the disclosure, reference should be made to the following detailed description taken in conjunction with the accompanying drawings, in which:FIGS. 1 and 2 are flowcharts illustrating embodiments of the DNN in accordance with the present disclosure; andFIG. 3 is a block diagram showing an exemplary inspection system.DETAILED DESCRIPTION OF THE DISCLOSURE

[0022] Although claimed subject matter will be described in terms of certain embodiments, other embodiments, including embodiments that do not provide all of the benefits and features set forth herein, are also within the scope of this disclosure. Various structural, logical, process step, and electronic changes may be made without departing from the scope of the disclosure. Accordingly, the scope of the disclosure is defined only by reference to the appended claims.

[0023] Embodiments disclosed herein describe a method for loading several DNNs based on the same backbone network. Most of the image processing and computer vision algorithms use a representation self-supervised network as a backbone for feature extraction from a given image. A backbone means that the DNN is composed of at least two consecutive sub networks: the backbone and the head network. For example, a frozen DNN can include a binary pre-trained DNN saved as a single binary file with all the network weights. The head network is a specific network that can be used for Al tasks, such as image classification, image regression, object detection, dense segmentation, instance segmentation, depth estimation, or other inspection techniques.

[0024] FIGS. 1 and 2 show embodiments of the DNN in accordance with the present disclosure. Embodiments disclosed herein can use a processor, such as a GPU, TPU, or CPU. In an embodiment, an image of a workpiece (such as a semiconductor wafer or a device on part of a semiconductor wafer) is received. This image can be generated using data from a detector in an inspection system. For example, a beam of light can be directed at the workpiece. Light reflected from the workpiece can be received at a detector. Data from the detector can be communicated to the processor to generate the image.

[0025] The image is then inspected using the DNN. The DNN includes a backbone network and a plurality of head networks connected to a top layer. Each of the head networks is configured for use during inspection of the image. For example, each of the head networks is configured to perform image classification, image regression, object detection, dense segmentation, instance segmentation, depth estimation, or other functions. Each head network can be configured to connect to the backbone network outputs shape. In FIG. 1, the head networks include a first classification head network, a second classification head network, a logistic regression head network, a dense segmentation head network, and a depth estimation head network. Other combinations of head networks are possible and FIG. 1 is merely one example. For example, an anomaly detection head network can be used.

[0026] A program client, such as a C++ program client, can train a set of DNN models for different Al tasks all based on the same backbone pretrained network. The network is packed as a multi-head network, meaning a single model that has a single backbone body and multiple head networks connected at the top layer of the backbone network. This means that multiple different Altasks are enabled, each with its own head network to be connected to the backbone network. A larger model is loaded into memory once and this model can serve multiple different Al tasks simultaneously after being loaded. This avoids loading multiple models into memory or the previous load / unload process. One or more of the head networks can process a different image simultaneously during the operation. For example, each of the head networks can process a different image simultaneously. In an example, a batch of thirty-two images was processed simultaneously by all the head networks. The output will have the shape of all the connected head networks.

[0027] FIG. 2 shows loading the backbone and changing between the head networks during the inspecting as shown with the numbered timeline at the bottom. The switchable head networks (#2) are attached after the backbone is loaded (#1). During inspection, the various head networks are attached and detached as necessary. The head networks can operate at least partly simultaneously or sequentially. For example, the first classification head network is detached in FIG. 2 at #3 on the timeline across the bottom and the logistic regression head network is attached at #4 on the timeline. Head networks can be switched when needed. During switching, the model may be unable to serve requests. One head network is detached and another head network may replace it. After the new head network replaces the previous head network, inference can continue as usual.

[0028] The embodiments disclosed herein enable several ways of switchable-head like packing four different Al image classification networks into a single model. For example, two, three, five, or more than five different Al classification networks can be used. The embodiments disclosed herein also enable packing two classification models together with dense segmentation models and a single depth estimation model all in one model. The number of head networks can depend on the processing memory. For example, a GPU with 1 GB of memory could load ten head networks if each head network is about 100 MB.

[0029] The embodiments disclosed herein also enable on-the-fly replacement of a head network or several head networks from a loaded model so the model can stay in memory while one of its head networks is being replaced. Thus, for a single image, the DNN can replace the head network several times for the same image. For example, unloading and reloading a model underprevious techniques can take seconds whereas detaching and attaching a new head network may take hundreds of milliseconds.

[0030] In an example, the backbone model can take 10 GB of memory. Using the embodiments disclosed herein, when a head network is replaced only the memory consumed by this head network is freed, In the usual case, the whole model is freed from main memory and needs to be allocated and loaded again each time.

[0031] In another example, there are three head networks: a classification head network that predicts yes / no if this image has a suspected defect, a head network that generates a defect detection map per pixel, and an image regression head network that predicts the size of the defect in pixels area unit. These three head networks can be connected to the same backbone network. Images can be processed sequentially by these three head networks. The head networks can review the same batch of input images.

[0032] The backbone network can use a pretrained network, such as DINOv2 or other backbone networks. DINOv2 is a method for training computer vision models that uses selfsupervised learning. Similar to other self-supervised systems, models using the DINOv2 method can be trained on a collection of images without labels. DINOv2 can provide features that can be directly used as inputs for simple linear classifiers, which means DINOv2 can be used to create multipurpose backbones for many different computer vision tasks. DINOv2 also can be used with depth estimation, image segmentation, classification, and instance retrieval. Additional details of DIN0v2 are described in Oquab et al., “DIN0v2: Learning Robust Visual Features without Supervision” (April 2023), the relevant portions of which are incorporated by reference.

[0033] The embodiments disclosed herein can execute more image-based inference per second on a single CPU or GPU device. Thus, on the same hardware, the inspection system can execute more pixels per second because a multi-head system based on a single backbone can be packed into a single Al model. A desired set of Al models can be packed into a single loadable model into the processor, such as based on memory capabilities, and can serve up to multiple different clients using a single loaded DNN model. Executing a set of DNNs all attached to a single backbone network means that a single model will have the output of several algorithms for a given input image.

[0034] A DNN (e.g., a convolutional neural network (CNN), vision transformer network (VIT), residual network (ResNet), or auto encoder (UNET)) can be executed by the processor. The DNN can have one of the configurations described further herein. Rooted in neural network technology, deep learning is a probabilistic graph model with many neuron layers, commonly known as a deep architecture. Deep learning technology processes the information such as image, text, voice, and so on in a hierarchical manner. In using deep learning in the present disclosure, feature extraction is accomplished automatically using learning from data.

[0035] Generally speaking, deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high level abstractions in data. In a simple case, there may be two sets of neurons: ones that receive an input signal and ones that send an output signal. When the input layer receives an input, it passes on a modified version of the input to the next layer. In a deep network, there are many layers between the input and output, allowing the algorithm to use multiple processing layers, composed of multiple linear and non-linear transformations.

[0036] Deep learning is part of a broader family of machine learning methods based on learning representations of data. An observation (e.g., a feature to be extracted for reference) can be represented in many ways such as a vector of intensity values per pixel, or in a more abstract way as a set of edges, regions of particular shape, etc. Some representations are better than others at simplifying the learning task (e.g., face recognition or facial expression recognition). Deep learning can provide efficient algorithms for unsupervised or semi-supervised feature learning and hierarchical feature extraction.

[0037] Research in this area attempts to make better representations and create models to learn these representations from large-scale data. Some of the representations are inspired by advances in neuroscience and are loosely based on interpretation of information processing and communication patterns in a nervous system, such as neural coding which attempts to define a relationship between various stimuli and associated neuronal responses in the brain.

[0038] There are many variants of neural networks with deep architecture depending on the probability specification and network architecture, including, but not limited to, Deep Belief Networks (DBN), Restricted Boltzmann Machines (RBM), and Auto-Encoders. Another type ofdeep neural network, a CNN, can be used for feature analysis. The actual implementation may vary depending on the size of input images, the number of features to be analyzed, and the nature of the problem. Other layers may be included in the deep learning module besides the neural networks disclosed herein.

[0039] In a further embodiment, the DNN can have a set of weights that model the world according to the data that it has been fed to train it. Neural networks can be generally defined as a computational approach, based on a relatively large collection of neural units loosely modeling the way a biological brain solves problems with relatively large clusters of biological neurons connected by axons. Each neural unit is connected with many others, and links can be enforcing or inhibitory in their effect on the activation state of connected neural units. These systems are self-learning and trained rather than explicitly programmed and excel in areas where the solution or feature detection is difficult to express in a traditional computer program.

[0040] Neural networks typically comprise multiple layers, and the signal path traverses from front to back. The goal of the neural network is to solve problems in the same way that the human brain would, although several neural networks are much more abstract. Modern neural network projects typically work with a few thousand to a few million neural units and millions of connections. The neural network may have any suitable architecture and / or configuration known in the art.

[0041] In general, the DNN described herein is a trained DNN. For example, the DNN may be previously trained by one or more other systems and / or methods. The DNN is already generated and trained and then the functionality of the module is determined as described herein, which can then be used to perform one or more additional functions for the deep learning module.

[0042] As stated above, although a CNN is used herein to illustrate the architecture of a DNN, the present disclosure is not limited to a CNN. Other variants of deep learning architectures may be used in the embodiments disclosed herein.

[0043] One embodiment of a system 200 is shown in FIG. 3. The system 200 includes optical based subsystem 201. In general, the optical based subsystem 201 is configured for generating optical based output for a workpiece 202 by directing light to (or scanning light over) anddetecting light from the workpiece 202. In one embodiment, the workpiece 202 includes a wafer. The wafer may include any wafer known in the art. In another embodiment, the workpiece 202 includes a reticle. The reticle may include any reticle known in the art.

[0044] In the embodiment of the system 200 shown in FIG. 3, optical based subsystem 201 includes an illumination subsystem configured to direct light to workpiece 202. The illumination subsystem includes at least one light source. For example, as shown in FIG. 3, the illumination subsystem includes light source 203. In one embodiment, the illumination subsystem is configured to direct the light to the workpiece 202 at one or more angles of incidence, which may include one or more oblique angles and / or one or more normal angles. For example, as shown in FIG. 3, light from light source 203 is directed through optical element 204 and then lens 205 to workpiece 202 at an oblique angle of incidence. The oblique angle of incidence may include any suitable oblique angle of incidence, which may vary depending on, for instance, characteristics of the workpiece 202.

[0045] The optical based subsystem 201 may be configured to direct the light to the workpiece 202 at different angles of incidence at different times. For example, the optical based subsystem 201 may be configured to alter one or more characteristics of one or more elements of the illumination subsystem such that the light can be directed to the workpiece 202 at an angle of incidence that is different than that shown in FIG. 3. In one such example, the optical based subsystem 201 may be configured to move light source 203, optical element 204, and lens 205 such that the light is directed to the workpiece 202 at a different oblique angle of incidence or a normal (or near normal) angle of incidence.

[0046] In some instances, the optical based subsystem 201 may be configured to direct light to the workpiece 202 at more than one angle of incidence at the same time. For example, the illumination subsystem may include more than one illumination channel, one of the illumination channels may include light source 203, optical element 204, and lens 205 as shown in FIG. 3 and another of the illumination channels (not shown) may include similar elements, which may be configured differently or the same, or may include at least a light source and possibly one or more other components such as those described further herein. If such light is directed to the workpiece at the same time as the other light, one or more characteristics (e.g., wavelength, polarization, etc.) of the light directed to the workpiece 202 at different angles of incidence may be different such thatlight resulting from illumination of the workpiece 202 at the different angles of incidence can be discriminated from each other at the detector(s).

[0047] In another instance, the illumination subsystem may include only one light source (e.g., light source 203 shown in FIG. 3) and light from the light source may be separated into different optical paths (e.g., based on wavelength, polarization, etc.) by one or more optical elements (not shown) of the illumination subsystem. Light in each of the different optical paths may then be directed to the workpiece 202. Multiple illumination channels may be configured to direct light to the workpiece 202 at the same time or at different times (e.g., when different illumination channels are used to sequentially illuminate the workpiece). In another instance, the same illumination channel may be configured to direct light to the workpiece 202 with different characteristics at different times. For example, in some instances, optical element 204 may be configured as a spectral filter and the properties of the spectral filter can be changed in a variety of different ways (e.g., by swapping out the spectral filter) such that different wavelengths of light can be directed to the workpiece 202 at different times. The illumination subsystem may have any other suitable configuration known in the art for directing the light having different or the same characteristics to the workpiece 202 at different or the same angles of incidence sequentially or simultaneously.

[0048] In one embodiment, light source 203 may include a broadband plasma (BBP) source. In this manner, the light generated by the light source 203 and directed to the workpiece 202 may include broadband light. However, the light source may include any other suitable light source such as a laser. The laser may include any suitable laser known in the art and may be configured to generate light at any suitable wavelength or wavelengths known in the art. In addition, the laser may be configured to generate light that is monochromatic or nearly-monochromatic. In this manner, the laser may be a narrowband laser. The light source 203 may also include a polychromatic light source that generates light at multiple discrete wavelengths or wavebands.

[0049] Light from optical element 204 may be focused onto workpiece 202 by lens 205. Although lens 205 is shown in FIG. 3 as a single refractive optical element, it is to be understood that, in practice, lens 205 may include a number of refractive and / or reflective optical elements that in combination focus the light from the optical element to the workpiece. The illumination subsystem shown in FIG. 3 and described herein may include any other suitable optical elements(not shown). Examples of such optical elements include, but are not limited to, polarizing component(s), spectral filter(s), spatial filter(s), reflective optical element(s), apodizer(s), beam splitter(s) (such as beam splitter 213), aperture(s), and the like, which may include any such suitable optical elements known in the art. In addition, the optical based subsystem 201 may be configured to alter one or more of the elements of the illumination subsystem based on the type of illumination to be used for generating the optical based output.

[0050] The optical based subsystem 201 may also include a scanning subsystem configured to cause the light to be scanned over the workpiece 202. For example, the optical based subsystem201 may include stage 206 on which workpiece 202 is disposed during optical based output generation. The scanning subsystem may include any suitable mechanical and / or robotic assembly (that includes stage 206) that can be configured to move the workpiece 202 such that the light can be scanned over the workpiece 202. In addition, or alternatively, the optical based subsystem 201 may be configured such that one or more optical elements of the optical based subsystem 201 perform some scanning of the light over the workpiece 202. The light may be scanned over the workpiece202 in any suitable fashion such as in a serpentine-like path or in a spiral path.

[0051] The optical based subsystem 201 further includes one or more detection channels. At least one of the one or more detection channels includes a detector 212 configured to detect light from the workpiece 202 due to illumination of the workpiece 202 by the subsystem and to generate output responsive to the detected light. For example, the optical based subsystem 201 shown in FIG. 3 includes two detection channels, one formed by collector 207, element 208, and detector 209 and another formed by collector 210, element 211, and detector 212. As shown in FIG. 3, the two detection channels are configured to collect and detect light at different angles of collection. In some instances, both detection channels are configured to detect scattered light, and the detection channels are configured to detect light that is scattered at different angles from the workpiece 202. However, one or more of the detection channels may be configured to detect another type of light from the workpiece 202 (e.g., reflected light).

[0052] As further shown in FIG. 3, both detection channels are shown positioned in the plane of the paper and the illumination subsystem is also shown positioned in the plane of the paper. Therefore, in this embodiment, both detection channels are positioned in (e.g., centered in) the planeof incidence. However, one or more of the detection channels may be positioned out of the plane of incidence. For example, the detection channel formed by collector 210, element 211, and detector 212 may be configured to collect and detect light that is scattered out of the plane of incidence. Therefore, such a detection channel may be commonly referred to as a “side” channel, and such a side channel may be centered in a plane that is substantially perpendicular to the plane of incidence.

[0053] Although FIG. 3 shows an embodiment of the optical based subsystem 201 that includes two detection channels, the optical based subsystem 201 may include a different number of detection channels (e.g., only one detection channel or two or more detection channels). In one such instance, the detection channel formed by collector 210, element 211, and detector 212 may form one side channel as described above, and the optical based subsystem 201 may include an additional detection channel (not shown) formed as another side channel that is positioned on the opposite side of the plane of incidence. Therefore, the optical based subsystem 201 may include the detection channel that includes collector 207, element 208, and detector 209 and that is centered in the plane of incidence and configured to collect and detect light at scattering angle(s) that are at or close to normal to the workpiece 202 surface. This detection channel may therefore be commonly referred to as a “top” channel, and the optical based subsystem 201 may also include two or more side channels configured as described above. As such, the optical based subsystem 201 may include at least three channels (i.e., one top channel and two side channels), and each of the at least three channels has its own collector, each of which is configured to collect light at different scattering angles than each of the other collectors.

[0054] As described further above, each of the detection channels included in the optical based subsystem 201 may be configured to detect scattered light. Therefore, the optical based subsystem 201 shown in FIG. 3 may be configured for dark field (DF) output generation for workpieces 202. However, the optical based subsystem 201 may also or alternatively include detection channel(s) that are configured for bright field (BF) output generation for workpieces 202. In other words, the optical based subsystem 201 may include at least one detection channel that is configured to detect light specularly reflected from the workpiece 202. Therefore, the optical based subsystems 201 described herein may be configured for only DF, only BF, or both DF and BF imaging. Although each of the collectors are shown in FIG. 3 as single refractive optical elements,it is to be understood that each of the collectors may include one or more refractive optical die(s) and / or one or more reflective optical element(s).

[0055] The one or more detection channels may include any suitable detectors known in the art. For example, the detectors may include photo-multiplier tubes (PMTs), charge coupled devices (CCDs), time delay integration (TDI) cameras, and any other suitable detectors known in the art. The detectors may also include non-imaging detectors or imaging detectors. In this manner, if the detectors are non-imaging detectors, each of the detectors may be configured to detect certain characteristics of the scattered light such as intensity but may not be configured to detect such characteristics as a function of position within the imaging plane. As such, the output that is generated by each of the detectors included in each of the detection channels of the optical based subsystem may be signals or data, but not image signals or image data. In such instances, a processor such as processor 214 may be configured to generate images of the workpiece 202 from the non-imaging output of the detectors. However, in other instances, the detectors may be configured as imaging detectors that are configured to generate imaging signals or image data. Therefore, the optical based subsystem may be configured to generate optical images or other optical based output described herein in a number of ways.

[0056] It is noted that FIG. 3 is provided herein to generally illustrate a configuration of an optical based subsystem 201 that may be included in the system embodiments described herein or that may generate optical based output that is used by the system embodiments described herein. The optical based subsystem 201 configuration described herein may be altered to optimize the performance of the optical based subsystem 201 as is normally performed when designing a commercial output acquisition system. In addition, the systems described herein may be implemented using an existing system (e.g., by adding functionality described herein to an existing system). For some such systems, the methods described herein may be provided as optional functionality of the system (e.g., in addition to other functionality of the system). Alternatively, the system described herein may be designed as a completely new system.

[0057] The processor 214 may be coupled to the components of the system 200 in any suitable manner (e.g., via one or more transmission media, which may include wired and / or wireless transmission media) such that the processor 214 can receive output. The processor 214 may beconfigured to perform a number of functions using the output. The system 200 can receive instructions or other information from the processor 214. The processor 214 and / or the electronic data storage unit 215 optionally may be in electronic communication with a wafer inspection tool, a wafer metrology tool, or a wafer review tool (not illustrated) to receive additional information or send instructions. For example, the processor 214 and / or the electronic data storage unit 215 can be in electronic communication with a scanning electron microscope.

[0058] The processor 214, other system(s), or other subsystem(s) described herein may be part of various systems, including a personal computer system, image computer, mainframe computer system, workstation, network appliance, internet appliance, or other device. The subsystem(s) or system(s) may also include any suitable processor known in the art, such as a parallel processor. In addition, the subsystem(s) or system(s) may include a platform with highspeed processing and software, either as a standalone or a networked tool.

[0059] The processor 214 and electronic data storage unit 215 may be disposed in or otherwise part of the system 200 or another device. In an example, the processor 214 and electronic data storage unit 215 may be part of a standalone control unit or in a centralized quality control unit. Multiple processors 214 or electronic data storage units 215 may be used.

[0060] The processor 214 may be implemented in practice by any combination of hardware, software, and firmware. Also, its functions as described herein may be performed by one unit, or divided up among different components, each of which may be implemented in turn by any combination of hardware, software and firmware. Program code or instructions for the processor 214 to implement various methods and functions may be stored in readable storage media, such as a memory in the electronic data storage unit 215 or other memory.

[0061] If the system 200 includes more than one processor 214, then the different subsystems may be coupled to each other such that images, data, information, instructions, etc. can be sent between the subsystems. For example, one subsystem may be coupled to additional subsystem(s) by any suitable transmission media, which may include any suitable wired and / or wireless transmission media known in the art. Two or more of such subsystems may also be effectively coupled by a shared computer-readable storage medium (not shown).

[0062] The processor 214 may be configured to perform a number of functions using the output of the system 200 or other output. For instance, the processor 214 may be configured to send the output to an electronic data storage unit 215 or another storage medium. The processor 214 may be configured according to any of the embodiments described herein. The processor 214 also may be configured to perform other functions or additional steps using the output of the system 200 or using images or data from other sources.

[0063] Program instructions implementing methods such as those described herein may be transmitted over or stored on carrier medium. The carrier medium may include a storage medium such as a read-only memory, a random access memory, a magnetic or optical disk, a non-volatile memory, a solid state memory, a magnetic tape, and the like. A carrier medium may include a transmission medium such as a wire, cable, or wireless transmission link. For instance, the various steps described throughout the present disclosure may be carried out by a single processor 214 or, alternatively, multiple processors 214. Moreover, different sub-systems of the system 200 may include one or more computing or logic systems. Therefore, the above description should not be interpreted as a limitation on the present disclosure but merely an illustration.

[0064] In an instance, the processor 214 is in communication with the system 200. The processor 214 is configured to operate a DNN that includes a backbone network and a plurality of head networks connected to a top layer as disclosed herein.

[0065] An additional embodiment relates to a non-transitory computer-readable medium storing program instructions executable on a controller for performing a computer-implemented method for inspection using a deep neural network that includes a backbone network and a plurality of head networks connected to a top layer, as disclosed herein. In particular, as shown in FIG. 3, electronic data storage unit 215 or other storage medium may contain non-transitory computer- readable medium that includes program instructions executable on the processor 214. The computer-implemented method may include any step(s) of any method(s) described herein, including the methods shown in FIGS. 1-2.

[0066] The program instructions may be implemented in any of various ways, including procedure-based techniques, component-based techniques, and / or object-oriented techniques, among others. For example, the program instructions may be implemented using ActiveX controls, C++objects, JavaBeans, Microsoft Foundation Classes (MFC), Streaming SIMD Extension (SSE), or other technologies or methodologies, as desired.

[0067] While disclosed with an inspection system and a semiconductor wafer, other systems and workpieces can benefit from the embodiments disclosed herein. For example, a review or metrology tool can use the embodiments disclosed herein. Other overlay measurement tools also can use the embodiments disclosed herein. The system can use an electron beam or an ion beam instead of a beam of light. The workpiece can be a flat panel, a PCB, or other substrate instead of a semiconductor wafer.

[0068] Each of the steps of the method may be performed as described herein. The methods also may include any other step(s) that can be performed by the processor and / or computer subsystem(s) or system(s) described herein. The steps can be performed by one or more computer systems, which may be configured according to any of the embodiments described herein. In addition, the methods described above may be performed by any of the system embodiments described herein.

[0069] Although the present disclosure has been described with respect to one or more particular embodiments, it will be understood that other embodiments of the present disclosure may be made without departing from the scope of the present disclosure. Hence, the present disclosure is deemed limited only by the appended claims and the reasonable interpretation thereof.

Claims

What is claimed is:

1. A system comprising: a light source that generates a beam of light; a stage configured to hold a workpiece in a path of the beam of light; a detector that receives the beam of light reflected from the workpiece; a processor in electronic communication with the detector, wherein the processor is configured to operate a deep neural network that includes a backbone network and a plurality of head networks connected to a top layer, wherein each of the head networks is configured for use during inspection of an image of the workpiece generated using data from the detector; and an electronic data storage unit in electronic communication with the processor, wherein the deep neural network is stored on the electronic data storage unit.

2. The system of claim 1, wherein each of the head networks is configured to perform image classification, image regression, object detection, dense segmentation, instance segmentation, or depth estimation.

3. The system of claim 1, wherein the processor is a GPU.

4. The system of claim 1, wherein the processor is a CPU.

5. The system of claim 1, wherein one or more of the head networks are configured to operate simultaneously.

6. The system of claim 1, wherein the deep neural network is configured to change between the plurality of head networks during the inspection of the image.

7. The system of claim 1, wherein the workpiece is a semiconductor wafer.

8. A method comprising: receiving, at a processor, an image of a workpiece; and inspecting the image using the processor, wherein the inspecting uses a deep neural network that includes a backbone network and a plurality of head networks connected to a top layer, and wherein each of the head networks is configured for use during inspection of the image.

9. The method of claim 8, further comprising: directing a beam of light at the workpiece; receiving the beam of light reflected from the workpiece at a detector; and communicating data from the detector to the processor to generate the image.

10. The method of claim 8, wherein each of the head networks is configured to perform image classification, image regression, object detection, dense segmentation, instance segmentation, or depth estimation.

11. The method of claim 8, wherein the processor is a GPU.

12. The method of claim 8, wherein the processor is a CPU.

13. The method of claim 8, wherein one or more of the head networks operate simultaneously.

14. The method of claim 8, wherein the deep neural network is configured to change between the plurality of head networks during the inspecting.

15. The method of claim 8, wherein the workpiece is a semiconductor wafer.

16. A non-transitory computer- readable storage medium, comprising one or more programs for executing the following steps on one or more computing devices: receiving an image of a workpiece; and inspecting the image using a deep neural network that includes a backbone network and a plurality of head networks connected to a top layer, and wherein each of the head networks is configured for use during inspection of the image.

17. The non-transitory computer-readable storage medium of claim 16, wherein each of the head networks is configured to perform image classification, image regression, object detection, dense segmentation, instance segmentation, or depth estimation.

18. The non-transitory computer-readable storage medium of claim 16, wherein one or more of the head networks are configured to operate simultaneously.

19. The non-transitory computer-readable storage medium of claim 16, wherein the deep neural network is configured to change between the plurality of head networks during the inspecting.

20. The non-transitory computer-readable storage medium of claim 16, wherein the workpiece is a semiconductor wafer.

Images