Dynamic design and inspection of adaptively patterned workpieces with deep learning-based rendering

Abstract

A reference optical image of the die is determined based on the design file using image-to-image transformation by a deep convolutional neural network. A difference image is generated by subtracting the reference optical image from the target image. The difference image can be binarized after applying a care area mask. The resulting binarized defect image can be used for optical inspection.

Description

[Technical Field]

[0001] This application claims priority to Indian Patent Application No. 202241056424, filed September 30, 2022, and U.S. Provisional Application No. 63 / 426803, filed November 21, 2022, the disclosures of which are incorporated herein by reference.

[0002] TECHNICAL FIELD This disclosure relates to optical inspection of workpieces, such as semiconductor wafers. [Background technology]

[0003] As the semiconductor manufacturing industry evolves, the demands on yield management, particularly for metrology and inspection systems, are increasing. As critical dimensions continue to shrink, the industry is still under pressure to reduce the time to achieve high yields and high-value production. Reducing the total time from detecting a yield problem to correcting it increases the return on investment for semiconductor manufacturers.

[0004] In the fabrication of semiconductor devices, such as logic and memory devices, workpieces, such as semiconductor wafers, are typically processed using a number of manufacturing processes to create various features and multiple layers in the semiconductor devices. For example, in a semiconductor manufacturing process known as lithography, a pattern is transferred from a reticle onto a photoresist array on the semiconductor wafer. Additional examples of semiconductor manufacturing processes include, but are not limited to, chemical mechanical polishing (CMP), etching, deposition, and ion implantation. An array of semiconductor devices may be fabricated on a single semiconductor wafer and then separated into individual semiconductor devices.

[0005] Inspection processes are used at various stages during semiconductor manufacturing, whereby detecting wafer-side defects can help increase yield and, therefore, profits in the manufacturing process. Inspection has always been an important part of semiconductor device, e.g., integrated circuit (IC) manufacturing. However, as semiconductor device dimensions shrink, inspection has become ever more important for the successful manufacture of acceptable semiconductor devices, since even smaller defects can cause device malfunctions. For example, as semiconductor device dimensions shrink, the detection of smaller defects becomes necessary, since even relatively small defects can cause unwanted errors in the semiconductor device.

[0006] Some workpieces contain dies with multiple chips attached. Some multi-chip die designs, such as those with system-in-package (SIP), film frame carrier (FFC), or 3D integrated circuits (3D-IC), can be difficult to inspect. Each die may be printed differently, limiting the ability to inspect by neighboring device comparison.

[0007] Existing techniques include adjacent die subtraction and reference-based subtraction. These techniques require that the images be structurally similar and aligned to one another. Running these techniques on a die with multiple chips can result in a large amount of nuisance. Suppressing such high levels of nuisance can be difficult. While alignment improvements can reduce nuisance, the manual alignment methods typically used for such improvements are lengthy and time-consuming. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] U.S. Patent No. 10,475,179 [Patent Document 2] International Publication No. 2007 / 120280 Summary of the Invention [Problem to be solved by the invention]

[0009] Therefore, there is a need for improved methods and systems. [Means for solving the problem]

[0010] A method is provided in a first embodiment. The method includes receiving, by a processor, a target image of a workpiece including a die having a plurality of chips attached thereto and a design file containing a design of the die. The processor is used to generate a reference optical image of the die based on the design file using image-to-image transformation using a deep convolutional neural network. The processor is used to generate a difference image by subtracting the reference optical image from the target image. The processor is used to generate a runtime care area mask for the die based on the design file containing the design of the die. The processor is used to generate a masked difference image by applying the runtime care area mask to the difference image. The processor is used to generate a binarized defect image by applying a threshold to the masked difference image. The die may include at least one SIP device, FFC, or 3D-IC.

[0011] The deep convolutional neural network can be a cycle generative adversarial network.

[0012] The design file may be a graphic design system file.

[0013] The method may involve aligning the target image and the reference optical image prior to subtraction.

[0014] The method may involve using the processor to extract a care area (region of interest) image from a target image using a run-time care area mask.

[0015] The method may comprise generating the target image using an optical inspection system.

[0016] A non-transitory computer readable medium having a program stored thereon may be configured to instruct a processor thereof to perform the method of the first embodiment.

[0017] A system is provided in a second embodiment. The system includes a light source generating a light beam, a stage configured to hold a workpiece in the path of the light beam, a detector configured to receive the light beam reflected from the workpiece, and a processor in electronic communication with the detector. The workpiece includes a die having a plurality of chips attached thereto. The processor is configured to generate a target image of the workpiece based on information from the detector, receive a design file containing a design of the die, generate a reference optical image of the die based on the design file using image-to-image transformation with a deep convolutional neural network, generate a difference image by subtracting the reference optical image from the target image, generate a runtime care area mask for the die based on the design file containing the design of the die, generate a masked difference image by applying the runtime care area mask to the difference image, and generate a binarized defect image by applying a threshold to the masked difference image. The die may include at least one SIP device, FFC, or 3D-IC.

[0018] The deep convolutional neural network can be a cycle generative adversarial network.

[0019] The design file may be a graphic design system file.

[0020] The processor may be further configured to align the target image and the reference optical image prior to subtraction.

[0021] The processor may be further configured to extract the care area image from the target image using the runtime care area mask.

[0022] For a more complete understanding of the nature and purpose of the present disclosure, reference should be made to the following detailed description taken in conjunction with the accompanying drawings, in which: [Brief explanation of the drawings]

[0023] [Figure 1] FIG. 1 is a diagram of an example SIP device. [Figure 2] 1 is a flowchart of a method according to the present disclosure. [Figure 3] FIG. 1 is a diagram of one embodiment of an optical inspection system according to the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0024] Although the claimed subject matter is described in terms of certain embodiments, other embodiments are within the scope of this disclosure, including embodiments that do not provide all of the benefits and features described herein. Various structural, logical, process step, and electronic changes may be made without departing from the scope of this disclosure. Accordingly, reference should be made solely to the appended claims for defining the scope of this disclosure.

[0025] With adaptive patterning, each die looks different when printed. In the disclosed embodiments, different design files are used for each die and workpiece, and deep learning methods are used for optical image rendering and care area alignment to detect defects on the workpiece. The alignment time constraint can be removed with physics-based optical image rendering using Cycle Generative Adversarial Networks (CycleGANs). According to these embodiments, dynamically loading design files and using deep learning-based rendering can help identify defects on workpieces with adaptive die patterns. Defect detection is enabled for adaptive die-patterned workpieces. The use of physics-based reference generation in conjunction with design files avoids a training step.

[0026] Figure 1 shows an example of a SIP device. Multiple chips are mounted within a die. Due to high-speed chip placement, translational and rotational offsets may occur as shown in Figure 1. As can be seen in Figure 1, chip A is positioned differently relative to chip B in die A, die B, and die C. The redistribution lines (RDLs) connecting these chips can be structurally adjusted based on this chip placement. These redistribution lines may be different between two dies.

[0027] The die-to-die pattern variations seen in Figure 1 cause problems during inspection. First, static care areas cannot be set during recipe generation. Second, the non-repetitive structure makes traditional die-to-die techniques that compare a target die to neighboring dies ineffective. For example, as seen in Figure 1, die A and die B are not the same. Because of the angle difference, it is not possible to depict a static care area that is common to both dies.

[0028] 2 is a flowchart of the method 100. Some or all of the steps in the method 100 may be performed using a processor.

[0029] A target image 101 is received. The target image 101 may be generated, for example, by an optical inspection system. The target image 101 may be of a workpiece including a die with multiple chips attached, such as a SIP device, an FFC device, a 3D-IC device, or other type of device. For ease of depiction, Figure 2 shows a generic SIP device.

[0030] A design file 102 corresponding to the die in the target image 101 is also received. The design file 102 contains information about the die layout, including planar geometry, text labels, and other information, in a hierarchical format. An example design file 102 is a graphic design system (GDS) file, such as a GDSII or OASIS file. A GDS graphic design system incorporates design information for a workpiece or die. While a die-level design may be present in the GDS for standard wafers, a workpiece-level design, i.e., a design with different die designs for each workpiece or at the die level, may be present in the GDS for workpieces with adaptive die patterning. The design file 102 may be a model die or a specific die that is essential for the method 100.

[0031] Using the design file 102 and image-to-image transformation with a deep convolutional neural network, a reference optical image 103 is generated. Image-to-image transformation maps input and output images so that the output image can be used to perform a specific task. For example, transforming an image from one domain to another allows learning of a mapping between the two images. Image-to-image transformation can be used for image synthesis, noise reduction, and other purposes. A cycle generative adversarial network can be used to perform these tasks. A separate reference optical image 103 can be generated for each die. Examples of cycle generative adversarial networks include CycleGAN, Pix2Pix, and other types of generative adversarial network models. Pix2Pix uses aligned and paired data for training. CycleGAN can be trained with an unpaired dataset. Pairing means that the source and target domain images should be in the same location and there should be the same number of images in each domain.

[0032] Pix2Pix is ​​a generative adversarial network (GAN) model designed for general-purpose image-to-image translation. The Pix2Pix model is a conditional GAN: the generation of the output image is conditional on the input, e.g., the source image. The discriminator is fed a source image and a target image and determines whether the target is the most likely translation of the source image. The generator is trained using an adversarial loss to encourage it to generate the most likely image in the target domain. Pix2PixGAN can be used to perform image-to-image translation tasks, such as generating an image based on another image (e.g., a detailed image based on a sketch, design, or blueprint).

[0033] CycleGAN is an image-to-image translation model similar to Pix2Pix. Unlike Pix2Pix, CycleGAN uses unpaired datasets to learn the mapping between input and output images. Like Pix2Pix, CycleGAN can generate an image based on another image (e.g., a detailed image based on a sketch, design, or blueprint).

[0034] Deep learning, which has its roots in neural network technology, is a probabilistic graph model with many layers of neurons, commonly known as deep architecture. Deep learning technology processes information such as images, text, and audio in a hierarchical manner. When using deep learning in the present disclosure, feature extraction is accomplished automatically by learning from data. For example, defects can be classified, sorted, or binned using a deep learning classification module based on one or more extracted features.

[0035] Broadly speaking, deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model higher-level abstractions in data. In a simple case, there might be two sets of neurons, one set that receives input signals and one set that sends out output signals. The input layer receives an input and passes a modified version of that input to the next layer. Because deep networks can have many layers between the input and output, the algorithms can use multiple processing layers, consisting of multiple linear and nonlinear transformations.

[0036] Deep learning is a member of a broad family of machine learning methods that are based on learning data representations. Observables (e.g., features to be extracted for reference) can be represented in many ways, such as vectors of pixel-by-pixel intensity values, or in more abstract ways, such as collections of edges, regions of specific shapes, etc. Some representations are better at simplifying the learning task (e.g., face recognition or facial expression recognition) than others. Deep learning can provide efficient algorithms for unsupervised or semi-supervised feature learning and hierarchical feature extraction.

[0037] In one embodiment, the deep learning model is configured as a neural network. In a further embodiment, the deep learning model can be a deep neural network with a set of weights that model the world according to the data fed to it to train it. A neural network can generally be defined as an information processing method based on a relatively large population of neural units that loosely model the way a biological brain solves problems with a relatively large cluster of axon-connected biological neurons. Each neural unit can be connected to many other neural units, and their effect on the activation state of the connected neural units can be forced or inhibited by the links. Such systems are self-learning, trained rather than explicitly programmed, and excel at solutions or feature detection strategies that are difficult to express in traditional computer programs.

[0038] Neural networks typically have multiple layers, with signal paths running from front to back. The goal of a neural network is to solve problems in the same way that a human brain would, although some neural networks are much more abstract than that. Modern neural network projects typically operate with thousands to millions of neural units and millions of connections. Neural networks can have any suitable architecture and / or configuration known in the art.

[0039] GANs are responsible for generative modeling using deep learning methods, such as convolutional neural networks. Generative modeling is an unsupervised learning task in machine learning that involves automatically discovering and learning regularities or patterns in input data so that the model can be used to generate or output new examples that are plausible as determined by the original dataset.

[0040] Training a generative model in a GAN is done by framing the problem as a supervised learning problem with two sub-models. First, there is a generator model that is trained to generate new examples. Second, there is a discriminator model that attempts to classify examples as true (originating from the domain) or false (generated). These two models are trained together in a zero-sum game (i.e., adversarial) until the discriminator model is sufficiently fooled into thinking that the generator model is generating the most likely examples.

[0041] Areas of the design file 102 that correspond to the optical image 101 can be used in the method 100. Features of the die will be present within both the design file 102 and the optical image 101.

[0042] A deep convolutional neural network can be trained to map images in the design file 102 to the target image 101 and generate the reference optical image 103. This training typically does not require perfect alignment; rough locations in the design file 102 and target image 101 are generally sufficient to train the network parameters. At runtime, the corresponding GDS portion of the target die is retrieved from the design file 102 and transformed into the reference optical image 103 using the model trained during recipe generation. The reference optical image 103 can be used as a reference image, aligned, and compared to the target image 101 to detect anomalies. Manual alignment during setup and between the target image 101 and the reference optical image 103 can be avoided.

[0043] Using an infrastructure with high performance information processing accelerators (e.g., graphics processing units (GPUs)), the network can be trained with example design clips and images.

[0044] The reference optical image 103 can be generated from the design file 102 in real time or in any other manner as needed.

[0045] The reference optical image 103 is then subtracted from the target image 101 using image subtraction, which produces a difference image 104. This image subtraction can be performed using the same neural network, a different neural network, or a conventional processor. As shown in Figure 2, the difference image 104 produced by the image subtraction contains subtle differences between the target image 101 and the reference optical image 103.

[0046] Optionally, prior to image subtraction, the target image 101 can be aligned to the reference optical image 103, not necessarily based on the reference optical image 103 being generated.

[0047] Optionally, the design file 102 can be used to generate a runtime care area mask 105. The runtime care area mask 105 incorporates the design of the die in the target image 101. The runtime care area mask 105 can also be used to exclude noisy areas or areas not of interest for inspection. A separate runtime care area mask 105 can be generated for each die.

[0048] Optionally, a runtime care area mask 105 is applied to the difference image 104 to generate a masked difference image 106. The masked difference image 106 can mask (or exclude) portions of the difference image 104, such as areas not of interest for inspection, areas where defects are rare, or areas known to lead to false positives during inspection.

[0049] A threshold can be applied to the masked difference image 106. This step produces a binarized defect image 107, which can be used for optical inspection. Pixels in the binarized defect image can be of two colors, e.g., black and white. A threshold can be used to determine which pixels represent one of the two colors. Post-processing, such as filtering and merging potential defective pixels in the binarized defect image 107, can provide defect information for all sites.

[0050] Optionally, a runtime care area mask 105 can be used to extract a care area image from the target image 101. The resulting image can be used for optical inspection and comparison with a binarized defect image 107. The runtime care area mask 105 can also be applied prior to one or both of the images used in the image subtraction to produce the difference image 104.

[0051] In one example, a recipe is generated by the user using layer design information and global alignment. A deep convolutional neural network is used to learn network parameters and render a reference optical image.

[0052] At runtime, design files are dynamically loaded for each workpiece, and an optical reference image is rendered using a deep convolutional neural network, utilizing high-performance configurations of computing nodes or central processing unit (CPU) clusters. This rendered optical reference image is used to update the design-based care areas into the runtime optical image, and also directly for anomaly detection.

[0053] According to one embodiment, a deep learning based classifier can reduce nuisances and isolate defects of interest (DOIs) in the binarized defect image 107.

[0054] One embodiment of an optical inspection system 200 is shown in FIG. 3. The system 200 includes an optical-based subsystem 201. Generally, the optical-based subsystem 201 is configured to generate an optical-based output for the sample 202 by directing light at (or scanning light over) the sample 202 and to detect light from the sample 202. In one embodiment, the sample 202 includes a wafer. The wafer may include any wafer known in the art, such as those used in method 100. In another embodiment, the sample 202 includes a reticle. The reticle may include any reticle known in the art.

[0055] In the embodiment of system 200 shown in FIG. 3, optically-based subsystem 201 includes an illumination subsystem configured to direct light toward sample 202. The illumination subsystem includes at least one light source. For example, the illumination subsystem shown in FIG. 3 includes light source 203. In some embodiments, the illumination subsystem is configured to direct light toward sample 202 at one or more angles of incidence, including one or more oblique angles and / or one or more perpendicular angles. For example, as shown in FIG. 3, light from light source 203 is directed first through optical element 204 and then through lens 205 toward sample 202 at an oblique angle of incidence. The oblique angle of incidence can include any suitable oblique angle of incidence and can vary depending, for example, on the characteristics of sample 202.

[0056] The optical-based subsystem 201 can be configured to direct light to the sample 202 at different angles of incidence at different times. For example, the optical-based subsystem 201 can be configured to change one or more properties of one or more elements of its illumination subsystem so that it can direct light to the sample 202 at angles of incidence different from that shown in Figure 3. In such examples, the optical-based subsystem 201 can be configured to move the light source 203, optical element 204, and lens 205 to direct light to the sample 202 at different oblique or orthogonal (or near-orthogonal) angles of incidence.

[0057] According to certain examples, the optically-based subsystem 201 can be configured to simultaneously direct light at multiple angles of incidence toward the sample 202. For example, the illumination subsystem can include multiple illumination channels, one of which can include a light source 203, optical element 204, and lens 205, as shown in FIG. 3, and another of which (not shown) can include similar elements, which may be different, similar, or include at least one light source and possibly one or more other components, such as those described herein. When such light is simultaneously directed toward the sample with other light, one or more characteristics (e.g., wavelength, polarization, etc.) of the light directed toward the sample 202 at the different angles of incidence can be different so that the light resulting from illumination of the sample 202 at the different angles of incidence can be distinguished from one another by a detector(s).

[0058] In another example, the illumination subsystem can include a single light source (e.g., light source 203 shown in FIG. 3 ), and light from the light source can be separated into separate optical paths (e.g., based on wavelength, polarization, etc.) by one or more optical elements (not shown) in the illumination subsystem. The light from each of the separate optical paths can then be directed toward the sample 202. Multiple illumination channels can be configured to direct light toward the sample 202 at different times or simultaneously (e.g., when sequentially illuminating the sample using separate illumination channels). In another example, the same illumination channel can be configured to direct light with different characteristics toward the sample 202 at different times. For example, in certain examples, the optical element 204 can be configured as a spectral filter, and the characteristics of the spectral filter can be changed in various ways (e.g., by retracting the spectral filter), thereby allowing different wavelengths of light to be directed toward the sample 202 at different times. The illumination subsystem may have any other configuration known in the art suitable for directing light having different or the same characteristics at different or the same angles of incidence, either sequentially or simultaneously.

[0059] According to certain embodiments, light source 203 may comprise a broadband plasma (BBP) light source, thereby allowing the light generated by light source 203 and directed toward sample 202 to include broadband light. However, light source 203 may comprise any other suitable light source, such as a laser. The laser may include any laser known in the art and may be configured to generate light at any suitable wavelength or wavelength group known in the art. Additionally, the laser may be configured to generate monochromatic or near-monochromatic light, thereby allowing the laser to be a narrowband laser. Light source 203 may also comprise a polychromatic light source that generates light at multiple discrete wavelengths or wavebands.

[0060] Light from optical element 204 can be focused onto sample 202 by lens 205. While lens 205 is depicted in FIG. 3 as a single refractive optical element, it is understood that lens 205 can actually include multiple refractive and / or reflective optical elements that cooperate to focus light from the optical element onto the sample. The illumination subsystem depicted in FIG. 3 and described herein can include any other suitable optical elements (not shown). Examples of such optical elements include, but are not limited to, polarizing element(s), spectral filter(s), spatial filter(s), reflective optical element(s), apodizer(s), beamsplitter(s) (e.g., beamsplitter 213), aperture(s), and the like, which can include any type of suitable optical element known in the art. Additionally, optical-based subsystem 201 can be configured to vary one or more of the elements of the illumination subsystem based on the type of illumination to be used to generate the optical-based output.

[0061] The optical-based subsystem 201 can also include a scanning subsystem configured to scan the light across the sample 202. For example, the optical-based subsystem 201 can include a stage 206 on which the sample 202 rests during optical-based output generation. Any suitable mechanical and / or robotic assembly (including the stage 206) can be included within the scanning subsystem and configured to move the sample 202, thereby enabling the light to be scanned across the sample 202. Additionally or alternatively, the optical-based subsystem 201 can be configured such that one or more optical elements of the optical-based subsystem 201 perform some optical scanning of the sample 202. The light can be scanned across the sample 202 in any suitable manner, such as along a serpentine or spiral path.

[0062] The optical-based subsystem 201 further includes one or more detection channels. At least one of the detection channels includes a detector configured to detect light emitted from the sample 202 resulting from illumination of the sample 202 by the illumination subsystem and to generate an output responsive to the detected light. For example, FIG. 3 shows the optical-based subsystem 201 with two detection channels: one formed by collector 207, element 208, and detector 209, and the other formed by collector 210, element 211, and detector 212. As shown in FIG. 3, the two detection channels are configured to collect and detect light at different collection angles. In some examples, both detection channels are configured to detect scattered light, and the detection channels are configured to detect light scattered from the sample 202 at different angles. Alternatively, one or more of the detection channels can be configured to detect a different type of light (e.g., reflected light) from the sample 202.

[0063] As detailed in FIG. 3 , both detection channels are located within the plane of the paper, and the illumination subsystem is also located within the plane of the paper. Thus, in this embodiment, both detection channels are located within (e.g., centered within) the plane of incidence. Alternatively, one or more of the detection channels can be located outside the plane of incidence. For example, the detection channel formed by collector 210, element 211, and detector 212 can be configured to collect and detect light scattered outside the plane of incidence. Therefore, such detection channels can be referred to as "side" channels, and can be centered within a plane substantially perpendicular to the plane of incidence.

[0064] While the embodiment shown in FIG. 3 includes two detection channels in the optical-based subsystem 201, the optical-based subsystem 201 may include any number of detection channels (e.g., a single detection channel or more than one detection channel). In some such examples, the detection channel formed by collector 210, element 211, and detector 212 may form one side channel as described above, and an additional detection channel (not shown) within the optical-based subsystem 201 may form another side channel located on the opposite side of the plane of incidence. Thus, in the optical-based subsystem 201, a detection channel including collector 207, element 208, and detector 209 may be centered in the plane of incidence and configured to collect and detect light at scattering angles perpendicular to or near the surface of sample 202. This detection channel may therefore be referred to as the "top" channel, and more than one side channel configured as described above may be included within the optical-based subsystem 201. Thus, the optical-based subsystem 201 can have at least three channels (i.e., one top channel and two side channels), each of the at least three channels having its own concentrator, and each concentrator configured to collect light at a different scattering angle than any of the other concentrators.

[0065] As detailed above, each detection channel within the optical-based subsystem 201 can be configured to detect scattered light. Accordingly, the optical-based subsystem 201 shown in FIG. 3 can be configured to generate a dark-field (DF) output for the sample 202. However, alternatively or additionally, the optical-based subsystem 201 can include a detection channel(s) configured to generate a bright-field (BF) output for the sample 202. In other words, the optical-based subsystem 201 can include at least one detection channel configured to detect light specularly reflected from the sample 202. Accordingly, the optical-based subsystem 201 described herein can be configured for DF-only imaging, BF-only imaging, or both DF and BF imaging. While each collector is shown in FIG. 3 as a single refractive optical element, it will be appreciated that each collector can also include one or more refractive optical dies and / or one or more reflective optical elements.

[0066] The one or more detection channels may include any suitable detector known in the art. For example, the detectors may include photomultiplier tubes (PMTs), charge-coupled devices (CCDs), time delay integration (TDI) cameras, or any other suitable detector known in the art. The detectors may include non-imaging and imaging detectors. In this manner, non-imaging detectors may be configured to detect a particular characteristic of scattered light, such as intensity, but not detect that characteristic as a function of position in the imaging plane. This allows the output generated by each detector in each detection channel of the optical-based subsystem to be a signal or data, but not an image signal or image data. In such an example, a processor, such as processor 214, may be configured to generate an image of sample 202 from the non-imaging output of the detector. However, in other examples, the detectors may be configured as imaging detectors configured to generate imaging signals or image data. Thus, depending on the configuration of the optical-based subsystem, optical images and other optical-based outputs described herein may be generated in a variety of ways.

[0067] FIG. 3 is provided herein to generally illustrate optical-based subsystem 201 configurations that may be included in or that may generate optical-based outputs used by system embodiments described herein. The optical-based subsystem 201 configurations described herein may be modified to optimize the performance of the optical-based subsystem 201, as is commonly done when designing commercial power capture systems. Additionally, the systems described herein may be implemented using existing systems (e.g., by adding the described functionality to an existing system). In such systems, the described methods may be provided as optional features of the system (e.g., in addition to other features of other systems). Alternatively, the systems described herein may be designed as entirely new systems.

[0068] Processor 214 may be coupled to components of system 200 in any suitable manner (e.g., via one or more types of transmission media, including wired and / or wireless transmission media) to receive output from processor 214. Processor 214 may be configured to perform multiple functions using the output. System 200 may receive instructions and other information from processor 214. Optionally, processor 214 and / or electronic data storage unit 215 may be in electronic communication with a wafer inspection tool, wafer metrology tool, or wafer review tool (not shown) to receive additional information or send instructions. For example, processor 214 and / or data storage unit 215 may be in electronic communication with a scanning electron microscope.

[0069] The processor 214, other system(s), or other subsystem(s) described herein can be part of a variety of systems, including personal computer systems, image computers, mainframe computer systems, workstations, network appliances, internet appliances, and other devices. The subsystem(s) or system(s) can incorporate any suitable processor, e.g., parallel processor, known in the art. Additionally, the system(s) or system(s) can incorporate high-speed processing platforms and software, either as standalone or networked tools.

[0070] The processor 214 and electronic data storage unit 215 may be part of, e.g., located within, the system 200 or other device. By way of example, the processor 214 and electronic data storage unit 215 may be part of a stand-alone control unit or may be located within a centralized quality control unit. Multiple processors 214 or electronic data storage units 215 may also be used.

[0071] Processor 214 may be implemented using any combination of hardware, software, and firmware. Furthermore, some of its functions described herein may be performed by a single unit or distributed among separate components, each of which may in turn be implemented using any combination of hardware, software, and firmware. Program code or instructions for causing processor 214 to implement various methods and functions may be stored on a readable storage medium, such as memory within electronic data storage unit 215 or other memory.

[0072] If there are multiple processors 214 in system 200, the separate subsystems may be coupled together to pass images, data, information, instructions, etc. between them. For example, one subsystem may be coupled to another subsystem(s) by any suitable transmission medium, including any suitable wired and / or wireless transmission medium known in the art. Two or more of these subsystems may also be operatively coupled by a shared computer-readable storage medium (not shown).

[0073] Processor 214 may be configured to perform multiple functions using the output of system 200 or other outputs. For example, processor 214 may be configured to send output to electronic data storage unit 215 or other storage medium. Processor 214 may be configured according to any of the embodiments described herein. Processor 214 may also be configured to perform other functions or additional steps using the output of system 200 or using images or data from other sources.

[0074] The various steps, functions, and / or operations of the system 200 and methods disclosed herein may be performed by one or more of electronic circuits, logic gates, multiplexers, programmable logic devices, ASICs, analog or digital controls / switches, microcontrollers, and information processing systems. Program instructions for implementing methods such as those described herein may be transmitted on or stored on a carrier medium. The carrier medium may include storage media such as read-only memory, random-access memory, magnetic or optical disks, non-volatile memory, solid-state memory, magnetic tape, and the like. The carrier medium may include transmission media such as wires, cables, wireless transmission links, and the like. For example, the various steps described throughout this disclosure may be performed by a single processor 214 or, alternatively, by multiple processors 214. Furthermore, there may be one or more information processing or logic systems within the various subsystems of the system 200. Accordingly, the foregoing description should be construed as merely illustrative and not limiting on the present disclosure.

[0075] In one example, processor 214 is in communication with system 200. Processor 214 is configured to generate a target image of the workpiece based on information from detector 209 and / or detector 212. At least one SIP device, FFC device, or 3D-IC device can be provided within the die. Processor 214 also receives a design file containing a design of the die. The design file can be a graphic design system file. A reference optical image of the die is generated by processor 214 based on the design file using image-to-image transformation with a deep convolutional neural network. The deep convolutional neural network can be a CycleGAN. Processor 214 is also configured to generate a difference image by subtracting the reference optical image from the target image, generate a runtime care area mask for the die based on the design file containing the design of the die, apply the runtime care area mask to the difference image to generate a masked difference image, and apply a threshold to the masked difference image to generate a binarized defect image.

[0076] The processor may be further configured to align the target image and the reference optical image prior to subtraction.

[0077] The processor may be further configured to extract the care area image from the target image using the runtime care area mask.

[0078] An additional embodiment relates to a non-transitory computer-readable medium having stored thereon program instructions executable on a controller for performing a computer-implemented method for testing as disclosed herein. Specifically, as shown in FIG. 3, a non-transitory computer-readable medium having program instructions executable on a processor 214 may be included in an electronic data storage unit 215 or other storage medium. The computer-implemented method may incorporate any step(s) of any method(s) described herein, including method 100.

[0079] The program instructions may be implemented in any of a variety of ways, including procedure-based, component-based, and / or object-oriented techniques, among others. For example, the program instructions may be implemented using ActiveX controls, C++ objects, JavaBeans, Microsoft Foundation Classes (MFC), Streaming SIMD Extensions (SSE), or any other technology or methodology, as desired.

[0080] Each step in the method may be performed as described herein. The methods may also include any other step(s) that may be performed by a processor and / or computer subsystem(s) or system(s) described herein. The steps may be performed by one or more computer systems, which may be configured according to any of the embodiments described herein. In addition, the methods described above may be performed by any of the system embodiments described herein.

[0081] Although the present disclosure has been described in connection with one or more specific embodiments, other embodiments of the present disclosure may be made without departing from the spirit and scope of the present disclosure, which is to be construed as limited only by the appended claims and their reasonable interpretation.

Claims

1. 1. A method comprising: receiving, at a processor, a target image of a workpiece including a die having a plurality of chips associated therewith; receiving, at the processor, a design file containing a design for the die; generating a reference optical image of the die based on the design file using an image-to-image transformation with a deep convolutional neural network using the processor; generating a difference image by subtracting the reference optical image from the target image using the processor; generating, with the processor, a runtime care area mask for the die based on the design file containing the design of the die; generating a masked difference image by applying the runtime care area mask to the difference image with the processor; and generating a binarized defect image by applying a threshold to the masked difference image using the processor; method.

2. 10. The method of claim 1, wherein the die comprises at least one system-in-package device.

3. 10. The method of claim 1, wherein the die comprises a film frame carrier.

4. 10. The method of claim 1, wherein the die comprises a 3D integrated circuit.

5. 2. The method of claim 1, wherein the deep convolutional neural network is a cycle generative adversarial network.

6. 10. The method of claim 1, wherein the design file is a graphic design system file.

7. 10. The method of claim 1, further comprising aligning the target image and the reference optical image prior to said subtraction.

8. The method of claim 1 , further comprising: using the processor to extract a care area image from the target image using the runtime care area mask.

9. 10. The method of claim 1, further comprising generating the target image using an optical inspection system.

10. A non-transitory computer-readable medium having stored thereon a program configured to instruct the processor to perform the method of claim 1.

11. 1. A system comprising: a light source for generating a light beam; a stage configured to hold a workpiece including a die having a plurality of chips attached thereto in the path of the light beam; a detector configured to receive the light beam reflected from the workpiece; a processor in electronic communication with the detector; wherein the processor: generating a target image of the workpiece based on information from the detector; receiving a design file containing a design for the die; generating a reference optical image of the die based on the design file by image-to-image transformation using a deep convolutional neural network; generating a difference image by subtracting the reference optical image from the target image; generating a runtime care area mask for the die based on the design file containing the design of the die; generating a masked difference image by applying the runtime care area mask to the difference image; and generating a binarized defect image by applying a threshold to the masked difference image; The system is configured as follows:

12. 12. The system of claim 11, wherein the die comprises at least one of a system-in-package device, a film frame carrier, or a 3D integrated circuit.

13. 12. The system of claim 11, wherein the deep convolutional neural network is a cycle generative adversarial network.

14. 12. The system of claim 11, wherein the design file is a graphic design system file.

15. 12. The system of claim 11, wherein the processor is further configured to align the target image and the reference optical image prior to the subtraction.

16. 12. The system of claim 11, wherein the processor is further configured to extract a care area image from the target image using the runtime care area mask.

Images