Fully automated SEM sampling system for e-beam image enhancement

Abstract

Disclosed herein is a method of automatically acquiring training images to train a machine learning model that improves image quality. The method can include analyzing a plurality of patterns of data related to a layout of a product to identify a plurality of training locations on a sample of the product for use in connection with training a machine learning model. The method can include acquiring, for each of the plurality of training locations, a first image having a first quality and acquiring, for each of the plurality of training locations, a second image having a second quality, the second quality being higher than the first quality. The method can include training the machine learning model using the first images and the second images.

Description

[0001] Cross Reference to Related Applications

[0002] This application claims priority to U.S. Application 62 / 787,031, filed December 31, 2018, which is incorporated by reference herein in its entirety. TECHNICAL FIELD

[0003] The present disclosure relates generally to systems and image enhancement methods for image acquisition, and more particularly to systems and methods for improving metrology by automatically acquiring training images to train machine learning models that improve image quality. BACKGROUND

[0004] In manufacturing processes for manufacturing integrated circuits (ICs), unfinished or finished circuit components are inspected to ensure that they are manufactured according to design and are free of defects. Inspection systems can be employed that utilize optical microscopes or charged particle (e.g., electron) beam microscopes, such as scanning electron microscopes (SEMs). As physical dimensions of IC elements continue to shrink, accuracy of defect detection and yield become increasingly important. However, imaging resolution and throughput of detection tools are difficult to keep pace with the ever-shrinking feature sizes of IC components. Further improvements are needed in the art. SUMMARY

[0005] The following presents a simplified summary of one or more aspects in order to provide a basic understanding of such aspects. This summary is not an extensive overview of all contemplated aspects, and is intended to neither identify key or critical elements of all aspects nor delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that is presented later.

[0006] In one aspect of the present disclosure, a method of automatically acquiring training images to train a machine learning model is provided. The method can include analyzing a plurality of patterns of data related to a layout of a product to identify a plurality of training locations on a sample of the product for use in relation to training the machine learning model. The method can include acquiring, for each of the plurality of training locations, a first image having a first quality, and acquiring, for each of the plurality of training locations, a second image having a second quality, the second quality being higher than the first quality. The method can include training the machine learning model using the first images and the second images.

[0007] In another aspect of the disclosure, an apparatus for automatically acquiring training images to train a machine learning model is provided. The apparatus can include a memory and one or more processors coupled to the memory. The processors can be configured to analyze a plurality of patterns of data related to a layout of a product to identify a plurality of training locations on a sample of the product for use in relation to training a machine learning model. The processors can be further configured to acquire, for each of the plurality of training locations, a first image having a first quality and acquire, for each of the plurality of training locations, a second image having a second quality, the second quality being higher than the first quality. The processors can be further configured to train the machine learning model using the first images and the second images.

[0008] In another aspect of the disclosure, a non-transitory computer-readable medium storing a set of instructions is provided, the set of instructions executable by a controller of an apparatus to cause the apparatus to perform a method comprising: analyzing a plurality of patterns of data related to a layout of a product to identify a plurality of training locations on a sample of the product for use in relation to training a machine learning model; acquiring, for each of the plurality of training locations, a first image having a first quality; acquiring, for each of the plurality of training locations, a second image having a second quality, the second quality being higher than the first quality; and training the machine learning model using the first images and the second images.

[0009] In another aspect of the disclosure, an electron beam inspection apparatus is provided, the electron beam inspection apparatus comprising a controller having circuitry to cause the electron beam inspection apparatus to: analyze a plurality of patterns of data related to a layout of a product to identify a plurality of training locations on a sample of the product for use in relation to training a machine learning model; acquire, for each of the plurality of training locations, a first image having a first quality; acquire, for each of the plurality of training locations, a second image having a second quality, the second quality being higher than the first quality; and train the machine learning model using the first images and the second images.

[0010] To the accomplishment of the foregoing and related ends, the aspects comprise the features hereinafter fully described and particularly pointed out in the claims. The following description and the annexed drawings set forth in detail certain illustrative features of the one or more aspects. These features are indicative, however, of but a few of the various ways in which the principles of various aspects can be employed. This description is intended to include all such aspects and their equivalents. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 is a flowchart of a process for improving images acquired by a SEM sampling system.

[0012] Figure 2is a block diagram illustrating an example of an automated SEM sampling system in accordance with some aspects of the present disclosure.

[0013] Figure 3 is a schematic diagram illustrating an example of an electron beam inspection (EBI) system in accordance with some aspects of the present disclosure.

[0014] Figure 4 is a schematic diagram illustrating an example of an electron beam tool that can be part of an example electron beam inspection (EBI) system in accordance with some aspects of the present disclosure. Figure 3

[0015] Figures 5A-5C illustrates multiple design patterns of a graphic database system (GDS) of a product in accordance with some aspects of the present disclosure.

[0016] Figure 5D is a graph illustrating multiple training locations in accordance with some aspects of the present disclosure.

[0017] Figure 6A is a flow diagram illustrating an example of a method of automatically acquiring training images to train a machine learning model in accordance with some aspects of the present disclosure.

[0018] Figure 6B is a block diagram illustrating details of an automated SEM sampling system in accordance with some aspects of the present disclosure.

[0019] Figure 7 is a block diagram illustrating an example of a method of automatically acquiring training images to train a machine learning model to improve image quality in accordance with some aspects of the present disclosure. DETAILED DESCRIPTION

[0020] Reference will now be made in detail to example aspects of the embodiments, examples of which are illustrated in the accompanying drawings. The following description refers to the accompanying drawings, in which the same numbers represent the same or similar elements unless otherwise represented. The implementation set forth in the following description of example aspects of the embodiments does not represent all of the implementations consistent with the present invention. Instead, they are merely examples of apparatuses and methods consistent with some aspects related to the present invention as set forth in the claims. For example, although some aspects of the embodiments are described in the context of utilizing an electron beam inspection (EBI) system, such as a scanning electron microscope (SEM), to generate images of a wafer, the present disclosure is not so limited. Other types of inspection systems and image generation systems are similarly applied.

[0021] ​Increasing the computing power of electronic devices while reducing the physical size of the devices can be achieved by significantly increasing the packing density of circuit components such as transistors, capacitors, diodes, etc. on IC chips. For example, in a smartphone, an IC chip (the size of a thumbnail) can include over 2 billion transistors, each smaller than 1 / 1000th the size of a human hair. Not surprisingly, semiconductor IC fabrication is a complex process with hundreds of individual steps. Even a single error in one step can significantly impact the functionality of the final product. Even a single “killer defect” can cause the device to fail. The goal of the manufacturing process is to improve the overall yield of the process. For example, for a 50-step process to achieve a yield of 75%, the yield of each individual step must be greater than 99.4%, whereas if the yield of the individual steps is 95%, the overall process yield will drop to 7%.

[0022] At various steps in the semiconductor manufacturing process, pattern defects can occur on at least one of the wafer, chip, or mask, and can cause the manufactured semiconductor device to fail, thereby greatly reducing the yield. As semiconductor device sizes (along with any defects) continue to become smaller and smaller, identifying defects becomes more challenging and costly. Currently, engineers of semiconductor production lines often spend hours (sometimes even days) to identify the location of small defects to minimize their impact on the final product.

[0023] Conventional optical inspection techniques are ineffective in inspecting small defects (e.g., nanometer-scale defects). Advanced electron beam inspection (EBI) tools, such as SEMs with high resolution and large depth of focus, have been developed to meet the needs of the semiconductor industry. Electron beam images can be used to monitor the semiconductor manufacturing process, particularly for higher nodes where optical detection cannot provide sufficient information.

[0024] Electron beam images can be characterized according to one or more quantities such as contrast, brightness, noise level, etc. Generally speaking, low-quality images typically require less parameter tuning and less scanning, but the information (such as defect type and location) embedded in low-quality images is difficult to extract, which can negatively impact the analysis. High-quality images can be acquired by increasing the number of scans, which do not suffer from this problem. However, high-quality images can have low throughput.

[0025] Furthermore, the electron beam image acquisition process can go through many steps such as identifying the pattern of interest, setting the scan area for inspection, tuning the SEM conditions, determining the quality enhancement method, etc. Many of these settings and parameters are factors that affect the system throughput and the quality of the electron beam images. There can be a tradeoff between throughput and image quality.

[0026] To acquire high quality images and at the same time achieve high throughput, an operator often needs to set many parameters and decide how the images should be acquired. However, determining these parameters is often not simple. To minimize possible inter-operator variations, a machine learning based enhancement method can be trained to learn an enhancement framework / network. In this case, acquiring sufficient and representative training samples is beneficial to improve the final performance of the trained system. However, the common procedure to acquire SEM image samples requires a lot of human intervention, including searching for appropriate design patterns for scanning, determining the number of images to collect and various imaging conditions, etc. This intensive human involvement hinders the full utilization of advanced machine learning based enhancement methods. Therefore, there is a need to develop a fully automated intelligent sampling system for e-beam image quality enhancement.

[0027] Disclosed herein, among other things, is an apparatus for automatically acquiring training images to train a machine learning model to improve image quality, and a method used by the apparatus. E-beam imaging plays an important role in inspecting very small defects (e.g., nanometer scale defects of 0.000000001 meters) in semiconductor manufacturing processes. Typically, a large number of e-beam images of relatively low quality can be acquired very quickly, but the images can not provide sufficient useful information about issues such as defect type and location. On the other hand, high quality images can be acquired, but doing so takes more time, thus reducing the speed at which the apparatus can be analyzed. This increases manufacturing costs. A sample can be scanned multiple times to improve image quality, such as by averaging multiple images to reduce noise. However, scanning each sample multiple times reduces system throughput and can cause charge accumulation or can damage the sample.

[0028] Some systems and methods disclosed herein embody ways to acquire high quality images by reducing the number of scans of a sample (in some embodiments, by automatically selecting sample locations to use to generate training images for a machine learning (ML) algorithm). The term “quality” refers to resolution, contrast, sensitivity, brightness, or noise level, among others. Some systems and methods disclosed herein can acquire the benefits of higher quality images without slowing production excessively. Some embodiments of the system can automatically analyze multiple patterns of data related to a layout of a product to identify multiple training locations on a sample of the product to use in relation to training a machine learning model. The data patterns can be SEM images of the product, or layout designs (e.g., Graphic Database System (GDS), Open Artwork System Interchange Standard (OASIS), Caltech Intermediate Format (CIF), etc.) representations of the product. For example, a GDS analyzer can be used to determine good locations on a sample to use for training a ML algorithm. These locations are scanned multiple times, resulting in image assets, where some images are progressively improved with each scan, such as due to noise averaging or using higher resolution settings on the SEM. These images (e.g., lower quality images and related higher quality images) are used as training samples to train a ML algorithm. The number of scans of other locations on the sample is reduced, and the ML algorithm modifies images to approximate how the images would be progressively improved by additional scans or by scans using higher resolution settings.

[0029] The present disclosure provides a method of automatically acquiring training images to train a machine learning model that improves image quality. The method can include analyzing multiple patterns of data related to a layout of a product to identify multiple training locations on a sample of the product to use in relation to training a machine learning model. For example, the method can include analyzing multiple patterns from a Graphic Database System (GDS) of the product, and identifying, based on the analysis, multiple training locations on a sample of the product to use in relation to training a machine learning model.

[0030] As an example, the method can obtain one or more low-quality images and one or more high-quality images for each of a plurality of training locations. The method can train a machine learning model using the low-quality images and the high-quality images. For example, the machine learning model can learn how images change between low-quality images and high-quality images, and can be trained to generate images from low-quality images that can approximate high-quality images. After training, the machine learning model can be used to automatically generate high-quality images from low-quality images for products. In this way, high-quality images can be quickly obtained. Moreover, the method can minimize the amount of human supervision needed, prevent inconsistencies caused by different operators using, and avoid various human errors. Thus, the method can improve inspection accuracy. Accordingly, the method can improve manufacturing efficiency and reduce manufacturing cost.

[0031] As an example, the method can also include using the machine learning model to modify the images to approximate results obtained by increasing the number of scans. The term “approximate” means close or similar in quality. For example, the quality of the images using the machine learning model can be within 5%, 10%, 15%, or 20% of the quality of the images obtained by increasing the number of scans.

[0032] Some of the methods disclosed herein facilitate generating high-quality images with high throughput. Moreover, some of the methods can require minimal human intervention, reducing or eliminating inconsistencies of different operators and various human errors, thereby improving accuracy of inspection. In this way, manufacturing efficiency can be improved and manufacturing cost can be reduced.

[0033] Some disclosed embodiments provide a fully automated intelligent e-beam image sampling system for e-beam image enhancement, which includes a GDS pattern analyzer and an intelligent inspection sampling planner for collecting sample images to feed into a machine learning quality enhancement system to generate a non-parametric quality enhancement module. The automated intelligent e-beam image sampling system can be used to enhance lower quality images collected in a higher throughput mode. The automated intelligent e-beam image sampling system is advantageous in that it requires minimal human intervention and generates high-quality images with high throughput for inspection and metrology analysis. Advantageously, the automated intelligent e-beam image sampling system can improve manufacturing efficiency and reduce manufacturing cost.

[0034] Figure 1 is a flowchart 100 of a process for improving images obtained by a SEM sampling system 102 in accordance with some aspects of the present disclosure. The automated intelligent SEM sampling system 102 can be, for example, the automated intelligent SEM sampling system 102 of FIG. 1. Figure 3The EBI system 300. The automated intelligent SEM sampling system 102 can be configured to acquire training images to train a machine learning model that improves image quality. The automated intelligent SEM sampling system 102 can be configured to analyze a plurality of patterns of data related to a layout of a product to identify a plurality of training locations on a sample of the product for use in relation to training a machine learning model. For example, the data can be in a database. For example, the database can be any of GDS, Open Artwork System Exchange Standard, or Caltech Intermediate Format. For example, GDS can include GDS and GDSII. Further, the automated intelligent SEM sampling system 102 can include a GDS pattern analyzer configured to analyze a plurality of patterns from a Graphic Database System (GDS) 101 of the product. The automated intelligent SEM sampling system 102 can additionally include an intelligent inspection sampling planner configured to identify a plurality of training locations on a sample of the product based on the analysis for use in relation to training a machine learning model.

[0035] The automated intelligent SEM sampling system 102 can be configured to acquire a first image having a first quality for each of the plurality of training locations. For example, the system 102 can be configured to enable a first scan for each of the plurality of training locations to acquire the first image for each of the plurality of training locations. The automated intelligent SEM sampling system 102 can be configured to acquire the first image for each of the plurality of training locations based on the first scan. Further, the system 102 can be configured to acquire more than one first image having the first quality for each of the plurality of training locations. For example, the first scan can include a lesser number of scans and the first image can be a low quality image. The lesser number of scans can be a number of scans ranging from about, for example, 1 to about 10. The automated intelligent SEM sampling system 102 can be configured to acquire a second image for each of the plurality of training locations, where the second image has a second quality that is higher than the first quality. For example, the second image can be a high quality image. The second image can be a higher quality image due to having a higher resolution, a higher contrast, a higher sensitivity, a higher brightness, or a lower noise level, etc., or some combination of these.

[0036] For example, the system 102 can be configured to acquire one or more second images having a second quality for each of the plurality of training locations. For example, the second images can be acquired by enabling a second set (or series) of scans, where the set (or series) of scans can include increasing the number of scans, resulting in higher quality images. For example, the increasing number of scans can be a number of scans ranging from about 32 to about 256. As another example, the higher quality images can be acquired by acquiring a plurality of low quality images and averaging the images to produce a higher quality image (e.g., less noise due to averaging). As yet another example, the higher quality images can be acquired by combining a plurality of low quality images to produce a higher quality image. As yet another example, the second images can be received as reference images, as shown at 105, by optional user input. As yet another example, the second images can be acquired based on quality improving scan acquisitions, such as using higher resolution or other setting changes that can improve scan quality.

[0037] The automated intelligent SEM sampling system 102 can use the first and second images for each of the plurality of training locations as training images to train the machine learning model. In some aspects, the automated intelligent SEM sampling system 102 can be configured to enable scans multiple times for each training location to create multiple training images for each location, where some of the training images reflect improvements in image quality resulting from additional numbers of scans of the training locations. For example, pairs of low and high quality images can be acquired and used as training images to train the machine learning model. For example, the automated intelligent SEM sampling system 102 can collect sample images (e.g., pairs of training images) to feed into the machine learning based quality enhancement system 107 to generate a non-parametric quality enhancement module. The automated intelligent SEM sampling system can be used to enhance low quality images 108 collected during operation in high throughput mode to generate enhanced high quality images 109. For example, the automated intelligent SEM sampling system 102 can be configured to use the machine learning model to modify the images to approximate results acquired by increasing the number of scans. An advantage of the fully automated intelligent SEM sampling system 102 is that it requires minimal human intervention while being able to generate high quality images with high throughput for inspection and metrology analysis.

[0038] Figure 2 is a block diagram 200 showing an example of an automated SEM sampling system 202 in accordance with some aspects of the present disclosure. As shown, the automated SEM sampling system 200 can include a computer system 202 (e.g., the computer system 309 in Figure 2 communication with an inspection system 212 and a reference storage device 210. For example, the inspection system 212 can be an EBI tool (e.g., the EBI tool 308 in Figure 3 communication with an inspection system 212 and a reference storage device 210. For example, the inspection system 212 can be an EBI tool (e.g., the EBI tool 308 in Figure 3EBI system 300). The computer system 202 can include a processor 204, a storage medium 206, and a user interface 208. The processor 204 can include multiple processors, and the storage medium 206 and the reference storage device 210 can be the same storage medium. The computer system 202 can communicate with the inspection system 212 and the reference storage device 210 via wired or wireless communication. For example, the computer system can be a controller of an EBI tool, and the controller can have circuitry to cause the EBI tool to perform automated SEM sampling.

[0039] The computer system 202 can include, but is not limited to, a personal computer, a workstation, a network computer, or any device having one or more processors. The storage medium 206 stores SEM sampling instructions, and the processor 204 is configured (via its circuitry) to execute the SEM sampling instructions to control the automated SEM sampling process. The processor 204 can be configured to acquire training images to train a machine learning model to improve image quality, as described in connection with Figure 1 The processor 204 can be configured to analyze a plurality of GDS patterns of a product and identify a plurality of training locations on a sample of the product. The processor 204 can communicate with the inspection system 212 to enable a first scan for each of the plurality of training locations to acquire a first image for each of the plurality of training locations. For example, the processor 204 can instruct the inspection system 212 to perform the first scan to acquire at least one image, which can be a low quality image with fewer scan times. The processor 204 can acquire the first image for each of the plurality of training locations based on the first scan from the inspection system 212. The processor 204 can also acquire a second image for each of the plurality of training locations, which can be a high quality image. For example, the processor 204 can instruct the inspection system 212 to perform a second scan with increased scan times to acquire the high quality image. For another example, the processor 204 can acquire the high quality image as a reference image from the reference storage device 210 through optional user input. The processor 202 can be configured to train a machine learning model using the first image (e.g., a low quality image) and the second image (e.g., a high quality image) for each of the plurality of training locations as training images. In some aspects, a plurality of low quality and high quality image pairs can be acquired and used as training images to train the machine learning model. The processor 204 can be further configured to modify a first image of a new location with low quality using the machine learning model and generate a high quality image of the new location.

[0040] The user interface 208 can include a display configured to display wafer images, input devices configured to transmit user commands to the computer system 202, and the like. The display can be any type of computer output surface and projection mechanism that can show text and graphics images for displaying information to computer users, including but not limited to cathode ray tubes (CRT), liquid crystal displays (LCD), light emitting diodes (LED), gas plasma, touchscreens, or other image projection technologies. The input devices can be any type of computer hardware devices that are used to provide data and control signals from operators to the computer system 202. The input devices can include, but are not limited to, keyboards, mice, scanners, digital cameras, joysticks, trackballs, cursor direction keys, touch screen displays, or audio / video commanders, and the like for transmitting directional information and command selections to the processor or for controlling cursor movement on the display.

[0041] The reference storage 210 can store a database of reference files accessed by the computer system 202 in the automated SEM sampling process. In some embodiments, the reference storage 210 can be part of the computer system 202. Reference image files for inspecting a wafer can be manually provided to the computer system 202 by an operator. Alternatively, the reference storage 210 can be implemented with a processor and the reference image files can be automatically provided to the computer system 202 by the reference storage 210. The reference storage 210 can be a remote server computer configured to store and provide any reference images, can be a cloud storage, and the like.

[0042] The inspection system 212 can be any inspection system capable of generating wafer images. For example, the wafer can be a sample of a product where multiple design patterns of the GDS are analyzed by the processor 204. The wafer can be a semiconductor wafer substrate, a semiconductor wafer substrate with one or more epitaxial layers or process films. Embodiments of the present disclosure are not limited to a particular type for the wafer inspection system 212 so long as the wafer inspection system is capable of generating wafer images with sufficiently high resolution to observe critical features on the wafer (e.g., less than 20 nm), which is consistent with contemporary semiconductor foundry technology. In some aspects of the present disclosure, the inspection system 212 is an electron beam inspection (EBI) system 304. Figure 3 The EBI system 304 is described.

[0043] Once the wafer images are acquired by the inspection system 212, the wafer images can be transmitted to the computer system 202. The computer system 202 and the reference storage 210 can be part of the inspection system 212 or remote from the inspection system 212.

[0044] In some aspects of the embodiments, the automated SEM sampling system 202 may further include an inspection system 212 and a reference storage device 210. For example, the automated SEM sampling system 202 may be further configured to perform a first scan for each of a plurality of training locations to acquire at least one first image for each of the plurality of training locations. As another example, the automated SEM sampling system 202 may be further configured to perform a second scan for each of the plurality of training locations to acquire at least one second image for each of the plurality of training locations. For example, due to the increased number of scans, at least one second image may have enhanced quality.

[0045] Figure 3 This is a schematic diagram illustrating an example electron beam inspection system according to some aspects of this disclosure. For example... Figure 3 As shown, the electron beam inspection system 300 includes a main chamber 302, a loading / locking chamber 304, an electron beam tool 306, a computer system 309, and a device front-end module 308. The computer system 309 may be the controller of the electron beam inspection system 300. The electron beam tool 306 is located within the main chamber 302. The device front-end module 308 includes a first loading port 308a and a second loading port 308b. The device front-end module 308 may include additional loading ports. The first loading port 308a and the second loading port 308b receive a wafer cassette containing a wafer (e.g., a semiconductor wafer or a wafer made of other materials) or a sample to be inspected (wafers and samples are collectively referred to as "wafers" below). One or more robotic arms (not shown) in the device front-end module 308 transfer the wafer to the loading / locking chamber 304. The loading / locking chamber 304 is connected to a loading / locking vacuum pump system (not shown) that removes gas molecules from the loading / locking chamber 304 to achieve a first pressure below atmospheric pressure. After the first pressure is reached, one or more robotic arms (not shown) transfer the wafer from the loading / locking chamber 304 to the main chamber 302. The main chamber 302 is connected to a main chamber vacuum pump system (not shown), which removes gas molecules from the main chamber 302 to achieve a second pressure below the first pressure. After the second pressure is reached, the wafer is inspected by an electron beam tool 306. The electron beam tool 306 can scan a location multiple times to obtain an image. Typically, low-quality images can be obtained in high throughput with fewer scans, while high-quality images can be obtained in low throughput with more scans.

[0046] Figure 4 This illustrates that some aspects of this disclosure may be... Figure 3 A schematic diagram of an example electron beam tool 400 (e.g., 306) as part of an example electron beam inspection system. Figure 4 Examples of components of an electron beam tool 306 according to some aspects of this disclosure are shown.Figure 4 As shown, the electron beam tool 400 can include a motorized stage 400 and a wafer holder 402 supported by the motorized stage 400 to hold a wafer 403 to be inspected. The electron beam tool 400 also includes an objective lens assembly 404, an electron detector 406 (which includes an electron sensor surface), an objective lens aperture 408, a condenser lens 410, a beam limiting aperture 412, a gun aperture 414, an anode 416, and a cathode 418. In some aspects, the objective lens assembly 404 can include a modified swing objective lens retardation immersion lens (SORIL) including a pole piece 404a, a control electrode 404b, a deflector 404c, and a field coil 404d. The electron beam tool 400 can also include an energy dispersive X-ray spectrometer (EDS) detector (not shown) to characterize materials on the wafer.

[0047] A primary electron beam 420 is emitted from the cathode 418 by applying a voltage between the anode 416 and the cathode 418. The primary electron beam 420 passes through the gun aperture 414 and the beam limiting aperture 412, both of which can determine the size of the electron beam that enters the condenser lens 410 located below the beam limiting aperture 412. The condenser lens 410 focuses the primary electron beam 420 before the beam enters the objective lens aperture 408 to set the size of the electron beam before it enters the objective lens assembly 404. The deflector 404c deflects the primary electron beam 420 to facilitate scanning of the beam on the wafer. For example, during scanning, the deflector 404c can be controlled to deflect the primary electron beam 420 onto different locations of the top surface of the wafer 403 at different points in time to provide data for image reconstruction of different portions of the wafer 403. Further, the deflector 404c can be further controlled to deflect the primary electron beam 420 to different sides of the wafer 403 at a particular location at different points in time to provide data for stereoscopic image reconstruction of the wafer structure at that location. Further, in some aspects, the anode 416 and the cathode 418 can be configured to generate multiple primary electron beams 420, and the electron beam tool 400 can include multiple deflectors 404c to project the multiple primary electron beams 420 to different portions / sides of the wafer simultaneously to provide data for image reconstruction of different portions of the wafer 203.

[0048] The field coil 404d and the pole piece 404a generate a magnetic field that starts at one end of the pole piece 404a and terminates at the other end of the pole piece 404a. A portion of the wafer 403 scanned by the primary electron beam 420 can be immersed in the magnetic field and can be charged, which in turn creates an electric field. The electric field reduces the energy of the primary electron beam 420 that hits near the surface of the wafer before the beam collides with the wafer. The control electrode 404b, which is electrically isolated from the pole piece 404a, controls the electric field on the wafer to prevent micro-arching of the wafer and to ensure proper beam focusing.

[0049] Upon receiving the primary electron beam 420, a secondary electron beam 422 can be emitted from a portion of the wafer 403. The secondary electron beam 422 can form a beam spot on the surface of a sensor of the electron detector 406. The electron detector 406 can generate a signal (e.g., voltage, current, etc.) representative of the beam spot intensity and provide the signal to a processing system (not shown). The intensity of the secondary electron beam 422, and the resulting beam spot, can vary depending on the external or internal structure of the wafer 403. Moreover, as described above, the primary electron beam 420 can be projected onto different locations on the wafer top surface to generate secondary electron beams 422 (and resulting beam spots) of different intensities. Thus, by mapping the intensity of the beam spot to the location of the wafer 403, the processing system can reconstruct an image reflecting the internal or external structure of the wafer 403. Once the electron beam tool 126 acquires the wafer image, the wafer image can be transmitted to the computer system 402 (e.g., as shown 202). Figure 2

[0050] Figures 5A-5C A plurality of design patterns of a database, such as a GDS database of products, are shown in accordance with some aspects of the present disclosure. The automated SEM sampling system disclosed herein can be configured to perform a method of automatically acquiring training images to train a machine learning model that improves image quality. For example, the automated SEM sampling system can be a controller of an EBI tool, and the controller can have circuitry to cause the EBI tool to perform automated SEM sampling. For example, the automated SEM sampling system can include a GDS analyzer (e.g., a GDS analyzer component). The GDS analyzer can be configured to perform pattern analysis and classification based on various features, i.e., line pattern, logic pattern, ID / 2D pattern, dense / isolated pattern, etc. The same pattern can be grouped together via pattern grouping.

[0051] For example, a plurality of manufacturing design patterns can be rendered from GDS input. At this stage, the plurality of patterns are scattered patterns. Various features of each pattern can be analyzed and extracted, such as pattern location within a die, shape, size, density, neighborhood layout, pattern type, etc.

[0052] Moreover, the plurality of design patterns can be classified into different categories based on the extracted features. As Figures 5A-5C ​As shown, patterns with similar or identical shapes can be grouped together via pattern grouping. For example, a first pattern subset (group 1) may include patterns with the same or similar shapes as pattern 501a. For example, a second pattern subset (group 2) may include patterns with the same or similar shapes as pattern 501b. For example, a third pattern subset (group 3) may include patterns with the same or similar shapes as pattern 501c. Each pattern group can be associated with corresponding metadata, which may include information such as the pattern location, pattern type, shape, size, and other extracted features within the die.

[0053] An automated SEM sampling system may include an intelligent inspection sampling planner (e.g., an inspection sampling planner component). The intelligent inspection sampling planner can identify multiple training locations on a sample of the product based on the analysis results of an analyzer, for use in conjunction with training a machine learning model. The product's GDS database may contain information related to the location associated with each pattern group. Therefore, the design patterns rendered from the GDS can contain location information. Thus, by analyzing and identifying pattern groups from the GDS, the location of the corresponding pattern group on the product's wafer can be determined.

[0054] Figure 5D This shows wafer 503 (e.g., combined with...). Figure 4 The image 500d describes multiple training locations 506t on pattern 403). For each pattern group, there are multiple potential locations 506 to obtain the training image, such as... Figure 5D As shown. The automatic SEM sampling system can be further configured to determine one or more specific training locations 506t for acquiring training images. For example, based on the analysis results of the analyzer, the SEM sampling system can identify one or more training locations based on one or more of the following: location within the die, inspection area, field of view (FOV), or other imaging parameters such as local alignment points (LAP) on the overlay area in the wafer and autofocus points. For example, for each pattern group, the SEM sampling system can determine one or more training locations based at least in part on the location within the die. The planner can automatically generate die samples across the entire wafer.

[0055] Scan paths can be analyzed and created based on the overall scan area of ​​all pattern groups. The scan path can be determined by parameters such as position, field of view (FOV), or a combination thereof. Furthermore, the scan path and other parameters, such as FOV, shape, and type, can be used according to the electron beam tool's scheme. The electron beam tool can be configured to automatically scan and capture training images for machine learning modules according to a scheme. For example, the LAP and autofocus point can be determined based on factors such as the number of fields of view (FOV) and the distance between each FOV.

[0056] Figure 6Ais a block diagram 600a illustrating a flowchart of a system for automatically acquiring training images 610 to train a machine learning model 615 that improves image quality, in accordance with some aspects of the present disclosure. Figure 6B is a block diagram 600b illustrating details of an automated SEM sampling system, in accordance with some aspects of the present disclosure. Reference is made to Figure 6A and Figure 6B The method implemented by the system can be performed by the automated SEM sampling system 602 (e.g., processor 604, computer system 309) in communication with the EBI tool 612 (e.g., EBI system 300). For example, the automated SEM sampling system can be a controller of the EBI tool, and the controller can have circuitry to cause the EBI tool to perform the method. For example, the method can include performing pattern analysis and classification by the GDS analyzer 603 (e.g., GDS analyzer component 603 of the processor). The pattern analysis and classification can be performed based on various features, such as line patterns, logic patterns, ID / 2D patterns, dense / isolated patterns, etc. The method can also include grouping the same or similar patterns together via pattern grouping.

[0057] The method can include determining, by the sampling planner 605 (e.g., sampling planner component 605 of the processor), scan areas as training locations based on the analysis results of the analyzing step. The analysis results of the analyzing step can include pattern locations with dies, inspection areas, FOV sizes, and LAP points and auto-focus points, and other imaging parameters based on the coverage area in the wafer.

[0058] The method can include enabling scanning multiple times for each training location through the user interface 608 to create multiple training images 610 for each training location, and obtaining the multiple training images 610 from the EBI tool 612. For example, some of the multiple images can be low-quality images, e.g., generated using a fewer number of scans. For example, some of the multiple images can have enhanced image quality, e.g., generated using an increased number of scans. The enhanced image quality can refer to higher resolution, higher contrast, higher sensitivity, higher brightness, or lower noise level, etc. For example, some of the training images 610 can reflect an improvement in image quality resulting from an additional number of scans of the training location. In some aspects, pairs of multiple low-quality images and high-quality images can be obtained via the user interface 608 and used as training images 610 to train the machine learning model 615. For example, the low-quality SEM image mode can be based on default settings or user inputted yield requirements. For example, the high-quality SEM image mode can be based on default settings or user inputted quality requirements. In some aspects, the user can also choose to directly input high-quality reference images 611. For example, the high-quality reference images 611 can be stored in the storage medium 606. In this case, the obtaining of high-quality images can be skipped.

[0059] The method can also include using the machine learning model 615 (e.g., a machine learning model component of the processor) to modify images to approximate results obtained by an increased number of scans. Various machine learning methods can be employed in the machine learning model 615 to learn an enhancement framework from the training image pairs 610. The machine learning model 615 can be parameterized. Data can be collected for the machine learning model 615.

[0060] The quality enhancement module 617 (e.g., a quality enhancement module 617 of the processor) can be learned at the end of the step of using the machine learning model for each type of pattern of interest. The quality enhancement module 617 can be used directly for inspection or metrology purposes in a high-yield mode. After being trained based on images sampled from the automated sampling system 602, the quality enhancement module 617 can be used for image enhancement without training data. Thus, the quality enhancement module 617 can be non-parametric, which does not involve using excessive parameter settings that can result in excessive overhead. Thus, the quality enhancement module 617 facilitates generating high-quality images at high yield, thereby improving manufacturing efficiency and reducing manufacturing cost.

[0061] Figure 7is a flowchart 700 illustrating an example of a method of automatically acquiring training images to train a machine learning model that improves image quality, in accordance with some aspects of the present disclosure. The method can be performed by an automated SEM sampling system (e.g., 102, 202, 602) in communication with an EBI tool (e.g., 212, 612). For example, the automated SEM sampling system can be a controller of the EBI tool, and the controller can have circuitry to cause the EBI tool to perform the method.

[0062] As shown in FIG. 7, at step 702, the method can include analyzing a plurality of patterns of data related to a layout of a product to identify a plurality of training locations on a sample of the product for use in relation to training a machine learning model. For example, the data can be in a database. For example, the database can be any of a graphic database system (GDS), an open art system exchange standard, or a Caltech intermediate format, etc. For example, the GDS can include GDS and GDSII. Figure 7

[0063] For example, the step of analyzing the plurality of patterns of data related to the layout of the product can further include classifying the plurality of patterns into a plurality of pattern subsets. For example, the step of analyzing the plurality of patterns of data related to the layout of the product can further include extracting features from the plurality of patterns. For example, classifying the plurality of patterns into a plurality of pattern subsets can be based on the extracted features. For example, each subset of the plurality of pattern subsets can be associated with information related to a location, a type, a shape, a size, a density, or a neighborhood layout. For example, identifying the plurality of training locations can be based on a field of view, a local alignment point, or an autofocus point. For example, identifying the plurality of training locations can include identifying one or more training locations for each pattern subset.

[0064] At step 704, the method can include acquiring, for each training location of the plurality of training locations, a first image having a first quality. For example, the step of acquiring, for each training location of the plurality of training locations, a first image having a first quality can include acquiring, for each training location of the plurality of training locations, more than one first image having the first quality.

[0065] For example, the method can further include determining a first scan path, the first scan path including a first scan for acquiring the first image. For example, the first scan path can be based on an overall scan area of the plurality of training locations. For example, the first scan path can be determined by some parameters such as a location, a FOV, etc. Further, the first scan path along with other parameters such as a FOV, a shape, a type, etc. can provide a first recipe to an e-beam tool. The e-beam tool can be configured to follow the first recipe to automatically scan and capture images for the machine learning module.

[0066] ​At step 706, the method can include obtaining, for each of the plurality of training positions, a second image having a second quality. For example, the second quality can be higher than the first quality. For example, the step of obtaining, for each of the plurality of training positions, a second image having a second quality includes obtaining, for each of the plurality of training positions, more than one second image having the second quality.

[0067] For example, the method can include determining a second scan path, the second scan path including a second scan for obtaining the second image. For example, the second scan path is based on an overall scan area of the plurality of training positions. For example, the second scan path can be determined by some parameters such as position, FOV, etc. Further, the second scan path along with other parameters such as FOV, shape, type, etc. can provide a second recipe to the e-beam tool. The e-beam tool can be configured to follow the second recipe to automatically scan and capture images for the machine learning module. For example, the first scan can include a first number of scans, and the second scan can include a second number of scans, where the second number of scans can be greater than the first number of scans.

[0068] For example, the second image can be obtained as a reference image through an optional user input.

[0069] At step 708, the method can include training the machine learning model using the first image and the second image.

[0070] At step 710, the method can include modifying an image using the machine learning model to approximate a result obtained by an increased number of scans.

[0071] For example, the method can further include modifying a first image of a certain position using the machine learning model to obtain a second image of the position, where the second image has an enhanced quality than the first image. In this way, the method facilitates obtaining high quality images with high throughput, thereby improving manufacturing efficiency and reducing manufacturing cost. Further, the method is fully automated. Therefore, the method can prevent human errors and inconsistencies from different operators. Accordingly, the method is more advantageous to improve detection accuracy.

[0072] Now referring again to Figure 2, the computer system 202 can be a controller of the inspection system 212 (e.g., an electron beam inspection system), and the controller can include circuitry to analyze a plurality of patterns of data related to a layout of a product to identify a plurality of training locations on a sample of the product for use in relation to training a machine learning model; acquire, for each of the plurality of training locations, a first image having a first quality; acquire, for each of the plurality of training locations, a second image having a second quality, the second quality being higher than the first quality; and train the machine learning model using the first images and the second images.

[0073] Further reference is made to Figure 2 , the storage medium 206 can be a non-transitory computer readable medium storing a set of instructions executable by a controller of a device to cause the device to perform a method comprising: analyzing a plurality of patterns of data related to a layout of a product to identify a plurality of training locations on a sample of the product for use in relation to training a machine learning model; acquiring, for each of the plurality of training locations, a first image having a first quality; acquiring, for each of the plurality of training locations, a second image having a second quality, the second quality being higher than the first quality; and training the machine learning model using the first images and the second images.

[0074] Embodiments can be further described using the following clauses:

[0075] 1. A method of automatically acquiring training images for training a machine learning model, the method comprising:

[0076] analyzing a plurality of patterns of data related to a layout of a product to identify a plurality of training locations for use in relation to training the machine learning model;

[0077] acquiring, for each of the plurality of training locations, a first image having a first quality;

[0078] acquiring, for each of the plurality of training locations, a second image having a second quality, the second quality being higher than the first quality; and

[0079] training the machine learning model using the first images and the second images.

[0080] 2. The method of clause 1, wherein the data is in a database.

[0081] 3. The method of clause 2, wherein the database is any of a Graph Database System (GDS), an Open Artwork System Exchange Standard, or a Caltech Intermediate Format.

[0082] 4. The method of clause 3, wherein the GDS comprises GDS format data or GDSII format data.

[0083] 5. The method of clause 1, wherein the step of acquiring, for each of the plurality of training locations, a first image having a first quality comprises acquiring, for each of the plurality of training locations, more than one first image having the first quality.

[0084] 6. The method of clause 1, wherein the step of acquiring, for each of the plurality of training locations, a second image having a second quality comprises acquiring, for each of the plurality of training locations, more than one second image having the second quality.

[0085] 7. The method of clause 1, wherein the step of analyzing a plurality of patterns of data related to a layout of a product further comprises classifying the plurality of patterns into a plurality of pattern subsets.

[0086] 8. The method of any one of clauses 1-7, wherein the step of analyzing a plurality of patterns of data related to a layout of a product further comprises extracting features from the plurality of patterns.

[0087] 9. The method of clause 8, wherein the extracted features comprise shape, size, density, or neighborhood layout.

[0088] 10. The method of clause 7, wherein classifying the plurality of patterns into a plurality of pattern subsets is based on the extracted features.

[0089] 11. The method of clause 7, wherein each of the plurality of pattern subsets is associated with information related to a location, type, shape, size, density, or neighborhood layout.

[0090] 12. The method of any one of clauses 1-11, wherein identifying the plurality of training locations is based on a field of view, a local alignment point, or an autofocus point.

[0091] 13. The method of any one of clauses 1-12, wherein the method further comprises determining a first scan path comprising a first scan for acquiring the first image, the first scan path based on an overall scan area of the plurality of training locations.

[0092] 14. The method of clause 13, wherein the method further comprises

[0093] determining a second scan path comprising a second scan for acquiring the second image, the second scan path based on the overall scan area of the plurality of training locations.

[0094] 15. The method of clause 14, wherein the first scan comprises a first number of scans, wherein the second scan comprises a second number of scans, and wherein the second number of scans is greater than the first number of scans.

[0095] 16. The method of any of clauses 1-13, wherein the second image is obtained as a reference image by optional user input.

[0096] 17. The method of any of clauses 1-16, further comprising

[0097] using the machine learning model to modify a first image of a location to obtain a second image of the location, wherein the second image has enhanced quality than the first image.

[0098] 18. The method of any of clauses 1-17, wherein identifying the plurality of training locations comprises identifying one or more training locations for each pattern subset.

[0099] 19. The method of any of clauses 1-18, wherein the quality comprises resolution, contrast, brightness, or noise level.

[0100] 20. The method of any of clauses 1-19, further comprising

[0101] using the machine learning model to modify an image to approximate a result obtained by an increased number of scans.

[0102] 21. An apparatus for automatically obtaining training images to train an ML model that improves image quality, the apparatus comprising:

[0103] a memory; and

[0104] at least one processor coupled to the memory and configured to:

[0105] analyze a plurality of patterns of data related to a layout of a product to identify a plurality of training locations to use in relation to training the machine learning model;

[0106] obtain, for each training location of the plurality of training locations, a first image having a first quality;

[0107] obtain, for each training location of the plurality of training locations, a second image having a second quality, the second quality being higher than the first quality; and

[0108] train the machine learning model using the first image and the second image.

[0109] 22. The apparatus of clause 21, wherein the data is in a database.

[0110] 23. The apparatus of clause 22, wherein the database is any one of a Graphic Database System (GDS), an Open Artwork System Exchange Standard, or a Caltech Intermediate Format.

[0111] 24. The apparatus of clause 23, wherein the GDS comprises GDS format data or GDSII format data.

[0112] 25. The apparatus of clause 21, wherein the at least one processor is further configured to acquire, for each of the plurality of training positions, more than one first image having a first quality.

[0113] 26. The apparatus of clause 21, wherein the at least one processor is further configured to acquire, for each of the plurality of training positions, more than one second image having a second quality.

[0114] 27. The apparatus of clause 21, wherein the at least one processor is further configured to classify the plurality of patterns into a plurality of pattern subsets.

[0115] 28. The apparatus of clause 21, wherein the at least one processor is further configured to extract features from the plurality of patterns.

[0116] 29. The apparatus of clause 28, wherein the extracted features comprise shape, size, density, or neighborhood layout.

[0117] 30. The apparatus of clause 27, wherein the at least one processor is further configured to classify the plurality of patterns into a plurality of pattern subsets based on the extracted features.

[0118] 31. The apparatus of clause 27, wherein each of the plurality of pattern subsets is associated with information related to position, type, shape, size, density, or neighborhood layout.

[0119] 32. The apparatus of any one of clauses 21 to 31, wherein the at least one processor is further configured to identify the plurality of training positions based on a field of view, a local alignment point, or an autofocus point.

[0120] 33. The apparatus of any one of clauses 21 to 32, wherein the at least one processor is further configured to

[0121] determine a first scan path comprising a first scan for acquiring the first images, the first scan path based on an overall scan area for the plurality of training positions.

[0122] 34. The apparatus of clause 33, wherein the at least one processor is further configured to

[0123] determine a second scan path comprising a second scan to acquire the second image, the second scan path based on an overall scan area for the plurality of training locations.

[0124] 35. The apparatus of clause 34, wherein the first scan comprises a first number of scans, wherein the second scan comprises a second number of scans, and wherein the second number of scans is greater than the first number of scans.

[0125] 36. The apparatus of any of clauses 21 to 33, wherein the second image is received as a reference image through optional user input.

[0126] 37. The apparatus of any of clauses 21 to 36, wherein the at least one processor is further configured to

[0127] modify a first image of a location using the machine learning model to acquire a second image of the location, wherein the second image has enhanced quality than the first image.

[0128] 38. The apparatus of any of clauses 21 to 37, wherein the at least one processor is further configured to

[0129] identify one or more training locations for each pattern subset.

[0130] 39. The apparatus of any of clauses 21 to 38, wherein the quality comprises resolution, contrast, brightness, or noise level.

[0131] 40. The apparatus of any of clauses 21 to 39, wherein the at least one processor is further configured to

[0132] modify an image using the machine learning model to approximate a result acquired through an increased number of scans.

[0133] 41. A non-transitory computer readable medium storing a set of instructions executable by a controller of a device to cause the device to perform a method, the method comprising:

[0134] analyzing a plurality of patterns of data related to a layout of a product to identify a plurality of training locations to use in relation to training a machine learning model;

[0135] acquiring a first image having a first quality for each training location of the plurality of training locations;

[0136] acquiring, for each of the plurality of training locations, a second image having a second quality, the second quality being higher than the first quality; and

[0137] training the machine learning model using the first image and the second image.

[0138] 42. The non-transitory computer readable medium of clause 41, wherein the data is in a database.

[0139] 43. The non-transitory computer readable medium of clause 42, wherein the database is any one of a Graphic Database System (GDS), an Open Artwork System Exchange Standard, a Caltech Intermediate Format, or an Electronic Design Interchange Format.

[0140] 44. The non-transitory computer readable medium of clause 43, wherein the GDS comprises at least one of a GDS or a GDSII.

[0141] 45. The non-transitory computer readable medium of clause 41, wherein the step of acquiring, for each of the plurality of training locations, a first image having a first quality further comprises acquiring, for each of the plurality of training locations, more than one first image having the first quality.

[0142] 46. The non-transitory computer readable medium of clause 41, wherein the step of acquiring, for each of the plurality of training locations, a second image having a second quality comprises acquiring, for each of the plurality of training locations, more than one second image having the second quality.

[0143] 47. The non-transitory computer readable medium of clause 41, wherein the step of analyzing a plurality of patterns of data related to a layout of a product further comprises classifying the plurality of patterns into a plurality of pattern subsets.

[0144] 48. The non-transitory computer readable medium of any one of clauses 41 to 47, wherein the step of analyzing a plurality of patterns of data related to a layout of a product further comprises extracting features from the plurality of patterns.

[0145] 49. The non-transitory computer readable medium of clause 48, wherein the extracted features comprise shape, size, density, or neighborhood layout.

[0146] 50. The non-transitory computer readable medium of clause 47, wherein classifying the plurality of patterns into a plurality of pattern subsets is based on the extracted features.

[0147] 51. The non-transitory computer-readable medium of clause 47, wherein each subset of the plurality of pattern subsets is associated with information related to a location, a type, a shape, a size, a density, or a neighborhood layout.

[0148] 52. The non-transitory computer-readable medium of any one of clauses 41-51, wherein identifying the plurality of training locations is based on a field of view, a local alignment point, or an autofocus point.

[0149] 53. The non-transitory computer-readable medium of any one of clauses 41-52, wherein the method further comprises

[0150] determining a first scan path comprising a first scan for acquiring the first image, the first scan path based on an overall scan area of the plurality of training locations.

[0151] 54. The non-transitory computer-readable medium of clause 53, wherein the method further comprises

[0152] determining a second scan path comprising a second scan for acquiring the second image, the second scan path based on an overall scan area of the plurality of training locations.

[0153] 55. The non-transitory computer-readable medium of clause 54, wherein the first scan comprises a first number of scans, wherein the second scan comprises a second number of scans, and wherein the second number of scans is greater than the first number of scans.

[0154] 56. The non-transitory computer-readable medium of any one of clauses 41-53, wherein the second image is acquired as a reference image through optional user input.

[0155] 57. The non-transitory computer-readable medium of any one of clauses 41-56, wherein the method further comprises using the machine learning model to modify a first image of a location to acquire a second image of the location, wherein the second image has enhanced quality than the first image.

[0156] 58. The non-transitory computer-readable medium of any one of clauses 41-57, wherein identifying the plurality of training locations comprises identifying one or more training locations for each pattern subset.

[0157] 59. The non-transitory computer-readable medium of any one of clauses 41-58, wherein the quality comprises resolution, contrast, brightness, or noise level.

[0158] 60. The non-transitory computer-readable medium of any one of clauses 41-59, wherein the method further comprises

[0159] using the machine learning model to modify the image to approximate a result obtained by an increase in a number of scans.

[0160] 61. An e-beam inspection apparatus comprising:

[0161] a controller having circuitry to cause the e-beam inspection apparatus to perform the following operations:

[0162] analyze a plurality of patterns of data related to a layout of a product to identify a plurality of training locations to use in relation to training a machine learning model;

[0163] obtain, for each training location of the plurality of training locations, a first image having a first quality;

[0164] obtain, for each training location of the plurality of training locations, a second image having a second quality, the second quality being higher than the first quality; and

[0165] train the machine learning model using the first image and the second image.

[0166] 62. The e-beam inspection apparatus of clause 61, wherein the data is in a database.

[0167] 63. The e-beam inspection apparatus of clause 62, wherein the database is any one of a Graphic Database System (GDS), an Open Artwork System Interchange Standard, or a Caltech Intermediate Format.

[0168] 64. The e-beam inspection apparatus of clause 63, wherein the GDS comprises at least one of a GDS or a GDSII.

[0169] 65. The e-beam inspection apparatus of clause 61, wherein the step of obtaining, for each training location of the plurality of training locations, a first image having a first quality comprises obtaining, for each training location of the plurality of training locations, more than one first image having the first quality.

[0170] 66. The e-beam inspection apparatus of clause 61, wherein the step of obtaining, for each training location of the plurality of training locations, a second image having a second quality comprises obtaining, for each training location of the plurality of training locations, more than one second image having the second quality.

[0171] 67. The e-beam inspection apparatus of clause 61, wherein the step of analyzing a plurality of patterns of data related to a layout of a product further comprises classifying the plurality of patterns into a plurality of pattern subsets.

[0172] 68. The e-beam inspection apparatus of any of clauses 61-67, wherein the step of analyzing a plurality of patterns of data related to a layout of a product further comprises extracting features from the plurality of patterns.

[0173] 69. The e-beam inspection apparatus of clause 68, wherein the extracted features comprise shape, size, density, or neighborhood layout.

[0174] 70. The e-beam inspection apparatus of clause 67, wherein classifying the plurality of patterns into a plurality of pattern subsets is based on the extracted features.

[0175] 71. The e-beam inspection apparatus of clause 67, wherein each of the plurality of pattern subsets is associated with information related to a location, type, shape, size, density, or neighborhood layout.

[0176] 72. The e-beam inspection apparatus of any of clauses 61-71, wherein identifying the plurality of training locations is based on a field of view, a local alignment point, or an auto focus point.

[0177] 73. The e-beam inspection apparatus of any of clauses 61-72, wherein the controller has circuitry to cause the e-beam inspection apparatus to further perform operations comprising:

[0178] determining a first scan path comprising a first scan for acquiring the first image, the first scan path based on an overall scan area of the plurality of training locations.

[0179] 74. The e-beam inspection apparatus of clause 73, wherein the controller has circuitry to cause the e-beam inspection apparatus to further perform operations comprising:

[0180] determining a second scan path comprising a second scan for acquiring the second image, the second scan path based on an overall scan area of the plurality of training locations.

[0181] 75. The e-beam inspection apparatus of clause 74, wherein the first scan comprises a first number of scans, wherein the second scan comprises a second number of scans, and wherein the second number of scans is greater than the first number of scans.

[0182] 76. The e-beam inspection apparatus of any of clauses 61-73, wherein the second image is acquired as a reference image by optional user input.

[0183] 77. The e-beam inspection apparatus of any of clauses 61-76, wherein the controller has circuitry to cause the e-beam inspection apparatus to further perform operations comprising:

[0184] modify a first image of a location using the machine learning model to obtain a second image of the location, wherein the second image has enhanced quality compared to the first image.

[0185] 78. The e-beam inspection apparatus of any of Clauses 61-77, wherein identifying the plurality of training locations comprises identifying one or more training locations for each pattern subset.

[0186] 79. The e-beam inspection apparatus of any of Clauses 61-78, wherein the quality comprises resolution, contrast, brightness, or noise level.

[0187] 80. The e-beam inspection apparatus of any of Clauses 61-79, wherein the controller has circuitry to cause the e-beam inspection apparatus to further perform operations comprising:

[0188] modify an image using the machine learning model to approximate a result that would be obtained by increasing a number of scans.

[0189] 81. The method of Clause 1, wherein the pattern of data is a scanning electron microscope (SEM) image of the product.

[0190] 82. The method of Clause 1, wherein the second image is obtained based on a plurality of low quality images.

[0191] 83. The method of Clause 1, wherein the second image is obtained by averaging a plurality of low quality images.

[0192] 84. The method of Clause 1, wherein the second image is obtained by combining a plurality of low quality images.

[0193] 85. The method of Clause 1, wherein the first image and the second image are images obtained by one or more scanning electron microscopes, and wherein the second image is a higher quality image than the first image.

[0194] 86. The method of Clause 85, wherein the second image has higher resolution, higher contrast, higher brightness, or reduced noise level compared to the first image.

[0195] 87. The apparatus of Clause 21, wherein the pattern of data is a scanning electron microscope (SEM) image of the product.

[0196] 88. The apparatus of Clause 21, wherein the second image is obtained based on a plurality of low quality images.

[0197] 89. The apparatus of clause 21, wherein the second image is acquired by averaging the plurality of low quality images.

[0198] 90. The apparatus of clause 21, wherein the second image is acquired by combining a plurality of low quality images.

[0199] 91. The non-transitory computer readable medium of clause 41, wherein the pattern of data is a scanning electron microscope (SEM) image of the product.

[0200] 92. The non-transitory computer readable medium of clause 41, wherein the second image is acquired based on a plurality of low quality images.

[0201] 93. The non-transitory computer readable medium of clause 41, wherein the second image is acquired by averaging a plurality of low quality images.

[0202] 94. The non-transitory computer readable medium of clause 41, wherein the second image is acquired by combining a plurality of low quality images.

[0203] 95. The e-beam inspection apparatus of clause 61, wherein the pattern of data is a scanning electron microscope (SEM) image of the product.

[0204] 96. The e-beam inspection apparatus of clause 61, wherein the second image is acquired based on a plurality of low quality images.

[0205] 97. The e-beam inspection apparatus of clause 61, wherein the second image is acquired by averaging a plurality of low quality images.

[0206] 98. The e-beam inspection apparatus of clause 61, wherein the second image is acquired by combining a plurality of low quality images.

[0207] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to produce a machine, such that the instructions that execute via the processor of the computer or other programmable data processing apparatus create means for implementing the functions / acts specified in the flowchart or flow diagram block or blocks. These computer program instructions can also be stored in a computer- readable medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable medium produce an article of manufacture including instructions which implement the functions / acts specified in the flowchart or flow diagram block or blocks.

[0208] These computer program instructions can also be stored in a computer- readable medium that can direct a hardware processor core, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer-readable medium implement the functions / acts specified in the flowchart or block diagram block or blocks.

[0209] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart or block diagram block or blocks.

[0210] Any combination of one or more computer readable medium can be utilized. The computer readable medium can be a non-transitory computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM, EEPROM, or Flash memory), an optical fiber, a cloud storage, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium can be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0211] Program code embodied on a computer readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, IR, etc., or any suitable combination of the foregoing.

[0212] Computer program code for carrying out operations of the example embodiments can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0213] The flow diagrams and block diagrams in the drawings are examples of architectures, functionality, and operations that might be implemented in accordance with various embodiments. In this regard, each block in the flow diagrams or block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending on the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

[0214] It should be understood that the described embodiments are not mutually exclusive and elements, components, materials or steps described in connection with one example embodiment can be combined with or removed from other embodiments in a suitable manner to achieve a desired design goal.

[0215] Reference herein to "some aspects," "some embodiments," or "some example embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one aspect or embodiment. The appearances of the phrase "one aspect," "some aspects," "one embodiment," "some embodiments," or "some example embodiments" in various places in the specification are not necessarily referring to the same embodiment, nor are separate or alternative embodiments necessarily mutually exclusive of other embodiments.

[0216] It should be understood that the steps of the example methods set forth herein are not necessarily required to be performed in the order described, and the order of the steps of such methods should be understood as merely an example. Likewise, additional steps can be included in such methods, and certain steps can be omitted or combined, in accordance with various embodiments.

[0217] As used herein, the term "or" encompasses all possible combinations, unless otherwise specifically stated, e.g., if A or B is stated, then A or B or A and B are encompassed unless otherwise specifically stated. As a second example, if A, B, or C is stated, then A, or B, or C, or A and B, or A and C, or B and C, or A and B and C are encompassed unless otherwise specifically stated.

[0218] As used in this application, the words “exemplary” and “demonstrative” are used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects or designs.

[0219] Further, as used in this application and the appended claims, the singular forms “a,” “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0220] Unless specifically stated otherwise, each numerical value and range should be interpreted as approximately as if the word “about” preceded the respective value and the term “approximately” followed the respective value.

[0221] The use of the singular throughout the claims denotes “one or more” unless otherwise indicated. The use of numbering or reference labels in the claims to identify one or more possible embodiments of the claimed subject matter is intended to facilitate the interpretation of the claims. Such usage should not be interpreted to necessarily limit the scope of those claims to the embodiments shown in the corresponding drawings.

[0222] Although elements in the following method claims, if any, are in a specific order, the order of the elements should not be construed as necessarily implying a specific order unless explicitly stated otherwise in the claim.

[0223] It will be further understood that those skilled in the art can make various changes to the details of the components described and illustrated as essential to the understanding of the nature of the aspects or embodiments described without departing from the scope of the appended claims.

Claims

1. An apparatus for automatically acquiring training images to train a machine learning model that improves image quality, the apparatus comprising: a memory; and at least one processor coupled to the memory and configured to: analyze a plurality of patterns of data related to a layout of a product to identify a plurality of training locations to use in relation to training the machine learning model; acquire, for each training location of the plurality of training locations, a first image having a first quality by a first number of scans of a first scan; acquire, for the each training location of the plurality of training locations, a second image having a second quality by a second number of scans of a second scan, the second quality being higher than the first quality; and use the first image and the second image to train the machine learning model.

2. The apparatus of claim 1, wherein the data is in a database.

3. The apparatus of claim 2, wherein the database is any one of a Graphic Database System (GDS), an Open Artwork System Exchange Standard, or a Caltech Intermediate Format.

4. The apparatus of claim 3, wherein the GDS includes GDS format data or GDSII format data.

5. The apparatus of claim 1, wherein the at least one processor is further configured to acquire, for each training location of the plurality of training locations, more than one first image having the first quality.

6. The apparatus of claim 1, wherein the at least one processor is further configured to acquire, for each training location of the plurality of training locations, more than one second image having the second quality.

7. The apparatus of claim 1, wherein the at least one processor is further configured to classify the plurality of patterns into a plurality of pattern subsets.

8. The apparatus of claim 1, wherein the at least one processor is further configured to extract features from the plurality of patterns.

9. The apparatus of claim 8, wherein the extracted features include shape, size, density, or neighborhood layout.

10. The apparatus of claim 7, wherein the at least one processor is further configured to classify the plurality of patterns into a plurality of pattern subsets based on the extracted features.

11. The apparatus of claim 7, wherein each subset of the plurality of pattern subsets is associated with information related to a location, a type, a shape, a size, a density, or a neighborhood layout.

12. The apparatus of claim 1, wherein the at least one processor is further configured to identify the plurality of training locations based on a field of view, a local alignment point, or an autofocus point.

13. The apparatus of claim 1, wherein the at least one processor is further configured to determine a first scan path that includes the first scan for acquiring the first image, the first scan path being based on a total scan area for the plurality of training locations.

14. The apparatus of claim 13, wherein the at least one processor is further configured to determining a second scan path for a second scan to acquire the second image, the second scan path based on an overall scan area for the plurality of training locations.

15. A non-transitory computer readable medium storing a set of instructions executable by a controller of a device to cause the device to perform a method, the method comprising: analyzing a plurality of patterns of data related to a layout of a product to identify a plurality of training locations to use in relation to training a machine learning model; acquiring, for each training location of the plurality of training locations, a first image having a first quality by a first number of scans; acquiring, for the each training location of the plurality of training locations, a second image having a second quality by a second number of scans, the second quality higher than the first quality; and training the machine learning model using the first images and the second images.

Images