Substrate Mapping Using Deep Neural Networks

Abstract

Various examples include systems and networks for mapping substrates (e.g., silicon wafers in a wafer cassette, etc.) within a substrate carrier and a classification of the state of each substrate and the carrier in which the substrate is placed. In various examples provided herein, an image acquisition system, such as a camera, acquires multiple images of the substrates within the carrier. The image(s) are then processed using a deep convolutional neural network to classify the state of the substrate relative to the substrate slot, including empty slot, occupied slot (e.g., properly loaded slot), double loaded slot, cross-slot insertion, and protrusion (if the substrate is not fully loaded within the slot).

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of priority to U.S. Patent Application No. 17 / 476,195, filed September 15, 2021, the contents of which are incorporated herein by reference in their entirety.

[0002] FIELD OF THE INVENTION The disclosed subject matter relates generally to the field of substrate inspection and metrology tools used in the semiconductor and related industries (e.g., flat panel display and solar cell production facilities). More specifically, in various embodiments, the disclosed subject matter relates to a mapping system capable of determining the classification status of a substrate (e.g., wafer) having a substrate carrier (e.g., wafer cassette). [Background technology]

[0003] Various types of substrates, such as semiconductor wafers, are typically placed in various types of substrate carriers, such as wafer cassettes, for processing and metrology operations within a manufacturing facility (e.g., an integrated circuit manufacturing facility, etc.). However, before an operation begins, it is desirable to know whether all of the substrates within the carrier are properly loaded within the carrier and which substrate slots within the carrier are occupied by substrates. Summary of the Invention

[0004] This document describes, among other things, mapping substrates within a substrate carrier and classifying the state of each substrate and the carrier in which the substrate is placed. In various examples provided herein, an image acquisition system, such as a camera, acquires one or more images of the substrates within the carrier. The images are then processed using a deep convolutional neural network to classify the state of the carrier's substrate slots, including empty slots, occupied slots (e.g., properly loaded slots), double-loaded slots, cross-slot insertion, and protrusions (when the substrate is not fully loaded within the slot).

[0005] In various embodiments, the disclosed subject matter is a method for classifying a state of a number of substrates within a location within a substrate carrier. The method includes detecting at least a portion of a substrate within the substrate carrier. The detecting includes capturing one or more images of the portion of the substrate and locations proximate to the portion of the substrate, transferring the one or more images to a pre-trained deep convolutional neural network, classifying a state of the portion of the substrate within the location within the substrate carrier from the images using the pre-trained deep convolutional neural network, and providing automatic tagging of each of a plurality of locations within the substrate carrier proximate to a plurality of portions of the substrate.

[0006] In various embodiments, the disclosed subject matter is a substrate mapping system. The substrate mapping system includes a camera that collects one or more images of a substrate and its potential location within a substrate carrier, and a data collection and control system. The one or more images include the substrate and its location relative to a number of substrate slots within the substrate carrier. The data collection and control system includes one or more hardware-based processors of a machine coupled to the camera and is configured to transfer the one or more images to a deep convolutional neural network, use the deep convolutional neural network to classify from the one or more images a state of the imaged portion of the substrate within the substrate location within the substrate carrier, and provide automatic tagging of each of the substrate locations within the substrate carrier that are proximate to the imaged portion of the substrate.

[0007] In various embodiments, the disclosed subject matter is a computer-readable medium that includes instructions that, when executed by a machine, cause the machine to perform operations including detecting at least a portion of a substrate within a substrate carrier, capturing one or more images of the portion of the substrate and locations proximate the portion of the substrate, transferring the one or more images to a pre-trained deep convolutional neural network, classifying a state of the portion of the substrate within the location within the substrate carrier from the images using the pre-trained deep convolutional neural network, and providing automatic tagging of each of a plurality of locations within the substrate carrier proximate to a plurality of portions of the substrate.

[0008] The various figures of the accompanying drawings are merely illustrative of example implementations of the present disclosure and should not be considered as limiting its scope. [Brief explanation of the drawings]

[0009] [Figure 1A] 1 illustrates a substrate carrier usable with various embodiments of the disclosed subject matter. [Figure 1B] 1 illustrates a front-opening unified-pod (FOUP) usable with various embodiments of the disclosed subject matter. [Figure 2A] 1 shows a cross-sectional portion of a substrate carrier with a substrate loaded therein. [Figure 2B] 1 shows a cross-sectional portion of a substrate carrier containing substrates, one of which is a cross-slot insert within the substrate carrier. [Figure 2C] 1 shows a cross-sectional portion of a substrate carrier with a substrate properly loaded within the substrate carrier, as well as a location within the substrate carrier where a substrate is missing. [Figure 2D] 1 shows a cross-sectional portion of a substrate carrier containing a double-loaded substrate, along with a substrate properly loaded within the substrate carrier. [Figure 3] 1 illustrates an example of a substrate mapping system according to an embodiment of the disclosed subject matter. [Figure 4A] 4 illustrates an example of a pre-processing system that may be used with the substrate mapping system of FIG. 3, according to various embodiments of the disclosed subject matter. [Figure 4B] 4 illustrates an example framework for detecting substrate loading errors in a carrier from images acquired from the substrate mapping system of FIG. 3, according to various embodiments of the disclosed subject matter. [Figure 4C] 4 illustrates an example of a multiple network pre-processing system that may be used with the substrate mapping system of FIG. 3, according to various embodiments of the disclosed subject matter. [Figure 4D] 4 illustrates an example of a single network pre-processing system that may be used with the substrate mapping system of FIG. 3, according to various embodiments of the disclosed subject matter. [Figure 5A] 4 illustrates an exemplary method for training the substrate mapping system of FIG. 3 in accordance with various embodiments of the disclosed subject matter. [Figure 5B] 4 illustrates an exemplary method for using the substrate mapping system of FIG. 3 in a normal operating mode, according to various embodiments of the disclosed subject matter. [Figure 6] 1 illustrates an example block diagram comprising a machine upon which any one or more of the techniques (eg, methodologies) discussed herein may be implemented. DETAILED DESCRIPTION OF THE INVENTION

[0010] The disclosed subject matter relates to detecting and mapping substrates (e.g., silicon wafers) within a substrate carrier (e.g., a wafer cassette or front-opening integrated pod (FOUP)). Current substrate-carrier mapping solutions often do not work well in manufacturing environments (e.g., semiconductor fabs). For example, compound fabs have a wide range of substrate diameters (e.g., 75 mm, 100 mm, 125 mm, 150 mm, 200 mm, and 300 mm), substrate types (e.g., silicon, sapphire, gallium arsenide (GaAs), silicon carbide (SiC), and other III-V and II-VI compound semiconductors, bonded wafers, etc.), and substrate thicknesses (e.g., from 150 μm or less to 2000 μm or more). Back-end of line (BEOL) processes using bonded substrates may also have a variety of different substrates within a single substrate carrier.

[0011] Currently, various attempts at substrate mapping solutions exist, however, current mapping solutions for substrate carriers (e.g., silicon wafer cassettes, etc.) do not work well for, for example, front-end compound fabrication facilities (fabs) and back-end fabs.

[0012] For example, one approach uses optical sensors mounted on the robot end effector used to remove and place substrates from the carrier. While end effector-mounted optical sensors work in certain situations, they require an additional sweep step performed by the robot handling system, making them slow and time-consuming to configure.

[0013] Certain other approaches use optical sensors mounted on the load port door. However, load port-mounted optical sensors cannot accommodate a variety of substrate types and thicknesses because (1) the optics are in a fixed location and (2) the optics rely on a fixed wavelength of light. Also, most complex fabs manually load carriers due to fab space constraints. Therefore, there are no movable load ports on which sensors can be mounted.

[0014] Certain other approaches use cameras to simultaneously observe the entire carrier of substrates. Camera-based systems can also be used with manually loaded systems because the camera does not need to be attached to a load port. For example, color cameras can operate in ambient light in a manufacturing environment as well as all or most broad-spectrum dedicated lighting, minimizing the lighting requirements of camera-based systems. However, attempted camera-based solutions have often been unsuccessful because edge detection of substrates such as silicon wafers can pose insurmountable challenges for classical image processing algorithms. For example, many substrates are highly specular, causing reflections from both adjacent substrates in the carrier and the fab room to appear in the image. Furthermore, the shape of a substrate's edge can vary. For example, a rounded edge has a distinct glare in the image. In contrast to a rounded edge, a square edge on a substrate appears black. Furthermore, fiducials on the substrate, used to identify the substrate's crystallographic orientation, can appear anywhere in the image. Also, as mentioned above, the slots in the substrate carrier are designed with a significant amount of tolerance (i.e., the slots are generally manufactured to be overly large), and as a result, the large slots make it difficult to rely on the edge of the substrate being in the same position on each load.

[0015] As disclosed herein, a deep convolutional artificial neural network (identified herein as a deep convolutional system or convnet for notational simplicity) can classify carrier types and all substrates contained therein. A generalized deep convolutional system is described in detail below. However, upon reading and understanding the disclosed subject matter, one skilled in the art will recognize that any type of deep convolutional system can be used with the disclosed subject matter described herein. The same or similar type of deep convolutional system can be used to classify the state of each slot in the carrier (e.g., proper loading, double slot insertion, cross slot insertion, missing, etc.). Various embodiments of the disclosed subject matter can scan and characterize an entire substrate carrier (e.g., a wafer cassette with 25 wafers) in less than two seconds.

[0016] In its simplest form, a deep convolutional neural network is used to generate a given output for a given input or set of inputs. In this case, an input consisting of several acquired images of substrates in a carrier (e.g., wafers in a cassette) generates an output indicating the classification state of each of the substrates. For example, the classification state of a substrate may include a substrate in each substrate slot in a carrier, including an empty slot, an occupied slot (e.g., a properly loaded slot), a double-loaded slot, a cross-slot insertion, and an protrusion (when a substrate is not fully loaded in a slot). The relationship between the input and the output is obtained by training the deep convolutional neural network. Such training operations are described herein.

[0017] For example, in a particular exemplary embodiment, the deep convolutional neural network may comprise a ResNet-18. This implementation of ResNet has 18 residual blocks in its architecture and may use up to millions of images during the training period of the network. For example, ResNet may use fewer layers (e.g., tens of layers, as opposed to hundreds of layers in other network types) to avoid degradation problems caused by very deep residual networks. The degradation problem in very deep residual networks often converges at a higher error rate, thereby degrading the performance of the network model. In ResNet, the degradation problem is generally ameliorated by introducing residual blocks, where intermediate layers of blocks learn residual functions by referencing the block inputs. Generally, inputs to a layer can be passed to another layer directly or as a shortcut ("skip connection"). The residual function can be thought of as a refinement step in which the input feature map is adjusted for higher quality features. This is compared to simple networks in which each layer is expected to learn a new, separate feature map. If no refinement is required, the hidden layers can be trained to gradually adjust their weights towards zero so that the residual block represents the identity function.

[0018] However, the use of ResNet-18 is just one example. A more generalized example of a neural network is described in detail below with reference to Figures 4A-4D.

[0019] As described above, the disclosed subject matter uses a deep convolutional neural network to treat each substrate as a classification problem. Camera images of the substrate and surrounding area (e.g., substrate carrier) are cropped and / or scaled to trained convnet-sized images that match the original image size used to train the system. In embodiments, the cropped or scaled images are partitioned into one image per substrate slot or images containing several slots (e.g., three or more slots). The image(s) are then processed using the deep convolutional neural network to classify the state of the substrate slot, including empty slot, occupied slot (e.g., properly loaded slot), double-loaded slot, cross-slot insertion, and protrusion (when the substrate is not fully loaded within the slot).

[0020] For example, referring now to FIG. 1A , a substrate carrier 100 usable with various embodiments of the disclosed subject matter is shown. The substrate carrier 100 is often used to store or transport substrates (e.g., silicon wafers) during semiconductor manufacturing operations, such as substrate processing (e.g., deposition and etching operations) or metrology operations (e.g., measuring film thickness and critical dimensions (CD) of planned features to be fabricated on the substrate). The substrate carrier 100 includes a carrier body 103 and several slots 101 into which substrates can be loaded. The carrier body 103 can include a plastic material such as polyetheretherketone (PEEK), polypropylene, or perfluoroalkoxyalkane (PFA). In a specific example, the substrate carrier 100 is a 25-slot cassette into which up to 25 silicon wafers (e.g., 75 mm to 200 mm in diameter) can be loaded.

[0021] FIG. 1B illustrates a front-opening integrated pod (FOUP) 150 usable with various embodiments of the disclosed subject matter. FOUP 150 is typically used to store or transport 300 mm wafers. FOUP 150 is shown to include a FOUP carrier body 151 having several slots 159, a substrate door retainer 153, a FOUP door 155, and a substrate center retainer 157. The substrate door retainer 153 and the substrate center retainer 157 help prevent excessive movement of substrates loaded within the FOUP 150, thereby minimizing or preventing damage to the substrates. While the FOUP serves a similar purpose to the substrate carrier 100 of FIG. 1A, features such as the substrate door retainer 153 and the substrate center retainer 157 enable the FOUP 150 to generally avoid many of the problems associated with improper substrate placement and loading, as discussed below with respect to FIGS. 2A-2D, which can occur with the substrate carrier 100 of FIG. 1A.

[0022] The FOUP 150 is a specialized plastic (e.g., polycarbonate) enclosure designed to securely and safely hold, for example, 300 mm silicon wafers in a controlled environment. The FOUP 150 can allow substrates to be transferred between machines (e.g., processing tools and metrology tools) for processing and measurement, typically by an automated material handling system.

[0023] 2A shows a cross-sectional portion 200 of a substrate carrier (e.g., similar to or the same as substrate carrier 100 of FIG. 1A) with substrates 205 loaded therein. The body 203 of the substrate carrier includes a number of substrate slots 201 (five pairs of slots are shown in this example) into which substrates may be loaded. Three of the substrates 205 are loaded into each appropriate one of the substrate slots 201.

[0024] Figure 2B shows a cross-sectional portion 210 of a substrate carrier with a substrate 205 loaded therein. Figure 2B also shows a "cross-slot inserted" substrate 207 within the substrate carrier. That is, the left edge of the substrate 207 is mounted in the second (from the top) one of the substrate slots 201 on the left side of the substrate carrier, and the right edge of the substrate 207 is mounted in the top-most one of the substrate slots 201 on the right side of the substrate carrier. Due to the cross-slot inserted mounting of the substrate 207, an end effector of a robot (e.g., a transfer robot) or other automated substrate extraction tool would generally not be able to remove the substrate 207 from the carrier due to the angle at which the substrate 207 is oriented.

[0025] 2C shows a cross-sectional portion 220 of a substrate carrier with two of the substrates 205 loaded into their appropriate slots in the substrate slots 201, and a position showing the area 209 where the substrates should be (the substrate is missing). Because the substrate is not loaded in area 209, a robot designed to transport the missing substrate wastes time and effort.

[0026] FIG. 2D shows a cross-sectional portion 230 of a substrate carrier containing double-loaded substrates 211 (two substrates loaded at the same height into a pair of substrate slots; two of the substrates 205 are loaded into their appropriate respective ones of the substrate slots 201). However, similar to the substrate carrier of FIG. 2B , a robot end effector or other automated substrate extraction tool will generally be unable to remove either of the double-loaded substrates 211 from the carrier due to the double loading, or may be able to load the bottom one of the double-loaded substrates 211. The top one of the double-loaded substrates 211 may be at least partially withdrawn, or may simply be improperly loaded into a processing or metrology tool. In other cases, the top one of the double-loaded substrates 211 may be at least partially withdrawn and may later fall off. In either case, the robot will subsequently be unable to return a single one of the double-loaded substrates 211 to the same pair of slots 201.

[0027] In addition to the potential problems described above, any one of the cross-sectional portions 200, 210, 220, 230 of the substrate carrier shown in Figures 2A-2D may have one or more of the substrates 205, 207, 211 not fully loaded into their respective substrate slots 201. That is, one or more of the substrates 205, 207, 211 may protrude from the substrate carrier (i.e., off the page). Thus, even if a robot end effector can attach to a substrate (e.g., substrate 205), the end effector may be attached too far back on the substrate, making proper placement on a process or metrology tool difficult or impossible. Aspects of the disclosed subject matter can also detect protruding substrates, as described below.

[0028] To detect these and other problems identified above, the disclosed subject matter uses a deep convolutional neural network based on several images to classify images captured by an image acquisition system. To avoid having to collect millions of images, transfer learning can be used. In an example, the number of images typically used to train the network can be about 5,000 to about 50,000 images. Training begins with a pre-trained network. In an embodiment, the last 5 to 10 layers are trained using a limited amount of image data. Such a training scheme limits the computational requirements frequently encountered in more complex deep convolutional neural network systems.

[0029] Training can begin with a mix of example images, including images of various types of substrates in various types of carriers, and the relationship of the substrate to the substrate slots in the carrier. The images can include multiple images of a given substrate or multiple images covering multiple slots of a carrier (e.g., three slots to detect cross-slot inserted substrates, even if the image may include the entire substrate carrier). Carrier size can be classified, as can the thickness(es) of the substrate within the carrier. In addition to the classification status described herein, other characteristics of the substrate, such as substrate thickness, substrate curvature, substrate bow, and substrate sag (e.g., from a thinned substrate), can also be noted to adjust the positional location (e.g., pick location) between the robot's end effector and the substrate due to one or more of the substrate characteristics. Characteristics can be noted, for example, to avoid collisions between the robot's end effector and the substrate due to one or more of the substrate characteristics. Thus, training can begin with generalizing the images down to specific substrate types, and then automatically tagging the substrates according to each substrate's classification status (e.g., double-slot inserted, missing, protruding) as described above. The automatic tagging of each substrate is based on the output from a deep convolutional neural network.

[0030] In addition to classification status, automatic tagging may also include an identification (e.g., an ID number) associated with each substrate within a given carrier, and automatic tagging of each of multiple locations within the substrate carrier that are proximate portions of multiple substrates.

[0031] 3 illustrates an example of a substrate mapping system 300 in accordance with an embodiment of the disclosed subject matter. The substrate mapping system 300 can be used to perform one or more of the techniques illustrated and described herein, including, for example, a system for detecting the placement (both accurate and inaccurate) of a substrate within a substrate carrier (such as, for example, the substrate carrier 100 of FIG. 1 ). The substrate mapping system 300 includes at least a portion of a machine learning network.

[0032] The substrate mapping system 300 may first be used in a training mode to train a machine learning network, and then may be used in a normal operation mode, for example, to detect the placement of a substrate within a substrate carrier. In various examples, the training mode may be performed by the manufacturer of the substrate mapping system 300. Data obtained from the training mode may then be used, for example, at a manufacturing facility, to determine the actual classification status in the normal operation mode (e.g., a properly loaded substrate, a cross-loaded substrate, a double-loaded substrate, or an ejected substrate, as described above with reference to FIGS. 2A-2D). An exemplary framework for using data from the substrate mapping system 300 is described below with reference to FIGS. 4A-4D. Examples of methods for using the training mode and the normal operation mode are described below with reference to FIGS. 5A and 5B.

[0033] In the exemplary embodiment, the substrate mapping system 300 is shown to include a substrate carrier 305, an optional light source 301 for illuminating the substrate in the substrate carrier 305, and a camera 303. In various embodiments, both the optional light source 301 and the camera 303 are coupled to a data acquisition and control system 310. In various embodiments, the camera 303 may include two or more cameras. For example, a second camera may be desirable to detect substrate protrusion from the substrate carrier 305. The camera 303 may also include multiple cameras to cover different load ports. For example, one or more cameras for load port A and a second set of one or more cameras for load port B. In various embodiments, a remote data storage and processing unit 330 may also be used. The remote data storage and processing unit 330 may include, for example, one or more instantiations of a server, a cloud processing system, a data warehouse, a general-purpose computing device such as a laptop, a tablet, a smartphone, or a desktop computer.

[0034] The optional light source 301 may include a broadband light source, several primarily monochromatic (e.g., single wavelength) light sources, or a combination of broadband and monochromatic light sources. The optional light source 301 may also include light sources incident on the substrate at one or more angles of incidence, with different polarization states and radiation intensities, etc. The selection of the optional light source 301 may be used based on the particular reflection and / or transmission characteristics of the substrate in the substrate carrier 305. The optional light source 301 may also be configured to work with various films or coatings on the substrate. For example, yellow light (e.g., having a wavelength of approximately 577 nm) may be used with substrates having photoresist coatings to avoid exposing the photoresist. In other embodiments, the wavelength of light may be selected to illuminate the substrate during image capture to avoid causing chemical reactions of one or more films on the substrate. However, in various embodiments, ambient light from within the fab may be sufficient to illuminate the substrate in the substrate carrier 305.

[0035] The camera 303 may include one or more lenses (e.g., there may be a single variable focal length lens or multiple single focal length lenses), an image sensor (e.g., a CCD array, a CMOS-based sensor, an active pixel sensor, or other sensor type), and a camera board with associated circuitry to facilitate image extraction. In one example, the camera 303 is a color camera, which can also aid in carrier detection. A color camera may be desirable because substrate carriers are often a different color than the substrates loaded within the carrier, making a color camera useful. Also, typical networks are often trained on color images, which otherwise poses integration challenges for grayscale images collected from a monochrome camera. However, for known carrier types that use networks trained using grayscale images, a monochrome camera may be used as well. In embodiments, multiple cameras may be used, as described above. For example, two cameras may be used to capture stereo images, which may be useful in making classification status determinations, such as whether a substrate protrudes from the substrate carrier 305.

[0036] The data acquisition and control system 310 is shown to include a central processing unit (CPU) 311, a graphics processing unit (GPU) 319, a field programmable gate array (FPGA) 317 (or other suitable hardware, such as an application-specific integrated circuit (ASIC) or a data processing unit (DPU), and an artificial neural network (ANN)), memory 321, a display 313, input devices 323, and a communication interface 315 (e.g., a high performance network (HPN)).

[0037] The data acquisition and control system 310 may also include front-end circuitry, such as, for example, a transmit signal chain, a receive signal chain, switch circuitry, digital circuitry, analog circuitry, etc. In an embodiment, the transmit signal chain may provide control signals to the optional light source 301. The receive signal chain may receive image signals from the camera 303. The front-end circuitry may be coupled to and controlled by one or more processor circuits, such as a CPU 311, a GPU 319, and an FPGA 317. The CPU 311 may be implemented as one or more multi-core processors. The GPU 319 and FPGA 317 may be used to speed up the processing of image data collected from the camera 303 and the performance of machine learning networks, as described herein. The techniques shown and described herein may be performed, for example, by the CPU 311 working in conjunction with the GPU 319 for faster processing.

[0038] The CPU 311 and GPU 319, as well as other components of the data acquisition and control system 310, may be coupled to memory 321 to execute instructions that, for example, cause the data acquisition and control system 310 to perform one or more of source illumination, image acquisition (described in more detail below), processing, or storage of data related to image acquisition, or to otherwise perform the techniques shown and described herein. The data acquisition and control system 310 may be communicatively coupled to other portions of the substrate mapping system 300, for example, using a wired or wireless version of the communications interface 315.

[0039] Execution of one or more techniques as shown and described herein can be accomplished in the data acquisition and control system 310 or using other processing or storage facilities, such as using a remote data storage and processing unit 330. For example, processing tasks that may be undesirably slow or exceed the capabilities of the data acquisition and control system 310 when performed in the data acquisition and control system 310 may be performed remotely (e.g., on a separate system), e.g., in response to a request from the data acquisition and control system 310. Similarly, storage of imaging data or intermediate data may be accomplished using remote facilities communicatively coupled to the data acquisition and control system 310. The data acquisition and control system 310 may also include a display 313, e.g., for presentation of configuration information or results, and input devices 323, including one or more of a keyboard, trackball, function or soft keys, mouse interface, touch screen, stylus, etc., for receiving operator commands, configuration information, or responses to queries.

[0040] As described above, the data collection and control system 310 may receive one or more images of the substrates in the substrate carrier 305, as well as images of the substrates and their potential positions within the substrate carrier 305 (to determine if there are any missing substrates). The images also include the substrates and their relationships to the substrate slots 201 (see FIG. 2A ) within the substrate carrier. The data collection and control system 310 may perform one or more techniques, as shown and described herein, to classify the substrates and carriers. Additionally, some or all aspects of the data collection and control system 310 may be performed and controlled remotely.

[0041] FIG. 4A illustrates an example preprocessing system 400 that may be used with the substrate mapping system of FIG. 3 in accordance with various embodiments of the disclosed subject matter. Upon reading and understanding the disclosed subject matter, those skilled in the art will recognize that several different deep convolutional neural networks, such as residual neural networks (ResNets such as ResNet-18 described above), may be used instead of or in addition to the example preprocessing system of FIG. 4A or the example preprocessing systems of FIG. 4C or 4D described below. The camera 303 of FIG. 3 is used to obtain multiple raw images of the substrate in the substrate carrier 305. Each of the images may then be processed simultaneously or sequentially. For example, the raw image 401 of the substrate may be manipulated by various techniques. In one example, the raw image may be manipulated using a transform technique to form a transformed image in an abstract Hilbert space. For example, the transform may include a Fourier transform, a Laplace transform, or other suitable transform technique. In the case of color images, these processes may be applied to one or a combination of the original colors provided by the camera 303.

[0042] The raw image 401 (or the transformed image) may then be filtered by one or more filters in real space and / or Hilbert space, including linear and nonlinear filters. For example, the raw image 401 (or the transformed image) may be filtered by a first filter 421 (filter 1) to generate a first set of intermediate images 403, 405 (although the example of FIG. 4A shows only two intermediate images, any number of intermediate images may be generated as needed). The first filter 421 may be provided, for example, as one or more linear filters at different bandwidths of light. The first set of intermediate images 403, 405 may then be combined and filtered by a second filter 423 (filter 2) to generate a second intermediate image 407. The second filter 423 may be provided as one or more linear filters or other types of filters. The second intermediate image 407 may be filtered by a third filter 425 (filter 3) to generate a preprocessed image 409. The third filter 425 may be provided as one or more non-linear or other types of filters. Pre-processing may make contrast in the image more pronounced or increase edge detection, thus aiding in detecting and classifying the location of the substrate as described herein.

[0043]

[0023] Figure 4B illustrates an example framework 450 for detecting substrate loading errors in a carrier from images acquired from the substrate mapping system of Figure 3, in accordance with various embodiments of the disclosed subject matter. As discussed above, framework 450 may be used in a training mode to train a machine learning network and then in a normal operation mode to classify substrates in a manufacturing environment. Examples of the training and normal operation modes are described below with reference to Figures 5A and 5B.

[0044] As shown in Figure 4B, framework 450 is shown to include a pre-processor 453 and a machine learning network 460. A raw image 451 is provided to pre-processor 453. In this example, pre-processor 453 filters or otherwise processes raw image 451, e.g., cropping, scaling, or otherwise modifying or enhancing raw image 451, as described above with reference to Figure 4A, to generate a pre-processed image 455.

[0045] The preprocessed image 455 may then be input to the machine learning network 460. The machine learning network 460 may be implemented as a multi-layer machine learning model. For example, the machine learning network 460 may include four layers, including an input layer 459, a feature extraction layer 461, a feature relation layer 463, and a decision layer 465. Pixel information from the preprocessed image 455 may be sent to the input layer 459. Each node in the input layer 459 may correspond to a pixel in the preprocessed image 455. The machine learning network 460 may be trained in one or more of layers 459-465 in an iterative manner. The decision layer 465 may output a decision regarding the classification of a given substrate. A classification result 467 is then generated. The classification result 467 may extract the type of classification detected from the raw image 451. The classification result 467 may provide a text indication of the classification status (e.g., properly loaded substrate, cross-loaded substrate, double-loaded substrate, or protruding substrate). In an embodiment, the classification result 467 may be input as a command instructing the robot to work on a potentially misplaced or missing substrate (e.g., to move further forward on the substrate before applying vacuum if it protrudes from the carrier, to skip the missing substrate slot, etc.).

[0046] As described above, framework 450 may first be used in a training mode to train machine learning network 460 to classify substrate states. Framework 450 may then be used in a normal operating mode to classify substrates in a manufacturing environment. Training of machine learning network 460 may be a supervised process and may be performed off-site from where the classification process is performed. Training may use a set of training images (e.g., one or more training images) with known classification states to train machine learning network 460.

[0047] 4C illustrates an example of a multiple-network preprocessing system 470 that may be used with the substrate mapping system of FIG. 3 in accordance with various embodiments of the disclosed subject matter. Upon reading and understanding the disclosed subject matter, those skilled in the art will recognize that a number of different deep convolutional neural networks, such as a residual neural network (ResNet, such as ResNet-18, described above), may be used in place of or in addition to the example preprocessing system of FIG. 4C. The multiple-network preprocessing system 470 is shown to include a raw image input block 472, a filtering block 474, a convnet substrate carrier size processing block 476, a convnet substrate detection processing block 478, and a decision block 480 that determines when all substrate slots (e.g., 25 slots) in a substrate carrier have been processed.

[0048] The camera 303 in FIG. 3 is used to acquire at least one raw image (or multiple raw images if a multiple network preprocessing system 470 is used for training purposes) of the substrate in the substrate carrier 305. Each of the images can then be processed sequentially (an example of simultaneous processing of images is provided below with reference to FIG. 4D). For example, a raw image of the substrate is provided as input to a raw image input block 472. The raw image can then be manipulated by various techniques. In one example, the raw image is cropped in a crop image block 471 to a rough position according to the physical structure of the substrate carrier. In a scale image block 473, the cropped image is interpolated or extrapolated to the trained convnet size, returning it to the training size used in the original machine learning network. The scaled image is then rescaled in a rescaling block 475 from the floating-point range of 0 to 1 to an integer range. The integer range can be selected to have a given precision value (e.g., as one of 256 discrete steps for 8-bit color depth).

[0049] After the operations in filtering 474 are completed, the filtered image is input to convnet substrate carrier size processing block 476, which performs a determination of the physical size of the substrate carrier (e.g., determining the maximum diameter of the substrate or the number of slots in the carrier). Those skilled in the art, upon reading and understanding the disclosed subject matter, will recognize that the substrate carrier size determination only needs to be performed once per processing of a carrier of substrates. However, the determination may be verified for each loop of operations.

[0050] Convnet substrate detection processing block 478 then determines, based on the original training data, whether a substrate is located within each slot, whether it is a cross-slot insertion, protruding, or double-slot insertion, or any other parameter on which the multiple network pre-processing system 470 was trained. Then, in decision block 480, it is determined whether all substrate slots (e.g., 25 slots) in the substrate carrier have been processed. If there are more substrates to process (e.g., 25 substrates have not been processed), operation returns to raw image input block 472. If all substrates have been processed, operation of the multiple network pre-processing system 470 proceeds to end block 482.

[0051] FIG. 4D illustrates an example of a single-network preprocessing system 490 that may be used with the substrate mapping system of FIG. 3 in accordance with various embodiments of the disclosed subject matter. Upon reading and understanding the disclosed subject matter, those skilled in the art will recognize that several different deep convolutional neural networks, such as residual neural networks (ResNets such as ResNet-18 described above), may be used in place of or in addition to the example preprocessing system of FIG. 4D . However, each of the various components of FIG. 4D may be the same as or similar to the equivalent components, if any, of the multiple-network preprocessing system 470 of FIG. 4C . However, in exemplary operation, the multiple-network preprocessing system 470 of FIG. 4C may function with approximately 10% or less of the total number of training images that may be used with the single-network preprocessing system 490.

[0052] The camera 303 in FIG. 3 is used to acquire at least one raw image (or multiple raw images if the multiple network preprocessing system 490 is used for training purposes) of the substrate in the substrate carrier 305. Each of the images can then be processed simultaneously. For example, a raw image of the substrate is provided as input to a raw image input block 492. The raw image can then be manipulated by various techniques. In one example, the raw image is cropped in a crop image block 491 within a filtering block 494 to a rough position according to the physical structure of the substrate carrier. In a scale image block 493, the cropped image is interpolated or extrapolated to the trained convnet size, returning it to the training size used in the original machine learning network. The scaled image is then rescaled in a rescaling block 495 from the floating-point range of 0 to 1 to an integer range. The integer range can be selected to have a given precision value (e.g., as one of 256 discrete steps for 8-bit color depth).

[0053] After the operations in filtering 494 are completed, the filtered image is input to a convnet substrate carrier size / slot / substrate detection and processing block 496, which determines the physical size of the substrate carrier (e.g., determine the maximum diameter of the substrate or the number of slots in the carrier), and whether the substrate is located in each slot, is a cross-slot insertion, protruding, or double-slot insertion, or any other parameter the single network pre-processing system 490 was trained on based on the original training data. A determination of all substrates in all slots (e.g., all 25 slots) is then output in block 498.

[0054] Referring now to FIG. 5A, an exemplary method 500 for training the substrate mapping system of FIG. 3 is shown, in accordance with various embodiments of the disclosed subject matter. The exemplary method 500 of the training process may be performed multiple times using a set of training images. In operation 501, training images may be received and preprocessed as described above. The training images may correspond to images of the same or similar types of substrate classifications encountered during normal operating modes. The training images may be selected based on the presence of a particular type of substrate classification condition (e.g., a properly loaded substrate, a cross-loaded substrate, a double-loaded substrate, or an ejected substrate). The preprocessing may include a series of filtering operations (at least some of which are described above) to assist in determining the classification of the substrate as described above.

[0055] At operation 509, a label (e.g., a slot number) associated with the preprocessed image may be received. In embodiments, the label may be generated by a manual process. At operation 511, the preprocessed image may be input to a machine learning network (e.g., machine learning network 460) as described herein. At operation 513, a reference output based on the label may be generated and provided to the machine learning network.

[0056] In operation 515, the machine learning network may perform iterative operations until the output of the machine learning network matches or substantially matches the reference output based on the classification type for each of the labels. The labels may be associated with known classification types (e.g., properly loaded substrate, cross-loaded substrate, double-loaded substrate, or ejected substrate). Upon reading and understanding the disclosed subject matter, one skilled in the art will recognize that more than one classification type may apply, such as a cross-loaded substrate ejecting from a substrate carrier or a double-loaded and cross-loaded substrate.

[0057] The exemplary method 500 may be repeated on a set of training images to train a machine learning network. The training images may include different classification types to train the machine learning network to use the classification types to detect different types of substrates (e.g., elemental, compound, bonded substrates, etc., each combined with different diameters and / or thicknesses). After the machine learning network completes the training process, the machine learning network may be used in a normal operating mode, for example, to detect classification types of substrates during a manufacturing or metrology process.

[0058] 5B shows an exemplary method 550 for using the substrate mapping system of FIG. 3 in a normal operating mode, according to various embodiments of the disclosed subject matter. In operation 551, at least one image of a device under inspection is received and may be preprocessed as described above. The image may be captured by camera 303, as described above with reference to FIG. 3. The preprocessing may be a number of filtering techniques, as described above.

[0059] In operation 553, the preprocessed images may be input to a machine learning network, as described herein. In operation 555, the machine learning network (e.g., machine learning network 460 of FIG. 4) may perform an operation to output a decision regarding each substrate classification state from the preprocessed images based on its training.

[0060] The techniques shown and described herein may be performed using part or all of a substrate mapping system 300 as shown in FIG. 3 , or otherwise using a machine 600 as discussed below in connection with FIG. 6 . FIG. 6 shows an example block diagram comprising a machine 600 on which any one or more of the techniques (e.g., methodologies) discussed herein may be performed. In various examples, the machine 600 may operate as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, the machine 600 may operate as a server machine, a client machine, or in both capacities in a server-client network environment. In one example, the machine 600 may function as a peer machine in a peer-to-peer (P2P) (or other distributed) network environment. Machine 600 may be a personal computer (PC), tablet device, set-top box (STB), personal digital assistant (PDA), mobile phone, web appliance, network router, switch, or bridge, or any machine capable of executing instructions (sequential or otherwise) that specify actions to be taken by the machine. Additionally, although only a single machine is illustrated, the term "machine" is intended to include any collection of machines individually or collectively executing a set of instructions (or multiple sets) to perform any one or more of the methodologies discussed herein, such as cloud computing, software as a service (SaaS), other computer cluster configurations, etc.

[0061] Examples described herein may include or operate by logic or multiple components or mechanisms. Circuitry is a collection of circuits implemented in tangible entities including hardware (e.g., simple circuits, gates, logic, etc.). Circuitry membership may be flexible over time and the variability of the underlying hardware. Circuitry includes elements that, alone or in combination, can perform specified operations when in operation. In one example, the hardware of a circuitry may be invariably designed (e.g., hardwired) to perform specific operations. In one example, hardware comprising circuitry may include variably connected physical components (e.g., execution units, transistors, simple circuits, etc.) including computer-readable media that are physically altered (e.g., magnetically, electrically, etc., via a change in physical state or a transformation of another physical property) to encode instructions for specific operations. When connecting the physical components, the underlying electrical properties of the hardware components may be altered, for example, from insulating to conductive properties or vice versa. The instructions enable embedded hardware (e.g., an execution unit or a load mechanism) to create, via variable connections, members of circuitry within the hardware to perform portions of specific operations during operation. Thus, the computer-readable medium is communicatively coupled to other components of the circuitry when the device is operating. In one example, any of the physical components may be used in more than one member of more than one circuitry. For example, during operation, an execution unit may be used in a first circuit of a first circuitry at one time and reused by a second circuit within the first circuitry or by a third circuit within the second circuitry at a different time.

[0062] The machine 600 (e.g., a computer system) may include a hardware processor 602 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a hardware processor core, or any combination thereof), a main memory 604, and a static memory 606, some or all of which may communicate with each other via an interlink 630 (e.g., a bus). The machine 600 may further include a display device 609, an input device 611 (e.g., an alphanumeric keyboard), and a user interface (UI) navigation device 613 (e.g., a mouse). In one example, the display device 609, the input device 611, and the UI navigation device 613 may include at least a portion of a touchscreen display. Machine 600 may additionally include a storage device 620 (e.g., a drive unit), a signal generating device 617 (e.g., a speaker), a network interface device 650, and one or more sensors 615, such as a global positioning system (GPS) sensor, a compass, an accelerometer, or other sensor. Machine 600 may include an output controller 619, such as a serial controller or interface (e.g., universal serial bus (USB)), a parallel controller or interface, or other wired or wireless (e.g., infrared (IR) controller or interface, near field communication (NFC)), coupled to communicate with or control one or more peripheral devices (e.g., a printer, a card reader, etc.).

[0063] Storage device 620 may include a machine-readable medium on which is stored one or more sets of data structures or instructions 624 (e.g., software or firmware) that embody or are utilized by any one or more of the techniques or functions described herein. Instructions 624 may also reside, completely or at least partially, within main memory 603, static memory 605, mass storage device 607, or hardware-based processor 601 during execution thereof by machine 600. In one example, one of hardware-based processor 601, main memory 603, static memory 605, or storage device 620, or any combination thereof, may constitute a machine-readable medium.

[0064] Although the machine-readable medium is considered a single medium, the term "machine-readable medium" may include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) configured to store one or more instructions 624.

[0065] The term "machine-readable medium" may include any medium that can store, encode, or carry instructions for execution by machine 600 and cause machine 600 to perform any one or more of the techniques of this disclosure, or that can store, encode, or carry data structures used by or associated with such instructions. Non-limiting examples of machine-readable media may include solid-state memory, and optical and magnetic media. Thus, a machine-readable medium is not a transitory, propagating signal. Specific examples of large-scale machine-readable media may include semiconductor memory devices (e.g., Electrically Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM)) and non-volatile memory such as flash memory devices, magnetic or other phase-change or state-change memory circuits, magnetic disks such as internal hard disks and removable disks, magneto-optical disks, and CD-ROM and DVD-ROM disks.

[0066] The instructions 624 may further be transmitted or received over the communications network 621 using a transmission medium via a network interface device 650 utilizing any one of several transport protocols (e.g., frame relay, internet protocol (IP), transmission control protocol (TCP), user datagram protocol (UDP), hypertext transfer protocol (HTTP), etc.). Exemplary communication networks may include local area networks (LANs), wide area networks (WANs), packet data networks (e.g., the Internet), mobile phone networks (e.g., cellular networks), Plain Old Telephone (POTS) networks, and wireless data networks (e.g., the Institute of Electrical and Electronics Engineers (IEEE) 802.22 family of standards known as Wi-Fi®, the IEEE 802.26 family of standards known as WiMax®), the IEEE 802.25.4 family of standards, peer-to-peer (P2P) networks, among others. In one example, the network interface device 650 may include one or more physical jacks (e.g., Ethernet, coaxial, or telephone jacks) or one or more antennas for connecting to the communication network 626. In one example, the network interface device 650 may include multiple antennas for wireless communication using at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) techniques.The term "transmission medium" is intended to include any intangible medium capable of storing, encoding, or carrying instructions for execution by machine 600, including digital or analog communication signals or other intangible media to facilitate communication of such software.

[0067] As used herein, the term "or" may be interpreted in an inclusive or exclusive sense. Additionally, other embodiments will be understood by those of ordinary skill in the art based on reading and understanding the disclosure provided. Furthermore, those of ordinary skill in the art will readily understand that the various combinations of techniques and examples provided herein may all be applied in various combinations.

[0068] Throughout this specification, multiple instances may implement components, operations, or structures that are described as a single instance. Although individual operations are illustrated and described as separate operations, one or more of the individual operations may be performed simultaneously, and unless otherwise specified, the operations do not necessarily require that they be performed in the order illustrated. Structures and functions presented as separate components in example configurations may be implemented as combined structures or components. Similarly, structures and functions presented as a single component may be implemented as separate components. These and other variations, modifications, additions, and improvements fall within the scope of the subject matter described herein.

[0069] Further, although not explicitly shown, one skilled in the art will understand that each of the various arrangements, quantities, and numbers of elements may be varied (e.g., the number of cameras, lenses, and illumination sources used). Also, the various wavelengths are provided merely to aid in understanding. Furthermore, each of the examples shown and described herein merely represent one possible configuration and should not be construed as limiting the scope of the present disclosure.

[0070] While various embodiments are discussed separately, these separate embodiments are not intended to be considered independent techniques or designs. As noted above, each of the various portions may be interrelated and each may be used separately or in combination with other embodiments discussed herein. For example, while various embodiments of operations, systems, and processes have been described, these methods, operations, systems, and processes may be used separately or in various combinations.

[0071] Thus, many modifications and variations are possible, as will be apparent to those skilled in the art upon reading and understanding the disclosure provided herein. Functionally equivalent methods and devices within the scope of the present disclosure, in addition to those enumerated herein, will be apparent to those skilled in the art from the foregoing description. Portions and features of some embodiments may be included in, or substituted for, portions and features of other embodiments. Such modifications and variations are intended to fall within the scope of the appended claims. Accordingly, the present disclosure should be limited only by the appended claims, along with the full scope of equivalents to which such claims are entitled. It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting.

[0072] The Abstract of the Disclosure is provided to allow the reader to quickly ascertain the nature of the present technical disclosure. The Abstract is submitted with the understanding that it will not be used to interpret or limit the scope of the claims. In addition, in the foregoing Detailed Description, it is noted that various features may be grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure should not be construed as limiting the scope of the claims. Accordingly, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separate embodiment.

[0073] The description provided herein includes illustrative examples, devices, and apparatuses that embody various aspects of the subject matter described herein. For purposes of explanation, numerous specific details are set forth herein to provide an understanding of various embodiments of the subject matter discussed. However, it will be apparent to those skilled in the art that various embodiments of the disclosed subject matter may be practiced without these specific details. Additionally, well-known structures, materials, and techniques have not been shown in detail so as not to obscure the various illustrated embodiments. As used herein, the terms "about," "approximately," and "substantially" can refer to values ​​that are, for example, within ±10% of a given value or range of values. The following numbered examples are specific embodiments of the disclosed subject matter.

[0074] Example 1: A method for classifying a state of a number of substrates within locations within a substrate carrier. The method includes detecting at least a portion of a substrate within the substrate carrier. The detecting includes capturing one or more images of the portion of the substrate and locations proximate to the portion of the substrate, transferring the one or more images to a pre-trained deep convolutional neural network, classifying a state of the portion of the substrate within the location within the substrate carrier from the images using the pre-trained deep convolutional neural network, and providing automatic tagging of each of a plurality of locations within the substrate carrier proximate to a plurality of portions of the substrate.

[0075] Example 2: The method of example 1, further comprising training a neural network to classify the state of the substrate position.

[0076] Example 3: The method of example 1 or example 2, wherein the automatic tagging includes classifying each of the plurality of locations as at least one classification type selected from types including proper load, cross-slot insertion, double load, protrusion, and empty.

[0077] Example 4: The method of any one of Examples 1 to 3, further comprising identifying each of the plurality of substrate portions for at least one substrate characteristic comprising a characteristic selected from substrate thickness, substrate curvature, substrate warpage, and substrate sag, wherein the at least one characteristic is provided for adjusting the location of a robot end effector using one or more of the plurality of substrates.

[0078] Example 5: The method of any one of Examples 1-4, wherein the classifying further comprises identifying a plurality of substrate types in the substrate carrier from the plurality of substrate portions.

[0079] Example 6: The method of any one of Examples 1 to 5, further comprising selecting a wavelength of the light source used in capturing the one or more images based on at least one of the reflective properties of the substrate in the substrate carrier, the transmissive properties of the substrate in the substrate carrier, and a film or coating on the substrate.

[0080] Example 7: The method of claim 6, further comprising selecting an angle of incidence, a polarization state, and an intensity of the radiation of the light source.

[0081] Example 8: The method of any one of Examples 1-7, further comprising scaling each of the one or more images to the trained convnet size.

[0082] Example 9: A substrate mapping system includes a camera that collects one or more images of a substrate and its potential location within a substrate carrier, and a data collection and control system. The one or more images include the substrate and its location relative to a number of substrate slots within the substrate carrier. The data collection and control system includes one or more hardware-based processors of a machine coupled to the camera and is configured to transfer the one or more images to a deep convolutional neural network, use the deep convolutional neural network to classify from the one or more images a state of the imaged portion of the substrate within the substrate location within the substrate carrier, and provide automatic tagging of each of the substrate locations within the substrate carrier proximate to the imaged portion of the substrate.

[0083] Example 10: The substrate mapping system of example 9, further comprising a light source that illuminates at least some of the substrates and their potential locations within the substrate carrier.

[0084] Example 11: A substrate mapping system as described in Example 10, wherein the light source is a broadband light source.

[0085] Example 12: A substrate mapping system as described in Example 10, wherein the light source is a monochromatic light source.

[0086] Example 13: A substrate mapping system as described in Example 10, wherein the wavelength of light emitted from the light source is selected based on at least one of the reflectance characteristics of the substrate in the substrate carrier, the transmittance characteristics of the substrate in the substrate carrier, and a film or coating on the substrate.

[0087] Example 14: The substrate mapping system of example 10, wherein the one or more characteristics of the light source include a selection of an angle of incidence, a polarization state, and an intensity of the radiation.

[0088] Example 15: A substrate mapping system described in any one of Examples 9 to 14, wherein the automatic tagging includes classifying each of the plurality of locations as at least one classification type selected from types including proper load, cross-slot insertion, double load, protrusion, and empty.

[0089] Example 16: A substrate mapping system described in any one of Examples 9 to 15, wherein the data collection and control system is further configured to identify each of the imaged substrates for at least one substrate characteristic including a characteristic selected from substrate thickness, substrate curvature, substrate warp, and substrate sag, and to transmit the at least one substrate characteristic to the transfer robot to adjust a positional location between an end effector of the transfer robot and a selected one of the substrates according to one or more of the substrate characteristics.

[0090] Example 17: The substrate mapping system of any one of Examples 9-16, further comprising characterizing the size of the substrate carrier.

[0091] Example 18: A substrate mapping system as described in any one of Examples 9 to 17, wherein the system is initially trained in a training mode based on one or more images to produce a pre-trained deep convolutional neural network, and then configured for use in a normal operating mode within a manufacturing facility such that the substrate mapping system detects the placement of substrates within substrate carriers within the manufacturing facility based on the pre-trained deep convolutional neural network.

[0092] Example 19: A substrate mapping system according to any one of Examples 9 to 18, wherein the data collection and control system is further configured to scale each of the one or more images to the trained convnet size.

[0093] Example 20: A computer-readable medium comprising instructions that, when executed by a machine, cause the machine to perform operations including detecting at least a portion of a substrate within a substrate carrier, wherein the detecting includes capturing one or more images of the portion of the substrate and locations proximate the portion of the substrate, transferring the one or more images to a pre-trained deep convolutional neural network, classifying a state of the portion of the substrate within the location within the substrate carrier from the images using the pre-trained deep convolutional neural network, and providing automatic tagging of each of a plurality of locations within the substrate carrier proximate to a plurality of portions of the substrate.

[0094] Example 21: The computer-readable medium of Example 20, further comprising training a neural network to classify the state of the substrate position.

[0095] Example 22: The computer-readable medium of example 20 or example 20, wherein the automatic tagging includes classifying each of the plurality of positions as at least one classification type selected from types including proper load, cross-slot insertion, double load, protrusion, and empty.

[0096] Example 23: A computer-readable medium described in any one of Examples 20 to 22, further comprising identifying each of the portions of the plurality of substrates for at least one substrate characteristic including a characteristic selected from substrate thickness, substrate curvature, substrate warpage, and substrate sag, wherein the at least one characteristic is provided for adjusting the location of a robot end effector using one or more of the plurality of substrates.

[0097] Example 24: The computer-readable medium of any one of Examples 20-23, wherein the classifying further comprises identifying a plurality of substrate types in the substrate carrier from the portions of the plurality of substrates.

[0098] Example 25: A computer-readable medium described in any one of Examples 20 to 24, further comprising selecting a wavelength of a light source used in capturing one or more images based on at least one of the reflective properties of the substrate in the substrate carrier, the transmissive properties of the substrate in the substrate carrier, and a film or coating on the substrate.

[0099] Example 26: The computer-readable medium of Example 25, further comprising selecting an angle of incidence, a polarization state, and an intensity of the light source radiation.

[0100] Example 27: The computer-readable medium of any one of Examples 20-26, further comprising scaling each of the one or more images to the trained convnet size.

Claims

1. 1. A method for classifying the condition of a plurality of substrates at a plurality of locations in a substrate carrier, the method comprising: detecting at least a portion of the plurality of substrates in the substrate carrier, the detecting including capturing one or more images of the portions of the plurality of substrates at the plurality of locations proximate the portions of the plurality of substrates; transferring the one or more images to a pre-trained deep convolutional neural network; classifying the conditions of the portions of the plurality of substrates at the plurality of locations within the substrate carrier from the one or more images using the pre-trained deep convolutional neural network; providing the condition for each of the plurality of locations within the substrate carrier proximate the portions of the plurality of substrates.

2. 2. The method of claim 1, wherein the state includes classifying each of the plurality of positions as at least one classification type selected from types including proper load, cross-slot insertion, double load, protrusion, and empty.

3. 10. The method of claim 1, further comprising identifying each of the portions of the plurality of substrates for at least one substrate characteristic comprising a characteristic selected from substrate thickness, substrate curvature, substrate warp, and substrate sag, wherein the at least one characteristic is provided for adjusting a location of a robot end effector with one or more of the plurality of substrates.

4. 10. The method of claim 1, further comprising selecting a wavelength of a light source used in capturing the one or more images based on at least one of a reflective characteristic of the substrate in the substrate carrier, a transmissive characteristic of the substrate in the substrate carrier, and a film or coating on the substrate.

5. The method of claim 4 further comprising selecting an angle of incidence, a polarization state, and an intensity of the light source radiation.

6. The method of claim 1 , further comprising scaling each of the one or more images to a trained convnet size.

7. 1. A substrate mapping system comprising: a camera that collects one or more images of portions of a plurality of substrates at a plurality of positions within a substrate carrier, the one or more images including a relationship of the portions of the plurality of substrates and the plurality of positions of the substrates relative to a plurality of substrate slots within the substrate carrier; a data acquisition and control system, the data acquisition and control system including one or more hardware-based processors of a machine coupled to the camera, the data acquisition and control system including: transferring the one or more images to a deep convolutional neural network; using the deep convolutional neural network to classify from the one or more images a condition of the portions of the plurality of substrates at the plurality of positions within the substrate carrier; and a substrate mapping system that provides the status of each of the positions of the substrate within the substrate carrier proximate the portion of the substrate that is imaged;

8. 8. The substrate mapping system of claim 7, further comprising a light source that illuminates at least some of the plurality of substrates and the plurality of positions of the substrates within the substrate carrier.

9. 9. The substrate mapping system of claim 8, wherein the light source is at least one of a broadband light source and a monochromatic light source.

10. 9. The substrate mapping system of claim 8, wherein the wavelength of light emitted from the light source is selected based on at least one of a reflective characteristic of the substrate within the substrate carrier, a transmissive characteristic of the substrate within the substrate carrier, and a film or coating on the substrate.

11. 9. The substrate mapping system of claim 8, wherein the one or more characteristics of the light source include a selection of an angle of incidence, a polarization state, and an intensity of the radiation.

12. 8. The substrate mapping system of claim 7, wherein the state includes classifying each of the plurality of locations as at least one classification type selected from types including proper load, cross-slot insertion, double load, protrusion, and empty.

13. said data acquisition and control system comprising: identifying each of the imaged substrates for at least one substrate characteristic, including a characteristic selected from substrate thickness, substrate curvature, substrate bow, and substrate sag; and 8. The substrate mapping system of claim 7, further configured to transmit the at least one substrate characteristic to the transfer robot to adjust a positional location between an end effector of the transfer robot and a selected one of the substrates according to one or more of the substrate characteristics.

14. The substrate mapping system of claim 7 further comprising characterizing a size of the substrate carrier.

15. 8. The substrate mapping system of claim 7, wherein the data acquisition and control system is further configured to scale each of the one or more images to a trained convnet size.

Images