Substrate mapping using deep neural networks

Abstract

To determine the classification status of a substrate (e.g., a wafer) having a substrate carrier (e.g., a wafer cassette). [Solution] Various examples include a system and network for mapping substrates within a substrate carrier (e.g., silicon wafers in a wafer cassette), a classification of the state of each substrate, and a carrier in which the substrates are placed. In the various examples provided herein, an image acquisition system, such as a camera, acquires multiple images of the substrates within the carrier. The images(s) are then processed using a deep convolutional neural network to classify the state of the substrates relative to the substrate slots, including empty slots, occupied slots (e.g., properly loaded slots), double-loaded slots, cross-slot insertions, and protrusions (when the substrate is not fully loaded into the slot).

Description

Technical Field

[0001] (Cross - Reference to Related Applications) This application claims the benefit of priority to U.S. Patent Application No. 17 / 476,195, filed on September 15, 2021, the content of which is hereby incorporated by reference in its entirety.

[0002] (Field of the Invention) The disclosed subject matter generally relates to the field of substrate inspection and measurement tools used in the semiconductor industry and related industries (e.g., flat panel display and solar cell production equipment). More specifically, in various embodiments, the disclosed subject matter relates to a mapping system capable of determining the classification status of substrates (e.g., wafers) having a substrate carrier (e.g., a wafer cassette).

Background Art

[0003] Various types of substrates, such as semiconductor wafers, are typically placed within various types of substrate carriers, such as wafer cassettes, for processing and measurement operations within manufacturing equipment (e.g., integrated circuit manufacturing equipment, etc.). However, prior to starting operations, it is desirable to know whether all substrates within the carrier are properly loaded within the carrier and which substrate slots within the carrier are occupied by substrates.

Summary of the Invention

[0004] This document describes, inter alia, the mapping of substrates within a substrate carrier, the classification of the state of each substrate, and the carrier in which the substrates are placed. In various examples provided herein, an image acquisition system, such as a camera, acquires one or more images of the substrates within the carrier. The images are then processed using a deep convolutional neural network to classify the state of the substrate slots of the carrier, including empty slots, occupied slots (e.g., properly loaded slots), double - loaded slots, cross - slot insertions, and protrusions (when the substrate is not fully loaded within the slot).

[0005] In various embodiments, the subject matter disclosed is a method for classifying the states of several substrates at locations within a substrate carrier. The method includes detecting at least a portion of a substrate within a substrate carrier. Detection includes capturing one or more images of the substrate portion and locations adjacent to the substrate portion, transferring the one or more images to a pre-trained deep convolutional neural network, using the pre-trained deep convolutional neural network to classify the states of the substrate portion at locations within the substrate carrier from the images, and providing automatic tagging of each of multiple locations within the substrate carrier adjacent to multiple substrate portions.

[0006] In various embodiments, the subject matter disclosed is a substrate mapping system. The substrate mapping system includes a camera that collects one or more images of a substrate and the potential locations of the substrate within a substrate carrier, and a data acquisition and control system. The one or more images include the relationship between the substrate and the location of the substrate relative to several substrate slots within the substrate carrier. The data acquisition and control system includes one or more hardware-based processors of a machine coupled to the camera, which transfers the one or more images to a deep convolutional neural network, which uses the deep convolutional neural network to classify the state of the imaged portion of the substrate within the location of the substrate within the substrate carrier from the one or more images, and is configured to provide automatic tagging of each of the locations of the substrate within the substrate carrier adjacent to the imaged portion of the substrate.

[0007] In various embodiments, the disclosed subject matter is a computer-readable medium, which, when executed by a machine, includes instructions causing the machine to perform an operation that includes detecting at least a portion of a substrate in a substrate carrier. The detection includes capturing one or more images of the portion of the substrate and locations adjacent to the portion of the substrate; transferring the one or more images to a pre-trained deep convolutional neural network; using the pre-trained deep convolutional neural network to classify the state of the portion of the substrate in the location within the substrate carrier from the images; and providing automatic tagging of each of the multiple locations within the substrate carrier adjacent to the multiple portions of the substrate.

[0008] The various drawings in the attached drawings are merely illustrative of exemplary implementations of the disclosure and should not be considered to limit its scope. [Brief explanation of the drawing]

[0009] [Figure 1A] The substrate carriers that can be used with various embodiments of the disclosed subject matter are shown. [Figure 1B] A front-opening unified-pod (FOUP) that can be used with various embodiments of the disclosed subject matter is shown. [Figure 2A] This shows a cross-sectional view of a substrate carrier, including a substrate loaded inside the substrate carrier. [Figure 2B] This shows a cross-section of a substrate carrier containing a substrate, where one of the substrates is inserted via a cross-slot within the substrate carrier. [Figure 2C] This shows a cross-section of a substrate carrier, including the location within the carrier where a substrate is missing, along with a substrate that is properly loaded into the carrier. [Figure 2D] This shows a cross-sectional view of a substrate carrier, including a double-loaded substrate, along with a substrate properly loaded within the carrier. [Figure 3] An example of a substrate mapping system according to an embodiment of the disclosed subject matter is shown. [Figure 4A] Examples of pre-processing systems that may be used with the substrate mapping system shown in Figure 3, according to various embodiments of the disclosed subject matter, are shown. [Figure 4B] An example of a framework for detecting substrate load errors in a carrier from images acquired from the substrate mapping system shown in Figure 3, according to various embodiments of the disclosed subject matter, is presented. [Figure 4C] Examples of multiple network preprocessing systems that may be used with the substrate mapping system shown in Figure 3, according to various embodiments of the disclosed subject matter, are shown. [Figure 4D] Examples of single-network preprocessing systems that may be used with the substrate mapping system shown in Figure 3, according to various embodiments of the disclosed subject matter, are shown. [Figure 5A] Exemplary methods for training the substrate mapping system of Figure 3 are shown, according to various embodiments of the disclosed subject matter. [Figure 5B] Exemplary methods for using the substrate mapping system of Figure 3 in normal operating mode are shown according to various embodiments of the disclosed subject matter. [Figure 6] A block diagram of an example machine capable of performing one or more of the techniques (e.g., methodologies) discussed herein is shown. [Modes for carrying out the invention]

[0010] The disclosed subject relates to the detection and mapping of substrates (e.g., silicon wafers) within a substrate carrier (e.g., wafer cassettes or front-opening integrated pods (FOUPs)). Modern substrate-carrier mapping solutions often do not work well in manufacturing environments (e.g., semiconductor fabs). For example, compound fabs have a wide range of substrate diameters (e.g., 75 mm, 100 mm, 125 mm, 150 mm, 200 mm, and 300 mm), substrate types (e.g., silicon, sapphire, gallium arsenide (GaAs), silicon carbide (SiC), and other III-V and II-VI compound semiconductors, bonded wafers, etc.), and substrate thicknesses (e.g., from less than 150 μm to more than 2000 μm). Back-end-of-line (BEOL) processes using bonded substrates can also have various different substrates within a single substrate carrier.

[0011] Currently, various attempts exist in substrate mapping solutions. However, current mapping solutions for substrate carriers (e.g., silicon wafer cassettes) do not work well for, for example, front-end compound manufacturing facilities (fabs) and back-end fabs.

[0012] For example, one approach uses an optical sensor mounted on a robotic end-effector, which is used to remove the substrate from the carrier and place it back into the carrier. While an optical sensor mounted on the end-effector works in certain situations, it is slow and time-consuming to configure because it requires an additional sweep step performed by the robotic handling system.

[0013] Certain other methods use optical sensors mounted on the load port door. However, optical sensors mounted on the load port cannot cope with the variety of substrate types and thicknesses because (1) the optics are in a fixed location and (2) the optics depend on a fixed wavelength of light. Also, most composite, fabs manually load carriers due to space constraints in the fab. Therefore, there are no movable load ports on which sensors can be mounted.

[0014] Certain other methods use a camera to simultaneously observe the entire carrier on the substrate. Camera-based systems can also be used with manually loaded systems because the camera does not need to be mounted in the load port. For example, color cameras have minimal illumination requirements for camera-based systems because they can operate in ambient light in the manufacturing environment and can function with all or most broad-spectrum dedicated illumination. However, camera-based solutions that have been attempted have often been unsuccessful because edge detection of substrates such as silicon wafers can present problems that classical image processing algorithms cannot overcome. For example, many substrates are highly specular, and reflections from both adjacent substrates and the fab chamber within the carrier appear in the image. Furthermore, the shape of the substrate edges can vary. For example, rounded edges have a distinct glare in the image. In contrast to rounded edges, square edges of the substrate appear black. In addition, references on the substrate used to identify the crystal plane orientation of the substrate can appear at arbitrary positions in the image. Furthermore, as mentioned above, the slots within the substrate carrier are designed with a considerable amount of tolerance (i.e., the slots are generally manufactured to be excessively large). As a result, the large slots make it difficult to rely on the substrate edge at the same location on each load.

[0015] As disclosed herein, a deep convolutional artificial neural network (identified herein as a deep convolutional system or convnet for brevity of notation) can classify a carrier type and all substrates contained therein. A generalized deep convolutional type system will be described in detail below. However, based on reading and understanding the disclosed subject matter, one of ordinary skill in the art will recognize that any type of deep convolutional type system can be used with the disclosed subject matter described herein. A deep convolutional type system of the same or similar type can be used to classify the state of each slot in a carrier (e.g., proper load, double slot insertion, cross slot insertion, missing, etc.). Various embodiments of the disclosed subject matter can scan and characterize an entire substrate carrier (e.g., a wafer cassette having 25 wafers) in less than 2 seconds.

[0016] Using a deep convolutional neural network, in its simplest form, for a given input or set of inputs, a given output is produced. In this case, an input consisting of several acquired images of substrates in a carrier (e.g., wafers in a cassette) produces an output indicating the classification state of each of the substrates. For example, the classification state of a substrate can include the substrate in each substrate slot in a carrier, including an empty slot, an occupied slot (e.g., a properly loaded slot), a double loaded slot, a cross slot insertion, and a protrusion (if the substrate is not fully loaded in the slot). The relationship between the input and the output is obtained from training the deep convolutional neural network. Such training operations are described herein.

[0017] For example, in a particular exemplary embodiment, a deep convolutional neural network could comprise ResNet-18. This implementation of ResNet has 18 residual blocks in its architecture and can use up to millions of images during the network's training period. For example, ResNet can use fewer layers (e.g., dozens of layers, as opposed to hundreds of layers in other network types) to avoid the degradation problem caused by very deep residual networks. The degradation problem in very deep residual networks often converges with a higher error rate, thereby degrading the performance of the network model. In ResNet, the degradation problem is generally mitigated by introducing residual blocks in which the intermediate layers of the block learn residual features by referencing the block input. Generally, inputs to a layer can be passed to another layer directly or as shortcuts ("skip connections"). The residual function can be thought of as a refinement step in which the input feature map is tuned to higher quality features. This is compared to a simple network where each layer is expected to learn a new, distinct feature map. If refinement is not required, the intermediate layers can be trained to gradually adjust their weights toward zero so that the residual blocks represent the identity function.

[0018] However, the use of ResNet-18 is just one example. More generalized examples of neural networks are described in detail below with reference to Figures 4A to 4D.

[0019] As described above, the disclosed subject matter uses deep convolutional neural networks and treats each substrate as a classification problem. Camera images of the substrate and surrounding area (e.g., substrate carrier) are trimmed and / or scaled to an image of the trained convnet size that matches the original image size used to train the system. In embodiments, the trimmed or scaled image is split as one image per substrate slot or as an image that includes several slots (e.g., three or more slots). The image(s) are then processed using a deep convolutional neural network to classify the state of the substrate slots, including empty slots, occupied slots (e.g., properly loaded slots), double-loaded slots, cross-slot insertions, and protrusions (if the substrate is not fully loaded within the slot).

[0020] For example, referring now to FIG. 1A, a substrate carrier 100 that can be used with various embodiments of the disclosed subject matter is shown. Substrate carrier 100 is often used to store or transport a substrate (e.g., a silicon wafer) during semiconductor manufacturing operations such as substrate processing (e.g., deposition and etching operations) or metrology operations (e.g., measurement of film thickness and critical dimension (CD) of planned features fabricated on the substrate). Substrate carrier 100 includes a carrier body 103 and several slots 101 into which a substrate can be loaded. Carrier body 103 can include a plastic material such as polyether ether ketone (PEEK), polypropylene, or perfluoroalkoxy alkane (PFA). In a specific example, substrate carrier 100 is a 25-slot cassette in which up to 25 silicon wafers (e.g., having a diameter of 75 mm to 200 mm) can be loaded.

[0021] Figure 1B shows a front-opening integrated pod (FOUP) 150 that can be used with various embodiments of the disclosed subject matter. The FOUP 150 is typically used for storing or transporting 300 mm wafers. The FOUP 150 is shown to include a FOUP carrier body 151 having several slots 159, a substrate door retainer 153, a FOUP door 155, and a substrate center retainer 157. The substrate door retainer 153 and the substrate center retainer 157 help to hold the substrate loaded into the FOUP 150 so as not to move excessively, thereby minimizing or preventing damage to the substrate. The FOUP is used for similar purposes as the substrate carrier 100 in Figure 1A, but features such as the substrate door retainer 153 and the substrate center retainer 157 allow the FOUP 150 to largely prevent many of the problems related to improper substrate placement and loading that can occur with the substrate carrier 100 in Figure 1A, as described later with reference to Figures 2A-2D.

[0022] The FOUP150 is a special plastic (e.g., polycarbonate) enclosure designed to securely and reliably hold, for example, 300mm silicon wafers in a controlled environment. The FOUP150 can typically enable the transfer of substrates between machines (e.g., processing tools and measurement tools) for processing and measurement by an automated material handling system.

[0023] Figure 2A shows a cross-sectional portion 200 of a substrate carrier (similar to or the same as, for example, the substrate carrier 100 in Figure 1A) containing substrates 205 loaded within the substrate carrier. The body 203 of the substrate carrier includes several substrate slots 201 (five pairs of slots are shown in this example) into which substrates can be loaded. Three of the substrates 205 are loaded into the appropriate substrate slots of the substrate slots 201.

[0024] Figure 2B shows a cross-sectional portion 210 of a substrate carrier containing a substrate 205 loaded within the substrate carrier. Figure 2B also shows a "cross-slot inserted" substrate 207 within the substrate carrier. That is, the left end of the substrate 207 is mounted in the second (from the top) of the left-side substrate slots 201 of the substrate carrier, and the right end of the substrate 207 is mounted in the top-side substrate slots 201 of the substrate carrier. The cross-slot inserted mounting of the substrate 207 will generally prevent an end effector of a robot (e.g., a transfer robot) or other automated substrate extraction tool from removing the substrate 207 from the carrier due to the angle at which the substrate 207 is oriented.

[0025] Figure 2C shows a cross-sectional portion 220 of a substrate carrier containing two of the substrates 205 loaded into the appropriate slots of the substrate slot 201, and a location indicating the area 209 where the substrate should be located (where the substrate is missing). Since the substrate is not loaded into area 209, a robot designed to transport the missing substrate wastes time and effort.

[0026] Figure 2D shows a cross-sectional portion 230 of a substrate carrier containing a double-loaded substrate 211 (two substrates are loaded into a pair of substrate slots at the same height; two of the substrates 205 are loaded into their respective appropriate substrate slots in the substrate slots 201). However, as with the substrate carrier in Figure 2B, the robot's end effector or other automated substrate extraction tool may generally be unable to remove either of the double-loaded substrates 211 from the carrier due to the double-loading, or may be able to load the bottom substrate of the double-loaded substrate 211. The top substrate of the double-loaded substrate 211 may be at least partially pulled out, or simply improperly loaded into the processing or measurement tool. In other cases, the top substrate of the double-loaded substrate 211 may be at least partially pulled out and subsequently fall. In any case, the robot is then unable to return a single substrate of the double-loaded substrate 211 to the same pair of slots 201.

[0027] In addition to the problems described above, in any one of the cross-sectional portions 200, 210, 220, and 230 of the substrate carrier shown in Figures 2A to 2D, one or more of the substrates 205, 207, and 211 may not be fully loaded into each of the substrate slots 201. That is, one or more of the substrates 205, 207, and 211 may protrude from the substrate carrier (i.e., outside the drawing page). Therefore, even if the robot's end effector can be mounted on a substrate (e.g., substrate 205), the end effector may be mounted on a substrate very far back on the substrate, making proper placement on the process or measurement tool difficult or impossible. Embodiments of the subject matter disclosed can also detect protruding substrates among the substrates, as described below.

[0028] To detect these and other problems identified above, the disclosed subject uses a deep convolutional neural network based on a number of images to classify images captured by an image acquisition system. Transfer learning may be used to avoid having to collect millions of images. In the example, the number of images typically used to train the network may be about 5,000 to about 50,000 images. Training begins with a pre-trained network. In the embodiment, the last 5 to 10 layers are trained using a limited amount of image data. Such a training method limits the computational requirements frequently encountered in more complex deep convolutional neural network systems.

[0029] Training can begin with a mixture of exemplary images containing images of various types of substrates within various types of carriers, and the relationship between the substrates and the substrate slots within the carrier. The images may include multiple images of a given substrate, or multiple images covering multiple slots in the carrier (e.g., three slots for detecting cross-slot inserted substrates, even if the image may include the entire substrate carrier). Carrier size can be classified as well as the thickness(s) of the substrates within the carrier. In addition to the classification statuses described herein, other substrate characteristics such as substrate thickness, curvature, warp, and substrate sagging (e.g., from thinned substrates) may be considered to adjust the positional location (e.g., pick location) between the robot's end effector and the substrate, due to one or more of these substrate characteristics. These characteristics may be considered, for example, to avoid collisions between the robot's end effector and the substrate due to one or more of these substrate characteristics. Therefore, training can begin with generalizing images down to a specific substrate type, and then automatically tag the substrates according to the classification status of each substrate as described above (e.g., double-slot inserted, missing, protruding). The automatic tagging of each board is based on the output from a deep convolutional neural network.

[0030] In addition to classification status, automatic tagging may also include an identification (e.g., an ID number) associated with each substrate within a given carrier, and automatic tagging of each of multiple locations within the substrate carrier adjacent to multiple substrate portions.

[0031] Figure 3 shows an example of a substrate mapping system 300 according to an embodiment of the subject matter disclosed. The substrate mapping system 300 can be used to perform one or more techniques illustrated and described herein, and includes, for example, a system for detecting the arrangement (both accurate and inaccurate) of substrates within a substrate carrier (e.g., substrate carrier 100 in Figure 1). The substrate mapping system 300 includes at least a portion of a machine learning network.

[0032] The substrate mapping system 300 may first be used in training mode to train a machine learning network, and then in normal operation mode to detect, for example, the placement of substrates within a substrate carrier. In various examples, the training mode may be performed by the manufacturer of the substrate mapping system 300. The data obtained from the training mode may then be used, for example, in the manufacturing equipment to determine the actual classification state in normal operation mode (e.g., properly loaded substrates, cross-loaded substrates, double-loaded substrates, or protruding substrates, as described above with reference to Figures 2A to 2D). An exemplary framework for using data from the substrate mapping system 300 is described below with reference to Figures 4A to 4D. Examples of how to use the training mode and normal operation mode are described later with reference to Figures 5A and 5B.

[0033] In exemplary embodiments, the substrate mapping system 300 is shown to include a substrate carrier 305, an optional light source 301 for illuminating the substrates within the substrate carrier 305, and a camera 303. In various embodiments, both the optional light source 301 and the camera 303 are coupled to a data acquisition and control system 310. In various embodiments, the camera 303 may include two or more cameras. For example, a second camera may be desirable to detect substrate protrusions from the substrate carrier 305. The camera 303 may also comprise multiple cameras to cover different load ports. For example, a second set of one or more cameras for load port A and one or more cameras for load port B. In various embodiments, a remote data storage and processing unit 330 may also be used. The remote data storage and processing unit 330 may include one or more instances of general-purpose computing devices such as servers, cloud processing systems, data warehouses, laptops, tablets, smartphones, or desktop computers.

[0034] The optional light source 301 may include a broadband light source, several primarily monochromatic (e.g., single-wavelength) light sources, or a combination of a broadband light source and a monochromatic light source. The optional light source 301 may also include light sources incident on the substrate at one or more incident angles, with different polarization states and radiant intensities, etc. The selection of the optional light source 301 may be used based on the specific reflectivity and / or transmission characteristics of the substrate in the substrate carrier 305. The optional light source 301 may also be configured to cooperate with various films or coatings on the substrate. For example, yellow light (e.g., having a wavelength of approximately 577 nm) may be used with a substrate having a photoresist coating to avoid exposure of the photoresist. In other embodiments, the wavelength of light may be selected to illuminate the substrate during image acquisition to avoid causing chemical reactions in one or more films on the substrate. However, in various embodiments, ambient light from within the fab may be sufficient to illuminate the substrate in the substrate carrier 305.

[0035] Camera 303 may comprise one or more lenses (e.g., a single variable focal length lens or multiple single focal length lenses), an image sensor (e.g., a CCD array, a CMOS-based sensor, an active pixel sensor, or other sensor type), and a camera substrate having associated circuitry for facilitating image extraction. In one example, camera 303 is a color camera, which can also assist in carrier detection. A color camera may be desirable because substrate carriers are often of a different color than the substrate loaded within the carrier, making a color camera useful. Also, general networks are often trained on color images, and if not, this creates an integration challenge with grayscale images collected from a monochrome camera. However, for known carrier types that use networks trained with grayscale images, a monochrome camera can also be used. In embodiments, multiple cameras may be used as described above. For example, two cameras can be used to capture a stereo image, which may be useful in determining classification status, such as whether a substrate is protruding from the substrate carrier 305.

[0036] The data acquisition and control system 310 is shown to include a central processing unit (CPU) 311, a graphics processing unit (GPU) 319, a field programmable gate array (FPGA) 317 (or other suitable hardware, such as an application-specific integrated circuit (ASIC) or data processing unit (DPU), and an artificial neuron network (ANN)), memory 321, a display 313, an input device 323, and a communication interface 315 (e.g., a high-performance network (HPN)).

[0037] The data acquisition and control system 310 may also include front-end circuits such as, for example, a transmit signal chain, a receive signal chain, a switch circuit, a digital circuit, and an analog circuit. In an embodiment, the transmit signal chain may provide control signals to an optional light source 301. The receive signal chain may receive image signals from the camera 303. The front-end circuit configuration may be coupled to and controlled by one or more processor circuits such as a CPU 311, a GPU 319, and an FPGA 317. The CPU 311 may be implemented as one or more multi-core processors. The GPU 319 and FPGA 317 may be used to accelerate the processing of image data acquired from the camera 303 and the performance of a machine learning network, as described herein. Techniques illustrated and described herein may be performed, for example, by the CPU 311 working with the GPU 319 for faster processing.

[0038] The CPU 311 and GPU 319, as well as other components of the data acquisition and control system 310, may be coupled to the memory 321, for example, to execute instructions causing the data acquisition and control system 310 to perform one or more of the following: light source illumination, image acquisition (described in more detail below), processing, or storage of data related to image acquisition, or to perform techniques illustrated and described herein in a different manner. The data acquisition and control system 310 may be coupled to communicate with other parts of the substrate mapping system 300, for example, using a wired or wireless version of the communication interface 315.

[0039] The execution of one or more techniques as illustrated and described herein can be achieved in the data acquisition and control system 310 or using other processing or storage equipment, for example, using a remote data storage and processing unit 330. For example, processing tasks that would be undesirably slow or exceed the capabilities of the data acquisition and control system 310 can be performed remotely (for example, on a separate system) in response to a request from the data acquisition and control system 310. Similarly, the storage of imaging data or intermediate data can be achieved using remote equipment communicatively coupled to the data acquisition and control system 310. The data acquisition and control system 310 may also include, for example, a display 313 for presenting configuration information or results, and an input device 323 for receiving operator commands, configuration information, or responses to queries, including one or more of the following: a keyboard, trackball, function keys or soft keys, mouse interface, touchscreen, stylus, etc.

[0040] As described above, the data acquisition and control system 310 may receive one or more images of the substrates in the substrate carrier 305, as well as images of the substrates and their potential locations within the substrate carrier 305 (to determine whether any missing substrates are present). The images also include the relationship between the substrates and their potential locations relative to the substrate slots 201 in the substrate carrier (see Figure 2A). The data acquisition and control system 310 may perform one or more techniques, as illustrated and described herein, to classify the substrates and carriers. Furthermore, some or all embodiments of the data acquisition and control system 310 may be performed and controlled remotely.

[0041] Figure 4A shows an example of a preprocessing system 400 that may be used with the substrate mapping system of Figure 3, according to various embodiments of the disclosed subject matter. Those skilled in the art will recognize, upon reading and understanding the disclosed subject matter, that several different deep convolutional neural networks, such as residual neural networks (ResNets such as the ResNet-18 described above), may be used instead of, or in addition to, the exemplary preprocessing system of Figure 4A, or the exemplary preprocessing system of Figure 4C or Figure 4D, as described below. The camera 303 of Figure 3 is used to obtain a number of raw images of the substrate in the substrate carrier 305. Each of the images may then be processed simultaneously or sequentially. For example, the raw images 401 of the substrate may be manipulated by various techniques. In one example, the raw images may be manipulated using transformation techniques to form a transformed image in an abstract Hilbert space. For example, the transformation may include the Fourier transform, the Laplace transform, or other preferred transformation techniques. In the case of color images, these processes may be applied to one or a combination of the original colors provided by the camera 303.

[0042] Next, the raw image 401 (or transformed image) may be filtered by one or more filters in real space and / or Hilbert space, including linear and nonlinear filters. For example, the raw image 401 (or transformed image) may be filtered by a first filter 421 (filter 1) to produce a first set of intermediate images 403, 405 (the example in Figure 4A shows only two intermediate images, but any number of intermediate images may be produced as needed). The first filter 421 may be provided, for example, as one or more linear filters in different bandwidths of light. The first set of intermediate images 403, 405 may then be combined and filtered by a second filter 423 (filter 2) to produce a second intermediate image 407. The second filter 423 may be provided as one or more linear filters or other types of filters. The second intermediate image 407 may be filtered by a third filter 425 (filter 3) to produce a preprocessed image 409. The third filter 425 may be provided as one or more nonlinear filters or other types of filters. Preprocessing may enhance contrast in the image or increase edge detection, and therefore assist in detecting and classifying the location of the substrate as described herein.

[0043] Figure 4B shows an example of a framework 450 that detects substrate load errors in a carrier from images acquired from the substrate mapping system of Figure 3, according to various embodiments of the disclosed subject matter. As described above, the framework 450 may be used in training mode to train a machine learning network and then in normal operation mode to classify substrates in a manufacturing environment. Examples of training mode and normal operation mode will be described later with reference to Figures 5A and 5B.

[0044] As shown in Figure 4B, the framework 450 is shown to include a preprocessor 453 and a machine learning network 460. A raw image 451 is provided to the preprocessor 453. In this example, the preprocessor 453 filters or otherwise processes the raw image 451, as described above with reference to Figure 4A, for example, by cropping, scaling, or otherwise modifying or enhancing the raw image 451 to produce a preprocessed image 455.

[0045] Next, the preprocessed image 455 can be input to the machine learning network 460. The machine learning network 460 may be provided as a multi-layer machine learning model. For example, the machine learning network 460 may include four layers, including an input layer 459, a feature extraction layer 461, a feature relation layer 463, and a decision layer 465. Pixel information from the preprocessed image 455 can be sent to the input layer 459. Each node in the input layer 459 may correspond to a pixel in the preprocessed image 455. The machine learning network 460 may be trained iteratively in one or more of the layers 459-465. The decision layer 465 may output a decision regarding the classification of a given substrate. Then, a classification result 467 is generated. The classification result 467 may extract the type of classification detected from the raw image 451. The classification result 467 may provide a text indication of the classification state (e.g., properly loaded substrate, cross-loaded substrate, double-loaded substrate, or protruding substrate). In the embodiment, the classification result 467 may be input as a command instructing the robot to operate on potentially misplaced or missing substrates (e.g., to move further forward on the substrate before applying vacuum if it is protruding from the carrier, to skip missing substrate slots, etc.).

[0046] As described above, framework 450 can first be used in training mode to train the machine learning network 460 to classify the state of the substrate. Framework 450 can then be used in normal operation mode to classify the substrate in the manufacturing environment. Training of the machine learning network 460 can be a supervised process and can be performed offsite where the classification process is performed. Training can use a set of training images (e.g., one or more training images) with known classification states to train the machine learning network 460.

[0047] Figure 4C shows an example of a multi-network preprocessing system 470 that may be used with the substrate mapping system of Figure 3, according to various embodiments of the disclosed subject matter. Upon reading and understanding the disclosed subject matter, those skilled in the art will recognize that several different deep convolutional neural networks, such as residual neural networks (ResNet, such as ResNet-18 described above), may be used instead of or in addition to the exemplary preprocessing system of Figure 4C. The multi-network preprocessing system 470 is shown to include a raw image input block 472, a filtering block 474, a convnet substrate carrier size processing block 476, a convnet substrate detection processing block 478, and a determination block 480 that determines when all substrate slots (e.g., 25 slots) in the substrate carrier have been processed.

[0048] The camera 303 in Figure 3 is used to acquire at least one raw image (or multiple raw images if the multi-network preprocessing system 470 is used for training purposes) of the substrate in the substrate carrier 305. Each of the images can then be processed sequentially (an example of simultaneous processing of images is provided below with reference to Figure 4D). For example, a raw image of the substrate is provided as input to the raw image input block 472. The raw image can then be manipulated by various techniques. In one example, the raw image is trimmed to a rough position according to the physical structure of the substrate carrier in the trimmed image block 471. In the scaled image block 473, the trimmed image is interpolated or extrapolated to the trained convnet size, relating it back to the training size used in the original machine learning network. The scaled image is then rescaled in the rescaling block 475 from a floating-point range of 0 to 1 to an integer range. The integer range can be selected to have a given precision value (for example, as one of 256 discrete steps for an 8-bit color depth).

[0049] After the operation within filtering 474 is complete, the filtered image is input to convnet substrate carrier size processing block 476, which determines the physical size of the substrate carrier (e.g., the maximum diameter of the substrate or the number of slots in the carrier). Those skilled in the art will recognize, upon reading and understanding the disclosed subject matter, that the determination of the substrate carrier size needs to be performed only once for each processing of the substrate carrier. However, the determination may be verified for each loop of the operation.

[0050] Next, the convnet substrate detection processing block 478 determines, based on the original training data, whether the substrate is located within each slot, is a cross-slot insertion, protruding, or a double-slot insertion, or any other parameters for which the multi-network preprocessing system 470 was trained. Then, in the determination block 480, it is determined whether all substrate slots in the substrate carrier (e.g., 25 slots) have been processed. If there are still substrates to be processed (e.g., 25 substrates have not been processed), the operation returns to the raw image input block 472. If all substrates have been processed, the operation of the multi-network preprocessing system 470 proceeds to the termination block 482.

[0051] Figure 4D shows an example of a single-network preprocessing system 490 that may be used with the substrate mapping system of Figure 3, according to various embodiments of the disclosed subject matter. Those skilled in the art, upon reading and understanding the disclosed subject matter, will recognize that several different deep convolutional neural networks, such as residual neural networks (ResNets such as the ResNet-18 described above), may be used instead of, or in addition to, the exemplary preprocessing system of Figure 4D. However, each of the various components of Figure 4D may be the same as or similar to the equivalent components of the multi-network preprocessing system 470 of Figure 4C, if any. However, in exemplary operation, the multi-network preprocessing system 470 of Figure 4C may function with approximately 10% or less of the total number of training images that may be used with the single-network preprocessing system 490.

[0052] The camera 303 in Figure 3 is used to acquire at least one raw image (or multiple raw images if the multi-network preprocessing system 490 is used for training purposes) of the substrate in the substrate carrier 305. Each of the images can then be processed simultaneously. For example, a raw image of the substrate is provided as input to the raw image input block 492. The raw image can then be manipulated by various techniques. In one example, the raw image is trimmed to a rough position according to the physical structure of the substrate carrier in the trimming image block 491 within the filtering block 494. In the scaling image block 493, the trimmed image is interpolated or extrapolated to the trained convnet size, relating it back to the training size used in the original machine learning network. The scaled image is then rescaled in the rescaling block 495 from a floating-point range of 0 to 1 to an integer range. The integer range can be selected to have a given precision value (for example, as one of 256 discrete steps for an 8-bit color depth).

[0053] After the operation in filtering 494 is complete, the filtered image is input to convnet substrate carrier size / slot / substrate detection and processing block 496, which determines the physical size of the substrate carrier (e.g., the maximum diameter of the substrate or the number of slots in the carrier), as well as whether the substrate is located in each slot, is cross-slot inserted, protruding, or double-slot inserted, or any other parameters for which the single-network preprocessing system 490 was trained, based on the original training data. Next, in block 498, the determination of all substrates in all slots (e.g., all 25 slots) is output.

[0054] Referring here to Figure 5A, an exemplary method 500 for training the substrate mapping system of Figure 3, according to various embodiments of the disclosed subject matter, is shown. The exemplary method 500 of the training process may be performed multiple times using a set of training images. In operation 501, training images are received and may be preprocessed as described above. The training images may correspond to images of the same or similar types of substrate classification encountered during normal operation modes. The training images may be selected based on the presence of a particular type of substrate classification state (e.g., properly loaded substrate, crossloaded substrate, double-loaded substrate, or protruding substrate). The preprocessing may include a series of filtering operations (at least some of which are described above) to assist in determining the classification of the substrate as described above.

[0055] In operation 509, a label (e.g., slot number) associated with the preprocessed image may be received. In the embodiment, the label may be generated by a manual process. In operation 511, the preprocessed image may be input to a machine learning network (e.g., machine learning network 460) as described herein. In operation 513, a reference output based on the label may be generated and provided to the machine learning network.

[0056] In operation 515, the machine learning network may perform iterative operations until the output of the machine learning network matches, or substantially matches, a reference output based on the classification type for each of the labels. Labels may be associated with known classification types (e.g., well-loaded substrate, cross-loaded substrate, double-loaded substrate, or protruding substrate). Upon reading and understanding the disclosed subject matter, a person skilled in the art will recognize that two or more classification types may apply, such as a cross-loaded substrate or a double-loaded and cross-loaded substrate that protrudes from the substrate carrier.

[0057] The exemplary method 500 may be repeated on a set of training images to train a machine learning network. The training images may include different classification types to train the machine learning network to detect different types of substrates (e.g., elements, compounds, bonded substrates, each with different diameters and / or thicknesses) using classification types. After the machine learning network has completed the training process, it can be used in normal operating mode to detect the classification type of a substrate, for example, during a manufacturing or measurement process.

[0058] Figure 5B shows an exemplary method 550 for using the substrate mapping system of Figure 3 in normal operating mode according to various embodiments of the disclosed subject matter. In operation 551, at least one image of the device under inspection is received and may be preprocessed as described above. The image may be captured by camera 303 as described above with reference to Figure 3. The preprocessing may be several filtering techniques as described above.

[0059] In operation 553, the preprocessed image may be input to a machine learning network as described herein. In operation 555, the machine learning network (e.g., machine learning network 460 in Figure 4) may, based on its training, perform an operation to output a determination regarding each substrate classification state from the preprocessed image.

[0060] The techniques illustrated and described herein may be performed using a portion or all of a substrate mapping system 300, as shown in Figure 3, or using a machine 600, as otherwise discussed below in relation to Figure 6. Figure 6 shows an exemplary block diagram of a machine 600 on which one or more of the techniques (e.g., methodologies) discussed herein may be performed. In various examples, the machine 600 may operate as a standalone device or be connected to other machines (e.g., networked). In a networked deployment, the machine 600 may operate as a server machine, a client machine, or both in a server-client network environment. In one example, the machine 600 may function as a peer machine in a peer-to-peer (P2P) (or other distributed) network environment. Machine 600 could be a personal computer (PC), a tablet device, a set-top box (STB), a personal digital assistant (PDA), a mobile phone, a web appliance, a network router, a switch or bridge, or any machine capable of executing instructions (sequential or otherwise) that specify actions to be taken by that machine. Furthermore, although only a single machine is illustrated, the term “machine” includes any set of machines that individually or collectively execute a set (or set) of instructions in order to perform one or more of the methodologies considered herein, such as cloud computing, software as a service (SaaS), or other computer cluster configurations.

[0061] The examples described herein may include or operate by logic or a number of components or mechanisms. A circuit configuration is a collection of circuits implemented on a tangible entity that includes hardware (e.g., simple circuits, gates, logic, etc.). Circuit configuration membership may be flexible with respect to time and the variability of the underlying hardware. A circuit configuration includes components that, individually or in combination, can perform a specified operation when in operation. In one example, the hardware of a circuit configuration may be designed (e.g., wired) immutably to perform a particular operation. In one example, hardware comprising a circuit configuration may include variably connected physical components (e.g., execution units, transistors, simple circuits, etc.) including a computer-readable medium that is physically modified (e.g., magnetically, electrically, etc., through changes in physical states or conversions of other physical properties, etc.) to encode instructions for a particular operation. When connecting physical components, the underlying electrical properties of the hardware components may be changed, for example, from insulating properties to conductive properties, or vice versa. Instructions enable embedded hardware (e.g., an execution unit or load mechanism) to create members of a circuit configuration within the hardware via variable connections to perform a specific part of an operation during operation. Thus, computer-readable media are communicatively coupled to other components of the circuit configuration while the device is operating. In one example, any of the physical components may be used by two or more members of two or more circuit configurations. For instance, during operation, an execution unit may be used at one point in time by a first circuit in a first circuit configuration, and then reused by a second circuit within the first circuit configuration, or at a different time by a third circuit within the second circuit configuration.

[0062] The machine 600 (e.g., a computer system) may include a hardware processor 602 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a hardware processor core, or any combination thereof), main memory 604, and static memory 606, some or all of which may communicate with each other via an interlink 630 (e.g., a bus). The machine 600 may further include a display device 609, an input device 611 (e.g., an alphanumeric keyboard), and a user interface (UI) navigation device 613 (e.g., a mouse). In one example, the display device 609, the input device 611, and the UI navigation device 613 may include at least a portion of a touchscreen display. The machine 600 may additionally include a memory device 620 (e.g., a drive unit), a signal generating device 617 (e.g., a speaker), a network interface device 650, and one or more sensors 615 such as a global positioning system (GPS) sensor, a compass, an accelerometer, or other sensors. The machine 600 may also include an output controller 619 coupled to communicate or control one or more peripheral devices (e.g., a printer, a card reader, etc.), such as a serial controller or interface (e.g., a universal serial bus, USB), a parallel controller or interface, or other wired or wireless (e.g., an infrared (IR) controller or interface, near field communication, NFC).

[0063] The storage device 620 may include a machine-readable medium that stores one or more data structures or instructions 624 (e.g., software or firmware) that embody or utilize any one or more of the techniques or functions described herein. The instructions 624 may also reside entirely or at least partially in the main memory 603, static memory 605, mass storage device 607, or hardware-based processor 601 during their execution by machine 600. In one example, one of the hardware-based processor 601, main memory 603, static memory 605, or storage device 620, or any combination thereof, may constitute the machine-readable medium.

[0064] A machine-readable medium is considered a single medium, but the term “machine-readable medium” may include a single or multiple mediums configured to store one or more instructions 624 (e.g., a centralized or distributed database, and / or associated caches and servers).

[0065] The term “machine-readable medium” may include any medium capable of storing, encoding, or carrying instructions for execution by machine 600, and capable of causing machine 600 to execute one or more of the techniques of the Disclosure, or storing, encoding, or carrying data structures used by or associated with such instructions. Examples of non-limiting machine-readable mediums include solid-state memory, as well as optical and magnetic media. Therefore, machine-readable mediums are not transient propagating signals. Specific examples of large-scale machine-readable mediums include semiconductor memory devices (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)), non-volatile memory such as flash memory devices, magnetic or other phase-change or state-change memory circuits, magnetic disks such as internal hard disks and removable disks, magneto-optical disks, as well as CD-ROM and DVD-ROM disks.

[0066] Instruction 624 may be further transmitted or received via the communication network 621 using a transmission medium via a network interface device 650 that utilizes one of several transmission protocols (e.g., Frame Relay, Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), etc.). Examples of communication networks include, in particular, local area networks (LANs), wide area networks (WANs), packet data networks (e.g., the Internet), mobile phone networks (e.g., cellular networks), plain old telephone (POTS) networks, and wireless data networks (e.g., the Institute of Electrical and Electronics Engineers (IEEE) 802.22 family of standards known as Wi-Fi®, the IEEE 802.26 family of standards known as WiMAX®), the IEEE 802.25.4 family of standards, and peer-to-peer (P2P) networks. For example, the network interface device 650 may include one or more physical jacks (e.g., Ethernet, coaxial, or telephone jacks) or one or more antennas for connecting to the communication network 626. For example, the network interface device 650 may include multiple antennas for wireless communication using at least one of the following techniques: single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO).The term "transmission medium" includes any intangible medium capable of storing, encoding, or carrying instructions for execution by machine 600, and includes digital or analog communication signals or other intangible medium for facilitating communication of such software.

[0067] As used herein, the term "or" may be interpreted in an inclusive or exclusive sense. Furthermore, other embodiments will be understood by those skilled in the art based on reading and understanding the provided disclosure. Moreover, those skilled in the art will readily understand that all the various combinations of techniques and examples provided herein may be applied in various combinations.

[0068] Throughout this specification, multiple instances may implement a component, operation, or structure described as a single instance. While individual operations are illustrated and described as separate operations, one or more of these operations may be performed simultaneously, and unless otherwise specified, the operations are not necessarily required to be performed in the order illustrated. Structures and functions presented as separate components in exemplary configurations may be implemented as combined structures or components. Similarly, structures and functions presented as single components may be implemented as separate components. These and other variations, modifications, additions, and improvements fall within the scope of the subject matter described herein.

[0069] Furthermore, although not expressly shown, each of the various arrangements, quantities, and numbers of the elements may be changed (e.g., the number of cameras, lenses, and light sources used), as will be understandable to those skilled in the art. Also, various wavelengths are provided merely to aid understanding. Moreover, each of the examples illustrated and described herein merely represents one possible configuration and should not be construed as limiting the scope of this disclosure.

[0070] Although various embodiments are considered separately, these separate embodiments are not intended to be considered as independent techniques or designs. As stated above, each of the various parts may be related to one another and may be used separately or in combination with other embodiments considered herein. For example, various embodiments of operations, systems, and processes have been described, and these methods, operations, systems, and processes may be used separately or in various combinations.

[0071] Accordingly, many modifications and variations are possible, as will be apparent to those skilled in the art by reading and understanding the disclosure provided herein. In addition to those enumerated herein, functionally equivalent methods and devices within the scope of this disclosure will be apparent to those skilled in the art from the foregoing description. Parts and features of some embodiments may be included in or replaced by parts and features of other embodiments. Such modifications and variations are intended to fall within the scope of the appended claims. Accordingly, this disclosure should be limited only by the appended claims and the entire scope of equivalents to which such claims are granted. It should also be understood that the terms used herein are intended solely to describe and not to limit specific embodiments.

[0072] This abstract is provided to enable readers to quickly confirm the nature of this technical disclosure. The abstract is submitted with the understanding that it is not to be used to interpret or limit the claims. In addition, it is evident in the preceding detailed description that, for the purpose of simplifying the disclosure, various features may be grouped together under a single embodiment. The method of disclosure should not be interpreted as limiting the claims. Therefore, the following claims are incorporated into the detailed description, with each claim standing on its own as a distinct embodiment.

[0073] The descriptions provided herein include exemplary examples, devices, and apparatus that embody various aspects of the subject matter described herein. For illustrative purposes, numerous specific details are provided herein to provide an understanding of the various embodiments of the subject matter under consideration. However, it will be apparent to those skilled in the art that various embodiments of the disclosed subject matter can be carried out without these specific details. Furthermore, well-known structures, materials, and techniques are not described in detail so as not to obscure the various embodiments illustrated herein. As used herein, the terms “about,” “approximately,” and “substantially” may refer, for example, to a value that is within ±10% of a given value or range of values. The following numbered embodiments are specific embodiments of the subject matter disclosed.

[0074] Example 1: A method for classifying the state of several substrates at locations within a substrate carrier. The method includes detecting at least a portion of a substrate within a substrate carrier. Detection includes capturing one or more images of the substrate portion and locations adjacent to the substrate portion, transferring the one or more images to a pre-trained deep convolutional neural network, using the pre-trained deep convolutional neural network to classify the state of the substrate portion at locations within the substrate carrier from the images, and providing automatic tagging of each of multiple locations within the substrate carrier adjacent to multiple substrate portions.

[0075] Example 2: The method according to Example 1, further comprising training a neural network to classify the state of the substrate position.

[0076] Example 3: The method according to Example 1 or Example 2, wherein the automatic tagging includes classifying each of a plurality of locations as at least one classification type selected from types including appropriate load, cross-slot insertion, double load, protrusion, and empty.

[0077] Example 4: The method according to any one of Examples 1 to 3, further comprising identifying each of a plurality of substrate portions with respect to at least one substrate characteristic, which includes a characteristic selected from substrate thickness, substrate curvature, substrate warp, and substrate sag, wherein at least one characteristic is provided for adjusting the position of a robot end effector using one or more of the plurality of substrates.

[0078] Example 5: The method according to any one of Examples 1 to 4, further comprising classifying multiple substrate types within a substrate carrier from multiple substrate portions.

[0079] Example 6: The method according to any one of Examples 1 to 5, further comprising selecting the wavelength of a light source used to capture one or more images based on at least one of the reflective properties of the substrate in the substrate carrier, the transmittance properties of the substrate in the substrate carrier, and a film or coating on the substrate.

[0080] Example 7: The method according to claim 6, further comprising selecting the incident angle, polarization state, and intensity of radiation from a light source.

[0081] Example 8: The method according to any one of Examples 1 to 7, further comprising scaling each of one or more images to the size of a trained convnet.

[0082] Example 9: The substrate mapping system includes a camera that collects one or more images of a substrate and the potential locations of the substrate within a substrate carrier, and a data acquisition and control system. The one or more images include the relationship between the substrate and the location of the substrate relative to several substrate slots within the substrate carrier. The data acquisition and control system includes one or more hardware-based processors of the machine coupled to the camera, which transfers the one or more images to a deep convolutional neural network, which uses the deep convolutional neural network to classify the state of the imaged portion of the substrate within the substrate location within the substrate carrier from the one or more images, and is configured to provide automatic tagging of each of the locations of the substrate within the substrate carrier adjacent to the imaged portion of the substrate.

[0083] Example 10: The substrate mapping system according to Example 9, further comprising at least some of the substrates and a light source for illuminating the potential positions of the substrates within the substrate carrier.

[0084] Example 11: The substrate mapping system according to Example 10, wherein the light source is a broadband light source.

[0085] Example 12: The substrate mapping system according to Example 10, wherein the light source is a monochromatic light source.

[0086] Example 13: The substrate mapping system according to Example 10, wherein the wavelength of light emitted from the light source is selected based on at least one of the reflective properties of the substrate in the substrate carrier, the transmittance properties of the substrate in the substrate carrier, and a film or coating on the substrate.

[0087] Example 14: The substrate mapping system according to Example 10, wherein one or more characteristics of the light source include selection of the angle of incidence of radiation, polarization state, and intensity.

[0088] Example 15: A substrate mapping system according to any one of Examples 9 to 14, wherein automatic tagging includes classifying each of a plurality of locations as at least one classification type selected from types including appropriate load, cross-slot insertion, double load, protrusion, and empty.

[0089] Example 16: A substrate mapping system according to any one of Examples 9 to 15, wherein the data acquisition and control system is further configured to identify each imaged substrate with respect to at least one substrate characteristic, including a characteristic selected from substrate thickness, substrate curvature, substrate warp, and substrate droop, and to transmit at least one of the substrate characteristics to the transfer robot in order to adjust the positional location between the end effector of the transfer robot and one selected substrate from among the substrates based on one or more of the substrate characteristics.

[0090] Example 17: A substrate mapping system according to any one of Examples 9 to 16, further comprising characterizing the size of the substrate carrier.

[0091] Example 18: A substrate mapping system according to any one of Examples 9 to 17, wherein the system is first trained in training mode based on one or more images to produce a pre-trained deep convolutional neural network, and subsequently configured to be used in normal operating mode within a manufacturing facility to detect the arrangement of substrates in substrate carriers within the manufacturing facility based on the pre-trained deep convolutional neural network.

[0092] Example 19: A substrate mapping system according to any one of Examples 9 to 18, wherein the data acquisition and control system is further configured to scale each of one or more images to a trained convnet size.

[0093] Example 20: A computer-readable medium containing instructions that, when executed by a machine, cause the machine to perform an action including detecting at least a portion of a substrate within a substrate carrier. The detection includes capturing one or more images of a portion of the substrate and locations adjacent to the portion of the substrate; transferring one or more images to a pre-trained deep convolutional neural network; using the pre-trained deep convolutional neural network to classify the state of the portion of the substrate within a location within the substrate carrier from the images; and providing automatic tagging of each of multiple locations within the substrate carrier adjacent to multiple portions of the substrate.

[0094] Example 21: The computer-readable medium according to Example 20, further comprising training a neural network to classify the state of the substrate position.

[0095] Example 22: A computer-readable medium according to Example 20 or Example 20, wherein automatic tagging includes classifying each of a plurality of locations as at least one classification type selected from types including appropriate load, cross-slot insertion, double load, protrusion, and empty.

[0096] Example 23: A computer-readable medium according to any one of Examples 20-22, further comprising identifying each of a plurality of substrate portions with respect to at least one substrate characteristic, including a characteristic selected from substrate thickness, substrate curvature, substrate warp, and substrate sag, wherein at least one characteristic is provided for adjusting the position of a robot end effector using one or more of the plurality of substrates.

[0097] Example 24: A computer-readable medium according to any one of Examples 20-23, further comprising classifying multiple substrate types within a substrate carrier from multiple substrate portions.

[0098] Example 25: A computer-readable medium according to any one of Examples 20 to 24, further comprising selecting the wavelength of a light source used to capture one or more images based on at least one of the reflective properties of the substrate in the substrate carrier, the transmittance properties of the substrate in the substrate carrier, and a film or coating on the substrate.

[0099] Example 26: The computer-readable medium according to Example 25, further comprising selecting the incident angle, polarization state, and intensity of radiation from a light source.

[0100] Example 27: A computer-readable medium according to any one of Examples 20-26, further comprising scaling each of one or more images to a trained convnet size.

Claims

1. A method for classifying the states of multiple substrates at multiple locations within a substrate carrier, wherein the method is Detection of at least a portion of the plurality of substrates within the substrate carrier, wherein the detection includes capturing one or more images of the portion of the plurality of substrates and the plurality of locations adjacent to the portion of the plurality of substrates. Transferring one or more of the aforementioned images to a pre-trained deep convolutional neural network, Using the pre-trained deep convolutional neural network, classify the state of the portions of the substrates at the multiple locations within the substrate carrier from one or more images. A method comprising providing automatic tagging of each of the plurality of positions within the substrate carrier adjacent to the portion of the plurality of substrates.

2. The method according to claim 1, further comprising training the neural network to classify the state of the substrate position.

3. The method according to claim 1, wherein the automatic tagging includes classifying each of the plurality of locations as at least one classification type selected from types including appropriate load, cross-slot insertion, double load, protrusion, and empty.

4. The method according to claim 1, further comprising identifying each of the portions of the plurality of substrates with respect to at least one substrate characteristic, including a characteristic selected from the thickness of the substrate, the curvature of the substrate, the warp of the substrate, and the droop of the substrate, wherein the at least one characteristic is provided for adjusting the position of a robot end effector using one or more of the plurality of substrates.

5. The method according to claim 1, further comprising classifying to identify a plurality of substrate types within the substrate carrier from the portions of the plurality of substrates.

6. The method according to claim 1, further comprising selecting the wavelength of a light source used to capture one or more images based on at least one of the reflective properties of the substrate in the substrate carrier, the transmittance properties of the substrate in the substrate carrier, and a film or coating on the substrate.

7. The method according to claim 6, further comprising selecting the incident angle, polarization state, and intensity of the radiation from the light source.

8. The method according to claim 1, further comprising scaling each of the one or more images to a trained convnet size.

9. A substrate mapping system, A camera for collecting one or more images of a substrate in a substrate carrier and the potential position of the substrate, wherein the one or more images include the relationship between the substrate and the position of the substrate relative to a plurality of substrate slots in the substrate carrier, The system comprises a data acquisition and control system, wherein the data acquisition and control system includes one or more hardware-based processors of a machine coupled to the camera, and the data acquisition and control system is The one or more images mentioned above are transferred to a deep convolutional neural network. Using the deep convolutional neural network, the state of the imaged portion of the substrate within the substrate carrier at the given location is classified from one or more images, and A substrate mapping system that provides automatic tagging of each of the positions of the substrate within the substrate carrier that are adjacent to the portion of the substrate that has been imaged.

10. The substrate mapping system according to claim 9, further comprising at least some of the substrates and a light source for illuminating the potential positions of the substrates within the substrate carrier.

11. The substrate mapping system according to claim 10, wherein the light source is a broadband light source.

12. The substrate mapping system according to claim 10, wherein the light source is a monochromatic light source.

13. The substrate mapping system according to claim 10, wherein the wavelength of light emitted from the light source is selected based on at least one of the reflective properties of the substrate in the substrate carrier, the transmittance properties of the substrate in the substrate carrier, and a film or coating on the substrate.

14. The substrate mapping system according to claim 10, wherein one or more characteristics of the light source include selection of the angle of incidence of radiation, polarization state, and intensity.

15. The substrate mapping system according to claim 9, wherein the automatic tagging includes classifying each of the plurality of locations as at least one classification type selected from types including appropriate load, cross-slot insertion, double load, protrusion, and empty.

16. The aforementioned data acquisition and control system Each of the imaged substrates is identified with respect to at least one substrate characteristic, which includes a characteristic selected from the substrate thickness, substrate curvature, substrate warp, and substrate sag, and The substrate mapping system according to claim 9, further configured to transmit at least one of the substrate characteristics to the transfer robot in order to adjust the positional location between the end effector of the transfer robot and one selected substrate from among the substrates, based on one or more of the substrate characteristics.

17. The substrate mapping system according to claim 9, further comprising characterizing the size of the substrate carrier.

18. The substrate mapping system according to claim 9, wherein the system is first trained in training mode based on one or more images to produce a pre-trained deep convolutional neural network, and subsequently configured to be used in normal operating mode within the manufacturing facility to detect the arrangement of substrates in substrate carriers within the manufacturing facility based on the pre-trained deep convolutional neural network.

19. The substrate mapping system according to claim 9, wherein the data acquisition and control system is further configured to scale each of the one or more images to a trained convnet size.

20. A computer-readable medium containing instructions, wherein when the instructions are executed by a machine, the machine, Detection of at least a portion of a plurality of substrates within a substrate carrier, wherein the detection includes capturing one or more images of the portion of the plurality of substrates and a plurality of locations adjacent to the portion of the plurality of substrates. Transferring one or more of the aforementioned images to a pre-trained deep convolutional neural network, Using the pre-trained deep convolutional neural network, classify the state of the portions of the plurality of substrates at the location within the substrate carrier from one or more images. A computer-readable medium that causes an operation to be performed, which includes providing automatic tagging of each of the plurality of positions in the substrate carrier adjacent to the portion of the plurality of substrates.

Images