Machine learning based root cause analysis of process cycle images

Abstract

The disclosed technology relates to classifying process cycle images to predict the success or failure of a process cycle. The disclosed technology includes capturing and processing images of sections disposed on an imaging chip during a genotyping process. Image description features of the production cycle images are created and provided as input to a classifier. The trained classifier separates successful production images from unsuccessful or failed production images. The failed production images are further classified into various categories of failure by a trained root cause classifier.

Description

[Technical Field]

[0001] (Priority application) This application claims the benefit of U.S. Provisional Patent Application No. 17 / 161,595 (Attorney Docket No. ILLM1026-2 / IP-1911-US), entitled "MACHINE LEARNING-BASED ROOT CAUSE ANALYSIS OF PROCESS CYCLE IMAGES," filed January 28, 2021, which claims the benefit of U.S. Provisional Patent Application No. 62 / 968,950 (Attorney Docket No. ILLM1026-1 / IP-1911-PRV), entitled "MACHINE LEARNING-BASED ROOT CAUSE ANALYSIS OF PROCESS CYCLE IMAGES," filed January 31, 2020, the priority application of which is incorporated by reference for all purposes.

[0002] FIELD OF THE INVENTION The disclosed technology relates to image classification for production process assessment and root cause failure analysis. [Background technology]

[0003] The subject matter discussed in this section should not be assumed to be prior art merely as a result of its mention in this section. Similarly, it should not be assumed that the problems mentioned in this section, or problems associated with the subject matter provided as background, have been previously recognized in the prior art. The subject matter in this section merely represents different approaches, which themselves may also correspond to embodiments of the claimed technology.

[0004] Genotyping is a process that can take multiple days to complete. The process is vulnerable to both mechanical and chemical processing errors. Collected samples for genotyping are extracted and distributed into sections and areas of an imaging chip. The samples are then chemically processed through multiple steps to generate a fluorescent image. The process generates a quality score for each section analyzed. This quality may not provide insight into the root cause of a low-quality process failure. In some cases, failed section images still produce acceptable quality scores.

[0005] Therefore, an opportunity has arisen to introduce new methods and systems for evaluating section images and determining root cause analysis of failures during production genotyping.

[0006] In the drawings, like reference characters generally refer to like parts throughout the different views. Also, the drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the disclosed technology. In the following description, various embodiments of the disclosed technology are described with reference to the following drawings: [Brief explanation of the drawings]

[0007] [Figure 1] 1 shows an architecture level schematic of a system in which process cycle images from a genotyping instrument are classified and the root cause of poor images is determined. [Figure 2] 2 illustrates the subsystem components of the feature generator of FIG. 1; [Figure 3] The process steps for an exemplary genotyping process are presented. [Figure 4] Presents an image of the segments located within the image generated after successful completion of the process. [Figure 5A] An example of a failed segmentation image due to hybridization failure during the genotyping process is presented. [Figure 5B] An example of a failed segmentation image due to hybridization failure during the genotyping process is presented. [Figure 5C] An example of a failed segmented image due to a spacer shift failure is presented. [Figure 5D] An example of a failed segmented image due to a spacer shift failure is presented. [Figure 5E] An example of a failed segmented image due to an offset failure is presented. [Figure 5F] An example of a failure segmentation image due to surface wear failure is presented. [Figure 5G] 10 presents an example of a failure segment image due to a reagent flow failure. [Figure 5H] 10 presents an example of a failure segment image due to a reagent flow failure. [Figure 5I] Examples of failed or unhealthy segmented images where the source of the failure is unknown are presented. [Figure 6A] 1 is an example of a measure of 96 intrinsic image components selected by ranking principal components generated using Principal Component Analysis (PCA). [Figure 6B] 1 is an example of a measure of 96 intrinsic image components selected by ranking principal components generated using Principal Component Analysis (PCA). [Figure 6C] Zoom in on the top 40 intrinsic image components selected from the 96 images of FIGS. 6A and 6B. [Figure 6D] Zoom in on the top 40 intrinsic image components selected from the 96 images of FIGS. 6A and 6B. [Figure 7A] 1 illustrates image rescaling and flattening of segmented images for input to principal component analysis. [Figure 7B] 1 illustrates the creation of eigenimages using principal component analysis. [Figure 8A] 1 illustrates an example of feature generation using the intensity of areas of a partition. [Figure 8B] 10 illustrates another example of feature generation using the intensity of areas of a partition. [Figure 9]1 is a diagrammatic illustration of a one-vs-the-rest (OvR) classifier. [Figure 10A] We illustrate the training of a binary (good vs. bad) classifier and a multi-class (root cause) classifier using labeled training data containing process cycle images. [Figure 10B] Illustrates a two-step process in which production process cycle images are classified as good versus bad and further determines a failure category for bad images. [Figure 11] FIG. 1 is a simplified block diagram of a computer system that can be used to implement the disclosed techniques. DETAILED DESCRIPTION OF THE INVENTION

[0008] The following discussion is presented to enable any person skilled in the art to make and use the disclosed technology and is provided in the context of a particular application and its requirements. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the disclosed technology. Thus, the disclosed technology is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein. Introduction

[0009] The disclosed technology applies vision systems and image classification for production genotyping evaluation and root cause failure analysis. Two distinct approaches are described, one based on eigenimages and the other on area thresholding. Principal component analysis (PCA) and non-negative matrix factorization (NMF) are among the disclosed techniques. Other dimensionality reduction techniques that can be applied to images include independent component analysis, dictionary learning, sparse principal component analysis, factor analysis, and mini-batch K-means. Variations of image decomposition and dimensionality reduction techniques can be used. For example, PCA can be implemented using singular value decomposition (SVD) or as kernel PCA. The output from these techniques is provided as input to a classifier. The applied classifier can include random forests, K-nearest neighbors (KNN), multinomial logistic regression, support vector machines (SVM), gradient boosted trees, naive Bayes, etc. As a larger body of labeled images becomes available, convolutional neural networks such as ImageNet can also be used.

[0010] This production process is vulnerable to both mechanical and chemical processing errors. Collected samples are extracted, distributed to sections and areas of the BeadChip, and then chemically processed through multiple steps to generate a fluorescent image. The final fluorescent image, or even intermediate fluorescent images, can be analyzed to monitor production and perform failure analysis.

[0011] The majority of production analysis is successful. The current failure analysis is understood to fit into five categories and a remaining failure category. The five failure categories are hybridization or hyb failure, spacer shift failure, offset failure, surface wear failure, and reagent flow failure. The remaining category is unhealthy patterns due to mixing effects, unidentified causes, and weak signals. More different causes can then be identified, especially since root cause analysis leads to improved production.

[0012] The first image processing techniques applied to quality control and failure analysis evolved from facial recognition using eigenface analysis. A linear measure of 40–100 image components was identified from tens of thousands of labeled images. One approach to forming the eigenvalues ​​was principal component analysis (PCA), followed by a ranking of the components according to a measure of explained variability. 40 components were observed to explain the majority of the variability. Beyond 100 components, additional components appeared to reflect patterns of noise or natural variability in sample processing. The number of relevant components is expected to be affected by image resolution. Here, resolution reduction was applied so that sections of the image-generating chip were analyzed at a resolution of 180 × 80 pixels. This was sufficient resolution to distinguish successful from unsuccessful production and then classify the root causes of failure among six failure categories. Formal sensitivity analysis was not applied, but images with slightly lower resolution also worked; images 4–22 times this resolution could be processed in the same manner, but with increased computational expense. Each image analyzed by eigenimage analysis is represented as a weighted linear combination of reference images. Each weight for the set of ordered reference components is used as a feature to train a classifier. For example, in one implementation, 96 weights of the components of the labeled images were used to train a random forest classifier. A random forest classifier with 200 trees and a depth of 20 performed well. Two tasks were performed by the random forest classifier: separation of successful and unsuccessful production images, and then root cause analysis of the unsuccessful production images. This two-stage classification was chosen due to the predominance of successful production runs, but one-stage classification could also be used.

[0013] The second image processing technique applied involved thresholding of the image area. A production image of a section of the imaging chip captures several physically separated areas. Structures that border the section and separate distinct physical areas of the section are visible in the production image. The thresholding strategy involves separating the active areas from border structures and then distinguishing the separated areas. Optionally, the structures that separate the physical areas can also be filtered out from the image. At least the active areas are subjected to luminescence thresholding. The thresholding determines how much of the active area produces the desired signal strength. Each active area is evaluated after thresholding for success or failure. Patterns of failure across areas and sections of the imaging chip can be further evaluated for root cause classification.

[0014] Processing of production images to detect failed production runs and determine root cause can be performed immediately during production, and results can be read from the imaging chip and quality judged faster. This image processing can be done more quickly because shrinking the image 20 times on the sides significantly reduces computational requirements, and direct processing of reduced resolution images does not require correlation of individual bright pixels within an area to individual probes. Fast turnaround root cause analysis can be used to correct upstream processes before chemicals and processing time are wasted.

[0015] environment We describe a system for early prediction of failure in a genotyping system. Genotyping is the process of determining differences in an individual's genetic makeup (genotype) by using biological assays to examine the individual's DNA sequence and compare it to a reference sequence. Genotyping allows researchers to search for genetic variants, such as single nucleotide polymorphisms (SNPs) and structural changes in DNA. The system will be described with reference to FIG. 1, which shows an architecture-level schematic diagram of the system according to an embodiment. Because FIG. 1 is an architecture diagram, certain details have been intentionally omitted to improve clarity of description. The discussion of FIG. 1 is organized as follows: first, the elements of the diagram are described, followed by a description of their interconnections. Then, the use of the elements within the system is described in more detail.

[0016] 1 includes system 100. This paragraph names labeled portions of system 100. The figure illustrates genotyping equipment 111, process cycle image database 115, failure category label database 117, labeled process cycle image database 138, trained good vs. bad classifier 151, reference database of eigenimages 168, trained root cause classifier 171, feature generator 185, and network 155.

[0017] The disclosed technology is applied to various genotyping devices 111, also referred to as genotyping scanners and genotyping platforms. A network 155 communicatively couples the genotyping devices 111, the process cycle image database 115, the failure category label database 117, the labeled process cycle image database 138, the trained good vs. bad classifier 151, the reference database of eigenimages 168, the trained root cause classifier 171, and the feature generator 185.

[0018] Genotyping instruments can include Illumina's BeadChip imaging system, such as the ISCAN™ system. The instrument can detect the fluorescence intensity of hundreds to millions of beads arranged in mapped sections on an imaging chip. Genotyping instruments can include an instrument control computer that controls various aspects of the instrument, such as laser control, precision machine control, excitation signal detection, image registration, image extraction, and data output. Genotyping instruments can be used in a wide variety of physical environments and operated by technicians with various skill levels. Sample preparation can take 2-3 days and can include manual and automated handling of samples.

[0019] We illustrate the process steps of an exemplary genotyping process 300 in Figure 3. This exemplary genotyping process is referred to as Illumina's INFINIUM™ assay workflow. The process is designed to interrogate many SNPs with broad-range locus multiplexing. Using a single bead type and a dual-color (red and green) channel approach, the process scales to genotyping hundreds to millions of SNPs per sample. The process begins with DNA sample receipt and extraction. The process can operate with a relatively low input sample, such as 200 ng, which can assay millions of SNP loci. The sample is amplified. The amplification process can take several hours to overnight to complete. The amplified sample undergoes controlled enzymatic fragmentation, followed by alcohol precipitation and resuspension. An imaging chip is prepared for hybridization in a capillary flow chamber. The sample is then applied to the prepared imaging chip and incubated overnight. During this overnight hybridization, the sample is annealed to locus-specific 50-mers covalently attached to up to millions of bead types, one bead type corresponding to each allele per SNP locus. Allele specificity is imparted by enzymatic base extension followed by fluorescent staining. A genotyping instrument or scanner (such as the ISCAN™ system) detects the fluorescence intensity of the beads and performs the genotyping.

[0020] In one example, genotyping results are presented using a metric called "call rate." This metric represents the percentage of genotypes that are correctly scanned on the imaging chip. A separate call rate is reported for each section of the imaging chip. A threshold can be used to accept or reject the results. For example, a call rate of 98% or greater can be used to accept the genotyping results for a section. Different thresholds, such as less than 98% or more than 98%, can be used. If the call rate for a section falls below the threshold, the genotyping process is considered a failure. The genotyping process can take many days and is therefore expensive to repeat. Failure in the genotyping process can occur due to operational errors (such as mechanical or handling errors) or chemical processing errors.

[0021] The genotyping system can provide process cycle images of the imaging chip sections along with their call rates upon completion of the genotyping process. The disclosed technology can process these section images to classify whether the genotyping process was successful (good images of the section) or not (bad or failed images of the section). The disclosed technology can further process the bad or failed images to determine the failure category. Currently, the system can classify failed images into one of six failure categories: hybridization or hyb failure, spacer shift failure, offset failure, surface wear failure, reagent flow failure, and overall unhealthy images due to mixing effects, unknown causes, weak signals, etc. More different causes can then be identified, especially since root cause analysis can lead to improved production.

[0022] We now refer to FIG. 1 to provide a description of the remaining components of system 100. Failure category labels for the six failure types may be stored in failure category label database 117. A training data set of labeled process image cycles is stored in database 138. The labeled training examples may consist of successful (good) and unsuccessful (bad) process cycle images. Unsuccessful process cycle images are labeled as belonging to one of the six failure categories listed above. In one embodiment, training database 138 consists of at least 20,000 training examples. The size of the training database may increase as more labeled image data is collected from laboratories using genotyping equipment.

[0023] The disclosed technology includes two independent image processing techniques for extracting features from process cycle images. A feature generator 185 can be used to apply one of the two techniques to extract features from process cycle images for input into a machine learning model. The first image processing technique evolves from facial recognition using eigenface analysis. A relatively small number of linear criteria, such as 40 to 100 or more image components, are identified from tens of thousands of labeled images. One approach for forming the eigencriteria is principal component analysis (PCA). The production cycle images are represented as a weighted linear combination of reference images for input to a classifier. For example, in one implementation, 96 weights of the components of the labeled images are used to train the classifier. The eigenimage criteria can be stored in the database 168.

[0024] A second image processing technique for extracting features involves thresholding the segment image. A production image of a segment of an imaging chip captures several physically separated areas. Structures that border the segment and separate distinct physical areas of the segment are visible in the production image. The thresholding technique determines how much active area produces a desired signal strength. The output from the thresholding technique can be provided as input to a classifier to distinguish good images from bad images. Patterns of failure across areas and segments of the imaging chip can be further evaluated for root cause analysis.

[0025] The image features of the production images generated by feature generator 185 are provided as input to trained classifiers 151 and 171. Two types of classifiers are trained: a good vs. bad classifier that can predict successful and unsuccessful production images; and a root cause analysis classifier that can predict the failure category of unsuccessful images. One example of a classifier used by the disclosed technology includes a random forest classifier. Other examples of classifiers that can be applied include K-nearest neighbors (KNN), multinomial logistic regression, and support vector machines. Convolutional neural networks (CNNs) can also be used when a larger body of labeled images becomes available.

[0026] 1, all of the components of system 100 described above are communicatively coupled to network 155. The actual communication paths may be point-to-point via public and / or private networks. Communications may occur across various networks, e.g., private networks, VPNs, MPLS circuits, or the Internet, and may use appropriate application programming interfaces (APIs) and data exchange formats, e.g., Representational State Transfer (REST), JavaScript Object Notation (JSON), Extensible Markup Language (XML), Simple Object Access Protocol (SOAP), Java Message Service (JMS), and / or Java Platform Module System. All communications may be encrypted. Communications are typically over networks such as local area networks (LANs), wide area networks (WANs), telephone networks (Public Switched Telephone Network (PSTN), Session Initiation Protocol (SIP)), wireless networks, point-to-point networks, star networks, token ring networks, hub networks, the Internet, including mobile Internet via protocols such as EDGE, 3G, 4G LTE, Wi-Fi, and WiMAX. The engine or system components of FIG. 1 are implemented by software running on various types of computing devices. Exemplary devices are workstations, servers, computing clusters, blade servers, and server farms. Additionally, various authorization and authentication technologies may be used to secure communications, such as username / password, Open Authorization (OAuth), Kerberos, Secured, digital certificates, etc.

[0027] Feature Generator - System Component FIG. 2 is a high-level block diagram 200 of the components of the feature generator 185. These components are computer implemented using a variety of different computer systems, as presented below in the description of FIG. 11. The illustrated components may be merged or further separated when implemented. The feature generator 185 consists of two high-level components that implement two image processing techniques: a principal component analysis, or PCA, based feature generator 235 and an image segmentation based feature generator 255. The PCA-based feature generator consists of an image scaler 237 and an eigenimage reference creator 239. The image segmentation based feature generator 255 consists of an image transformer 257 and an intensity extractor 259. In the following sections, we present further details of the implementation of these components.

[0028] PCA-based feature generator The first image processing technique evolved from face recognition via eigenface analysis. One approach to forming eigencriteria is principal component analysis (PCA). A PCA-based feature generator 235 applies PCA to the resized process image. An image scaler component 237 resizes (or rescales) the process cycle image. Scaling reduces the size of the process image so that it can be processed in a computationally efficient manner by the eigenimage criteria creator component 239. We present details of these components in the following sections.

[0029] Image Scaler Higher resolution images obtained from the genotyping device or scanner may require more computational resources to process. Images obtained from the genotyping scanner are resized by the image scaler 237 so that images of the image-generating chip segments are analyzed at a reduced resolution of 180 x 80 pixels. In one example, the segmented images obtained from the scanner have a resolution of 3600 x 1600 pixels, and a 20x reduction in resolution is applied to resize the images. This is sufficient resolution to distinguish successful production images from unsuccessful production images and then classify the root cause of the failure among six failure categories. Images rescaled to 4-25x their original resolution can be processed in the same manner.

[0030] The disclosed technology may apply various interpolation techniques to reduce the size of the produced image. In one embodiment, bilinear interpolation is used to reduce the size of the segmented image. Linear interpolation is a curve-fitting method using linear polynomials to construct new data points within a discrete set of known data points. Bilinear interpolation is an extension of linear interpolation to interpolate functions of two variables (e.g., x and y) on a two-dimensional grid. Bilinear interpolation is performed using linear interpolation, first in one direction and then again in the second direction. Each step is linear in the sampled value and position, but the overall interpolation is not linear but rather quadratic in the sample locations. Other interpolation techniques may also be used to reduce the size (rescaling) of the segmented image, such as neighborhood interpolation and resampling using pixel area relationships.

[0031] Intrinsic image reference generator The first image processing technique applied to the segmented image to generate input features for the classifier has evolved from face recognition by eigenface analysis. From tens of thousands of labeled images, linear criteria for 40 to 100 or more image components are identified. One approach to forming the criteria for eigenimages is principal component analysis (PCA). A set B of elements (vectors) in a vector space V is called a basis if every element of V can be written uniquely as a linear combination of the elements of B. Equivalently, B is a basis if its elements are linearly independent and every element of V is a linear combination of the elements of B. A vector space can have several bases. However, all bases have the same number of elements, called the dimension of the vector space. In the technique of the present inventors, the basis of the vector space is an eigenimage.

[0032] PCA is often used to reduce the dimension of a d-dimensional dataset by projecting it onto a k-dimensional subspace, where k < d. For example, the resized labeled images in the training database of the present inventors represent vectors in a d = 14,400-dimensional space (180 × 80 pixels). In other words, an image is a point in a 14,400-dimensional space. The eigenspace-based approach approximates an image vector with a lower-dimensional feature vector. The main assumption behind this technique is that the image space (given by the feature vectors) has a lower dimension than the image space (given by the number of pixels in the image), and image recognition can be performed in this reduced space. The segmented images of the image generation chip, which have a similar overall configuration, will not be randomly distributed in this large space and can thus be explained by a relatively low-dimensional subspace. The PCA technique finds vectors that best explain the distribution of the segmented images throughout the image space. These vectors define a subspace of the image, also referred to as the "image space". In the embodiments of the present inventors, each vector represents an 180 × 80 pixel image and is a linear combination of the images in the training data. In the following text, the present inventors present details on how principal component analysis (PCA) can be used to create the criteria for eigenimages.

[0033] PCA-based analysis of labeled training images can consist of the following five steps:

[0034] Step 1: Accessing multidimensional correlation data The first step in applying PCA is to access high-dimensional data. In one example, we used 20,000 labeled images as training data. Each image was resized to 180x80 pixel resolution and represented as a point in a 14,400-dimensional space, one dimension per pixel. This technique can handle images with higher or lower resolution than specified above. The size of the training dataset is expected to increase as we collect more labeled images from our lab.

[0035] Step 2: Standardize the data Standardization (or Z-score normalization) is the process of rescaling features so that they have the properties of a Gaussian distribution with a mean equal to zero, or μ=0, and a standard deviation from the mean equal to one, or σ=1. Standardization is performed to construct features that have similar ranges to each other. The standard score of an image can be calculated by subtracting the mean (image) from image and dividing the result by the standard deviation. Once PCA has obtained a feature subspace that maximizes the variance along an axis, it helps to standardize the data so that the data is centered on the axis.

[0036] Step 3: Calculate the covariance matrix A covariance matrix is ​​a dxd matrix in d-dimensional space, where each element represents the covariance between two features. The covariance of two features measures their tendency to fluctuate together. The variance is the average of the squared deviations of the feature from its mean. The covariance is the average of the products of the deviations of the features from their mean. Consider feature k and feature j. Let {x(1,j),x(2,j),...,x(i,j)} be the set of i examples of feature j, and {x(1,k),x(2,k),...,x(i,k)} be the set of i examples of feature k. Similarly,

number

number

number

number

number

[0037] The mean vector is a d-dimensional vector, where each value in this vector represents the sample mean of a feature column in the training dataset. jk is -(σ ij )(σ ik ), i.e., +(σ ij )(σ ik ) can vary between linear correlation and inverse linear correlation. When there is no dependency between two features, σ jk The value of is zero.

[0038] Step 4: Compute the eigenvectors and eigenvalues The eigenvectors and eigenvalues ​​of the covariance matrix represent the core of PCA. The eigenvectors (or principal components) determine the orientation of the new feature space, and the eigenvalues ​​determine their magnitude. In other words, the eigenvalues ​​explain the variance of the data along the axes of the new feature space. Eigenvalue decomposition is a method of matrix decomposition by representing a matrix using its eigenvectors and eigenvalues. An eigenvector is defined as a vector that changes only by a scalar when a linear transformation is applied to the vector. If A is a matrix representing a linear transformation, v is an eigenvector and λ is the corresponding eigenvalue, which can be expressed as Av = λv. A square matrix can have many eigenvectors because it has dimensions. If we represent all eigenvectors as columns of a matrix V and the corresponding eigenvalues ​​as entries of a diagonal matrix L, the above equation can be expressed as AV = VL. For a covariance matrix, all eigenvectors are orthogonal to each other and are principal components of the new feature space.

[0039] Step 5: Use the explained variance to select a criterion for the eigenimages The above steps may result in 14,400 principal components for our implementation, equal to the dimension of the feature space. An eigenpair consists of an eigenvector and a scalar eigenvalue. We may sort the eigenpairs based on their eigenvalues ​​and create a measure of the eigenimages using a metric called "explained variance." Explained variance indicates how much information (or variance) can be attributed to each of the principal components. We may plot the results of the explained measure on a two-dimensional graph. The sorted principal components are represented along the x-axis. A graph showing cumulative explained variance may be plotted. The first m components representing a major portion of the variance may be selected.

[0040] In our implementation, the first 40 principal components represent a high percentage of explained variance, and therefore we selected the first 40 principal components to form the baseline for our new feature space. In other implementations, 25 to 100 principal components or more than 100 principal components, up to 256 or 512 principal components, may be selected to create the baseline for eigenimages. Each production image analyzed by eigenimage analysis is represented as a weighted linear combination of baseline images. Each weight in the set of ordered baseline components is used as a feature for training the classifier. For example, in one implementation, 96 weights for the components of labeled images were used to train the classifier.

[0041] The disclosed technology may use other image decomposition and dimensionality reduction techniques. For example, non-negative matrix factorization (NMF) learns a portion-based representation of an image, compared to PCA, which learns a complete representation of the image. Unlike PCA, NMF learns to represent an image with a set of reference images that resemble portions of the image. NMF factorizes matrix X into two matrices, W and H, with the property that all three matrices have no negative elements. We assume that matrix X is set so that there are n data points (such as images of a section on an image generation chip), each with p dimensions (e.g., 14,400). Therefore, matrix X has p rows and n columns. We wish to reduce the p dimensions to r dimensions, or in other words, create a rank-r approximation. NMF approximates matrix X as the product of two matrices: W (p rows and r columns) and H (r rows and n columns).

[0042] The interpretation of matrix W is that each column is a reference element. By reference elements, we average several components present in the n original data points (or images). These are the components from which we can reconstruct an approximation to all of the original data points or images. The interpretation of matrix H is that each column gives the coordinates of a data point in the reference matrix W. In other words, it shows us how to reconstruct an approximation to the original data points from a linear combination of the components of matrix W. In the case of a face image, the reference elements of matrix W (or reference images) may include features such as eyes, nose, lips, etc. The columns of matrix H indicate which features are present in which images.

[0043] Image segmentation-based feature generator A second image processing technique for extracting features from process cycle images is based on image area thresholding. Image segmentation-based feature generator 255 first segments the image of a section of the imaging chip using image segmenter 257, and then applies thresholding by extracting the intensity of active areas or regions of interest in the section image. Thresholding determines how much of the active area produces the desired signal intensity.

[0044] The image generation chip consists of multiple segments, such as 24, 48, 96, or more, organized into rows and columns. This design allows for the processing of multiple samples in one process cycle, as many samples (one per segment) can be processed in parallel. The segments are physically separated from each other to prevent samples from mixing with each other. In addition, the segments can be organized into multiple parallel regions called "slots." Therefore, the structure of the segment and slot boundaries is visible in the process cycle image from the genotyping scanner. We present below details of two components of the image segmentation-based feature generator 255, which can implement techniques for transforming segmented images for the extraction of image features.

[0045] Image Converter The image converter 257 applies a series of image transformation techniques to prepare the segmented image for intensity extraction from the region of interest. In one embodiment, this process of image transformation and intensity extraction is performed by some or all of the following five steps: Image transformation converts the segmented grayscale image into a binary image consisting of black and light pixels. The average intensity values ​​of the active regions of the grayscale and binary images are provided as input features to a classifier for classifying the image as healthy (good) or unhealthy (bad) images. In the following text, we present details of the image transformation steps, including applying thresholding to convert the grayscale image into a binary image. A process step includes applying a filter to remove noise.

[0046] The first step in the image transformation process is to apply a bilateral filter to process the segmented cycle image. Bilateral filtering is a technique for smoothing an image while preserving edges. It replaces the intensity of each pixel with a weighted average of the intensity values ​​from its neighboring pixels. Each neighborhood is weighted by a spatial component, which penalizes distant pixels, and a range component, which penalizes pixels with different intensities. The combination of both components ensures that only nearby similar pixels contribute to the final result. Therefore, bilateral filtering is an efficient way to smooth an image while preserving its discontinuities or edges. Other filters, such as median filters and anisotropic diffusion, may also be used.

[0047] The second step of image transformation can be to apply thresholding to output the image from step 1. In one embodiment, we apply Otsu's method (Otsu, N., 1979, "A threshold selection method from gray-level histograms," IEEE Transactions on Systems, Man, and Cybernetics, Volume 9, Issue 1), which uses intensity histograms and searches for a threshold to maximize the weighted sum of grayscale variance between pixels assigned to dark and light intensity classes. Otsu's method attempts to maximize the variance between classes. The basic idea is that well-thresholded classes should differ in terms of their pixel intensity values, and conversely, the best threshold is the threshold that gives the best separation between classes in terms of their intensity values. Additionally, Otsu's method has the property that it is entirely based on calculations performed on the image's histogram, which is an easily obtainable one-dimensional array. For further details, see Section 10.3.3 of Gonzalez and Woods, "Digital Image Processing," 3 rd Please refer to the Edition.

[0048] The third step in image transformation is the application of a noise-reducing Gaussian blur filter to remove speckle-like noise. Noise can contaminate process cycle images with small speckles. Gaussian filtering is a weighted average of the intensities of neighboring locations, with weights decreasing with the spatial distance to the central location.

[0049] The fourth step of image transformation involves image morphology operations. The binary output image from the third step is processed by a morphological transformation to fill holes in the image. A hole can be defined as a background region (represented by 0s) surrounded by a connected boundary of foreground pixels (represented by 1s). Two basic image morphology operations are "erosion" and "dilation." In an erosion operation, a kernel slides (or moves) over the binary image. If all pixels under the kernel are 1s, then the pixel (1 or 0) in the binary image is considered to be 1; otherwise, it is eroded (changed to 0). The erosion operation is useful for removing isolated 1s in a binary image. However, erosion also shrinks clusters of 1s by shrinking their edges. The dilation operation is the opposite of erosion. In this operation, as the kernel slides over the binary image, if at least one pixel under the kernel has a value of 1, then the values ​​of all pixels in the binary image area overlapped by the kernel are changed to 1. When a dilation operation is applied to a binary image followed by an erosion operation, the effect is to close small holes in the clusters of 1s (represented by 0s in the image). The output from this step is provided as input to the intensity extractor component 259, which implements the fifth step of this image transformation technique.

[0050] intensity extractor The intensity extractor 259 divides the segmented image into active regions, or segments, by filtering out structures at the boundaries of the segments and slots. The intensity extractor may apply different segmentations to divide the segmented image into 8 to 17 active areas. Examples of areas within the segmented image include the four slots, the four corners, the four edges between the corners, and the various vertical and horizontal lines at the boundaries of the segments and slots. Areas corresponding to known structures separating the active areas are then removed from the image. The remaining active area image portions are processed by the intensity extractor 259. Intensity values ​​are extracted and averaged for each active area in the transformed image and the corresponding untransformed image. For example, if intensity values ​​are extracted from 17 active areas in the transformed image, the intensity extractor also extracts intensity values ​​from the same 17 active areas in the untransformed image. Thus, a total of 34 features are extracted per segmented image.

[0051] For a binary image, the average intensity of an active area can be between 1 and 0. For example, a black pixel is considered to have an intensity of 0, and a bright (or blank) pixel is considered to have an intensity of 1. If all pixels in an active area are black, the average intensity of the active area will be 0. Similarly, if all pixels in an active area are bright, the intensity of that area will be 1. Active areas of a healthy image will appear as blank or bright in the binary image, while black pixels represent an unhealthy image. The average intensity of the corresponding active area in the grayscale image is also extracted. The average intensities of the active areas from both the grayscale image and the converted binary image are provided as inputs to a good vs. bad classifier. In one embodiment, the classification confidence score from the classifier is compared to a threshold to classify the image as a healthy (good) image or an unhealthy (bad) image. An example threshold is 80%. A higher threshold value may result in more images being classified as unhealthy.

[0052] Process Cycle Image Here, we present examples of successful and unsuccessful production images of segments on an imaging chip. FIG. 4 is an example 400 of a production image of 24 segments on an imaging chip. The segments are arranged in 12 rows and 2 columns. Each segment has four slots. Example 400 shows a segment image of a successful production cycle. Imaging chips with other configurations of segments, such as including 48, 96, or more segments, can also be used. In the following figures, we present examples of segment images of unsuccessful production cycles. Production processes are vulnerable to both operational and chemical processing errors. Operational defects can be caused by mechanical or sample handling problems. Chemical processing errors can be caused by problems with the sample or the chemical processing of the sample. The disclosed technology attempts to classify bad process image cycles that occur due to both operational and chemical processing errors.

[0053] FIG. 5A shows an example section image 510 from an unsuccessful production cycle. The image of section 512 in the second column and seventh row of the imaging chip in FIG. 5A is dark in the bottom half and slightly lighter in the top. The cause of this failure is related to the hybridization process. Therefore, the failed section image is labeled as a "Hyb" failure. Hybridization failures can also occur due to failure of the robot handling the sample during the sample preparation process on the imaging chip. The call rate for this section is below the 98 percent threshold, as shown in the figure. In some cases, the call rate for the section from the genotyping instrument can exceed the pass threshold, and then the section image can fail due to hybridization errors.

[0054] In example 510, it can be noted that the image for section 514 at row 11 and column 2 has a dark area on the right wall. This could also indicate a processing problem, but the overall call rate for this image is above the pass threshold and is not labeled as a failed image. There is sufficient redundancy of samples on the section, so any small areas of the section with apparent failures can be ignored and may not cause errors in the results. For example, in one case, the scanner reads fluorescence from approximately 700K probes on a section with a redundancy of 10. Therefore, the call rate is based on the readout of approximately 7 million probes. We present a further example of hybridization failure in example 515 of FIG. 5B. The four sections on the imaging chip with dashed boundaries show poorly produced images of the sections due to hybridization failures. Note that although the call rate values ​​for these four sections are above the pass threshold, the images for these sections are labeled as failed due to hybridization errors.

[0055] Figure 5C presents an example 520 of nine section images showing unsuccessful processing due to spacer shift failures. When samples are prepared on a section on the imaging chip, dark markers are placed around the section. The spacers separate the samples in each section from other samples in adjacent sections. If the markers are not placed correctly, they can block part of the image signal. Offset errors can occur across multiple adjacent sections, as shown in Figure 5C. The top portions of the nine sections in this figure appear dark. The dark areas at the top of the sections increase as you move from left to right. Space shift issues are caused by incorrect placement of markers by laboratory technicians during sample preparation on the imaging chip, and are therefore operational errors. Figure 5D presents three more examples of failed section images due to spacer shift errors. Box 525 shows five section images with spacer shift failures, where the top portions of the section images are dark and increase in width from top right to top left. Box 527 shows two section images showing unsuccessful processing due to spacer shift issues at the bottom portions of the sections. Similarly, box 529 shows an image of two segments that failed due to space shift issues.

[0056] 5E shows an example of a failed image of a segment due to unsuccessful processing caused by an offset failure. In an offset failure, the image of the segment on the imaging chip is shifted to one side. For example, in example 530, all segment images on the imaging chip are shifted toward the left, so that the dark outer boundary of the imaging chip on the left edge is cut off from the image. An offset failure can be caused by a scanning error, such as a misalignment or misplacement of the scanner of the imaging chip on the chip carrier.

[0057] FIG. 5F shows an example of a failed segment image due to a surface wear failure. Surface wear is caused by scratches on the surface of a segment of an imaging chip during the manufacturing process or during preparation of samples on the segment. The scratches are visible as lines on the image of the segment, as shown in example 535. Note that the call rate values ​​are above the pass threshold for the three segments in the dashed box on the left, yet they are labeled as failed due to a surface wear failure.

[0058] Figure 5G is an example 540 of a failed section image due to a reagent flow failure. Ten section images in box 542 are labeled as failed images due to reagent flow failure. The section images failed due to an unsuccessful process caused by improper reagent flow. During the genotyping process, reagents are introduced into the imaging chip from one side. The reagents flow from one end of the imaging chip to the other, completely covering all sections. Sometimes, there is a problem with the reagent flow and it does not propagate evenly to all sections. In this case, when a sufficient amount of reagent does not cover a section, the reagent may dry out. Improper reagent flow may reduce the intensity of the emission signal from some sections because the fluorescent dye may not be uniformly distributed across all sections, thus affecting image quality. A failed image due to a reagent flow failure may appear darker in color compared to a section image representing a successful process cycle. Figure 5H shows another example of a failed section image due to a reagent flow failure in example 545. A reagent flow failure may affect multiple adjacent sections within a region of the imaging chip, as shown in Figures 5G and 5H.

[0059] FIG. 5I presents an example of a failed image due to unknown reasons. The failed segment image is labeled as "unhealthy." Failed images in the unhealthy class of failures may be due to mixed or unidentified causes and weak signals. The segment image example 550 also shows an example of a spacer failure for the top left segment of the image generation chip. The image segment on the top left location (row 1 and column 2) is labeled as a spacer failure. The top portion of the failed segment image can be seen to be dark. The portion of the upper dark region increases from the right corner to the left corner of the segment image.

[0060] Principal Component Analysis-Based Feature Generation Here, we present examples of eigenimages, referred to as eigenfaces in the field of face recognition. From tens of thousands of labeled images, a linear measure of 40 to 100 image components is identified. Figures 6A and 6B present examples of 96 eigenimages (610 and 611) obtained by applying principal component analysis (PCA). The 96 eigenimages are selected based on the ranking of the components according to the explained variability measures, as presented above. Figures 6C and 6D show the top 40 ranked eigenimages (620 and 621) from the 96 eigenimages of Figures 6A and 6B. In one implementation, 40 components were observed to explain the majority of the variability. The additional components selected appeared to reflect patterns of noise or natural variability in sample processing.

[0061] Here, we describe the creation of a dimensionality reduction and eigenimage criterion using PCA. The first step is to reduce the resolution of a segmented image and prepare the reduced image for input to PCA. FIG. 7A (labeled 710) shows a 3600×1600 pixel segmented image that is rescaled by a factor of 20, resulting in a reduced segmented image of 180×80 pixels. The rescaled segmented image is flattened. The resulting flattened rescaled segmented images are each one-dimensional arrays, i.e., 14,400×1 pixels. The flattened rescaled segmented images are normalized as described above, thus resulting in the normalized flattened rescaled segmented image shown in FIG. 7B (labeled 740), which is provided as input to PCA. PCA therefore generates 14,400 principal components or eigenimages. Each input image is a vector in the 14,400-dimensional space. We then use the explained variance to rank the principal components or eigenimages and create a criterion, for example a criterion of 40-100 components. The components form a criterion in linear space.

[0062] Image segmentation-based feature generation A second image processing technique for generating features from segmented images involves thresholding image areas or segments. Figure 8A shows an example 810 of segmentation-based features applied to a segmented image. Example 812 is a production image of a segment of an image-generating chip. We apply an image transform to convert this grayscale produced image 812 to generate a companion binary image 814. In one embodiment, some or all of the five steps presented above with reference to image converter 257 can be performed to convert a grayscale image to a binary image. Black pixels in binary image 814 indicate unhealthy or bad image pixels, while light pixels indicate healthy or good image pixels.

[0063] The illustration 816 on the right is an exemplary schematic diagram of a segment, showing the various areas of the segment and the borders or lines surrounding these areas. The areas where the intensity of the fluorescent signal is recorded are also referred to as active areas or regions of interest. For example, the segment schematic 816 shows four slot active areas extending parallel to each other from top to bottom. Areas of the segment image that are not active areas are filtered out of the image. For example, slot boundary areas separated from each other by vertical lines indicating the slot boundaries or borders. Similarly, borders on the four sides of the segment image can be filtered out. Segmentation techniques can divide a segment image into 4 to 20 or more segments or active areas. Thresholding determines how much active area produces the desired signal intensity.

[0064] The number of active areas determines the number of features generated per image. For example, if a segmented image is segmented into eight active areas, the image intensities from the eight active areas in the transformed image and the image intensity values ​​from the same eight active areas in the original segmented image before transformation are provided as inputs to the classifier. Therefore, in this example, a total of 16 features are provided to the classifier per segmented image. The average signal intensity from the active areas can be used as an input to the classifier. For example, if a segmented image is segmented into eight active areas, the average intensity of these eight active areas is calculated for both the grayscale image and the binary image. These 16 intensity values ​​are provided as inputs to the classifier to classify the segmented image as good versus bad. Other segmentation schemes, such as dividing into fewer or more segments per image, such as 4, 12, 17 or more segments per image, may also be used. When provided as inputs to a random forest classifier, a subset of features is randomly selected for each decision tree. The decision tree votes the image as healthy or unhealthy. The majority of votes in the random forest are used to classify the image. In one embodiment, the value of the number of trees in the random forest classifier is in the range of 200 to 500, and the value of the model depth is in the range of 5 to 40. Patterns of failures across areas and sections of the imaging chip can be further evaluated for root cause classification.

[0065] Figure 8B presents an example 820 of three pairs of segmented images 822, 824, and 826. The left image in each pair is a pre-transformed grayscale segmented image, and the right image in each pair is a processed binary image after applying image transformations as described with reference to Figure 2. The first image pair 822 is a product image of a successful genotyping process. The second image pair 824 is a product image of a failed production image due to hybridization (or hyb) failure. The third image pair 826 is a failed image due to a surface wear problem.

[0066] One-vs-other (OvR) classification FIG. 9 presents a diagrammatic example 900 of running a one-vs-others classifier. The graphs show an example of running a one-vs-others (OvR) classifier on a dataset consisting of samples belonging to three classes (squares, circles, and triangles), as shown in graph 931 on the left. As shown in graph 915 on the top right, a first hyperplane 916 indicates the hyperplane determination of the square class as ground truth. Hyperplane 916 partitions data points in the square class from the rest of the data points (circles and triangles). Similarly, graphs 936 and 955 partition data points in the circle and triangle classes, respectively, to other classes of data via hyperplanes 937 and 956, respectively. The location of the hyperplane is determined by a weight vector. The training algorithm attempts to maximize the margin of the hyperplane from the ground truth classes for generalization, which may result in misclassification of one or more data points. We apply OvR classification to distinguish segmented images from a process cycle that belong to a good class from images that belong to multiple bad classes.

[0067] Random Forest Classifier The disclosed technology can apply various classifiers to distinguish images from good or healthy images from bad or unhealthy images that belong to multiple failure classes. Applied classifiers include random forest, K-nearest neighbors, multinomial logistic regression, and support vector machine. We present an embodiment of the disclosed technology using a random forest classifier as an example.

[0068] A random forest classifier (also called a random decision forest) is an ensemble machine learning technique. An ensemble technique or algorithm combines more than one technique, of the same or different types, to classify objects. A random forest classifier consists of multiple decision trees operating as an ensemble. Each individual decision tree in the random forest acts as a base classifier and outputs a class prediction. The class with the most votes becomes the prediction of the random forest model. The basic idea behind random forests is that a large number of relatively uncorrelated models (decision trees) operating as a committee will favor one of the individual constituent models.

[0069] The disclosed technology applies a random forest classifier in a two-stage classification process. A first trained random forest classifier performs the task of separating unsuccessful production images from successful production images. A second trained random forest classifier performs the task of root cause analysis of unsuccessful production images by predicting the failure class of the unsuccessful images. This two-stage classification was chosen due to the predominance of successful production runs, but one-stage classification could also be used. Another reason for choosing a two-stage approach is that it allows us to control the sensitivity threshold for classifying images as healthy or successful production images versus unhealthy or failed production images. We can increase the threshold in the first stage classification, thus causing the classifier to classify more production images as failure images. These failure images are then processed by the second stage classifier for root cause analysis by identifying the failure class.

[0070] Training a Random Forest Classifier FIG. 10A illustrates the training of two random forest classifiers as shown in example 1000. The training data consists of input features for labeled process cycle images stored in training database 138, as shown in FIG. 1. In one exemplary training of the classifiers, we used 20,000 labeled production images. The labeled images include both good images from successful production cycles and failed images from unsuccessful production cycles. The size of training database 138 will grow as labeled production images are received from laboratories performing the genotyping process.

[0071] In one implementation, we used 96 weights of components from labeled production images to train a random forest classifier. A random forest classifier with 200 decision trees and a depth of 20 performed well. It is understood that a random forest classifier with a range of 200 to 500 decision trees and a depth range of 10 to 40 is expected to provide good results for this implementation. We used randomized search cross-validation to tune the hyperparameters. The search range for depth was 5 to 150, and the search range for the number of trees was 100 to 500. Increasing the number of trees may improve model performance but may also increase the time required for training. A training database 1001 containing features for 20,000 production cycle images is used to train a binary classifier labeled as a good vs. bad classifier 151. The same training database can be used to train a root cause classifier 171 to predict failure classes. The root cause classifier 171 is trained on a training database 1021 consisting only of bad or failed production images, as shown in FIG. 10A.

[0072] Decision trees are prone to overfitting. To overcome this problem, bagging techniques are used to train decision trees in random forests. Bagging is a combination of bootstrapping and aggregation techniques. In bootstrapping, during training, we take row samples from our training database and use them to train each decision tree in the random forest. For example, a subset of selected row features may be used to train decision tree 1. Therefore, the training data for decision tree 1 may be referred to as row sample 1 with column sample 1, or RS1 + CS1. The columns or features may be selected randomly. Decision tree 2 and subsequent decision trees in the random forest are trained in a similar manner by using subsets of the training data. Note that the training data for decision trees is generated with replacement, i.e., the same row data may be used in training multiple decision trees.

[0073] The second part of the bagging technique is the aggregation part, which is applied during production. Each decision tree outputs a classification for each class. For binary classification, it can be 1 or 0. The output of the random forest is an aggregation of the outputs of the decision trees in the random forest, with the majority of votes selected as the output of the random forest. By using votes from multiple decision trees, the random forest reduces the high variance in the decision tree results, thus resulting in better prediction results. By using row and column sampling to train individual decision trees, each decision tree becomes an expert on the training records with the selected features.

[0074] During training, the output of the random forest is compared to the ground truth labels and the prediction error is calculated. During backpropagation, the weights of the 96 components (or eigenimages) are adjusted so that the prediction error is reduced. The number of components or eigenimages depends on the number of components selected from the output of principal component analysis (PCA) using a measure of explained variance. During binary classification, the good vs. bad classifier uses image description features from the training data and applies one-vs-other (OvR) classification of a good class (or healthy-labeled images) versus multiple bad classes (images labeled with one of six failure classes). The parameters of the trained random forest classifier (e.g., component weights) are stored for use in good vs. bad classification of production cycle images during inference.

[0075] Training of the root cause classifier 171 is performed in a similar manner. The training database 1021 consists of features from labeled process cycle images from faulty process cycles belonging to multiple failure classes. The random forest classifier 171 is trained using image description features for one-vs-other (OvR) classification of each failure class versus the rest of the labeled training examples.

[0076] Classification using a Random Forest classifier Here, we describe the classification of production images using trained classifiers 151 and 171. FIG. 10B illustrates a two-stage classification 1080 of production images, using a good vs. bad classifier 151 in the first stage and a root cause classifier 171 in the second stage. The process is presented using a series of process flow steps labeled 1 through 9. The process begins in step 1 by accessing a trained random forest classifier, labeled as the good vs. bad classifier 151. Input features of the production images stored in database 1030 are provided as input to classifier 151. The classifier distinguishes good images belonging to successful process cycles from bad images belonging to unsuccessful process cycles. Bad images, for example, may belong to multiple failure classes, each of which may belong to one of the six failure classes described above. The trained classifier accesses the eigenimage criteria used to analyze the production images. The trained classifier creates image description features for the production images based on a linear combination of the eigenimages. The eigenimage weights are learned during training of the classifier as described above.

[0077] When we apply one-versus-all classification, every decision tree in the random forest classifier predicts an output for each class, i.e., whether the image belongs to one of seven classes (one good class and six failure classes). Therefore, each decision tree in the random forest will output seven probability values, one value per class. The results from the decision trees are aggregated, and the majority votes are used to predict the image as good or bad. For example, if more than 50% of the decision trees in the random forest classify the image as good, the image is classified as a good image belonging to a successful production cycle. The sensitivity of the classifier can be adjusted, for example, by setting a higher threshold, which will result in more images being classified as bad. In process step 2, the output 151 from the classifier is checked. If the image is classified as a good image (step 3), the process ends (step 4). Otherwise, if the image is classified as a bad image, indicating a failed process cycle (step 5), the system invokes the root cause classifier 171 (step 6).

[0078] The root cause classifier is applied in the second stage of the two-stage process to determine the failure class of the defective image. The process continues in the second stage by accessing the production image input features for the defective image (step 7) and providing the input features to the trained root cause classifier 171 (step 8). Each decision tree in the root cause classifier 171 votes on the input image features by applying one-versus-other classification. In this case, the classification determines whether the image belongs to one of the six failure classes versus the rest of the five failure classes. Each decision tree provides a classification for each class. The majority votes from the decision trees determine the failure class of the image (step 9).

[0079] We can use other classifiers to classify good versus bad segmented images and perform root cause analysis. For example, the disclosed technology may apply a k-nearest neighbor (k-NN or KNN) algorithm to classify segmented images. The k-NN algorithm assumes that similar examples (or segmented images in our implementation) exist in close proximity. The k-NN algorithm captures the idea of ​​similarity (also called proximity or closeness) by calculating the distance between data points or images. For this purpose, linear distance (or Euclidean distance) is commonly used. In k-NN classification, the output is class membership, e.g., a good image class or a bad image class. An image is classified by the multiple votes of its neighbors, and an object is assigned to the most common class among its k nearest neighbors. The value of k is a positive integer.

[0080] To select the right value of k for our data, we run the kNN algorithm several times with different values ​​of k and select a value of k that reduces the number of errors we encounter while maintaining the algorithm's ability to make accurate predictions when given data it has not seen before. We assume that we set the value of k to 1. This can result in inaccurate predictions. We consider two clusters of data points: good images and bad images. Suppose we have a query example surrounded by many good image data points, but it is close to one bad image data point that is also within the cluster of good image data points. With k=1, the k-NN will inaccurately predict that the query example is a bad image. As we increase the value of k, the k-NN algorithm's predictions become more stable due to majority voting (in classification) and averaging (in regression). Therefore, the algorithm is more likely to make accurate predictions up to a certain value of k. As the value of k increases, we begin to observe an increasing number of errors. Values ​​of k in the range of 6 to 50 are expected to work.

[0081] Examples of other classifiers that can be trained and applied by the disclosed technology include multinomial logistic regression, support vector machines (SVM), gradient boosting trees, naive Bayes, etc. We evaluated the performance of the classifiers using three criteria: training time, accuracy, and interpretability of results. The random forest classifier performed better than the other classifiers. We briefly present the other classifiers in the following text.

[0082] Support vector machine classifiers also performed as well as random forest classifiers. SVM classifiers locate a hyperplane between the feature vector for the good class and the feature vectors for multiple bad classes. The disclosed techniques may include training a multinomial logistic regression. A multinomial regression model can be trained to predict the probability of different possible outcomes (multi-class classification). The model is used when the output is categorical. Therefore, the model can be trained to predict whether an image belongs to a good class or one of multiple bad classes. The performance of the logistic regression classifier was lower than that of the random forest and SVM classifiers. The disclosed techniques may include training a gradient boosting model, which is an ensemble of predictive models such as decision trees. The model attempts to optimize a cost function over a function space by iteratively selecting a function that points in the negative gradient direction. For example, the model can be trained to minimize the mean squared error over the training dataset. The gradient boosting model required more training time compared to other classifiers. The disclosed techniques may include training a naive Bayes classifier, which assumes that the value of a particular feature is independent of the value of any other feature. The naive Bayes classifier considers each feature to contribute independently to the probability of an example belonging to a class. The naive Bayes classifier may be trained to classify images into a good class versus multiple bad classes.

[0083] Specific Implementations The disclosed technology applies image classification for evaluation and root cause analysis of genotyping processes. Two tasks are performed by the classifier: separation of successful and unsuccessful (or failed) images, and then root cause analysis of the unsuccessful production images. The disclosed technology can be applied to process cycle images from genotyping process equipment. While the disclosed technology is described for classifying images from a genotyping process, classification can be applied to images from other types of processes that produce images of samples positioned on an imaging chip during or at the end of the process.

[0084] We first present classification of successful and unsuccessful production images. In one embodiment of the disclosed technology, a method for training a random forest classifier to classify genotyping process cycle images is described. The method for training the classifier includes accessing labeled training examples for images from the process cycle belonging to a successful (or good or healthy) class and multiple unsuccessful (or poor or unhealthy) classes. The method may include accessing a criterion for eigenimages used to analyze the images. The method includes creating image description features for each labeled training example based on a linear combination of the eigenimages. The method includes training a random forest classifier using the image description features for one-vs-other determination of the successful class versus the multiple unsuccessful classes of the labeled training examples. The method may include training another classifier for one-vs-other determination of the successful class versus the multiple unsuccessful classes of the labeled training examples. For example, the method may include training a support vector machine (SVM) classifier. The SVM classifier locates a hyperplane between the feature vector for the successful class versus the feature vectors for the rest of the classes (or, in our case, the multiple unsuccessful classes). The method may include training a multinomial logistic regression. A multinomial regression model may be trained to predict the probability of different possible outcomes (multi-class classification). The model is used when the output is categorical. Therefore, the model may be trained to predict whether an image belongs to a success class or one of multiple failure classes. The method may include training a gradient boosting model, which is an ensemble of predictive models such as decision trees. The model attempts to optimize a cost function over a function space by iteratively selecting a function that points in the negative gradient direction. For example, the model may be trained to minimize the mean squared error over a training dataset. The method may include training a naive Bayes classifier, which assumes that the value of a particular feature is independent of the value of any other feature. A naive Bayes classifier considers each feature to independently contribute to the probability of an example belonging to a class. The naive Bayes classifier may be trained to classify images into a success class versus multiple failure classes.The trained classifier parameters are stored for use in production of success vs. failure classification of process cycle images. Classifying production images using any of the classifiers described above can detect failed production runs. The classifier can provide quick feedback to the operator about the quality of the genotyping process. Feedback from the classifier can be used to correct upstream processes that can reduce processing time and waste of valuable samples.

[0085] The method for training a classifier for genotyping process cycle images may further include creating a criterion of eigenimages to be used to analyze the images. The method may include accessing based on the eigenimages. The eigenimage criterion is ordered according to a measure of explained variability. The highest-ordered eigenimage criterion that cumulatively explains more than a threshold amount of variability is selected for analyzing the process cycle image. The method may include analyzing the process cycle image using the selected eigenimage criterion. The random forest classifier may include 100 to 400 decision trees. The depth of the random forest classifier may be 10 to 40.

[0086] In a production embodiment, a method classifies production process cycle images as successful or unsuccessful. The method includes accessing a random forest classifier trained to distinguish images from process cycles belonging to a successful class from images belonging to multiple failure classes. The method may include accessing a criterion of eigenimages used to analyze the production images and creating image description features for the production images based on a linear combination of the eigenimages. The method applies the trained random forest classifier to the image description features for a one-versus-all determination of a successful class versus multiple failure classes. The method includes providing a notification on a user interface to repeat the process cycle based on the determination of the failed process cycle.

[0087] Here, we present a root cause analysis of unsuccessful (or failed) images. In one embodiment of the disclosed technology, a method is described for training a root cause random forest classifier for process cycle images or genotyping process cycle images. The method for training the classifier includes accessing labeled training examples for images from the process cycles belonging to multiple failure classes. The method may also include accessing a criterion for eigenimages used to analyze the images and creating image description features for each labeled training example based on a linear combination of the eigenimages. The method includes training a random forest classifier using the image description features for each failure class vs. other one-vs.-other decision of the labeled training examples. Parameters of the trained random forest classifier are stored for use in producing root cause classifications for process cycle images from the failed process cycles.

[0088] The method for training a root cause classifier may further include creating an eigenimage criterion used to analyze the image. The eigenimage criterion is ordered according to a measure of explained variability. The highest-ordered eigenimage criterion that cumulatively explains variability above a threshold is selected to analyze the process cycle image. The random forest classifier may include 100 to 400 decision trees. The depth of the random forest classifier may be 10 to 40.

[0089] In a production embodiment, a method for root cause analysis from failed process cycle images includes accessing a criterion of eigenimages used to analyze the failed production images and creating image description features for the failed production images based on a linear combination of the eigenimages. The method includes accessing a random forest classifier trained to distinguish images from process cycles belonging to one of a plurality of failure classes. The method includes applying the random forest classifier to the image description features for one-versus-other decisions between the plurality of failure classes, including scoring each of the plurality of failure classes versus the others, and using the resulting scores to select from among the plurality of failure classes as the likely root cause of the faulty process cycle.

[0090] The disclosed technology may use other feature generation and dimensionality reduction techniques to generate inputs for the classifier. Examples of such techniques include nonnegative matrix factorization (NMF), independent component analysis, dictionary learning, sparse principal component analysis, factor analysis, and mini-batch K-means. Variations of image decomposition and dimensionality reduction techniques may be used. The disclosed technology may also apply classifiers other than random forest classifiers to classify process cycle images. Applied classifiers may include random forests, K-nearest neighbors (KNN), multinomial logistic regression, support vector machines (SVMs), gradient boosting trees, naive Bayes, etc. Convolutional neural networks such as ImageNet may also be used as a larger body of labeled images becomes available.

[0091] In one embodiment, a single method may implement a success-versus-failure classifier and a root cause classifier in an end-to-end manner. An input image is provided as input to a first classifier that separates success images from failure images. The method includes accessing a second random forest classifier upon determining that the image-descriptive features do not belong to the success class. The second random forest classifier is trained to distinguish images from process cycles that belong to one of a plurality of failure classes. The method applies the second random forest classifier to the image-descriptive features for one-versus-others decision making among the plurality of failure classes. The method also includes scoring each of the plurality of failure classes and using the resulting score to select from among the plurality of failure classes as a possible root cause of the bad process cycle.

[0092] Another embodiment of the disclosed technology uses thresholding of areas of a segmented image to classify process cycle images. The method includes determining and applying a threshold from the intensities of pixels in a grayscale production cycle image, where the threshold is used to classify the image pixels into light and dark classes, and converting the grayscale image to a binary image. The method may include producing the light and dark image pixel classifications. The method includes segmenting the grayscale image and the converted binary image into eight or more areas that separate systematically expected noise from signal in the image. The method includes calculating average intensity values ​​for the areas in the grayscale image and the converted binary image. The method includes accessing a random forest classifier trained to classify images from process cycles based on the average intensity values ​​for the areas in the grayscale and binary images. The image is classified as belonging to a success or failure class, and a confidence score for the classification is generated. The method may include comparing the confidence score generated by the trained random forest classifier to a threshold and classifying the image as success or failure.

[0093] In one embodiment, the method presented above further comprises applying a bilateral filter to preserve edges in the production cycle image before transforming the production cycle image.

[0094] In one embodiment, the method further includes, after transforming the production cycle image, applying a Gaussian blur filter to remove speckle-like noise from the binary image.

[0095] In one embodiment, the method further includes applying morphological operations to fill holes in the binary image after transforming the production cycle image, the morphological operations including erosion and dilation.

[0096] The above computer-implemented methods may be executed on a system including computer hardware. The computer-implemented system may execute one or more of the above methods. The computer-implemented system may incorporate any of the method features described immediately above or throughout this application that apply to the method implemented by the system. For brevity, alternative combinations of system features are not individually listed. Features applicable to systems, methods, and articles of manufacture are not repeated for each statutory classification set of basic features. The reader will understand how the features identified in this section can be readily combined with basic features in other statutory classifications.

[0097] As an article of manufacture rather than a method, program instructions executable by a processor may be stored on a non-transitory computer readable medium (CRM). When executed, the program instructions perform one or more of the computer-implemented methods described above. Alternatively, the program instructions may be stored on a non-transitory CRM, which, when combined with appropriate hardware, may become one or more components of a computer-implemented system that performs the disclosed methods.

[0098] Each of the features discussed in this specific embodiment section for the method embodiment applies equally to the CRM and system embodiments. As noted above, all method features are not repeated here and should be considered again by reference. item 1. A method for training a random forest classifier for genotyping process cycle images, comprising: accessing labeled training examples of images from the process cycle belonging to a success class and a plurality of failure classes; creating image description features for each labeled training example based on a linear combination of the eigenimages; training a random forest classifier using the image description features of the labeled training examples; and storing parameters of the trained random forest classifier. 2. accessing a metric of the intrinsic image; ordering the eigenimage criteria according to the explained measure of variability; selecting the top-ordered measure of the eigenimages that cumulatively explains the variability above a threshold; Item 10. The method of item 1, further comprising: analyzing the process cycle image using the selected criteria of the intrinsic image. 3. The method of item 1, further comprising using the image description features to train a random forest classifier for one-versus-other determination of a success class versus multiple failure classes. 4. The method according to item 1, wherein the random forest classifier comprises 100 to 400 decision trees. 5. The method according to item 1, wherein the random forest classifier has a depth of 10 to 40. 6. A method for classifying genotyping process cycle images, comprising: accessing a random forest classifier trained to distinguish images from process cycles belonging to a success class from images belonging to a plurality of failure classes; generating image description features for the production image based on a linear combination of the eigenimages; applying a random forest classifier to the image description features; providing a notification on a user interface and repeating the process cycle based on the failed process cycle determination. 7. The method of item 6, further comprising using the image description features to train a random forest classifier for one-versus-other determination of a success class versus multiple failure classes. 8. A method of training a root cause random forest classifier for genotyping process cycle images, comprising: accessing labeled training examples for images from the process cycle belonging to multiple failure classes; creating image description features for each labeled training example based on a linear combination of the eigenimages; training a random forest classifier using the image description features of the labeled training examples; and storing parameters of the trained random forest classifier. 9. The method of item 8, further comprising using the image description features to train a random forest classifier for one-versus-other determination of a success class versus multiple failure classes. 10. A method for root cause analysis from a failed process cycle image, comprising: accessing a criterion for the intrinsic image used to analyze the failed production image; creating image description features for the failed production images based on a linear combination of the eigenimages; accessing a random forest classifier trained to distinguish images from process cycles belonging to one of a plurality of failure classes; 1. A method comprising: applying a random forest classifier to image description features for one-versus-other decisions between a plurality of failure classes, the method comprising scoring each of the plurality of failure classes versus the others and using the resulting scores to select from among the plurality of failure classes as the likely root cause of a failed process cycle. 11. accessing a second random forest classifier upon determining that the image description features do not belong to the success class, the second random forest classifier being trained to distinguish images from process cycles that belong to one of a plurality of failure classes; Item 3. The method of item 2, further comprising applying a second random forest classifier to the image description features, the applying including scoring each of a plurality of failure classes versus the others and using the resulting scores to select from among the plurality of failure classes as the likely root cause of the failed process cycle. 12. A method for classifying genotyping process cycle images, comprising: determining and applying a threshold from the intensities of pixels in the grayscale production cycle image, the threshold being used to classify the image pixels into light and dark classes; and converting the grayscale image to a binary image; Segmenting the grayscale image and the converted binary image into eight or more areas that separate systematically expected noise from signal in the image; Calculating a mean intensity value for an area in the grayscale image and the converted binary image; accessing a random forest classifier trained to classify images from the process cycle as belonging to a success class or a failure class based on average intensity values ​​for areas within the grayscale and binary images; comparing a confidence score generated by the trained random forest classifier to a threshold; and classifying the image as a success or failure. 13. Item 13. The method of item 12, further comprising applying a bilateral filter to preserve edges in the production cycle image before transforming the production cycle image. 14. Item 13. The method of item 12, further comprising, after transforming the production cycle image, applying a Gaussian blur filter to remove speckle-like noise from the binary image. 15. Item 13. The method of item 12, further comprising applying morphological operations to fill holes in the binary image after transforming the production cycle image, the morphological operations including erosion and dilation. 16. A non-transitory computer-readable storage medium having stored thereon computer program instructions for training a random forest classifier for genotyping process cycle images, the instructions, when executed on a processor, performing a method comprising: accessing labeled training examples of images from the process cycle belonging to a success class and a plurality of failure classes; creating image description features for each labeled training example based on a linear combination of the eigenimages; training a random forest classifier using the image description features of the labeled training examples; and storing parameters of the trained random forest classifier. 17. Further performing the method comprises: Item 17. The non-transitory computer-readable storage medium of item 16, further comprising using the image description features to train a random forest classifier for one-versus-other determination of a success class versus multiple failure classes. 18. Performing the method accessing a metric of the intrinsic image; ordering the eigenimage criteria according to the explained measure of variability; selecting the top-ordered measure of the eigenimages that cumulatively explains the variability above a threshold; Item 17. The non-transitory computer-readable storage medium of item 16, further comprising: analyzing the process cycle image using the selected criteria of the intrinsic image. 19. The non-transitory computer-readable storage medium of item 16, wherein the random forest classifier comprises between 100 and 400 decision trees. 20. The non-transitory computer-readable storage medium of item 16, wherein the random forest classifier has a depth of 10 to 40. 21. Performing the method accessing a second random forest classifier upon determining that the image description features do not belong to the success class, the second random forest classifier being trained to distinguish images from process cycles that belong to one of a plurality of failure classes; Item 19. The non-transitory computer-readable storage medium of item 18, further comprising: applying a second random forest classifier to the image description features, the applying including scoring each of a plurality of failure classes versus the others, and using the resulting scores to select from among the plurality of failure classes as the likely root cause of the failed process cycle. 22. A non-transitory computer-readable storage medium having stored thereon computer program instructions for classifying genotyping process cycle images, the instructions, when executed on a processor, performing a method comprising: determining and applying a threshold from the intensities of pixels in the grayscale production cycle image, the threshold being used to classify the image pixels into light and dark classes; and converting the grayscale image to a binary image; Segmenting the grayscale image and the converted binary image into eight or more areas that separate systematically expected noise from signal in the image; Calculating a mean intensity value for an area in the grayscale image and the converted binary image; and accessing a random forest classifier trained to classify images from a process cycle as belonging to a success class or a failure class based on average intensity values ​​for areas within the grayscale and binary images; comparing a confidence score generated by the trained random forest classifier to a threshold; and classifying the image as a success or failure. 23. Performing the method Item 23. The non-transitory computer-readable storage medium of item 22, further comprising applying a bilateral filter to preserve edges in the production cycle image before transforming the production cycle image. 24. A system including one or more processors coupled to a memory, the memory being loaded with computer instructions for training a random forest classifier for process cycle images, the computer instructions executing the instructions set forth in item 16 when executed on the processor. twenty five. accessing a metric of the intrinsic image; ordering the eigenimage criteria according to the explained measure of variability; selecting the top-ordered measure of the eigenimages that cumulatively explains the variability above a threshold; Item 25. The system of item 24, further performing the steps including: analyzing the process cycle image using the selected criteria of the intrinsic image. 26. The system of item 24, wherein the random forest classifier comprises 100 to 400 decision trees. 27. The system of item 24, wherein the random forest classifier has a depth of 10 to 40. 28. accessing a second random forest classifier upon determining that the image description features do not belong to the success class, the second random forest classifier being trained to distinguish images from process cycles that belong to one of a plurality of failure classes; 26. The system of claim 25, further performing actions including applying a second random forest classifier to the image description features, the applying including scoring each of the plurality of failure classes versus the others, and using the resulting scores to select from among the plurality of failure classes as the likely root cause of the failed process cycle. 29. A system including one or more processors coupled to a memory, the memory being loaded with computer instructions for classifying process cycle images that, when executed on the processor, perform the instructions set forth in item 22.

[0099] Computer Systems 11 is a simplified block diagram of a computer system 1100 that can be used to implement the disclosed techniques. The computer system typically includes at least one processor 1172 that communicates with a number of peripheral devices via a bus subsystem 1155. These peripheral devices can include, for example, a storage subsystem 1110 that includes a memory subsystem 1122 and a file storage subsystem 1136, user interface input devices 1138, user interface output devices 1176, and a network interface subsystem 1174. The input and output devices enable user interaction with the computer system. The network interface subsystem provides an interface to external networks, including interfaces to corresponding interface devices in other computer systems.

[0100] In one embodiment, a root cause classifier 171 for classifying failed (or bad) images is communicatively linked to the storage subsystem and the user interface input device.

[0101] The user interface input devices 1138 may include pointing devices such as keyboards, mice, trackballs, touchpads, or graphics tablets, scanners, touchscreens integrated into displays, audio input devices such as voice recognition systems and microphones, and other types of input devices. In general, use of the term "input device" is intended to encompass all possible types of devices and ways of inputting information into a computer system.

[0102] The user interface output devices 1176 can include a display subsystem, a printer, a fax machine, or a non-visual display such as an audio output device. The display subsystem may include a flat panel device such as a cathode ray tube (CRT), a liquid crystal display (LCD), a projection device, or some other mechanism for producing a visible image. The display subsystem can also provide a non-visual display such as an audio output device. In general, use of the term "output device" is intended to include all possible types of devices and methods for outputting information from the computer system to a user or to another machine or computer system.

[0103] The storage subsystem 1110 stores programming and data constructs that provide the functionality of some or all of the modules and methods described herein. These software modules are generally executed by the processor alone or in combination with other processors.

[0104] The memory used in the storage subsystem may include several memories, such as a main random access memory (RAM) 1132 for storing instructions and data during program execution, and a read only memory (ROM) 1134 with fixed instructions stored therein. The file storage subsystem 1136 may provide persistent storage for program and data files and may include a hard disk drive, associated removable media, a CD-ROM drive, an optical drive, or a removable media cartridge. Modules that implement the functionality of particular embodiments may be stored by the file storage subsystem within the storage subsystem or in other machines accessible by the processor.

[0105] The bus subsystem 1155 provides a mechanism for allowing the various components and subsystems of the computer system to communicate with each other as intended. Although the bus subsystem is shown schematically as a single bus, alternative implementations of the bus subsystem may use multiple buses.

[0106] The computer system itself can be of various types, including a personal computer, a portable computer, a workstation, a computer terminal, a network computer, a television, a mainframe, a server farm, a loosely distributed set of loosely networked computers, or any other data processing system or user device. Due to the ever-changing nature of computers and networks, the description of the computer system depicted in Figure 11 is intended only as a specific example for purposes of illustrating the disclosed technology. Many other configurations of a computer system can have more or fewer components than the computer system depicted in Figure 11.

[0107] The computer system 1100 includes a GPU or FPGA 1178. It may also include a machine learning processor hosted by a machine learning cloud platform such as Google Cloud Platform, Xilinx, and Cirrascale. Examples of deep learning processors include Google's Tensor Processing Unit (TPU), rackmount solutions such as the GX4 Rackmount Series and GX8 Rackmount Series, NVIDIA DGX-1, Microsoft's Stratix V FPGA, Graphcore's Intelligent Processor Unit (IPU), Qualcomm's Zeroth Platform with Snapdragon processors, NVIDIA's Volta, NVIDIA's DRIVE PX, NVIDIA's JETSON TX1 / TX2 MODULE, Intel's Nirvana, Movidius VPU, Fujitsu DPI, ARM's DynamicIQ, IBM TrueNorth, and others. [Explanation of symbols]

[0108] 100 systems 111 Genotyping equipment 115 Process Cycle Image Database 117 Failure Category Label Database 138 Labeled Process Cycle Image Database 151 trained good vs. bad classifiers 155 Network 168 Eigenimage Reference Database 171 Trained Root Cause Classifiers 185 Feature Generator

Claims

1. 1. A processor-implemented method for training a random forest classifier for genotyping process cycle images, comprising: accessing labeled training examples for images from the process cycle belonging to a success class and a plurality of failure classes, each failure class corresponding to a different root cause of failure, each different root cause of failure being either an operational processing error or a chemical processing error; creating image description features for each labeled training example based on a linear combination of the eigenimages; training the random forest classifier to identify specific features corresponding to the plurality of failure classes using the image description features of the labeled training examples; and storing parameters of the trained random forest classifier.

2. accessing a metric of the intrinsic image; ordering the eigenimage criteria according to the explained variability measure; selecting the top-ordered measure of the eigenimages that cumulatively explains the variability above a threshold; The method of claim 1 further comprising: analyzing the process cycle images using the selected criteria of the intrinsic images.

3. The method of claim 1 , further comprising using the image description features to train the random forest classifier for one-vs-other decision of the success class versus the plurality of failure classes.

4. 2. The method of claim 1, wherein the random forest classifier comprises 100 to 400 decision trees, the manipulation processing error is one of a spacer shift failure, an offset failure, or a surface abrasion failure, and the chemical processing error is one of a hybridization error or a reagent flow error.

5. The method of claim 1 , wherein the random forest classifier has a depth of 10 to 40.

6. accessing a second random forest classifier upon determining that the image-descriptive features do not belong to the success class, the second random forest classifier being trained to distinguish images from process cycles that belong to one of the plurality of failure classes; 3. The method of claim 2, further comprising: applying the second random forest classifier to the image description features, the applying comprising scoring each of the plurality of failure classes versus the others and using the resulting scores to select from among the plurality of failure classes as the likely root cause of a failed process cycle.

7. 1. A non-transitory computer-readable storage medium having stored thereon computer program instructions for training a random forest classifier for genotyping process cycle images, the instructions, when executed on a processor, causing the processor to: accessing labeled training examples for images from the process cycle belonging to a success class and a plurality of failure classes, each failure class corresponding to a different root cause of failure, each different root cause of failure being either an operational processing error or a chemical processing error; creating image description features for each labeled training example based on a linear combination of the eigenimages; training the random forest classifier to identify specific features corresponding to the plurality of failure classes using the image description features of the labeled training examples; and storing parameters of the trained random forest classifier.

8. The instructions further cause the processor to: The non-transitory computer-readable storage medium of claim 7 , wherein the image description features are used to train the random forest classifier for one-vs-other decision of the success class versus the plurality of failure classes.

9. The instructions further include: Access the intrinsic image criteria, ordering the eigenimage criteria according to the explained measure of variability; Select the top-ordered criteria of the eigenimages that cumulatively explain the variability above the threshold; The non-transitory computer-readable storage medium of claim 7 configured to analyze the process cycle images using selected criteria of the characteristic images.

10. 8. The non-transitory computer-readable storage medium of claim 7, wherein the random forest classifier comprises 100 to 400 decision trees, the manipulation processing error is one of a spacer shift failure, an offset failure, or a surface abrasion failure, and the chemical processing error is one of a hybridization error or a reagent flow error.

11. 8. The non-transitory computer-readable storage medium of claim 7, wherein the random forest classifier has a depth of 10 to 40.

12. The instructions further cause the processor to: accessing a second random forest classifier upon determining that the image-descriptive features do not belong to the success class, the second random forest classifier being trained to distinguish images from process cycles that belong to one of the plurality of failure classes; 10. The non-transitory computer-readable storage medium of claim 9, configured to cause the processor to: apply the second random forest classifier to the image-descriptive features, the applying including scoring each of the plurality of failure classes versus the others and using the resulting scores to select from among the plurality of failure classes as the likely root cause of a failed process cycle.

13. 1. A system including one or more processors coupled to a memory, the memory loaded with computer instructions for training a random forest classifier for genotyping process cycle images, the computer instructions, when executed on the one or more processors, causing the one or more processors to: accessing labeled training examples for images from the process cycle belonging to a success class and a plurality of failure classes, each failure class corresponding to a different root cause of failure, each different root cause of failure being either an operational processing error or a chemical processing error; creating image description features for each labeled training example based on a linear combination of the eigenimages; training the random forest classifier to identify specific features corresponding to the plurality of failure classes using the image description features of the labeled training examples; storing parameters of the trained random forest classifier.

14. The instructions further cause the one or more processors to: accessing a metric of the intrinsic image; ordering the eigenimage criteria according to the explained variability measure; selecting the top-ordered measure of the eigenimages that cumulatively explains the variability above a threshold; and analyzing the process cycle images using selected criteria of the intrinsic images.

15. 14. The system of claim 13, wherein the random forest classifier comprises 100 to 400 decision trees, the manipulation processing error is one of a spacer shift failure, an offset failure, or a surface abrasion failure, and the chemical processing error is one of a hybridization error or a reagent flow error.

16. The system of claim 13 , wherein the random forest classifier has a depth of 10 to 40.

17. The instructions further cause the one or more processors to: accessing a second random forest classifier upon determining that the image-descriptive features do not belong to the success class, the second random forest classifier being trained to distinguish images from process cycles that belong to one of the plurality of failure classes; 14. The system of claim 13, further comprising: applying the second random forest classifier to the image description features, wherein the one or more processors are configured to score each of the plurality of failure classes versus the others and use the resulting scores to select from among the plurality of failure classes as the likely root cause of a failed process cycle.

Images