Root Cause Analysis of Process Cycle Images Based on Machine Learning

Abstract

This invention discloses a technique for classifying process cycle images to predict the success or failure of a process cycle. The disclosed technique includes capturing and processing images of segments arranged on an image generation chip during genotyping. Image descriptive features of the production cycle images are created and provided as input to a classifier. The trained classifier separates successful production images from unsuccessful or failed production images. These failed production images are further classified into various categories of failure by a trained root cause classifier.

Description

[0001] Priority application

[0002] This application claims the benefit of U.S. non-provisional patent application No. 17 / 161,595 (Attorney's File No. ILLM 1026-2 / IP-1911-US), filed January 28, 2021, entitled "MACHINE LEARNING-BASED ROOT CAUSE ANALYSIS OF PROCESS CYCLE IMAGES," which claims the benefit of U.S. provisional patent application No. 62 / 968,950 (Attorney's File No. ILLM 1026-1 / IP-1911-PRV), filed January 31, 2020, entitled "MACHINE LEARNING-BASED ROOT CAUSE ANALYSIS OF PROCESS CYCLE IMAGES." The priority application is incorporated herein by reference for all purposes. Technical Field

[0003] The disclosed techniques involve image classification for evaluation and root cause failure analysis of production processes. Background Technology

[0004] The topics discussed in this section should not be considered prior art simply because they are mentioned here. Similarly, problems mentioned in this section or related to the topics provided as background art should not be assumed to have been previously recognized in the prior art. The topics in this section merely represent different methods, which themselves may correspond to specific implementations of the technology protected by the claims.

[0005] Genotyping is a multi-day process. It is susceptible to errors in mechanical and chemical processing. Samples collected for genotyping are extracted and distributed across segments and regions of an image generation chip. The samples are then chemically processed through multiple steps to generate fluorescence images. This process generates a quality score for each segment analyzed. However, this quality score does not reveal the root cause of low-quality process failures. In some cases, images of failed segments may still produce acceptable quality scores.

[0006] Therefore, there is an opportunity to introduce new methods and systems to evaluate segment images and identify the root causes of failures during genotyping. Attached Figure Description

[0007] In the accompanying drawings, similar reference numerals generally refer to similar parts in all different views. Furthermore, the drawings are not necessarily drawn to scale, but rather emphasize the principles of the disclosed technology. In the following description, various specific embodiments of the disclosed technology are described with reference to the following drawings, wherein:

[0008] Figure 1 A schematic diagram of the system architecture is shown, in which process cycle images from a genotyping instrument are classified and the root cause of defective images is determined.

[0009] Figure 2 Examples Figure 1 The feature generator is a subsystem component.

[0010] Figure 3 The process steps of an exemplary genotyping procedure are presented.

[0011] Figure 4 The image presents the segments arranged in the image generation process after successful processing.

[0012] Figure 5A and Figure 5B Examples of images of failed segments due to hybridization failure during the genotyping process are presented.

[0013] Figure 5C and Figure 5D An example of a failed segment image is presented, resulting from a failed interval shift.

[0014] Figure 5E An example of a failed segment image due to an offset failure is presented.

[0015] Figure 5F Examples of failed segment images are presented, resulting from surface abrasion failure.

[0016] Figure 5G and Figure 5H An example of a failed segment image is presented, caused by reagent flow failure.

[0017] Figure 5I Examples of failed or unhealthy segment images with unknown sources of failure are presented.

[0018] Figure 6A and Figure 6B This is an example of a basis of 96 intrinsic image components selected using the ordered principal components generated by principal component analysis (PCA).

[0019] Figure 6C and Figure 6D From Figure 6A and Figure 6B A magnified view of the first 40 intrinsic image components selected from 96 images.

[0020] Figure 7A The example illustrates image scaling and flattening for segment images input into principal component analysis.

[0021] Figure 7B An example is given of using principal component analysis to create a basis for an intrinsic image.

[0022] Figure 8A An example is shown that uses the intensity of the region of a segment to generate a feature.

[0023] Figure 8B Another example is shown that uses the intensity of the region of a segment to generate a feature.

[0024] Figure 9 This is a diagrammatic illustration of a one-to-many (OvR) classifier.

[0025] Figure 10A Examples are given of training binary (good vs. bad) classifiers and multi-class (root cause) classifiers using labeled training data that includes process cycle images.

[0026] Figure 10B A two-step process is illustrated, in which production process cycle images are classified as good and bad, and the failure category of bad images is further determined.

[0027] Figure 11 It is a simplified block diagram of a computer system that can be used to implement the disclosed technology. Detailed Implementation

[0028] The following discussion is presented to enable any person skilled in the art to implement and use the disclosed technology, and is provided in the context of a particular application and its requirements. Various modifications to the specific embodiments disclosed will be apparent to those skilled in the art, and the general principles defined herein can be applied to other specific embodiments and applications without departing from the spirit and scope of the disclosed technology. Therefore, the disclosed technology is not intended to be limited to the specific embodiments shown, but is to be accorded the broadest scope consistent with the principles and features disclosed herein.

[0029] Introduction

[0030] The disclosed techniques apply visual systems and image classification to the evaluation of genotyping and root cause failure analysis. Two distinct approaches are described: one involving intrinsic images and the other based on region-wise thresholding. Principal Component Analysis (PCA) and Nonnegative Matrix Factorization (NMF) are among the techniques disclosed. Other dimensionality reduction techniques that can be applied to images include Independent Component Analysis, dictionary learning, sparse principal component analysis, factor analysis, and mini-batch K-means. Variations of image decomposition and dimensionality reduction techniques can be used. For example, PCA can be implemented using Singular Value Decomposition (SVD) or as kernel PCA. The outputs of these techniques are fed as input to a classifier. Applied classifiers can include Random Forest, K-Nearest Neighbors (KNN), Multinomial Logistic Regression, Support Vector Machine (SVM), Gradient Boosting Tree, Naive Bayes, etc. As larger subjects of labeled images become available, convolutional neural networks, such as ImageNet, can also be used.

[0031] The production process is susceptible to errors in mechanical and chemical handling. Collected samples are extracted, distributed across segments and regions of the BeadChip, and then chemically processed through multiple steps to generate fluorescence images. The final fluorescence images, and even intermediate fluorescence images, can be analyzed to monitor production and perform failure analysis.

[0032] The vast majority of production analyses are successful. Failed analyses are currently understood to fall into five categories plus any remaining failure categories. The five failure categories are hybridization or mixing failure, spacer displacement failure, offset failure, surface abrasion failure, and reagent flow failure. The remaining categories are unhealthy patterns due to mixing effects, unidentified causes, and weak signals. Over time, especially when root cause analysis leads to improved production, more diverse causes can be identified.

[0033] The first image processing technique applied to quality control and failure analysis evolved from facial recognition via intrinsic face analysis. From tens of thousands of labeled images, a linear basis of 40 to 100 or more image components is identified. One method for forming the eigenbase is to perform principal component analysis (PCA), followed by ranking the components according to a measure of interpretable variability. It was observed that 40 components explained the majority of the variability. Over 100 components, additionally, appeared to reflect noise patterns or natural variability in sample processing. The number of expected relevant components is affected by image resolution. Here, a resolution reduction was applied, allowing analysis of segments of the image generation chip at a resolution of 180 × 80 pixels. This was sufficient to distinguish between successful and unsuccessful production, and then the root cause of failure was categorized into six failure categories. Formal sensitivity analysis was not applied, but it was expected that images with slightly lower resolutions could also work, and images with resolutions 4 to 22 times that of the original resolution could be processed in the same way, but with increased computational cost. Each image to be analyzed by intrinsic image analysis is represented as a weighted linear combination of the base images. Each weight in the ordered set of base components is used as a feature for training the classifier. For example, in one implementation, a random forest classifier is trained using 96 weights of the components of a labeled image. The random forest classifier has 200 trees and a depth of 20. The random forest classifier performs two tasks: separating successful and unsuccessful production images, and then performing root cause analysis on the unsuccessful production images. This two-stage classification is chosen because successful production runs are dominant, but a one-stage classification can also be used.

[0034] The second image processing technique applied involves thresholding image regions. The production image of a segment of the image generation chip captures several physically separated regions. Structures bordering the segments and the structures of the physical regions separating the segments are visible in the production image. The thresholding strategy involves separating the effective regions from the boundary structures and then distinguishing between the separated regions. Optionally, structures separating the physical regions can also be filtered out from the image. At least the effective regions are thresholded for emission. This thresholding determines how many effective regions are generating the desired signal intensity. The success or failure of each effective region is evaluated after thresholding. Failure modes between regions and segments of the image generation chip can be further evaluated for root cause classification.

[0035] Production image processing can be performed immediately during production to detect failed production runs and determine root causes, even faster than reading results from an image generation chip and judging quality. This image processing can be completed much faster because shrinking the image 20 times on one side significantly reduces computational requirements, and direct processing of the reduced-resolution image does not require associating individual luminescent pixels in the area with individual probes. This rapid turnaround for root cause analysis allows for correcting upstream processes before wasting chemicals and processing time.

[0036] environment

[0037] This describes a system for early prediction of failures in genotyping systems. Genotyping is the process of determining differences in an individual's genetic makeup (genotype) by examining their DNA sequence using bioassays and comparing it to a reference sequence. Genotyping allows researchers to explore genetic variants, such as single nucleotide polymorphisms (SNPs) and structural variations in DNA. Reference Figure 1 To describe the system, this diagram illustrates an architecture-level schematic of the system according to a specific implementation. Because Figure 1 It is an architecture diagram, so some details have been intentionally omitted to improve the clarity of the description. Figure 1 The discussion is organized as follows. First, the elements in the attached diagram are described, followed by their interconnections. Then, the purpose of the elements in the system is described in more detail.

[0038] Figure 1 This includes system 100. This paragraph names the labeled parts of system 100. The illustration shows the genotyping instrument 111, the process cycle image database 115, the failure category label database 117, the labeled process cycle image database 138, the trained good and bad classifier 151, the feature image base database 168, the trained root cause classifier 171, the feature generator 185, and the network 155.

[0039] The disclosed technology is applicable to various genotyping instruments 111, also known as genotyping scanners and genotyping platforms. Network 155 communicatively couples the genotyping instrument 111, process cycle image database 115, failure category label database 117, labeled process cycle image database 138, trained good and bad classifiers 151, feature image base database 168, trained root cause classifier 171, and feature generator 185 to each other.

[0040] Genotyping instruments may include Illumina's BeadChip imaging system, such as ISCAN. TMThe system and instrument can detect the fluorescence intensity of hundreds to millions of microbeads arranged in segments at mapped locations on an image generation chip. Genotyping instruments may include an instrument control computer that controls various aspects of the instrument, such as laser control, precision mechanical control, excitation signal detection, image registration, image extraction, and data output. Genotyping instruments can be used in a variety of physical environments and can be operated by technicians of varying skill levels. Sample preparation can take two to three days and may include both manual and automated sample handling.

[0041] Examples Figure 3 The exemplary genotyping process 300 in Illumina is described in the process steps of this exemplary genotyping process. This exemplary genotyping process is referred to as Illumina's INFINIUM. TM Assay Workflow. This process is designed to study numerous SNPs at a wide locus reuse level. Using a single bead type and a two-color (e.g., red and green) channel approach, the process scales genotyping from hundreds to millions of SNPs per sample. The process begins with the addition and extraction of a DNA sample. The process can be operated with relatively low input samples, such as 200 ng, which allows for the determination of millions of SNP loci. The sample is then amplified. The amplification process can take several hours to overnight to complete. The amplified sample undergoes controlled enzymatic cleavage. Alcohol precipitation and resuspension are then performed. An image generation chip is prepared for hybridization in a capillary flow chamber. The sample is then applied to the prepared image generation chip and incubated overnight. During this overnight hybridization, the sample is annealed to locus-specific 50-mers covalently linked to up to millions of bead types. One bead type corresponds to each allele at each SNP locus. Allele specificity is conferred by enzymatic base extension and subsequent fluorescent staining. Genotyping instruments or scanners (such as ISCAN) are used. TM The system detects the fluorescence intensity of the microbeads and performs genotyping.

[0042] In one example, genotyping results are presented using a metric called the “call rate.” This metric represents the percentage of genotypes correctly scanned on the image generation chip. Individual call rates are reported for each segment of the image generation chip. Thresholds can be used to accept or reject results. For example, a genotyping result for a segment can be accepted using a call rate of 98% or higher. Different thresholds, such as below 98% or above 98%, can be used. If a segment’s call rate is below the threshold, the genotyping process is considered a failure. Genotyping processes can span many days, and therefore, repetition is very costly. Failures in the genotyping process can occur due to operational errors (such as mechanical or processing errors) or chemical processing errors.

[0043] The genotyping system can provide process cycle images of image generation chip segments and their recall rates upon completion of the genotyping process. The disclosed technique can process these segment images to classify whether the genotyping process was successful (good images of the segments) or unsuccessful (poor or failed images of the segments). The disclosed technique can further process poor or failed images to determine the failure category. Currently, the system can classify failed images into one of six failure categories: hybridization or mixing failure due to mixing effects, unknown causes, weak signals, etc.; spacer region shift failure; offset failure; surface abrasion failure; reagent flow failure; and images of overall unhealthy conditions. Over time, especially when root cause analysis leads to improved production, more diverse causes can be identified.

[0044] Now for reference Figure 1 This provides a description of the remaining components of system 100. Failure category labels for six failure types can be stored in a failure category label database 117. A training dataset of labeled process image cycles is stored in database 138. Labeled training examples can include successful (good) and unsuccessful (bad) process cycle images. Unsuccessful process cycle images are labeled as belonging to one of the six failure categories listed above. In one embodiment, training database 138 includes at least 20,000 training examples. The size of the training database can be increased as more labeled image data is collected from the laboratory using a genotyping instrument.

[0045] The disclosed techniques include two independent image processing techniques for extracting features from process cycle images. A feature generator 185 can be used to apply one of these techniques to extract features from the process cycle images for input into a machine learning model. The first image processing technique is derived from facial recognition via intrinsic facial analysis. A relatively small number of linear bases, such as 40 to 100 or more image components, are identified from tens of thousands of labeled images. One method for forming intrinsic bases is Principal Component Analysis (PCA). The process cycle images are represented as a weighted linear combination of base images used as input to a classifier. For example, in one implementation, the classifier is trained using 96 weights of the components of the labeled images. The bases of the intrinsic images can be stored in a database 168.

[0046] The second image processing technique for feature extraction includes thresholding of the segmented image. The generated image of a segment from the image generation chip captures several physically separated regions. The generated image shows the structures bordering the segments and the structures of the physical regions separating the segments. The thresholding technique determines how many effective regions are generating the desired signal strength. The output of the thresholding technique can be provided as input to a classifier to distinguish between good and bad images. Failure modes between regions and segments of the image generation chip can be further evaluated for root cause analysis.

[0047] Image features of the production images generated by feature generator 185 are fed as input to trained classifiers 151 and 171. Two types of classifiers are trained. A good vs. bad classifier can predict successful and unsuccessful production images. A root cause analysis classifier can predict the failure category of unsuccessful images. An example of a classifier used in the disclosed technique includes a random forest classifier. Other examples of classifiers that can be applied include K-Nearest Neighbors (KNN), multinomial logistic regression, and support vector machines. As larger subjects of labeled images become available, convolutional neural networks (CNNs) can also be used.

[0048] Finish Figure 1 The components of the aforementioned system 100 are communicatively coupled to network 155. The actual communication path can be point-to-point over public and / or private networks. Communication can occur on various networks, such as private networks, VPNs, MPLS circuits, or the Internet, and can use appropriate application programming interfaces (APIs) and data exchange formats, such as Representational State Transfer (REST), JavaScript Object Notation (JSON), Extensible Markup Language (XML), Simple Object Access Protocol (SOAP), Java Message Service (JMS), and / or Java Platform Modules (MPLM) systems. All communication can be encrypted. Communication is typically conducted via protocols such as EDGE, 3G, 4G LTE, Wi-Fi, and WiMAX, through networks such as LANs (Local Area Networks), WANs (Wide Area Networks), telephone networks (Public Switched Telephone Network (PSTN), Session Initiation Protocol (SIP), wireless networks, point-to-point networks, star networks, token ring networks, hub networks, and the Internet (including mobile Internet). Figure 1 The engine or system component is implemented by software running on different types of computing devices. Exemplary devices are workstations, servers, computing clusters, blade servers, and server groups. Additionally, various authorization and authentication technologies, such as username / password, Open Authorization (OAuth), Kerberos, secure digital certificates, and more, can be used to secure communications.

[0049] Feature generator—system component

[0050] Figure 2 This is a high-level block diagram 200 of the components of the feature generator 185. These components are used as follows... Figure 11 The description presents various computer systems implemented in this computer. The illustrated components may be combined or further separated during implementation. Feature generator 185 consists of two high-level components implementing two image processing techniques: a feature generator 235 based on principal component analysis or PCA and a feature generator 255 based on image segmentation. The PCA-based feature generator includes an image scaler 237 and an intrinsic image basis creator 239. The image segmentation-based feature generator 255 includes an image converter 257 and an intensity extractor 259. Further details of the specific implementation of these components are presented in the following sections.

[0051] PCA-based feature generator

[0052] The first image processing technique evolved from facial recognition through intrinsic face analysis. One method for forming intrinsic bases is Principal Component Analysis (PCA). A PCA-based feature generator 235 applies PCA to the resized processed image. An image scaler component 237 resizes (or rescales) the images in the process cycle. Scaling reduces the size of the processed images, allowing them to be processed in a computationally efficient manner by the intrinsic image base creator component 239. Details of these components are presented in the following sections.

[0053] Image scaler

[0054] Higher-resolution images obtained from genotyping instruments or scanners may require more computational resources to process. Images obtained from genotyping scanners are resized by image scaler 237 to analyze images of segments of the generated chip at a reduced resolution of 180×80 pixels. In one example, the segment image obtained from the scanner has a resolution of 3600×1600 pixels, and the image size is resized by applying a reduction to 1 / 20 of the resolution. This resolution is sufficient to distinguish between successful and unsuccessful production images, and then the root cause of failure is categorized into six failure categories. Images rescaled from 4x to 25x the original resolution can be processed in the same manner.

[0055] The disclosed techniques can be applied to various interpolation techniques to reduce the size of produced images. In one specific implementation, bilinear interpolation is used to reduce the size of a segmented image. Linear interpolation is a curve fitting method that uses a linear polynomial to construct a range of new data points with a known set of data points. Bilinear interpolation is an extension of linear interpolation used to interpolate functions of two variables (e.g., x and y) on a two-dimensional grid. Bilinear interpolation is performed by first using linear interpolation in one direction and then similarly using linear interpolation in the second direction. Although each step is linear in both sampled values ​​and locations, the interpolation as a whole is not linear but quadratic in terms of sampled locations. Other interpolation techniques can also be used to reduce the size of segmented images (rescaling), such as nearest-neighbor interpolation using pixel area relationships and resampling.

[0056] Intrinsic Image Base Creator

[0057] The first image processing technique applied to segmented images to generate input features for a classifier evolved from face recognition through feature face analysis. From tens of thousands of labeled images, a linear basis of 40 to 100 or more image components is identified. One method for forming the basis of intrinsic images is Principal Component Analysis (PCA). The set B of elements (vectors) in a vector space V is called a basis if each element of V can be uniquely written as a linear combination of elements of B. Equivalently, B is a basis if the elements of B are linearly independent, and each element of V is a linear combination of elements of B. A vector space can have several bases. However, all bases have the same number of elements, called the dimension of the vector space. In our technique, the basis of the vector space is the intrinsic image.

[0058] PCA is typically used to reduce the dimensionality of a d-dimensional dataset by projecting it onto a k-dimensional subspace, where k < d. For example, the resized labeled images in a training database describe vectors in a 14,400-dimensional space (180 × 80 pixels), i.e., d = 14,400. In other words, an image is a point in a 14,400-dimensional space. Eigenspace-based methods approximate image vectors with lower-dimensional feature vectors. The main assumption behind this technique is that the image space (given by the feature vectors) has a lower dimension than the image space (given by the number of pixels in the image), and image recognition can be performed in this reduced space. The images of the segments of an image generation chip are similar in overall configuration and will not be randomly distributed in this huge space, and thus can be described by a relatively low-dimensional subspace. The PCA technique finds vectors that best account for the distribution of the segment images within the entire image space. These vectors define a subspace of the image, also called the "image space". In our specific implementation, each vector describes an image of 180 × 80 pixels and is a linear combination of the images in the training data. In the following text, details of how to use principal component analysis (PCA) to create a basis of eigenimages are presented.

[0059] The PCA-based analysis of labeled training images may include the following five steps.

[0060] Step 1: Access multidimensional related data

[0061] The first step in applying PCA is to access the high-dimensional data. In one instance, 20,000 labeled images are used as training data. Each image is resized to a resolution of 180 × 80 pixels and represented as a point in a 14,400-dimensional space, one dimension per pixel. This technique can handle images with higher or lower resolutions than those specified above. As more labeled images are collected from the laboratory, the size of the training dataset is expected to increase.

[0062] Step 2: Data Standardization

[0063] Normalization (or Z-score normalization) is the process of rescaling features so that they have the properties of a Gaussian distribution with a mean equal to zero or μ = 0 and a standard deviation from the mean equal to 1 or σ = 1. Normalization is performed to build features with similar ranges to each other. The standard score of an image can be calculated by subtracting the mean (of the image) from the image and dividing the result by the standard deviation. Since PCA produces a feature subspace that maximizes the variance along the axes, it helps to normalize the data so that it is centered on the axes.

[0064] Step 3: Calculate the covariance matrix

[0065] The covariance matrix is ​​a d×d matrix in d-dimensional space, where each element represents the covariance between two features. The covariance of two features measures how much they change together. This change is the average of the squared deviations of the features from their means. The covariance is the average of the products of the deviations of the feature values ​​from their means. Consider features k and j. Let {x(1,j),x(2,j),…,x(i,j)} be the set of i examples of feature j, and let {x(1,k),x(2,k),…,x(i,k)} be the set of i examples of feature k. Similarly, let… Let be the mean of feature j, and Let j be the mean of feature k. The covariance of feature j and feature k is calculated as follows:

[0066]

[0067] The calculation of the covariance matrix can be expressed by the following matrix equation:

[0068]

[0069] The average vector can be represented as:

[0070]

[0071] The mean vector is a d-dimensional vector, where each value in the vector represents the sample mean of a feature column in the training dataset. The covariance value σ... jk It can be in -(σ ij )(σ ik (i.e., inverse linear correlation) to +(σ) ij )(σ ik The variation between linear correlations. When there is no correlation between two features, σ jk The value is zero.

[0072] Step 4: Calculate the eigenvectors and eigenvalues

[0073] The eigenvectors and eigenvalues ​​of the covariance matrix represent the core of PCA. Eigenvectors (or principal components) determine the orientation of the new feature space, and eigenvalues ​​determine their magnitudes. In other words, eigenvalues ​​explain the variation of the data along the axes of the new feature space. Eigenvalue decomposition is a matrix factorization method that uses the eigenvectors and eigenvalues ​​of a matrix to represent the matrix. An eigenvector is defined as a vector that changes only by a scalar when a linear transformation is applied to it. If A is a matrix representing a linear transformation, v is an eigenvector, and λ is the corresponding eigenvalue, then it can be expressed as Av = λv. A square matrix can have as many eigenvectors as its dimension. If all eigenvectors are represented as columns of a matrix V and the corresponding eigenvalues ​​as entries of a diagonal matrix L, then the above equation can be expressed as AV = VL. In the case of the covariance matrix, all eigenvectors are orthogonal to each other and are the principal components of the new feature space.

[0074] Step 5: Use interpretable variance to select the basis of the intrinsic image.

[0075] In our specific implementation, the above steps can produce 14,400 principal components, equal to the dimension of the feature space. Eigen pairs consist of eigenvectors and scalar eigenvalues. Feature pairs can be classified based on their eigenvalues, and a basis for the eigenimage is created using a metric called "interpretable variance." Interpretable variance indicates how much information (or variance) can be attributed to each principal component. The results of the interpretable metric can be plotted on a two-dimensional graph. The principal components for classification are represented along the x-axis. A graph indicating the cumulative interpretable variance can be plotted. The top m components representing the principal portion of the variance can be selected.

[0076] In our implementation, the first 40 components represent a high percentage of the explained variance; therefore, the first 40 principal components were selected to form the basis of the new feature space. In other implementations, 25 to 100 principal components, or more than 100, up to 256 or 512 principal components, can be selected to create the basis of the intrinsic images. Each production image to be analyzed by intrinsic image analysis is represented as a weighted linear combination of the basis images. Each weight of the ordered set of basis components is used as a feature for training the classifier. For example, in one implementation, 96 weights of the components of the labeled image are used to train the classifier.

[0077] The disclosed techniques can utilize other image decomposition and dimensionality reduction techniques. For example, compared to PCA, which learns a complete representation of an image, Nonnegative Matrix Factorization (NMF) learns a part-based representation of an image. Unlike PCA, NMF learns to represent an image using a set of base images that are partly similar to the image. NMF decomposes matrix X into two matrices W and H, with the property that none of the three matrices have negative elements. Assume that matrix X is set such that there are n data points (such as images of segments on an image generation chip), each data point having p dimensions (e.g., 14,400). Therefore, matrix X has p rows and n columns. The goal is to reduce the p dimensions to r dimensions, or in other words, to create an r-order approximation. NMF approximates matrix X as the product of two matrices: W (p rows and r columns) and H (r rows and n columns).

[0078] The matrix W is interpreted as follows: each column represents a basis element. A basis element refers to a component present in the n original data points (or images). These are building blocks from which an approximation of all the original data points or images can be reconstructed. The matrix H is interpreted as follows: each column gives the coordinates of the data points in the basis matrix W. In other words, it explains how to reconstruct an approximation of the original data points based on a linear combination of the building blocks in matrix W. In the case of facial images, the basis elements (or base images) in matrix W can include features such as eyes, nose, lips, etc. The columns of matrix H indicate which features are present in which images.

[0079] Feature generator based on image segmentation

[0080] A second image processing technique for extracting features from process periodic images is based on thresholding of image regions. The image segmentation-based feature generator 255 applies thresholding by first segmenting the image of the image generation chip using the image segmenter 257, and then extracting the intensity of the effective region or region of interest from the segmented image. This thresholding determines how much of the effective region is generating the desired signal intensity.

[0081] The image generation chip may include multiple segments organized into rows and columns, such as 24, 48, 96, or more. This design allows for the processing of multiple samples in a single process cycle, as many samples can be processed in parallel (one sample per segment). The segments are physically separated from each other, preventing samples from mixing. Furthermore, the segments may be organized into multiple parallel regions called "grooves." Therefore, the structures at the boundaries of the segments and grooves are visible in the process cycle images from the genotyping scanner. Details of two components of the image segmentation-based feature generator 255 are presented below, which implements techniques for transforming segment images to extract image features.

[0082] Image converter

[0083] Image converter 257 employs a series of image conversion techniques to prepare a segmental image for extracting intensity from the region of interest. In one specific implementation, the image conversion and intensity extraction process is performed by some or all of the following five steps. Image conversion transforms the grayscale image of the segment into a binary image composed of dark and bright pixels. The average intensity values ​​of the effective regions in the grayscale image and the binary image are provided as input features to a classifier to classify the image as healthy (good) or unhealthy (bad). Details of the image conversion steps are presented in the following text, including applying thresholding to convert the grayscale image into a binary image. Processing steps include applying filters to remove noise.

[0084] The first step in the image transformation process is to apply a bidirectional filter to process the periodic segments of the image. A bidirectional filter is a technique for smoothing an image while preserving edges. The bidirectional filter replaces the intensity of each pixel with a weighted average of the intensity values ​​from its neighboring pixels. Each neighboring pixel is weighted by a spatial component that penalizes distant pixels and a range component that penalizes pixels with different intensities. The combination of these two components ensures that only nearby, similar pixels contribute to the final result. Therefore, a bidirectional filter is an efficient way to smooth an image while preserving its discontinuities or edges. Other filters, such as median filters and anisotropic diffusion, can be used.

[0085] The second step in image transformation can be to apply thresholding to the output image from step 1. In one specific implementation, Otsu's method (Otsu, N., 1979, "A threshold selection method from gray-level histograms", IEEE Transactions on Systems, Man, and Cybernetics, Vol. 9, No. 1) is applied. This method uses an intensity histogram and searches for a threshold to maximize the weighted sum of the gray-level variances assigned to pixels of intensity levels that are dark and light. Otsu's method attempts to maximize the inter-class variance. The basic idea is that a well-thresholded class should be distinct relative to the intensity values ​​of its pixels, and conversely, the threshold that gives the best separation between classes based on the class intensity values ​​would be the optimal threshold. Furthermore, Otsu's method has the characteristic of being based entirely on computations performed on the histogram of the image, which is an easily obtainable one-dimensional array. For more details, see Section 10.3.3 of Gonzalez and Woods' "Digital Image Processing" (3rd edition).

[0086] The third step in image transformation is to apply a denoising Gaussian blur filter to remove speckled noise. Noise can contaminate the process periodic image with small specks. Gaussian filtering is a weighted average of the intensities of neighboring locations, with weights decreasing with spatial distance from the center.

[0087] The fourth step of the image transformation involves image morphological operations. Morphological transformations are used to process the binary output image from the third step to fill holes in the image. A hole can be defined as a background region (represented by 0) surrounded by connected boundaries of foreground pixels (represented by 1). Two basic image morphological operations are "erosion" and "dilation." In the erosion operation, a kernel slides (or moves) over the binary image. If all pixels below the kernel are 1, a pixel (1 or 0) in the binary image is considered a 1. Otherwise, it is eroded (becomes 0). The erosion operation is used to remove isolated 1s from the binary image. However, erosion also shrinks clusters of 1s by eroding the edges. The dilation operation is the opposite of erosion. In this operation, when the kernel slides over the binary image, if at least one pixel below the kernel has a value of 1, the values ​​of all pixels in the binary image region overlaid by the kernel are changed to 1. If a dilation operation is applied to the binary image after the erosion operation, the effect is to close small holes (represented by 0s in the image) within clusters of 1s. The output from this step is provided as input to the intensity extractor component 259, which performs the fifth step of the image conversion technique.

[0088] Strength extractor

[0089] Intensity extractor 259 divides the segment image into valid regions or segments by filtering out structures at the boundaries of segments and slots. The intensity extractor can apply different segmentations to divide the segment image from eight into seventeen or more valid regions. Examples of regions in the segment image include four slots, four corners, four edges between corners, and various vertical and horizontal lines at the boundaries of segments and slots. Regions corresponding to known structures that separate the valid regions are then removed from the image. The remaining valid region portion of the image is processed by intensity extractor 259. Intensity values ​​are extracted from each valid region of the transformed image and the corresponding untransformed image and averaged. For example, if intensity values ​​are extracted from 17 valid regions of the transformed image, the intensity extractor also extracts intensity values ​​from the same 17 valid regions of the untransformed image. Therefore, a total of 34 features are extracted from each segment image.

[0090] In the case of a binary image, the average intensity of the effective region can be between 1 and 0. For example, the intensity of dark pixels is considered to be 0 and the intensity of bright (or blank) pixels to be 1. If all pixels in the effective region are dark pixels, the average intensity of the effective region will be 0. Similarly, if all pixels in the effective region are bright pixels, the intensity of the region will be 1. Effective regions in a healthy image appear as blank or bright in the binary image, while dark pixels represent an unhealthy image. The average intensity of the corresponding effective regions in the grayscale image is also extracted. The average intensity of the effective regions from the grayscale image and the converted binary image is provided as input to the good and bad classifiers. In one specific implementation, the classification confidence score from the classifier is compared to a threshold to classify the image as a healthy (good) image or an unhealthy (bad) image. An example threshold is 80%. A higher threshold may result in more images being classified as unhealthy.

[0091] Process cycle image

[0092] Now we present examples of successful and unsuccessful image generation from segments on the image generation chip. Figure 4 This is illustration 400 of a production image of 24 segments on an image generation chip. The segments are arranged in twelve rows and two columns. Each segment has four slots. Illustration 400 shows a segment image of a successful production cycle. Other configurations with segments, such as image generation chips including 48, 96, or more segments, can also be used. Examples of segment images of unsuccessful production cycles are presented in the figures below. The production process is susceptible to operational and chemical processing errors. Operational defects can be caused by mechanical or sample handling problems. Chemical processing errors can be caused by problems in the sample or by the chemical treatment of the sample. The disclosed technique attempts to classify poorly processed image cycles due to operational and chemical processing errors.

[0093] Figure 5A Example 510 shows a segment image from an unsuccessful production cycle. Figure 5A In the image generation chip, the image 512 of the segment in the second column and seventh row is dark in the lower half and slightly lighter in the upper half. This failure is related to the hybridization process. Therefore, the failed image of the segment is marked as a "Hyb" failure. Hybridization failure can also occur due to failure of the robot handling the sample during sample preparation on the image generation chip. As shown, the recall rate of this segment is below the 98% threshold. In some cases, the recall rate of segments from genotyping instruments may be higher than the threshold, but even so, the segment image may still fail due to hybridization errors.

[0094] It can be noted that in Figure 510, the image of segment 514 at row 11 and column 2 has a dark area on the right wall. This could also indicate a processing problem; however, the overall recall rate of this image is above the threshold and it is not marked as a failed image. There is sufficient sample redundancy on the segments, so small segments with obvious failures can be ignored and will not cause errors in the results. For example, in one instance, the scanner read fluorescence from approximately 700K probes on a segment with redundancy of 10. Therefore, the recall rate is based on readings from approximately 7 million probes. Figure 5B Figure 515 presents another example of hybridization failure. Due to hybridization failure, the four segments within the dashed boxes on the image generation chip show poorly produced images of those segments. Note that the call rate values ​​for these four segments are above the pass threshold, but the images of these segments are marked as failed due to hybridization errors.

[0095] Figure 5C Illustration 520 presents nine segment images illustrating unsuccessful processing due to spacer region shifting failure. When preparing samples on the segments on the image generation chip, dark markers are placed around the segments. Spacer regions separate the sample in each segment from other samples in adjacent segments. Incorrect placement of the markers can obstruct part of the image signal. Figure 5C As shown, offset errors can occur in multiple adjacent segments. The top portions of the nine segments in this figure appear dark. The darker areas at the top of the segments increase as you move from left to right. This spatial offset problem is an operational error, caused by inaccurate placement of markers by lab technicians during sample preparation on the image generation chip. Figure 5D Three more examples of segment failure images are presented, showing examples of failures due to gap shifting. Box 525 shows five segment images with gap shifting failure, as the top portion of the segment image is dark and the width increases from the upper right to the upper left. Box 527 shows two segment images indicating a failure process due to gap shifting problems at the bottom portion of the segment. Similarly, box 529 shows images of two segments that failed due to spatial offset problems.

[0096] Figure 5E An example of a failed image segment due to unsuccessful processing caused by an offset failure is shown. In an offset failure, the image of a segment on the image generation chip is offset to one side. For example, in Figure 530, all segment images on the image generation chip are shifted to the left, thus the dark outer boundary of the left edge of the image generation chip is truncated from the image. Offset failures can be caused by scanning errors, such as scanner misalignment or incorrect placement of the image generation chip on the chip carrier.

[0097] Figure 5FAn example of a failed segment image due to surface abrasion failure is shown. Surface abrasion is caused by scratches on the surface of the segment in the image generation chip during the manufacturing process or sample preparation on the segment. As shown in Figure 535, scratches are displayed as lines on the segment image. Note that although the call rate values ​​of the three segments in the dashed box on the left are higher than the pass threshold, the image is marked as failed due to surface abrasion failure.

[0098] Figure 5G Illustration 540 shows failed segment images due to reagent flow failure. Ten segment images in box 542 are marked as failed images due to reagent flow failure. Segment images fail due to unsuccessful processes caused by improper reagent flow. During the genotyping process, reagents are introduced into the image generation chip from one side. The reagents flow from one end of the image generation chip to the opposite end and completely cover all segments. Sometimes, there are problems with reagent flow, and it cannot spread evenly to all segments. In this case, the reagent may dry out when a sufficient amount of reagent does not cover a segment. Inappropriate reagent flow can reduce the emission signal intensity from some segments, and since the fluorescent dye may not be evenly distributed across all segments, it affects image quality. The colors of failed images due to reagent flow failure may appear darker compared to segment images representing successful process cycles. Figure 5H Figure 545 shows another example of a failed segment image due to reagent flow failure. (See Figure 545 for further examples.) Figure 5G and Figure 5H As shown, reagent flow failure can affect multiple adjacent segments in the region of the image generation chip.

[0099] Figure 5I Examples of failed images due to unknown causes are presented. Failed segment images are labeled "unhealthy". Failed images in the unhealthy failure category can be caused by mixed or unidentified causes and weak signals. Illustration 550 of the segment images also shows an example of a failed interval region in the upper left segment of the image generation chip. The image segment in the upper left position (row 1 and column 2) is labeled as an interval region failure. It can be seen that the top portion of the failed segment image is dark. The dark area at the top increases from the right corner of the segment image to the left corner.

[0100] Feature generation based on principal component analysis

[0101] An example of an intrinsic image is now presented, which in the field of face recognition is called an intrinsic face. From tens of thousands of labeled images, a linear basis of 40 to 100 or more image components is identified. Figure 6A and Figure 6BExamples of 96 intrinsic images (610 and 611) obtained by applying principal component analysis (PCA) are presented. The 96 intrinsic images were selected based on the ranking of the components according to the measures of interpretable variability as described above. Figure 6C and Figure 6D Showing from Figure 6A and Figure 6B The 96 intrinsic images are arranged in the top 40 (620 and 621). In one specific implementation, 40 components were observed to explain most of the variability. The selected additional components appear to reflect noise patterns or natural variability in the sample processing.

[0102] The following describes the dimensionality reduction and creation of the basis of the intrinsic image using PCA. The first step is to reduce the resolution of the image segments and prepare the reduced image for input into PCA. Figure 7A (Labeled as 710) shows segment images with a resolution of 3600 × 1600 pixels, which have been rescaled by a factor of 20, resulting in reduced segment images of size 180 × 80 pixels. The rescaled segment images are then flattened. The resulting flattened rescaled segment images are a one-dimensional array, i.e., each segment image contains 14,400 × 1 pixels. As described above, the flattened rescaled segment images are normalized to obtain... Figure 7B The normalized, flattened, and rescaled segment image shown (labeled 740) is fed as input to PCA. PCA thus produces 14,400 principal component or intrinsic images. Each input image is a vector in a 14,400-dimensional space. The principal component or intrinsic images are then sorted using interpretable variance and a basis is created, for example, a basis of 40 to 100 components. These components form the basis of the linear space.

[0103] Feature generation based on image segmentation

[0104] The second image processing technique for generating features based on image segments includes thresholding image regions or segments. Figure 8A An example 810 of segment-based feature generation applied to a segmented image is shown. Figure 812 is a produced image of a segment from an image generation chip. An image conversion is applied to transform this grayscale produced image 812 to generate a corresponding binary image 814. In one specific implementation, some or all of the five steps presented by the above-described reference image converter 257 may be performed to convert the grayscale image into a binary image. Dark pixels in the binary image 814 indicate unhealthy or poor image pixels, while bright pixels indicate healthy or good image pixels.

[0105] Figure 816 on the right is an exemplary schematic diagram indicating the various regions of a segment and the segments surrounding the boundaries or lines of these regions. The regions from which fluorescence signal intensity is recorded are also called effective regions or regions of interest. For example, the cross-sectional schematic 816 indicates the effective regions of four grooves extending parallel to each other from top to bottom. Regions in the segment image that are not effective regions are filtered out from the image. For example, the boundary regions of the grooves are separated from each other by vertical lines indicating the boundaries or limits of the grooves. Similarly, boundaries on the four sides of the segment image can be filtered out. Segmentation techniques can divide the segment image into 4 to 20 or more segments or effective regions. This thresholding determines how many effective regions are generating the desired signal intensity.

[0106] The number of effective regions determines the number of features generated for each image. For example, if a segmented image is segmented into eight effective regions, the image intensities of the eight effective regions of the transformed image and the image intensities of the same eight effective regions of the original segmented image before transformation are provided as input to the classifier. Therefore, in this example, a total of 16 features for each segmented image are provided to the classifier. The average intensity of the signal intensity from the effective regions can be used as input to the classifier. For example, if the segmented image is segmented into eight effective regions, the average intensity of these eight effective regions is calculated for both the grayscale and binary images. These sixteen intensity values ​​are provided as input to the classifier to classify the segmented image as good or bad. Other segmentation schemes can be used that divide the image into fewer or more segments, such as 4, 12, 17, or more segments per image. If provided as input to a random forest classifier, a subset of features is randomly selected for each decision tree. The decision trees vote on whether the image is healthy or unhealthy. The majority vote in the random forest is used to classify the image. In one specific implementation, the number of trees in the random forest classifier ranges from 200 to 500, and the model depth ranges from 5 to 40. Failure modes between regions and segments of the image generation chip can be further evaluated for root cause classification.

[0107] Figure 8B Illustration 820 presents three pairs of segment images 822, 824, and 826. The left image in each pair is a pre-converted grayscale segment image, and the right image in each pair is an image obtained by applying a reference... Figure 2 The images described are binary images after processing following image conversion. The first image pair 822 is a production image of a successful genotyping process. The second image pair 824 is a production image of a failed process due to hybridization (or mixing) failure. The third image pair 826 is a failed image due to surface abrasion issues.

[0108] One-to-many (OvR) classification

[0109] Figure 9 A graphical illustration 900 depicts the operation of a one-to-many (OvR) classifier. The illustration shows an example of running an OvR classifier on a dataset consisting of samples belonging to three categories (square, circle, and triangle) as shown in Figure 931 on the left. The first hyperplane 916 shown in Figure 915 on the upper right illustrates the hyperplane determined as the ground truth for the square category. Hyperplane 916 separates data points in the square category from the remaining data points (circles and triangles). Similarly, Figures 936 and 955 separate data points in the circle and triangle categories from the other categories in the data via hyperplanes 937 and 956, respectively. The location of the hyperplanes is determined by the weight vectors. The training algorithm attempts to maximize the bounds between the hyperplanes and the ground truth categories for generalization; however, this can lead to incorrect classification of one or more data points. OvR classification is applied to distinguish segment images from process cycles belonging to the good category from images belonging to multiple bad categories.

[0110] Random Forest Classifier

[0111] The disclosed technique can apply various classifiers to distinguish between images from good or healthy images and poor or unhealthy images belonging to multiple failure categories. The classifiers applied include random forest, K-nearest neighbors, multinomial logistic regression, and support vector machines. A specific implementation of the disclosed technique is presented using a random forest classifier as an example.

[0112] Random forest classifiers (also known as random decision forests) are an ensemble machine learning technique. An ensemble technique or algorithm combines more than one of the same or different types of techniques for classifying objects. A random forest classifier consists of multiple decision trees operating as a whole. Each individual decision tree in a random forest acts as a base classifier and outputs a class prediction. The class with the most votes becomes the prediction of the random forest model. The basic concept behind random forests is that a large number of relatively unrelated models (decision trees) running as a committee will outperform any single constituent model.

[0113] The disclosed technique applies a random forest classifier in a two-stage classification process. A first-trained random forest classifier performs the task of separating successful production images from unsuccessful production images. A second-trained random forest classifier performs the task of root cause analysis of unsuccessful production images by predicting the failure category of unsuccessful images. This two-stage classification was chosen because successful production runs predominate, although a one-stage classification could also be used. Another reason for choosing the two-stage approach is that it allows control over a sensitivity threshold to classify images as healthy or successful production images versus unhealthy or failed production images. A threshold can be increased in the first-stage classification, causing the classifier to classify more production images as failed images. These failed images are then processed by the second-stage classifier for root cause analysis by identifying the failure category.

[0114] Training a random forest classifier

[0115] Figure 10A The training of two random forest classifiers, as shown in Figure 1000, is described. Figure 1 As shown, the training data includes input features of labeled process cycle images stored in training database 138. In an exemplary training of the classifier, 20,000 labeled production images of segments are used. The labeled images include good images from successful production cycles and failure images from unsuccessful production cycles. The size of training database 138 will increase as more labeled production images are received from the laboratory performing the genotyping process.

[0116] In one implementation, a random forest classifier was trained using 96 weights of the components of the labeled production images. The random forest classifier had 200 decision trees and a depth of 20. It should be understood that a random forest classifier with a range of 200 to 500 decision trees and a depth range of 10 to 40 is expected to provide good results for this implementation. Hyperparameters were tuned using random search cross-validation. The search depth ranged from 5 to 150, and the search number of trees ranged from 100 to 500. Increasing the number of trees can improve the performance of the model; however, it can also increase the training time. A training database 1001 containing features of 20,000 production cycle images was used to train a binary classifier 151 labeled as good and bad. The same training database was used to train a root cause classifier 171 to predict the failure category. Figure 10A As shown, the root cause classifier 171 is trained on a training database 1021 consisting only of poor or failed production images.

[0117] Decision trees are prone to overfitting. To overcome this problem, bagging is used to train decision trees in random forests. Bagging is a combination of bootstrapping and aggregation techniques. During bootstrapping, samples of rows are taken from the training database during training and used to train each decision tree in the random forest. For example, a subset of features from the selected rows can be used to train decision tree 1. Therefore, the training data used for decision tree 1 can be referred to as row sample 1 and column sample 1, or RS1+CS1. Columns or features can be randomly selected. Decision tree 2 and subsequent decision trees in the random forest are trained in a similar manner using subsets of the training data. It is important to note that the training data used for the decision trees is generated through substitution; that is, the same row data can be used to train multiple decision trees.

[0118] The second part of the bagging technique is the aggregation part applied during the production process. Each decision tree outputs a classification for each class. In the case of binary classification, it can be 1 or 0. The output of the random forest is an aggregation of the outputs of the decision trees in the random forest, where the majority vote is selected as the output of the random forest. By using votes from multiple decision trees, the random forest reduces the high variance in the decision tree results, thus producing good prediction results. By training individual decision trees using row and column sampling, each decision tree becomes an expert on the training record with selected features.

[0119] During training, the output of the random forest is compared to the baseline ground truth labels, and the prediction error is calculated. During backpropagation, the weights of the 96 components (or intrinsic images) are adjusted to reduce the prediction error. The number of components or intrinsic images depends on the number of components selected from the output of Principal Component Analysis (PCA) using a measure of interpretable variance. During binary classification, good and bad classifiers use image descriptive features from the training data and apply one-to-many (OvR) classification to the good category (or images labeled as healthy) and multiple bad categories (images labeled with one of the six failure categories). The parameters of the trained random forest classifier, such as the component weights, are stored for good and bad classification of images in the production cycle during inference.

[0120] The root cause classifier 171 is trained in a similar manner. The training database 1021 includes features from labeled process cycle images, which are derived from defective process cycles belonging to multiple failure categories. The image description features are used to train a random forest classifier 171 for one-to-many (OvR) classification of each failure category with the remainder of the labeled training examples.

[0121] Classification using a random forest classifier

[0122] The classification of produced images using trained classifiers 151 and 171 is now described. Figure 10B A two-stage classification 1080 of production images is illustrated, using a good vs. bad classifier 151 in the first stage and a root cause classifier 171 in the second stage. The process is presented using a sequence of process steps labeled 1 to 9. The process begins at step 1, accessing a trained random forest classifier labeled as good vs. bad classifier 151. Input features of the production images stored in database 1030 are provided as input to classifier 151. The classifier distinguishes good images belonging to successful process cycles from bad images belonging to failed process cycles. Bad images belong to multiple failure categories; for example, each image may belong to one of the six failure categories described above. The trained classifier accesses a base of intrinsic images used to analyze the production images. The trained classifier creates image descriptive features of the production images based on a linear combination of the intrinsic images. As described above, the weights of the intrinsic images are learned during the training of the classifier.

[0123] When applying one-to-many classification, all decision trees in the random forest classifier predict the output for each category, i.e., whether the image belongs to one of seven categories (one good category and six failure categories). Therefore, each decision tree in the random forest will output seven probability values, one for each category. The results from the decision trees are aggregated, and the majority vote is used to predict whether the image is good or bad. For example, if more than 50% of the decision trees in the random forest classify the image as good, the image is classified as a good image belonging to a successful production cycle. The sensitivity of the classifier can be adjusted, for example, by setting a higher threshold, which will result in more images being classified as bad. In processing step 2, the output of classifier 151 is examined. If the image is classified as a good image (step 3), the process ends (step 4). Otherwise, if the image is classified as a bad image indicating a failed process cycle (step 5), the root cause classifier 171 is invoked (step 6).

[0124] In the second stage of the two-stage processing, a root cause classifier is applied to determine the failure category of the poor image. This process continues in the second stage (step 8) by accessing the production image input features of the poor image (step 7) and feeding these input features to the trained root cause classifier 171. Each decision tree in the root cause classifier 171 votes on the input image features by applying a one-to-many classification. In this case, the classification determines whether the image belongs to one of the six failure categories or the remaining five failure categories. Each decision tree provides a classification for each category. The majority vote from the decision trees determines the failure category of the image (step 9).

[0125] Other classifiers can be used to classify good and bad segment images and perform root cause analysis. For example, the disclosed technique can apply the K-Nearest Neighbors (k-NN or KNN) algorithm to classify segment images. The k-NN algorithm assumes that similar examples (or segment images in our specific implementation) are very close. The k-NN algorithm captures the concept of similarity (also known as proximity or proximity) by calculating the distance between data points or images. Straight-line distance (or Euclidean distance) is commonly used for this purpose. In k-NN classification, the output is a category membership, such as a good image category or a bad image category. Images are classified by multiple votes from their neighboring images, where an object is assigned to the category most common among its k nearest neighbors. The value of k is a positive integer.

[0126] To select the correct value of k for the data, the k-NN algorithm is run multiple times with different k values, aiming to choose a k value that reduces the number of errors encountered while maintaining the algorithm's ability to accurately predict when given previously unseen data. Let's assume k is set to 1. This can lead to incorrect predictions. Consider a cluster of two data points: good images and bad images. If there is a query example surrounded by many good image data points, but close to a bad image data point also in the good image data point cluster, when k = 1, k-NN incorrectly predicts the query example as a bad image. As the value of k increases, the predictions of the k-NN algorithm become more stable due to majority voting (in classification) and averaging (in regression). Therefore, the algorithm is more likely to make more accurate predictions until a certain value of k is reached. As the value of k increases, an increase in the number of errors begins to be observed. Values ​​of k in the range of 6 to 50 are expected to work.

[0127] Examples of other classifiers that can be trained and applied using the disclosed techniques include multinomial logistic regression, support vector machines (SVM), gradient boosting trees, Naive Bayes, and others. The performance of the classifiers is evaluated using three criteria: training time, accuracy, and interpretability of results. The random forest classifier outperforms the other classifiers. Other classifiers are briefly introduced in the text below.

[0128] Support Vector Machine (SVM) classifiers perform as well as Random Forest classifiers. SVM classifiers locate a hyperplane between feature vectors of the good class and feature vectors of multiple bad classes. The disclosed techniques may include training a multinomial logistic regression. A multinomial regression model can be trained to predict the probabilities of different possible outcomes (multi-class classification). This model is used when the output is categorical. Therefore, a model can be trained to predict whether an image belongs to the good class or one of multiple bad classes. Logistic regression classifiers perform worse than Random Forest and SVM classifiers. The disclosed techniques may include training a gradient boosting model, which is an ensemble of predictive models such as decision trees. The model attempts to optimize a cost function in the function space by iteratively selecting a function pointing in the direction of the negative gradient. For example, a model can be trained to minimize the mean squared error on the training dataset. Gradient boosting models require more training time compared to other classifiers. The disclosed techniques may include training a Naive Bayes classifier, which assumes that the value of a particular feature is independent of the values ​​of any other feature. The Naive Bayes classifier assumes that each feature independently contributes to the probability of an example belonging to a class. A Naive Bayes classifier can be trained to classify images into good categories and multiple bad categories.

[0129] Specific implementation

[0130] The disclosed technique applies image classification to the evaluation and root cause analysis of the genotyping process. A classifier performs two tasks: separating successful and unsuccessful (or failed) production images, and then performing root cause analysis on the unsuccessful images. The disclosed technique can be applied to process cycle images from genotyping processing instruments. Although the disclosed technique is described as classifying images from a genotyping process, this classification can be applied to images from other types of processes that produce images of samples positioned on an image generation chip during or at the end of the process.

[0131] The classification of successful and unsuccessful production images is presented first. In a specific implementation of the disclosed technique, a method for training a random forest classifier to classify images from a genotyping process cycle is described. The method of training the classifier includes accessing labeled training examples from images belonging to a successful (or good or healthy) category and multiple failure (or bad or unhealthy) categories from the process cycle. The method may include accessing a basis of intrinsic images used to analyze the images. The method includes creating image descriptive features for each labeled training example based on a linear combination of the intrinsic images. The method includes training a random forest classifier using the image descriptive features to determine a one-to-many relationship between the successful classification and multiple failure categories for the labeled training examples. The method may include training other classifiers to determine a one-to-many relationship between the successful category and multiple failure categories for the labeled training examples. For example, the method may include training a support vector machine (SVM) classifier. The SVM classifier locates a hyperplane between the feature vector of the successful category and the feature vectors of the remaining categories (or, in our case, multiple failure categories). The method may include training a multinomial logistic regression. A multinomial regression model can be trained to predict the probabilities of different possible outcomes (multi-class classification). This model is used when the output is a classification. Therefore, models can be trained to predict whether an image belongs to the success category or one of multiple failure categories. This method can include training a gradient boosting model, which is an ensemble of predictive models such as decision trees. The model attempts to optimize a cost function in the function space by iteratively selecting a function pointing in the direction of the negative gradient. For example, the model can be trained to minimize the mean squared error on the training dataset. This method can also include training a Naive Bayes classifier that assumes the value of a particular feature is independent of the values ​​of any other feature. The Naive Bayes classifier assumes that each feature independently contributes to the probability of an example belonging to a category. A Naive Bayes classifier can be trained to classify images into a success category and multiple failure categories. The parameters of the trained classifier are stored for use in the success and failure classification of images in the production process cycle. Classifying production images using any of the above classifiers can detect failed production runs. The classifier can provide operators with rapid feedback on the quality of the genotyping process. Feedback from the classifier can be used to correct upstream processes, thus reducing processing time and the waste of valuable samples.

[0132] Methods for training a classifier for genotyping process cycle images may also include creating a basis for intrinsic images used to analyze the images. This method may include accessing the basis of the intrinsic images. The basis of the intrinsic images is ranked according to a measure of interpretable variability. The highest-ranked basis of the intrinsic images that cumulatively explains variability above a threshold is selected for analyzing the process cycle images. This method may include analyzing the process cycle images using the selected basis of the intrinsic images. The random forest classifier may include 100 to 400 decision trees. The depth of the random forest classifier may be between 10 and 40.

[0133] In practical production implementation, this method classifies production process cycle images into successes or failures. The method includes accessing a random forest classifier trained to distinguish images from process cycles belonging to the success category from images belonging to multiple failure categories. The method may include accessing a base of intrinsic images used to analyze the production images, and creating image descriptive features for the production images based on linear combinations of the intrinsic images. The method applies the trained random forest classifier to the image descriptive features for a one-to-many determination of success and multiple failure categories. The method includes providing notifications on the user interface to repeat the process cycle based on the failure determination.

[0134] Root cause analysis of unsuccessful (or failed) images is now presented. In a specific implementation of the disclosed technique, a method for training a root cause random forest classifier for process cycle images or genotyping process cycle images is described. The method of training the classifier includes accessing labeled training examples from images of failed process cycles belonging to multiple failure categories. The method also includes accessing a base of intrinsic images used to analyze the images and creating image descriptive features for each labeled training example based on a linear combination of the intrinsic images. The method includes training a random forest classifier using the image descriptive features for one-to-many determination of each failure category with the remainder of the labeled training examples. The parameters of the trained random forest classifier are stored for generating root cause classifications from process cycle images of failed process cycles.

[0135] Methods for training the root cause classifier may also include creating bases of intrinsic images used to analyze the images. The bases of intrinsic images are ranked according to a measure of interpretable variability. The highest-ranked base of the intrinsic images that cumulatively explains variability above a threshold is selected for analyzing the process cycle images. Random forest classifiers may include 100 to 400 decision trees. The depth of a random forest classifier can be between 10 and 40.

[0136] In the specific implementation of production, a method for root cause analysis based on images of failed process cycles includes accessing a base of intrinsic images used to analyze the failed production images, and creating image descriptive features of the failed production images based on a linear combination of the intrinsic images. This method includes accessing a random forest classifier trained to distinguish images from process cycles belonging to one of multiple failure categories. The method involves applying the random forest classifier to the image descriptive features for one-to-many determination among the multiple failure categories, including scoring each of the multiple failure categories against the remainder, and using the resulting scores to select the likely root cause of the poor process cycle among the multiple failure categories.

[0137] The disclosed techniques can utilize other feature generation and dimensionality reduction techniques to generate inputs for the classifier. Examples of such techniques include Nonnegative Matrix Factorization (NMF), Independent Component Analysis, Dictionary Learning, Sparse Principal Component Analysis, Factor Analysis, and Mini-Batch K-Means. Variations of image decomposition and dimensionality reduction techniques can be used. The disclosed techniques can also apply classifiers other than Random Forest classifiers to classify process periodic images. Applied classifiers may include Random Forest, K-Nearest Neighbors (KNN), Multinomial Logistic Regression, Support Vector Machine (SVM), Gradient Boosting Tree, Naive Bayes, etc. As larger subjects of labeled images become available, convolutional neural networks, such as ImageNet, can also be used.

[0138] In one implementation, a single method can implement a success / failure classifier and a root cause classifier in an end-to-end manner. Input images are provided as input to a first classifier, which separates successful images from failed images. The method includes accessing a second random forest classifier when it is determined that an image descriptive feature does not belong to the success category. The second random forest classifier is trained to distinguish images from process cycles belonging to one of multiple failure categories. The method includes applying the second random forest classifier to the image descriptive features for one-to-many determination among the multiple failure categories. The method also includes scoring each of the multiple failure categories against the remainder, and using the resulting scores to select the likely root cause of the undesirable process cycle among the multiple failure categories.

[0139] Another specific implementation of the disclosed technique uses thresholding of regions of a segmented image for classifying process cycle images. The method includes determining and applying a threshold based on pixel intensity in a grayscale production cycle image, using this threshold to classify image pixels into bright and dark categories, and converting the grayscale image into a binary image. The method may include generating bright and dark image pixel classifications. The method includes segmenting the grayscale image and the converted binary image into eight or more regions that separate expected system noise from the signal in the image. The method includes calculating the average intensity value of the regions in the grayscale image and the converted binary image. The method includes accessing a random forest classifier trained to classify images from the process cycle based on the average intensity value of the regions in the grayscale and binary images. The image is classified as belonging to a successful category or a failed category, and a confidence score is generated for the classification. The method may include comparing the confidence score generated by the trained random forest classifier with a threshold and classifying the image as successful or failed.

[0140] In one specific implementation, the method further includes applying a bilateral filter to preserve edges in the production cycle image before converting the production cycle image.

[0141] In one specific implementation, the method further includes applying a Gaussian blur filter to remove speckled noise from the binary image after the production cycle image has been transformed.

[0142] In one specific implementation, the method further includes applying morphological operations to fill holes in a binary image after converting the production cycle image, wherein the morphological operations include erosion and dilation.

[0143] The computer-implemented methods described above can be practiced in systems including computer hardware. The computer-implemented system can practice one or more of the methods described above. The computer-implemented system can incorporate any features immediately preceding or throughout the methods described herein, which are applicable to the methods implemented by that system. For the sake of brevity, no alternative combinations of system features are listed separately. The features applicable to the system, method, and article of manufacture do not repeat the basic features for each group of statutory classifications. The reader will understand how easily the features identified in this section can be combined with basic features in other statutory classifications.

[0144] As an article of manufacture rather than a method, a non-transitory computer-readable medium (CRM) may be loaded with processor-executable program instructions. When the program instructions are executed, one or more of the computer-implemented methods described above are implemented. Alternatively, the program instructions may be loaded on a non-transitory CRM and, when combined with suitable hardware, become part of one or more computer-implemented systems practicing the methods disclosed herein.

[0145] Each of the features discussed in this particular implementation section of the method also applies to the CRM and system implementations. As shown above, for the sake of brevity, all method features are not repeated here and should be considered as repeated by reference.

[0146] Terms and Conditions

[0147] 1. A method for training a random forest classifier for periodic images in a genotyping process, comprising:

[0148] Access labeled training examples from process cycles belonging to the success category and multiple failure categories;

[0149] Image descriptive features for training examples of each label are created based on a linear combination of intrinsic images;

[0150] The random forest classifier is trained using the image description features of the labeled training examples; and

[0151] Store the parameters of the trained random forest classifier.

[0152] 2. The method described in Clause 1 further includes:

[0153] Access the basis of the intrinsic image;

[0154] The basis of the intrinsic images is ordered according to a measure of the explained variability; and

[0155] Select the highest-ranked basis of the intrinsic images that cumulatively explain variability above a threshold; and

[0156] The process cycle image is analyzed using the basis of the selected intrinsic image.

[0157] 3. The method according to Clause 1 further includes using the image description features to train the random forest classifier for one-to-many determination of the successful category and the plurality of failed categories.

[0158] 4. The method according to Clause 1, wherein the random forest classifier comprises 100 to 400 decision trees.

[0159] 5. The method according to Clause 1, wherein the random forest classifier has a depth of 10 to 40.

[0160] 6. A method for classifying periodic images of the genotyping process, comprising:

[0161] Access a random forest classifier, which is trained to distinguish images from process cycles belonging to the success category from images belonging to multiple failure categories;

[0162] Image descriptive features for production images are created based on linear combinations of intrinsic images;

[0163] The random forest classifier is applied to the image description features; and

[0164] Based on the determination of the failed process cycle, notifications for recurring process cycles are provided on the user interface.

[0165] 7. The method according to Clause 6 further includes using the image description features to train the random forest classifier for one-to-many determination of the successful category and the plurality of failed categories.

[0166] 8. A method for training a root cause random forest classifier for periodic images of a genotyping process, comprising:

[0167] Access labeled training examples from images of failure process cycles belonging to multiple failure categories;

[0168] Image descriptive features for training examples of each label are created based on a linear combination of intrinsic images; and

[0169] The random forest classifier is trained using the image description features of the labeled training examples;

[0170] Store the parameters of the trained random forest classifier.

[0171] 9. The method according to Clause 8 further includes using the image description features to train the random forest classifier for one-to-many determination of the successful category and the plurality of failed categories.

[0172] 10. A method for root cause analysis based on a failure process cycle diagram, comprising:

[0173] Access the base of the intrinsic image used to analyze the failed production image; and

[0174] Image descriptive features of the failed production images are created based on a linear combination of the intrinsic images;

[0175] Access a random forest classifier, which has been trained to distinguish images from process cycles belonging to one of multiple failure categories; and

[0176] The random forest classifier is applied to the image description features for one-to-many determination of the plurality of failure categories, including scoring each of the plurality of failure categories against the remaining categories, and using the obtained scores to select the possible root cause of the failure process cycle from the plurality of failure categories.

[0177] 11. The method described in Clause 2 further includes:

[0178] When it is determined that the image descriptive features do not belong to the success category, a second random forest classifier is accessed, wherein the second random forest classifier is trained to distinguish images from process cycles belonging to one of the plurality of failure categories; and

[0179] The second random forest classifier is applied to the image description features, including scoring each of the plurality of failure categories against the remaining categories, and using the obtained scores to select the possible root cause of the failure cycle in the plurality of failure categories.

[0180] 12. A method for classifying periodic images of a genotyping process, comprising:

[0181] A threshold is determined and applied based on the pixel intensity in the grayscale production cycle image. The image pixels are then classified into bright and dark categories using the threshold, and the grayscale image is converted into a binary image.

[0182] The grayscale image and the converted binary image are divided into eight or more regions, which separate the expected system noise from the signal in the image;

[0183] Calculate the average intensity value of the region in the grayscale image and the converted binary image;

[0184] Access a random forest classifier, which is trained to classify the image from the process period into a success or failure category based on the average intensity value of the region in the grayscale image and the binary image, and compare the confidence score generated by the trained random forest classifier with a threshold to classify the image as success or failure.

[0185] 13. The method described under Clause 12 further includes:

[0186] A bilateral filter is applied before converting the production cycle image to preserve the edges in the production cycle image.

[0187] 14. The method described under Clause 12 further includes:

[0188] After converting the production cycle image, a Gaussian blur filter is applied to remove speckled noise from the binary image.

[0189] 15. The method described under Clause 12 further includes:

[0190] After the production cycle image is transformed, morphological operations are applied to fill holes in the binary image, wherein the morphological operations include erosion and dilation.

[0191] 16. A non-transitory computer-readable storage medium printed with computer program instructions for training a random forest classifier for periodic images of a genotyping process, the instructions, when executed on a processor, implementing a method comprising:

[0192] Access labeled training examples from process cycles belonging to the success category and multiple failure categories;

[0193] Image descriptive features for training examples of each label are created based on a linear combination of intrinsic images;

[0194] The random forest classifier is trained using the image description features of the labeled training examples; and

[0195] Store the parameters of the trained random forest classifier.

[0196] 17. Further implementation of the method according to the non-transitory computer-readable storage medium described in Clause 16 further includes:

[0197] The image description features are used to train the random forest classifier for one-to-many determination of the successful category and the plurality of failed categories.

[0198] 18. The method further comprises, according to the non-transitory computer-readable storage medium described in Clause 16, including:

[0199] Access the basis of the intrinsic image;

[0200] The basis of the intrinsic image is sorted according to the measure of the explained variability;

[0201] Select the highest-ranked basis of the intrinsic images that cumulatively explain variability above a threshold; and

[0202] The process cycle image is analyzed using the basis of the selected intrinsic image.

[0203] 19. The non-transitory computer-readable storage medium as described in Clause 16, wherein the random forest classifier comprises 100 to 400 decision trees.

[0204] 20. The non-transitory computer-readable storage medium as described in Clause 16, wherein the random forest classifier has a depth of 10 to 40.

[0205] 21. The method further comprises, according to Clause 18, a non-transitory computer-readable storage medium:

[0206] When it is determined that the image descriptive features do not belong to the success category, a second random forest classifier is accessed, wherein the second random forest classifier is trained to distinguish images from process cycles belonging to one of the plurality of failure categories; and

[0207] The second random forest classifier is applied to the image description features, including scoring each of the plurality of failure categories against the remaining categories, and using the obtained scores to select the possible root cause of the failure cycle in the plurality of failure categories.

[0208] 22. A non-transitory computer-readable storage medium printed with computer program instructions for classifying periodic images of a genotyping process, the instructions, when executed on a processor, implementing a method comprising:

[0209] A threshold is determined and applied based on the pixel intensity in the grayscale production cycle image. The image pixels are then classified into bright and dark categories using the threshold, and the grayscale image is converted into a binary image.

[0210] The grayscale image and the converted binary image are divided into eight or more regions, which separate the expected system noise from the signal in the image;

[0211] Calculate the average intensity value of the region in the grayscale image and the converted binary image;

[0212] Access a random forest classifier, which is trained to classify the image from the process period into a success or failure category based on the average intensity value of the region in the grayscale image and the binary image, and compare the confidence score generated by the trained random forest classifier with a threshold to classify the image as success or failure.

[0213] 23. The method further comprises, according to Clause 22, a non-transitory computer-readable storage medium:

[0214] A bilateral filter is applied before converting the production cycle image to preserve the edges in the production cycle image.

[0215] 24. A system comprising one or more processors coupled to a memory loaded with computer instructions for training a random forest classifier for process periodic images, the computer instructions executing on the processor to implement the instructions as described in Clause 16.

[0216] 25. Further actions to be taken for the system described in Clause 24 include:

[0217] Access the basis of the intrinsic image;

[0218] The basis of the intrinsic images is ordered according to a measure of the explained variability; and

[0219] Select the highest-ranked basis of the intrinsic images that cumulatively explain variability above a threshold; and

[0220] The process cycle image is analyzed using the basis of the selected intrinsic image.

[0221] 26. The system according to Clause 24, wherein the random forest classifier comprises 100 to 400 decision trees.

[0222] 27. The system according to Clause 24, wherein the random forest classifier has a depth of 10 to 40.

[0223] 28. Further actions to be taken for the system described in Clause 25 include:

[0224] When it is determined that the image descriptive features do not belong to the success category, a second random forest classifier is accessed, wherein the second random forest classifier is trained to distinguish images from process cycles belonging to one of the plurality of failure categories; and

[0225] The second random forest classifier is applied to the image description features, including scoring each of the plurality of failure categories against the remaining categories, and using the obtained scores to select the possible root cause of the failure cycle in the plurality of failure categories.

[0226] 29. A system comprising one or more processors coupled to a memory loaded with computer instructions for classifying process cycle images, the computer instructions executing on the processor to implement the instructions as described in Clause 22.

[0227] Computer System

[0228] Figure 11 This is a simplified block diagram of a computer system 1100 that can be used to implement the techniques disclosed in this invention. The computer system typically includes at least one processor 1172 that communicates with a plurality of peripheral devices via a bus subsystem 1155. These peripheral devices may include a storage subsystem 1110, which includes, for example, a memory subsystem 1122 and a file storage subsystem 1136, a user interface input device 1138, a user interface output device 1176, and a network interface subsystem 1174. The input and output devices allow users to interact with the computer system. The network interface subsystem provides an interface to external networks, including interfaces to corresponding interface devices in other computer systems.

[0229] In one implementation, the root cause classifier 171 for classifying failed (or bad) images is communicatively linked to the storage subsystem and the user interface input device.

[0230] User interface input device 1138 may include: a keyboard; pointing devices such as a mouse, trackball, touchpad, or graphics tablet; a scanner; a touchscreen integrated into a display; audio input devices such as a voice recognition system and a microphone; and other types of input devices. Generally, the term "input device" is intended to encompass all possible types of devices and methods for inputting information into a computer system.

[0231] User interface output device 1176 may include a display subsystem, printer, fax machine, or non-visual display such as an audio output device. The display subsystem may include a cathode ray tube (CRT), a flat panel device such as a liquid crystal display (LCD), a projection device, or some other mechanism for producing visible images. The display subsystem may also provide a non-visual display, such as an audio output device. Generally, the term "output device" is intended to encompass all possible types of devices and methods for outputting information from a computer system to a user or to another machine or computer system.

[0232] The storage subsystem 1110 provides programming and data construction for some or all of the functions and methods of the modules described herein. These software modules are typically executed by a processor alone or in combination with other processors.

[0233] The memory used in the storage subsystem may include multiple memories, including main random access memory (RAM) 1132 for storing instructions and data during program execution and read-only memory (ROM) 1134 for storing fixed instructions. The file storage subsystem 1136 may provide persistent storage for program files and data files and may include hard disk drives, floppy disk drives, and associated removable media, CD-ROM drives, optical disk drives, or removable media magnetic tape disks. Modules implementing certain specific functionalities may be stored by the file storage subsystem within the storage subsystem or on other machines accessible to the processor.

[0234] The bus subsystem 1155 provides a mechanism for enabling various components and subsystems of a computer system to communicate with each other as intended. Although the bus subsystem is schematically shown as a single bus, alternative implementations of the bus subsystem may use multiple buses.

[0235] Computer systems themselves can be of different types, including personal computers, portable computers, workstations, computer terminals, network computers, televisions, mainframes, server clusters, a loosely networked group of widely distributed computers, or any other data processing system or user equipment. Due to the constantly evolving nature of computers and networks, [the following text appears to be incomplete and requires further context: "for..."] Figure 11 The description of the computer system depicted is intended only as a specific example to illustrate the disclosed techniques. Many other configurations of the computer system are possible, which have more advanced features than those described above. Figure 11 The computer system depicted in the text has more or fewer components.

[0236] Computer system 1100 includes a GPU or FPGA 1178. It may also include machine learning processors hosted by machine learning cloud platforms such as Google Cloud Platform, Xilinx, and Cirrascale. Examples of deep learning processors include Google's Tensor Processing Unit (TPU), rack-mount solutions (such as the GX4 rack-mount series and GX8 rack-mount series), NVIDIA DGX-1, Microsoft's Stratix V FPGA, Graphcore's Intelligent Processing Unit (IPU), Qualcomm's Zeroth Platform with Snapdragon processors, NVIDIA's Volta, NVIDIA's DRIVE PX, NVIDIA's JETSON TX1 / TX2 MODULE, Intel's Nirvana, Movidius VPU, Fujitsu DPI, ARM's DynamicIQ, IBM TrueNorth, and others.

Claims

1. A method for training a random forest classifier for periodic images in a genotyping process, the method comprising: Access labeled training examples from process cycles belonging to a success category and multiple failure categories, each failure category corresponding to a different root cause of failure, where each different root cause of failure is an operational error or a chemical processing error; Image descriptive features for training examples of each label are created based on a linear combination of intrinsic images; The random forest classifier is trained using the image description features of the labeled training examples; and Store the parameters of the trained random forest classifier.

2. The method according to claim 1, further comprising: Access the basis of the intrinsic image; The basis of the intrinsic image is sorted according to the measure of the explained variability; as well as Select the highest-ranked basis of the intrinsic images that cumulatively explain variability above a threshold; and The process cycle image is analyzed using the basis of the selected intrinsic image.

3. The method of claim 1, further comprising using the image description features to train the random forest classifier for one-to-many determination of the successful category and the plurality of failed categories.

4. The method according to claim 1, wherein the random forest classifier comprises 100 to 400 decision trees, wherein, The operational processing error is one of a spacer region shift failure, an offset failure, or a surface wear failure, and the chemical processing error is one of a hybridization error or a reagent flow error.

5. The method of claim 1, wherein the random forest classifier has a depth of 10 to 40.

6. The method according to claim 2, further comprising: When it is determined that the image description features do not belong to the success category, a second random forest classifier is accessed, wherein the second random forest classifier is trained to distinguish images from process cycles belonging to one of the plurality of failure categories; as well as The second random forest classifier is applied to the image description features, including scoring each of the plurality of failure categories against the remaining categories, and using the obtained scores to select the possible root cause of the failure cycle in the plurality of failure categories.

7. A non-transitory computer-readable storage medium printed with computer program instructions for training a random forest classifier for periodic images of a genotyping process, the instructions causing the processor, when executed on a processor, to: Access labeled training examples from process cycles belonging to a success category and multiple failure categories, each failure category corresponding to a different root cause of failure, where each different root cause of failure is an operational error or a chemical processing error; Image descriptive features for training examples of each label are created based on a linear combination of intrinsic images; The random forest classifier is trained using the image description features of the labeled training examples; and Store the parameters of the trained random forest classifier.

8. The non-transitory computer-readable storage medium of claim 7, wherein the instructions are further configured to cause the processor to: The image description features are used to train the random forest classifier for one-to-many determination of the successful category and the plurality of failed categories.

9. The non-transitory computer-readable storage medium of claim 7, wherein the instructions are further configured to cause the processor to: Access the basis of the intrinsic image; The basis of the intrinsic image is sorted according to the measure of the explained variability; Select the highest-ranked basis of the intrinsic images that cumulatively explain variability above a threshold; and The process cycle image is analyzed using the basis of the selected intrinsic image.

10. The non-transitory computer-readable storage medium of claim 7, wherein the random forest classifier comprises 100 to 400 decision trees, wherein, The operational processing error is one of a spacer region shift failure, an offset failure, or a surface wear failure, and the chemical processing error is one of a hybridization error or a reagent flow error.

11. The non-transitory computer-readable storage medium of claim 7, wherein the random forest classifier has a depth of 10 to 40.

12. The non-transitory computer-readable storage medium of claim 9, wherein the instructions are further configured to cause the processor to: When it is determined that the image descriptive features do not belong to the success category, a second random forest classifier is accessed, wherein the second random forest classifier is trained to distinguish images from process cycles belonging to one of the plurality of failure categories; and Applying the second random forest classifier to the image description features includes the processor being further configured to score each of the plurality of failure categories against the remaining categories, and using the obtained scores to select the possible root cause of the failed process cycle from the plurality of failure categories.

13. A system for training a random forest classifier, the system comprising one or more processors coupled to a memory loaded with computer instructions for training a random forest classifier for periodic images, the computer instructions causing the one or more processors, when executed, to: Access labeled training examples from process cycles belonging to a success category and multiple failure categories, each failure category corresponding to a different root cause of failure, where each different root cause of failure is an operational error or a chemical processing error; Image descriptive features for training examples of each label are created based on a linear combination of intrinsic images; The random forest classifier is trained using the image description features of the labeled training examples to identify specific features corresponding to the plurality of failure categories; as well as Store the parameters of the trained random forest classifier.

14. The system of claim 13, wherein the instructions further cause the one or more processors to: Access the basis of the intrinsic image; The basis of the intrinsic images is ordered according to a measure of the explained variability; and Select the highest-ranked basis of the intrinsic images that cumulatively explain variability above a threshold; and The process cycle image is analyzed using the basis of the selected intrinsic image.

15. The system of claim 13, wherein the random forest classifier comprises 100 to 400 decision trees.

16. The system of claim 13, wherein the random forest classifier has a depth of 10 to 40.

17. The system of claim 13, wherein the instructions further cause the one or more processors to: When it is determined that the image descriptive features do not belong to the success category, a second random forest classifier is accessed, wherein the second random forest classifier is trained to distinguish images from process cycles belonging to one of the plurality of failure categories; and Applying the second random forest classifier to the image description features includes the processor being further configured to score each of the plurality of failure categories against the remaining categories, and using the obtained scores to select the possible root cause of the failed process cycle from the plurality of failure categories.

Images