Image-driven quality control for array-based PCR

Abstract

A system and method for image-driven quality control of array-based PCR is provided. The system includes a PCR unit, a reaction array plate, a convolutional neural network (CNN) configured to receive an image sequence of the reaction array plate in a PCR system, and an output of the CNN coupled to control of the reaction array plate. The method includes: applying image sequences from multiple subarrays of the reaction array plate to multiple CNNs during operation of the PCR system on the reaction plate array; operating the CNNs to generate a prediction of a failure mode for the reaction plate based on the image sequence; and coupling the output of the CNN to one or more of settings for manufacturing the reaction array plate or controlling the PCR system.

Description

Background Art

[0001] Deep learning is a large deep neural network trained with large amounts of data to learn complex and abstract patterns directly from unstructured raw data. Traditional machine learning algorithms rely heavily on handcrafted features, which are task-specific and designed by engineers with extensive domain expertise. In contrast, deep learning has the powerful ability to discover effective features directly from raw data and learn complex features by combining simpler features. Deep learning has demonstrated current state-of-the-art performance in a variety of fields such as image classification, computer vision, speech recognition, natural language processing (NLP), and medical image diagnosis. For example, Google and Microsoft have applied convolutional neural networks (CNNs) to the ImageNet image classification task and reported 4.9% and 4.6% error rates, which beat humans in image classification.

[0002] Currently, manufacturing quality control of assay reaction plates (e.g., those used to perform polymerase chain reaction (PCR)) performs manual visual inspection of reformatted and quality control (QC) images to identify problems and determine the root cause of failures in manufacturing assay reaction plates. However, visual inspection of images during the manufacturing process is time-consuming, error-prone, and tedious for humans. This also limits production capacity and often delays delivery of reaction plate array products to customers.

[0003] Some PCR systems are designed for maximum throughput, which can run up to four 3,072 reaction plates simultaneously in about four hours. Such systems and reaction plates typically utilize sample volumes of only nanoliters, requiring specialized instruments with complex workflows to load samples and assays onto the plate array. The complex workflows and challenging consumable usage can impact array reaction quality by reducing sensitivity and accuracy. Another side effect of this complexity is lower array manufacturing yields. Array manufacturing failures related to plate coating, reformatting, or sample loading are common. For example, array plates running PCR may have certain loading-related artifacts, including but not limited to:

[0004] 1. Bridging - where the fluid overflows around some adjacent reaction units and forms bridges between them, compromising the integrity of the reaction;

[0005] 2. Leakage - the sealing unit for the reaction plate may break, causing fluid to escape into the external cavity; and

[0006] 3. Assay spotting issues - For pre-spotted assay formats, some reaction units may not be loaded with the assay.

[0007] These sheet coating, reformatting, or sample loading issues can be detected by visual inspection of reformatting and quality control (QC) images. Therefore, it is important to automatically identify issues from QC images and quickly determine the root cause of failures in the manufacturing process. Because PCR reactions can show abnormal behavior, such as sample loading issues in images taken before, during, or after running PCR on OpenArray or other array-based PCR platforms, it is desirable to automatically detect these types of failure modes in customer-facing software and make the troubleshooting steps more efficient and easier for customers. Summary of the invention

[0008] The system utilizes deep neural networks such as convolutional neural networks (CNNs) to detect anomalies in array plate reformatting and QC images to identify failure modes. QC images are retrieved from a manufacturing data server, including both qualified and unqualified thin plates within a set time period. The data is classified based on qualified or different failure modes. Since each QC image may have more than one failure mode associated with one or more subarrays, the QC images are divided into (e.g., 48) subarray images, and each subarray is annotated as qualified or five failure modes, including: bright Rox, Rox spotting, Rox peeling, stop point bridging, and reformatting errors. The annotated subarray images are used to train, evaluate, and test the CNN model. The trained and verified model can then be used to identify faults and their modes in array plate reformatting and QC images.

[0009] Convolutional neural networks (CNNs) are particularly well suited for classifying features in datasets modeled in two or three dimensions. This makes CNNs popular for image classification, since images can be presented in computer memory in three dimensions (two dimensions representing width and height, and the third dimension representing pixel features such as color component and intensity). For example, a colored JEG image of size 480×480 pixels can be modeled in computer memory using an array of 480×480×3, where each value of the third dimension is the intensity of the red, green, or blue color component of the pixel ranging from 0 to 255. Inputting this array of numbers into a trained CNN will generate an output that describes the probability that the image belongs to a particular class (0.80 for a cat, 0.15 for a dog, 0.05 for a bird, etc.). Image classification is the task of taking an input image and outputting a class (cat, dog, etc.) or the probability of a class that best describes the image.

[0010] Fundamentally, a CNN takes in a dataset, passes it through a series of convolutions, non-linear activation functions (e.g., RELU), and pooling operations (downsampling, e.g., max pooling), and an output layer (e.g., softmax) to produce a classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] To facilitate identification of the discussion of any particular element or act, the most significant digit(s) in a reference number refers to the figure number in which the element is first introduced.

[0012] Figure 1 A qPCR system 100 is shown according to one embodiment.

[0013] Figure 2 Process 200 is depicted according to one embodiment.

[0014] Figure 3 Plate preparation 300 is shown according to one embodiment.

[0015] Figure 4 A QC image condition 400 is shown according to one embodiment.

[0016] Figure 5 A convolutional neural network 500 is depicted according to one embodiment.

[0017] Figure 6 A convolutional neural network layer 600 is depicted according to one embodiment.

[0018] Figure 7 A neural network 700 is depicted according to one embodiment.

[0019] Figure 8 A PCR quality control system 800 is shown according to one embodiment.

[0020] Fig. 9 A PCR quality control system 900 is shown according to one embodiment.

[0021] Fig.10 A verification result 1000 is shown according to one embodiment.

[0022] Fig.11 A verification result 1100 is shown according to one embodiment.

[0023] Fig.12 A PCR quality control system 1200 is shown according to one embodiment.

[0024] Fig.13 A recurrent neural network 1300 is depicted according to one embodiment.

[0025] Fig.14A PCR quality control system 1400 is shown according to one embodiment.

[0026] Fig.15 A PCR quality control system 1500 is shown according to one embodiment.

[0027] Fig.16 A cloud learning and control system 1600 is depicted according to one embodiment.

[0028] Fig.17 is a block diagram of an example computing device 1700 that may incorporate embodiments of the present invention. DETAILED DESCRIPTION

[0029] The loading problems discussed in the context of PCR assay arrays can be detected by evaluating images taken before, during, or after a PCR reaction run. A convolutional neural network (CNN) is trained and applied to recognize failure modes and to identify unknown (unclassified) failure modes based on training using data for PCR runs without anomalies.

[0030] Compared to traditional troubleshooting methods (which rely on manual identification of features that indicate failure modes), CNNs provide improved sensitivity and specificity, and can learn to identify previously unclassified problems. Single or multiple CNNs can be applied in series or in parallel. In some embodiments, a recurrent neural network (RNN) can be applied in conjunction with a CNN.

[0031] According to one embodiment, a deep neural network, more specifically a convolutional neural network (CNN), can be used to detect Anomalies in reformatting and QC images to identify failure modes. OpenArray QC images can be downloaded from the OpenArray manufacturing data server, containing both qualified and unqualified thin plates from previous instrument runs. The data can then be classified based on qualified or different failure modes. Since each QC image may contain more than one failure mode associated with one or more subarrays, the process divides the QC image into (for example) 48 subarray images and annotates each subarray as qualified or one or more of five failure modes, including: bright Rox, Rox spotting, Rox peeling, stop point bridging, and reformatting errors. The annotated subarray images are used to train, evaluate, and test the CNN model. The trained and verified model can then be used to identify faults and their modes in OpenArray reformatting and QC images.

[0032] In one embodiment, the process of operating a deep learning neural network to identify failure modes may involve:

[0033] 4. Download array quality control (QC) images of qualified and unqualified plates from the manufacturing data server.

[0034] 5. For each QC image, a point-finding algorithm was used to find the centers of all through-holes (reaction units) in the array, and the entire array image was divided into 48 sub-array images based on the identified through-hole centers.

[0035] 6. All 48 subarray images were loaded into the annotation tool and visually inspected and marked as either good or five failure modes including: Bright Rox, Rox Spotting, Rox Peeling, Stop Point Bridging, and Reformatting Errors.

[0036] 7. Repeat 2) and 3) for all QC images to build annotated training and validation datasets.

[0037] 8. Use a convolutional neural network with one or more hidden layers as the network to be trained.

[0038] 9. Use a fully connected layer as the output layer to output the predictions for the input subarray image.

[0039] 10. Apply the Softmax cross entropy cost function to calculate the loss between the predictions and the annotated tokens.

[0040] 11. At each training step, use the Adam optimizer or other gradient descent optimizer to update the weights of the above network to minimize the loss for a small number of training samples randomly selected from the training dataset.

[0041] 12. Continue the training process until a predetermined number of training steps are reached and save the trained network for each specific training step.

[0042] 13. Evaluate the trained model saved during the training process by validating an independent subset of subarray images in the dataset that were not included in the training process. The trained model with the smallest evaluation loss or error rate is selected as the best trained model.

[0043] 14. For a new QC image with the identified through-hole centers, divide the entire array image into some number (e.g., 48) of sub-array images. Apply the selected trained model on those sub-array images to predict pass or fail. If all sub-array images are good, the array QC passes, otherwise the array QC fails. The system can also predict which sub-array or sub-arrays failed and in which failure mode.

[0044] 15. Data augmentation techniques such as adding noise, rotation, translation offset, brightness or contrast changes, or sub-array images simulated by Generative Adversarial Net (GAN) can be used to improve the robustness of the trained model.

[0045] 16. During training, techniques such as dropout or weight decay can be used to improve the generalizability of the trained model.

[0046] Figure 1A qPCR system 100 is depicted, the system comprising a reaction plate 104, a sample loading instrument 102, a real-time PCR instrument 108, a sample mixture 106, a computer system 124, and a user interface 130. The reaction plate 104 comprises a plurality of subarrays, each of which comprises a plurality of through-holes used as reaction locations for a qPCR experiment. Each through-hole may be coated with an assay 110. In some configurations, the assay 110 is a probe specifically targeting a nucleotide sequence in a sample DNA. During amplification of the sample DNA, the probe indicates the presence of the target sequence by releasing a reporter dye detected by the real-time PCR instrument 108. The reaction plate 104 is combined with a target polynucleotide sequence 112 in the sample loading instrument 102. Prior to combining the reaction plate 104 with the target polynucleotide sequence 112, the target polynucleotide sequence 112 is prepared in a sample mixture 106 comprising a reaction mixture 118. The reaction mixture 118 comprises at least a polymerase 116, a primer 136, and an inert reference dye 114. The polymerase 116 amplifies double-stranded DNA during a PCR reaction. The sample loading instrument 102 loads a specific volume of sample mixture 106 into each expected through hole in the reaction plate 104. After the sample loading instrument 102 has completed preparing the reaction plate 104, the reaction plate 104 is loaded into the real-time PCR instrument 108. The real-time PCR instrument 108 includes a thermal cycler that circulates in different temperature ranges, which triggers specific stages of DNA replication. The first stage is a high temperature stage that denatures double-stranded DNA. The next stage anneals primers and polymerases to positions near the target polynucleotide sequence 112, and allows the assay 110 to hybridize with the target polynucleotide sequence 112. The third stage is the extension / replication stage in which the polymerase begins to replicate the DNA chain. During the replication stage, the polymerase 116 may encounter the hybridized assay 110 and the target polynucleotide sequence 112. When the polymerase 116 encounters the assay 110, the reporter dye of the assay 110 is cleaved, thereby generating a fluorescent signal observed in the through hole. The real-time PCR instrument 108 includes a detector 120, which detects the reporter dye from each through hole during the replication process. The detected signals are reported to a computer system 124, which includes a memory 126 and a processor 128, which stores and processes information to generate a cluster analysis graph 132 that displays the number of copies and instances of the target sequence in the sample mixture 106. The real-time PCR instrument 108 additionally includes a camera 122 that can capture an inert reference dye image 134. The inert reference dye image 134 can be used as a qualitative indication that the sample loading instrument 102 has successfully loaded the reaction plate 104, and can also provide insight into any abnormal results in the cluster analysis graph 132. Both the cluster analysis graph 132 and the inert reference dye image 134 can be displayed via the user interface 130.

[0047] As known to those of ordinary skill in the art, PCR analysis is performed on a thermal cycler instrument having various schemes for cycling through multiple thermal cycles to amplify a gene target. In various embodiments of the present teachings, the number of cycles performed for amplification may be between about 20 and 40 cycles. For various embodiments of the present teachings, the number of cycles performed for amplification may be greater than 40 cycles. In order to amplify a gene target, a thermal cycler instrument may perform the first thermal cycle of a PCR experiment within a certain cycle time, which cycle time may be associated with the first thermal cycle number.

[0048] In various embodiments of the genotyping analysis, two or more DNA samples are probed with a first probe and a second probe. The processor can receive a first data set at a first time from a qPCR instrument based on any of a plurality of schemes for data collection, the first data set comprising a first probe intensity and a second probe intensity for each of the two or more DNA samples at the first time. The processor can receive a second data set at a second time from a qPCR instrument based on any of a plurality of schemes for data collection, the second data set comprising a first probe intensity and a second probe intensity for each of the two or more DNA samples at the second time.

[0049] According to various embodiments of the present teachings, a user interface can present a visualization tool for analyzing the data set received at the first time and the second time to the terminal user. As previously mentioned, multiple samples can be processed to perform batch genotyping analysis, thereby generating a data set that is intensive in data. Various embodiments of the system and method according to the present teachings provide an embodiment of a visualization tool, which can help the terminal user evaluate and analyze such data-intensive data sets. For various embodiments of the system and method according to the present teachings, in response to input from the terminal user, the processor can use the first data set to generate a first curve graph of the first probe intensity and the second probe intensity. In addition, the processor can use the second data set to generate the second curve graph of the first probe intensity as a function of the second probe intensity in response to input from the terminal user. According to various embodiments of the system and method of the present teachings, the processor can display the first curve graph and the second curve graph in response to input from the terminal user. In various embodiments, the input can be an interactive process with a user interface to display data in a step-by-step manner. In such embodiments, the terminal user can select any data set in any order for display.

[0050] In various embodiments, the processor can receive data during the run time of the PCR experiment. For example, the processor can receive a first data set from the qPCR instrument after collecting the first data set and before collecting a second data set. In addition, this approach can be extended throughout the entire run time, such that, for example, the processor can receive a second data set from the qPCR instrument after collecting the second data set and before collecting a subsequent data set.

[0051] In some embodiments, after the thermal cycle is completed, the processor can receive the first data set and the second data set from the qPCR instrument. For example, the processor can receive the first data set and the second data set after the first data set and the second data set have been stored on a computer-readable medium.

[0052] In some configurations, visualization tools can help terminal users display various aspects of genotyping data sets, thereby promoting the analysis of genotyping data. In various embodiments, the processor can display the curve graph showing the trajectory line between the second data set and the first data set. In various embodiments, the processor can display on the first curve graph quality value of the first data set, and display on the second curve graph quality value of the second data set. According to various embodiments, the user interface provides the interaction between the selection made on the sample table and the selection dynamically displayed on the curve graph of genotyping data. In various embodiments, the selection made by the terminal user from the user interface of the visualization tool can be, for example, but not limited to, dynamic analysis is provided, so that the terminal user can, for example, but not limited to, troubleshoot ambiguous endpoint data, make manual calls, use trajectory lines to help visualize clustering to enhance genotyping distribution, optimize assay conditions (i.e., labeled probes, assay buffers, etc.) and optimize analysis conditions.

[0053] In various embodiments, the system utilizes a data set that can be represented by the diagram depicted in the cluster analysis curve graph 132, for example, but not limited to. Such representation can be generated from the analysis performed using two dyes with emission under different wavelengths, and the dye can be associated with each label probe in one of the two alleles of the locus in the guiding biological sample. In such duplex reactions, a discrete signal set is generated for each of three possible genotypes. In the Cartesian coordinate system of signal 2 and signal 1, as shown in the cluster analysis curve graph, each data point shown on such graphical representation can have a coordinate in one of the three given discrete signal sets. Therefore, for each data point, the discrete signal set for multiple samples can be stored in the data set as a data point. Such data sets can be stored in a variety of computer-readable media, and can be dynamically analyzed during or after analysis, as will be discussed in more detail later.

[0054] One such type of assay for demonstrating features of embodiments of methods and systems for visualizing genotyping data can utilize Reagents, and can be labeled with, for example, but not limited to, FAM and VIC dyes, as will be discussed later. However, one of ordinary skill in the art will recognize that a variety of assays comprising labeled probe reagents can be used to generate data that can be analyzed according to various embodiments of the methods and systems of the present teachings.

[0055] According to various embodiments, the term "label probe" generally refers to a molecule used in an amplification reaction, typically for quantitative or qPCR analysis and endpoint analysis. Such labeled probes can be used to monitor the amplification of a target polynucleotide. In some embodiments, the oligonucleotide labeled probe present in the amplification reaction is suitable for monitoring the amount of amplicon produced over time. Such oligonucleotide labeled probes include, but are not limited to, the 5'-exonuclease assay described herein. labeled probes (see also U.S. Pat. No. 5,538,848), various stem-loop molecular beacons (see, e.g., U.S. Pat. Nos. 6,103,476 and 5,925,517 and Tyagi and Kramer, 1996, Nature Biotechnology, 14:303-308), stemless or linear beacons (see, e.g., WO 99 / 21881), PNA Molecular Beacons™ (see, e.g., U.S. Pat. Nos. 6,355,421 and 6,593,091), linear PNA beacons (see, e.g., Kubista et al., 2001, SPIE 4264:53-58), non-FRET labeled probes (see, e.g., U.S. Pat. No. 6,150,097), Labeled probes (U.S. Pat. No. 6,548,250), stem-loop and duplex Scorpion™ labeled probes (Solinas et al., 2001, Nucleic Acids Research 29:E96 and U.S. Pat. No. 6,589,743), bulge-loop labeled probes (U.S. Pat. No. 6,590,091), pseudoknot labeled probes (U.S. Pat. No. 6,589,250), cyclicons (U.S. Pat. No. 6,383,752), MGB Eclipse™ probes (Epoch Biosciences), hairpin labeled probes (U.S. Pat. No. 6,596,490), peptide nucleic acids (peptide nucleic acids), The invention also provides a method for preparing a nucleic acid (PNA) luminescent labeled probe, a self-assembled nanoparticle labeled probe, and a ferrocene-modified labeled probe, such as those described in U.S. Pat. No. 6,485,901; Mhlanga et al., 2001, Methods 25:463-471; Whitcombe et al., 1999, Nature Biotechnology 17:804-807; Isacsson et al., 2000, Molecular Cell Labeling probes 14:321-328; Svanvik et al., 2000, Anal Biochem. 281:26-35; Wolffs et al., 2001, Biotechniques 766:769-771; Tsourkas et al., 2002, Nucleic Acids Res. Research, 30:4208-4215; Riccelli et al., 2002, Nucleic Acids Research, 30:4088-4093; Zhang et al., 2002, Shanghai, 34:329-332; Maxwell et al., 2002, J. Am. Chem. Soc., 124:9606-9612; Broude et al., 2002, Trends Biotechnol., 20:249-56; Huang et al., 2002, Chem Res. Toxicol., 15:118-126; and Yu et al., 2001, J. Am. Chem. Soc., 14:11155-11161.The label probe can also include a black hole quencher (Biosearch), Iowa Black (IDT), QSY quencher (Molecular Labeling Probes), and dimethylaminoazobenzoyl and Dabcel sulfonate / carboxylate quenchers (Times). The label probe can also include two label probes, where, for example, the fluorophore is on one probe and the quencher is on another probe, where the two label probes are hybridized together on the target to quench the signal, or where hybridization on the target changes the signal characteristics via fluorescence changes. The label probe can also include a sulfonate derivative of a fluorescein dye, which has a sulfonic acid group instead of a carboxylate group, a phosphoramidite form of fluorescein, a phosphoramidite form of CY 5 (e.g., available from Amersham).

[0056] As used herein, the term "nucleic acid sample" refers to nucleic acids found in biological samples according to the present teachings. It is expected that samples can be collected invasively or non-invasively. Samples can be found on, in, within, from, or in combination with the following: fibers, fabrics, cigarettes, chewing gum, adhesive materials, soil, or inanimate objects. As used herein, "sample" is used in its broadest sense and refers to a sample containing nucleic acids from which a gene target or target polynucleotide can be derived. Samples can include cells, chromosomes separated from cells (e.g., diffusion of metaphase chromosomes), genomic DNA, RNA, cDNA, etc. Samples can be of animal or plant origin, covering any organism containing nucleic acids, including but not limited to plants, livestock, domestic pets, and human samples, and can be derived from multiple sources. These sources may include, but are not limited to, whole blood, hair, blood, urine, tissue biopsy, lymph, bone, bone marrow, teeth, amniotic fluid, hair, skin, semen, anal secretions, vaginal secretions, sweat, saliva, oral swabs, various environmental samples (e.g., agricultural, water and soil), research samples, purified samples, and lysed cells. It should be understood that nucleic acid samples containing target polynucleotide sequences can be isolated from samples using any of a variety of sample preparation procedures known in the art, including, for example, using such procedures as mechanical force, sonication, restriction endonuclease cleavage, or any method known in the art.

[0057] As used herein, the terms "target polynucleotide", "gene target" and the like are used interchangeably herein and refer to a specific nucleic acid sequence of interest. "Target" can be a polynucleotide sequence that is attempted to be amplified and can exist in the presence of other nucleic acid molecules or be present in a larger nucleic acid molecule. The target polynucleotide can be obtained from any source and can include any number of different constituent components. For example, the target can be a nucleic acid (e.g., DNA or RNA). The target can be methylated, unmethylated or both. In addition, it will be appreciated that the "target" used in the context of a particular nucleic acid sequence of interest additionally refers to its substitute, such as an amplified product and a native sequence. In certain embodiments, the particular nucleic acid sequence of interest is a short DNA molecule derived from a degradation source, such as can be found in, for example, but not limited to, a forensic sample. As described above, the particular nucleic acid sequence of interest of the present teaching content can be derived from any of a variety of organisms and sources.

[0058] As used herein, "DNA" refers to deoxyribonucleic acid in its various forms as understood in the art, such as genomic DNA, cDNA, isolated nucleic acid molecules, vector DNA, and chromosomal DNA. "Nucleic acid" refers to DNA or RNA in any form. Examples of isolated nucleic acid molecules include, but are not limited to, recombinant DNA molecules contained in vectors, recombinant DNA molecules maintained in heterologous host cells, partially or substantially purified nucleic acid molecules, and synthetic DNA molecules. Typically, "isolated" nucleic acids do not contain sequences of nucleic acids (i.e., sequences located at the 5' and 3' ends of the nucleic acid) in the genomic DNA of the organism from which the nucleic acid is derived. In addition, "isolated" nucleic acid molecules (e.g., cDNA molecules) are generally substantially free of other cellular materials or culture medium when produced by recombinant techniques, or chemical precursors or other chemicals when chemically synthesized.

[0059] In some embodiments, the PCR amplification product can be detected by a fluorescent dye bound to the PCR amplification primer, for example, as described in PCT patent application WO 2009 / 059049. The PCR amplification product can also be detected by other techniques, including but not limited to staining of the amplification product, such as silver staining, etc.

[0060] In some embodiments, the detection includes an instrument, i.e., using an automatic or semi-automatic detection component, which may, but need not, include a computer algorithm. In some embodiments, the instrument is portable, transportable, or includes a portable component that can be inserted into a less mobile or less transportable component, such as a laboratory, hospital, or other environment where the amplification product detection is performed. In certain embodiments, the detection step is combined with or is a continuation of at least one amplification step, a sequencing step, an isolation step, a separation step, such as, but not limited to, a capillary electrophoresis instrument comprising at least one fluorescence scanner and at least one mapping, recording or reading component; a chromatographic column coupled to an absorbance monitor or a fluorescence scanner and a graph recorder; a chromatographic column coupled to a mass spectrometer comprising a recording and / or detection component; a spectrophotometer instrument comprising at least one UV / visible light scanner and at least one mapping, recording or reading component; a microarray having a data recording device such as a scanner or a CCD camera; or a sequencing instrument having a detection component selected from the following: a sequencing instrument comprising at least one fluorescence scanner and at least one mapping, recording or reading component, sequencing by a synthesis instrument comprising fluorescently labeled, reversible terminator nucleotides, a pyrophosphate sequencing method comprising detecting the release of pyrophosphate (PPi) after the incorporation of a nucleotide by a DNA polymerase, paired-end sequencing, polymerase clone sequencing, single-molecule sequencing, nanopore sequencing, and sequencing by hybridization or by ligation, as described by Lin, B. et al. in Recent Patents in Biomedical Engineering. Patents on Biomedical Engineering) (2008) 1(1) 60-67, which is incorporated herein by reference.

[0061] In certain embodiments, the detection step is combined with an amplification step, such as, but not limited to, real-time analysis, such as Q-PCR. An exemplary method for performing the detection step comprises Gene Analyzer Instrument Series, DNA analyzer instrument series, Sequence Detection System instrument series and Applied Biosystems real-time PCR instrument series (all from Applied Biosystems); and microarrays and related software, such as the Applied Biosystems microarray and the Applied Biosystems 1700 chemiluminescent microarray analyzer, and other commercially available microarray and analysis systems available from Affymetrix, Agilent, and Amersham Biosciences, etc. (see also Gerry et al., J. Mol. Biol. 292:251-62, 1999; DeBellis et al., Minerva Biotec 14:247-52, 2002; and Stears et al., Nat. Med. 9:140-45, including supplements, 2003) or bead array platforms (Illumina, San Diego, California). Exemplary software includes GeneMapperTM software, Analysis software and software (all from Applied Biosystems).

[0062] In some embodiments, the amplification product can be detected and quantified based on the mass-to-charge ratio (m / z) of at least a portion of the amplicon. For example, in some embodiments, the primer includes a mass spectrometry compatible reporter group, including but not limited to a mass tag, a charge tag, a cleavable portion, or an isotope that is incorporated into the amplification product and can be used for mass spectrometry detection (see, for example, Haff and Smirnov, "Nucleic Acids Research" 25:3749-50, 1997; and Sauer et al., "Nucleic Acids Research" 31:e63, 2003). The amplification product can be detected by mass spectrometry. In some embodiments, the primer includes a restriction enzyme site, a cleavable portion, etc., to facilitate the release of a portion of the amplification product for detection. In certain embodiments, a plurality of amplification products are separated by liquid chromatography or capillary electrophoresis, subjected to ESI or MALDI, and detected by mass spectrometry. A description of mass spectrometry can be found in "The Expanding Role of Mass Spectrometry in Biotechnology" by Gary Siuzdak, MCC Press, 2003, and elsewhere.

[0063] In some embodiments, the detection comprises a manual or visual readout or assessment, or a combination thereof. In some embodiments, the detection comprises an automatic or semi-automatic digital or analog readout. In some embodiments, the detection comprises a real-time or endpoint analysis. In some embodiments, the detection comprises a microfluidic device, including but not limited to Low Density Array (Applied Biosystems). In some embodiments, the detection comprises a real-time detection instrument. Exemplary real-time instruments include ABI 7000 Sequence Detection System, ABI 7700 Sequence Detection System, Applied Biosystems 7300 Real-Time PCR System, Applied Biosystems 7500 Real-Time PCR System, Applied Biosystems 7900HT Fast Real-Time PCR System (all from Applied Biosystems); LightCycler™ System (Roche Molecular); Mx3000P™ Real-Time PCR System, Mx3005P™ Real-Time PCR System, and Multiplex Quantitative PCR System (Stratagene, La Jolla, CA); and Smart Cycling System (Cepheid, distributed by Fisher Scientific). Descriptions of real-time instruments can be found in, among other places, their respective manufacturers' user manuals; McPherson; "DNA Amplification: Current Technologies and Applications", edited by Demidov and Broude, Horizon Bioscience, 2004; and U.S. Pat. No. 6,814,934.

[0064] The term "amplification reaction mixture" and / or "master mixture" may refer to an aqueous solution including various (some or all) reactants for amplifying target nucleic acids. Such reactions may also be performed using solid supports or semisolid supports (e.g., arrays). The reaction may also be performed in a single or multiplexed form according to the user's expectations. These reactions typically include enzymes, aqueous buffers, salts, amplification primers, target nucleic acids, and nucleoside triphosphates. In certain embodiments, the amplification reaction mixture and / or master mixture may include one or more of the following: such as buffers (e.g., Tris), one or more salts (e.g., MgC, KCl), glycerol, dNTPs (dA, dT, dG, dC, dU), recombinant BSA (bovine serum albumin), dyes (e.g., ROX inert reference dyes), one or more detergents, polyethylene glycol (PEG), polyvinyl pyrrolidone (PVP) and / or gelatin (e.g., fish or cattle) and / or defoamers. Depending on the context, the mixture may be a complete or incomplete amplification reaction mixture. In certain embodiments, the master mixture does not include amplification primers before being used for the amplification reaction. In some embodiments, the master mix does not contain a target nucleic acid prior to use in an amplification reaction. In some embodiments, the amplification master mix is ​​mixed with a target nucleic acid sample prior to contacting with an amplification primer.

[0065] In some embodiments, the amplification reaction mixture comprises amplification primers and a master mix. In some embodiments, the amplification reaction mixture comprises amplification primers, a detectably labeled probe, and a master mix.

[0066] In certain embodiments, the reaction mixture of amplification primer and master mixture or amplification primer, probe and master mixture is dried in a storage container or reaction vessel. In certain embodiments, the reaction mixture of amplification primer and master mixture or amplification primer, probe and master mixture is lyophilized in a storage container or reaction vessel. In certain embodiments, the disclosure generally relates to amplification of multiple target-specific sequences from a single control nucleic acid molecule. For example, in certain embodiments, a single control nucleic acid molecule can include RNA, and in other embodiments, a single control nucleic acid molecule can include DNA. In certain embodiments, target-specific primers and primer pairs are target-specific sequences that can amplify a specific region of a nucleic acid molecule (for example, a control nucleic acid molecule). In certain embodiments, target-specific primers can trigger reverse transcription of RNA to generate target-specific cDNA. In certain embodiments, target-specific primers can amplify target DNA or cDNA. In certain embodiments, the amount of DNA required for selective amplification can be about 1ng to 1 microgram. In certain embodiments, the amount of DNA required for selective amplification of one or more target sequences can be about 1ng, about 5ng or about 10ng. In certain embodiments, the amount of DNA required for selective amplification of a target sequence is about 10ng to about 200ng.

[0067] As used herein, the term "reaction vessel" generally refers to any vessel, cavity, device, or assembly in which a reaction can occur according to the present teachings. In some embodiments, the reaction vessel can be a microtube, such as, but not limited to, a 0.2 mL or 0.5 mL reaction tube, such as a Micro Amp™ optical tube (Life Technologies Corp., Carlsbad, California) or a microcentrifuge tube, or other vessels of the type customary in molecular biology laboratories. In some embodiments, the reaction vessel comprises a well of a multiwell plate (e.g., a 48-well, 96-well, or 384-well microtiter plate), a spot on a slide, a well in a TaqMan™ array card, or a channel or cavity of a microfluidic device, including, but not limited to, a through-hole of a TaqMan™ low-density array or a TaqMan™ OpenArray™ real-time PCR plate (Applied Biosystems, ThermoFisher Scientific). For example, but not limited to, multiple reaction vessels can be present on the same support. For example, an OpenArray TM The plate is a reaction plate 3072 through-hole. Each such through-hole in such a plate can contain a single TaqMan TM Assay. In some embodiments, a lab-on-a-chip device available from, for example, Caliper or Fluidigm can provide a reaction vessel. It will be appreciated that a variety of reaction vessels are commercially available or can be designed for use in the context of the present teachings.

[0068] The terms "annealing" and "hybridizing" (including but not limited to variations of the roots "hybridize" and "anneal") are used interchangeably and refer to nucleotide base pairing interactions of one nucleic acid with another nucleic acid that result in the formation of a duplex, triplex, or other higher order structure. Primary interactions are typically nucleotide base specific, such as A:T, A:U, and G:C, through Watson-Crick and Hoogsteen type hydrogen bonds. In certain embodiments, base stacking and hydrophobic interactions may also contribute to duplex stability. Conditions for annealing primers and probes to complementary sequences are well known in the art, for example, as described in Nucleic Acid Hybridization, A Practical Approach, ed., Hames and Higgins, IRL Press, Washington, D.C. (1985); and Wetmur and Davidson, Mol. Biol. 31:349 (1968).

[0069] In general, whether the annealing is performed is affected by, among other things, the complementary portion of the complementary portion of the primer and the length of its corresponding binding site in the target side-joining sequence and / or amplicon, or the length of the corresponding complementary portion of the reporter probe and its binding site; pH; temperature; the presence of monovalent and divalent cations; the ratio of G and C nucleotides in the hybridization region; the viscosity of the culture medium; and the presence of denaturants. Such variables affect the time required for hybridization. Therefore, the preferred annealing conditions will depend on the specific application. However, such conditions can be routinely determined by a person of ordinary skill in the art without excessive experimentation. Preferably, the annealing conditions are selected to allow the primer and / or probe to selectively hybridize with the corresponding target side-joining sequence or the complementary sequence in the amplicon, but not to hybridize to any significant extent with different target nucleic acids or non-target sequences in the reaction composition at the second reaction temperature.

[0070] Figure 2 A process 200 for PCR amplification, and more specifically, a 5′ nuclease assay for PCR amplification, is depicted according to one embodiment. The process 200 shows four stages of this assay process, which occur in each cycle and do not interfere with the exponential accumulation of the product. The four stages include a polymerization stage 202, a strand displacement stage 204, a cleavage stage 206, and a completion stage 208. During the polymerization stage 202, a forward primer and a reverse primer begin to replicate a segment of double-stranded DNA 214 near a target sequence 210. The forward primer (5′ to>3′) includes a hot start polymerase 224 (Taq polymerase), which works at a temperature at which DNA polymerases are inactive to avoid unnecessary replication. The probe 212 includes a reporter dye 218, a complementary sequence 226, a non-fluorescent quencher 220, and a minor groove binder 222. The probe 212 hybridizes with the target sequence 210 through the complementary sequence 226. The non-fluorescent quencher 220 and the minor groove binder 222 act as molecules attached to the 3′ end of the probe 212. When the probe is intact, the non-fluorescent quencher 220 (NFQ) prevents the reporter dye 218 from emitting a fluorescent signal. Because the non-fluorescent quencher 220 does not fluoresce, it produces a lower background signal, thereby improving quantitative accuracy. The minor groove binder 222 (MGB) increases the melting temperature (Tm) of the probe without increasing the length of the probe, allowing shorter probes to be designed. During the polymerization stage 202, the hot start polymerase 224 moves toward the probe 212, where the reporter dye 218 is attached to the 5′ side of the probe 212.

[0071] In the strand displacement phase 204, the hot start polymerase 224 interacts with the hybridized probe 212 that displaces the reporter dye 218. In the cleavage phase 206, the hot start polymerase 224 cleaves the reporter dye 218 from the probe 212. The cleavage separates the reporter dye from the quencher dye; wherein the non-fluorescent quencher 220 no longer blocks the reporter dye 218, and the separated reporter dye 216 increases its fluorescence. The fluorescence increases only when the target sequence is complementary to the probe and is amplified during PCR. The instrument detects the fluorescence emitted by the reporter dye, which indicates the presence of the target sequence on the double-stranded DNA 214. Since the probe 212 hybridizes with the target sequence 210, the hot start polymerase 224 terminates at the complementary sequence 226, indicating the completion of phase 208.

[0072] Figure 3 The plate preparation 300 of the reaction plate 104 before loading the reaction plate into the rtPCR instrument is depicted. The reaction plate 104 includes a plurality of subarrays, each subarray 308 including a plurality of through-array through-holes 304. Each through-hole can be used as a reaction site for an assay 110. In some configurations, the reaction plate 104 includes 48 subarrays, each of which includes 64 through-holes, each of which can accommodate a 33-nL reaction volume. In the aforementioned configuration, the reaction plate 104 includes 3072 through-holes.

[0073] Depending on the configuration of the reaction plate 104, some of the array through-holes 304 will contain assays 110 spotted therein. Each through-hole includes a hydrophilic interior into which the assay 110 may be spotted. The hydrophilic through-holes are also surrounded by a hydrophobic surface, thereby allowing the reaction to be controlled.

[0074] In order to accurately load a set volume into each desired array through-hole 304, a sample loading instrument 102 is used. The sample loading instrument 102 aliquots the set volume of the sample mixture 106 into each desired through-hole in the reaction plate 104. In some configurations, the sample loading instrument 102 utilizes a tip block 310 to dispense the sample mixture into the through-holes of the reaction plate 104.

[0075] When operating the sample loading instrument 102, the tip block 310 can be moved across the reaction plate 104 to allow a set volume of the sample mixture 106 to be delivered to a specific array through-hole 304. When the sample loading instrument 102 completes its operation, the reaction plate 104 is converted to a loaded reaction plate 306, wherein a plurality of subarrays (e.g., subarray 312) include loaded through-holes 302, which include target polynucleotide sequences 112.

[0076] refer to Figure 4, QC image condition 400 depicts an inert reference dye image that demonstrates characteristic defects that lead to poor data collection. When the spotted sample plate is loaded into the rtPCR instrument, an image of the spotted plate is taken under a fluorescent lamp. The fluorescent lamp causes the inert reference dye to fluoresce, indicating the presence of a sample mixture (i.e., target polynucleotide and reaction mixture). The inert reference dye image is collected and stored by a computer system associated with the instrument. The inert reference dye image can be analyzed early in the quality control process to determine if there are any errors associated with spotting the sample onto the array plate.

[0077] Image 402 depicts a qualified image showing brightly illuminated dots arranged in a grid pattern. Each dot represents a via, and the qualified image shows all vias with similar brightness against a uniform background.

[0078] Image 404, image 406, image 408, image 410 and image 412 represent five common fault conditions associated with the spotting of array plates. Image 404 represents one of the conditions in which a fault occurs, which is referred to as bright ROX. Bright ROX indicates that there are visible bright spots 416 in some through holes compared to other through holes. Bright spots 416 may indicate that there is an inert reference dye to a higher degree in the brighter through holes, which may indicate improper spotting of the plate. Image 406 represents ROX spotting, in which random through holes appear darker than other through holes and are distributed on the entire array plate. In some cases, darker through holes may indicate partially loaded through holes. Image 408 represents ROX stripping, in which some of the darker observation through holes are arranged on the array plate in an almost linear pattern. In some cases, ROX stripping may indicate that the plate is not tightly against the block on the right side of the plate holder, the automatic loader is misaligned, or there are too many bubbles and / or foams at the tip of the automatic loader. In image 404, image 406, and image 408, if there are at least seven affected vias, the image is considered rejected, but since some rejection identification is subjective, there is no absolute limit as the determination is based on the relative brightness of the vias.

[0079] Image 410 represents a fault condition known as a dot-to-dot bridge. Dot-to-dot bridge 414 is shown as a gray "bridge" in the background area connecting several (two or four) adjacent through-holes. Dot-to-dot bridge 414 may be caused by improper removal of the autoloader tip block, which causes some sample mixture to overflow from the through-hole. Image 412 represents a reformatting error and is characterized by at least one random empty through-hole. A reformatting error may indicate a problem with the array plate or a problem with the array configuration in the autoloader.

[0080] Figure 5An exemplary convolutional neural network 500 is depicted. The convolutional neural network 500 arranges its neurons in three dimensions (width, height, depth), as shown in the convolutional layer 504. Each layer of the convolutional neural network 500 converts a 3D input volume into a 3D output volume of neuron activations. In this example, the input layer 502 encodes an image, so its width and height will be the dimensions of the image, and the depth will be 3 (red, green, blue channels). The convolutional layer 504 further transforms the output of the input layer 502, and the output layer 506 transforms the output of the convolutional layer 504 into one or more classifications of the image content. In one embodiment, the convolutional neural network 500 can utilize one input layer 502, six hidden layers (one or more of which can be convolutional layers 504), and one output layer 506. In addition to one or more convolutional layers 504, the convolutional neural network 500 can utilize pooling layers and fully connected layers.

[0081] Figure 6 An exemplary convolutional neural network layer 600 is depicted in more detail. An example sub-region of an input layer region 604 of an input layer region 602 of an image is analyzed by a set of convolutional layer sub-regions 608 in a convolutional layer 606. The length and width of the input layer region 602 are 32×32 neurons (e.g., 32×32 pixels), while the depth is three neurons (e.g., three color channels per pixel). Each neuron in the convolutional layer 606 is only spatially connected (in height and width) to a local region in the input layer region 602, but is connected to the entire depth (i.e., if the input is an image, to all color channels). Note that there are multiple neurons (5 in this example) along the depth of the convolutional layer sub-region 608 that analyze a sub-region of the input layer region 604 of the input layer region 602, where each neuron of the convolutional layer sub-region 608 can receive input from each neuron of the sub-region of the input layer region 604.

[0082] refer to Figure 7 , the neural network 700 includes an input layer 702, a first convolutional layer 704, a first pooling layer 706, a second convolutional layer 708, a second pooling layer 710, a first fully connected layer 712 and a second fully connected layer 714.

[0083] The input layer 702 may be a 54×54 array. The 54×54 array may be a subarray of an array board. For example, the array board may be arranged into a 4×12 array of subarrays. Thus, each array board may be divided into 48 subarrays, each of which is sent to the neural network 700 to determine whether it is qualified or unqualified. Each value in the array may represent a color channel, such as an R, G, B color channel or a black / white color scale. The input layer 702 is then convolved into the first convolutional layer 704. The first convolutional layer 704 includes 32 arrays, each of which is 54×54. The first convolutional layer 704 is pooled into the first pooling layer 706. The pooling function may be a maximum pooling function operating on a 3×3 region of the first convolutional layer 704. The first pooling layer 706 may then include 32 arrays, each of which is 18×18. The first pooling layer 706 is then convolved into the second convolutional layer 708. The second convolutional layer 708 may have 64 arrays, each of which is 18×18. The second convolutional layer 708 is pooled into the second pooling layer 710. The pooling function may be a maximum pooling function operating on a 3×3 region of the second convolutional layer 708. The second pooling layer 710 may then include 64 arrays, each of which is 6×6. Each of the 64 6×6 arrays is sent to the first fully connected layer 712, which may have 192 nodes. The first fully connected layer 712 is then fully connected to the second fully connected layer 714, which may have 48 nodes. The output layer 716 then has two states, either qualified or unqualified, and is fully connected to the second fully connected layer 714. The output layer 716 may also predict the type (category) of unqualified.

[0084] The neural network 700 can be trained and operated by first retrieving historical QC images of qualified and unqualified boards from the manufacturing data server. For each image, the center of the through-hole (reaction unit) in the array is found by a point-finding algorithm, and the entire array image is divided into 48 sub-array images based on the identified through-hole centers. The 48 sub-array images are loaded into the annotation tool, and they are inspected and marked as qualified or unqualified, and the unqualified has five failure modes, including: bright Rox, Rox spotting, Rox peeling, stop point bridging, and reformatting errors. Repeat the first two steps for the QC images to build annotated training and validation data sets. Initialize a convolutional neural network with one or more hidden layers to be used as the network to be trained. A fully connected layer is used as the output layer to output the prediction of the input sub-array image. A Softmax cross entropy cost function is applied to calculate the loss between the prediction and the annotated labeling. In each training step, the weights of the above network are updated using an Adam optimizer or other gradient descent optimizer to minimize the loss for a small number of training samples randomly selected from the training data set. The training process is continued until a predetermined number of training steps is reached, and the trained network is saved for each specific training step. The trained models saved during the training process are evaluated by an independent subset of subarray images in the validation dataset, which are not included in the training process. The trained model with the smallest evaluation loss or error rate is then selected as the best trained model. During training, techniques such as dropout or weight decay can be used to improve the generalizability of the trained model.

[0085] During operation, a new QC image has the through-hole centers identified and the entire array image is divided into 48 sub-array images. The selected trained model is applied to those sub-array images to predict pass or fail. If all sub-array images are good, the array QC may be pass, otherwise the array QC fails. The neural network 700 can also predict which sub-array fails and in which failure mode.

[0086] Further data augmentation techniques such as adding noise, rotation, translation offset, brightness or contrast changes, or sub-array images simulated by generative adversarial networks (GANs) can be used to improve the robustness of the trained model.

[0087] Transfer learning is a technique that makes full use of multiple data sources to overcome data scarcity problems. By using transfer learning, a network that has been trained for a similar problem is modified and then retrained using a portion of labeled data. Here, the neural network 700 can use a model learned from other image datasets (such as ImageNet data) and replace the final classification layer with a new fully connected layer to predict pass and fail modes for array-based PCR data.

[0088] refer to Figure 8 , PCR quality control system 800 includes PCR system 802, CNN 804 and computer system 806. PCR system 802 generates QC images and sends the images to CNN 804. CNN 804 can be an embodiment of neural network 700 described above. CNN 804 determines whether the QC image is qualified or unqualified. The result is sent to computer system 806. The result may include whether the QC image is qualified or unqualified, which subarray is unqualified and which failure mode it is. Computer system 806 can display the results to the user, store the results, perform additional transformations of the results, and control PCR system 802 or set process parameters to improve the manufacture of reaction plate arrays. Computer system 806 can also change PCR system 802 based on the results, such as stopping the operation of PCR system 802 that provides unqualified results, or correcting PCR system 802 when unqualified situations occur. PCR system 802 can generate a series of QC images, each QC image is at a different time. These QC images can be sent to CNN 804 in series. For each image, CNN 804 provides results to computer system 806 .

[0089] refer to Fig. 9 , PCR quality control system 900 includes PCR system 802, computer system 806, CNN 902 and CNN 904. PCR quality control system 900 can be operated similarly to PCR quality control system 800. However, PCR quality control system 900 can initialize multiple CNNs (CNN 902, CNN 904, etc.). Each CNN is then used for one of the subarrays. Some QC images include 48 subarrays, and 48 CNNs can be operated. Computer system 806 can then combine the results from each CNN. The results can include whether a subarray has a fault condition, which subarrays have a fault condition, and which fault condition each subarray is in.

[0090] refer to Fig.10, depicting the validation results 1000. The rows represent the annotated classifications, and the columns represent the predictions from the selected network. The upper left box is when the annotated markers are qualified and the predicted classifications are qualified. For a given test, there are 1166 such results. The upper right box is when the annotated markers are qualified and the predicted classifications are not qualified. For a given test, there are 73 such results. The lower left box is when the annotated markers are not qualified and the predicted classifications are qualified. For a given test, there are 70 such results. The lower right box is when the annotated markers are not qualified and the predicted classifications are not qualified. For a given test, there are 1196 such results. The resulting sensitivity (true positives / annotated positives) is 94.1%. The resulting specificity (true negatives / annotated negatives) is 94.5%. The resulting accuracy ((true positives + true negatives) / total) is 94.3%.

[0091] refer to Fig.11 , depicts verification results 1100. Verification results 1100 compare annotated classifications with predicted classifications. The classifications contain pass and five failure modes: bright ROX, ROX spotting, ROX stripping, stop point bridging, and reformatting.

[0092] refer to Fig.12, PCR quality control system 1200 includes PCR system 802, computer system 806, fully connected network 1202 and CNN 1204. PCR system 802 generates QC images and sends the images to fully connected network 1202. Fully connected network 1202 may include a fully connected layer with one or more hidden layers. Unsupervised learning (e.g., autoencoder) can be applied to QC images to learn simple or complex features, which are then used as input to CNN 1204 or other fully connected networks to classify QC images into qualified and different failure modes. CNN 1204 may be an embodiment of neural network 700 described above. CNN 1204 determines whether the QC image is qualified or unqualified. The result is sent to computer system 806. The result may include whether the QC image is qualified or unqualified, which subarray is unqualified, and which failure mode it is. Computer system 806 may display the results to a user, store the results, perform additional transformations of the results, adjust future manufacturing parameters of the reaction plate array, and control PCR system 802. The computer system 806 can also modify the PCR system 802 based on the results, such as stopping the operation of the PCR system 802 that provides substandard results, or correcting the PCR system 802 when substandard conditions occur. The PCR system 802 can generate a series of QC images, each QC image at a different time. These QC images can be sent to the CNN 1204 in series. For each image, the CNN 1204 provides the results to the computer system 806. The PCR quality control system 1200 can also utilize multiple CNNs 1204, similar to Fig. 9 As depicted, each CNN 1204 receives a subarray. In addition, multiple fully connected networks 1202 may be initialized, each fully connected network receiving a subarray. The output of the fully connected network 1202 for each subarray may be sent to the CNN 1204.

[0093] Fig.13A recurrent neural network 1300 (RNN) is depicted. The variable x[t] is the input at stage t. For example, x[1] can be a one-hot vector that selects a particular element in a sequence. The variable s[t] is the hidden state at stage t. This is the "memory" of the network. The variable s[t] is calculated based on the previous hidden state and the input at the current stage: s[t] = f(Ux[t] + Ws[t-1]). The activation function f is typically a nonlinearity, such as tanh or ReLU. The input s(-1) required to calculate the first hidden state is typically initialized to all zeros. The variable o[t] is the output at stage t. For example, the prediction can be a vector of probabilities over the entire vocabulary: o[t] = softmax(Vs[t]). The recurrent neural network 1300 can be used to look at the evolution of a QC image during a PCR reaction. One or more CNNs can be input to the recurrent neural network 1300 (or there can be multiple RNNs, such as one RNN per CNN), and the recurrent neural network 1300 outputs a classification prediction based on how the CNN image classification evolves over time. The CNN may have a modified output layer so that instead of predicting pass / fail, the CNN sends a set of categorical features as input to the recurrent neural network 1300. These features may be, for example, the result of the second fully connected layer. If multiple RNNs are used (e.g., one RNN per CNN), the analysis system may combine the RNN predictions into a final prediction of the failure mode using a multi-layer perceptron (general ANN).

[0094] refer to Fig.14 , PCR quality control system 1400 includes PCR system 802, computer system 806, CNN 902 and RNN1402. PCR system 802 sends a series of QC images to CNN 902. The series is based on periodic sampling of the reaction plate. Then, each image will apply CNN 902 to determine the classification or features (for example, without applying the final fully connected output layer). Then send it to RNN 1402. RNN 1402 receives the classification or features of the series. Then, RNN 1402 determines the output, for example, due to the change in the QC image determined by RNN 1402, the PCR system 802 has failed or the PCR system 802 may fail. RNN 1402 can also determine the failure mode. Computer system 806 can display the results to the user, store the results, perform additional transformations of the results, and control PCR system 802. The computer system 806 may also modify the PCR system 802 based on the results, such as ceasing operation of a PCR system 802 that provides unacceptable results, or calibrating the PCR system 802 when an unacceptable condition occurs.

[0095] refer to Fig.15, PCR quality control system 1500 includes PCR system 802, computer system 806, CNN 902, CNN 904 and RNN 1502. PCR system 802 sends a series of QC images to CNN 902 and CNN 904. CNN 902, CNN 904 and other CNNs can receive subarrays of QC images in series. The series is based on periodic sampling of the reaction plate. Then, each image will apply CNN to determine the classification or characteristics of the subarray (for example, the final fully connected output layer is not applied). Then it is sent to RNN 1502. RNN 1502 can also be multiple RNNs in parallel, one RNN for each subarray. RNN 1502 receives the classification or characteristics of the series. Then, RNN 1502 determines the output, for example, due to the change in the QC image determined by RNN 1502, the PCR system 802 has failed or the PCR system 802 may fail. RNN 1502 can also determine the failure mode. Computer system 806 can display the results to a user, store the results, perform additional transformations on the results, and control PCR system 802. Computer system 806 can also modify PCR system 802 based on the results, such as ceasing operation of a PCR system 802 that provides unacceptable results, or calibrating PCR system 802 when an unacceptable condition occurs.

[0096] Fig.16 A cloud learning and control system 1600 is depicted according to one embodiment. The cloud learning and control system 1600 includes a cloud analysis system 1610, which includes a learning system 1612, such as one or more of the embodiments disclosed herein. Experimental data from multiple PCR runs or other experiments (e.g., PCR laboratory instrument 1604, PCR laboratory instrument 1606, and PCR laboratory instrument 1608) is monitored by the cloud analysis system 1610 via the Internet 1602 or other network. The cloud analysis system 1610 processes the experimental data and provides learned configuration parameters as feedback to adjust the configuration settings of the PCR instruments for current or future experiments.

[0097] Fig.17 is a block diagram of an example computing device 1700 that may incorporate embodiments of the present invention. Fig.17 The machine system of the aspect of the technical process described herein is only described and does not limit the scope of the claims. Those skilled in the art will recognize other changes, modifications and alternatives. In one embodiment, the computing device 1700 generally includes a monitor or graphical user interface 1702, a data processing system 1720, a communication network interface 1712, one or more input devices 1708, one or more output devices 1706, etc.

[0098] like Fig.17As depicted, data processing system 1720 may include one or more processors 1704 that communicate with a number of peripheral devices via a bus subsystem 1718. These peripheral devices may include one or more input devices 1708, one or more output devices 1706, a communication network interface 1712, and storage subsystems such as volatile memory 1710 and non-volatile memory 1714.

[0099] The volatile memory 1710 and / or the non-volatile memory 1714 may store computer executable instructions, and thus form logic 1722, which when applied to and executed by one or more processors 1704 implements embodiments of the processes and neural networks disclosed herein.

[0100] One or more input devices 1708 include devices and mechanisms for inputting information into the data processing system 1720. These may include a keyboard, a keypad, a touch screen incorporated into the monitor or graphical user interface 1702, an audio input device such as a voice recognition system, a microphone, and other types of input devices. In various embodiments, one or more input devices 1708 may be implemented as a computer mouse, a trackball, a trackpad, a joystick, a wireless remote control, a drawing tablet, a voice command system, an eye tracking system, etc. One or more input devices 1708 typically allow a user to select objects, icons, control areas, text, etc. that appear on the monitor or graphical user interface 1702 via commands such as clicking a button, etc.

[0101] Output device(s) 1706 include devices and mechanisms for outputting information from data processing system 1720. These may include a monitor or graphical user interface 1702, speakers, printer, infrared LEDs, etc. as are well known in the art.

[0102] The communication network interface 1712 provides an interface to communication networks (e.g., communication network 1716) and devices external to the data processing system 1720. The communication network interface 1712 can act as an interface for receiving data from other systems and transmitting data to other systems. Embodiments of the communication network interface 1712 can include an Ethernet interface, a modem (telephone, satellite, cable, ISDN), a (asynchronous) digital subscriber line (DSL), FireWire, USB, a wireless communication interface (e.g., Bluetooth or WiFi), a near field communication wireless interface, a cellular interface, etc.

[0103] Communication network interface 1712 may be coupled to communication network 1716 via an antenna, cable, etc. In some embodiments, communication network interface 1712 may be physically integrated on a circuit board of data processing system 1720, or in some cases may be implemented in software or firmware such as a "soft modem" or the like.

[0104] The computing device 1700 may include logic that allows communication over a network using schemes such as HTTP, TCP / IP, RTP / RTSP, IPX, UDP, etc.

[0105] The volatile memory 1710 and the non-volatile memory 1714 are examples of tangible media that are configured to store computer-readable data and instructions to implement various embodiments of the processes described herein. Other types of tangible media include removable memory (e.g., pluggable USB memory devices, mobile device SIM cards), optical storage media (e.g., CD-ROMs), DVDs, semiconductor memories (e.g., flash memories), non-transitory read-only memories (ROMs), volatile memories with backup batteries, networked storage devices, etc. The volatile memory 1710 and the non-volatile memory 1714 can be configured to store basic programming and data constructs that provide the functionality of the disclosed processes and other embodiments thereof within the scope of the present invention.

[0106] Logic 1722 implementing an embodiment of the present invention may be implemented by volatile memory 1710 and / or non-volatile memory 1714. Instructions of the logic 1722 may be read from volatile memory 1710 and / or non-volatile memory 1714 and executed by one or more processors 1704. Volatile memory 1710 and non-volatile memory 1714 may also provide a repository for storing data used by the logic 1722.

[0107] The volatile memory 1710 and the non-volatile memory 1714 may include multiple memories, including a main random access memory (RAM) for storing instructions and data during program execution and a read-only memory (ROM) in which read-only non-temporary instructions are stored. The volatile memory 1710 and the non-volatile memory 1714 may include a file storage subsystem that provides permanent (non-volatile) storage for program and data files. The volatile memory 1710 and the non-volatile memory 1714 may include a removable storage system, such as a removable flash memory.

[0108] Bus subsystem 1718 provides a mechanism for allowing the various components and subsystems of data processing system 1720 to communicate with each other as needed. Although communication network interface 1712 is schematically depicted as a single bus, some embodiments of bus subsystem 1718 may utilize multiple distinct busses.

[0109] It will be readily apparent to those skilled in the art that computing device 1700 may be a device such as a smart phone, a desktop computer, a laptop computer, a rack-mounted computer system, a computer server, or a tablet computer device. As is generally known in the art, computing device 1700 may be implemented as a series of multiple networked computing devices. In addition, computing device 1700 will typically include operating system logic (not shown), the type and nature of which are well known in the art.

[0110] Additional terms and explanations

[0111] Terms used herein should be given their ordinary meaning in the relevant art, or the meaning indicated by their usage in context, but if an explicit definition is provided, that meaning shall prevail.

[0112] "Support Vector Machine" refers to a supervised learning model with an associated learning algorithm that analyzes data for classification and regression analysis. Given a set of training instances, each labeled as belonging to one or the other of two categories, the SVM training algorithm builds a model that assigns new instances to one category or the other, making it a non-probabilistic binary linear classifier. The SVM model represents instances as points in space, and maps them so that instances of separate categories are separated by as wide a clear gap as possible. New instances are then mapped to the same space and predicted to belong to a category based on which side of the gap they fall. In addition to performing linear classification, SVMs can also perform nonlinear classification efficiently, using so-called kernel strategies, implicitly mapping their inputs to a high-dimensional feature space.

[0113] "Kernel" refers to a kernel function that operates in a high-dimensional implicit feature space without computing the coordinates of the data in that space, but rather by simply computing the inner products between the projections of all pairs of data in the feature space. This operation is usually computationally cheaper than explicit computation of the coordinates. When used with SVM, this approach is called the "kernel strategy."

[0114] "ReLU" refers to the rectification function, that is, an activation function defined as the positive part of its input. It is also called a ramp function and is similar to half-wave rectification in electrical signal theory. ReLU is a commonly used activation function in deep neural networks.

[0115] A "loss function", also called a cost function or error function (not to be confused with the Gaussian error function), is a function that maps the values ​​of one or more variables to real numbers that intuitively represent some "cost" associated with those values.

[0116] "Hyperbolic tangent function" refers to a function of the form tanh(x) = sinh(x) / cosh(x). The tanh function is a commonly used activation function in artificial neural networks. Like the sigmoid, the tanh function is also sigmoid-shaped ("s" shaped), but the output range is (-1, 1). Therefore, strongly negative inputs to the tanh will be mapped to negative outputs. In addition, only zero-valued inputs are mapped to near-zero outputs. These properties make the network less likely to get "stuck" during training.

[0117] "Sigmoid function" refers to a function of the form f(x) = 1 / (exp(-x)). The sigmoid function is used as an activation function in artificial neural networks. It has the property of mapping a wide range of input values ​​to a range of 0 to 1 or sometimes -1 to 1.

[0118] "Backpropagation" refers to an algorithm used in artificial neural networks to compute the gradients needed to calculate the weights to be used in the network. It is often used to train deep neural networks, a term that refers to neural networks with more than one hidden layer. For backpropagation, the loss function calculates the difference between the network output and its expected output after an example has been propagated through the network.

[0119] "Softmax function" refers to a function of the form f(xi) = exp(xi) / sum(exp(x)), where the sum is taken over a set of x. Softmax is used at different layers of artificial neural networks (usually at the output layer) to predict the classification of the input to these layers. The Softmax function calculates the probability distribution of event xi among 'n' different events. In general, this function calculates the probability of each target class among all possible target classes. The calculated probabilities help predict the target class represented in the input. The main advantage of using Softmax is the output probability range. The range will be from 0 to 1, and the sum of all probabilities will be equal to one. If the Softmax function is used for a multi-classification model, it will return the probability of each class, and the target class will have a high probability. The formula calculates the exponential (power) of a given input value and the sum of the exponential values ​​of all values ​​in the input. Then, the ratio of the exponential of the input value to the sum of the exponential values ​​is the output of the Softmax function.

[0120] "CTC loss function" refers to connectivity temporal classification, a type of neural network output and associated scoring function used to train recurrent neural networks (RNNs) such as LSTM networks to solve sequence problems with variable timing. CTC networks have continuous outputs (such as Softmax), which can be fitted by training to model the probability of labels. CTC does not attempt to learn boundaries and timing: if the label sequences differ only in alignment, ignoring gaps, they are considered equivalent. Equivalent label sequences can occur in many ways - making scoring a difficult task. Fortunately, there is an efficient forward-backward algorithm. The CTC score can then be used with the backpropagation algorithm to update the neural network weights. Alternative methods for neural networks suitable for CTC include hidden Markov models (HMMs).

[0121] "Gated Recurrent Unit (GRU)" refers to the gating mechanism in recurrent neural networks. Compared to LSTM, GRU can show better performance on smaller datasets. Compared to LSTM, GRU has fewer parameters because they do not have output gates.

[0122] "Beam search" refers to a heuristic search algorithm that explores a graph by expanding the most promising nodes in a finite set. Beam search is an optimization of best-first search that reduces its memory requirements. Best-first search is a graph search that ranks all partial solutions (states) according to some heuristic algorithm. But in beam search, only a predetermined number of the best local solutions are kept as candidates. Therefore, it is a greedy algorithm. Beam search uses breadth-first search to build its search tree. At each level of the tree, it generates all the successors of the state at the current level and sorts them in ascending order of heuristic cost. However, it only stores a predetermined number β of the best states (called the beam width) at each level. Only those states are expanded next. The larger the beam width, the fewer states are deleted. For infinite beam width, no states are deleted, and beam search is the same as breadth-first search. The beam width limits the memory required to perform the search. Beam search sacrifices completeness (the guarantee that the algorithm will terminate with a solution if one exists) because the target state may be deleted. Beam search is not optimal (that is, there is no guarantee that it will find the best solution). Typically, a beam search returns the first solution found. Beam search for machine translation is another case: once the configured maximum search depth (i.e., translation length) is reached, the algorithm will evaluate the solutions found during the search at various depths and return the best solution (the one with the highest probability). The beam width can be fixed or variable. One approach using a variable beam width starts with the minimum width. If no solution is found, the beam is widened and the procedure is repeated.

[0123] "Adam optimizer" refers to an optimization algorithm that can be used instead of the classic stochastic gradient descent procedure to iteratively update network weights based on training data. Stochastic gradient descent maintains a single learning rate (called α) for all weight updates, and the learning rate does not change during training. For each network weight (parameter), the learning rate is maintained and adjusted separately as learning develops. Adam combines the advantages of two other extensions of stochastic gradient descent. Specifically, the adaptive gradient algorithm (AdaGrad) can maintain a learning rate per parameter, thereby improving performance on sparse gradient problems (such as natural language and computer vision problems), while root mean square propagation (RMSProp) can also maintain a learning rate per parameter, which is adjusted according to the average of the most recent weight gradient magnitudes (e.g., how fast it changes). This means that the algorithm works well on online and non-static problems (such as noisy). Adam recognizes the benefits of both AdaGrad and RMSProp. Instead of adjusting parameter learning rates based on the average first moment (mean) as in RMSProp, Adam uses the average of the gradient second moment (center-free variance). Specifically, the algorithm computes exponential moving averages of the gradient and squared gradient, and parameters β1 and β2 control the decay rate of these moving averages. Initial values ​​of the moving averages and values ​​of β1 and β2 that are close to 1.0 (recommended) can cause the moment estimates to be biased toward zero. This bias can be overcome by first computing the bias estimate and then computing the bias-corrected estimate.

[0124] "Assay" refers to two target-specific primers or two primers and a probe used to amplify the target in a PCR reaction mixture.

[0125] "Forward primer" refers to the oligonucleotide flanking the 5' end of the amplicon. Reverse primer and forward primer are used together in a PCR reaction to amplify the target.

[0126] "Diluent" refers to a reagent used to dilute a sample or standard before adding it to a PCR reaction.

[0127] "Data collection" refers to the process by which the instrument detects fluorescence data from each well of the reaction plate during an instrument run. The instrument converts the signal into electronic data and saves the data in an experiment file.

[0128] "ROX dye" refers to an inert dye whose fluorescence does not change during the reaction and can be added to quantitative real-time PCR reactions to normalize well-to-well differences that may occur due to artifacts (e.g., pipetting errors or instrument limitations). The ROX inert reference dye consists of a 25 μM solution of 5-carboxy-X-rhodamine in 10 mM Tris-HCl (pH 8.6), 0.1 mM EDTA, and 0.01% TweenTM-20. Although many instruments require the use of a final concentration of 500 nM of the dye, newer instruments with optimized filter settings require the use of 50 nM of the dye.

[0129] "Reference dye" refers to ROX dye.

[0130] An "inert reference" is a dye that produces a fluorescent signal that is unrelated to PCR amplification and is added to each reaction at a constant concentration. Because the inert reference signal should be consistent in all wells, it is used to normalize the reporter dye signal to account for non-PCR related fluorescence fluctuations due to small differences in well-to-well volume. Normalization to the inert reference signal typically produces data with very high precision across technical replicates.

[0131] "Reporter" refers to the fluorescent dye used to detect amplification. Reagent, reporter dye attached to the 5' end.

[0132] "Reverse primer" refers to an oligonucleotide flanking the 3' end of an amplicon. A reverse primer and a forward primer are used together in a PCR reaction to amplify the target.

[0133] As used herein, "circuit system" refers to a circuit system having at least one discrete circuit; a circuit system having at least one integrated circuit; a circuit system having at least one application-specific integrated circuit; a circuit system forming a general-purpose computing device that is configured by a computer program (e.g., a general-purpose computer configured by a computer program that at least partially executes a process or device described herein, or a microprocessor configured by a computer program that at least partially executes a process or device described herein); a circuit system forming a memory device (e.g., various forms of random access memory); or a circuit system forming a communication device (e.g., a modem, a communications switch, or an optoelectronic device).

[0134] "Firmware" as used herein refers to software logic embodied as processor-executable instructions stored in read-only memory or media.

[0135] "Hardware" as used herein refers to logic embodied as analog or digital circuitry.

[0136] "Logic" herein refers to machine memory circuits, non-transitory machine-readable media, and / or circuitry by means of which material and / or material energy configuration includes control and / or program signals and / or settings and values ​​(e.g., resistance, impedance, capacitance, inductance, current / voltage levels, etc.) that can be applied to affect device operation. Magnetic media, electronic circuits, electrical and optical memory (both volatile and non-volatile), and firmware are examples of logic. Logic specifically excludes pure signals or software per se (however, machine memory that includes software and thereby forms a configuration of matter is not excluded).

[0137] "Software" herein refers to logic implemented as processor-executable instructions in machine memory (eg, read / write volatile or non-volatile memory or media).

[0138] Herein, references to "one embodiment" or "an embodiment" do not necessarily refer to the same embodiment, but they may refer to the same embodiment. Unless the context clearly requires otherwise, throughout the specification and claims, the words "comprise / comprising" and the like should be interpreted in an inclusive sense rather than an exclusive or exhaustive sense; that is, in the sense of "including but not limited to". Unless expressly limited to a single or multiple, words using the singular or plural also include the plural or singular, respectively. In addition, the words "herein," "above," "below," and words of similar meaning, when used in this application, should refer to the application as a whole, rather than to any particular part of the application. When the claims use the word "or" when referring to a list of two or more items, the word covers all of the following interpretations of the word: any one of the items in the list, all of the items in the list, and any combination of the items in the list, unless expressly limited to one or the other. Any term not expressly defined herein has a conventional meaning as commonly understood by a person skilled in the relevant art.

[0139] The various logical function operations described herein can be implemented using logic referenced by nouns or noun phrases reflecting the operation or function. For example, an association operation can be implemented by an "associator" or "correlator". Similarly, an on-off operation can be performed by a "switch", a selection can be performed by a "selector", etc.

[0140] Although the foregoing disclosure discusses illustrative aspects and / or embodiments, it should be noted that various changes and modifications may be made herein without departing from the scope of the described aspects and / or embodiments as defined by the appended claims. Furthermore, although elements of the described aspects and / or embodiments may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated. Additionally, unless otherwise stated, all or a portion of any aspect and / or embodiment may be utilized together with all or a portion of any other aspect and / or embodiment.

Claims

1. A system for quality control, comprising: PCR system; Reaction array plates; a convolutional neural network (CNN) configured to receive an image sequence of the reaction array plate in the PCR system and generate a failure mode prediction for the reaction array plate based on the image sequence; as well as The output of the CNN is coupled to the control of the reaction array plate. The system according to claim 1 , wherein the control is a stop control.

3. The system of claim 1, further comprising: Mapping functions between the centers of the through-holes of the reaction array plate and the arrays in the layers of the CNN.

4. The system of claim 1, wherein the output of the CNN transforms the image sequence into ROX dot-like predictions, ROX stripped predictions, bright ROX predictions, dot-dot-bridged predictions, and reformatted error predictions.

5. The system of claim 1, wherein the CNN lacks a final fully connected output layer.

6. The system of claim 5, further comprising at least one recurrent neural network (RNN) coupled to receive feature indications from the CNN and transform the feature indications into failure mode predictions.

7. The system of claim 6, comprising a plurality of RNNs, wherein predictions of the plurality of RNNs are combined into a final prediction of at least one failure mode by a perceptron.

8. The system of claim 1, wherein the CNN further comprises: Input layer, which consists of a 54×54 array; The first convolutional layer, which consists of 32 54×54 arrays; as well as The second convolutional layer consists of 64 18×18 arrays.

9. The system of claim 8, wherein the CNN further comprises: The first fully connected layer, which includes 192 nodes; a second fully connected layer coupled to the first fully connected layer and comprising 48 nodes; as well as An output layer is coupled to the second fully connected layer and generates a binary pass or fail output signal.

10. The system of claim 9, the output layer further generating a prediction of a failure mode of the reaction array plate.

11. A method for quality control, comprising: Operate the PCR system on a reaction array plate; applying image sequences from a plurality of subarrays of the reaction array plate to a plurality of convolutional neural networks (CNNs) during operation of the PCR system on the reaction plate array; operating the CNN to generate a failure mode prediction for the reaction board based on the image sequence; as well as The output of the CNN is coupled to one or more of a setup for manufacturing the reaction array plate or controlling the PCR system. The method according to claim 11 , wherein the control is a stop control.

13. The method according to claim 11, further comprising: The centers of the through-holes of the reaction array plate are mapped to different ones in the CNN.

14. The method of claim 11, wherein the output of the CNN transforms the image sequence into ROX dot-like prediction, ROX stripped prediction, bright ROX prediction, dot-dot bridged prediction, and reformatted error prediction.

15. The method of claim 11, wherein one or more of the CNNs lack a final fully connected output layer.

16. The method of claim 15, further comprising coupling the output of the CNN to at least one recurrent neural network (RNN) to receive feature indications from the CNN and transform the feature indications into the failure mode predictions.

17. The method of claim 16, comprising a plurality of RNNs, wherein predictions of the plurality of RNNs are combined into a final prediction of at least one failure mode by a perceptron.

Images