Methods for genotype detection using melt curve signatures

Abstract

Disclosed herein are methods of determining a presence of a target analyte, the method including: receiving data representing at least one melt curve representing denaturation of the target analyte; generating at least one extracted feature for the at least one melt curve, thus generating at least one extracted features; providing the at least one extracted feature to a machine learning model trained to determine whether the target analyte is present in a set of known target analytes using the at least one extracted feature; and receiving, from the machine learning model, a prediction value for the at least one extracted feature of the set of extracted features wherein each prediction value is indicative of a probability of whether the at least one extracted feature is present in a training feature set representing the set of known target analytes on which the machine learning model was trained.

Description

METHODS FOR GENOTYPE DETECTION USING MELT CURVESIGNATURESTECHNICAL FIELD

[0001] The present document relates to genotype detection and related methods.STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH

[0002] This invention was made with government support under grant number R01AI134982 awarded by the NIH, grant number R01AI134982 awarded by the NIAID, grant number DGE-1650112 awarded by the NSF, and grant number 2T15LM011271-1 1 awarded by the National Library of Medicine. The government has certain rights in this invention.BACKGROUND

[0003] Effective large-scale monitoring and surveillance of novel pathogens are critical components of contemporary public health strategy. Genotypic screening of pathogens, in particular, enables researchers and clinicians to gain nuanced insights into disease transmission patterns, virulence factors, antibiotic resistance profiles, and potential outbreak: sources. Modern techniques for genotypic screening, however, can be cost-prohibitive, slow, and intractable to scale.SUMMARY

[0004] Disclosed herein are various embodiments of methods for unique bacterial genotype detection using melt curve signature features. In some embodiments, methods include receiving a biological sample. In some embodiments, methods include performing, using a digital high-resolution melt (DHRM) analysis apparatus, DHRM on the sample. In some embodiments, methods include receiving from the DHRM apparatus a data set of melt curves, the data set may include one melt curve for each of a set of target analytes in the sample. In some embodiments,methods include generating a set of extracted features from the data set. In some embodiments, methods include providing the set of extracted features to a trained machine learning model previously trained to determine unique bacterial genomes based on a training feature set for known bacterial genomes. In some embodiments, methods include receiving from the machine learning model a prediction value for each extracted feature of the set of extracted features where each prediction value is indicative of a probability of whether the extracted feature is present in the training feature set.

[0005] Examples may include one or more of the following features. Generating the set of extracted features may include processing the data set through a feature extraction algorithm. The feature extraction algorithm can be a non-parametric feature extraction algorithm. The feature extraction algorithm can be based on a rolling window transformation having a kernel size of 20, a stride of 8, and / or a signature depth of 3. The feature extraction algorithm can be time-shift invariant. The method may include training the machine learning model may include providing to an untrained machine learning model the training set of melt curves and a sequence of bacterial genome values for each of the training set of melt curves to provide the trained learning model. The method may include validating the trained learning model using a cross- validation method. The cross-validation method can be a leave-one-group-out (LOGO) analysis method. Receiving from the DHRM apparatus a data set of melt curves may include receiving a raw set of melt curves. The method may include performing a preprocessing step on the raw set of melt curves. The preprocessing step may include a background subtraction step, a noise reduction step, or both. The trained machine learning model can be a logistic regression model, a naive bayes (NB) model, a support vector machine (SVM) model, a neural network model, and / or a random forest (Randf) model. The biological sample can be a blood sample. The method may include comparing the prediction value to a threshold value and, if the prediction value is below the threshold value, determining that the extracted feature is not preset in the training feature set. The threshold value can be determined using a Youden’s index value. The set of target analytes may include a set of nucleotide sequences from the unique bacterial genome.

[0006] In general, an aspect disclosed herein is a computer program product for determining the presence of a unique bacterial genome. In some embodiments, the computer program product includes receiving from the DHRM apparatus a dataset of melt curves. In some embodiments, the data set may include one melt curve for each of a set of target analytes in the sample. In some embodiments, the computer program product includes generating a set of extracted features from the data set. In some embodiments, the computer program product includes providing the set of extracted features to a trained machine learning model previously trained to determine unique bacterial genomes based on a training feature set for known bacterial genomes. In some embodiments, the computer program product includes receiving from the machine learning model a prediction value for each extracted feature of the set of extracted features where each prediction value is indicative of a probability of whether the extracted feature is present in the training feature set.

[0007] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Although methods and materials similar or equivalent to those described herein can be used to practice the invention, suitable methods and materials are described below. All publications, patent applications, patents, and other references mentioned herein are incorporated by reference in their entirety. In case of conflict, the present specification, including definitions, will control. In addition, the materials, methods, and examples are illustrative only and not intended to be limiting.

[0008] The details of one or more embodiments are set forth in the accompanying drawings and the description below. Other features and advantages will be apparent from the description and drawings, and from the claims.BRIEF DESCRIPTION OF THE DRAWINGS

[0009] FIG. 1 A is a flow chart diagram showing methods provided herein.

[0010] FIG. 1 B is a flow chart diagram showing a method provided herein.

[0011] FIG. 2A is a line chart showing experimentally-obtained dHRM melt curves.

[0012] FIG. 2B is a line chart showing examples of synthesized melt curves.

[0013] FIG. 3A is a boxplot of classification probability results for ten bacterial species.

[0014] FIG. 3B is a line chart showing a receiver operating characteristic (ROC) curve for determining a threshold value.

[0015] FIG. 4A is a line chart showing a ROC curve for accumulated cross- validation experiments.

[0016] FIG. 4B is a line chart showing a ROC curve for individual cross- validation experiments.

[0017] FIG. 5 is a bar chart showing the determined Youden’s Index for a number of machine-learning (ML) models.

[0018] FIG. 6A is a boxplot for bacterial species indicating the probability of a novel bacterial genomic prediction.

[0019] FIG. 6B is a line chart showing a ROC curve for determining the practical and optimal threshold values for the boxplot of FIG. 6A.

[0020] FIG. 6C is a boxplot for bacterial species indicating the probability of a novel bacterial genomic prediction.

[0021] FIG. 6D is a line chart showing a ROC curve for determining the practical and optimal threshold values for the boxplot of FIG. 6C.

[0022] FIG. 7A is a bar chart comparing Youden’s Index for each of the ML methods using various validation thresholds.

[0023] FIG. 7B is a bar chart comparing Youden’s Index for each of the ML methods using various validation thresholds and compared to those of FIG. 7A.

[0024] FIGS. 8A-8C are showing steps of the data preparation workflow.

[0025] FIGS. 9A-9D are line charts showing an overview of experimentally obtained and synthesized melt curves.

[0026] FIG. 10 is line charts showing an overview of the synthetically created melt curves for 50 pathogens.

[0027] FIGS. 11A-11J are multiple bar charts depicting the probability of a bacterial species to be known or novel according to a classification algorithm.

[0028] FIGS. 12A-12D are four bar charts comparing the Youden’s Index value for each of classification algorithms on the x-axis.

[0029] FIG. 13 is a line chart shows the correlation between melt curve distance and patristic distance from E. Coli.

[0030] In the figures, like references indicate like elements.DETAILED DESCRIPTION

[0031] Recent advancements have positioned high resolution melt analysis (HRM) as a reliable method for heterozygote detection. Moreover, the melt curves produced using HRM can now be used to determine a sequence-specific signature. When applied to the problem of identifying and differentiating bacteria, universal primers have been used to target the 16S rRNA gene, which is highly conserved in bacteria but contains variable regions that are specific to different species or strains. This enables broad-based amplification of the bacterial 16S gene with universal primers, while relying on melt curves to genotype the hypervariable sequences specific to organism identity.

[0032] The use of machine learning (ML) classification, where each unique sequence-specific curve signature represents a pathogen class, can be considered as framework to utilize HRM as a broad-based sequence profiling tool. This could be especially valuable in clinical diagnostics, where specifically identifying pathogenic bacteria is crucial for determining the appropriate treatment.

[0033] The growth in the amount of HRM data available for ML training has further accelerated the practical application of these methods for pathogen profiling, e.g., bacterial genome detection and classification. The rising prominence of digital PCR (dPCR) has led to the evolution from traditional HRM to digital HRM (dHRM) which enables 200-fold increases in the number of melt curves.

[0034] This methodology originated from a unique dHRM platform that employs heat transfer and imaging mechanisms to simultaneously melt thousands of dPCR reactions. A distinct feature of this methodology is the digital design, in which each reaction is characterized by the presence or absence of a genome as its DNA template.

[0035] While traditional HRM might typically operate on a 96-well plate, dHRM utilizes a commercial dPCR chip with 20,000 partitions, resulting in 20,000 HRM curves. The application of Naive Bayes (NB), Support Vector Machines (SVM), Nearest Neighbors using Dynamic Time Warping, Random Forest (RandF), and Neural Networks, have been demonstrated as effective for the purpose of pathogen classification using melt curve data. SVM algorithms in particular have shown notable performance even with relatively few melt curves. Thus, ML models as disclosed herein can accurately identify the presence of novel (e.g., unique) genotypes to enable broad-based pathogen detection and classification.

[0036] FIG. 1A is a flowchart diagram showing an exemplary method for detecting unique bacterial genotypes in a sample. Bacterial genomes are used as an example in FIG. 1A, so other examples may include fungal genomes, viral genomes, archaeal genomes, or a combination of these. The method of FIG. 1A includes a bacterial genomic material (e.g., DNA or RNA) extraction step. A bacterial species is cultured and the genomic material extracted as described herein. In some embodiments, the quality and concentration of the extracted genomic material is assessed. For instance, this can be assessed using spectrophotometric absorbance measurements. In some embodiments, the identities of at least one species can be confirmed using sequencing.

[0037] In some embodiments, amplification is performed on the extracted genome to amplify the target analytes within the extracted genome. Techniques to amplify the target analytes are known to the field. In embodiments in which the target analyte is the nucleic acid, polymerase chain reaction (PCR) can be used to amplify the target analyte. PCR methods include digital PCR (dPCR). Briefly, dPCR allows the amplification of many target analytes within a sample. A dPCR chip has multiple wells, each well containing the necessary components to amplify at least a portion, e.g., a target analyte, of the extracted genome. dPCR can be performed using commercially available dPCR instruments.

[0038] As used herein, the term ‘target analyte’ refers to at least a portion of a genome present in the biological sample, e.g., a nucleic acid sequence. The target analyte can be at least a portion of the genome of the host of the biological sample, or at least a portion of the genome of an organism that is not the host of the biological sample. In some examples, the target analyte is at least a portion of a genome that is not natively present in the host of the biological sample. As used herein, the term ‘host’ refers to the organism from which the biological sample was sampled.

[0039] As used herein, ‘natively present’ refers to organisms normally present in the natural flora of the host, while ‘not natively present’ refers to organisms not normally present in the natural flora. Organisms that can be natively, or not natively, present include bacteria, fungi, viruses, archaea, or a combination of these. In some examples, the organism that is not natively present in human natural flora includes a non-native bacteria. In such examples, the target analyte may be at least a portion of the bacterial genome that is not natively present the natural flora of a human.

[0040] In some embodiments, dHRM is performed on the amplified genome to generate melt curves. Briefly, a dye can be added to the amplified target analytes. The dye fluoresces upon binding with the target analytes. In some embodiments, the target analyte is a nucleic acid target analyte, e.g., DNA, or RNA, and the fluorescent dye is an intercalating fluorescent dye.

[0041] In some embodiments, the target analytes are heated such that the target analytes denature. In some embodiments, denaturation results in a loss of fluorescence as the dye ceases to bind to the target analytes. In some embodiments, the loss in fluorescence is recorded to produce a melt curve. In some embodiments, some or all of the wells of the dPCR chip are monitored in parallel. In some embodiments, one melt curve can be generated for each well, for each target analyte, or both. In some embodiments, this process can be highly parallel. In some embodiments, more than 10,000 (e.g., 20,000 or more) melt curves can be generated in parallel, one for each target analyte.

[0042] In some embodiments, the melt curves are preprocessed to provide preprocessed melt curves. In some embodiments, preprocessing includes subtraction of a background reading from each of the melt curves. Such pre-processing can align the curves along a common axis for display to a user. Preprocessing can reduce noise associated with a background signal in the melt curves. Optionally, the melt curves can be truncated and a portion of each melt curve used in preprocessing, downstream processing, or both.

[0043] In some embodiments, the preprocessed melt curves represent the temperature dependence of the denaturation of the target analyte, e.g., the denaturation of double-stranded DNA into single-stranded DNA, the denaturation of secondary or tertiary RNA structures into single-stranded RNA, or both. In some embodiments, the melt curves represent a change in fluorescence as dye molecules cease to fluoresce as the target analyte denatures. The preprocessed melt curves can be a time-series of fluorescence values, e.g., a 1-dimensional time series. In some embodiments, the preprocessed melt curves represent fluorescence values (e.g., change in fluorescence over time, e.g., dF / dT) for a number of temperature steps in a temperature range over which the melt curve was recorded. The preprocessed melt curves can represent fluorescence loss curves over time as the fluorescent dye ceases to be intercalated as the double-stranded DNA melts.

[0044] In some embodiments, the preprocessed melt curves are provided to a feature extraction algorithm. In some embodiments, the feature extraction algorithm extracts features from the preprocessed melt curves. In some embodiments, the extracted features represent one or more features of the melt curve associated with the target analyte. Extracting features can reduce the dimensionality of a data set thus enabling improved efficiency and performance in ML models.

[0045] The feature extraction algorithm can include applying one or more transformations on the preprocessed melt curves. In some embodiments, the preprocessed melt curves are modeled as 1 -dimensional time-series before the one or more transformations are applied.

[0046] In some embodiments, the transformation includes a feature extractor, e.g., a non-parametric feature extractor. In some embodiments, the feature extractor computes at least one integral, e.g., a series of integrals, along a data path. In some embodiments, the feature extractor determines at least one parameter representing the integral. Some examples of the integral parameter include, without limitation, the order, the area, or both. In some embodiments, the feature extractor determines the at least one parameter of at least one, e.g., each, of the preprocessed melt curves using the at least one computed integral.

[0047] In some embodiments, the transformation is time-shift invariant. In some embodiments, the transformation is sensitive to the shape, e.g., the geometric shape, of the path. For example, a transformation can be applied using a rolling window. The rolling window may be described with a kernel size, and a stride, or both, across the time-series. In some embodiments, the transformation generates a set of features for the downstream ML model processing using the transformation. In some embodiments, the transformation can have parameters representing a kernel size of the rolling window, a stride or the rolling window, or both. In some embodiments, the one or more parameters of the transformation depends on one or more parameters of the preprocessed melt curves. In some embodiments, the kernel size, the stride, or both, are less than a sequence length, a signature depth, or both, of the preprocessed melt curves. The kernel size can range from 2 to 20, (e.g., 3, 4, 5, 6, 7, 8, 9, 10, 12, 14, 16, or 18).

[0048] In some embodiments, regularization is used to address correlation between the extracted features provided to the ML model. Regularization may be used to reduce model overfitting, e.g., when a model is trained to perform a function usinga first set of training data and has reduced performance when generalizing to newly provided or unseen data. In some embodiments, the regularization adds a penalty term to a loss function of a model. The regularization may include dropout, L1 regularization (e.g., Lasso), L2 regularization (e.g., Ridge), or any combination of these.

[0049] In some embodiments, the set of extracted features is provided to an ML model for classification. In some embodiments, the one or more ML model is selected for analysis of the extracted features. Examples of machine learning models suitable for detecting novel target analytes include, without limitation, logistic regression, naive bayes (NB), support vector machines (SVM), neural networks, random forest (RandF), or any combination of these.

[0050] In some embodiments, the cross-validation analysis is performed between the one or more selected machine learning models. Cross-validation is a statistical method used analyze one or more machine learning models. Cross- validation can be used to compare and select a model for an example modeling problem. One example of cross-validation analysis is leave-one-group-out (LOGO) analysis. In LOGO analysis, a portion of the melt curves are grouped and reserved from the analysis, e.g., into a ‘hold out’ group. The remaining melt curves are grouped, e.g., into a test group. In some embodiments, a ratio of 80:20 ‘test group’:’hold out group’ is used.

[0051] In some embodiments, of LOGO analysis, the test group is used to train the selected machine learning models. In some embodiments, the hold out group is then added to the test set. In some embodiments, the selected machine learning models are then tested on the respective ability to recognize the curves of the held- out species as novel within the modified test set is tested.

[0052] In some embodiments, a threshold for determining target analyte novelty is set. The threshold can be set according to one or more criteria. One example of a non-limiting criteria for setting the threshold is a Youden’s Index, e.g., a Youden’s J statistic. Briefly and without expressing limitation, Youden’s Index is a statistic that captures the performance of a two variable, e.g., binary, diagnostic test. A Youden’s Index threshold value can be calculated by accumulating the LOGO experiments results and determining the Youden Index from the combined receiver operating characteristic (ROC) curve.

[0053] In some embodiments, a ROC curve illustrates the performance of the ML model at varying threshold values. A ROC curve compares of the true positive rate (TPR) against the false positive rate (FPR) at each threshold setting. The threshold value can be set to the point on the ROC curve at which sensitivity plus specificity minus one is maximized, e.g., the maximum of TPR + FPR + 1 at each threshold value.

[0054] In some embodiments, the threshold value is validated. One non-limiting example of validating the threshold includes comparing the threshold value, e.g., the operating point on the ROC curve to the determined optimal threshold value based on the Youden's index. The threshold value can be within a range of the Youden’s index to be validated.

[0055] In some embodiments, the ML model is pre-trained to recognize features extracted from melt curves associated with a set of target analytes, e.g., a set of pre-determined target analytes, e.g., one or more melt curves associated with each target analyte. In some embodiments, the ML model is pre-trained to recognize features from 10 or more target analytes, e.g., 50 or more, 100 or more, or 200 or more.

[0056] In some embodiments, a probability value for each input feature is received from the ML model. In some embodiments, the ML model outputs a probability value for each input feature. As each input feature corresponds to a processed melt curve, each probability value therefore corresponds to probability of the input feature being present in the training data set. Thus, a high probability value is indicative of a high probability that the input feature is present in the training data set.

[0057] In some embodiments, the probability value of the unique target analyte is compared to the threshold value to determine whether the feature corresponds to a unique target analyte, e.g., a target analyte not associated with a feature set in the training data set. The threshold value can be determined through statistical analysis of the cross-validation results as described herein.

[0058] In some embodiments, if the probability value is equal to or greater than the threshold value, the ML model returns a classification of non-unique, e.g., known, e.g., present in the training data set, for the extracted feature associated with the probability value. In some embodiments, if the probability value is less than the threshold value, the ML model returns a classification of a bacterial genome that is notpresent in the training data set, e.g., unique, for the extracted feature associated with the probability value.

[0059] In some embodiments, the method of determining the presence of a unique target analyte is performed using a dHRM apparatus. In some embodiments, the dHRM apparatus performing the method can be termed a ‘detection system’ for determining the presence of a unique bacterial genome. In some embodiments, the method is a computer-implemented method which can be stored on a computer program product including a non-transitory computer readable medium, e.g, a data processing device, encoded with instructions to cause one or more processors to perform the method.

[0060] In some embodiments, the presence of a unique bacterial genome in the biological sample is determined. The biological sample can be a blood sample, a urine sample, a saliva sample, a sweat sample, a hair sample, a skin tissue sample, a nail sample, a sputum sample, a bone marrow sample, a cerebrospinal fluid (CSF) sample, a tear sample, a breath sample, a feces sample, a semen sample, a vaginal secretion sample, an amniotic fluid sample, or a placental tissue sample.

[0061] In some examples, the host from which biological sample is obtained is an animal such as a mouse, a rat, a rabbit, a dog, a cat, a non-human primate, a fish, a bird, a pig, or a sheep. In some examples, the host from which biological sample is obtained is a human, such as a neonate, a newborn, an infant, a toddler, a preschooler, a school-age child, an adolescent, a young adult, a middle-aged adult, or an older adult.

[0062] In some embodiments, disclosed herein are methods of determining the presence of a unique target analyte from a biological sample. FIG. 1 B is a flow chart diagram detailing exemplary steps of the method 120 to determine the presence of the unique target analyte in a biological sample.

[0063] In FIG. 1 B, data representing at least one melt curve, the at least one melt curve representing denaturation of a target analyte is received (122). The at least one melt curve represents the denaturation of the target analyte as a change in fluorescence over time. The target analyte can be provided from a biological sample. In one example, at least one target analyte is extracted from the biological sample.

[0064] In FIG. 1 B, at least one extracted feature is generated using the at least one melt curve (124). A feature extraction algorithm receives the at least one melt curve and generates the at least one extracted feature based on the at least one meltcurve. The extracted features represent one or more features of the melt curve associated with the target analyte. Extracting the features reduces the dimensionality of data provided to downstream machine learning models. Reducing the dimensionality can improve efficiency and performance of the machine learning models.

[0065] In some embodiments, the feature extraction algorithm applies one or more transformations to the melt curves to provide preprocessed melt curves. In some embodiments, the feature extraction engine applies a rolling window transformation to the melt curves.

[0066] In some embodiments, the feature extraction algorithm applies regularization to the extracted features. Examples of regularization are described herein, include, without limitation, dropout, L1 regularization, L2 regularization, or any combination of these.

[0067] In FIG. 1 B, the at least one extracted feature is provided to a machine learning model (126). In some embodiments, the machine learning model is previously trained to determine whether the at least one target analyte is present in a set of known target analytes. In some embodiments, the machine learning model uses the at least one extracted feature to determine whether the extracted feature is represented in the set of extracted features of known target analytes.

[0068] In FIG. 1 B, prediction value is received from the machine learning model (step 128). In some embodiments, the prediction value is correlated with the at least one extracted feature provided to the machine learning model. In some embodiments, the prediction value represents an estimation of the predicted confidence whether the extracted feature represents a target analyte that is represented in the known target analytes on which the machine learning model was trained. In some embodiments, ahigh prediction value can indicate increased confidence that the extracted feature represents a target analyte in the known target analytes compared to a low prediction value.

[0069] In FIG. 1 B, the prediction value is compared to a presence threshold value (130). In some embodiments, the presence threshold value represents a target degree of confidence for which the extracted feature is present in the training feature set. In some embodiments, if the prediction value satisfies the president's threshold value, the machine learning model provides an indication that the target analyte represented by the extracted feature is present in the set of known target analytes onwhich the machine learning model was trained. In some embodiments, if the prediction value does not satisfy the presence threshold value, the machine learning provides an indication that the target analyte represented by the extracted feature is not present in the set of known target analytes. A target analyte represented by an extracted feature not present in the set of known target analytes can be a unique target analyte, such as a portion of a bacterial genome that is not natively present in the biological sample.

[0070] In some embodiments, the target analyte can be amplified using amplification techniques known in the field. In some embodiments, the target analyte is it DNA or RNA sequence representing a portion of a bacterial genome. The target analyte can be amplified using DNA or RNA amplification techniques known to the field.

[0071] In some embodiments, the at least one melt curve can be preprocessed before being provided to the feature extraction algorithm. Preprocessing can include background subtraction or any preprocessing method described herein.

[0072] While one exemplary order of the method 120 is shown above, the various steps of the method 120 can be performed in a different order. For example, at least one extracted feature can be produced for at least one melt curve, additional data received representing at least one additional melt curve, and an additional extracted feature can be produced from the additional melt curve before feeding both extracted features to the machine learning model.EXAMPLESExample 1 - Machine Learning Based DNA Melt Curve Profiling Enables Automated Novel Genotype Detection.

[0073] Bacterial Strains

[0074] The ten bacterial species used in this study and their corresponding melt curves are the primary causative pathogen in cases of neonatal sepsis. Bacteria were cultured in Lurie-Bertani (LB) broth or Tryptic Soy broth (TSB), as required, and incubated overnight at 37°C.

[0075] The bacterial species used in this study included Streptococcus agalactiae (S. agalactiae), Streptococcus gallolyticus (S. gallolyticus), Citrobacter koseri (C. koseri), Listeria monocytogenes (L. monocytogenes), Haemophilusinfluenzae (H. influenzae), Streptococcus sanguinis (S. sanguinis), Staphylococcus aureus (S. aureus), Streptococcus pneumoniae (S. pneumoniae), Enterococcus faecium (E. faecium), and Escherichia coli (E. coli).

[0076] Bacterial genomic DNA extraction and PCR

[0077] Following culturing, DNA extraction was performed using Wizard Genome DNA Purification kit (Promega Corporation, Madison, Wl). The quality and concentration of the extracted DNA was assessed using spectrophotometric absorbance measurements and confirmed the identities of the species from sequencing, genomic DNA dilutions were prepared for use with dPCR and used the commercially available QuantStudio 3D Digital PCR 20IC chip v2 (Applied Biosystems, Foster City, CA) for amplification, the manufacturer’s recommended process was followed, the reagents were customized. The composition of the dPCR master mix included 1 pL of sample, 0.15 pM forward primer 5'-- GYGGCGNACGGGTGAGTAA-3' (Integrated DNA Technologies, Coralville, IA), 0.15 pM reverse primer 5'AGCTGACGACANCCATGCA- 3' (Integrated DNA Technologies, Coralville, IA), 0.02 U / pL of Phusion HotStart Polymerase (Thermo Fisher Scientific, Waltham, MA), 0.2 mM dNTPs (Invitrogen, Carlsbad, CA), 1X Phusion HF Buffer containing 1.5 mM MgCI2 (Thermo Fisher Scientific, Waltham, MA), 2.5X EvaGreen (Biotium, Fremont, CA), 2X ROX (Thermo Fisher Scientific, Waltham, MA), and ultrapure PCR water (Quality Biological. Ine., Gaithersburg, MD) to bring the total volume to 14.5 pL. the chip was loaded by spreading 14.5 pL, of the master mix per the manufacturer's recommendation The dPCR was cycled on a flatbed thermocycler with the following cycle settings: an initial enzyme activation (98 °C, 30 s), followed, by 70 cycles (95 °C, 30 s, 59 °C, 30 s, 72 °C, 60 s).

[0078] DNA melt curve generation and preprocessing

[0079] The architecture of the U-dHRM device has been previously described. Briefly, a copper plate hosts a microfluidic dPCR chip, separated by a thin layer of thermal grease to ensure efficient heat transfer. Temperature control was achieved through a thermoelectric module (TE Technology Inc., Traverse City, Ml). PID controller (Meerstetter Engineering GmbH, Rubigen, Switzerland). Class 1 / 3B resistance temperature detector (RID) (Herseus, Hanau, Germany) embedded in the copper block, IC-type thermocouple (OMEGA Engineering. Stamford, CT), and heatsink. The device was secured on-stage for optimal fluorescent imaging using a custom- made adapter. A Nikon Eclipse Ti microscope (Nikon, Tokyo, Japan) captured simultaneous fluorescent images from heat ramping the DNA-intercalating dye, EvaGreen™ (EX / Em: 488 nm / 561 nm) and the control dye, ROX™ (EX / Em: 405 nm / 488 nm). An automated image processing algorithm implemented in MATLAB was used to generate the melt curves, background subtraction was performed using a linear method. This horizontally aligns the tails of the melt curves with the x-axis, to ensure they are most similar to the theoretically predicted uMelt curves.

[0080] Feature Engineering

[0081] The resulting preprocessed melt curves contained fluorescence loss values (e.g., -dF / dT) for 410 temperature steps in the range (51 °C, 92 °C). This was modeled as a 1 -dimensional time-series and apply a ‘signature transform’ to each melt curve. The signature method was a non-parametric feature extractor that computes a series of integrals along a data path that captures its order and area. The signature transform method was optionally time-shift invariant and was sensitive to the geometric shape of the path. The signature transform was applied on a rolling window with a kernel size of 20 and a stride of 8 across the time-series with time and basepoint augmentations and a signature depth of 3 to generate a set of features for the downstream classification tasks.

[0082] Machine learning model selection

[0083] Five ML methods were compared: Logistic Regression, Naive Bayes (NB), Support Vector Machines (SVM), Neural Networks and Random Forest (RandF). To address correlation between the input signature features, L2 regularization for the logistic regression, SVM, and neural network models were used. Discussion of how calibrating the probabilities affects the results is presented herein. All algorithms were built and implemented using the scikit-learn package within Python programming language. All data and code are available on https: / / github.com / aboussina / dHRM- novelty-detection.

[0084] Quantification of Genotypic Differences

[0085] To further assess the utility of the derived HRM signatures for the identification of distinct genotypes, the ability of the aforementioned ML models toquantify the degree of genotypic differences between species were analyzed. That is, beyond classifying a melt curve to a given species, this was reformulated as a regression problem where the target variable was a metric for genotypic difference. This metric was calculated by mapping ten bacterial species on the SILVA phylogenetic tree and computing the patristic distance between the node of each species and the E. coli node. Patristic distance describes the evolutionary distance between two species or taxa. The patristic distance can be calculated by summing the branch lengths of a phylogenetic tree that connect two species. A lower patristic distance indicates that two species are closer together on the phylogenetic tree compared to two species with a higher patristic distance. As used herein, smaller patristic distances indicate fewer evolutionary changes have occurred between the two species. The performance of this regressor was evaluated using the c-statistic.

[0086] Leave-one-group-out (LOGO) experiments

[0087] The capability of these machine learning models for the task of identifying novel melt curves, e.g., those belonging to species unrepresented in the training set, were evaluated using a leave-one-group-out experimental design. For each of our ten bacterial species, the respective melt curves were held out and split the curves of the remaining nine species into training and test sets (e.g., an 80:20 ratio). Then, the held-out species' curves were added to the test set. The machine learning model was trained on the training set and tested on its ability to recognize the curves of the held-out species as novel within the modified test set. This process was repeated for each of the ten species. The schematic of this approach is presented in FIG. 1 above. To measure the model’s efficacy in novelty detection, the Youden’s index was used.

[0088] Threshold selection

[0089] The threshold for novelty identification on a single LOGO experiment was set to the Youden Index. However, to practically identify novel organisms, the threshold should apply across all ten LOGO experiments. Thus, a 'practical' threshold was calculated by accumulating all the LOGO experiments and determining the Youden Index from the combined ROC curve, the performance of each LOGO experiment was assessed using this combined 'practical' threshold.

[0090] Threshold validation with synthetic melt curves

[0091] In-silico melt curves were generated using uMelt for 50 clinically relevant bacterial pathogens, including category A and B biothreat agents and their surrogates from Table 1. Real dHRM noise was added to the synthetic melt curves to more realistically capture sample variation as described. 100 melt curves per species were created, with a unique noise residual applied to each individual melt curve.Table 1 : clinically relevant bacterial pathogens used to generate in-silico melt curves.

[0092] Preprocessing

[0093] FIGS. 2A-2B and FIGS. 8A-8C show the results of background subtraction on the experimental melt curves. FIG. 2A is a line charts showing the experimentally-obtained dHRM melt curves. FIG. 2B is a line chart showing ten examples of synthesized melt curves using a combination of uMelts and real dHRM melt curve noise. FIGS. 8A-8C show line charts showing steps of the data preparation workflow. A linear background fit was subtracted from the original melt curves such that the experimental melt curves resemble synthetic ones. Shown at the top of FIG. 8A shows a line chart showing an example original melt curve, a linear fit to the melt curve, and the melt curve aligned to the x-axis. The aligned melt curve is the original melt curve following subtraction of the linear fit. FIG. 8B shows ten original melt curves corresponding to the bacterial species listed in the key. FIG. 8C shows8 is the experimental curves after subtraction of the linear fit representing the background signal.

[0094] FIGS. 9A-9D show line charts showing an overview of experimentally obtained and synthesized melt curves. FIG. 9A is a line chart showing the full melt curves for all synthetic uMelt curves. FIG. 9B is a line chart showing is ten original melt curves corresponding to the bacterial species listed in the key. FIG. 9C is a line chart showing the short version of the synthetic uMelt curves. FIG. 9D is a line chart showing the short version of the ten original melt curves of FIG. 9B.

[0095] For the experimental curves (FIG. 9B and FIG. 9D), the mean curve is shown. For the synthetic uMelts curves (FIG. 9A and FIG. 9C), the original curve is shown (e.g., without applying dHRM noise). These depictions indicate similar curve characteristics as the noise residuals to the mean (e.g., from the experimental data) are added to the uMelts.

[0096] FIGS. 2A and 2B compare the change in fluorescent signal (dF) versus change in temperature (dT) on the x-axis to temperature (°C) on the y-axis. ‘Full melt’refers to using the entire length of the melt curve and ‘short melt’ refers to a shorter window around the melt peak location. FIG. 10 is line charts showing an overview of the synthetically created melt curves for all 50 pathogens.

[0097] Each line chart depicts the residuals to the mean from each experimental melt curve are applied to the u Melts. Background noise was scaled to the ratio of the peak heights (e.g., the peak value of the experimental curves divided by the uMelt peak) and shifted (e.g., the position peak of the uMelt curve is subtracted from the position of the peak of the experimental curves). The noise is considered to be in a similar position relative to the peak. No residual is used twice which means all the curves shown are unique. Each uMelt curve used ten residuals from each of the experimental melts, which results in 100 curves per synthetic uMelt species.

[0098] As shown in FIGS. 2A and 2B, the background subtraction effectively aligns the curves with the x-axis. Further, the simulated curves show similar characteristics to the experimental observations.

[0099] Classification and Regression performance

[0100] The machine learning methods show similar classification accuracy, which is summarized in Table 2.Table 2: ML methods overview and classification results.

[0101] Table 2 shows calculated c-statistics when each ML model (e.g., Logistic regression, Gaussian Naive Bayes, etc.) was utilized on the ‘full melt’ data compared to the ‘short melt’ data. Higher c-statistics indicate higher accuracy when determining novel bacterial genomes.

[0102] No significant difference was observed between the 'full melt' and 'short melt' classification results, which implies there might not be any additional information in the tail of the melt curve, e.g., the portion of the melt curves to the left of the vertical dashed line in FIGS. 2A and 2B. FIG. 13 is a line chart showing the correlation between melt curve distance (e.g., defined as the average pointwise distance from the average E.coli curve) and patristic distance from E. Coli. The correlation between melt curve distance (defined as the average pointwise distance from the average E. coli curve) and patristic distance from E. coli was determined. Modest correlation was observed with a couple of distinct outliers. The performance of a select ML method (RandF, n=S00) to quantify the patristic distance is listed in Table 3. Strong performance was observed (c-statistic > 0.96) for this regression task indicating that the derived features enable quantification of the genotypic differences between organisms.Table 3. C-statistics for patristic distance regression using random forest with 500 trees.

[0103] Leave-one-group-out (LOGO) experiments and threshold selection

[0104] FIGS. 3A and 3B show the results of one LOGO experiment for one ML method (RandF, n=S00). FIG. 3A is a boxplot of the classification probabilities of each of the ten species. The tested species is shown on the x-axis and the classification probabilities is shown on the y-axis. The threshold value determined by the Youden index is shown as a horizontal dashed line at about 0.85. In this experiment C. koseri was the left-out species, which means it was left out of the training set and added to the test set. This experiment was repeated for each of the ten species, and an optimal threshold was found for each of them. This experiment was repeated for all ML methods (method shown here is RandF (n=500)).

[0105] FIG. 3B is a line chart showing a ROC curve that was used to find the threshold value. Youden's Index is chosen as the threshold value, e.g., the point on the ROC curve where sensitivity + specificity - 1 is maximized. FIG. 3B is a line chart comparing sensitivity (TPR) to specificity (FPR). The threshold value was found by plotting the ROC curve for the left-out species (e.g., FIG. 3B) and selecting the Youden index. The threshold value is shown at the upper left corner of the ROC curve as a cross symbol. FIGS. 4 and 5 summarize the results of the accumulated LOGO experiments. The accumulated ROC curve for all 10 species is shown in FIG. 4A. FIG. 4A is a ROC curve for all ten LOGO experiments accumulated with RandF (n=500). The threshold value (shown as a cross symbol) is found by determining the Youden's Index. This is termed the 'practical' threshold value as one threshold has to be chosen (e.g., as opposed to a separate threshold for each LOGO experiment) when further validating the model on unseen 'novel' melt curves. The practical threshold value was the threshold value for all ten LOGO experiments combined.

[0106] The ROC curves of the 10 individual LOGO experiments are shown in FIG. 4B. An enlarged portion of the line chart showing the threshold values for eachof the traces is shown inset. A table showing the bacterial species associated with each ROC curve is shown to the left of FIG. 4B. Choosing a practical threshold value implies that each LOGO experiment individually will be performing at a suboptimal threshold, which translates to a suboptimal operating point on the ROC curve.

[0107] The performance, as measured by Youden's index is shown as a function of classification method in FIG. 5. FIG. 5 shows a summary of LOGO novelty detection results. FIG. 5 is a bar chart comparing the determined Youden’s Index for each of the tested ML models. A higher Youden’s Index indicates a higher novel bacterial genome identification probability. Each ML model has four bars. From left to right, the bars correspond to the Youden’s Index determined using the optimum threshold value using the full melt data, the Youden’s Index determined using the practical threshold value using the full melt data, the Youden’s Index determined using the optimum threshold value using the short melt data, and the Youden’s Index determined using the practical threshold value using the short melt data.

[0108] FIG. 5 shows the novelty detection performance across ten species measured by Youden's index as a function of classification method. Optimal threshold value means selecting the best threshold for each leave-one-species-out experiment. Practical threshold value means selecting one threshold and applying it to all leave- one-species-out experiments. Each bar shows the average performance across ten LOGO experiments. 'Optimal thresholds' means selecting the best threshold for each LOGO experiment individually. 'Practical threshold' means selecting the optimal threshold for the accumulated LOGO experiments and applying it to all LOGO experiments separately. Random Forest outperformed the other methods, but Neural Nets and SVMs still perform relatively well.

[0109] Threshold validation and novelty detection

[0110] FIGS. 6A-6D shows boxplots and ROC curves used to validate the previously obtained 'practical' threshold using the signature features (FIGS. 6A, 6B) as well as the raw melt curves (FIGS. 6C, 6D). The ML method shown in FIG. 6 is RandF (n = 100), which was one of the best performing methods. Similar figures for all other methods are shown in FIGS. 11A-11J. FIGS. 11A-11J show multiple bar charts depicting the probability of a bacterial species to be known or novel according to a classification algorithm, listed at the top of each bar chart. The classification algorithms include a Gaussian Naive Bayes (Gaussian NB) classification algorithm, aLogisticRegression classification algorithm, a neural net activation classification algorithm (e.g., a tanh, a logistic, a rectified linear unit (relu), or an identity neural net activation classification algorithm), a support vector machine (SVM) kernel classification algorithm (e.g., a linear, or a radial basis function (rbf) SVM kernel classification algorithm), or a random forest (RandF) classification algorithm (e.g., a RandF classification algorithm with 100 decision trees (n=100), or 500 decision trees (n=500)). FIGS. 11A-1 1J include a line chart for each bar chart, each line chart comparing the true positive rate to the false positive rate for the respective classification algorithm.

[0111] FIGS. 6A and 6C show boxplots for a number of bacterial species indicating the probability of a novel bacterial genomic prediction. Species having a mean probability value below the practical threshold (lower horizontal dashed line) and / or the optimal threshold (upper horizontal dashed line) are determined to be a novel bacterial genomic detection compared to the primary ten species.

[0112] When the practical threshold is close to the optimal threshold, the practical operating point on the ROC curve (Figure 6B) will be close to the optimal one (Youden's index). When this is the case, it confirms that the proposed method for obtaining a practical threshold for novelty detection, is indeed valid. Both feature sets (e.g., signatures and the raw melt curves) achieve strong discriminative performance of novel organisms based on the model score.

[0113] The use of signature features enables an overall slight improvement compared to raw melt covers only but notably results in reduced novelty detection of T. pallidum.

[0114] Figure 7A is a bar chart comparing Youden’s Index for each of the ML methods used in the analysis and shows an overview of the results for all M L methods. The average difference between the optimal and practically attained Youden index across the ten ML methods is just 0.19 with a standard deviation of 0.025. This serves as a confirmation of the threshold selection process using the accumulated LOGO experiments. Random Forest and SVM (RBF) perform the best, with the Neural Nets a close third.

[0115] Further improvements

[0116] It was observed that the classification probabilities for the E. coli group of melt curves are more spread out (see FIG. 6A) and might be an outlier groupcompared to the other species. This was apparent for multiple ML methods and was confirmed with one-tailed t-tests (e.g., for the short melt curves: P < 0.01 for Logistic Regression, Gaussian NB, SVM (RBF and linear), Neural Net (identity and relu) and RandF (n= 100 and n=500)). As a result of this, all steps were run again, but this time leaving out E. coli, to see if the novelty detection method could be further optimized. Figure 7B is a bar chart comparing Youden’s Index for each of the ML methods and shows that results do indeed improve when leaving E. coli out. Random Forest and SVM (RBF) were the top performers, and those results are further summarized in Table 4. The best performance achieved was a Youden index of 0.93, corresponding to a specificity of 0.97 and sensitivity of 0.96.Table 4 - Overview of Best Results

[0117] It was also investigated whether calibrating the probabilities using scilci. t-learn' s 'CalibratedClassifierCV function would improve the outcome. Both the 'sigmoid' method, which corresponds to Platt's method (i.e., a logistic regression model) or the 'isotonic' method, which is a non-parametric approach, were tested. Results are summarized in FIGS. 12A-12D. FIGS. 12A-12D four bar charts comparing the Youden’s Index value for each of classification algorithms on the x-axis. Three bars indication no probability calibration, a sigmoid probability calibration, or an isotonic probability calibration are shown. FIGS. 12A and 12C show results using E. coli. FIGS. 12B and 12D show results using dropped E. coli. The left column of charts show results using the full melt curves and the right column of charts show results using the short melt curves. Calibrating probabilities improves novelty detection for Gaussian Naive Bayes and for Logistic Regression. None of the calibrated methods outperform the best results (e.g., SVM, Neural Net, RandF) as outlined in Figure 7 and Table 4.

[0118] DISCUSSION

[0119] This example demonstrated the utility of time-series classification algorithms in resolving multiple bacterial organisms melt curves, and in identifying previously unknown (novel) melt curves that are not represented in the database. The large amount of dPCR chip generated melt curve data enabled the development of machine learning classifiers and novelty detection algorithms. The most widely used ML methods in HRM analysis were selected and shown that they are all able to classify the dHRM database with very high accuracy. Interestingly, some outperform others when it comes to novelty detection. It was found that that Neural Nets, SVMs, and Random Forest outperform the other ML methods, even after calibrating the probabilities. Random forests utilizing features extracted from a time-series have been shown to perform well on time-series classification tasks.

[0120] Here, it has been shown that its well-calibrated probabilities are also particularly useful for conducting HRM novelty detection. The performance of the approach was improved with the removal of data for the outlier species E. coli. One reason for the lower performance of E. coli compared to the other species could be that is has the lowest number of melt curves available which results in a smaller amount of training data available for the ML methods. It is not expected that E. coli inherently has more heterogeneity in its sequence compared to other species. Melt curve shape variance might be another contributor to its outlier behavior as it has the third most variance in shape (from the ten species) as measured by dynamic time warping (DTW).

[0121] No major differences were found between using the full length and short version of the melt curves, although for most methods the short version does outperform the full length, showing that there might not be any additional information in the low-temperature tail of the curve, and including it might even confound the novelty detection performance.

[0122] This approach provides an effective way to incorporate novelty detection within a large machine learning framework. In conclusion, advances in machine learning and 'big data' generation are opening up more opportunities for the advancement of HRM, which due to its speed, low cost, and simplicity was already attractive. The opportunity to use HRM as a discovery tool as well as profiling technology will further advance HRM technology towards its application in research and clinical diagnostics.

Claims

CLAIMSWe claim:1 . A method of determining a presence of a target analyte, the method comprising: receiving data representing at least one melt curve representing denaturation of the target analyte; generating at least one extracted feature for the at least one melt curve, thus generating at least one extracted features; providing the at least one extracted feature to a machine learning model trained to determine whether the target analyte is present in a set of known target analytes using the at least one extracted feature; and receiving, from the machine learning model, a prediction value for the at least one extracted feature of the set of extracted features wherein each prediction value is indicative of a probability of whether the at least one extracted feature is present in a training feature set representing the set of known target analytes on which the machine learning model was trained.

2. The method of claim 1 , wherein the at least one melt curve comprises at least one melt curve for each of a set of target analytes in a biological sample.

3. The method of claim 1 or 2, wherein the biological sample is a blood sample, a urine sample, a saliva sample, a sweat sample, a hair sample, a skin tissue sample, a nail sample, a sputum sample, a bone marrow sample, a cerebrospinal fluid (CSF) sample, a tear sample, a breath sample, a feces sample, a semen sample, a vaginal secretion sample, an amniotic fluid sample, or a placental tissue sample.

4. The method of claim 1 or 2, wherein each target analyte of the set of target analytes represents a nucleotide sequence present in the biological sample.

5. The method of claim 4, wherein the nucleotide sequence is a DNA sequence or an RNA sequence.

6. The method of any one of claims 1-5, comprising performing melt curve analysis to generate the at least one melt curve, wherein the melt curve analysis is dHRM melt analysis.

7. The method of any one of claims 1-6, comprising comparing the prediction value to a presence threshold value representing a degree of confidence to which the extracted feature is present in the training feature set, and, if the prediction value does not satisfy the presence threshold value, determining that the extracted feature is not present in the training feature set.

8. The method of claim 7, wherein the threshold value is determined using a Youden’s Index value.

9. The method of any one of claims 1-8, where generating the at least one extracted feature comprises processing the data set through a feature extraction algorithm.

10. The method of claim 9, wherein the feature extraction algorithm is a nonparametric feature extraction algorithm.11 . The method of claim 9, wherein the feature extraction algorithm is based on a rolling window transformation having a kernel size and a stride less than a sequence length, and a signature depth in a range from 2 to 20.

12. The method of claim 9, wherein the feature extraction algorithm is time-shift invariant.

13. The method of any one of claims 1-12, further comprising training the machine learning model by providing to an untrained machine learning model the training feature set, wherein the training feature set comprises a set of melt curves representing denaturation of a known target analyte.

14. The method of any one of claims 1-13, wherein receiving data representing at least one melt curve comprises receiving data representing at least one raw melt curve.

15. The method of claim 14, further comprising preprocessing the data representing at least one raw melt curve.

16. The method of claim 15, wherein the preprocessing comprises preprocessing using a background subtraction step, a noise reduction step, or both.

17. The method of any one of claims 1-16, wherein the trained machine learning model is a logistic regression model, a naive bayes (NB) model, a support vector machine (SVM) model, a neural network model, or a random forest (RandF) model.

18. The method of any one of claims 1-17, wherein the target analyte comprises a set of target analytes comprising a set of nucleotide sequences.

19. A computer program product comprising a non-transitory computer readable medium encoded with instructions to cause one or more processors to: receive data representing at least one melt curve representing denaturation of a target analyte; generate at least one extracted feature for the at least one melt curve, thus generating at least one extracted features; provide the at least one extracted feature to a machine learning model trained to determine whether the target analyte is present in a set of known target analytes using the at least one extracted feature; and receive, from the machine learning model, a prediction value for the at least one extracted feature of the set of extracted features wherein each prediction value is indicative of a probability of whether the at least one extracted feature is present in a training feature set representing the set of known target analytes on which the machine learning model was trained.

Images