Systems and methods for detecting and classifying pre-analytical errors in clinical laboratory diagnostics

Abstract

In some embodiments, a method of creating a machine-learning-based autoverification system for clinical laboratory diagnostics includes obtaining a biomarker training dataset, the biomarker training dataset including biomarker values for a plurality of biomarkers. The method also includes training an autoencoder employing the biomarker training dataset as both an input and a target for the autoencoder to create a trained autoencoder. The method further includes deploying the trained autoencoder in a hospital or laboratory. Numerous other embodiments are provided.

Description

SYSTEMS AND METHODS FOR DETECTING AND CLASSIFYING PRE- ANALYTICAL ERRORS IN CLINICAL LABORATORY DIAGNOSTICSCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims benefit under 35 USC § 119(e) of US Provisional Application No. 63 / 579,060, filed August 28, 2023. The entire contents of the abovereferenced patent application(s) are hereby expressly incorporated herein by reference.FIELD

[0002] The present application relates to medical testing and more particularly to systems and methods for detecting and classifying pre-analytical errors.BACKGROUND

[0003] Clinical laboratories play a crucial role in the diagnosis and treatment of diseases by analyzing patient samples and providing accurate and reliable test results. However, the process of analyzing these samples is not without its challenges. One such challenge is the detection of deviations from expected patient reference intervals, which can arise from a multitude of factors including pre-analytical errors.

[0004] Pre-analytical errors refer to any mistakes or inconsistencies that occur before a sample is even analyzed. These errors can include issues with sample collection, handling, and storage, as well as inadequate labeling or documentation. Pre-analytical errors can have a significant impact on the accuracy and reliability of test results, making it essential for clinical laboratories to detect and address these errors before they cause any harm.

[0005] To address this challenge, clinical laboratories have developed a method called autoverification of laboratory results. Autoverification involves using rule-based software algorithms to automatically review and verify the accuracy of laboratory test results before they are reported to patients and healthcare providers. This helps to ensure that only accurate and reliable results are reported, reducing the risk of incorrect or misleading information being provided.

[0006] Despite the benefits of autoverification, there are still several challenges associated with this method. One such challenge is the cost and scarcity of qualified laboratory personnel who are needed to manually review any results that do not passthe autoverification criteria. These personnel are highly trained and experienced professionals in short supply, making it difficult for clinical laboratories to maintain a sufficient workforce to meet the demands of their patients.

[0007] Furthermore, the manual review process can be time-consuming and slow, which can further delay the availability of results to patients and healthcare providers. This delay can have serious consequences, particularly in emergency situations where timely diagnosis and treatment are critical.

[0008] Therefore, there is a need for improved methods and apparatus for detecting and addressing pre-analytical errors in clinical laboratories that is more efficient, accurate, and cost-effective than current methods.SUMMARY

[0009] In some embodiments, a method of creating a machine-learning-based autoverification system for clinical laboratory diagnostics includes obtaining a biomarker training dataset, the biomarker training dataset including biomarker values for a plurality of biomarkers; training an autoencoder employing the biomarker training dataset as both an input and a target for the autoencoder to create a trained autoencoder; and deploying the trained autoencoder in a hospital or laboratory.

[0010] In some embodiments, a method of creating a machine-learning-based autoverification system for clinical laboratory diagnostics includes obtaining a first list of biomarkers for autoverification; obtaining a first training dataset for the first list of biomarkers, the first training dataset including biomarker values for each biomarker on the first list of biomarkers; training a first autoencoder employing the first training dataset as both an input and a target for the first autoencoder to create a first trained autoencoder; obtaining a second list of biomarkers for autoverification; obtaining a second training dataset for the second list of biomarkers, the second training dataset including biomarker values for each biomarker on the second list of biomarkers; training a second autoencoder employing the second training dataset as both an input and a target for the second autoencoder to create a second trained autoencoder; and deploying the first trained autoencoder and the second trained autoencoder.

[0011] In some embodiments, a machine-learning-based autoverification system for clinical laboratory diagnostics includes a processor; a memory coupled to the processor, the memory including an autoencoder trained on a biomarker training dataset by using the biomarker training dataset as both an input and a target for theautoencoder; and computer program instructions stored in the memory that, when executed by the processor, cause the processor to receive one or more biomarkers measured for a patient; input the measured biomarkers to the autoencoder to generate output biomarkers; and based on a comparison of the measured biomarkers input to the autoencoder and the output biomarkers of the autoencoder, determine whether the measured biomarkers are valid.

[0012] In some embodiments, a method of autoverification of clinical laboratory diagnostics includes receiving one or more biomarkers measured for a patient; inputting the measured biomarkers to an autoencoder to generate output biomarkers, wherein the autoencoder is trained on a biomarker training dataset by using the biomarker training dataset as both an input and a target for the autoencoder; and based on a comparison of the measured biomarkers input to the autoencoder and the output biomarkers of the autoencoder, determining whether the measured biomarkers are valid.

[0013] Other features and aspects of the present invention will become more fully apparent from the following detailed description, the appended claims, and the accompanying drawings.BRIEF DESCRIPTION OF THE DRAWINGS

[0014] FIG. 1A illustrates an example flow diagram of a method of training and deploying a machine-learning-based autoverification system for clinical laboratory diagnostics in accordance with embodiments provided herein.

[0015] FIG. 1 B illustrates an example computer in which the method of FIG. 1A may be implemented in accordance with one or more embodiments.

[0016] FIG. 2 illustrates a flowchart of a method of creating a machine-learning- based autoverification system for clinical laboratory diagnostics in accordance with one or more embodiments.

[0017] FIG. 3A illustrates an example machine-learning model that may be trained using a biomarker training dataset in accordance with embodiments provided herein.

[0018] FIG. 3B illustrates an example autoverification system that employs the ML model of FIG. 3A after training in accordance with embodiments provided herein.

[0019] FIG. 4 illustrates a flowchart of another method of creating a machinelearning-based autoverification system for clinical laboratory diagnostics in accordance with one or more embodiments.

[0020] FIG. 5. illustrates a flowchart of a method of autoverification of clinical laboratory diagnostics in accordance with one or more embodiments.

[0021] FIG. 6A illustrates a first machine-learning-based autoverification system provided in accordance with one or more embodiments.

[0022] FIG. 6B illustrates a second machine-learning-based autoverification system provided in accordance with one or more embodiments.

[0023] FIG. 7 illustrates an example graph of mean square error between measured biomarkers and predicted (reconstructed) biomarkers generated by an ML model (in response to the measured biomarkers) in accordance with one or more embodiments.

[0024] FIGS. 8A and 8B illustrate planar distributions of the latent space representations of measured biomarkers before (FIG. 8A) and after (FIG. 8B) application of a clustering model in accordance with one or more embodiments.

[0025] FIG. 9 illustrates a flowchart of a method of autoverifying measured biomarkers in accordance with one or more embodiments.

[0026] FIG. 10 illustrates a flowchart of a method of analyzing measured biomarkers subjected to IV contamination in accordance with one or more embodiments.DETAILED DESCRIPTION

[0027] Independent of the grammatical term usage, individuals with male, female or other gender identities are included within the term.

[0028] Embodiments provided herein provide systems and methods that may more efficiently, accurately, and cost-effectively detect and identify pre-analytical errors in clinical laboratories. In some embodiments, deep neural network (DNN) architectures based on one or more autoencoders in outlier detection mode may be utilized within one or more machine-learning-based autoverification systems. Other machinelearning models may be employed for outlier detection (e.g., other dimensionality reduction algorithms). In one or more embodiments, an autoencoder model may be trained on a set of valid laboratory results (e.g., biomarkers) from a clinical laboratory instrument, middleware, or laboratory information system (LIS). Use of an autoencoder may also be seen as a dimensionality reduction method, the autoencoder learning an internal representation of data in “latent space.” Only important features pertinent to a dataset (e.g., a plurality of biomarkers) are encoded in latent space whereas noise and outliers are suppressed. In a prediction (outlier detection) scenario, reconstructionerror (e.g., measured biomarkers versus autoencoder reconstructed biomarkers) is calculated. High reconstruction error is an indication of one or more outliers (e.g., an invalid sample). Once an outlier is identified, a second step may be performed to try to cluster outliers with respect to their root causes utilizing, in some embodiments, machine learning clustering methods.

[0029] In one or more embodiments, an autoencoder-based autoverification system of a hospital or laboratory, or other machine-learning-based autoverification system, may be trained on a dataset from the hospital or laboratory using the autoverification system. Such a dataset represents the hospital’s or laboratory’s target population (e.g., eliminating cohort risk of a machine-learning model).

[0030] In some embodiments, multiple models (e.g., multiple autoencoders) may be trained for different types of blood or other tests or biomarker panels. For example, a different autoencoder may be trained for each of a lipid panel, a complete blood count (CBC) panel, a basic metabolic panel (BMP), a comprehensive metabolic panel (CMP), a thyroid panel, an immunoassay panel, a urine panel, a combination of multiple panels, etc.

[0031] In one or more embodiments, a generic model (e.g., utilizing all available input biomarkers) and multiple specific models for different failure models (e.g., utilizing only a subset of relevant biomarkers for each failure model) may be trained (e.g., automatically). As an example, failure models may be developed for detecting IV fluid contamination or other error sources of interest.

[0032] Note that, in general, the systems and methods provided herein may be generic and rolled out to any clinical laboratory. That is, in some embodiments, an ML model (e.g., autoencoder) may be trained on biomarkers for multiple tests and / or trained on generic biomarker data (e.g., from multiple hospitals, laboratories, healthcare data providers, etc.).

[0033] In some embodiments, by creating individual models (e.g., ML-models such as autoencoders trained on specific sets of biomarkers), a fact-based model results report may be produced (e.g., prevalence of disease state x, y, and preanalytical error z have been observed for a specific biomarker pattern). Traditionally, model results are reported as an absence / presence prediction with some sort of confidence bounds, such as a 70% chance that a patient has colon cancer. The applicability of these confidence bounds greatly depends on a factor called “cohort risk,” which is a theoretical measure of differences in a model training population and a modelprediction population (especially regarding the mathematical relationships between the dependent variable and each independent variable). Due to high variability in biological and ecological systems, cohort risk is often prohibitively high, leading to nonperforming models. Fact-based results reporting highlights the most relevant information so that a practitioner may cross reference with additional information not directly captured in biomarkers, such as symptoms.

[0034] As stated above, in some embodiments provided herein, a two-step machine-learning approach may be employed (e.g., error detection followed by error classification). A two-step machine-learning approach may allow for superior pro- analytical error detection and classification with respect to probable root causes of errors. The general approach (e.g., training a model on a general biomarker dataset from multiple hospitals or laboratories) may be deployed in any clinical laboratory. Currently in vitro diagnostic (IVD) and laboratory information system (LIS) manufacturers provide software solutions that allow laboratories to set up rules for autoverification, but those rules must be developed, monitored, and updated by each laboratory individually. Such rules are complex and time consuming to design, implement, and validate. The machine-learning-based autoverification systems and methods described herein are customizable, easily implemented, and outperform rule-based approaches.

[0035] These and other embodiments of the invention are described below with reference to FIGS. 1A-10.

[0036] FIG. 1A illustrates an example flow diagram 100a of a method of training and deploying a machine-learning-based autoverification system for clinical laboratory diagnostics in accordance with embodiments provided herein. With reference to FIG. 1A, and as described in further detail below, the method of flow diagram 100a includes obtaining a biomarker dataset 102 (e.g., from one or more hospitals, one or more laboratories, a healthcare data provider, etc.). The biomarker dataset 102 is then employed to generate a training dataset (biomarker training dataset 104). In some embodiments, the biomarker dataset 102 is examined and biomarkers of interest are identified. For example, if autoverification is to be performed on blood panels such as a lipid panel, a CBC panel, a BMP, a CMP, a thyroid panel, or an immunoassay panel, biomarkers relevant to one or more of these panels may be selected from the biomarker dataset 102 for inclusion in the biomarker training dataset 104. A plurality of the biomarker values (e.g., 3 or more, 10 or more, etc.) may be employed for each biomarker within the biomarker training dataset 104. Thereafter, the biomarker trainingdataset 104 may be employed to train a machine-learning (ML) model 106. As described below, in some embodiments, the ML model 106 may include one or more autoencoders, although other ML models may be employed. In some embodiments, the entire biomarker dataset 102 may be used as the biomarker training dataset 104.

[0037] The trained ML model 106 may be used within a clinical laboratory setting as a deployed ML model 108 to predict whether test results on a patient sample are valid (e.g., autoverification) or whether the results should be manually reviewed, the test should be repeated on the patient sample, and / or a new patient sample should be obtained. As described further below, a prediction model, such as a clustering algorithm, may be employed to facilitate identification of sources of error associated with testing (e.g., to identify pre-analytical errors, such as errors during sample collection, handling, and / or storage, as well as inadequate labeling or documentation).

[0038] FIG. 1 B illustrates an example computer 120 in which the method of FIG. 1A may be implemented in accordance with one or more embodiments. With reference to FIG. 1 B, computer 120 includes a processor 122 coupled to a memory 124. Memory 124 may include biomarker dataset 102, biomarker training dataset 104, and ML model 106. Memory 124 may also include one or more programs 126 for carrying out the methods described herein when executed by processor 122, such as examining biomarker dataset 102 for biomarkers of interest, creating biomarker training dataset 104 based on biomarker dataset 102, and the like. In some embodiments, processor 122, executing one or more of programs 126, may train ML model 106 based on biomarker training dataset 104 (e.g., generating trained ML model 108 (FIG. 1A) that may be deployed in a hospital, laboratory, etc.). Memory 124 may include multiple memory units and / or types of memory. In some embodiments, all or a portion of memory 124 may be external to and / or remote from computer 120. Additionally, in some embodiments, multiple processors may be employed.

[0039] FIG. 2 illustrates a flowchart of a method 200 of creating a machinelearning-based autoverification system for clinical laboratory diagnostics in accordance with one or more embodiments. With reference to FIG. 2, method 200 includes, in block 202, obtaining a biomarker training dataset, the biomarker training dataset including biomarker values for a plurality of biomarkers. In some embodiments, the biomarker training dataset may be obtained from a hospital, a laboratory, a healthcare data provider such as Dandelion Health, Inc. of New York, NY, Prognos Health, Inc. of New York, NY, or the like, or created based on a biomarkerdataset obtained from a hospital, laboratory, healthcare data provider, etc. For example, a biomarker training dataset may be formed from a subset of biomarkers in a larger biomarker dataset (e.g., biomarkers relevant to a CBC panel, a BMP, a CMP, a lipid panel, an immunoassay panel, a urine panel, another biomarker panel, combinations of biomarker panels, etc., may be selected from a larger dataset and used to form a biomarker training dataset). FIGS. 1A and 1 B illustrate an example biomarker training dataset 104 formed from a biomarker dataset 102.

[0040] In some embodiments, the biomarker training dataset may contain biomarkers determined for at least one of a plurality of hospitals and a plurality of laboratories. In other embodiments, the biomarker training dataset may contain only biomarkers determined for a specific hospital (e.g., biomarkers determined for at least one of an emergency room, an in-patient clinic, an outpatient clinic, a doctor’s office, and / or a laboratory affiliated with the specific hospital).

[0041] In instances in which the biomarker training dataset includes hospital or laboratory data, in some embodiments, the biomarkers present within the biomarker training dataset may be obtained from a laboratory information system (e.g., validated biomarkers, such as biomarkers validated by a rules-based methodology). In other embodiments, the biomarker training dataset may include unvalidated biomarkers (e.g., from middleware).

[0042] In one or more embodiments, obtaining the biomarker training dataset may include obtaining a list of biomarkers to autoverify (e.g., from a hospital or laboratory) and creating the biomarker training dataset based on the list of biomarkers. For example, a hospital or laboratory may want to autoverify biomarkers related to one or more blood panels. The hospital or laboratory may provide a list of the particular biomarkers it plans to autoverify. Based on the provided list of biomarkers, a biomarker training dataset may be generated from a more general biomarker dataset. For example, a biomarker dataset may be obtained from the hospital or laboratory and the biomarker training dataset may be created based on the biomarker dataset and the list of biomarkers. Alternatively, the biomarker dataset may be provided by another hospital or laboratory, a plurality of hospitals or laboratories, etc.

[0043] In some embodiments, the list of biomarkers may include a list of biomarkers in a biomarker panel such as biomarkers from at least one of a lipid panel, a CBC panel, a BMP, a CMP, a thyroid panel, a urine panel, and an immunoassay panel. In another example, the biomarker panel may include routine diagnostic tests withcommon biomarkers measured during yearly physicals, routine health checks, or the like (e.g., biomarkers from a lipid panel, a CBC panel, and a CMP).

[0044] In some embodiments, a biomarker training dataset may include 3 or more biomarker values for each biomarker in the training dataset. Other numbers of biomarker values may be employed (e.g., 4, 5, 10, 20, 50, or more).

[0045] After the biomarker training dataset has been obtained (e.g., biomarker training dataset 104 of FIGS. 1A and 1 B), in block 204, method 200 includes training an autoencoder employing the biomarker training dataset as both an input and a target for the autoencoder (thereby creating a trained autoencoder). It will be understood that other ML-based models may be employed, such as another dimensionality-reduction algorithm (e.g., principal component analysis, isomap, t-distributed stochastic neighbor embedding, linear discrimination analysis, uniform manifold approximation and projection, etc.).

[0046] FIG. 3A illustrates an example machine-learning model, referred to as initial ML model 300, that may be trained using a biomarker training dataset (e.g., biomarker training dataset 104 of FIGS. 1A and 1 B) in accordance with embodiments provided herein. In the example embodiment of FIG. 3A, the initial ML model 300 is an autoencoder that maps biomarkers within the biomarker training dataset to a reduced- dimensionality latent space representation (as described further below). Any suitable autoencoder may be employed such as a vanilla autoencoder, an undercomplete autoencoder, a denoising autoencoder, a sparse autoencoder, a contractive autoencoder, a convolutional autoencoder, a recurrent autoencoder, a variational autoencoder, etc. Other dimensionality-reduction algorithms may be employed.

[0047] In some embodiments, the latent space representation may encode data across different scales. For example, normalizing flows, autoregressive models, variational autoencoders (VAEs), and deep energy-based models are examples for deep generative learning. Furthermore, the latent space may be constrained to be a known parametric distribution (e.g., Gaussian or mixture-of-Gaussian) or a nonparametric distribution, such as with a vector quantized variational autoencoder (VQ- VAE). It will be understood that other ML models may be employed.

[0048] Referring to FIG. 3A, ML model 300 includes an encoder 302 configured to map input biomarkers to a latent space L and a decoder 304 configured to map latent space L into output biomarkers (e.g., reconstructed biomarkers). A biomarker training dataset may be input to the encoder 302 to generate latent features in a latent variableLA in a multidimensional latent space (e.g., generating latent features in lowdimensional space). The features of the latent variable L may be partitioned into multiple groups of features such as W, X, Y, and Z as shown in FIG. 3A. Each of the features may correspond to a specific one of the attributes of the biomarkers of the biomarker training dataset input to the encoder 302. In some embodiments, at least one of the features (e.g., feature Z) may be reserved for an intrinsic property of the biomarker dataset in which variations cannot be controlled by a setup.

[0049] During training, the decoder 304 may use the features of the latent variable LA to reconstruct the biomarkers input to the encoder 302 (e.g., as similar to the input biomarkers as possible). A reconstruction loss module 306 may compare the input biomarkers to the output (reconstructed) biomarkers to determine whether the input and output biomarkers are close to each other. For example, the reconstruction loss module 306 may analyze the attributes of the input biomarkers and the output biomarkers to determine whether the latent variable LA is correct and whether the encoder model (encoder 302) and decoder model (decoder 304) are trained correctly.

[0050] Once the ML model (e.g., an autoencoder in the embodiment of FIG. 2) has been trained using the biomarker training dataset as both an input and target for the ML model, method 200 includes, in block 206, deploying the trained autoencoder in a hospital or laboratory. In some embodiments, the trained autoencoder may be deployed in the same hospital or laboratory that supplied the biomarker dataset used to generate the biomarker training dataset.

[0051] FIG. 3B illustrates an example a utoverifi cation system 350 that employs the ML model 300 of FIG. 3A after training in accordance with embodiments provided herein. With reference to FIG. 3B, the trained ML model 300 (e.g., an autoencoder in the embodiment of FIG. 3B) may be employed for autoverification by inputting measured biomarkers 352 to the trained model 300 to generate output (reconstructed) biomarkers 354. An error calculation module 356 may be employed to determine an amount of reconstruction error between the measured biomarkers 352 input to the trained ML model 300 and the output (reconstructed) biomarkers 354 output from the trained ML model 300. In some embodiments, the reconstruction error may represent one or more mean square error values for biomarkers. High reconstruction error may identify that the patient sample is invalid, while a low reconstruction error may indicate that a patient sample is valid.

[0052] As described further below, in some embodiments, if a sample is determinedto be invalid, a classification model may be employed to predict a source of error associated with the patient sample. In the embodiment of FIG. 3B, the latent space representation (e.g., latent variable LA) may be employed with a clustering model 358 to classify outlier latent space features into clusters associated with pre-analytical errors (e.g., a clinical condition cluster, a contamination cluster, or any other relevant cluster). The clustering model 358 may include one or more clustering algorithms. For example, a clustering algorithm such as k-means clustering or another dimensionality reduction and / or grouping algorithm may be employed. Other example clustering or grouping algorithms that may be employed include t-distributed stochastic neighbor embedding, isomap embedding, locally linear embedding, spectral embedding, multidimensional scaling, latent Dirichlet allocation, or the like.

[0053] In some embodiments, it may be desirable for a hospital or laboratory to employ multiple-machine-learning based autoverfication models trained on different sets of biomarkers. For example, separate autoencoders may be trained for a CBC panel, a BMP, a CMP, a lipid panel, an immunoassay panel, different combinations of biomarker panels, etc.

[0054] FIG. 4 illustrates a flowchart of another method 400 of creating a machinelearning-based autoverification system for clinical laboratory diagnostics in accordance with one or more embodiments. With reference to FIG. 4, method 400 includes, in block 402, obtaining a first list of biomarkers for autoverification; in block 404, obtaining a first training dataset for the first list of biomarkers, the first training dataset including biomarker values for each biomarker on the first list of biomarkers; and, in block 406, training a first autoencoder employing the first training dataset as both an input and a target for the first autoencoder to create a first trained autoencoder. Method 400 further includes, in block 408, obtaining a second list of biomarkers for autoverification; in block 410, obtaining a second training dataset for the second list of biomarkers, the second training dataset including biomarker values for each biomarker on the second list of biomarkers; and in block 412, training a second autoencoder employing the second training dataset as both an input and a target for the second autoencoder to create a second trained autoencoder. Thereafter, method 400 includes, in block 414, deploying the first trained autoencoder and the second trained autoencoder.

[0055] In some embodiments, the first and second lists of biomarkers may be obtained from a hospital or laboratory (e.g., that plans to use the trainedautoencoders). The first and second training datasets may be created from one or more biomarker datasets obtained from the hospital or laboratory planning to use the trained autoencoders or from a different source, such as another hospital or laboratory, a healthcare data provider, etc.

[0056] In one or more embodiments, the first list of biomarkers may include a list of biomarkers in a first biomarker panel and the second list of biomarkers may include a list of biomarkers in a second biomarker panel. For example, the first and second lists may include different sets of biomarkers from at least one of a lipid panel, a CBC panel, a BMP, a CMP, a thyroid panel, a urine panel, an immunoassay panel, etc.

[0057] FIG. 5. illustrates a flowchart of a method 500 of autoverification of clinical laboratory diagnostics in accordance with one or more embodiments. With reference to FIG. 5, method 500 includes, in block 502, receiving one or more biomarkers measured for a patient. For example, a patient sample may be collected from a patient at a hospital, inpatient clinic, outpatient clinic, walk-in laboratory, etc. A test may then be performed to measure one or more biomarkers for the patient sample, such as biomarkers for a CBC panel, a BMP, a CMP, a thyroid panel, a urine panel, an immunoassay panel, another biomarker panel, etc. Thereafter, method 500 includes, in block 504, inputting the measured biomarkers to an autoencoder to generate output biomarkers, wherein the autoencoder is trained on a biomarker training dataset by using the biomarker training dataset as both an input and a target for the autoencoder. As described above, in some embodiments, the biomarker training set used to train the autoencoder may be from the hospital or laboratory in which the autoencoder is deployed. In other embodiments, the biomarker training set may be from one or more different hospitals and / or laboratories.

[0058] Method 500 further includes, in block 506, based on a comparison of the measured biomarkers input to the autoencoder and the output biomarkers of the autoencoder, determining whether the measured biomarkers are valid. In some embodiments, determining whether the measured biomarkers are valid may include determining reconstruction error between the measured biomarkers and the output biomarkers (e.g., at least one mean squared error value or another measure of error based on the measured biomarkers for the patient sample and the output (reconstructed) biomarkers of the autoencoder in response to the measured biomarkers).

[0059] Because the autoencoder is trained using the same biomarkers as both aninput and a target for the autoencoder, the autoencoder will result in a small reconstruction error if the measured biomarkers from a patient sample input to the autoencoder are similar to the biomarkers in the biomarker training dataset. Furthermore, if the biomarker training dataset used to train the autoencoder includes validated biomarkers (e.g., from one or more laboratory information systems), a low reconstruction error between measured biomarkers from a patient sample and the output (reconstructed) biomarkers from the autoencoder in response to the measured biomarkers indicates that the measured biomarkers are valid. Likewise, a high reconstruction error between measured biomarkers from a patient sample and the output biomarkers from the autoencoder in response to the measured biomarkers indicates that the measured biomarkers are invalid (e.g., possibly requiring manual checking, retaking of the patient sample, etc.). In this manner, the trained autoencoder may be used for autoverification of patient biomarkers.

[0060] FIG. 6A illustrates a first machine-learning-based autoverification system 600a provided in accordance with one or more embodiments. With reference to FIG. 6A, the first machine-learning-based autoverification system 600a may be in communication with a first analyzer 602a configured to determine one or more biomarkers of a patient blood sample (e.g., creatinine, BUN, BUN / creatinine ratio, chloride, red blood cell width (RDW), sodium, albumin, hematocrit, platelet count, red or white blood cell counts, neutrophil, lymphocyte, eosinophil, etc.). Resultant biomarkers are fed to a processor 604 and run through a deployed ML model 606 (e.g., a trained autoencoder such as trained ML model 300 of FIG. 3B) stored in a memory 608. ML model 606 inputs measured biomarkers and provides predicted (reconstructed) biomarkers as outputs.

[0061] An error calculation module 610 (labelled error calculator in FIG. 6A) may be employed to determine an amount of reconstruction error between the measured biomarkers input to the ML model 606 and the reconstructed biomarkers output from the ML model 606 (e.g., mean square error values for biomarkers). High reconstruction error may identify that the patient sample is invalid, while low reconstruction error may indicate that the patient sample is valid. FIG. 7 illustrates an example graph of mean square error between measured biomarkers and predicted (reconstructed) biomarkers generated by the ML model 606 (in response to the measured biomarkers) in accordance with one or more embodiments. As shown in FIG. 7, the majority of normal biomarkers have mean square error values below a predefined threshold (e.g.,approximately 0.013 in FIG. 7, although any suitable threshold may be employed). The majority of abnormal biomarkers have mean square error values above the predefined threshold.

[0062] The error calculation module 610 (within the memory 608) may include computer program instructions that, when executed by the processor 604, cause the processor 604 to determine reconstruction error between the measured biomarkers and the output (reconstructed) biomarkers of the trained ML model 606 (e.g., determine at least one mean squared error value based on the measured biomarkers and the output biomarkers of a trained autoencoder).

[0063] A clustering model 612 may be employed to predict a source of error in measured biomarkers of a patient. For example, in some embodiments, the latent space representation of measured biomarkers (e.g., latent variable LA in FIG. 3B) may be employed with the clustering model 612 (e.g., similar to clustering model 358 of FIG. 3B) to classify outlier latent space features into clusters associated with pre- analytical errors (e.g., a clinical condition cluster, a contamination cluster, and / or any other relevant cluster). The clustering model 612 may include one or more clustering algorithms (as described above with regarding to clustering model 358).

[0064] In one or more embodiments, memory 608 may include program(s) 614 having computer program instructions that, when executed by the processor 604, cause the processor 604 to employ the clustering model 612 to predict a cause of a difference between measured biomarkers input to the ML model 606 (e.g., a trained autoencoder) and the resulting output biomarkers of the ML Model 606. For example, the program(s) 614 may use the clustering model 612 to determine a classification of a representation of the measured biomarkers, such as a latent space representation of the measured biomarkers generated by the ML model 606. In some embodiments, the clustering model 612 may cluster the latent space representation of the measured biomarkers into one of a plurality of clusters which classify (e.g., predict) the source of reconstruction error between the measured biomarkers and the output (reconstructed) biomarkers. Example clusters include a clinical condition cluster, a contamination cluster, and / or any other relevant cluster that identifies a source of error.

[0065] FIGS. 8A and 8B illustrate planar distributions of the latent space representations (e.g., features W, X, Y, and Z of FIG. 3B) of measured biomarkers before (FIG. 8A) and after (FIG. 8B) application of the clustering model 612 (FIG. 6A) in accordance with one or more embodiments. As shown in FIGS. 8A and 8B, theclustering model 612 spatially separates the latent features 800 (FIG. 8A) into a plurality of clusters 802, 804, 806, and 808 (FIG. 8B). (More generally, the clusters are separated in feature space.) Each cluster may represent a specific source of pre- analytical error (e.g., a clinical condition, contamination, wrong sample, etc.). Such clusters may be determined, for example, by training the clustering model 612 on a training dataset including known pre-analytical errors.

[0066] Thus, in some embodiments, program(s) 614 stored in memory 608 may include computer program instructions that cause the processor 604 to employ a clustering algorithm to cluster a representation of measured biomarkers into one of a plurality of clusters which may be used to classify (e.g., predict the cause of) the difference (e.g., reconstruction error) between the measured biomarkers and output biomarkers of a trained ML model in response to the measured biomarkers. For example, prediction of the source of error in the measured biomarkers may be determined by identifying the cluster(s) in which the latent representation of the measured biomarkers reside(s) after use of the clustering model 612.

[0067] FIG. 6B illustrates a second machine-learning-based autoverification system 600b provided in accordance with one or more embodiments. With reference to FIG. 6B, the second machine-learning-based autoverification system 600b may be in communication with a first analyzer 602a (and / or a second analyzer 602b or other analyzers not shown) configured to determine one or more biomarkers of a patient sample. Resultant biomarkers are communicated to the processor 604 and run through the first ML model 606 stored in memory 608. For example, the first ML model 606 may be a first autoencoder trained on a biomarker dataset for a first blood panel. The first ML model 606 may be used to autoverify biomarkers measured by the first analyzer 602a (or another analyzer) such as by generating output (reconstructed) biomarkers in response to measured biomarkers input to the first ML model 606. For example, the program(s) 614 executed by the processor 604 may receive biomarkers from the first analyzer 602a (e.g., directly, from medical staff, or via a central computer system), use the first ML model 606 to generate reconstructed biomarkers based on the received biomarkers from the first analyzer 602a, determine the validity of the biomarkers (e.g., using error calculation module 610 as described above), if the biomarkers are invalid, predict a cause (e.g., using clustering model 612 as described above), and / or make recommendations.

[0068] A plurality of uniquely trained ML models 606a-606n (e.g., uniquely trainedautoencoders) may be employed to autoverify different biomarker panels (e.g., using error calculation model 610) and to identify sources of errors in such biomarker panels (e.g., using clustering model 612 or other clustering models).

[0069] FIG. 9 illustrates a flowchart of a method 900 of autoverifying measured biomarkers in accordance with one or more embodiments. With reference to FIG. 9, in block 902, method 900 includes obtaining the reconstruction error between measured biomarkers and output (reconstructed) biomarkers produced by an ML model in response to the measured biomarkers. For example, measured biomarkers may be input to a trained autoencoder, such as a generically-trained or specifically- trained autoencoder, to generate output (reconstructed) biomarkers. The amount of reconstruction error between the input and output biomarkers may then be determined.

[0070] In block 904, method 900 includes determining if the reconstruction error is high (e.g., above a predetermined threshold). If the reconstruction error is not high, the sample is determined to be valid (block 906); otherwise, in block 908, clustering is performed based on the measured biomarkers. For example, a reduced- dimensionality representation of the measured biomarkers (such as a latent space representation generated by an autoencoder) may be clustered using a clustering algorithm (as described above).

[0071] Method 900 includes, in block 910, determining whether the clustering results in a distinct pattern (e.g., if at least one defined cluster is produced). If a distinct pattern is not produced, no determination is made as to the source of error in the sample (block 912); otherwise, if a distinct pattern is observed after clustering, a determination may be made as to which cluster(s) the representation of the measured biomarkers resides, such as a first outlier class (e.g., cluster) representing a sample error due to a clinical condition (block 914), an nth outlier class representing sample contamination (block 916), and / or the like. Any number of outlier classes may be employed.

[0072] FIG. 10 illustrates a flowchart of a method 1000 of analyzing measured biomarkers subjected to IV contamination in accordance with one or more embodiments. With reference to FIG. 10, in block 1002, method 1000 includes obtaining the reconstruction error between measured biomarkers and output (reconstructed) biomarkers produced by an ML model in response to the measured biomarkers. For example, measured biomarkers may be input to a trainedautoencoder, such as a generically-trained or specifically-trained autoencoder, to generate output (reconstructed) biomarkers. The amount of reconstruction error between the input and output biomarkers may then be determined.

[0073] In block 1004, method 1000 includes determining if the reconstruction error is high (e.g., above a predetermined threshold). If the reconstruction error is not high, the sample is determined to be valid (block 1006); otherwise, in block 1008, clustering is performed based on the measured biomarkers. For example, a reduced- dimensionality representation of the measured biomarkers (such as a latent space representation generated by an autoencoder) may be clustered using a clustering algorithm (as described above).

[0074] Method 1000 includes, in block 1010, determining whether the clustering results in a distinct pattern (e.g., if at least one defined cluster is produced). If a distinct pattern is not produced, no determination is made as to the source of error in the sample (block 1012); otherwise, if a distinct pattern is observed after clustering, a determination may be made as to which cluster(s) the representation of the measured biomarkers resides. In some embodiments, a first outlier class (e.g., cluster) may represent a first level or type of IV contamination (block 1014), a second outlier class may represent a second level or type of IV contamination (block 1016), and a third outlier class may represent a third level or type of IV contamination (block 1018). Any number and / or type of outlier classes may be employed (e.g., clusters for additional levels of IV contamination, type of IV contamination, specific biomarkers and / or biomarker relationships that are outliers, etc.).

[0075] In some embodiments, clusters (e.g., outlier classes) may be developed for identifying diseases and / or other clinical conditions such as acute kidney injury (AKI), myocardial infarction, segment elevation myocardial infarction (STEMI), non-ST elevation myocardial infarction (NSTEMI), unstable angina, acute coronary syndrome, heart failure, atrial fibrillation, acute liver failure, or the like.

[0076] In some embodiments, a dimensionality reduction model and clustering model may be initially trained to identify only a few clinical conditions. The dimensionality reduction model and / or clustering model may be retrained to identify additional clinical conditions as training data becomes available (e.g., from use of the trained models in real-world, clinical settings). Further, a dimensionality reduction model and / or clustering model deployed in a hospital or laboratory may be initially trained with training data from one or more different hospitals or laboratories (or ahealthcare data provider). Thereafter, the dimensionality reduction model and / or clustering model may be retrained on data from the location in which the models are deployed (as training data becomes available).

[0077] In some embodiments, a dimensionality reduction model and clustering model may be trained to aid microscopic analysis of patient samples. For example, a dimensionality reduction model and clustering model may be trained to identify, quantify, flag, or otherwise facilitate analysis of, one or more features of cells, formed elements, bacteria, parasites, etc., such as white blood cell differentials, red blood cell morphology, body fluid differentials, microscopic urine analysis, gram stains, parasite smears, pathology smears, etc.

[0078] While some embodiments have been described herein with regard to autoencoders, it will be understood that other dimensionality reduction models may be employed such as normalizing flows, autoregressive models, deep energy-based models, generative adversarial networks (GANs), other deep generative learning models, etc.NON-LIMITING ILLUSTRATIVE EMBODIMENTS

[0079] Illustrative embodiment 1. A method of creating a machine-learning-based autoverification system for clinical laboratory diagnostics, comprising: obtaining a biomarker training dataset, the biomarker training dataset including biomarker values for a plurality of biomarkers; training an autoencoder employing the biomarker training dataset as both an input and a target for the autoencoder to create a trained autoencoder; and deploying the trained autoencoder in a hospital or laboratory.

[0080] Illustrative embodiment 2. The method of illustrative embodiment 1 wherein obtaining the biomarker training dataset comprises obtaining a list of biomarkers to autoverify from the hospital or laboratory and creating the biomarker training dataset based on the list of biomarkers.

[0081] Illustrative embodiment 3. The method according to one of the preceding embodiments wherein creating the biomarker training dataset comprises obtaining a biomarker dataset from the hospital or laboratory and creating the biomarker training dataset based on the biomarker dataset and the list of biomarkers.

[0082] Illustrative embodiment 4. The method according to one of the preceding embodiments wherein the biomarker training dataset includes at least 3 biomarker values for each biomarker on the list of biomarkers.

[0083] Illustrative embodiment 5. The method according to one of the preceding embodiments wherein the list of biomarkers comprises a list of biomarkers in a biomarker panel.

[0084] Illustrative embodiment 6. The method according to one of the preceding embodiments wherein the biomarker panel comprises biomarkers from at least one of a lipid panel, a complete blood count (CBC) panel, a basic metabolic panel (BMP), a comprehensive metabolic panel (CMP), a thyroid panel, a urine panel, and an immunoassay panel.

[0085] Illustrative embodiment 7. The method according to one of the preceding embodiments wherein the biomarker panel comprise routine diagnostic tests.

[0086] Illustrative embodiment 8. The method according to one of the preceding embodiments wherein the routine diagnostic tests comprise biomarkers from a lipid panel, a complete blood count (CBC) panel, and a comprehensive metabolic panel (CMP).

[0087] Illustrative embodiment 9. The method according to one of the preceding embodiments further comprising deploying an error calculation module in the hospital or laboratory, the error calculation module configured to determine reconstruction error between measured biomarkers input to the trained autoencoder and reconstructed biomarkers output from the autoencoder in response to the measured biomarkers.

[0088] Illustrative embodiment 10. The method according to one of the preceding embodiments further comprising employing the deployed autoencoder on biomarkers measured for a patient sample.

[0089] Illustrative embodiment 11 . The method according to one of the preceding embodiments wherein employing the deployed autoencoder comprises: obtaining the patient sample from a patient; performing at least one test on the patient sample to generate measured biomarkers from the patient sample; inputting the measured biomarkers to the autoencoder to generate output biomarkers; and based on a comparison of the measured biomarkers and the output biomarkers, determining whether the measured biomarkers are valid.

[0090] Illustrative embodiment 12. The method according to one of the preceding embodiments wherein the measured biomarkers comprise at least one of a lipid panel, a complete blood count (CBC) panel, a basic metabolic panel (BMP), a comprehensive metabolic panel (CMP), a thyroid panel, a urine panel, and an immunoassay panel.

[0091] Illustrative embodiment 13. The method according to one of the preceding embodiments wherein the comparison comprises determining reconstruction error between the measured biomarkers and the output biomarkers.

[0092] Illustrative embodiment 14. The method according to one of the preceding embodiments wherein determining the reconstruction error comprises determining at least one mean squared error value.

[0093] Illustrative embodiment 15. The method according to one of the preceding embodiments further comprising determining a cause of a difference between the measured biomarkers and the output biomarkers.

[0094] Illustrative embodiment 16. The method according to one of the preceding embodiments wherein determining the cause of the difference between the measured biomarkers and the output biomarkers comprises determining a classification of a representation of the measured biomarkers.

[0095] Illustrative embodiment 17. The method according to one of the preceding embodiments wherein the representation of the measured biomarkers comprises a latent space representation of the measured biomarkers generated by the autoencoder.

[0096] Illustrative embodiment 18. The method according to one of the preceding embodiments wherein determining the classification of the representation of the measured biomarkers comprises: employing a clustering algorithm to cluster the representation of the measured biomarkers into one of a plurality of clusters; and classifying the difference between the measured biomarkers and the output biomarkers based on the plurality of clusters.

[0097] Illustrative embodiment 19. The method according to one of the preceding embodiments wherein the plurality of clusters includes a clinical condition cluster and a contamination cluster.

[0098] Illustrative embodiment 20. A method of creating a machine-learning-based autoverification system for clinical laboratory diagnostics, comprising: obtaining a first list of biomarkers for autoverification; obtaining a first training dataset for the first list of biomarkers, the first training dataset including biomarker values for each biomarker on the first list of biomarkers; training a first autoencoder employing the first training dataset as both an input and a target for the first autoencoder to create a first trained autoencoder; obtaining a second list of biomarkers for autoverification; obtaining a second training dataset for the second list of biomarkers, the second training datasetincluding biomarker values for each biomarker on the second list of biomarkers; training a second autoencoder employing the second training dataset as both an input and a target for the second autoencoder to create a second trained autoencoder; and deploying the first trained autoencoder and the second trained autoencoder.

[0099] Illustrative embodiment 21. The method of illustrative embodiment 20 wherein: obtaining the first and second lists of biomarkers comprises obtaining the first and second lists of biomarkers from a hospital or laboratory; obtaining the first and second training datasets comprises obtaining at least one biomarker dataset from the hospital or laboratory and creating the first and second training datasets based on the at least one biomarker dataset; and deploying the first and second trained autoencoders comprises deploying the first and second trained autoencoders in the hospital or laboratory.

[0100] Illustrative embodiment 22. The method according to one of the preceding embodiments wherein the first list of biomarkers comprises a list of biomarkers in a first biomarker panel and the second list of biomarkers comprises a list of biomarkers in a second biomarker panel.

[0101] Illustrative embodiment 23. The method according to one of the preceding embodiments wherein the first biomarker panel and second biomarker panel comprise different sets of biomarkers from at least one of a lipid panel, a complete blood count (CBC) panel, a basic metabolic panel (BMP), a comprehensive metabolic panel (CMP), a thyroid panel, a urine panel, and an immunoassay panel.

[0102] Illustrative embodiment 24. A machine-learning-based autoverification system for clinical laboratory diagnostics, comprising: a processor; a memory coupled to the processor, the memory including an autoencoder trained on a biomarker training dataset by using the biomarker training dataset as both an input and a target for the autoencoder; and computer program instructions stored in the memory that, when executed by the processor, cause the processor to: receive one or more biomarkers measured for a patient; input the measured biomarkers to the autoencoder to generate output biomarkers; and based on a comparison of the measured biomarkers input to the autoencoder and the output biomarkers of the autoencoder, determine whether the measured biomarkers are valid.

[0103] Illustrative embodiment 25. The autoverification system illustrative embodiment 24 wherein the biomarker training set comprises a biomarker dataset from a hospital or laboratory.

[0104] Illustrative embodiment 26. The autoverification system according to one of the preceding embodiments wherein the autoencoder is deployed at the hospital or laboratory that provided the biomarker dataset.

[0105] Illustrative embodiment 27. The autoverification system according to one of the preceding embodiments wherein the autoencoder is deployed at a different hospital or laboratory than the hospital or laboratory that provided the biomarker dataset.

[0106] Illustrative embodiment 28. The autoverification system according to one of the preceding embodiments wherein the biomarker training set comprises biomarkers in a biomarker panel.

[0107] Illustrative embodiment 29. The autoverification system according to one of the preceding embodiments wherein the biomarker panel comprises biomarkers from at least one of a lipid panel, a complete blood count (CBC) panel, a basic metabolic panel (BMP), a comprehensive metabolic panel (CMP), a thyroid panel, a urine panel, and immunoassay panel.

[0108] Illustrative embodiment 30. The autoverification system according to one of the preceding embodiments wherein the biomarker training set comprises biomarkers in routine diagnostic tests.

[0109] Illustrative embodiment 31 . The autoverification system according to one of the preceding embodiments wherein the routine diagnostic tests comprise biomarkers from a lipid panel, a complete blood count (CBC) panel, and a comprehensive metabolic panel (CMP).

[0110] Illustrative embodiment 32. The autoverification system according to one of the preceding embodiments further comprising computer program instructions stored in the memory that, when executed by the processor, cause the processor to determine reconstruction error between the measured biomarkers and the output biomarkers.

[0111] Illustrative embodiment 33. The autoverification system according to one of the preceding embodiments further comprising computer program instructions stored in the memory that, when executed by the processor, cause the processor to determine at least one mean squared error based on the measured biomarkers and the output biomarkers of the autoencoder.

[0112] Illustrative embodiment 34. The autoverification system according to one of the preceding embodiments further comprising computer program instructions storedin the memory that, when executed by the processor, cause the processor to predict a cause of a difference between the measured biomarkers and the output biomarkers of the autoencoder.

[0113] Illustrative embodiment 35. The autoverification system according to one of the preceding embodiments further comprising computer program instructions stored in the memory that, when executed by the processor, cause the processor to determine a classification of a representation of the measured biomarkers.

[0114] Illustrative embodiment 36. The autoverification system according to one of the preceding embodiments wherein the representation of the measured biomarkers comprises a latent space representation of the measured biomarkers, wherein the latent space representation is generated by the autoencoder.

[0115] Illustrative embodiment 37. The autoverification system according to one of the preceding embodiments further comprising computer program instructions stored in the memory that, when executed by the processor, cause the processor to employ a clustering algorithm to cluster the representation of the measured biomarkers into one of a plurality of clusters; and classify the difference between the measured biomarkers and the output biomarkers based on the plurality of clusters.

[0116] Illustrative embodiment 38. The autoverification system according to one of the preceding embodiments wherein the plurality of clusters includes a clinical condition cluster and a contamination cluster.

[0117] Illustrative embodiment 39. The autoverification system according to one of the preceding embodiments further comprising computer program instructions stored in the memory that, when executed by the processor, cause the processor to recommend manually processing of a test used to obtain the one or more biomarkers measured for a patient based on a difference between the measured biomarkers and the output biomarkers of the autoencoder.

[0118] Illustrative embodiment 40. A method of autoverification of clinical laboratory diagnostics, comprising: receiving one or more biomarkers measured for a patient; inputting the measured biomarkers to an autoencoder to generate output biomarkers, wherein the autoencoder is trained on a biomarker training dataset by using the biomarker training dataset as both an input and a target for the autoencoder; and based on a comparison of the measured biomarkers input to the autoencoder and the output biomarkers of the autoencoder, determining whether the measured biomarkers are valid.

[0119] Illustrative embodiment 41. The method according to illustrative embodiment 40 wherein the biomarker training set comprises a biomarker dataset from a hospital or laboratory and wherein the autoencoder is deployed at the hospital or laboratory that provided the biomarker dataset.

[0120] Illustrative embodiment 42. The method according to one of the preceding embodiments wherein the biomarker training set comprises a biomarker dataset from a first hospital or laboratory and wherein the autoencoder is deployed at a second hospital or laboratory that is different than the first hospital or laboratory that provided the biomarker dataset.

[0121] Illustrative embodiment 43. The method according to one of the preceding embodiments wherein autoencoder is a variational autoencoder.

[0122] The foregoing description discloses only example embodiments of the invention. Modifications of the above disclosed apparatus and methods which fall within the scope of the invention will be readily apparent to those of ordinary skill in the art.

[0123] Accordingly, while the present invention has been disclosed in connection with example embodiments thereof, it should be understood that other embodiments may fall within the spirit and scope of the invention, as defined by the following claims.

Claims

WHAT IS CLAIMED IS:

1. A method of creating a machine-leaming-based autoverification system for clinical laboratory diagnostics, comprising: obtaining a biomarker training dataset, the biomarker training dataset including biomarker values for a plurality of biomarkers; training an autoencoder employing the biomarker training dataset as both an input and a target for the autoencoder to create a trained autoencoder; and deploying the trained autoencoder in a hospital or laboratory.

2. The method of claim 1 wherein obtaining the biomarker training dataset comprises obtaining a list of biomarkers to autoverify from the hospital or laboratory and creating the biomarker training dataset based on the list of biomarkers.

3. The method of claim 2 wherein creating the biomarker training dataset comprises obtaining a biomarker dataset from the hospital or laboratory and creating the biomarker training dataset based on the biomarker dataset and the list of biomarkers.

4. The method of claim 3 wherein the biomarker training dataset includes at least 3 biomarker values for each biomarker on the list of biomarkers.

5. The method of claim 2 wherein the list of biomarkers comprises a list of biomarkers in a biomarker panel.

6. The method of claim 5 wherein the biomarker panel comprises biomarkers from at least one of a lipid panel, a complete blood count (CBC) panel, a basic metabolic panel (BMP), a comprehensive metabolic panel (CMP), a thyroid panel, a urine panel, and an immunoassay panel.

7. The method of claim 5 wherein the biomarker panel comprise routine diagnostic tests.

8. The method of claim 7 wherein the routine diagnostic tests comprise biomarkers from a lipid panel, a complete blood count (CBC) panel, and a comprehensive metabolic panel (CMP).

9. The method of claim 1 further comprising deploying an error calculation module in the hospital or laboratory, the error calculation module configured to determine reconstruction error between measured biomarkers input to the trained autoencoder and reconstructed biomarkers output from the autoencoder in response to the measured biomarkers.

10. The method of claim 1 further comprising employing the deployed autoencoder on biomarkers measured for a patient sample.

11. The method of claim 10 wherein employing the deployed autoencoder comprises: obtaining the patient sample from a patient; performing at least one test on the patient sample to generate measured biomarkers from the patient sample; inputting the measured biomarkers to the autoencoder to generate output biomarkers; and based on a comparison of the measured biomarkers and the output biomarkers, determining whether the measured biomarkers are valid.

12. The method of claim 11 wherein the measured biomarkers comprise at least one of a lipid panel, a complete blood count (CBC) panel, a basic metabolic panel (BMP), a comprehensive metabolic panel (CMP), a thyroid panel, a urine panel, and an immunoassay panel.

13. The method of claim 11 wherein the comparison comprises determining reconstruction error between the measured biomarkers and the output biomarkers.

14. The method of claim 13 wherein determining the reconstruction error comprises determining at least one mean squared error value.

15. The method of claim 11 further comprising determining a cause of a difference between the measured biomarkers and the output biomarkers.

16. The method of claim 15 wherein determining the cause of the difference between the measured biomarkers and the output biomarkers comprises determining a classification of a representation of the measured biomarkers.

17. The method of claim 16 wherein the representation of the measured biomarkers comprises a latent space representation of the measured biomarkers generated by the autoencoder.

18. The method of claim 16 wherein determining the classification of the representation of the measured biomarkers comprises: employing a clustering algorithm to cluster the representation of the measured biomarkers into one of a plurality of clusters; and classifying the difference between the measured biomarkers and the output biomarkers based on the plurality of clusters.

19. The method of claim 18 wherein the plurality of clusters includes a clinical condition cluster and a contamination cluster.

20. A method of creating a machine-learning-based autoverification system for clinical laboratory diagnostics, comprising: obtaining a first list of biomarkers for autoverification; obtaining a first training dataset for the first list of biomarkers, the first training dataset including biomarker values for each biomarker on the first list of biomarkers; training a first autoencoder employing the first training dataset as both an input and a target for the first autoencoder to create a first trained autoencoder; obtaining a second list of biomarkers for autoverification; obtaining a second training dataset for the second list of biomarkers, the second training dataset including biomarker values for each biomarker on the second list of biomarkers;training a second autoencoder employing the second training dataset as both an input and a target for the second autoencoder to create a second trained autoencoder; and deploying the first trained autoencoder and the second trained autoencoder.

21. The method of claim 20 wherein: obtaining the first and second lists of biomarkers comprises obtaining the first and second lists of biomarkers from a hospital or laboratory; obtaining the first and second training datasets comprises obtaining at least one biomarker dataset from the hospital or laboratory and creating the first and second training datasets based on the at least one biomarker dataset; and deploying the first and second trained autoencoders comprises deploying the first and second trained autoencoders in the hospital or laboratory.

22. The method of claim 21 wherein the first list of biomarkers comprises a list of biomarkers in a first biomarker panel and the second list of biomarkers comprises a list of biomarkers in a second biomarker panel.

23. The method of claim 22 wherein the first biomarker panel and second biomarker panel comprise different sets of biomarkers from at least one of a lipid panel, a complete blood count (CBC) panel, a basic metabolic panel (BMP), a comprehensive metabolic panel (CMP), a thyroid panel, a urine panel, and an immunoassay panel.

24. A machine-learning-based autoverification system for clinical laboratory diagnostics, comprising: a processor; a memory coupled to the processor, the memory including an autoencoder trained on a biomarker training dataset by using the biomarker training dataset as both an input and a target for the autoencoder; and computer program instructions stored in the memory that, when executed by the processor, cause the processor to: receive one or more biomarkers measured for a patient;input the measured biomarkers to the autoencoder to generate output biomarkers; and based on a comparison of the measured biomarkers input to the autoencoder and the output biomarkers of the autoencoder, determine whether the measured biomarkers are valid.

25. The autoverification system of claim 24 wherein the biomarker training set comprises a biomarker dataset from a hospital or laboratory.

26. The autoverification system of claim 25 wherein the autoencoder is deployed at the hospital or laboratory that provided the biomarker dataset.

27. The autoverification system of claim 25 wherein the autoencoder is deployed at a different hospital or laboratory than the hospital or laboratory that provided the biomarker dataset.

28. The autoverification system of claim 24 wherein the biomarker training set comprises biomarkers in a biomarker panel.

29. The autoverification system of claim 28 wherein the biomarker panel comprises biomarkers from at least one of a lipid panel, a complete blood count (CBC) panel, a basic metabolic panel (BMP), a comprehensive metabolic panel (CMP), a thyroid panel, a urine panel, and immunoassay panel.

30. The autoverification system of claim 24 wherein the biomarker training set comprises biomarkers in routine diagnostic tests.31 . The autoverification system of claim 30 wherein the routine diagnostic tests comprise biomarkers from a lipid panel, a complete blood count (CBC) panel, and a comprehensive metabolic panel (CMP).

32. The autoverification system of claim 24 further comprising computer program instructions stored in the memory that, when executed by the processor,cause the processor to determine reconstruction error between the measured biomarkers and the output biomarkers.

33. The autoverification system of claim 32 further comprising computer program instructions stored in the memory that, when executed by the processor, cause the processor to determine at least one mean squared error based on the measured biomarkers and the output biomarkers of the autoencoder.

34. The autoverification system of claim 24 further comprising computer program instructions stored in the memory that, when executed by the processor, cause the processor to predict a cause of a difference between the measured biomarkers and the output biomarkers of the autoencoder.

35. The autoverification system of claim 34 further comprising computer program instructions stored in the memory that, when executed by the processor, cause the processor to determine a classification of a representation of the measured biomarkers.

36. The autoverification system of claim 35 wherein the representation of the measured biomarkers comprises a latent space representation of the measured biomarkers, wherein the latent space representation is generated by the autoencoder.

37. The autoverification system of claim 36 further comprising computer program instructions stored in the memory that, when executed by the processor, cause the processor to employ a clustering algorithm to cluster the representation of the measured biomarkers into one of a plurality of clusters; and classify the difference between the measured biomarkers and the output biomarkers based on the plurality of clusters.

38. The autoverification system of claim 37 wherein the plurality of clusters includes a clinical condition cluster and a contamination cluster.

39. The autoverification system of claim 24 further comprising computer program instructions stored in the memory that, when executed by the processor, cause the processor to recommend manually processing of a test used to obtain the one or more biomarkers measured for a patient based on a difference between the measured biomarkers and the output biomarkers of the autoencoder.

40. A method of autoverification of clinical laboratory diagnostics, comprising: receiving one or more biomarkers measured for a patient; inputting the measured biomarkers to an autoencoder to generate output biomarkers, wherein the autoencoder is trained on a biomarker training dataset by using the biomarker training dataset as both an input and a target for the autoencoder; and based on a comparison of the measured biomarkers input to the autoencoder and the output biomarkers of the autoencoder, determining whether the measured biomarkers are valid.41 . The method of claim 40 wherein the biomarker training set comprises a biomarker dataset from a hospital or laboratory and wherein the autoencoder is deployed at the hospital or laboratory that provided the biomarker dataset.

42. The method of claim 40 wherein the biomarker training set comprises a biomarker dataset from a first hospital or laboratory and wherein the autoencoder is deployed at a second hospital or laboratory that is different than the first hospital or laboratory that provided the biomarker dataset.

43. The method of claim 40 wherein autoencoder is a variational autoencoder.

Images