System and method for determining accuracy of test results in diagnostic laboratory systems

Abstract

A method of determining accuracy of a test performed by a diagnostic laboratory system includes obtaining one or more first measurements during a first operation of the test performed by the diagnostic laboratory system. One or more second measurements are obtained during a second operation of the test performed by the diagnostic laboratory system. The first measurement and the second measurement are co-analyzed using a trained model that calculates an uncertainty score for the test based on a learned correlation between the first operation and the second operation. The uncertainty score may be used to determine whether the test results are reliable or whether the test should be retested. Other methods and systems are disclosed.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Patent Application No. 63 / 374,885, filed on September 7, 2022, entitled “SYSTEMS AND METHODS FOR DETERMINING TEST RESULT ACCURACIES INDIAGNOSTIC LABORABORY SYSTEMS,” the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0002] Embodiments of the present disclosure relate to determining test result accuracy in a diagnostic laboratory system. Background Art

[0003] Clinical diagnostic laboratory systems process patient samples, such as serum, plasma, urine, interstitial fluid, cerebrospinal fluid, etc., to obtain test results. Clinicians then use the test results to screen, diagnose and / or monitor different patient conditions and diseases. Each test includes multiple different operations, such as aspirating a sample and adding reagents to the sample. In addition, different types of tests perform operations in different orders. If any operation fails during the test and the failure is not detected, the undetected failure will have a significant impact on the test results and any clinical decisions made based on the test results. Therefore, it is necessary to determine the accuracy of the tests performed by the diagnostic laboratory system. Summary of the invention

[0004] In some embodiments, a method for determining the accuracy of a test performed by a diagnostic laboratory system includes (a) obtaining one or more first measurements during a first operation performed by the diagnostic laboratory system, wherein the first operation is one of a plurality of operations for performing a first test on a sample; (b) obtaining one or more second measurements during a second operation performed by the diagnostic laboratory system, wherein the second operation is one of a plurality of operations for performing the first test; (c) jointly analyzing the one or more first measurements and the one or more second measurements using a model trained using learned correlations between the first operation and the second operation; (d) determining an uncertainty score for the first test based on the joint analysis; and (e) determining whether to rerun the first test based on the uncertainty score.

[0005] In some embodiments, a method for determining the accuracy of a test performed by a diagnostic laboratory system includes (a) generating a graphical representation of a workflow of a test that can be performed by the diagnostic laboratory system, wherein the graphical representation includes multiple nodes and is configured to be analyzed by a graphical neural network; (b) obtaining one or more first measurements from a first operation performed by the diagnostic laboratory system during the test; (c) converting the one or more first measurements into a first vector, wherein the first vector is a first node of the graphical representation; (d) obtaining one or more second measurements from a second operation performed by the diagnostic laboratory system during the test; (e) converting the one or more second measurements into a second vector, wherein the second vector is a second node of the graphical representation; (f) analyzing the first node and the second node using the graphical neural network; (g) determining an uncertainty score for the test based on the analysis; and (h) determining whether to rerun the test based on the uncertainty score.

[0006] In some embodiments, a diagnostic laboratory system includes one or more modules configured to perform a test, the test having a workflow of a sequence of operations; a plurality of sensors configured to generate one or more measurements for each operation; a processor coupled to the sensors; and a memory coupled to the processor. The memory includes a graph neural network and computer program code, which, when executed by the processor, causes the processor to (a) generate a graph representation of the workflow, wherein the graph representation includes at least a first node corresponding to a first operation of the workflow and a second node corresponding to a second operation of the workflow; (b) convert one or more first measurements generated by the first operation into a first vector, wherein the first vector corresponds to the first node; (c) convert one or more second measurements generated by the second operation into a second vector, wherein the second vector corresponds to the second node; (d) analyze the first vector and the second vector using the graph neural network; (e) determine an uncertainty score for the test based on analyzing the first vector and the second vector; and (f) determine whether to rerun the test based on the uncertainty score.

[0007] Other aspects, features and advantages of the present disclosure will become apparent from the following description and illustration of multiple exemplary embodiments, including the best mode for implementing the present invention. The present disclosure may also have other and different embodiments, and its several details may be modified in various aspects, all without departing from the scope of the present disclosure. The present disclosure is intended to cover all modifications, equivalents and substitutes falling within the scope of the claims and their equivalents. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The drawings described below are provided for illustrative purposes and are not necessarily drawn to scale. Therefore, the drawings and descriptions are to be regarded as illustrative rather than restrictive in nature. The drawings are not intended to limit the scope of the present disclosure in any way.

[0009] Figure 1 A block diagram of a diagnostic laboratory system including multiple instruments is shown in accordance with one or more embodiments.

[0010] Figure 2 A block diagram of an instrument of a diagnostic laboratory system is shown, illustrating modules and operations that may be performed by the instrument according to one or more embodiments.

[0011] Figure 3 A block diagram of an aspiration and dispense module that may be implemented in an instrument of a diagnostic laboratory system according to one or more embodiments is shown.

[0012] Figure 4 is a graph illustrating pressure traces of a pipette assembly of an aspiration and dispense module located in a diagnostic laboratory system according to one or more embodiments.

[0013] Figure 5A Example methods for determining an uncertainty score for a test of a diagnostic laboratory system and / or for determining whether to retest a sample in accordance with one or more embodiments are shown.

[0014] Figure 5B An example diagram of a test workflow processed by a graph neural network and additional AI algorithms to generate uncertainty scores from a compact vector representation of operational data from operations performed during the test workflow in accordance with embodiments provided herein.

[0015] Figure 5C is a block diagram illustrating the use of an algorithm to calculate an uncertainty score resulting from testing of a sample by a diagnostic laboratory system in accordance with one or more embodiments.

[0016] Figure 6 A block diagram is shown illustrating different sequences of operations that may be performed on a sample during different tests performed by a diagnostic laboratory system in accordance with one or more embodiments.

[0017] Figure 7 A flow chart illustrating a method of determining the accuracy of a test performed by a laboratory system in accordance with one or more embodiments.

[0018] Figure 8 A flow chart illustrating another method of determining the accuracy of a test performed by a laboratory system in accordance with one or more embodiments. DETAILED DESCRIPTION

[0019] Diagnostic laboratory systems perform clinical chemistry and / or assays to identify analytes or other components in biological samples such as serum, plasma, urine, interstitial fluid, cerebrospinal fluid, etc. Samples are collected in sample containers and transported to instruments and modules throughout the laboratory system where the samples are processed and analyzed. For example, instruments and modules may prepare samples for testing and perform those tests on the samples.

[0020] When receiving a sample for testing in a diagnostic laboratory system, the sample may undergo a set of complex sequential operations of the workflow of each type of test. Different tests in the test may require different sequences of operations. In some tests, the operation may begin with sample container processing, for example, a sample container containing a sample is loaded into the laboratory system. The sample container processing may include other operations, such as reading a label on the sample container. Subsequent operations may include sample aspiration, reagent aspiration, and dispensing the sample and / or reagent into a test tube. The final operation in the sequence may include photometric measurement of the liquid in the test tube to determine the concentration of a chemical substance or analyte in the sample. Other operations may be performed on the sample and / or sample container. Operations may be performed using one or more instruments or modules configured to perform specific operations.

[0021] During each operation, multiple measurements may be performed. For example, the measurements may include measurements of instrument performance, samples, chemicals added to samples, sample containers, position and / or pressure during aspiration operations, etc. A single operation may result in one or more measurements. In some embodiments, the instruments and / or modules of the diagnostic laboratory system may be configured to perform quality checks to verify the operation of the diagnostic laboratory system. These quality checks generate measurements of instrument performance.

[0022] The operations performed during a test are performed sequentially. In some cases, multiple small errors or measurements that are well within the acceptable validity limits associated with the individual operations may accumulate and ultimately lead to inaccurate test results. These inaccurate test results may include underestimation or overestimation of the analyte concentration in the sample, which may lead to the prescription of unnecessary treatment for the patient.

[0023] Unlike traditional laboratory systems, the diagnostic laboratory systems and methods disclosed herein jointly analyze the set of operations performed during laboratory testing, and associate an uncertainty score (also known as a "biomarker confidence value") with each laboratory test based on the cumulative effect of small errors or measurements performed during each operation within the set. The uncertainty score or biomarker confidence value indicates whether the overall test result is valid. Therefore, for clinical decision-making, the uncertainty score can be used to determine whether the test result is reliable or whether retesting is needed, even if the individual operation measurements may be within acceptable limits. The uncertainty score can also indicate whether the instrument is malfunctioning. As described below, the uncertainty score or biomarker confidence value is not a simple average or median of the validity scores based on the individual operation measurements.

[0024] As previously mentioned, the test of the diagnostic laboratory system can include multiple sequential operations, and each operation can include multiple measurements. In some embodiments provided herein, a vector representation can be created for the measurement of each operation. For example, the vector representation can be an array of measurements that can be obtained by pre-processing raw measurement data (e.g., normalized data, using a dimensionality reduction technique such as principal component analysis (PCA) or independent component analysis (ICA) to map data to a vector space, using an autoencoder or other AI algorithms, etc.). The vector representation of each operation represents the "fingerprint" of the dynamic operation. Thereafter, the entire workflow of the test can be represented as a graph, wherein each node of the graph corresponds to a specific operation of the test. That is, each node of the graph is a vector representation (fingerprint) of the measurement of the different test operations in the test operation.

[0025] The graph neural network can be trained to map the above-mentioned workflow diagram to a compact vector space representing all vector representation fingerprints of the test operations. For example, in some embodiments, a graph autoencoder can be trained for this purpose. In one or more embodiments, the graph autoencoder may include a graph encoder that maps all input operation data (e.g., a vector representation of each test operation) to a compact vector space, and a graph decoder that reconstructs the operation data into its original form (e.g., for training purposes). The graph neural network can be trained on the operational workflow data of the test obtained from the daily operation of the diagnostic laboratory system. For example, such data can be collected from a deployed and fully operational diagnostic laboratory system. Training can be performed continuously, periodically, or at any suitable time. Training can be performed when the diagnostic laboratory system is online (e.g., in use) or offline.

[0026] After obtaining a compact vector space representation of the operational data of the test, the compact vector space can be used with a neural network or other AI algorithm to estimate test uncertainty (e.g., the probability of test success via a vector representation of a graphical encoding from each test operation). Additionally or alternatively, the compact vector space and the neural network can be used to determine the uncertainty score of each operation of the test. In some embodiments, a neural network (or other AI algorithm) for determining test and / or operational uncertainty based on the operational fingerprint (e.g., vector representation) of the test can be trained on data collected for failed operations or tests performed in a controlled diagnostic laboratory or factory setting. Importantly, in such a controlled setting, each test in an acceptable range (e.g., passing the validity check of the operation) can be marked as a failed test. For example, if all test operations produce valid results individually, but several operations are close to failure, it may be desirable to rerun the test (e.g., if multiple operations are close to the upper or lower limit of the validity of the operation, one operation is close to the upper limit, and another operation is close to the lower limit, etc.). Therefore, a neural network can be trained to provide an uncertainty score for the test and provide guidance on whether to allow retesting, regardless of whether the individual operations in the test pass the internal validity check. Likewise, by training on many vector representations of individual operations within a test, a neural network can be trained to identify an uncertainty score for each operation within a test and / or whether to recommend a retest based on the uncertainty scores of the individual operations (and / or any combination of the individual operation uncertainty scores).

[0027] Example measurements may include pressure sensor measurements obtained during the aspiration and dispensing of samples and reagents. Other measurements may include, for example, photometric measurements, acoustic measurements, temperature measurements, and optical measurements. In some embodiments, the measurements may include the results of quality inspection algorithms performed by instruments in the laboratory system. The algorithms used here may include, for example, deep neural networks, generative neural networks, graph neural networks, and other network or AI algorithms.

[0028] refer to Figure 1-8 These and other diagnostic laboratory systems and methods for determining test accuracy or uncertainty are described in greater detail.

[0029] refer to Figure 1, which shows a block diagram of an embodiment of a diagnostic laboratory system 100. The diagnostic laboratory system 100 may include a plurality of instruments 102 configured to process samples and sample containers 104 (some labeled) and perform tests (e.g., assays or other tests) on the samples. Performing a test may include performing one or more operations on the sample. Each operation may include one or more measurements. As described herein, one or more instruments 102 may include a plurality of different modules configured to perform the operations described herein. The sample may be a variety of biological samples collected from individuals (such as patients being evaluated by medical professionals). The samples may be collected in sample containers 104 and delivered to the laboratory system 100, where the sample containers 104 may be transported throughout the laboratory system 100 by tracks 108, such as to different instruments in the instruments 102. For example, the sample containers 104 may be transported by sample carriers 110 (a few of which are labeled). In Figure 1 In the embodiment of the present invention, the system 100 has three instruments 102, which include a sample handler 114, a first analyzer 116, and a second analyzer 118. The laboratory system 100 may include a sample handler 114, a first analyzer 116, and a second analyzer 118. Figure 1 Fewer or more instruments shown.

[0030] In some embodiments, the track 108 may extend near or around the instrument 102, such as Figure 1 As described herein, a portion or module of the instrument 102 may have a device for transferring the sample container 104 to and from the sample carrier 110, such as a robot ( Figure 1 10). The track 108 may include a plurality of segments 120 (several are labeled) that may be interconnected. The sample carrier 110 may be movable as indicated by dashed lines 126 in the segments 120. In some embodiments, some of the segments 120 may be integral with one or more of the instruments 102.

[0031] A diagnostic laboratory system such as the laboratory system 100 may have many instruments and may have tracks that link to other laboratory systems. A laboratory system including the laboratory system 100 may simultaneously move and process multiple sample carriers 110 and their corresponding sample containers 104. In some embodiments, the laboratory system 100 may simultaneously move and process hundreds or thousands of sample carriers 110 and their corresponding sample containers 104.

[0032] The laboratory system 100 may include or be coupled to a computer 130 configured to execute one or more programs configured to control the laboratory system 100. The computer 130 may be configured to communicate with the instrument 102 and other components of the laboratory system 100, such as components in a transport system. The transport system may include some or all components (e.g., motors, sensors, power supplies, etc.) configured to transport samples throughout the laboratory system 100. The computer 130 may include a processor 132 configured to execute programs, including programs other than those described herein. The programs may be implemented in computer code.

[0033] The computer 130 may include or have access to a memory 134, which may store one or more programs and / or data. The memory 134 and / or the programs stored therein may be referred to as non-transitory computer-readable media. The program may be a computer code that may be executed on or by the processor 132. The memory 134 may include an analysis program 136, which is configured to analyze the operations performed by the instrument 102 and / or determine the accuracy or uncertainty score of the test performed by the instrument 102, as described herein. The analysis program 136 may include a plurality of different programs as described herein, including one or more AI algorithms (e.g., a graph neural network 137, other generative neural networks, other deep networks or AI algorithms, including supervised, semi-supervised or unsupervised AI models, etc.). In some embodiments, the analysis program 136, a portion of the analysis program 136, or a copy of the analysis program 136 may reside in each instrument 102 or at a location outside the diagnostic laboratory system 100.

[0034] The computer 130 can be coupled to a workstation 138 that is configured to enable a user to interface with the laboratory system 100. The workstation 138 can include a display 140, a keyboard 142, and other peripherals. The analysis program 136 or other program can cause the display 140 to display the results of the data analysis, including an uncertainty score (e.g., a laboratory biomarker confidence value), a test validity score, and an indication as to whether the test should be rerun. Thus, the computer 130 in conjunction with the workstation 138 can be configured to generate a notification of the accuracy of the test, such as an uncertainty score.

[0035] The analysis program 136 can perform many functions. In some embodiments, the analysis program 136 can operate in conjunction with other programs to perform functions. For example, the analysis program 136 can be configured to detect operational failures in the laboratory system 100. If any operation of the laboratory system 100 fails and is not detected, the undetected failure can have a significant impact on clinical decision making. For example, a test resulting from a failed surgery may be inaccurate, which may cause a medical professional who relies on the results to provide inaccurate treatment.

[0036] When a sample is received for testing in the laboratory system 100, the sample undergoes a complex set of sequential operations or processes in a specific workflow defined by the specific test. Each type of test may have a unique sequence of operations. The workflow sequence may begin with a sample handling or sample container handling operation, followed by operations of aspirating and dispensing the sample and / or reagents into a test tube. The mixture in the test tube may undergo other operations required for the test. The workflow sequence may end with a measurement operation, such as a photometric measurement, to determine the chemical properties of the sample.

[0037] Each operation in the workflow sequence can be performed using one or more instruments 102. Because the operations occur sequentially, small instrument errors and / or measurements associated with one or more operations that are within but close to acceptable limits may accumulate and lead to larger errors in the resulting test. In some embodiments, the diagnostic laboratory system 100 collectively analyzes the set of sequential operations performed during a test and determines uncertainty scores for the results of tests performed within the laboratory system 100. For example, each uncertainty score can be based on an end-to-end validity check of an operation or set of operations performed during a test to determine the accuracy of the test. In some embodiments, the uncertainty score can be used to determine whether a retest is needed to obtain a valid or more accurate test result.

[0038] Additional references Figure 2 , which shows a block diagram of the instrument 202, which shows the modules and / or components associated with the instrument 202, and the operations associated therewith that can be performed by the instrument 202. (For example, the instrument 202 can be similar to Figure 1) In some embodiments, the modules and / or components of the instrument 202 may be within the instrument 202 (e.g., rather than being separate units occupying separate areas). In one or more embodiments, the instrument 202 may include a robotic handler 210 that may be configured to grasp and move sample containers 104, sample carriers 110, and / or other containers (vials, test tubes, etc.) within the instrument 202. The robotic handler 210 may operate in conjunction with an internal transport system 212 that is configured to transport sample containers 104, carriers 110, and / or other containers to specific locations within the instrument 202 via an internal track. The internal transport system 212 may be connected to the track 108 of the laboratory system 100 to receive and return sample containers 104, carriers 110, and / or other containers.

[0039] In some embodiments, the instrument 202 may include a reagent storage 214. The reagent storage 214 may be located in a module within the instrument 202 that is accessible by components of an aspiration and dispensing module 216 that is configured to aspirate and dispense reagents and samples. The photometric analyzer 218 may perform photometric analysis of the sample with or without the addition of one or more reagents to the sample. The quality checker 220 may perform self-tests and other analyses to determine whether a module (e.g., modules 210-218) is performing correctly and / or the likelihood that the results of the operation of the instrument 202 are accurate. The quality checker 220 may communicate with the analysis program 136 ( Figure 1 ) and / or transmit the quality inspection measurement results to the analysis program 136.

[0040] Additional references Figure 3 , which shows a block diagram of an embodiment of the aspiration and dispensing module 216. The aspiration and dispensing module 216 can be configured to be configured to be operable in one or more instruments 102 ( Figure 1 ) or instrument 202( Figure 2 Other embodiments of the aspiration and dispensing module 216 may be used in the instrument 102 and / or the instrument 202. Figure 3 The embodiment of FIG. 1 shows a sample container 304 located in a carrier 310, which shows a sample container 104 and a carrier 110 ( Figure 1 ). The carrier 310 may have been transported by the internal transport system 212 ( Figure 2 ) is transmitted to the aspiration and dispensing module 216. The sample container 304 may contain samples to be collected by the instrument 202 and / or other instruments in the instrument 102 ( Figure 1 ) samples 306 analyzed, processed and / or tested.

[0041] The aspiration and dispensing module 216 may include a reagent 312 stored in a reagent pack 314. As described in more detail herein, components of the aspiration and dispensing module 216 may aspirate the reagent 312 from the reagent pack 314 and aspirate the sample 306 from the sample container 304. In some embodiments, the aspiration and dispensing module 216 may include a tip dispenser 316 configured to change an aspiration probe tip as described herein prior to an aspiration operation. The aspiration and dispensing module 216 may have a test tube 320 configured to receive aspirated portions of the reagent 312 and the sample 306 via a dispensing operation. In some embodiments, the contents of the test tube 320 may be subjected to a photometric analyzer 218 ( Figure 2 ) performed by photometric analysis.

[0042] The aspiration and dispensing module 216 may include a robot 331 configured to move a straw assembly 332 within the aspiration and dispensing module 216. Figure 3 In the embodiment of the present invention, the probe 334 of the pipette assembly 332 is shown ready to aspirate the reagent 312 from the reagent package 314. The probe 334 is shown with the tip 322 attached to one end of the probe 334. The tip 322 may have been placed on the probe 334, such as by the tip dispenser 316 before aspirating the reagent 312. A new pipette tip may be placed on the probe 334 by a tip replacement operation, such as by using the tip dispenser 316 before aspirating the reagent 312 or the sample 306.

[0043] Figure 3 A sample container 304 is shown without a lid, which may have been removed by a decapping module (not shown) in the instrument 202 or another module (not shown) in the laboratory system 100 that performs a decapping operation. Removing the lid allows the sample 306 to be aspirated. The pipette assembly 332 can be configured to aspirate and dispense the reagent 312 and the sample 306 by positioning the probe 334 using the robot 331. By moving the probe 334 to the appropriate position and performing the dispensing operation, the reagent 312, other reagents, and a portion of the sample 306 can be dispensed into a reaction container, such as a test tube 320. The test tube 320 can be made of a material that passes light for use with the photometric analyzer 218 ( Figure 2 ) as described herein.

[0044] Some components of the aspiration and dispensing module 216 may be electrically coupled to the computer 330. Figure 3 In the embodiment of the present invention, the computer 330 may include a processor 330A and a memory 330B. The program 330C may be stored in the memory 330B and may be executed by the processor 330A. In other embodiments, the computer 330 and / or components of the computer 330 may be located in the computer 130 ( Figure 1 ) is implemented in. One of the programs 330C may be the quality check program 220 ( Figure 2 ). Computer 330 may also include a suction / dispensing controller 330D and a position controller 330E, which may be controlled by a program, such as program 330C stored in memory 330B. In some embodiments, position controller 330E and / or suction / dispensing controller 330D may be implemented in a separate device (e.g., another computer). Program 330C may include algorithms for controlling and / or monitoring components within suction and dispensing module 216. In some embodiments, the algorithms may include position controller 330E and / or suction / dispensing controller 330D.

[0045] The robot 331 may include one or more arms and motors configured to move the straw assembly 332 within the aspiration and dispensing module 216. Figure 3 In an embodiment of the present invention, the robot 331 may include an arm 350 coupled between a first motor 352 and a pipette assembly 332. The first motor 352 may be electrically coupled to the computer 330 and may receive instructions generated by a position controller 330E. The instructions may instruct the first motor 352 to move in a particular direction and speed. The first motor 352 may be configured to move the arm 350 so that the detector 334 can aspirate and / or dispense the sample 306 and / or reagent 312, as described herein. The first motor 352 may include or be associated with a position sensor 352A, which is configured to generate measurements (e.g., sensor data) indicating the position of the arm 350. The measurement data generated by the position sensor 352A may be transmitted to the computer 330 and / or the computer 130 ( Figure 1 ) and can be used by analysis program 136 as described herein.

[0046] The second motor 354 can be coupled between the arm 350 and the pipette assembly 332 and can be configured to move the probe 334 in a vertical direction (e.g., a Z direction) to aspirate and / or dispense liquid as described herein and replace the tip 322. The second motor 354 can move the probe 334 in response to instructions generated by the program 330C. For example, the second motor 354 can enable the probe 334 to enter and exit the sample container 304, the test tube 320, the tip dispenser 316, and / or the reagent pack 314. The second motor 354 may include or be associated with a current sensor 354A that is configured to measure the current drawn by the second motor 354. The measurement or sensor data (e.g., measured current) generated by the current sensor 354A can be transmitted to the computer 330 and / or the computer 130 ( Figure 1 ) and can be used by analysis program 136 as described herein.

[0047] The aspiration and dispensing module 216 may include a plurality of position sensors configured to generate measurements related to the position of the components. Figure 3 In some embodiments, the position sensor 356 can be mechanically coupled to the robot 331. In some embodiments, the position sensor 356 can be coupled to other components in the aspiration and dispense module 216. The position sensor 356 can be configured to sense the position of one or more components of the robot 331 or other components within the aspiration and dispense module 216 (e.g., the straw assembly 332). Figure 3 In an embodiment of the present invention, position sensor 356 can measure the position of arm 350, straw assembly 332, and / or detector 334. The measurements (e.g., position data) can be transmitted to computer 330 and / or computer 130 for processing by analysis program 136, as described herein.

[0048] The aspiration and dispense module 216 may also include a pump 360 mechanically coupled to the conduit 362 and electrically coupled to the aspiration / dispensing controller 330D. During an aspiration operation, the pump 360 may generate a vacuum or negative pressure (e.g., an aspiration pressure) in the conduit 362. During a dispense operation, the pump 360 may generate a positive pressure (e.g., a dispense pressure) in the conduit 362.

[0049] Pressure sensor 364 can be configured to measure pressure in conduit 362 and generate a measurement (e.g., pressure data) indicative of the pressure. In some embodiments, pressure sensor 364 can be configured to measure aspiration pressure and generate a pressure measurement. In some embodiments, pressure sensor 364 can be configured to measure dispense pressure and generate a pressure measurement. For example, the pressure measurement can be in the form of a pressure trace as a function of time, as described below with reference to Figure 4 The pressure measurement can ultimately be transmitted to the computer 130 ( Figure 1 ) and / or computer 330 for processing by analysis program 136. The pressure trace may change as a function of time or when one or more components of aspiration and dispense module 216 are replaced or fail.

[0050] Additional references Figure 4 , Figure 4 is a graph showing an example of a pressure trace 400 of the straw assembly 332 as a function of time as measured by the pressure sensor 364. Figure 4 In the embodiment of the present invention, pressure trace 400 shows the pressure in pipette assembly 332 during tip pick-up, aspiration, and dispensing operations. As tip 322 is replaced, the pressure rises slightly and drops significantly during aspiration. Then, during a dispensing operation, the pressure rises significantly. Pressure trace 400 represents one or more measurements that can be analyzed by analysis program 136, as described herein.

[0051] Reference again Figure 3 , the aspiration and dispensing module 216 may include an imaging device 366 configured to capture an image of the detector 334 and / or the liquid in the detector 334. For example, the detector 334 may be transparent, so that the imaging device 366 can capture an image of the liquid located in the detector 334. The captured image may include image data that is transmitted to the computer 330 and / or the computer 130 ( Figure 1 ) for processing by analysis program 136. The image data may include measurements generated during an imaging operation, such as during photometric analysis. Program 330C (or Figure 1 The image data may be analyzed by the analysis program 136 of the program 330C to determine the quality of the liquid in the detector 334. For example, the program 330C or the analysis program 136 may determine whether the liquid in the detector 334 contains bubbles or other anomalies.

[0052] As described herein, one or more modules or components of the instrument, such as the aspiration and dispensing module 216, may include one or more sensors that may be monitored by one or more programs, such as program 330C. Example sensors include position sensors, pressure sensors, imaging sensors, etc. A program, such as program 330C, may also perform a quality check (e.g., self-test) routine on the sensors. This information may be provided to computer 130 and / or analysis program 136 ( Figure 1 As further described below, the data generated by the self-test routines and sensors include measurements that can be encoded into a vector space and / or analyzed by one or more AI algorithms, such as by analysis program 136 ( Figure 1 ), for example, a first AI algorithm may generate a dynamic fingerprint of each operation tested via a vector representation of operational data (e.g., sensor measurements, self-test measurements, etc.), and a second AI algorithm may analyze the vectors (e.g., fingerprints) of the various operations tested to determine the accuracy of the test, such as by calculating or otherwise determining an uncertainty score (e.g., a biometric confidence value).

[0053] In more detail, analysis program 136 may employ an AI algorithm to analyze the common operational data generated by instrument 102. Based on the analysis, analysis program 136 may calculate an uncertainty score (e.g., a biomarker confidence value) that is indicative of the validity or accuracy of a test performed on a sample such as sample 306. In some embodiments, instrument measurements may include, but are not limited to, pressure sensor measurements obtained during aspiration and dispensing of samples and reagents, such as those obtained by pressure trace 400 ( Figure 4Other measurements may include position data, image data, photometric measurements, acoustic measurements, temperature measurements, optical measurements, quality inspection or self-test measurements, etc.

[0054] In some embodiments, the measurements obtained during each operation of the test are encoded as a specific fingerprint (e.g., a vector) representing the dynamics of the operation. A first AI algorithm trained with learned correlations between operations (e.g., learned correlations between a first operation and a second operation) can then map the fingerprint vector representation to a compact vector space. One or more other AI algorithms are configured to jointly analyze the compact vector space to determine the accuracy of the test, such as by calculating an uncertainty score. In some embodiments, if the uncertainty score is below a predetermined value, the computer 130 ( Figure 1 ) can generate a notification that a test is invalid and / or may need to be retested.

[0055] Figure 5A An example method 500 is shown for determining an uncertainty score for a test of a diagnostic laboratory system and / or for determining whether to retest a sample in accordance with one or more embodiments. Figure 5B An example graph 502 of a test workflow processed by a graph neural network 504 and additional AI algorithms (e.g., neural network 506) that generate uncertainty scores 508 from a compact vector representation 510 of operational data from operations (e.g., operations O1-O6) performed during the test workflow is shown in accordance with embodiments provided herein.

[0056] refer to Figure 5A , in block 512, measurements are obtained for each operation within the test. For example, pressure, temperature, photometric, acoustic or other parameters, self-test or quality check measurements, etc. may be obtained for each operation of the test (e.g., provided to computer 130 and / or analysis program 136). Example operations include sample container handling operations, sample aspiration and dispensing, reagent aspiration and dispensing, photometric measurements to determine chemical concentration and / or assay type, and / or any other test operation.

[0057] In block 514, a vector representation may be created for the measurements of each test operation. For example, the vector representation may be an array of measurements obtained by preprocessing the raw measurement data (e.g., normalizing the data, mapping the data to a vector space using a dimensionality reduction technique such as principal component analysis (PCA) or independent component analysis (ICA), using an autoencoder or other AI algorithm, etc.). The vector representation of each operation represents a dynamic "fingerprint" of the operation.

[0058] Thereafter, in block 516, a workflow diagram for the test may be created. Specifically, the entire workflow of the test may be represented as a graph, where each node of the graph corresponds to a specific operation of the test. That is, each node in the graph is a vector representation (fingerprint) of measurements of different test operations in the test operation. For example, Figure 5B 502 shows an example test workflow with six operations (labeled O1-O6). Other numbers and / or orders of operations may be used. The measurements taken during each operation O1-O6 are encoded in a vector representation corresponding to a node (e.g., nodes N1-N6, respectively) of the graph 502 (see also the description below). Figure 6 ).

[0059] In block 518, a graph neural network is used to map the workflow diagram to a compact vector space. For example, a graph neural network can be trained to map the test workflow diagram to a compact vector space representing all vector representation fingerprints of the test operations. Figure 5B In , a graph neural network 504 has been trained to map the workflow diagram 502 to a compact vector space 510. In some embodiments, a graph autoencoder may be trained for this purpose. In one or more embodiments, the graph autoencoder may include a graph encoder that maps all input operation data (e.g., a vector representation of each test operation) to a compact vector space, and a graph decoder that reconstructs the operation data into its original form (e.g., for training purposes). Figure 5B , graph encoder 517 maps graph 502 to compact vector space 510, and graph decoder 517' reconstructs graph 502 (as graph 502'). The graph neural network can be trained on operational workflow data for testing, for example, operational workflow data obtained from daily operation of a diagnostic laboratory system. For example, such data can be collected from a deployed and fully operational diagnostic laboratory system. Training can be performed continuously, periodically, or at any suitable time. Training can be performed while the diagnostic laboratory system is online (e.g., in use) or offline.

[0060] After obtaining the compact vector space representation of the operational data of the test, in block 520, the compact vector space can be used with a neural network or other AI algorithm to estimate the test uncertainty (e.g., computing an uncertainty score for the test, such as the likelihood of success of the test, from the graph-encoded vector representation of each test operation). Additionally or alternatively, in block 522, the compact vector space and the neural network (or other AI algorithm) can be employed to determine an uncertainty score for each operation of the test. For example, in Figure 5BIn some embodiments, a neural network (or other AI algorithm) for determining test and / or operation uncertainty based on an operation fingerprint (e.g., a vector representation) of a test can be trained on data collected for failed operations or tests performed in a controlled diagnostic laboratory or factory setting. Importantly, in such a controlled setting, tests where each operation is within an acceptable range (e.g., passing a validity check for the operation) can be marked as a failed test. For example, if all test operations individually produce valid results, but several operations are close to failure, it may be desirable to rerun the test (e.g., if multiple operations are close to an upper or lower limit of the validity of the operation, one operation is close to an upper limit and another operation is close to a lower limit, etc.). Therefore, a neural network can be trained to provide an uncertainty score for a test and provide guidance on whether to grant retesting, regardless of whether the individual operations within the test pass the internal validity check. Similarly, by training on many vector representations of individual operations within a test, a neural network can be trained to identify the uncertainty score for each operation within a test and / or whether to recommend retesting based on the uncertainty score of the individual operation (and / or any combination of individual operation uncertainty scores).

[0061] Any suitable neural network may be employed. Example architectures include Inception, ResNet, ResNeXt, DenseNet, etc., although other CNN architectures may also be employed.

[0062] In block 524, a determination as to whether to retest may be made and / or provided to a user based on the uncertainty score for the overall test workflow and / or based on one or more uncertainty scores for the individual operations. In some embodiments, method 500, graph neural network 504, and / or neural network 506 may be implemented in computer 130, memory 134, and / or analysis program 136 (e.g., as computer program code), and analysis program 136 may make a retest recommendation and / or perform a retest.

[0063] As described above, in some embodiments, a neural network 506 (or another AI algorithm) for determining test and / or operation uncertainty based on a fingerprint (e.g., a vector representation) of a test operation can be trained on data collected for failed operations or tests performed in a controlled diagnostic laboratory or factory setting. For example, in a controlled diagnostic laboratory, it can be determined that in some cases, a creatinine concentration test can produce creatinine concentration values ​​with a variance of + / -0.1 mg / dL. The neural network 506 can be trained to provide an estimated creatinine concentration variance, and in some embodiments, recommend retesting based on the variance. For example, if a test produces a creatinine concentration value of 0.5 mg / dL+ / -0.1 mg / dL, the neural network 506 can suggest that the creatinine concentration should be retested for such a low value, even if each operation of the creatinine concentration test produces a measurement within the valid range (and / or each operation passes its own internal self-test or quality check). Additionally or alternatively, neural network 506 may be trained to generate an uncertainty score representing a confidence level (such as 50%, 70%, 90%, etc.) based on the operational fingerprint of the test. In some embodiments, a creatinine concentration value of 0.5 mg / dL with 50% confidence may be flagged for retesting even if each operation of the creatinine concentration test produces a measurement within the valid range.

[0064] In another example embodiment, the test may include obtaining a first measurement during a first operation of the test, wherein the first measurement has a value below but close to an upper validity limit of the first measurement. For example, the first measurement may be a reagent aspiration pressure that is below but close to the upper validity limit. The test may also include obtaining a second measurement during a second operation of the test, wherein the second measurement value has a value above but close to a lower validity limit of the second measurement. For example, the second measurement value may be a sample aspiration pressure that is above but close to the lower validity limit. A detailed analysis in a controlled or factory setting may indicate that, in this case, retesting is recommended. The neural network 506 may be trained to provide a low (e.g., failure) uncertainty score in this case. Specifically, a model (e.g., neural network 506) trained using a learned correlation between the first operation and the second operation may be used to jointly analyze the first and second measurements, and the neural network 506 may provide a failure uncertainty score for the test based on the joint analysis.

[0065] A neural network or other AI algorithm (e.g., neural network 506) can be trained to determine whether an individual operation has succeeded or failed, and / or in some embodiments, provide an uncertainty score for an individual operation. For example, in one embodiment, the uncertainty score for an individual operation can be obtained by training an ensemble of neural networks, where each network is trained to provide a "likelihood" of whether the operation is successful. Given a set of possibilities from the network ensemble, the final decision on whether the operation is successful or failed can be based on a majority or average probability score. In addition, the variance of the possibilities from the network ensemble can be used to estimate the uncertainty (and / or uncertainty score) of the individual operation. If the variance is high, it means that the networks in the ensemble are not consistent in whether the operation is successful, so the uncertainty (and uncertainty score) will be high, and vice versa. For example, an ensemble of three neural network models can be used to evaluate the same operation. In some embodiments, the neural network can be a different type of network and / or a neural network trained differently. Assume that network models 1, 2, and 3 output the probability of success as 1, 0.8, and 0.1, respectively (1 is a high probability of success). Using the majority method, the operation will be reported as having succeeded. However, the uncertainty score of the evaluation is high because the variance is high (e.g., the standard deviation is 0.47). If the same network ensemble outputs are 1.0, 0.8, and 0.9, the uncertainty score will be low (e.g., with a standard deviation of 0.1). In another embodiment, the neural network can output a probability of success (or uncertainty score) that is proportional to the network's confidence. For example, a neural network with a Gaussian process classification layer can be used (e.g., see Amersfoort et al., "On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertainty," Arxiv, arXiv.2102.11409, February 22, 2021, https: / / arxiv.org / abs / 2102.11409). In yet another embodiment, the output of the neural network can be explicitly calibrated to be proportional to the confidence (e.g., outputting the probability of success and / or the uncertainty score of a separate process), such as by using a Pratt calibration or similar algorithm.

[0066] Thus, a neural network or other AI algorithm can be trained to provide an uncertainty score for a test and provide guidance on whether to warrant a retest, regardless of whether the individual operations in the test pass or fail internal validity checks. Likewise, by training on many vector representations of the individual operations within a test, a neural network can be trained to identify an uncertainty score for each operation within a test and / or whether to recommend a retest based on the uncertainty scores of the individual operations (and / or any combination of the individual operation uncertainty scores).

[0067] Additional references Figure 1 and Figure 5C , Figure 5C 5 is a block diagram 520 illustrating the use of an AI algorithm to calculate an uncertainty score 545 according to one or more embodiments. Figure 5C In the embodiment of the present invention, the diagnostic laboratory system 100 may use a number of operations 530 or processes to test the sample 306 ( Figure 3 ). The measurements from each operation 530 can be encoded as a specific fingerprint or vector representing the individual operation. Figure 5C In the embodiment of the present invention, operation 530 includes sample handling 532, sample aspiration 534, reagent aspiration 536, and photometric analysis 538. The laboratory system 100 may perform other operations. Each operation 530 may be performed in one or more instruments 102. In operation block 540, measurements such as test measurements and measurements generated by self-test procedures may be received.

[0068] The measurements generated by the sample handling 532 may be in the form of a system log that records the actions performed during the sample handling and any anomalies that occurred during the sample handling 532. The measurements generated by the sample handling 532 may be in other forms. For example, the measurements may include the pressure applied to the gripper to grip the sample container 104, the weight of the sample container 104, identification information in the form of image data, and other measurements.

[0069] Sample aspiration 534 and reagent aspiration 536 can generate information about Figure 3 The measurements or data described by the aspiration and dispensing module 216 can include, for example, Figure 4 The pressure trace and sensor measurements from the sensor are shown in pressure trace 400. The measurements can be related to other operations, such as image data indicating whether the aspirated liquid contains bubbles. Photometric analysis 538 can generate a signal trace or other types of measurements or data typically generated by a photometric analyzer.

[0070] The measurements from operation block 540 may be received in operation block 542, where a specific fingerprint representing the dynamics of operation 530 is generated. The fingerprint may be encoded into a vector, such as a compact vector. The vector representation may be obtained by preprocessing the data by one or more instruments 102 ( Figure 1 ) is an array of measurements obtained from raw data generated during execution of operation 601. In some embodiments, preprocessing may include normalizing the data over the set of measurements within a feasible or predetermined measurement range. In other embodiments, preprocessing may involve projecting the raw data into a vector space using a dimensionality reduction technique such as principal component analysis (PCA), independent principal component analysis (ICA), or an autoencoder.

[0071] In some embodiments, AI such as deep networks, generative neural networks, and other trained models can be used to generate vectors. In some embodiments, each vector can represent a separate operational effectiveness score for the measurements received from operation 530.

[0072] The vector generated by operation block 542 may be analyzed by one or more AI algorithms in operation block 544 to determine the accuracy of the test. For example, one or more AI algorithms in operation block 544 may generate an uncertainty score. In some embodiments, operation block 544 may use a graph neural network (GNN) plus an additional AI algorithm (e.g., a neural network) to calculate the uncertainty score, as previously described with reference to Figure 5A and 5B Thus, one or more AI algorithms can learn a combined representation of heterogeneous data or measurements from modules and / or operations and estimate a confidence score associated with a test. The methods described herein, such as by analysis program 136, can be applied to determine an uncertainty score (e.g., accuracy) for an entire set of operations performed during laboratory testing.

[0073] Figure 5C The process described in can be applied to include reference Figure 3 and Figure 4 The sample test of described operation is to analyze the suction and distribution operation in addition to other operations. Because invalid operation, such as invalid suction operation, increases the possibility that the subsequent distribution operation will be invalid, and increases the uncertainty of the test, the joint analysis is more suitable for generating accurate uncertainty score than analyzing each operation separately. Another example is related to the sample and reagent volume used in the test. For example, the sample volume of suction may be lower than expected but within the predetermined limit, and the reagent volume of suction may be greater than expected but within the predetermined limit. When analyzed separately, the test will show a high possibility of effectiveness. However, even if the individual volume is within its respective predetermined limit, the analyte concentration calculated by the test may also be inaccurate. Therefore, for example, an uncertainty score less than a predetermined value can be calculated, which indicates that the associated test should be rerun. In some embodiments, the uncertainty score less than the predetermined value can cause the computer 130 (executing the analysis program 136) to automatically rearrange the associated test.

[0074] We will now describe additional testing procedures and descriptions using a graph neural network (GNN). Figure 2 The instrument 202 can be configured to perform a variety of different tests. For example, some tests can be performed using reagents, and some tests can be performed without using reagents. Figure 6, which shows a block diagram 600 (also referred to as diagram 600) that shows different sequences of operations 601 that can be performed on a sample to perform different tests or different types of tests according to one or more embodiments. Different instruments can perform different operations 601, and different sequences of operations 601 can be performed, depending on the test being performed. The sequence of operations 601 for a separate test can be referred to as a separate workflow. Thus, block diagram 600 shows a graphical representation of a workflow.

[0075] One or more operations 601 may be a graphical representation of a test workflow (e.g., Figure 5B 600 or 502), and the paths between operations 601 can be edges of the graph. Each operation 601 can generate a measurement, which can be encoded into a vector or compact vector (e.g., a reduced dimensionality vector) as described above. The GNN can map the vector representation fingerprint to a compact vector space (e.g., compact vector space 510), and another AI algorithm (e.g., neural network 506) can jointly analyze the measurement (via the fingerprint vector representation) to determine the accuracy of the test (e.g., uncertainty score).

[0076] Diagram 600 shows a workflow, including a sequence of operations for different tests that may be performed by instrument 202. For example, the first operation may be tip pick 602, where probe 334 ( Figure 3 ) can replace the tip (e.g., tip 322- Figure 3 ). Tip pick 602 may include moving detector 334 to tip dispenser 316 and replacing tip 322. After tip pick 602, processing may proceed to sample aspiration 604 or reagent aspiration 606. If reagent is not added to the sample, for example, processing may proceed directly to sample aspiration 604. The path from sample aspiration 604 may extend to sample distribution 608, and the path from reagent aspiration 606 may extend to reagent distribution 610. In some embodiments, sample aspiration 604 and sample distribution 608 may be a single operation or a single node of a graph. In some embodiments, reagent aspiration 606 and reagent distribution 610 may be a single operation or a single node of a graph. The paths from both sample distribution 608 and reagent distribution 610 may extend to photometric analysis 612. Before adding new liquid (e.g., new reagent) to test tube 320, the path from reagent distribution 610 may extend back to tip pick 602. Other embodiments of diagram 600 may include different paths, depending on the configuration of the instrument and the test that the instrument is configured to perform.

[0077] Different tests or different types of tests may have different workflows, such as different paths or sides from start to finish. For example, a first test having a first workflow may begin with tip pick 602, followed by sample aspiration 604. After sample aspiration 604, the test may continue with sample dispensing 608, followed by photometric analysis 612. A second test may have a second workflow and may begin with tip pick 602, followed by sample aspiration 604. After sample aspiration 604, the test may continue with sample dispensing 608, followed by tip pick 602 receiving detector 334 ( Figure 3 ) on a new tip (e.g., tip 322- Figure 3 ). The test may proceed to reagent aspiration 606, reagent dispensing 610, and may end with photometric analysis 612. A third test may have a third workflow and begin with tip pick-up 602, followed by sample aspiration 604. After sample aspiration 604, the test may proceed to sample dispensing 608, followed by tip pick-up 602 to receive a new tip on the detector 334. The test may proceed to reagent aspiration 606, then reagent dispensing 610, and back to tip pick-up 602. When a new tip is received on the detector 334, the test may proceed to reagent aspiration 606 and reagent dispensing 610 to add a new reagent to the test tube 320. The cycle may continue to add a new reagent to the test tube 320. The test may then terminate with photometric analysis 612.

[0078] The measurements from each operation 601 may be encoded into a vector, as described with respect to operation block 542 ( Figure 5C ) as described. The nodes in the graph (graph) 600 can then be analyzed by the GNN and then by additional AI algorithms, such as with respect to operation block 544 ( Figure 5C ) described. In some embodiments, the AI ​​algorithm may be a neural network. The workflow (e.g., path or edge) of each test may be analyzed, such as with respect to calculating the uncertainty score 545 ( Figure 5C ). The AI ​​algorithm may also generate an uncertainty score 545, as described herein.

[0079] In more detail, in some embodiments, analysis program 136 may obtain measurements of individual operations in operations 601 that collectively perform a test. Analysis program 136 may encode the measurements generated during each operation 601 into a vector representation. For example, the vector representation may be a vector representation generated by preprocessing one or more instruments 102 ( Figure 1) is an array of measurements obtained from raw data generated during execution of operation 601. In some embodiments, preprocessing may involve normalizing the data over the set of measurements within a feasible or predetermined measurement range. In other embodiments, preprocessing may involve projecting the raw data into a vector space using a dimensionality reduction technique such as principal component analysis (PCA), independent principal component analysis (ICA), or an autoencoder.

[0080] The analysis program 136 can then use GNNs and / or other neural networks to learn the operational manifold of the instrument 102 and / or laboratory system 100. The operational manifold can be a workflow for the different tests described herein. An autoencoder or other algorithm such as a variational graph autoencoder (which can include a graph decoder) can be trained to map all input operational data (e.g., measurements obtained during testing) to a compact vector space or other vector space. A graph decoder or other decoder can be trained to reconstruct operational data from a compact vector space or other space (for training purposes). In some embodiments, a model including an encoder, a decoder, and / or a neural network can be trained on a dataset obtained from the laboratory system 100 ( Figure 1 ) is trained on a large amount of operational workflow data and / or measurements obtained in the daily operation of the test. The edges (paths) of the graph (graph) 600 can be directed to represent the order or sequence of operations performed for each type of test. Therefore, the sequence models the causal structure of the workflow for executing the test. The above process can be implemented in an AI algorithm in the analysis program 136 and can enable the projection of the operational workflow data to a vector representation.

[0081] The model(s) implemented in the analysis program 136 can then be used to detect operational anomalies or identify operational failures in a particular test. Detection of operational anomalies can be performed using a compact vector space generated by a trained GNN and constrained to be a Gaussian or mixed Gaussian distribution. Operational instances that are projected further away, such as using the Mahalanobis distance, can be considered abnormal and can be attributed to operational anomalies. The Mahalanobis distance is a multivariate distance metric that measures the distance between a point and a distribution.

[0082] Training of an AI algorithm that generates uncertainty scores from a compact vector space generated by a GNN can be conducted in a controlled laboratory or factory setting where data corresponding to failed operations and / or tests is available. This data can then be combined with a large amount of data corresponding to successful operations and / or tests to train a neural network to estimate the likelihood of success of an operation based on the vector representation of the graph encoding used in the GNN. Along with the calculated likelihood scores, the neural network can also estimate a validity score associated with each operation, with the validity score being related to operations that are sources of high or low uncertainty scores. Equipped with such training, the analysis program 136 or other program can recommend retesting, or a user can determine whether retesting is necessary based on the uncertainty score. The status of the operation can also be recorded in a machine log, such as stored in memory 134 ( Figure 1 ) and can be used to determine whether a particular operation results in a consistently low uncertainty score and whether the corresponding module needs to be revised, maintained, or replaced.

[0083] Examples of the methods and apparatus disclosed herein may be illustrated by a test performed by at least one instrument 102, the test comprising a reagent being added to a sample followed by photometric analysis. Figure 6 , the workflow of the test has the following sequence: tip pick 602, sample aspiration 604, sample dispensing 608, tip pick 602, reagent aspiration 606, reagent dispensing 610, and photometric analysis 612. Measurements are obtained during each operation in the workflow. For example, during tip pick 602, measurements may include Figure 4 The pressure measurement (trace) shown and the position sensor measurement generated by the position sensor 352A and the position sensor 356. These measurements can be encoded into vectors. The measurements generated by other operations in the operation of the workflow can also be encoded into vectors. In some embodiments, the operation block 542 can generate a vector. Generate an adversarial network (GAN), GNN or other networks or models trained on the test workflow can analyze the vector to generate a compact vector space representation of the operational fingerprint of the test, which is analyzed by another AI algorithm (e.g., neural network 506) to determine the uncertainty score 545. Based on the uncertainty score 545, the computer 130 can recommend retesting or indicate that the test is valid. For example, information about the uncertainty score 545 can be output to the display 140.

[0084] Reference now Figure 7 , Figure 77 is a flow chart illustrating a method 700 for determining the accuracy of a test performed by a laboratory system (e.g., laboratory system 100) according to one or more embodiments. The method 700 includes, at block 702, obtaining one or more first measurements during a first operation performed by the laboratory system, wherein the first operation is one of a plurality of operations for performing a first test on a sample (e.g., sample 306). The method 700 includes, at block 704, obtaining one or more second measurements during a second operation performed by the laboratory system, wherein the second operation is one of a plurality of operations for performing the first test. The method 700 includes, at block 706, jointly analyzing the one or more first measurements and the one or more second measurements using a model (e.g., graph neural network 504 and / or neural network 506) trained based on learned correlations between the first operation and the second operation. The method 700 includes, at block 708, calculating an uncertainty score for the first test based on the analysis. And the method 700 includes, at block 710, determining whether to rerun the first test based on the uncertainty score. For example, the analysis program 136 may alert the user to rerun the first test. The method 700 may optionally include rerunning the first test in response to a determination made at block 710, such as when the calculated uncertainty score is less than a predetermined value. In some embodiments, the laboratory system 100 (e.g., the computer 130 executing the analysis program 136) may automatically initiate a rerun of the first test.

[0085] Reference now Figure 8 , Figure 8 800 for determining the accuracy of a test performed by a laboratory system (e.g., laboratory system 100) according to one or more embodiments. The method 800 includes, at block 802, generating a graphical representation of a workflow of a test that can be performed by the laboratory system, wherein the graphical representation includes a plurality of nodes and is configured to be analyzed by a graphical neural network (e.g., see Figure 5A Test workflow diagram 502 or Figure 6600 of the test workflow diagram of FIG. 800). The method 800 includes, at block 804, obtaining one or more first measurements from a first operation performed by the laboratory system during the test. The method 800 includes, at block 806, converting the one or more first measurements into a first vector, wherein the first vector is a first node of the graphical representation. The method 800 includes, at block 808, obtaining one or more second measurements from a second operation performed by the laboratory system during the test. The method 800 includes, at block 810, converting the one or more second measurements into a second vector, wherein the second vector is a second node of the graphical representation. The method 800 includes, at block 812, jointly analyzing the first node and the second node using a graphical neural network. For example, the graphical neural network 504 can generate a compact vector space representation of the graphical representation (e.g., graph 502 or 600). The method 800 includes, at block 814, determining an uncertainty score for the test based on the analysis. As described, in some embodiments, the compact vector space can be fed to a trained neural network (e.g., neural network 506) to determine an uncertainty score for the test. The method 800 includes, at block 816 , determining whether to rerun the test based on the uncertainty score.

[0086] Although the present disclosure is susceptible to various modifications and alternative forms, specific method and device embodiments have been shown by way of example in the accompanying drawings and are described in detail herein. However, it should be understood that the specific methods and devices disclosed herein are not intended to limit the present disclosure, but on the contrary, all modifications, equivalents and alternatives falling within the scope of the claims are covered.

Claims

1. A method of determining the accuracy of a test performed by a diagnostic laboratory system, comprising: obtaining one or more first measurements during a first operation performed by the diagnostic laboratory system, wherein the first operation is one of a plurality of operations for performing a first test on a sample; obtaining one or more second measurements during a second operation performed by the diagnostic laboratory system, wherein the second operation is one of a plurality of operations for performing the first test; jointly analyzing the one or more first measurements and the one or more second measurements using a model trained using a learned correlation between the first operation and the second operation; determining an uncertainty score for the first test based on the common analysis; as well as A determination is made based on the uncertainty score whether to rerun the first test. 2 . The method of claim 1 , further comprising re-running the first test in response to the uncertainty score being less than a predetermined value.

3. The method according to claim 1, wherein: Obtaining one or more first measurements during the first operation includes obtaining first measurements having values ​​below an upper limit of first measurement validity; Obtaining one or more second measurements during the second operation includes obtaining second measurements having values ​​above a lower limit of second measurement validity; The co-analyzing includes co-analyzing the first measurement and the second measurement using a model trained using a learned correlation between the first operation and the second operation; as well as Determining the uncertainty score includes determining a failure uncertainty score for the first test based on the common analysis.

4. The method according to claim 1, further comprising: encoding the one or more first measurements into a first vector representation representing a first operation; as well as The one or more second measurements are encoded into a second vector representation representing a second operation.

5. The method according to claim 4, wherein: Encoding the one or more first measurements includes normalizing the one or more first measurements; and Encoding the one or more second measurements includes normalizing the one or more second measurements.

6. The method according to claim 4, wherein: Encoding the one or more first measurements includes generating a reduced dimension vector from the one or more first measurements; and Encoding the one or more second measurements includes generating a reduced dimension vector from the one or more second measurements.

7. The method according to claim 1, wherein: jointly analyzing the one or more first measurements and the one or more second measurements using a model trained using a learned correlation between the first operation and the second operation includes encoding the one or more first measurements and the one or more second measurements into a vector space using a neural network; as well as Determining the uncertainty score includes determining the uncertainty score based on the vector space.

8. The method according to claim 1, further comprising: A workflow of at least the first operation and the second operation is represented as a graph having at least a first node and a second node, wherein the first node includes a vector representation of one or more first measurements and the second node includes a vector representation of one or more second measurements.

9. The method according to claim 8, wherein: Jointly analyzing the one or more first measurements and the one or more second measurements using a model trained using learned correlations between the first operation and the second operation includes encoding the graph into a vector space using a graph neural network; as well as Determining the uncertainty score includes determining the uncertainty score based on the vector space.

10. The method of claim 9, further comprising training a graphical neural network on the workflow.

11. The method of claim 1 , wherein the one or more first measurements include at least one of pressure, photometric, acoustic, temperature, and optical measurements.

12. The method of claim 1, wherein the first test is performed by a plurality of modules, and further comprising, in response to an uncertainty score being less than a predetermined value, determining which module of the plurality of modules caused the uncertainty score to be less than the predetermined value.

13. The method of claim 1, further comprising, in response to the uncertainty score being less than a predetermined value, determining which of the first and second operations resulted in the uncertainty score being less than the predetermined value.

14. A method of determining the accuracy of a test performed by a diagnostic laboratory system, comprising: generating a graphical representation of a workflow of a test executable by the diagnostic laboratory system, wherein the graphical representation includes a plurality of nodes and is configured to be analyzed by a graphical neural network; obtaining one or more first measurements from a first operation performed by the diagnostic laboratory system during testing; converting the one or more first measurements into a first vector, wherein the first vector is a first node of the graphical representation; obtaining one or more second measurements from a second operation performed by the diagnostic laboratory system during the test; converting the one or more second measurements into a second vector, wherein the second vector is a second node of the graphical representation; analyzing the first node and the second node using a graph neural network; determining an uncertainty score for the test based on the analyzing; as well as Determine whether to rerun the test based on the uncertainty score.

15. The method according to claim 14, wherein: The diagnostic laboratory system is configured to perform a plurality of operations in a first sequence during a first operation; The diagnostic laboratory system is configured to perform the plurality of operations in a second sequence during a second operation; as well as A graph neural network is trained on the first sequence and the second sequence.

16. The method of claim 14, wherein: Converting the one or more first measurements includes encoding the one or more first measurements into a first vector representation representing a first operation; Converting the one or more second measurements includes encoding the one or more second measurements into a second vector representation representing a second operation, wherein the analyzing includes jointly analyzing the first vector representation and the second vector representation using a graph neural network to map the first and second vector representations into a vector space; and Determining the uncertainty score includes determining the uncertainty score based on the vector space.

17. The method of claim 16, wherein the encoding comprises using principal component analysis, independent component analysis, or an autoencoder.

18. The method of claim 14, further comprising training a graphical neural network on the workflow.

19. A diagnostic laboratory system comprising: one or more modules configured to execute a test having a workflow of a sequence of operations; a plurality of sensors configured to generate one or more measurements for each operation; a processor coupled to the sensor; as well as a memory coupled to the processor, wherein the memory includes a graph neural network and computer program code that, when executed by the processor, causes the processor to: generating a graphical representation of the workflow, wherein the graphical representation includes at least a first node corresponding to a first operation of the workflow and a second node corresponding to a second operation of the workflow; converting one or more first measurements resulting from the first operation into a first vector, wherein the first vector corresponds to the first node; converting one or more second measurements resulting from the second operation into a second vector, wherein the second vector corresponds to the second node; analyzing the first vector and the second vector using a graph neural network; determining an uncertainty score for the test based on analyzing the first vector and the second vector; as well as Determine whether to rerun the test based on the uncertainty score.

20. The diagnostic laboratory system of claim 19, wherein: The memory comprises computer program code which, when executed by the processor, causes the processor to: encoding the one or more first measurements into a first vector representing a first operation; encoding the one or more second measurements into a second vector representing a second operation; analyzing the first vector and the second vector by using a graph neural network to map the first vector and the second vector into a vector space; as well as An uncertainty score is determined based on the vector space.

Images