Systems and methods for determining test result accuracy in a diagnostic laboratory system

Abstract

A method of determining accuracy of a test performed by a diagnostic laboratory system includes obtaining one or more first measurements during a first operation of a test performed by the diagnostic laboratory system. One or more second measurements are obtained during a second operation of the test performed by the diagnostic laboratory system. The first measurements and the second measurements are jointly analyzed using a trained model that computes an uncertainty score for the test using a correlation based on learning between the first operation and the second operation. The uncertainty score can be used to determine whether a test result is reliable or whether the test should be re-run. Other methods and systems are disclosed.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 374,885, filed September 7, 2022, entitled “Systems and methods for determining the accuracy of test results in diagnostic laboratory systems”, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] Embodiments of this disclosure relate to determining the accuracy of test results in a diagnostic laboratory system. Background Technology

[0004] Clinical diagnostic laboratory systems process patient samples, such as serum, plasma, urine, interstitial fluid, and cerebrospinal fluid, to obtain test results. Clinicians then use these results to screen, diagnose, and / or monitor various patient conditions and diseases. Each test involves several different procedures, such as aspirating the sample and adding reagents. Furthermore, different types of tests are performed in different sequences. If any procedure fails during testing and this failure is not detected, the undetected failure can have a significant impact on the test results and any clinical decisions made based on those results. Therefore, it is necessary to determine the accuracy of the tests performed by diagnostic laboratory systems. Summary of the Invention

[0005] In some embodiments, a method for determining the accuracy of a test performed by a diagnostic laboratory system includes (a) obtaining one or more first measurements during a first operation performed by the diagnostic laboratory system, wherein the first operation is one of a plurality of operations for performing a first test on a sample; (b) obtaining one or more second measurements during a second operation performed by the diagnostic laboratory system, wherein the second operation is one of a plurality of operations for performing the first test; (c) jointly analyzing the one or more first measurements and the one or more second measurements using a model trained on a correlation learned between the first and second operations; (d) determining an uncertainty score for the first test based on the joint analysis; and (e) determining whether to rerun the first test based on the uncertainty score.

[0006] In some embodiments, a method for determining the accuracy of a test performed by a diagnostic laboratory system includes (a) generating a graphical representation of the workflow of the test that can be performed by the diagnostic laboratory system, wherein the graphical representation includes a plurality of nodes and is configured to be analyzed by a graph neural network; (b) obtaining one or more first measurements from a first operation performed by the diagnostic laboratory system during the test; (c) converting the one or more first measurements into a first vector, wherein the first vector is a first node of the graphical representation; (d) obtaining one or more second measurements from a second operation performed by the diagnostic laboratory system during the test; (e) converting the one or more second measurements into a second vector, wherein the second vector is a second node of the graphical representation; (f) analyzing the first and second nodes using a graph neural network; (g) determining an uncertainty score for the test based on the analysis; and (h) determining whether to rerun the test based on the uncertainty score.

[0007] In some embodiments, a diagnostic laboratory system includes one or more modules configured to perform a test having a workflow of an operational sequence; a plurality of sensors configured to generate one or more measurements for each operation; a processor coupled to the sensors; and a memory coupled to the processor. The memory includes a graph neural network and computer program code that, when executed by the processor, causes the processor to (a) generate a graph representation of the workflow, wherein the graph representation includes at least a first node corresponding to a first operation of the workflow and a second node corresponding to a second operation of the workflow; (b) convert one or more first measurements generated by the first operation into a first vector, wherein the first vector corresponds to the first node; (c) convert one or more second measurements generated by the second operation into a second vector, wherein the second vector corresponds to the second node; (d) analyze the first and second vectors using the graph neural network; (e) determine an uncertainty score for the test based on the analysis of the first and second vectors; and (f) determine whether to rerun the test based on the uncertainty score.

[0008] Other aspects, features, and advantages of this disclosure will become apparent from the following description and illustrations of several exemplary embodiments, including the preferred mode for carrying out the invention. Other and different embodiments of this disclosure are also possible, and several details thereof may be modified in various respects without departing from the scope of this disclosure. This disclosure is intended to cover all modifications, equivalents, and substitutions falling within the scope of the claims and their equivalents. Attached Figure Description

[0009] The accompanying drawings described below are for illustrative purposes and are not necessarily drawn to scale. Therefore, the drawings and description are intended to be illustrative rather than restrictive. The drawings are not intended to limit the scope of this disclosure in any way.

[0010] Figure 1 A block diagram of a diagnostic laboratory system comprising multiple instruments according to one or more embodiments is shown.

[0011] Figure 2 A block diagram of an instrument for a diagnostic laboratory system is shown, illustrating the modules and operations that can be performed by the instrument according to one or more embodiments.

[0012] Figure 3 A block diagram of a suction and dispensing module that can be implemented in an instrument of a diagnostic laboratory system according to one or more embodiments is shown.

[0013] Figure 4 It is a graph showing the pressure trajectory of the pipette assembly of the suction and dispensing module in a diagnostic laboratory system according to one or more embodiments.

[0014] Figure 5A Example methods are shown for determining the uncertainty score of a test for a diagnostic laboratory system and / or for determining whether a sample should be retested, according to one or more embodiments.

[0015] Figure 5B An example diagram of a test workflow, processed by a graph neural network and additional AI algorithms according to embodiments provided herein, is shown to generate uncertainty scores from a compact vector representation of operational data from operations performed during the test workflow.

[0016] Figure 5C This is a block diagram illustrating the use of an algorithm according to one or more embodiments to calculate an uncertainty score generated by testing a sample by a diagnostic laboratory system.

[0017] Figure 6 A block diagram is shown illustrating different sequences of operations that can be performed on a sample during different tests performed by a diagnostic laboratory system, according to one or more embodiments.

[0018] Figure 7 A flowchart is shown of a method for determining the accuracy of a test performed by a laboratory system according to one or more embodiments.

[0019] Figure 8 A flowchart is shown of another method for determining the accuracy of a test performed by a laboratory system, according to one or more embodiments. Detailed Implementation

[0020] Diagnostic laboratory systems perform clinical chemistry and / or laboratory tests to identify analytes or other components in biological samples such as serum, plasma, urine, interstitial fluid, and cerebrospinal fluid. Samples are collected in sample containers and transported to instruments and modules throughout the laboratory system, where they are processed and analyzed. For example, instruments and modules may prepare samples for testing and perform those tests on the samples.

[0021] When samples are received for testing in a diagnostic laboratory system, they may undergo a complex sequence of operations within the workflow for each type of test. Different tests within a single test may require different sequences of operations. In some tests, the operation may begin with sample container handling, for example, the sample container containing the sample being loaded into the laboratory system. Sample container handling may include other operations such as reading the label on the sample container. Subsequent operations may include sample aspiration, reagent aspiration, and dispensing of the sample and / or reagents into test tubes. The final operation in the sequence may include photometric measurements of the liquid in the test tubes to determine the concentration of a chemical or analyte in the sample. Other operations may be performed on the sample and / or sample container. Operations may be performed using one or more instruments or modules configured to perform specific operations.

[0022] Multiple measurements can be performed during each operation. For example, measurements may include instrument performance, the sample, chemicals added to the sample, the sample container, position and / or pressure during aspiration, etc. A single operation may result in one or more measurements. In some embodiments, the instruments and / or modules of a diagnostic laboratory system may be configured to perform quality checks to validate the operation of the diagnostic laboratory system. These quality checks generate measurements of instrument performance.

[0023] The procedures performed during testing are executed sequentially. In some cases, multiple small errors or measurements that happen to fall within the acceptable validity limits associated with individual procedures may accumulate and ultimately lead to inaccurate test results. These inaccurate test results may include underestimating or overestimating the analyte concentration in the sample, which could result in unnecessary treatment prescriptions for patients.

[0024] Unlike traditional laboratory systems, the diagnostic laboratory systems and methods disclosed in this paper jointly analyze a set of operations performed during laboratory testing and correlate an uncertainty score (also known as a “biomarker confidence value”) with each laboratory test based on the cumulative effect of small errors or measurements performed during each operation within the set. The uncertainty score, or biomarker confidence value, indicates whether the overall test result is valid. Therefore, for clinical decision-making, the uncertainty score can be used to determine whether a test result is reliable or whether a retest is necessary, even if the individual operational measurements may be within acceptable limits. The uncertainty score can also indicate whether the instrument is malfunctioning. As described below, the uncertainty score, or biomarker confidence value, is not based on a simple average or median of the validity scores of individual operational measurements.

[0025] As previously described, testing of a diagnostic laboratory system can include multiple sequential operations, and each operation can include multiple measurements. In some embodiments provided herein, vector representations can be created for the measurements of each operation. For example, a vector representation can be an array of measurements that can be obtained by preprocessing the raw measurement data (e.g., normalizing the data, mapping the data to a vector space using dimensionality reduction techniques such as principal component analysis (PCA) or independent component analysis (ICA), using an autoencoder, or other AI algorithms, etc.). The vector representation of each operation represents a "fingerprint" of the operation's dynamics. Subsequently, the entire testing workflow can be represented as a graph, where each node of the graph corresponds to a specific operation of the test. That is, each node of the graph is a vector representation (fingerprint) of the measurements of different test operations within the test operations.

[0026] A graph neural network can be trained to map the aforementioned workflow graph to a compact vector space representing fingerprints of all vector representations of test operations. For example, in some embodiments, a graph autoencoder can be trained for this purpose. In one or more embodiments, the graph autoencoder may include a graph encoder that maps all input operation data (e.g., vector representations of each test operation) to a compact vector space, and a graph decoder that reconstructs the operation data into its original form (e.g., for training purposes). The graph neural network can be trained on operational workflow data of tests obtained from the routine operation of a diagnostic laboratory system. For example, such data can be collected from a deployed and fully operational diagnostic laboratory system. Training can be performed continuously, periodically, or at any suitable time. Training can be performed while the diagnostic laboratory system is online (e.g., in use) or offline.

[0027] After obtaining a compact vector space representation of the operational data for the test, the compact vector space can be used with a neural network or other AI algorithm to estimate test uncertainty (e.g., the probability of test success via a vector representation of a graphically encoded representation from each test operation). Additionally or alternatively, the compact vector space and the neural network can be used to determine an uncertainty score for each operation of the test. In some embodiments, the neural network (or other AI algorithm) used to determine test and / or operational uncertainty based on the operational fingerprint (e.g., vector representation) of the test can be trained on data collected from failed operations or tests performed in a controlled diagnostic laboratory or factory setting. Importantly, in such a controlled setting, tests where each operation falls within acceptable limits (e.g., passes a validity check for that operation) can be marked as failed tests. For example, if all test operations individually produce valid results, but several operations are close to failure, it may be desirable to rerun the test (e.g., if multiple operations are close to an upper or lower limit of the operation's validity, one operation is close to the upper limit while another is close to the lower limit, etc.). Therefore, the neural network can be trained to provide an uncertainty score for the test and to provide guidance on whether a retest is permitted, regardless of whether individual operations in the test passed an internal validity check. Similarly, by training on many vector representations of individual operations within a test, a neural network can be trained to identify the uncertainty score for each operation within a test and / or whether a retest is recommended based on the uncertainty scores of individual operations (and / or any combination of the uncertainty scores of individual operations).

[0028] Example measurements may include pressure sensor measurements obtained during sample and reagent aspiration and dispensing. Other measurements may include, for example, photometric measurements, acoustic measurements, temperature measurements, and optical measurements. In some embodiments, measurements may include the results of quality control algorithms performed by instruments in a laboratory system. Algorithms used herein may include, for example, deep neural networks, generative neural networks, graph neural networks, and other network or AI algorithms.

[0029] refer to Figure 1-8 These and other diagnostic laboratory systems and methods for determining the accuracy or uncertainty of tests are described in more detail.

[0030] refer to Figure 1This diagram illustrates a block diagram of an embodiment of a diagnostic laboratory system 100. The diagnostic laboratory system 100 may include multiple instruments 102 configured to process samples and sample containers 104 (some labeled) and to test the samples (e.g., laboratory tests or other tests). Performing a test may include performing one or more operations on the sample. Each operation may include one or more measurements. As described herein, one or more instruments 102 may include multiple different modules configured to perform the operations described herein. Samples may be various biological samples collected from individuals (such as patients evaluated by a healthcare professional). Samples may be collected in sample containers 104 and delivered to the laboratory system 100, wherein sample containers 104 may be transported through the laboratory system 100 by rail 108, such as to different instruments in the instruments 102. For example, sample containers 104 may be transported by sample carriers 110 (a few labeled). Figure 1 In one embodiment, system 100 has three instruments 102, including a sample processor 114, a first analyzer 116, and a second analyzer 118. Laboratory system 100 may include... Figure 1 The instrument shown is fewer or more.

[0031] In some embodiments, the track 108 may extend near or around the instrument 102, such as Figure 1 As shown. As described herein, a portion or module of instrument 102 may have means for transferring sample container 104 to and from sample carrier 110 and sample container 104, such as a robot. Figure 1 (Not shown in the diagram). Track 108 may include multiple interconnected segments 120 (several are marked). Sample carrier 110 may be movable as shown by dashed lines 126 in the segments 120. In some embodiments, some segments 120 may be integrated with one or more instruments 102.

[0032] A diagnostic laboratory system, such as laboratory system 100, may have numerous instruments and may have tracks that link to other laboratory systems. A laboratory system including laboratory system 100 can simultaneously move and process multiple sample carriers 110 and their corresponding sample containers 104. In some embodiments, laboratory system 100 can simultaneously move and process hundreds or thousands of sample carriers 110 and their corresponding sample containers 104.

[0033] Laboratory system 100 may include or be coupled to computer 130, which is configured to execute one or more programs configured to control laboratory system 100. Computer 130 may be configured to communicate with instrument 102 and other components of laboratory system 100, such as components in a transport system. The transport system may include some or all components (e.g., motors, sensors, power supplies, etc.) configured to transport samples throughout laboratory system 100. Computer 130 may include processor 132, which is configured to execute programs, including programs other than those described herein. These programs may be implemented using computer code.

[0034] Computer 130 may include or have access to memory 134, which may store one or more programs and / or data. Memory 134 and / or programs stored therein may be referred to as non-transitory computer-readable media. Programs may be computer code executable on or by processor 132. Memory 134 may include analysis program 136 configured to analyze operations performed by instrument 102 and / or determine accuracy or uncertainty scores for tests performed by instrument 102, as described herein. Analysis program 136 may include multiple different programs as described herein, including one or more AI algorithms (e.g., graph neural network 137, other generative neural networks, other deep networks, or AI algorithms, including supervised, semi-supervised, or unsupervised AI models, etc.). In some embodiments, analysis program 136, portions of analysis program 136, or copies of analysis program 136 may reside in the respective instrument 102 or at a location outside the diagnostic laboratory system 100.

[0035] Computer 130 can be coupled to workstation 138, which is configured to allow a user to interface with laboratory system 100. Workstation 138 may include monitor 140, keyboard 142, and other peripherals. Analysis program 136 or other programs may cause monitor 140 to display the results of data analysis, including uncertainty scores (e.g., laboratory biomarker confidence values), test validity scores, and instructions on whether the test should be rerun. Thus, computer 130, coupled with workstation 138, can be configured to generate notifications of test accuracy, such as uncertainty scores.

[0036] The analysis program 136 can perform a variety of functions. In some embodiments, the analysis program 136 can be combined with other program operations to perform functions. For example, the analysis program 136 can be configured to detect operational failures in the laboratory system 100. If any operational failure of the laboratory system 100 goes undetected, the undetected failure can have a significant impact on clinical decision-making. For example, a test resulting from a failed procedure may be inaccurate, which could lead to inaccurate treatment by medical professionals who rely on the result.

[0037] When samples are received for testing in laboratory system 100, they undergo a complex sequence of operations or procedures within a specific workflow defined by the particular test. Each type of test may have a unique sequence of operations. The workflow sequence may begin with sample handling or sample container handling operations, followed by the aspiration and dispensing of the sample and / or reagents into test tubes. The mixture in the test tubes may undergo additional operations required for the test. The workflow sequence may end with a measurement operation, such as a photometric measurement, to determine the chemical properties of the sample.

[0038] Each operation in the workflow sequence can be performed using one or more instruments 102. Because the operations occur sequentially, small instrument errors and / or measurements associated with one or more operations, within but close to acceptable limits, may accumulate and lead to larger errors in the resulting test. In some embodiments, the diagnostic laboratory system 100 collectively analyzes the set of sequential operations performed during testing and determines an uncertainty score for the test results performed within the laboratory system 100. For example, each uncertainty score may be based on an end-to-end validity check of the operations or set of operations performed during testing to determine the accuracy of the test. In some embodiments, the uncertainty score can be used to determine whether a retest is needed to obtain a valid or more accurate test result.

[0039] For further reference Figure 2 It shows a block diagram of instrument 202, illustrating the modules and / or components associated with instrument 202, and the associated operations that can be performed by instrument 202. (For example, instrument 202 may resemble...) Figure 1(One of the instruments 102.) In some embodiments, modules and / or components of instrument 202 may be within instrument 202 (e.g., not separate units occupying a separate area). In one or more embodiments, instrument 202 may include a robotic handler 210 configured to grasp and move sample containers 104, sample carriers 110, and / or other containers (vials, test tubes, etc.) within instrument 202. The robotic handler 210 may operate in conjunction with an internal transport system 212 configured to transport sample containers 104, carriers 110, and / or other containers to specific locations within instrument 202 via internal tracks. The internal transport system 212 may be connected to tracks 108 of laboratory system 100 to receive and return sample containers 104, carriers 110, and / or other containers.

[0040] In some embodiments, instrument 202 may include a reagent storage 214. The reagent storage 214 may be located within a module of instrument 202, accessible by components of a suction and dispensing module 216 configured to suction and dispense reagents and samples. A photometric analyzer 218 can perform photometric analysis on a sample with or without the addition of one or more reagents. A quality control procedure 220 may perform self-tests and other analyses to determine whether modules (e.g., modules 210-218) are performing correctly and / or the likelihood of accurate operational results from instrument 202. The quality control procedure 220 may be integrated with analysis procedure 136 (… Figure 1 They can be operated together, and / or the quality inspection measurement results can be transmitted to the analysis program 136.

[0041] For further reference Figure 3 The diagram illustrates a block diagram of an embodiment of the suction and dispensing module 216. The suction and dispensing module 216 can be installed in one or more instruments 102 ( Figure 1 ) or instrument 202 ( Figure 2 This can be implemented in [the instrument]. Other embodiments of the suction and dispensing module 216 can be used in instrument 102 and / or instrument 202. Figure 3 The embodiment shows a sample container 304 located in a carrier 310, which shows a sample container 104 and a carrier 110 ( Figure 1 The carrier 310 may have been already connected to the internal transmission system 212. Figure 2 The sample is transferred to the aspiration and dispensing module 216. The sample container 304 may contain samples to be dispensed by instruments 202 and / or other instruments in instrument 102. Figure 1 ) Sample 306 for analysis, processing and / or testing.

[0042] The aspiration and dispensing module 216 may include reagent 312 stored in reagent pack 314. As described in more detail herein, components of the aspiration and dispensing module 216 may aspirate reagent 312 from reagent pack 314 and sample 306 from sample container 304. In some embodiments, the aspiration and dispensing module 216 may include a tip dispenser 316 configured to change the tip of the aspiration detector as described herein prior to an aspiration operation. The aspiration and dispensing module 216 may have a test tube 320 configured to receive aspirated portions of reagent 312 and sample 306 via a dispensing operation. In some embodiments, the contents of the test tube 320 may undergo a process by a photometric analyzer 218 (… Figure 2 Photometric analysis performed.

[0043] The suction and dispensing module 216 may include a robot 331 configured to move the suction tube assembly 332 within the suction and dispensing module 216. Figure 3 In this embodiment, the detector 334 of the pipette assembly 332 is shown prepared to aspirate reagent 312 from reagent pack 314. The detector 334 is shown having a tip 322 attached to one end of the detector 334. The tip 322 may have already been placed on the detector 334, such as by a tip dispenser 316 before aspirating reagent 312. A new tip can be placed on the detector 334 by a tip-changing operation, such as by using the tip dispenser 316 before aspirating reagent 312 or sample 306.

[0044] Figure 3 The image shows a sample container 304 without a cap, the cap possibly having been removed by a cap removal module (not shown) in instrument 202 or by another module (not shown) in laboratory system 100 that performs the cap removal operation. Removing the cap allows sample 306 to be aspirated. Pipette assembly 332 can be configured to position detector 334 using robot 331 to aspirate and dispense reagent 312 and sample 306. By moving detector 334 to the appropriate position and performing the dispensing operation, reagent 312, other reagents, and a portion of sample 306 can be dispensed into a reaction vessel, such as test tube 320. Test tube 320 may be made of a light-transmitting material and is used in photometric analyzer 218 (…). Figure 2 Photometric analysis, as described in this article.

[0045] Some components of the suction and dispensing module 216 can be electrically coupled to the computer 330. Figure 3 In one embodiment, computer 330 may include processor 330A and memory 330B. Program 330C may be stored in memory 330B and may be executed by processor 330A. In other embodiments, computer 330 and / or components of computer 330 may be in computer 130 ( Figure 1 Implemented in ). One of the programs 330C can be quality inspection program 220 ( Figure 2 The computer 330 may also include a suction / dispensing controller 330D and a position controller 330E, which may be controlled by a program, such as a program 330C stored in memory 330B. In some embodiments, the position controller 330E and / or the suction / dispensing controller 330D may be implemented in a separate device (e.g., another computer). The program 330C may include algorithms for controlling and / or monitoring the components within the suction and dispensing module 216. In some embodiments, the algorithm may include the position controller 330E and / or the suction / dispensing controller 330D.

[0046] The robot 331 may include one or more arms and motors configured to move the straw assembly 332 within the suction and dispensing module 216. Figure 3 In one embodiment, robot 331 may include an arm 350 coupled between a first motor 352 and a suction assembly 332. The first motor 352 may be electrically coupled to computer 330 and may receive instructions generated by position controller 330E. The instructions may instruct the first motor 352 to move in a specific direction and speed. The first motor 352 may be configured to move arm 350 to enable detector 334 to aspirate and / or dispense sample 306 and / or reagent 312, as described herein. The first motor 352 may include or be associated with position sensor 352A, which is configured to generate measurements (e.g., sensor data) indicating the position of arm 350. The measurement data generated by position sensor 352A may be transmitted to computer 330 and / or computer 130 (…). Figure 1 ), and can be used by the analyzer 136, as described herein.

[0047] A second motor 354 may be coupled between the arm 350 and the pipette assembly 332 and may be configured to move the detector 334 in a vertical direction (e.g., the Z direction) to aspirate and / or dispense liquid as described herein, and replace the tip 322. The second motor 354 may move the detector 334 in response to instructions generated by program 330C. For example, the second motor 354 may enable the detector 334 to enter and exit the sample container 304, test tube 320, tip dispenser 316, and / or reagent kit 314. The second motor 354 may include or be associated with a current sensor 354A, configured to measure the current drawn by the second motor 354. Measurement or sensor data (e.g., measured current) generated by the current sensor 354A may be transmitted to computer 330 and / or computer 130. Figure 1 ), and can be used by analyzer 136, as described herein.

[0048] The suction and dispensing module 216 may include multiple position sensors configured to generate measurements related to the position of the component. Figure 3 In some embodiments, position sensor 356 may be mechanically coupled to robot 331. In some embodiments, position sensor 356 may be coupled to other components in suction and dispensing module 216. Position sensor 356 may be configured to sense the position of one or more components of robot 331 or other components within suction and dispensing module 216 (e.g., straw assembly 332). Figure 3 In some embodiments, position sensor 356 may measure the position of arm 350, straw assembly 332, and / or detector 334. Measurements (e.g., position data) may be transmitted to computer 330 and / or computer 130 for processing by analysis program 136, as described herein.

[0049] The suction and dispensing module 216 may also include a pump 360 mechanically coupled to conduit 362 and electrically coupled to suction / dispensing controller 330D. During suction operation, pump 360 may generate a vacuum or negative pressure (e.g., suction pressure) in conduit 362. During dispensing operation, pump 360 may generate a positive pressure (e.g., dispensing pressure) in conduit 362.

[0050] Pressure sensor 364 can be configured to measure pressure in conduit 362 and generate a measurement indicating the pressure (e.g., pressure data). In some embodiments, pressure sensor 364 can be configured to measure suction pressure and generate a pressure measurement. In some embodiments, pressure sensor 364 can be configured to measure dispensing pressure and generate a pressure measurement. For example, the pressure measurement can be in the form of a pressure trajectory as a function of time, as referenced below. Figure 4 The pressure measurement can ultimately be transmitted to computer 130 (as described above). Figure 1 (and / or computer 330, for processing by analysis program 136. The pressure trajectory may be a function of time or may change when one or more components of the suction and dispensing module 216 are replaced or fail.)

[0051] For further reference Figure 4 , Figure 4 This is a graph illustrating an example of the pressure trajectory 400 of the straw assembly 332 as a function of time, measured by pressure sensor 364. Figure 4 In this embodiment, pressure trajectory 400 illustrates the pressure in the straw assembly 332 during tip pickup, aspiration, and dispensing operations. The pressure increases slightly as the tip 322 is replaced and decreases significantly during aspiration. Then, during the dispensing operation, the pressure increases significantly. Pressure trajectory 400 represents one or more measurements that can be analyzed by analysis procedure 136, as described herein.

[0052] Refer again Figure 3 The suction and dispensing module 216 may include an imaging device 366 configured to capture images of the detector 334 and / or the liquid within the detector 334. For example, the detector 334 may be transparent, thus allowing the imaging device 366 to capture images of the liquid located within the detector 334. The captured images may include image data, which is transmitted to computer 330 and / or computer 130. Figure 1 The image data may include measurements generated during the imaging operation, such as during photometric analysis. Program 330C (or...) Figure 1 Analysis program 136 can analyze image data to determine the mass of the liquid in detector 334. For example, program 330C or analysis program 136 can determine whether the liquid in detector 334 contains bubbles or other anomalies.

[0053] As described herein, one or more modules or components of the instrument, such as the suction and dispensing module 216, may include one or more sensors that can be monitored by one or more programs (such as program 330C). Example sensors include position sensors, pressure sensors, imaging sensors, etc. Programs such as program 330C may also perform quality check (e.g., self-test) routines on the sensors. This information can be provided to computer 130 and / or analysis program 136. Figure 1 As further described below, the data generated by the self-test routines and sensors includes measurements that can be encoded into a vector space and / or analyzed by one or more AI algorithms, such as those generated by analysis program 136 ( Figure 1 The analysis of computer program code in the test, for example, a first AI algorithm can generate a dynamic fingerprint of each operation of the test via a vector representation of the operational data (e.g., sensor measurements, self-test measurements, etc.), while a second AI algorithm can analyze the vectors (e.g., fingerprints) of various operations of the test to determine the accuracy of the test, such as by calculating or otherwise determining an uncertainty score (e.g., a biometric confidence value).

[0054] More specifically, the analysis procedure 136 may employ AI algorithms to analyze the cooperating data generated by the instrument 102. Based on this analysis, the analysis procedure 136 may calculate an uncertainty score (e.g., a biomarker confidence value), which is an indication of the validity or accuracy of a test performed on a sample such as sample 306. In some embodiments, instrument measurements may include, but are not limited to, pressure sensor measurements obtained during sample and reagent aspiration and dispensing, such as those obtained by pressure trajectory 400 (…). Figure 4As shown in the figure. Other measurements may include location data, image data, photometric measurements, acoustic measurements, temperature measurements, optical measurements, quality checks, or self-testing quantities.

[0055] In some embodiments, measurements obtained during each operation of the test are encoded as specific fingerprints (e.g., vectors) representing the dynamics of the operation. A first AI algorithm trained with learned correlations between operations (e.g., learned correlations between a first operation and a second operation) can then map the fingerprint vector representations to a compact vector space. One or more other AI algorithms are configured to jointly analyze the compact vector space to determine the accuracy of the test, such as by calculating an uncertainty score. In some embodiments, if the uncertainty score is below a predetermined value, computer 130 ( Figure 1 It can generate notifications that tests are invalid and / or may need to be retested.

[0056] Figure 5A An example method 500 is shown, according to one or more embodiments, for determining an uncertainty score for a test of a diagnostic laboratory system and / or for determining whether a sample should be retested. Figure 5B An example graph 502 is shown illustrating a test workflow processed by a graph neural network 504 and an additional AI algorithm (e.g., a neural network 506) according to embodiments provided herein, which generates an uncertainty score 508 from a compact vector representation 510 of operational data from operations performed during the test workflow (e.g., operations O1-O6).

[0057] refer to Figure 5A In block 512, measurements are obtained for each operation within the test. For example, pressure, temperature, photometry, acoustic or other parameters, self-test or quality check measurements, etc., may be obtained for each operation of the test (e.g., provided to computer 130 and / or analysis program 136). Example operations include sample container handling operations, sample aspiration and dispensing, reagent aspiration and dispensing, photometric measurements to determine chemical concentration and / or assay type, and / or any other testing operations.

[0058] In block 514, a vector representation can be created for the measurement of each test operation. For example, the vector representation can be an array of measurements obtained by preprocessing the raw measurement data (e.g., normalizing the data, mapping the data to a vector space using dimensionality reduction techniques such as principal component analysis (PCA) or independent component analysis (ICA), using an autoencoder or other AI algorithm, etc.). The vector representation of each operation represents a dynamic "fingerprint" of the operation.

[0059] Subsequently, in block 516, a workflow diagram of the test can be created. Specifically, the entire test workflow can be represented as a graph, where each node of the graph corresponds to a specific operation of the test. That is, each node in the graph is a vector representation (fingerprint) of the measurement of different test operations within the test operations. For example, Figure 5B Figure 502 illustrates an example test workflow with six operations (labeled O1-O6). Other numbers and / or sequences of operations can be used. Measurements performed during each operation O1-O6 are encoded in vector representations corresponding to the nodes in Figure 502 (e.g., nodes N1-N6 respectively) (see also the description below). Figure 6 ).

[0060] In block 518, a graph neural network is used to map the workflow graph to a compact vector space. For example, a graph neural network can be trained to map a test workflow graph to a compact vector space of fingerprints representing all vector representations of test operations. Figure 5B In this embodiment, a graph neural network 504 has been trained to map the workflow graph 502 to a compact vector space 510. In some embodiments, a graph autoencoder may be trained for this purpose. In one or more embodiments, the graph autoencoder may include a graph encoder that maps all input operation data (e.g., vector representations of each test operation) to a compact vector space, and a graph decoder that reconstructs the operation data to its original form (e.g., for training purposes). Figure 5B In this process, the graph encoder 517 maps graph 502 to a compact vector space 510, and the graph decoder 517' reconstructs graph 502 (as graph 502'). The graph neural network can be trained on operational workflow data used for testing, such as operational workflow data obtained from the daily operation of a diagnostic laboratory system. For example, such data can be collected from a deployed and fully operational diagnostic laboratory system. Training can be performed continuously, periodically, or at any suitable time. Training can be performed while the diagnostic laboratory system is online (e.g., in use) or offline.

[0061] After obtaining a compact vector space representation of the test operation data, in block 520, the compact vector space can be used with a neural network or other AI algorithm to estimate test uncertainty (e.g., calculating a test uncertainty score, such as the probability of test success, from the graphically encoded vector representation of each test operation). Additionally or alternatively, in block 522, the compact vector space and a neural network (or other AI algorithm) can be employed to determine the uncertainty score for each operation of the test. For example, in Figure 5BIn this context, a neural network 506 can be trained for this purpose. In some embodiments, a neural network (or other AI algorithm) for determining test and / or operational uncertainty based on the operational fingerprint (e.g., vector representation) of the test can be trained on data collected from failed operations or tests performed in a controlled diagnostic laboratory or factory setting. Importantly, in such a controlled setting, tests where each operation falls within acceptable limits (e.g., passes a validity check for that operation) can be marked as failed tests. For example, if all test operations individually produce valid results, but several operations are close to failure, it may be desirable to rerun the test (e.g., if multiple operations are close to an upper or lower limit of operational validity, one operation is close to the upper limit while another is close to the lower limit, etc.). Therefore, a neural network can be trained to provide an uncertainty score for the test and provide guidance on whether a retest is permitted, regardless of whether individual operations within the test passed an internal validity check. Similarly, by training on numerous vector representations of individual operations within the test, a neural network can be trained to identify the uncertainty score for each operation within the test and / or whether a retest is recommended based on the uncertainty scores of individual operations (and / or any combination of individual operation uncertainty scores).

[0062] Any suitable neural network can be used. Example architectures include Inception, ResNet, ResNeXt, DenseNet, etc., although other CNN architectures can also be used.

[0063] In block 524, a determination regarding whether to retest can be made and / or provided to the user, based on the uncertainty score of the overall test workflow and / or based on one or more uncertainty scores of individual operations. In some embodiments, method 500, graph neural network 504, and / or neural network 506 can be implemented in computer 130, memory 134, and / or analysis program 136 (e.g., as computer program code), and analysis program 136 can make retest recommendations and / or perform retests.

[0064] As described above, in some embodiments, a neural network 506 (or another AI algorithm) for determining test and / or operational uncertainty based on a fingerprint (e.g., vector representation) of the test operation can be trained on data collected from failed operations or tests performed in a controlled diagnostic laboratory or factory setting. For example, in a controlled diagnostic laboratory, it can be determined that in certain situations, creatinine concentration tests can produce creatinine concentration values ​​with a variance of + / - 0.1 mg / dL. The neural network 506 can be trained to provide an estimated variance of the creatinine concentration, and in some embodiments, a retest is recommended based on this variance. For example, if a test produces a creatinine concentration value of 0.5 mg / dL + / - 0.1 mg / dL, the neural network 506 can suggest that for such a low value, the creatinine concentration should be retested, even if each operation of the creatinine concentration test produces a measurement within the effective range (and / or each operation passes its own internal self-check or quality check). Additionally or alternatively, the neural network 506 can be trained to generate uncertainty scores representing confidence levels (such as 50%, 70%, 90%, etc.) based on the operational fingerprint of the test. In some embodiments, a creatinine concentration value of 0.5 mg / dL with 50% confidence can be labeled for retesting, even if each operation of the creatinine concentration test produces a measurement within the effective range.

[0065] In another example embodiment, the test may include obtaining a first measurement during a first operation of the test, wherein the first measurement has a value below but close to the upper limit of validity for the first measurement. For example, the first measurement may be a reagent aspiration pressure below but close to the upper limit of validity. The test may also include obtaining a second measurement during a second operation of the test, wherein the second measurement value has a value above but close to the lower limit of validity for the second measurement. For example, the second measurement value may be a sample aspiration pressure above but close to the lower limit of validity. Detailed analysis in a controlled or factory setting may indicate that, in this case, a retest is recommended. The neural network 506 may be trained to provide a low (e.g., failure) uncertainty score in this case. Specifically, a model (e.g., neural network 506) trained using the correlation learned between the first and second operations may be used to jointly analyze the first and second measurements, and the neural network 506 may provide a failure uncertainty score for the test based on the joint analysis.

[0066] Neural networks or other AI algorithms (e.g., neural network 506) can be trained to determine whether an individual operation has succeeded or failed, and / or, in some embodiments, to provide an uncertainty score for the individual operation. For example, in one embodiment, the uncertainty score for an individual operation can be obtained by training an ensemble of neural networks, where each network is trained to provide a “probability” of whether the operation is successful. Given a set of probabilities from the network ensemble, the final decision about whether the operation is successful or failed can be based on a majority or average probability score. Furthermore, the variance of the probabilities from the network ensemble can be used to estimate the uncertainty (and / or uncertainty score) of the individual operation. If the variance is high, it means that the networks in the ensemble are inconsistent on whether the operation is successful, and therefore the uncertainty (and uncertainty score) will be high, and vice versa. For example, an ensemble of three neural network models can be used to evaluate the same operation. In some embodiments, the neural networks can be different types of networks and / or differently trained neural networks. Suppose network models 1, 2, and 3 output probabilities of success of 1, 0.8, and 0.1, respectively (1 being a high probability of success). Using the majority approach, the operation would be reported as successful. However, the evaluated uncertainty score is high because the variance is high (e.g., a standard deviation of 0.47). If the same network ensemble outputs 1.0, 0.8, and 0.9, the uncertainty score will be low (e.g., a standard deviation of 0.1). In another embodiment, the neural network can output a probability of success (or uncertainty score) proportional to the network's confidence level. For example, a neural network with a Gaussian process classification layer can be used (e.g., see Amersfoort et al., "On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertainty", Arxiv, arXiv.2102.11409, February 22, 2021, https: / / arxiv.org / abs / 2102.11409). In yet another embodiment, the neural network's output can be explicitly calibrated to be proportional to the confidence level (e.g., outputting the probability of success and / or the uncertainty score of a single process), such as by using Pratt calibration or a similar algorithm.

[0067] Therefore, neural networks or other AI algorithms can be trained to provide an uncertainty score for a test and to provide guidance on whether a retest should be permitted, regardless of whether an individual operation in the test passes or fails an internal validity check. Similarly, by training on numerous vector representations of individual operations within a test, a neural network can be trained to identify the uncertainty score for each operation within the test and / or whether a retest should be recommended based on the uncertainty scores of individual operations (and / or any combination of the uncertainty scores of individual operations).

[0068] For further reference Figure 1 and Figure 5C , Figure 5C This is a block diagram 520 illustrating the use of an AI algorithm to calculate an uncertainty score 545 according to one or more embodiments. Figure 5C In an embodiment, the diagnostic laboratory system 100 may use several operations 530 or procedures to test sample 306. Figure 3 Measurements from each operation 530 can be encoded as a specific fingerprint or vector representing the individual operation. Figure 5C In this embodiment, operation 530 includes sample handling 532, sample aspiration 534, reagent aspiration 536, and photometric analysis 538. Laboratory system 100 can perform other operations. Each operation 530 can be performed in one or more instruments 102. In operation block 540, measurements, such as test measurements and measurements generated by a self-test program, can be received.

[0069] The measurements generated by sample processing 532 can be in the form of a system log, which records the actions performed during sample processing and any anomalies that occur during sample processing 532. The measurements generated by sample processing 532 can also take other forms. For example, measurements may include the pressure applied to the gripper to grip sample container 104, the weight of sample container 104, identification information in the form of image data, and other measurements.

[0070] Sample aspiration 534 and reagent aspiration 536 can generate information about Figure 3 The suction and dispensing module 216 describes the measurements or data. Measurements may include, for example, the measurements or data described. Figure 4 The pressure trajectory 400 shows the pressure trajectory and sensor measurements from the sensor. Measurements can be correlated with other operations, such as image data indicating whether the aspirated liquid contains air bubbles. The photometric analysis 538 can generate signal trajectories or other types of measurements or data typically generated by a photometric analyzer.

[0071] Measurements from operation block 540 can be received in operation block 542, where a specific fingerprint representing the dynamics of operation 530 is generated. The fingerprint can be encoded as a vector, such as a compact vector. This vector representation can be achieved through preprocessing by one or more instruments 102. Figure 1 An array of measurements obtained from the raw data generated during the execution of operation 601. In some embodiments, preprocessing may include normalizing the data on the measurement set within a feasible or predetermined range of measurements. In other embodiments, preprocessing may involve projecting the raw data into a vector space using dimensionality reduction techniques such as principal component analysis (PCA), independent principal component analysis (ICA), or an autoencoder.

[0072] In some embodiments, AI, such as deep networks, generative neural networks, and other trained models, can be used to generate vectors. In some embodiments, each vector can represent a separate operational validity score of a measurement received from operation 530.

[0073] The vector generated by operation block 542 can be analyzed by one or more AI algorithms in operation block 544 to determine the accuracy of the test. For example, one or more AI algorithms in operation block 544 can generate an uncertainty score. In some embodiments, operation block 544 can use a graph neural network (GNN) plus additional AI algorithms (e.g., neural networks) to compute the uncertainty score, as previously referenced. Figure 5A and 5B Therefore, one or more AI algorithms can learn combined representations of heterogeneous data or measurements from modules and / or operations, and estimate confidence scores associated with tests. The methods described herein, such as through analysis procedure 136, can be applied to determine uncertainty scores (e.g., accuracy) for the entire set of operations performed during laboratory testing.

[0074] Figure 5C The process described in [the document] can be applied to [various applications], including [those with reference to] [other applications]. Figure 3 and Figure 4 The described operations involve sample testing to analyze aspiration and dispensing operations in addition to other operations. Because invalid operations, such as invalid aspiration, increase the likelihood that subsequent dispensing operations will be invalid and increase the uncertainty of the test, joint analysis is more suitable for generating accurate uncertainty scores than analyzing each operation individually. Another example relates to the sample and reagent volumes used in the test. For example, the aspirated sample volume may be lower than expected but within predetermined limits, and the aspirated reagent volume may be larger than expected but within predetermined limits. When analyzed individually, the test will show a high probability of being valid. However, even if the individual volumes are within their respective predetermined limits, the analyte concentration calculated by the test may be inaccurate. Therefore, for example, an uncertainty score lower than a predetermined value can be calculated, indicating that the associated test should be rerun. In some embodiments, an uncertainty score lower than a predetermined value may cause computer 130 (executing analysis program 136) to automatically reschedule the associated test.

[0075] The following will describe the further testing procedures and descriptions using Graphical Neural Networks (GNNs). References Figure 2 Instrument 202 can be configured to perform a variety of different tests. For example, some tests can be performed using reagents, and some tests can be performed without reagents. See also... Figure 6The diagram illustrates block diagram 600 (also known as Figure 600), which shows different sequences of operations 601 that can be performed on a sample to perform different tests or different types of tests according to one or more embodiments. Depending on the test being performed, different instruments can perform different operations 601, and different sequences of operations 601 can be performed. A sequence of operations 601 for a single test can be referred to as a separate workflow. Therefore, block diagram 600 shows a graphical representation of the workflow.

[0076] One or more operations 601 can be a graphical representation of the test workflow (e.g., Figure 5B The nodes of Figure 600 or Figure 502, and the paths between operations 601 can be edges of the graph. Each operation 601 can generate a measurement, which can be encoded as a vector or compact vector (e.g., a reduced-dimensional vector) as described above. The GNN can map the vector representation fingerprint to a compact vector space (e.g., compact vector space 510), and another AI algorithm (e.g., neural network 506) can jointly analyze the measurement (via the fingerprint vector representation) to determine the accuracy of the test (e.g., uncertainty score).

[0077] Figure 600 illustrates the workflow, including a sequence of operations for different tests that instrument 202 can perform. For example, the first operation could be tip pickup 602, where detector 334 ( Figure 3 The tip can be replaced before aspirating the sample or reagent (e.g., tip 322-). Figure 3 Tip pickup 602 may include moving detector 334 to tip dispenser 316 and replacing tip 322. Following tip pickup 602, processing may proceed to sample aspiration 604 or reagent aspiration 606. If no reagent is added to the sample, for example, processing may proceed directly to sample aspiration 604. Paths from sample aspiration 604 may extend to sample dispensing 608, and paths from reagent aspiration 606 may extend to reagent dispensing 610. In some embodiments, sample aspiration 604 and sample dispensing 608 may be a single operation or a single node in a graph. In some embodiments, reagent aspiration 606 and reagent dispensing 610 may be a single operation or a single node in a graph. Paths from both sample dispensing 608 and reagent dispensing 610 may extend to photometric analysis 612. Paths from reagent dispensing 610 may extend back to tip pickup 602 before adding new liquid (e.g., new reagent) to tube 320. Other embodiments of Figure 600 may include different paths depending on the instrument configuration and the test the instrument is configured to perform.

[0078] Different tests or different types of tests can have different workflows, such as different paths or edges from start to finish. For example, a first test with a first workflow can begin with tip pickup 602, followed by sample aspiration 604. After sample aspiration 604, the test can proceed to sample dispensing 608, followed by photometric analysis 612. A second test can have a second workflow and can begin with tip pickup 602, followed by sample aspiration 604. After sample aspiration 604, the test can proceed to sample dispensing 608, followed by tip pickup 602 receiving detector 334 (…). Figure 3 The new tip on the ) (e.g., tip 322-) Figure 3 The test can proceed to reagent aspiration 606, reagent dispensing 610, and can end with photometric analysis 612. A third test can have a third workflow and begin with tip pickup 602, followed by sample aspiration 604. After sample aspiration 604, the test can continue to sample dispensing 608, followed by tip pickup 602 to receive a new tip on detector 334. The test can proceed to reagent aspiration 606, then reagent dispensing 610, and back to tip pickup 602. When a new tip is received on detector 334, the test can proceed to reagent aspiration 606 and reagent dispensing 610 to add new reagent to test tube 320. This cycle can continue to add new reagent to test tube 320. The test can then terminate with photometric analysis 612.

[0079] Measurements from each operation 601 can be encoded into vectors, as shown regarding operation block 542. Figure 5C As described in Figure 600, the nodes in Figure 600 can then be analyzed by a GNN, and then by additional AI algorithms, such as those relating to operation block 544. Figure 5C As described in the description. In some embodiments, the AI ​​algorithm can be a neural network. The workflow of each test (e.g., path or edge) can be analyzed, such as regarding the calculation of the uncertainty score 545 ( Figure 5C As described in the paper, AI algorithms can also generate uncertainty scores of 545.

[0080] More specifically, in some embodiments, the analysis program 136 can obtain measurements of individual operations within the jointly performed test operation 601. The analysis program 136 can encode the measurements generated during each operation 601 into vector representations. For example, the vector representations can be obtained through preprocessing by one or more instruments 102 (…). Figure 1The array of measurements obtained from the raw data generated during the execution of operation 601. In some embodiments, preprocessing may involve normalizing the data on the measurement set within a feasible or predetermined range of measurements. In other embodiments, preprocessing may involve projecting the raw data into a vector space using dimensionality reduction techniques such as principal component analysis (PCA), independent principal component analysis (ICA), or an autoencoder.

[0081] Analysis procedure 136 can then use GNNs and / or other neural networks to learn the operational manifold of instrument 102 and / or laboratory system 100. The operational manifold can be a workflow used for the different tests described herein. An autoencoder, such as a variational graphical autoencoder (which may include a graphical decoder), or other algorithms can be trained to map all input operational data (e.g., measurements acquired during testing) to a compact vector space or other vector space. A graphical decoder or other decoder can be trained to reconstruct the operational data from the compact vector space or other space (for training purposes). In some embodiments, a model including an encoder, decoder, and / or neural network can be learned from laboratory system 100 ( Figure 1 The system is trained on a large amount of operational workflow data and / or measurements obtained from the daily operations of the system. The edges (paths) of the graph (graphics) 600 can be guided to represent the order or sequence of operations performed for each type of test. Thus, the sequence models the causal structure of the workflow used to perform the tests. The above process can be implemented in the AI ​​algorithm in the analysis program 136, and it can enable the operational workflow data to be projected onto a vector representation.

[0082] The models(s) implemented in analysis procedure 136 can then be used to detect operational anomalies or identify operational malfunctions in specific tests. Detection of operational anomalies can be performed using a compact vector space generated by a trained GNN and constrained to a Gaussian or Gaussian mixture distribution. Operational instances projected further away, such as using Mahalanobis distance, can be considered anomalous and can be attributed to operational anomalies. Mahalanobis distance is a multivariate distance metric that measures the distance between a point and a distribution.

[0083] Training of an AI algorithm that generates uncertainty scores from a compact vector space generated by a GNN can be performed in a controlled laboratory or factory setting, where data corresponding to failed operations and / or tests can be obtained. This data can then be combined with a large amount of data corresponding to successful operations and / or tests to train the neural network to estimate the probability of an operation's success based on the graph-encoded vector representation used in the GNN. Along with the calculated probability scores, the neural network can also estimate a validity score associated with each operation, which is related to the operation as a source of high or low uncertainty scores. Equipped with this training, analysis program 136 or other programs can recommend retesting, or the user can determine whether retesting is necessary based on the uncertainty scores. The state of the operation can also be logged in a machine log, such as stored in memory 134. Figure 1 The logs in the database can be used to determine whether a particular operation results in a sustained low uncertainty score, and whether the corresponding module needs to be revised, maintained, or replaced.

[0084] Examples of the methods and apparatus disclosed herein can be illustrated by a test performed by at least one instrument 102, which involves adding a reagent to a sample followed by photometric analysis. Reference Figure 6 The testing workflow has the following sequence: tip pickup 602, sample aspiration 604, sample dispensing 608, tip pickup 602, reagent aspiration 606, reagent dispensing 610, and photometric analysis 612. Measurements are obtained during each operation in the workflow. For example, during tip pickup 602, measurements may include... Figure 4 The pressure measurement (trajectory) shown is accompanied by position sensor measurements generated by position sensors 352A and 356. These measurements can be encoded into vectors. Measurements generated by other operations in the workflow can also be encoded into vectors. In some embodiments, operation block 542 can generate vectors. A generative adversarial network (GAN), GNN, or other network or model trained on the testing workflow can analyze the vectors to generate a compact vector space representation of the operation fingerprint of the test, which is analyzed by another AI algorithm (e.g., neural network 506) to determine an uncertainty score 545. Based on the uncertainty score 545, computer 130 can suggest a retest or indicate that the test is valid. For example, information about the uncertainty score 545 can be output to display 140.

[0085] Now for reference Figure 7 , Figure 7This is a flowchart illustrating a method 700 for determining the accuracy of a test performed by a laboratory system (e.g., laboratory system 100) according to one or more embodiments. Method 700 includes, at block 702, obtaining one or more first measurements during a first operation performed by the laboratory system, wherein the first operation is one of a plurality of operations for performing a first test on a sample (e.g., sample 306). Method 700 includes, at block 704, obtaining one or more second measurements during a second operation performed by the laboratory system, wherein the second operation is one of a plurality of operations for performing the first test. Method 700 includes, at block 706, jointly analyzing the one or more first measurements and the one or more second measurements using a model (e.g., graph neural network 504 and / or neural network 506) trained based on a learned correlation between the first and second operations. Method 700 includes, at block 708, calculating an uncertainty score for the first test based on the analysis. And method 700 includes, at block 710, determining whether to rerun the first test based on the uncertainty score. For example, analysis procedure 136 may warn the user to rerun the first test. Method 700 may optionally include rerunning the first test in response to a determination made in block 710, such as when the calculated uncertainty score is less than a predetermined value. In some embodiments, laboratory system 100 (e.g., computer 130 executing analysis program 136) may automatically initiate the rerun of the first test.

[0086] Now for reference Figure 8 , Figure 8 This is a flowchart illustrating a method 800 for determining the accuracy of tests performed by a laboratory system (e.g., laboratory system 100) according to one or more embodiments. Method 800 includes, at block 802, generating a graphical representation of the workflow of the tests that can be performed by the laboratory system, wherein the graphical representation includes multiple nodes and is configured for analysis by a graph neural network (e.g., see...). Figure 5A Test workflow diagram 502 or Figure 6The testing workflow diagram is shown in 600. Method 800 includes, in block 804, obtaining one or more first measurements from a first operation performed by the lab system during testing. Method 800 includes, in block 806, converting the one or more first measurements into a first vector, wherein the first vector is a first node of a graphical representation. Method 800 includes, in block 808, obtaining one or more second measurements from a second operation performed by the lab system during testing. Method 800 includes, in block 810, converting the one or more second measurements into a second vector, wherein the second vector is a second node of a graphical representation. Method 800 includes, in block 812, using a graphical neural network to jointly analyze the first and second nodes. For example, graphical neural network 504 may generate a compact vector space representation of the graphical representation (e.g., graphical 502 or 600). Method 800 includes, in block 814, determining an uncertainty score for the test based on the analysis. As described, in some embodiments, the compact vector space may be fed into a trained neural network (e.g., neural network 506) to determine the uncertainty score for the test. Method 800 includes, in block 816, determining whether to rerun the test based on the uncertainty score.

[0087] While this disclosure is susceptible to various modifications and alternatives, specific methods and apparatus embodiments have been illustrated by way of example in the accompanying drawings, and are described in detail herein. However, it should be understood that the specific methods and apparatus disclosed herein are not intended to limit this disclosure, but rather to cover all modifications, equivalents, and alternatives falling within the scope of the claims.

Claims

1. A method for determining the accuracy of a test performed by a diagnostic laboratory system, comprising: One or more first measurements are obtained during a first operation performed by the diagnostic laboratory system, wherein the first operation is one of a plurality of operations for performing a first test on a sample; One or more second measurements are obtained during a second operation performed by the diagnostic laboratory system, wherein the second operation is one of a plurality of operations for performing the first test; A model trained using the correlation learned between a first operation and a second operation is used to jointly analyze one or more first measurements and one or more second measurements, wherein the training data used to train the model is collected from multiple failed operations or tests performed in a controlled diagnostic laboratory or factory setting. The uncertainty score of the first test is determined based on a joint analysis, wherein a specific combination of one or more first measurements and one or more second measurements, each having an acceptable value equal to or below the upper limit of measurement validity or equal to or above the lower limit of measurement validity, results in a high uncertainty score, and a specific combination of the correlations learned indicates inaccurate test results caused by these specific combinations. as well as The decision to rerun the first test is based on the uncertainty score.

2. The method of claim 1, further comprising rerunning the first test in response to an uncertainty score being less than a predetermined value.

3. The method according to claim 1, wherein: During the first operation, obtaining one or more first measurements includes obtaining a first measurement having a value below the upper limit of validity of the first measurement; Obtaining one or more second measurements during the second operation includes obtaining a second measurement with a value higher than the validity lower limit of the second measurement; as well as Determining the uncertainty score involves determining the failure uncertainty score of the first test based on joint analysis.

4. The method of claim 1, further comprising: Encode one or more first measurements into a first vector representation representing a first operation; as well as Encode one or more second measurements into a second vector representation that represents the second operation.

5. The method according to claim 4, wherein: Encoding one or more first measurements includes normalizing one or more first measurements; and Encoding one or more second measurements includes normalizing one or more second measurements.

6. The method according to claim 4, wherein: Encoding one or more first measurements includes generating a dimension-reduced vector from one or more first measurements; and Encoding one or more second measurements includes generating a dimension-reduced vector from one or more second measurements.

7. The method according to claim 1, wherein: Analyzing one or more first measurements and one or more second measurements together using a model trained by learning the correlation between the first and second operations includes encoding one or more first measurements and one or more second measurements into a vector space using a neural network; as well as Determining the uncertainty score includes determining the uncertainty score based on the vector space.

8. The method according to claim 1, further comprising: The workflow of at least the first operation and the second operation is represented as a graph having at least a first node and a second node, wherein the first node includes a vector representation of one or more first measurements, and the second node includes a vector representation of one or more second measurements.

9. The method according to claim 8, wherein: Analyzing one or more first measurements and one or more second measurements together using a model trained by learning the correlation between the first and second operations includes encoding graphs into a vector space using a graph neural network; as well as Determining the uncertainty score includes determining the uncertainty score based on the vector space.

10. The method of claim 9, further comprising training a graph neural network over a workflow.

11. The method of claim 1, wherein the one or more first measurements include at least one of pressure, photometry, acoustic, temperature and optical measurements.

12. The method of claim 1, wherein the first test is performed by a plurality of modules, and further comprising, in response to an uncertainty score being less than a predetermined value, determining which of the plurality of modules caused the uncertainty score to be less than the predetermined value.

13. The method of claim 1, further comprising, in response to an uncertainty score being less than a predetermined value, determining which of the first and second operations causes the uncertainty score to be less than the predetermined value.

14. A method for determining the accuracy of a test performed by a diagnostic laboratory system, comprising: A graphical representation of the workflow for generating tests that can be performed by the diagnostic laboratory system, wherein the graphical representation includes multiple nodes and is configured to be analyzed by a graphical neural network; One or more first measurements are obtained from the first operation performed by the diagnostic laboratory system during the test; The one or more first measurements are converted into a first vector, wherein the first vector is a first node of the graphical representation; One or more second measurements are obtained from a second operation performed by the diagnostic laboratory system during the test; The one or more second measurements are converted into a second vector, wherein the second vector is a second node of the graphical representation; The first and second nodes are analyzed using a graph neural network, which is trained on operational workflow data of tests obtained from the operation of the diagnostic laboratory system. The uncertainty score of a test is determined based on analysis using a second neural network trained on data collected from failed operations or tests performed in a controlled diagnostic laboratory or factory setting, wherein a specific combination of one or more first measurements and one or more second measurements, each having an acceptable value equal to or below the upper limit of measurement validity or equal to or above the lower limit of measurement validity, results in a high uncertainty score, and the specific combination of data collected for training the second neural network indicates inaccurate test results caused by these specific combinations; as well as The decision to rerun the test is based on the uncertainty score.

15. The method of claim 14, wherein: The diagnostic laboratory system is configured to perform multiple operations in a first sequence during the first operation; The diagnostic laboratory system is configured to perform multiple operations in a second sequence during the second operation; as well as A graph neural network is trained on the first and second sequences.

16. The method of claim 14, wherein: Transforming one or more first measurements includes encoding one or more first measurements into a first vector representation representing a first operation; Transforming one or more second measurements involves encoding one or more second measurements into a second vector representation that represents a second operation. The analysis includes using a graph neural network to jointly analyze the first and second vector representations to map them into a vector space; and Determining the uncertainty score includes determining the uncertainty score based on the vector space.

17. The method of claim 16, wherein the encoding includes employing principal component analysis, independent component analysis, or an autoencoder.

18. The method of claim 14, further comprising training a graph neural network over a workflow.

19. A diagnostic laboratory system, comprising: One or more modules are configured to perform tests, which have a workflow of operation sequences; Multiple sensors, configured to generate one or more measurements for each operation; A processor coupled to the sensor; as well as A memory coupled to the processor, wherein the memory includes a graph neural network, a second neural network, and computer program code, which, when executed by the processor, causes the processor to: A graphical representation of a workflow is generated, wherein the graphical representation includes at least a first node corresponding to a first operation of the workflow and a second node corresponding to a second operation of the workflow; One or more first measurements generated by the first operation are converted into a first vector, wherein the first vector corresponds to the first node; One or more second measurements generated by the second operation are converted into a second vector, wherein the second vector corresponds to the second node; as well as The first and second vectors are analyzed using a graph neural network trained on operational workflow data for tests obtained from the operation of the diagnostic laboratory system. The uncertainty score of the test is determined by analyzing the first and second vectors using a second neural network trained on data collected from failed operations or tests performed in a controlled diagnostic laboratory or factory setting, wherein a specific combination of one or more first measurements and one or more second measurements, each having an acceptable value equal to or below the upper limit of measurement validity or equal to or above the lower limit of measurement validity, results in a high uncertainty score, and the specific combination of data collected for training the second neural network indicates inaccurate test results caused by these specific combinations. as well as The decision to rerun the test is based on the uncertainty score.

20. The diagnostic laboratory system of claim 19, wherein, The memory includes computer program code that, when executed by the processor, causes the processor to: Encode one or more first measurements into a first vector representing a first operation; Encode one or more second measurements into a second vector representing a second operation; By using a graph neural network to analyze the first and second vectors, the first and second vectors are mapped into a vector space; as well as Uncertainty scores are determined based on vector space.

Images