Assay accuracy and reliability improvements

Abstract

In particular, the present invention relates to devices and methods that improve the accuracy and reliability of assays even when an assay device and / or operation of an assay device has certain errors, and in some embodiments, the errors are random.

Description

[0001] Cross-references

[0002] This application is a divisional application of the Chinese national phase application with application number 2020800415198 and application date 2020-04-06. Like the parent application, this application claims the priority benefit of U.S. Provisional Patent Application No. 62 / 830,311 filed on April 5, 2019, and is a continuation-in-part application of International Patent Application No. PCT / US2019 / 048678 filed on August 28, 2019, which claims U.S. Provisional Application Serial No. 62 / 742,247 filed on October 5, 2018 and U.S. Provisional Application Serial No. 62 / 742,247 filed on August 28, 2018. No. 62 / 724,025; this application is also a continuation-in-part of International Patent Application No. PCT / US2019 / 046971, filed on August 16, 2019, which claims the benefit of U.S. Provisional Application Serial No. 62 / 764,886, filed on August 16, 2018, and U.S. Provisional Application Serial No. 62 / 719,129, filed on August 16, 2018, the contents of which are incorporated herein by reference in their entirety. The entire disclosure of any publication or patent document mentioned herein is incorporated by reference in its entirety. Technical Field

[0003] The present invention relates particularly to apparatus and methods for performing biological and chemical assays, and more particularly to improving assay detection accuracy and reliability when performed under imperfect conditions with distortion and random variation (eg, limited resource settings). Background Art

[0004] When measuring the biomarkers in a sample from a subject (e.g., a human) for diagnosing a condition or disease, the accuracy of the assay is essential. False results may be harmful to the subject. Traditionally, the accuracy of the assay has been achieved by a "perfect protocol paradigm" (i.e., accurately performing everything including sample processing). This method requires complex machines, professional operations, ideal environments, etc., in order to ensure "perfect" assay devices and "perfect" assay performance and operation. However, it is highly desirable to develop systems and methods that improve assay accuracy, comprising at least one parameter, each parameter having an error that may come from an assay device or device operation, and an error that may be random according to a specific device or specific operation. Summary of the Invention

[0005] In particular, the present invention relates to devices and methods that improve the accuracy and reliability of an assay, even when the assay device and / or the operation of the assay device has some errors, and in some embodiments the errors are random.

[0006] One aspect of the invention is to overcome random errors or defects in an assay device or operation of an assay device by measuring, in addition to the analyte in a sample to produce an analyte test result, the reliability of the analyte test result. The analyte test result is only reported if the reliability meets a predetermined threshold, otherwise the analyte test result is discarded.

[0007] In some embodiments, the measurement of reliability is performed by imaging one or more parameters of the sample being assayed and processing the image using an algorithm.

[0008] In some embodiments, one or more monitoring structures (i.e. array of pillars) are placed on the sample contact area of the sample holder to provide information for the reliability measurement.

[0009] One aspect of the invention is to overcome distortion of the optical system in an image-based assay by having a monitoring mark on the sample holder, where one or more optical properties of the monitoring mark without distortion of the optical system are determined prior to the assay test. The monitoring mark is imaged with the optical system with distortion along with the sample. An algorithm is used to compare the monitoring mark with distortion in the optical system to the monitoring mark without distortion to correct for the distortion in the image-based assay.

[0010] In some embodiments, the algorithm is a machine learning model.

[0011] In some embodiments, a method for improving the accuracy of an assay for detecting an analyte in a sample, where one or more parameters of the assay have random variations, the method comprising:

[0012] detecting the analyte in the sample using the assay, producing a detection result;

[0013] determining the reliability of the detection result by (i) imaging the sample in the assay and (ii) processing the image using an algorithm; and

[0014] reporting the detection result only if the reliability meets a predetermined threshold.

[0015] In some embodiments, an apparatus for improving the accuracy of an assay for detecting an analyte in a sample, where one or more parameters of the assay have random variations, the apparatus comprising:

[0016] an assay for detecting the analyte in the sample to produce a detection result, where the assay has a sample holder; and

[0017] an imager that images a sample in the sample holder; and

[0018] A non-transitory storage medium storing an algorithm that uses the image to determine a confidence of the detection result.

[0019] In some embodiments, the method of any of the preceding embodiments further comprises using one or more monitoring marks on the sample holder for the assay, and imaging the monitoring marks in the image to determine the confidence, wherein the monitoring marks have predetermined optical properties in the manufacture of the sample holder.

[0020] In some embodiments, the apparatus of any of the preceding embodiments further comprises one or more monitoring marks on the sample holder, wherein the monitoring marks have predetermined optical properties in the manufacture of the sample holder, and are imaged in the image to determine the confidence.

[0021] In some embodiments, a method for improving the accuracy of an image-based assay for detecting an analyte in a sample, wherein the assay has an optical system with distortion, the method comprises:

[0022] having a sample holder having a sample contact surface, wherein (i) a sample forms a thin layer of 200 nm thick or thinner on the sample contact surface, and (ii) one or more monitoring marks on the sample contact surface of the sample, wherein the monitoring marks have a first set of parameters predetermined during the manufacture of the sample holder;

[0023] taking one or more images of the sample in the sample holder along with the monitoring marks using the optical system of the assay, wherein the monitoring marks have a second set of parameters in the image;

[0024] processing the one or more images using a processor, wherein the processor detects the distortion of the optical system by using the algorithm and the first set of parameters and the second set of parameters.

[0025] In some embodiments, an apparatus for improving the accuracy of an image-based assay for detecting an analyte in a sample, wherein the assay has an optical system with distortion, the apparatus comprises:

[0026] a sample holder having a sample contact surface, wherein (i) a sample forms a thin layer of 200 nm thick or thinner on the sample contact surface, and (ii) one or more monitoring marks on the sample contact surface of the sample, wherein the monitoring marks have a first set of parameters predetermined during the manufacture of the sample holder;

[0027] an optical system of the assay to take one or more images of a sample in the sample holder along with the monitoring indicia, wherein the monitoring indicia has a second set of parameters in the images;

[0028] a processor having a non-transitory storage medium storing an algorithm that processes the one or more images and corrects for distortion of the optical system using the algorithm and the first set of parameters and the second set of parameters.

[0029] In some embodiments, the method of any of the preceding embodiments, wherein the algorithm is a machine learning model.

[0030] In some embodiments, the method of any of the preceding embodiments, wherein the trustworthiness includes (1) blood origin, (2) air bubbles in blood, (3) too little or too much blood volume, (4) blood cells under spacers, (5) clumped blood cells, (6) lysed blood cells, (7) overexposed images of the sample, (8) underexposed images of the sample, (8) poor focus of the sample, (9) optical system error with wrong lever position, (10) open card, (12) wrong card due to no spacers in card, (12) dust in the card, (14) oil in the card, (14) smudges on the card out of the focal plane, (15) card not in the correct position in the reader, (16) empty card, (17) manufacturing error in the card, (18) wrong card for other applications, (19) dry blood, (20) expired card, (21) large variation in blood cell distribution, (22) non-blood sample, or (23) non-target blood sample.

[0031] In some embodiments, the method of any of the preceding embodiments, wherein the algorithm is machine learning.

[0032] In some embodiments, the method of any of the preceding embodiments, wherein the sample includes at least one of the parameters with random variation, wherein the parameters include dust, air bubbles, non-sample material, or any combination thereof.

[0033] In some embodiments, the method of any of the preceding embodiments, wherein the assay is a cellular assay, an immunoassay, a nucleic acid assay, a colorimetric assay, a luminescent assay, or any combination thereof.

[0034] In some embodiments, the method of any of the preceding embodiments, wherein the assay device includes two plates facing each other with a gap, and at least a portion of the sample is in the gap.

[0035] In some embodiments, the method of any of the preceding embodiments, wherein the assay device comprises a QMAX comprising two plates that are movable relative to each other and spacers that adjust the spacing between the plates.

[0036] In some embodiments, the method of any of the preceding embodiments, wherein some of the monitoring structures are arranged at a fixed interval.

[0037] In some embodiments, the method of any of the preceding embodiments, wherein the sample is selected from the group consisting of a cell, a tissue, a bodily fluid, and a stool.

[0038] The apparatus and method of any of the preceding claims, wherein the sample is amniotic fluid, aqueous humor, vitreous humor, blood (e.g., whole blood, fractionated blood, plasma, serum), breast milk, cerebrospinal fluid (CSF), cerumen (earwax), chyle, chime, endolymph, perilymph, excrement, gastric acid, gastric juice, lymphatic fluid, mucus (including nasal drainage and sputum), pericardial fluid, peritoneal fluid, pleural fluid, pus, rheum, saliva, sebum (skin oil), semen, sputum, sweat, synovial fluid, tears, vomit, urine, or exhaled condensate.

[0039] The apparatus and method of any of the preceding claims, wherein the analyte comprises a molecule (e.g., a protein, a polypeptide, DNA, RNA, a nucleic acid, or other molecule), a cell, a tissue, a virus, and a nanoparticle.

[0040] The apparatus and method of any of the preceding claims, wherein the sample is a non- flowable but deformable sample.

[0041] The apparatus and method of any of the preceding claims, wherein the algorithm is machine learning, artificial intelligence, a statistical method, or a combination thereof.

[0042] The apparatus and method of any of the preceding embodiments, wherein the spacers are the monitoring markers, wherein the spacers have a substantially uniform height equal to or less than 200 microns and a fixed spacer interval distance (ISD).

[0043] The apparatus and method of any of the preceding claims, wherein the monitoring markers are used to estimate TLD (true lateral dimension) and true volume estimation.

[0044] The apparatus and method of any of the preceding claims, wherein step (b) further comprises image segmentation for image-based assays.

[0045] In some embodiments, the method of any of the preceding embodiments, wherein step (b) further comprises focus check in image-based assays.

[0046] In some embodiments, the method of any of the preceding embodiments, wherein step (b) further comprises uniformity of distribution of the analyte in the sample.

[0047] In some embodiments, the method of any of the preceding embodiments, wherein step (b) further comprises analyzing and detecting aggregated analyte in the sample.

[0048] In some embodiments, the method of any of the preceding embodiments, wherein step (b) further comprises analyzing dry texture structures in the image of the sample.

[0049] In some embodiments, the method of any of the preceding embodiments, wherein step (b) further comprises analyzing defects in the sample.

[0050] In some embodiments, the method of any of the preceding embodiments, wherein step (b) further comprises correction of camera parameters and conditions such as distortion removal, temperature correction, brightness correction, contrast correction. BRIEF DESCRIPTION OF DRAWINGS

[0051] Those skilled in the art will appreciate that the figures described below are for purposes of illustration only. The figures are not intended to limit the scope of the present application in any way. The figures are not drawn to scale. In figures presenting experimental data points, the lines connecting the data points are for the purpose of guiding the observation of the data only, and serve no other purpose.

[0052] Figure 1 A flowchart of an image-based assay using a special sample holder is shown.

[0053] Figure 2 A side view and a top view of a sample holder Q-card with a monitoring marker column is shown.

[0054] Figure 3 A block diagram with parallel confidence risk estimation is shown.

[0055] Figure 4 A control flow of an imaging-based assay with confidence risk estimation is shown.

[0056] Figure 5 A flowchart of true lateral size correction using a machine learning model trained for column detection is shown.

[0057] Figure 6 A flowchart of training a machine learning (ML) model for detecting columns from images of samples is shown.

[0058] Figure 7 A schematic diagram representing the relationship between training a machine learning model and applying the trained machine learning model in prediction (inference) is shown.

[0059] Figure 8 Schematic diagram showing defects (such as dust, bubbles, etc.) that may appear in the sample.

[0060] Figure 9 Shown is a real image from a blood test, with defects in the sample image used for the assay.

[0061] Figure 10 Defect detection and segmentation are shown on images of samples using the described method.

[0062] Figure 11 shows an autofocus diagram in photomicrography;

[0063] Figure 12 A schematic diagram showing distortion cancellation with known distortion parameters is shown;

[0064] Figure 13 A schematic diagram showing distortion removal and camera adjustment when the distortion parameters are unknown;

[0065] Figure 14 The distorted position of the monitoring marker post in the sample image due to the distortion of the imager is shown. DETAILED DESCRIPTION

[0066] The following detailed description illustrates some embodiments of the present invention by way of example and not limitation. The section headings and any subtitles used herein are for organizational purposes only and should not be interpreted as limiting the subject matter described in any way. The content under a section heading and / or subheading is not limited to the section heading and / or subheading, but applies to the entire description of the present invention.

[0067] The following detailed description illustrates some embodiments of the present invention by way of example and not limitation. The section headings and any subtitles used herein are for organizational purposes only and should not be interpreted as limiting the subject matter described in any way. The content under a section heading and / or subheading is not limited to the section heading and / or subheading, but applies to the entire description of the present invention.

[0068] The citation of any publication is for its disclosure prior to the filing date and should not be construed as an admission that the present claims are not entitled to antedate such publication by virtue of prior invention. Further, the publication dates provided may be different from the actual publication dates, which may need to be independently confirmed.

[0069] The terms "confidence," "risk of error," "error risk factor," and "risk factor" are interchangeable terms to describe the likelihood that a test result is inaccurate. More "confidence" means a smaller "likelihood" (i.e., smaller risk), and therefore a lower "risk factor."

[0070] The terms "instructions" and "algorithms" are interchangeable.

[0071] The term "imaging-based assay" refers to an assay that includes an imager for detecting an analyte in a sample.

[0072] Improving assay accuracy by checking the reliability of test results

[0073] A reliability-based test reporting example to improve test accuracy. In many assay cases, there are random variations in assay operations, sample handling, and other processes related to assay operations, and these random variations can affect the test result accuracy of the assay.

[0074] According to the present invention, a monitoring structure for monitoring assay operation parameters is placed on the sample holder (in some embodiments, in the sample area being tested). As Figure 1 As an example shown in the flowchart, during testing, both the analyte in the sample and the monitoring structure are measured (in parallel or sequentially), and the reliability of the analyte measurement is determined from the monitoring structure measurement by instructions (non-transitory storage medium storing the instructions). If the reliability is above a threshold (i.e., the risk factor is below a threshold), the analyte measurement result will be reported, otherwise the analyte measurement result will not be reported.

[0075] If the analyte measurement result is not reported due to poor reliability (above a threshold), optionally, a second sample and / or a second test will be tested in the same way as the first sample and / or the first test. If the second sample and / or the second test still has poor reliability, a third sample and / or a third test will be performed. The process can continue until a reliable analyte measurement result is reported.

[0076] Examples of monitoring structures include, but are not limited to, spacers, scale marks, imaging marks, and position marks on Q-cards (i.e., QMAX cards). Figure 2 A schematic diagram of a sample holder Q-card. Monitoring parameters include, but are not limited to, images of the monitoring structure, light transmission, light scattering, color (wavelength spectrum), polarization, etc. Monitoring parameters also include images of the sample on the Q-card. Instructions to determine whether or not to report can be determined by testing under various conditions that deviate from ideal conditions. In some embodiments, machine learning is used to learn how to determine the reliability threshold.

[0077] In some embodiments, the monitoring parameters are related to sample holder operations and conditions, sample conditions, or reagent conditions, or measurement instrument conditions.

[0078] A method for improving the test result accuracy of an assay, the method comprising:

[0079] Having a sample holder with monitoring marks;

[0080] having a sample on the sample holder, wherein the sample is suspected of containing an analyte;

[0081] measuring the analyte in parallel or sequentially to produce an analyte measurement (i.e., test result), and measuring the monitoring label to produce a monitoring parameter;

[0082] using the instructions and the monitoring parameter to determine the trustworthiness of the analyte measurement;

[0083] determining publication of the analyte measurement.

[0084] Figure 3 A schematic of an assay with trustworthiness check is shown, and in some embodiments, the analyte measurement is only published when the trustworthiness is high (risk factor is low). In some embodiments, when the trustworthiness is low, the analyte measurement will not be published. In some embodiments, when the trustworthiness is low, the analyte measurement of the first sample will not be published, and a second sample will be used to undergo the same process as described above. In some embodiments, the process is repeated until a high trustworthiness analyte measurement is obtained.

[0085] An apparatus for improving test accuracy by measuring a monitoring parameter, comprising: a sample card having a monitoring structure, an imager, and a medium storing instructions for determining trustworthiness of a test result.

[0086] Examples of methods and apparatus of mobile assays (using QMAX cards) - concentration estimation (CBC), segmentation, etc. using image processing and machine learning.

[0087] AA-1. A method for correcting system error of an image system containing a thin layer sample, the method comprising:

[0088] receiving, by a processing device of the image system, an image associated with a sample card containing a sample and a monitoring standard, and a first parameter;

[0089] determining, by the processing device, a system error of the image system using a first machine learning model by comparing the first parameter to a second parameter associated with the monitoring standard determined during manufacturing of the sample card;

[0090] correcting, by the processing device, the image of the sample card taking into account the system error; and

[0091] determining, by the processing device, a biological property of the sample using the corrected image.

[0092] AA-2. The method of example AA-1, wherein the monitoring standard comprises a plurality of nanostructures on a plate of the sample card, and wherein the sample is deposited on the plate.

[0093] AA-3. The method of example AA-1, wherein the sample is a biological sample collected from an animal.

[0094] Figure 5 A flowchart is shown that utilizes columns for true lateral dimension (TLD) estimation and uses the markers (columns) for image correction from Q-card imaging.

[0095] BA-1. A smart assay monitoring method comprising:

[0096] receiving, by a processing device, an image encoding first information of a biological sample deposited in a sample card and second information of a plurality of monitoring markers;

[0097] determining, by the processing device, a measurement of a geometric feature associated with the plurality of monitoring markers by executing a first machine learning model on the image;

[0098] determining, by the processing device, a change between the measurement of the geometric feature and a ground truth value of the geometric feature provided by the sample card;

[0099] correcting, by the processing device, the image encoding the first information and the second information based on the change; and

[0100] determining, by the processing device, a biological property of the biological sample using the corrected image.

[0101] BA-2. The method of example BA-1, wherein the sample card comprises a first plate, a plurality of columns integrated substantially perpendicularly to a surface of the first plate, and a second plate capable of enclosing the first plate to form a thin layer, wherein the biological sample is deposited in the thin layer.

[0102] BA-3. The method of example BA-2, wherein the plurality of monitoring markers correspond to the plurality of columns.

[0103] BA-4. The method of example BA-3, wherein at least two of the plurality of columns are separated by a true lateral dimension (TLD), and wherein determining, by the processing device, the measurement of the geometric feature associated with the plurality of monitoring markers by executing the first machine learning model on the image comprises determining, by the processing device executing the first machine learning model on the image, the TLD.

[0104] Figure 6 A flowchart is shown that trains a machine learning model for column detection in TLD correction using monitoring marker columns. Figure 7 A relationship is shown between training a machine learning model and applying the trained machine learning model in prediction (derivation).

[0105] BB-1 An imaging system comprising:

[0106] A sample card comprising a first plate, a plurality of pillars integrated substantially perpendicularly to a surface of the first plate, and a second plate capable of enclosing the first plate to form a thin layer, wherein a biological sample is deposited in the thin layer;

[0107] A computing device comprising:

[0108] A processing device communicatively coupled to the optical sensor to:

[0109] receive, from the optical sensor, an image encoding first information of a biological sample deposited in a sample card and second information of a plurality of monitoring markers;

[0110] determine, using a first machine learning model for the image, a measure of a geometric feature associated with the plurality of monitoring markers;

[0111] determine a change between the measure of the geometric feature and a ground truth value of the geometric feature provided by the sample card;

[0112] correct the image encoding the first information and the second information based on the change; and

[0113] determine a biological property of the biological sample based on the corrected image.

[0114] On-car object detection for detecting improper card closure in an assay

[0115] CA-1. A method for correcting human handling errors recorded in an assay image of a thin layer sample, the method comprising:

[0116] receiving, by a processing device of an image system, an image of a sample card comprising a sample and a monitoring standard, wherein the monitoring standard comprises a plurality of nanostructures integrated on a first plate of the sample card, and wherein the sample is deposited on the first plate of the sample card in an open configuration and is enclosed by a second plate of the sample card in a closed configuration;

[0117] segmenting, by the processing device, the image into a first sub-region corresponding to the sample and a second sub-region corresponding to the plurality of nanostructures;

[0118] comparing, by the processing device, the second sub-region to the monitoring standard provided during manufacture of the sample to determine whether at least one of the second sub-region contains a foreign object other than the nanostructures;

[0119] in response to determining that at least one of the second sub-region contains a foreign object other than the nanostructures, determining an error associated with handling the sample card; and

[0120] correcting the image of the sample card by removing the at least one sub-region from the image.

[0121] CA-2. The method of embodiment CA-1, wherein the foreign object is one of a portion of the sample, a bubble, or an impurity.

[0122] Examples of defect (e.g., bubble, dust, etc.) and auxiliary structure (e.g., pillar) removal in an assay

[0123] DA-1. A method for measuring a volume of a sample in a thin layer sample card, the method comprising:

[0124] receiving, by a processing device of an image system, an image of a sample card containing a sample and a monitoring standard, wherein the monitoring standard contains a plurality of pillars integrated vertically to a first plate of the sample, and each of the plurality of pillars has substantially the same height (H);

[0125] determining, by the processing device, a plurality of non-sample sub-areas using a machine learning model, wherein the plurality of non-sample sub-areas correspond to at least one of a pillar, a bubble, or an impurity element;

[0126] calculating, by the processing device, an area occupied by the sample by removing the plurality of non-sample sub-areas from the image;

[0127] calculating, by the processing device, a volume of the sample based on the calculated area and the height (H), and

[0128] determining, by the processing device, a biological property of the sample based on the volume.

[0129] Examples of determining the reliability of an assay result

[0130] a. Shape segmentation combining ML-based bounding box detection and image processing-based shape determination

[0131] b. Uniformity of analyte detection in an assay (IQR-based outlier detection)

[0132] c. Aggregated analyte detection using ML

[0133] d. Dry texture structures on a card detection using ML

[0134] e. Defects detection using ML, e.g., dust, oil, etc.

[0135] f. Bubble detection using ML

[0136] EA-1. A method for determining the reliability of a measurement associated with an image assay result, the method comprising:

[0137] receiving, by a processing device of an image system, an image of a sample card containing a sample and a monitoring standard, wherein the monitoring standard contains a plurality of nanostructures integrated to a first plate of the sample;

[0138] segmenting, by the processing device, the image into a first sub-region corresponding to the sample and a second sub-region corresponding to the plurality of nanostructures;

[0139] determining, by the processing device, non-compliant elements in at least one of the first sub-region or the second sub-region using the first machine learning model;

[0140] determining, by the processing device, a biological property of the sample based on the first sub-region and the second sub-region;

[0141] calculating, by the processing device, a measure of confidence associated with the biological property based on a statistical analysis of the non-compliant elements; and

[0142] determining, by the processing device, a further action on the sample based on the measure of confidence.

[0143] EA-2. The method of example EA-1, further comprising:

[0144] determining, by the processing device, that the biological property is reliable based on the measure of confidence; and

[0145] providing, by the processing device, the biological property to a display device.

[0146] EA-3. The method of example EA-1, further comprising:

[0147] determining, by the processing device, that the biological property is not reliable based on the measure of confidence; and

[0148] providing, by the processing device, the biological property and the corresponding measure of confidence to a display device to allow a user to determine whether to accept or discard the biological property.

[0149] EA-4. The method of example EA-1, wherein segmenting, by the processing device, the image into a first sub-region corresponding to the sample and a second sub-region corresponding to the plurality of nanostructures;

[0150] first segmenting, by the processing device, the image using an image processing method to generate a first segmentation result;

[0151] second segmenting, by the processing device, the image using a second machine learning model to generate a second segmentation result; and

[0152] combining, by the processing device, the first segmentation result and the second segmentation result to segment the image into a first sub-region corresponding to the sample and a second sub-region corresponding to the plurality of nanostructures.

[0153] EA-5. The method of example EA-1, wherein determining, by the processing device, the out-of-compliance element in at least one of the first sub-region or the second sub-region using the first machine learning model comprises: at least one of

[0154] determining, by the processing device, the out-of-compliance element based on a non-uniformity of distribution of the at least one analyte in the sample;

[0155] determining, by the processing device, the out-of-compliance element based on a collective analyte detection of the sample;

[0156] determining, by the processing device, the out-of-compliance element based on a detection of dry texture structures in the sample;

[0157] determining, by the processing device, the out-of-compliance element based on a detection of impurities in the sample; or

[0158] determining, by the processing device, the out-of-compliance element based on a detection of air bubbles in the sample.

[0159] Multi-segment testing of a single card with content-based segmentation

[0160] FA-1. A method of determining measurement values of a plurality of analytes using a single sample card, the method comprising:

[0161] receiving, by a processing device of an image system, an image of a sample card comprising a sample and a monitoring standard, wherein the monitoring standard comprises a plurality of nanostructures integrated to a first plate of the sample;

[0162] segmenting, by the processing device, the image into a first sub-region associated with a first analyte contained in the sample, a second sub-region associated with a second analyte contained in the sample, and a third sub-region corresponding to the plurality of nanostructures using a first machine learning model;

[0163] determining, by the processing device, a true lateral distance (TLD) between two adjacent nanostructures based on the third sub-region corresponding to the plurality of nanostructures;

[0164] determining, by the processing device, a first cumulative area of the first sub-region based on the TLD, and further determining a first volume based on the first cumulative area and a height associated with the plurality of nanostructures;

[0165] determining, by the processing device, a second cumulative area of the sub-region based on the TLD, and further determining a second volume based on the second cumulative area and the height associated with the plurality of nanostructures;

[0166] determining, by the processing device, a first measurement value of the first analyte based on a count of the first analyte in the first volume; and

[0167] Determining, by a processing device, a second measurement of a second analyte based on a count of the second analyte in the second volume. System and device of a column-based spectrophotometer

[0168] GA-1. A method for measuring a biological property of a sample provided in a sample card comprising a plurality of nanopillars, the method comprising:

[0169] Receiving, by a processing device of an image system, an image of a sample card comprising a sample and a monitoring standard comprising a plurality of nanostructures integrated to a first plate of the sample;

[0170] Segmenting, by the processing device, the image into a first sub-region corresponding to the sample and a second sub-region corresponding to the plurality of nanopillars;

[0171] Determining, by the processing device, a first spectrophotometric measurement of the first sub-region;

[0172] Determining, by the processing device, a second spectrophotometric measurement of the second sub-region; and

[0173] Determining, by the processing device, the biological property of the sample based on a ratio between the first spectrophotometric measurement and the second spectrophotometric measurement.

[0174] GA-2. A method for analyzing a concentration of a compound using a measured response of the compound at a specific optical wavelength or a plurality of optical wavelengths, the method comprising:

[0175] Receiving, by a processing device of an image system, an image or a plurality of images taken on a sample card having a specific or a plurality of optical wavelengths, wherein the sample card comprises a sample and a monitoring standard comprising a plurality of nanostructures integrated to a first plate of the sample;

[0176] Segmenting, by the processing device, each image into a first sub-region corresponding to the sample and a second sub-region corresponding to the plurality of nanopillars;

[0177] Determining, from the images of the sample taken at different wavelengths, light absorption from the first sub-region of the sample;

[0178] Determining, from the images of the sample taken at different wavelengths, light absorption from the second sub-region of the nanopillars; and

[0179] Determining, from the light absorption measurements at one or more wavelengths from both sub-regions, the concentration of the compound.

[0180] Further, in some embodiments of the invention, the detection and segmentation of each image into a first sub-region corresponding to the sample and a second sub-region corresponding to the plurality of nanopillars is based on a machine learning model trained on a training image sample having labeled nanopillars.

[0181] Labeling large numbers of small and repetitive objects

[0182] HA-1. A method for labeling a plurality of objects in an image to prepare training data, the method comprising:

[0183] receiving, by a processing device, an image in a graphical user interface;

[0184] receiving, by the processing device, a selection of a location in the image through the graphical user interface;

[0185] calculating, by the processing device, a bounding box that encloses the location based on a plurality of pixels local to the location;

[0186] providing, by the processing device, a display of the bounding box superimposed on the image in the graphical user interface; and

[0187] in response to receiving a user confirmation, labeling a region within the bounding box as training data.

[0188] IA-1. A method for preparing an assay image, the method comprising:

[0189] providing a sample card comprising a plurality of marker elements;

[0190] depositing a sample on a first plate of the sample card in an open configuration;

[0191] closing the sample card to press a second plate of the sample card against the first plate to form a closed configuration, wherein the first plate and the second plate in the closed configuration form a thin layer, the thin layer comprising a substantially uniform thickness of the sample and the plurality of marker elements; and

[0192] providing an image system comprising a processing device and a non-transitory storage medium storing instructions that, when executed by the processing device, will:

[0193] capture an image of the sample card, the image comprising a substantially uniform thickness of the sample and the plurality of marker elements;

[0194] detect the plurality of marker elements in the image;

[0195] compare the detected plurality of marker elements to a monitoring standard associated with the sample card to determine a geometric mapping between the plurality of marker elements and the monitoring standard;

[0196] determine a non-ideal factor of the image system based on the geometric mapping; and process the image of the sample card to correct for the non-ideal factor.

[0197] IB-1. A method for preparing an assay image, the method comprising:

[0198] providing a sample card comprising a plurality of marker elements;

[0199] depositing a sample on a first plate of a sample card in an open configuration;

[0200] closing the sample card to press a second plate of the sample card against the first plate to form a closed configuration, wherein the first plate and the second plate in the closed configuration form a thin layer, the thin layer comprising a substantially uniform thickness of the sample and the plurality of marker elements; and

[0201] providing an image system comprising a processing device and a non-transitory storage medium storing instructions that, when executed by the processing device, will:

[0202] capturing an image of a sample card, the image comprising a substantially uniform thickness of a sample and a plurality of marker elements;

[0203] segmenting the image into a plurality of sub-regions;

[0204] determining, using a machine learning model and the plurality of marker elements, whether each of the plurality of sub-regions meets a requirement of the image system;

[0205] in response to determining that a sub-region does not meet the requirement, flagging the first sub-region as non-compliant;

[0206] in response to determining that a sub-region meets the requirement, flagging the first sub-region as compliant; and

[0207] performing an assay analysis using the compliant sub-regions of the image.

[0208] IC-1. A method for correcting a non-ideal factor of an image system, the method comprising:

[0209] receiving, by a processing device of the image system, an image of a sample card, the image comprising a substantially uniform sample layer deposited on a plate of the sample card and a plurality of marker elements associated with the sample card;

[0210] detecting, by the processing device, the plurality of marker elements in the image;

[0211] comparing the detected plurality of marker elements to a monitoring standard associated with the sample card to determine a geometric mapping between the plurality of marker elements and the monitoring standard;

[0212] determining a non-ideal factor of the image system based on the geometric mapping; and

[0213] processing the image of the sample card to correct the non-ideal factor.

[0214] ID-1. A method for correcting a non-ideal factor of an image system, the method comprising:

[0215] receiving, by a processing device of an image system, an image of a sample card, the sample card including a substantially uniform sample layer deposited on a plate of the sample card and a plurality of marker elements associated with the sample card;

[0216] segmenting, by the processing device, the image into a plurality of sub-regions;

[0217] determining, by the processing device, whether each of the plurality of sub-regions satisfies a requirement of the image system using a machine learning model and the plurality of marker elements;

[0218] responsive to determining that a sub-region does not satisfy the requirement, flagging, by the processing device, the first sub-region as non-compliant;

[0219] responsive to determining that a sub-region satisfies the requirement, flagging, by the processing device, the first sub-region as compliant; and

[0220] performing, by the processing device, an assay analysis using compliant sub-regions of the image.

[0221] IF-1. An image system, comprising:

[0222] an adapter for holding a sample card, the sample card including a first plate, a second plate, and a plurality of marker elements;

[0223] a mobile computing device coupled to the adapter, the mobile computing device including:

[0224] an optical sensor for capturing an image of the plurality of marker elements and the sample card, the image including a substantially uniform sample layer deposited between the first plate and the second plate of the sample card;

[0225] a processing device communicatively coupled to the optical sensor so as to:

[0226] receive the image captured by the optical sensor;

[0227] detect the plurality of marker elements in the image;

[0228] compare the detected plurality of marker elements to a monitoring standard associated with the sample card to determine a geometric mapping between the plurality of marker elements and the monitoring standard;

[0229] determine a non-ideal factor of the image system based on the geometric mapping; and

[0230] process the image of the sample card to correct for the non-ideal factor.

[0231] IF-1. An image system, comprising:

[0232] an adapter for holding a sample card, the sample card including a first plate, a second plate, and a plurality of marker elements;

[0233] a mobile computing device coupled to the adapter, the mobile computing device comprising:

[0234] an optical sensor to capture an image of a plurality of marker elements and a sample card, the image comprising a substantially uniform sample layer deposited between a first plate and a second plate of the sample card;

[0235] a processing device communicatively coupled to the optical sensor to:

[0236] receive the image captured by the optical sensor;

[0237] segment the image into a plurality of sub-regions;

[0238] determine, using a machine learning model and the plurality of marker elements, whether each of the plurality of sub-regions meets a requirement of an image system;

[0239] in response to determining that a sub-region does not meet the requirement, flag the first sub-region as non-compliant;

[0240] in response to determining that a sub-region meets the requirement, flag the first sub-region as compliant; and

[0241] perform an assay analysis using the compliant sub-regions of the image.

[0242] IH-1. A mobile imaging device comprising:

[0243] an optical sensor; and

[0244] a processing device communicatively coupled to the optical sensor to:

[0245] receive an image captured by the optical sensor, the image comprising a plurality of marker elements and a substantially uniform sample layer deposited between a first plate and a second plate of a sample card;

[0246] detect the plurality of marker elements in the image;

[0247] compare the detected plurality of marker elements to a monitoring standard associated with the sample card to determine a geometric mapping between the plurality of marker elements and the monitoring standard;

[0248] determine a non-ideal factor of an image system based on the geometric mapping; and

[0249] process the image of the sample card to correct for the non-ideal factor.

[0250] IH-1. A mobile imaging device comprising:

[0251] an optical sensor; and

[0252] a processing device communicatively coupled to the optical sensor to:

[0253] receiving an image captured by an optical sensor, the image containing a plurality of marker elements and a substantially uniform sample layer deposited between a first plate and a second plate of a sample card;

[0254] segmenting the image into a plurality of sub-regions;

[0255] determining, using a machine learning model and the plurality of marker elements, whether each of the plurality of sub-regions meets a requirement of an image system;

[0256] responsive to determining that a sub-region does not meet the requirement, marking the first sub-region as non-compliant;

[0257] responsive to determining that a sub-region meets the requirement, marking the first sub-region as compliant; and

[0258] performing assay analysis using the compliant sub-regions of the image.

[0259] II-1. An intelligent assay monitoring method, comprising:

[0260] receiving, by a processing device, an image encoding first information of a biological sample deposited in a sample card and second information of a plurality of monitoring markers;

[0261] determining, by the processing device, a measurement of a geometric feature associated with the plurality of monitoring markers by executing a first machine learning model on the image;

[0262] determining, by the processing device, a change between the measurement of the geometric feature and a ground truth value of the geometric feature provided by the sample card;

[0263] correcting, by the processing device, the image encoding the first information and the second information based on the change; and

[0264] determining, by the processing device, a biological property of the biological sample using the corrected image.

[0265] II-2. The method of example II-1, wherein the sample card comprises a first plate, a plurality of pillars integrated substantially perpendicularly to a surface of the first plate, and a second plate capable of enclosing the first plate to form a thin layer, wherein the biological sample is deposited in the thin layer.

[0266] II-3. The method of claim II-2, wherein the plurality of monitoring markers correspond to the plurality of pillars.

[0267] II-4. The method of claim II-2, wherein the plurality of monitoring markers are disposed in at least one of the first plate or the second plate.

[0268] II-5. The method of example II-1, wherein determining, by the processing device, the measurement of the geometric feature associated with the plurality of monitoring markers by executing the first machine learning model on the image further comprises:

[0269] identifying, by the processing device, the plurality of monitoring markers from the image by executing the first machine learning model; and

[0270] determining, by the processing device, the measurement of the geometric feature based on the identified plurality of monitoring markers.

[0271] II-6. The method of claim II-1, wherein determining, by the processing device, the change between the measurement of the geometric feature and a ground truth value of the geometric feature provided by the sample card further comprises:

[0272] determining, by the processing device, one of a system error or a human operator error; and

[0273] presenting, on a display device associated with the processing device, the determined one of the system error or the human operator error.

[0274] II-7. An image system, comprising:

[0275] a sample card comprising a first plate, a plurality of posts integrated substantially perpendicularly to a surface of the first plate, and a second plate capable of enclosing the first plate to form a thin layer, wherein a biological sample is deposited in the thin layer;

[0276] a computing device, comprising:

[0277] a processing device communicatively coupled to the optical sensor to:

[0278] receive, from the optical sensor, an image encoding first information of a biological sample deposited in the sample card and second information of a plurality of monitoring markers;

[0279] determine, using a first machine learning model for the image, a measurement of a geometric feature associated with the plurality of monitoring markers;

[0280] determine a change between the measurement of the geometric feature and a ground truth value of the geometric feature provided by the sample card;

[0281] correct the image encoding the first information and the second information based on the change; and

[0282] determine a biological property of the biological sample based on the corrected image.

[0283] IJ-1. A method for correcting non-ideal factors in an assay image of a thin layer sample, the method comprising:

[0284] A sample card comprising a monitoring standard is provided, the monitoring standard comprising a plurality of nanostructures on a plate of the sample card;

[0285] depositing a sample on the plate of the sample card; and

[0286] An image system is provided, the image system comprising a processing device and a non-transitory storage medium storing instructions that, when executed by the processing device, will:

[0287] capture an image of the sample card comprising the sample and the monitoring standard;

[0288] determine a non-ideal factor of the image system by comparing the image of the sample card to a plurality of geometric values of the monitoring standard determined during manufacture of the sample card; and

[0289] correct the image of the sample card in view of the non-ideal factor.

[0290] IJ-2. A method for correcting a non-ideal factor in an assay image of a thin layer sample, the method comprising:

[0291] receiving, by a processing device of an image system, an image of a sample card comprising a sample and a monitoring standard, wherein the monitoring standard comprises a plurality of nanostructures on a plate of the sample card, and wherein the sample is deposited on the plate;

[0292] determining, by the processing device, a non-ideal factor of the image system by comparing the image of the sample card to a plurality of geometric values of the monitoring standard determined during manufacture of the sample card; and

[0293] correcting the image of the sample card in view of the non-ideal factor.

[0294] IJ-3. A method for correcting a non-ideal factor in an assay image of a thin layer sample, the method comprising:

[0295] receiving, by a processing device of an image system, an image of a sample card comprising a sample and a monitoring standard, wherein the monitoring standard comprises a plurality of nanostructures on a plate of the sample card, and wherein the sample is deposited on the plate;

[0296] determining, by the processing device, a non-ideal factor of the image system using a machine learning model, wherein the machine learning model is trained by comparing the image of the sample card to geometric values of the monitoring standard determined during manufacture of the sample card; and

[0297] correcting the image of the sample card in view of the non-ideal factor.

[0298] IJ-4. A mobile imaging device comprising:

[0299] an optical sensor; and

[0300] a processing device communicatively coupled to the optical sensor to:

[0301] receive an image of a sample card containing a sample and a monitoring standard, wherein the monitoring standard contains a plurality of nanostructures on a plate of the sample card, and wherein the sample is deposited on the plate;

[0302] determine a non-ideal factor of the mobile imaging system by comparing the image of the sample card to a plurality of geometric values of the monitoring standard determined during manufacture of the sample card; and

[0303] correct the image of the sample card in view of the non-ideal factor.

[0304] IJ-5. An image system, comprising:

[0305] an adapter for holding a sample card containing a monitoring standard, wherein the monitoring standard contains a plurality of nanostructures on a plate of the sample card, and wherein the sample is deposited on the plate;

[0306] a mobile computing device coupled to the adapter, the mobile device comprising:

[0307] an optical sensor; and

[0308] a processing device communicatively coupled to the optical sensor to:

[0309] receive an image of the sample card;

[0310] determine a non-ideal factor of the image system by comparing the image of the sample card containing the sample deposited on the plate to a plurality of geometric values of the monitoring standard determined during manufacture of the sample card; and

[0311] correct the image of the sample card in view of the non-ideal factor.

[0312] IJ-6. A sample card, comprising:

[0313] a monitoring standard containing a plurality of nanostructures on a plate of the sample card, wherein the sample is deposited on the plate, and wherein the sample card is inserted into an adapter coupled to an image system, the image system comprising a processing device and an optical sensor to capture an image of the sample card, the processing device to:

[0314] receive an image of the sample card;

[0315] determine a non-ideal factor of the image system by comparing the image of the sample card containing the sample deposited on the plate to a plurality of geometric values of the monitoring standard determined during manufacture of the sample card; and

[0316] correct the image of the sample card in view of the non-ideal factor.

[0317] IJ-7. A method for correcting systematic errors of an image system containing a thin layer sample, the method comprising:

[0318] receiving, by a processing device of the image system, an image and a first parameter associated with a sample card containing a sample and a monitoring standard, wherein the monitoring standard contains a plurality of nanostructures on a plate of the sample card, and wherein the sample is deposited on the plate;

[0319] determining, by the processing device, a systematic error of the image system by comparing the first parameter to a second parameter associated with the monitoring standard determined during manufacturing of the sample card;

[0320] correcting the image of the sample card taking into account the systematic error.

[0321] IJ-8. A method for correcting human handling errors recorded in an assay image of a thin layer sample, the method comprising:

[0322] receiving, by a processing device of the image system, an image of a sample card containing a sample and a monitoring standard, wherein the monitoring standard contains a plurality of nanostructures on a plate of the sample card, and wherein the sample is deposited on the plate;

[0323] determining, by the processing device, human handling errors reflected in the image using a machine learning model by comparing the image of the sample card to a plurality of geometric values of the monitoring standard determined during manufacturing of the sample card, wherein the human handling errors contain erroneous handling of the image system; and

[0324] correcting the image of the sample card by removing the human handling errors reflected in the image.

[0325] Monitoring and correction of assay device handling errors

[0326] A method of monitoring and correcting errors occurring in operating an assay device, comprising:

[0327] (a) a Q-CARD having a monitoring mark on a plate (inside the sample);

[0328] (b) performing sample deposition,

[0329] (c) imaging the monitoring mark using an imager during measurement;

[0330] (d) determining an operating error by comparing the image of the monitoring mark to an ideal image of the monitoring mark;

[0331] wherein the ideal image of the monitoring mark is an image of the monitoring mark when the operation is performed correctly;

[0332] wherein the monitoring mark is pre-fabricated.

[0333] A method of monitoring a defect in operating an assay device, comprising:

[0334] (a) a Q-CARD having a monitoring mark on the plate (inside the sample);

[0335] (b) performing sample deposition,

[0336] (c) imaging the monitoring mark using an imager during the measurement;

[0337] (d) determining an operational error by comparing the image of the monitoring mark to an ideal image of the monitoring mark;

[0338] wherein the ideal image of the monitoring mark is an image of the monitoring mark when the operation is performed correctly;

[0339] wherein the monitoring mark is pre-fabricated.

[0340] OA-1 In some embodiments of the application, a method for improving the accuracy of an assay having one or more unpredictable and random operating conditions, comprising:

[0341] (a) detecting an analyte in a sample containing or suspected of containing the analyte, comprising:

[0342] (i) feeding the sample into a detection instrument, and

[0343] (ii) measuring the sample using the detection instrument to detect the analyte, producing a detection result of the detection;

[0344] (B) determining the reliability of the detection result in step (a), comprising:

[0345] (i) taking one or more images of (1) a portion of the sample and / or (2) a portion of the detection instrument surrounding the portion of the sample, wherein the images are substantially representative of the conditions under which the portion of the sample was measured in step (a) to produce the detection result; and

[0346] (ii) analyzing the images taken in step (b)(i) using a computing device having an algorithm to determine the reliability of the detection result in step (a); and

[0347] (c) reporting both the detection result and the reliability;

[0348] wherein step (a) has one or more unpredictable and random operating conditions.

[0349] In certain embodiments, the method OA-1 further comprises the step of discarding the detection result generated in step (a) if the reliability determined in step (b) is below a threshold value.

[0350] In certain embodiments, method OA-1 further comprises the step of: if the confidence determined in step (b) is below a threshold, then modifying the detection result generated in step (a).

[0351] OA-2. An apparatus for improving the accuracy of an assay having one or more unpredictable and random operating conditions, comprising:

[0352] (1) a detection device that detects an analyte in a sample to produce a detection result, wherein the sample contains or is suspected to contain the analyte;

[0353] (2) an inspection device that inspects the confidence of a particular detection result produced by the detection device, comprising:

[0354] (i) an imager that is capable of taking one or more images of (1) a portion of the sample and / or (2) a portion of the detection instrument that surrounds the portion of the sample, wherein the images are substantially representative of the conditions under which the portion of the sample was measured in step (a); and

[0355] (ii) a computing unit having an algorithm that is capable of analyzing features in the images taken in step (b)(i) to determine the confidence of the detection result;

[0356] (c) if step (b) determines that the detection result is not trustworthy, then discarding the detection result produced in step (a);

[0357] wherein step (a) has one or more unpredictable and random operating conditions.

[0358] In certain embodiments, the algorithm in OA-1 is machine learning, artificial intelligence, statistical methods, etc., or a combination thereof.

[0359] The term “operating conditions” when referring to conducting an assay refers to the conditions under which the assay is conducted. Operating conditions include, but are not limited to, at least three categories: (1) sample-related defects, (2) sample holder-related defects, (3) measurement process-related defects. The term “defect” refers to a deviation from ideal conditions.

[0360] Examples of sample-related defects include, but are not limited to, air bubbles in the sample, dust in the sample, a foreign object (i.e., an object that is not from the original sample but enters the sample later), a certain portion of the sample dried out, a dry texture structure in the sample, an insufficient amount of sample, an incorrect sample, no sample, a sample with an incorrect matrix (e.g., blood, saliva), an incorrect reaction reagent with the sample, an incorrect detection range of the sample, an incorrect signal uniformity of the sample, an incorrect distribution of the sample, a sample holder with incorrect sample position (e.g., blood cells under a spacer), etc.

[0361] Examples of defects associated with the sample holder include, but are not limited to, missing spacers in the sample holder, sample holder not properly closed, sample holder damaged, sample holder surface contaminated, reagents on the sample not properly prepared, sample holder in incorrect position, sample holder with incorrect spacer height, large surface roughness, incorrect transparency, incorrect absorbance, no sample holder, sample holder with incorrect optical properties, sample holder with incorrect electrical properties, incorrect geometry (size), thickness of sample holder, etc.

[0362] Examples of defects associated with the measurement process include, but are not limited to, light intensity, camera conditions, out-of-focus sample in the image taken by the imager, temperature of the light, color of the light, leakage of ambient light, distribution of the light, lens conditions, filter conditions, optical component conditions, electrical component conditions, assembly conditions of the instrument, relative position of the sample, sample holder and instrument, etc.A. Thin layer sample on a solid surface.

[0363] A1. A method for assaying a sample having random and unpredictable one or more operating conditions, comprising:

[0364] (a) providing a sample containing or suspected of containing an analyte;

[0365] (b) depositing the sample onto a solid surface;

[0366] (c) measuring the sample after step (b) to detect the analyte and generating a result of the detection, wherein the result is achievable by one or more operating conditions at which the assay is performed, and wherein the operating conditions are random and unpredictable;

[0367] (d) imaging a portion of the sample area / volume in which the analyte in the sample is measured in step (c); and

[0368] (e) determining a probability of error risk of the result measured in step (c) by analyzing one or more operating conditions shown in one or more images generated in step (d).

[0369] Further, if step (e) determines that the result measured in step (c) has a high probability of error risk, the result will be discarded.

[0370] AD1. An apparatus for assaying an analyte present in a sample under one or more operating variables, comprising:

[0371] (a) a solid surface having a sample contact area for receiving a thin layer of a sample, the sample containing an analyte to be measured;

[0372] (b) an imager configured to image a portion of the sample contact area that contains the analyte under measurement; and

[0373] (c) a non-transitory computer readable medium having instructions that, when executed, perform a determination of the reliability of the assay result by analyzing operational variables shown in the image of the portion of the sample.

[0374] B. Thin layer sample between two plates

[0375] B1. A method for assaying a sample having one or more operational variables, comprising:

[0376] (a) depositing a sample containing an analyte between a first plate and a second plate; wherein the sample is sandwiched between the substantially parallel first plate and second plate;

[0377] (b) measuring the analyte contained in the sample to produce a result, wherein the measurement involves one or more random and unpredictable operational variables;

[0378] (c) imaging a region portion of the first plate and the second plate to produce an image, wherein the region portion contains the sample and the analyte contained in the sample is measured; and

[0379] (d) determining whether the result measured in step (b) is reliable by analyzing operational variables shown in the image of the region portion containing the sample.

[0380] Further, if the analysis in step (d) determines that the result measured in step (b) is not reliable, discarding the result.C. Thin layer sample with a spacer between two plates

[0381] C1. A method for assaying a sample having one or more operational variables, comprising:

[0382] (a) depositing a sample containing or suspected of containing an analyte between a first plate and a second plate, wherein the sample is sandwiched between the first plate and the second plate, the first plate and the second plate are movable relative to each other into a first configuration or a second configuration;

[0383] (b) measuring the analyte in the sample to produce a result, wherein the measurement involves one or more random and unpredictable operational variables;

[0384] (c) imaging a region portion of the sample that contains the analyte under measurement; and

[0385] (d) determining whether the result measured in step (b) is reliable by analyzing operational variables shown in the image of the region portion.

[0386] wherein the first configuration is an open configuration in which the two plates are partially or completely separated, the spacing between the plates is not regulated by spacers, and a sample is deposited on one or both of the plates; and

[0387] wherein the second configuration is a closed configuration configured after the sample is deposited in the open configuration and the plates are forced to the closed configuration by exerting an imprecise pressing force on the force area; and in the closed configuration at least a portion of the sample is compressed by the two plates into a layer of highly uniform thickness and is substantially stagnant with respect to the plates, wherein the uniform thickness of the layer is limited by the sample contact area of the two plates and regulated by the plates and spacers.

[0388] Further, if step (d) determines that the result measured in step (b) is not trustworthy, the result is discarded.

[0389] D1. A method for assaying a sample having one or more operational variables, comprising:

[0390] (a) depositing a sample containing or suspected of containing an analyte, wherein the sample is deposited in an area in a device of any of the embodiments described in the present disclosure;

[0391] (b) measuring the analyte in the sample, wherein the measurement involves one or more random and unpredictable operational variables;

[0392] (c) imaging a portion of the sample area, wherein the portion is where the analyte is measured; and

[0393] (d) determining whether the result measured in step (b) is trustworthy by analyzing the operational variables shown in the image of the portion of the sample.

[0394] Further, if the analysis in step (d) determines that the result measured in step (b) is not trustworthy, the result is discarded.

[0395] E. Assay of a benefit assay device

[0396] E.1 In certain embodiments, the assay is performed using multiple assay devices, wherein the assay has a step of using image analysis to check whether the assay result is trustworthy, and wherein if a first assay device is found to be untrustworthy, a second assay device is used until the assay result is found to be trustworthy.

[0397] In some embodiments, the sample is a biological or chemical sample.

[0398] In certain embodiments, in step (d) the analysis uses machine learning with a training set to determine whether the result is trustworthy, wherein the training set uses operational variables with known analytes in samples.

[0399] In certain embodiments, in step (d), the analysis uses a lookup table to determine whether the result is trustworthy, wherein the lookup table contains operational variables for known analytes in the sample.

[0400] In certain embodiments, in step (d), the analysis uses a neural network to determine whether the result is trustworthy, wherein the neural network is trained using operational variables for known analytes in the sample.

[0401] In certain embodiments, in step (d), the analysis uses a threshold value for the operational variable to determine whether the result is trustworthy.

[0402] In certain embodiments, in step (d), the analysis uses machine learning, a lookup table, or a neural network to determine whether the result is trustworthy, wherein the operational variable comprises a condition of bubbles and / or dust in the image of the portion of the sample.

[0403] In certain embodiments, in step (d), the analysis uses machine learning (which determines whether the result is trustworthy), uses machine learning, a lookup table, or a neural network to determine an operational variable of bubbles and / or dust in the image of the portion of the sample.

[0404] In some embodiments, in step (b) of measuring the analyte, the measuring uses imaging.

[0405] In some embodiments, step (b) of measuring the analyte, the measuring uses imaging, and the same image used for the analyte measurement is used for the trustworthiness determination in step (d).

[0406] In some embodiments, step (b) of measuring the analyte, the measuring uses imaging, and the same imager used for the analyte measurement is used for the trustworthiness determination in step (d).

[0407] In some embodiments, the device used in A1, B1, and / or C1 further comprises a monitoring marker.

[0408] The method, device, computer program product, or system of any of the preceding embodiments, wherein the monitoring marker is used as a parameter with the imaging processing method in the algorithm that (i) adjusts the image, (ii) processes the image of the sample, (iii) determines a characteristic related to the microfeature, or (iv) any combination of the above.

[0409] The method, device, computer program product, or system of any of the preceding embodiments, wherein the monitoring marker is used as a parameter with step (b).

[0410] The method, device, computer program product or system of any of the preceding embodiments, wherein the spacers are the monitoring markers, wherein the spacers have a substantially uniform height equal to or less than 200 microns and a fixed spacer interval (ISD);

[0411] In some embodiments of the application, the monitoring markers are used to estimate TLD (true lateral dimension) as well as true volume estimation.

[0412] In certain embodiments, step (b) further comprises image segmentation for image-based assays.

[0413] In certain embodiments, step (b) further comprises focus check in image-based assays.

[0414] In certain embodiments, step (b) further comprises uniformity of analyte distribution in the sample.

[0415] In certain embodiments, step (b) further comprises analysis and detection of aggregated analytes in the sample.

[0416] In certain embodiments, step (b) further comprises analyzing dry texture structures in the sample image in the sample.

[0417] In certain embodiments, step (b) further comprises analysis of defects in the sample.

[0418] In certain embodiments, step (b) further comprises correction of camera parameters and conditions such as distortion correction, temperature correction, brightness correction, contrast correction.

[0419] In certain embodiments, step (b) further comprises methods and operations with histogram-based operations, math-based operations, convolution-based operations, smoothing operations, derivative-based operations, morphology-based operations.

[0420] F. System for reliability check

[0421] Figure 8 is a schematic of possible defects (e.g. dust, bubbles, etc.) that can occur in a sample at the sample holder. Figure 9 is an image of a real blood sample containing multiple defects that occur in image-based assays.

[0422] F1. A system for assaying a sample with one or more unknown operating conditions, comprising:

[0423] a) adding the sample to a sample holding device, e.g. a QMAX device, with a gap proportional to the size of the analyte to be analyzed or the analyte forms a monolayer between the gaps;

[0424] b) taking an image of the sample in the sample holding device on the area of interest (AoI) with an imager for assay;

[0425] c) segmenting the image of the sample taken by the imager from (b) into equal-sized and non-overlapping sub-image patches (e.g. 8x8 equal-sized small image patches);

[0426] d) performing machine learning based inference on analyte detection and segmentation of each image patch using a trained machine learning model to determine, but not limited to, analyte count and its concentration;

[0427] e) ordering the constructed analyte concentration of sub-image patches in ascending order, determining its 25th percentile Q1 and 75th percentile Q3;

[0428] f) determining the uniformity of analyte in the sample image using a confidence measure based on inter-quantile range: confidence - IQR = (Q3 - Q1) / (Q3 + Q1); and

[0429] g) raising a flag if the confidence - IQR from (f) exceeds a certain threshold (e.g. 30%), where the threshold is derived from training / evaluation data or from physical rules governing analyte distribution, and the assay result is not trustworthy.

[0430] F2. A system for assaying a sample with one or more unknown operating conditions, comprising:

[0431] a) adding analytes to a sample holding device, e.g. a QMAX device, whose gap is proportional to the size of the analytes to be analyzed, or the analytes form a monolayer between the gaps;

[0432] b) taking an image of the sample in the sample holding device on the area of interest (AoI) with an imager for assay;

[0433] c) performing machine learning based inference using a trained machine learning model for dry texture structure detection and segmentation, thereby detecting dry texture structure regions, and determining the area of dry texture structure in the AoI associated with the segmentation contour masks of those dry texture structure regions in the area covering the image of the sample;

[0434] d) determining the area ratio between the area of dry texture structure in the AoI and the area of the AoI: Area ratio of dry texture structure in the AoI = Area of dry texture structure in the AoI / Area of the AoI; and

[0435] h) If the area ratio from (d) clustered analyte area in AoI exceeds a certain threshold (e.g. 40%), raise a flag and the assay result is not trustworthy, where the threshold is derived from training / evaluation data or from physical rules governing analyte distribution.

[0436] F3. A system for assaying a sample with one or more unknown operating conditions, comprising:

[0437] a) adding analytes to a sample holding device, e.g. QMAX device, whose gap is proportional to the size of the analytes to be analyzed, or the analytes form a monolayer between the gaps;

[0438] b) taking an image of the sample in the sample holding device on an area of interest (AoI) for assaying with an imager;

[0439] c) performing machine learning based inference with a trained machine learning model for clustered analyte detection and segmentation, to detect clustered analytes and determine the area associated with the segmentation contour mask covering the clustered analytes in its AoI (clustered analyte area in AoI);

[0440] d) assaying the area ratio between clustered analyte area in AoI and AoI area: clustered analyte area in AoI ratio = clustered analyte area in AoI / area of AoI; and

[0441] i) If the clustered analyte area ratio from (d) exceeds a certain threshold (e.g. 40%), raise a flag and the assay result is not trustworthy, where the threshold is derived from training / evaluation data or from physical rules governing analyte distribution.

[0442] F4. A system for assaying a sample with one or more unknown operating conditions, comprising:

[0443] a) adding analytes to a sample holding device, e.g. QMAX device, whose gap is proportional to the size of the analytes to be analyzed, or the analytes form a monolayer between the gaps;

[0444] b) taking an image of the sample in the sample holding device on an area of interest (AoI) for assaying with an imager;

[0445] c) performing machine learning based inference with a trained machine learning model for defect detection and segmentation in the image of the sample, where the defects include (but not limited to) dust, oil, etc., to detect the defects and determine the area associated with the segmentation contour mask covering the defects in its AoI (defect area in AoI);

[0446] d) determining the ratio of the defect area in the AoI to the area of ​​the AoI: defect area ratio in the AoI = defect area in the AoI / area of ​​the AoI; and

[0447] e) If the defect area ratio in the AoI from (d) exceeds a certain threshold (e.g., 15%), a flag is raised and the assay result is not trustworthy, where the threshold is derived from training / evaluation data or from physical rules governing analyte distribution.

[0448] F5. A system for measuring a sample having one or more unknown operating conditions, comprising:

[0449] a) adding the analyte to a sample holding device, such as a QMAX device, whose gaps are proportional to the size of the analyte to be analyzed, or the analyte forms a monolayer between the gaps;

[0450] b) capturing an image of the sample in the sample holding device over an area of ​​interest (AoI) for measurement using an imager;

[0451] c) performing machine learning-based inference using the trained machine learning model for bubble and air gap detection and segmentation to detect bubbles and air gaps and determine the area of ​​the bubble air gaps in the AoI associated with the segmentation contour mask covering them in their AoI;

[0452] d) Determine the area ratio between the area of ​​the interstitial spaces in the AoI and the area of ​​the AoI:

[0453] Bubble gap area ratio in AoI=bubble gap area in AoI / area of ​​AoI; and

[0454] k) If the ratio of the bubble interstitial area in the AoI from (d) exceeds a certain threshold (e.g., 10%), a flag is raised and the assay result is not trustworthy, where the threshold is derived from training / evaluation data or from physical rules governing analyte distribution.

[0455] F6. A system for measuring a sample having one or more unknown operating conditions, comprising:

[0456] a) loading the analyte into a sample holding device, such as a QMAX device, wherein the sample holding device has gaps proportional to the size of the analyte to be analyzed, or the analyte forms a monolayer between the gaps, and there are monitoring labels (e.g., pillars) that reside in the device and are not immersed and can be imaged along with the sample on the sample holding device by an imager;

[0457] b) capturing an image of the sample in the sample holding device over an area of ​​interest (AoI) for measurement using an imager;

[0458] c) performing a machine learning based inference using the trained machine learning model to detect and segment the monitoring marker (column) with the analyte on top, determining the area associated with the detected monitoring marker (column) based on the segmented contour mask of the detected monitoring marker (column) in the AoI (AoI column on analyte area);

[0459] d) determining the area ratio between the column on analyte area in the AoI and the area of the AoI:

[0460] Column on analyte area ratio in the AoI = Column on analyte area in the AoI / Area of the AoI; and

[0461] l) if the column on analyte area ratio in the Ao1 from (d) exceeds a certain threshold (e.g. 10%), raise a flag and the assay result is not trustworthy, wherein the threshold is derived from training / evaluation data or from physical rules governing analyte distribution.

[0462] F7. A system for assaying a sample with one or more unknown operating conditions, comprising:

[0463] a) adding an analyte to a sample holding device, e.g. a QMAX device;

[0464] b) taking an image of the sample in the sample holding device on an area of interest (AoI) with an imager for assaying;

[0465] c) performing a machine learning based focus check to detect whether the image of the sample taken by the imager is in focus on the sample, wherein the machine learning model for detecting the focus of the imager is constructed from a plurality of images of the imager with known in-focus and out-of-focus conditions; and

[0466] d) if the image of the sample taken by the imager is detected to be out of focus in (c), raise a flag and the assay result based on the image is not trustworthy.

[0467] F8. A system for assaying a sample with one or more unknown operating conditions, comprising:

[0468] a) loading an analyte to a sample holding device, e.g. a QMAX device;

[0469] b) taking an image of the sample in the sample holding device on an area of interest (AoI) with an imager for assaying;

[0470] c) performing a machine learning based analyte detection; and

[0471] d) if the analyte count is outside of a pre-set acceptable range very low, raise a flag, the result is unreliable, where the acceptable range is specified based on the physical or biological conditions of the assay.

[0472] F9. A system for assaying a sample with one or more unknown operating conditions, comprising:

[0473] a) adding an analyte to a sample holding device, such as a QMAX device;

[0474] b) taking an image of the sample in the sample holding device on an area of interest (AoI) with an imager for assaying;

[0475] c) segmenting the image of the sample into non-overlapping, equal-sized sub-image patches;

[0476] d) performing machine learning based analyte detection on each of its sub-image patches; and

[0477] e) if for some sub-image patches, the count of detected analyte is unrealistically low (e.g., in a complete blood count, the number of red blood cells in the sample is lower than the human acceptable range), raise a flag, and the result is unreliable for a sample with insufficient or non-uniform distribution in the assay.

[0478] F10. In all the methods, devices, and embodiments described in the preceding claims, wherein the detection and segmentation of anomalies from the image of the sample taken by the imager in the image-based assay is based on image processing, machine learning, or a combination of image processing and machine learning.

[0479] F11. In all the methods, devices, and embodiments described in the preceding claims, the estimation of the area covered by the segmentation mask in the area of interest (AoI) of the image of the sample is compensated for distortions in micro-imaging, including but not limited to, spherical distortions from the lens, imperfections at the micro-level, misalignments in focusing, etc., with real lateral dimension (or field of view (FoV)) estimation based on each image or each sub-image patch.

[0480] F12. In the methods, devices, and embodiments described in F11, a monitoring marker (e.g., a post) is built-in in the sample holding device (e.g., a QMAX card); and the monitoring marker (e.g., a post) is used as a detectable anchor point to make the estimation of real lateral dimension (or field of view (FoV)) estimation accurate in the face of distortions in micro-imaging.

[0481] F13. In the methods, devices, and embodiments described in F12, the monitoring markers (e.g., pillars) of the sample holding device have some known configuration of distribution with prescribed fixed spacing in the sample holding device, such as QMAX cards, to make the detection and localization of the monitoring markers reliable and robust as anchor points in the estimation of true lateral dimensions (TLDs) (or field of view (FoV)).

[0482] F14. In the methods, devices, and embodiments described in F1, the detection and characterization of outliers in the image-based assay is based on non-overlapping sub-image patches of the input image of the sample described herein, and the determination of outliers can be based on non-parametric methods, parametric methods, and combinations of both during the assay process.

[0483] G-1 A method comprising:

[0484] (a) detecting an analyte in a sample comprising or suspected of comprising the analyte, the detecting comprising:

[0485] (i) depositing the sample into a detection instrument, and

[0486] (ii) measuring the sample using the detection instrument to detect the analyte, thereby generating a detection result;

[0487] (B) determining a reliability of the detection result, the determining comprising:

[0488] (i) taking one or more images of a portion of the sample and / or a portion of the detection instrument proximal to the portion of the sample, wherein the one or more images reflect one or more operating conditions under which the detection result was generated; and

[0489] (ii) analyzing the one or more images using a computing device having an algorithm to determine the reliability of the detection result in step (a); and

[0490] (c) reporting the detection result and the reliability of the detection result;

[0491] wherein the one or more operating conditions are unpredictable and / or random.

[0492] The term “unreliable” in the context of an assay result means that for an assay of a given sample, the assay result is not always accurate: sometimes the result of the assay is accurate, but sometimes the result is inaccurate, where an inaccurate result is substantially different from an accurate result. Such an inaccurate result is referred to as a “false result”. In some literature, a false result is also referred to as an “outlier”.

[0493] The term “accurate” in the context of an assay result means that within the allowable arrangement, the assay result is consistent with the result of the same sample assayed by a gold standard instrument operated by a trained professional in an ideal environment.

[0494] Traditionally, diagnostic assays are typically performed using complex (and often expensive) instruments and require highly trained personnel and complex infrastructure, which are not available in limited resource settings.

[0495] The term "limited resource setting" or "LRS" for an assay of a sample refers to a setting in which the assay is performed using a simplified / low cost assay process or a simplified / low cost instrument, by untrained personnel, in an adverse environment (e.g. an open and non-laboratory environment with dust), or any combination thereof.

[0496] The term "LRS assay" refers to an assay performed in a LRS.

[0497] The term "credibility" describing the reliability of a particular assay result (or data) refers to a reliability analysis of a particular assay result determining that the result has a low probability of inaccuracy.

[0498] The term "incredibility" describing the reliability of a particular assay result (or data) refers to a reliability analysis of a particular assay result determining that the result has a high probability of inaccuracy.

[0499] The term "operating conditions" when performing an assay refers to the conditions in which the assay is performed. Operating conditions include, but are not limited to, air bubbles in the sample, dust in the sample, foreign objects (i.e. objects that are not from the original sample but rather enter the sample at a later time), imperfections of the solid surface, and / or processing conditions of the assay.

[0500] When assaying a sample in a limited resource setting (LRS), the assay result can be unreliable. Traditionally, however, the reliability of a particular result is not examined during or after a particular test of a given sample.

[0501] The present invention observes that in a LRS assay (or even in a laboratory testing environment), one or more unpredictable random operating conditions can occur and affect the assay result. When this happens, even using the same sample, from one particular assay to the next assay can be substantially different. Instead of taking the assay result as is, however, the reliability of a particular result in a particular test of a given sample can be assessed by analyzing one or more factors related to the assay operating conditions in the particular assay.

[0502] The present invention observes that in a LRS assay with one or more unpredictable random operating conditions, by using an analysis of the reliability of each particular assay and by rejecting incredulous assay results, the overall accuracy of the assay can be significantly improved.

[0503] One aspect of the present invention is such devices, systems and methods that perform the analysis by not only measuring the analyte in a particular test, but also by analyzing the operating conditions of the particular test to check the reliability of the measurement results.

[0504] In some embodiments of the present invention, the checking of the reliability of the measurement results of the assay is modeled in a machine learning framework, and machine learning algorithms and models are designed and applied to handle unpredictable random operating conditions that occur and affect the assay results.

[0505] The term“machine learning” refers to algorithms, systems and devices in the field of artificial intelligence that generally use statistical techniques and artificial neural networks to provide computers with the ability to“learn” from data (i.e., to progressively improve performance on a particular task) without being explicitly programmed.

[0506] The term“artificial neural network” refers to a hierarchical connected system inspired by biological networks that can“learn” to perform a task by considering examples, typically without being programmed with any task-specific rules.

[0507] The term“convolutional neural network” refers to a class of multi-layer feed-forward artificial neural networks that are most commonly applied to the analysis of visual images.

[0508] The term“deep learning” refers to a broad class of machine learning methods in artificial intelligence (AI) that learn from data with network structures composed of many connected layers.

[0509] The term“machine learning model” refers to a trained computational model built from the training process in machine learning from data. The machine learning model trained by a computer during the derivation phase gives the computer the ability to perform a specific task (e.g., detecting and classifying objects). Examples of machine learning models include ResNet, DenseNet, etc., which are also referred to as“deep learning models” due to the hierarchical depth in their network structure.

[0510] The term“image segmentation” refers to an image analysis process that segments a digital image into multiple segments (sets of pixels, with a set of bit-masked masks covering the image segment along their segment boundary contours). Image segmentation can be achieved by image segmentation algorithms in image processing such as watershed, iterative graph cut, mean shift, etc., or by machine learning algorithms such as MaskRCNN, etc.

[0511] The term“object” or“object of interest” in an image refers to an object that is visible in the image and has a fixed shape or form.

[0512] In the present invention, the innovative use of machine learning has the advantage that the process of automating the determination of the confidence of the assay result is directly confronted with unpredictable random operating conditions from the data in the assay without making explicit assumptions about the unpredictable conditions that can be complex, difficult to predict and prone to error.

[0513] The machine learning framework in the present invention involves a process that includes:

[0514] (a) collecting training task data;

[0515] (b) preparing the data with labels;

[0516] (c) selecting a machine learning model;

[0517] (d) training the selected machine learning model with the training data;

[0518] (e) adjusting the hyperparameters and model structure with training and evaluation data until the model achieves satisfactory performance on the evaluation and test data; and

[0519] (f) using the trained machine learning model from (e) to perform inference on the test data.

[0520] Image segmentation for image-based assays: In some embodiments of the present invention for validating the confidence of the test results, it is required to segment the objects of interest from the images of the samples for analysis. Although the machine learning based image segmentation algorithms (e.g. Mask RCNN) are powerful, they require precise contour labeling of the shape of the objects in the microscopic images of the samples to train the machine learning model, which has become a bottleneck for many applications. In addition, they are very sensitive to the shape of the objects in the images. For image-based assays, it is difficult to make such labeling of the shape contour of the objects because the objects in the samples can be very small, their appearance is random, and, they have huge variations in shape, size, and color (e.g. dust, bubbles, etc.).

[0521] In some embodiments of the present invention, a fine image segmentation algorithm is designed based on the combination of machine learning based coarse bounding box segmentation and image processing based fine shape determination. It is applied to the image segmentation in image-based assays, where each object only needs to be labeled in a coarse bounding box, independent of its shape and shape contour details. In this way, the need for fine labeling of the shape-dependent contour of the objects in the images of the samples is eliminated, which is difficult, complex, expensive and difficult to be accurate. The fine image segmentation algorithm includes:

[0522] a) collecting a plurality of images of a sample taken by an imager, the plurality of images containing objects to be detected in the images of the sample for further analysis;

[0523] b) labeling each object in the collected images with a coarse bounding box containing the object for model training;

[0524] c) training a machine learning model (e.g., FRCNN) to detect objects in images of samples having coarse bounding boxes containing the objects;

[0525] d) using images of a sample as input in an assay;

[0526] e) applying the trained machine learning model to detect objects in images of the sample having their coarse bounding boxes;

[0527] f) transforming each image patch corresponding to a detected bounding box to gray and then to binary with adaptive thresholding;

[0528] g) performing morphological dilation (7x7) and erosion (3x3) to enhance the outline of the shape from background noise;

[0529] h) performing a convex hull analysis on each image patch, taking the longest connected outline found in the image patch as the outline of the shape of the object, thereby determining an image mask of the object (e.g., a binary bitmap covering the object in the sample image); and

[0530] i) completing image segmentation by collecting all image masks from (h).

[0531] (if additional margin delta segmentation mask is needed, dilate each detected outline in (h) by margin delta as new mask)

[0532] Figure 10 is an example of the described fine image segmentation algorithm applied to images of blood samples depicted in Figure 9 The described fine image segmentation algorithm can be suitably applied in image-based assays with objects of different sizes and shapes, where very tight mask coverage of objects in sample images is required, as shown in the example.

[0533] Focus check in image-based assays: In image-based assays, the image of the sample taken by the imager needs to be in focus on the sample by the imager to conduct the assay, and out-of-focus in the image of the sample taken by the imager blurs the analytes in the image of the sample, thus the assay result becomes untrustworthy. However, there are many random factors that can cause the image of the sample to be partially or even completely out-of-focus, including but not limited to vibration / hand-shake during the image taken, misalignment of the sample holding device with the image sensor plane, etc. Moreover, the prior art mainly relies on certain edge content based measurements, such as Tenengrad, etc., and some pre-set content dependent thresholds, which are unreliable, fragile, and insufficient for the requirements in micro-imaging based assays.

[0534] In some embodiments of the present invention, a machine learning based verification process is designed and applied to determine whether the image of the sample taken by the imager in an image-based assay is in focus or out-of-focus, where the images of the sample taken by the imager under in-focus and out-of-focus conditions are collected as training data, and are labeled based on their known in-focus conditions. A machine learning model is selected and trained with the labeled training data. During the assay process, the trained machine learning model is applied to the image of the sample taken by the imager to deduce / predict whether the image of the sample taken by the imager for the assay is in focus or not, and to decide whether the assay result is trustworthy or not, without the need of pre-set content dependent thresholds in the prior art.

[0535] In some embodiments, monitoring marks in the form of columns are located on the sample holding QMAX card to keep the gap between the two parallel plates of the sample holding QMAX card uniform. In this way, the sample volume under the area of interest (AoI) on the sample holding QMAX card taken by the imager can be determined by the AoI and the gap between its two parallel plates.

[0536] Uniformity of analyte distribution in the sample: One factor that can affect the reliability of the assay result is the uniformity of the analytes distributed in the sample, and it is difficult for even experienced technicians to detect them by eyeball check.

[0537] In some embodiments of the present invention, an algorithm with a machine learning based dedicated process is designed and applied to determine from the image of the sample taken by the imager whether the analytes are uniformly distributed in the sample for the assay, where multiple images of the sample taken by the imager are collected, from which the analytes in the images of the sample are identified and labeled. A machine learning model, such as F-RCNN, is selected and trained with the labeled training images to detect the analytes in the sample images.

[0538] During the assay process, the algorithm with its dedicated process includes:

[0539] a) using images of the sample from an imager as input;

[0540] b) segmenting the sample images into equally sized and non-overlapping image patches (e.g. 8x8 equally sized small image patches);

[0541] c) applying a trained machine learning model to each constructed image patch to determine, without limitation, the analyte concentration in each patch;

[0542] d) ordering the constructed image patches in ascending order of their analyte concentration and determining the 25th percentile Qi and the 75th percentile Q3 of the ordered concentration sequence of the constructed image patches;

[0543] e) applying a robust, non-parametric outlier detection algorithm to determine the uniformity / homogeneity of the analyte in the sample under assay from the analyte concentration distribution of the constructed image patches, wherein in some embodiments of the invention a confidence measure based on the inter-quartile range is constructed and applied:

[0544] confidence - IQR = (Q3 - Qi) / (Q3 + Qi); and

[0545] if the confidence - IQR exceeds a certain threshold (e.g. 30%), a flag is raised and the assay result is not trustworthy, wherein the threshold depends on the impact of the non-uniform distribution on the final estimate, which can be estimated from the training and evaluation data.

[0546] Aggregated analyte in the sample: In addition, aggregated analytes in the sample can affect the accuracy of the assay result, especially if they occupy a significant portion of the sample. For example, in a complete blood count, certain portions of red blood cells can aggregate in the sample, especially if they are exposed to open air for a period of time. Aggregated analytes in the sample have various sizes and shapes depending on how they aggregate together. If the portion of aggregated analytes in the sample exceeds a certain percentage, the sample should not be used for the assay.

[0547] In some embodiments of the application, a machine learning based process is designed and applied to determine whether the analyte is clustered / aggregated in the sample for analysis from the images of the sample taken by the imager, where images of good samples and samples with varying degrees of aggregated analyte in the sample are taken by the imager and collected as training data. The aggregated analytes in the images are first roughly labeled by bounding boxes regardless of their shape and shape contour details following the fine image segmentation algorithm described in the present application. A machine learning model (e.g. Fast RCNN) is selected and trained with the labeled training images to detect the aggregated analyte clusters in the images of the sample with their bounding boxes, then additional processing steps are performed to determine their fine segmentation based on the fine segmentation image segmentation algorithm described in the present application.

[0548] During the assay process, the operation of the aggregated analyte includes:

[0549] a) using the images of the sample from the imager as input;

[0550] b) applying a trained machine learning model for the aggregated analyte to detect the aggregated analyte in the images of the sample under analysis in the bounding boxes;

[0551] c) determining their segmentation contour masks in the images of the sample following the fine image segmentation algorithm in the present application;

[0552] d) determining the total area occupied by the aggregated analyte in the area of interest (AoI) in the sample image by summing up all the areas associated with their segmentation contour masks from (c);

[0553] e) determining the area ratio between the aggregated analyte area in the AoI and the AoI area in the sample image:

[0554] Aggregated analyte area ratio in the AoI = Aggregated analyte area in the AoI / Area of the AoI; and

[0555] f) if the aggregated analyte area ratio in the AoI exceeds a certain threshold, a confidence in the assay result is raised, where in some embodiments of the method, the threshold is about 10-20%, where the threshold depends on the impact of the aggregated analyte area on the final estimate, which can be estimated from the training and evaluation data.

[0556] Dry texture structures in the images of the sample: Dry texture structures in the images of the sample are another factor that affects the confidence in the assay result in image based assays. This happens when the amount of the sample used for the assay is lower than the required amount or certain parts of the sample in the image holding device become dry due to some unpredictable factors.

[0557] In some embodiments of the present application, a machine learning based process is designed and applied to detect dry texture structure regions in images of samples taken by an imager in an image based assay, where images of good samples without dry texture structure regions and images of samples with various degrees of dry texture structure regions in the samples are collected as training data, from which dry texture structure regions in images are roughly labeled by bounding boxes regardless of shape and shape contour details. A machine learning model (e.g., Fast RCNN) is selected and trained using the labeled training images to detect dry texture structure regions in images of samples with bounding boxes, and then a fine image segmentation algorithm described in the present application is followed to determine segmentation contour masks covering them.

[0558] In an image based assay process, it performs the following processing operations, including:

[0559] a) taking images of samples from an imager as input;

[0560] b) applying a trained machine learning model (e.g., Fast RCNN) for dry texture structure to the images of samples for assay and detecting their dry texture structure regions in bounding boxes;

[0561] c) if dry texture structure regions are detected in the images of samples in (b), determining segmentation contour masks by a fine image segmentation algorithm in the present application;

[0562] d) determining the total area occupied by dry texture structure in the area of interest (AoI) in the images of samples for assay by summing up all the areas of detected dry texture structure based on the segmentation contour masks covering the dry texture structure in (c);

[0563] e) determining the area ratio between the dry texture structure area in the AoI and the area of the AoI in the sample images:

[0564] Dry texture structure area ratio in AoI = dry texture structure area in AoI / area of AoI; and

[0565] f) if the dry texture structure area ratio in the AoI exceeds a certain threshold, a confidence level of the assay result is raised, where in some embodiments of the method, the threshold is about 10%, where the threshold depends on the impact of the dry texture structure area on the final estimate, which can be estimated from training and evaluation data.

[0566] Sample Defects: Defects in a sample can severely affect the reliability of the assay result, where these defects can be any unwanted objects in the sample, including but not limited to dust, oil, etc. They are difficult to handle with the prior art because their occurrence and shape in the sample are random.

[0567] In some embodiments of the present application, a dedicated process for defect detection is designed and applied to image-based assays, where images of good samples without defects and images of samples with different degrees of defects in the sample are collected as training data, according to which the defect areas in the images are labeled with coarse bounding boxes. A machine learning model (e.g., Fast RCNN) is selected and trained using the labeled training images to detect defects in the images of the sample in the bounding boxes, and then the fine image segmentation algorithm described in the present application is applied to determine the segmentation contour masks covering them.

[0568] During the image-based assay process, defect detection and area determination are performed to verify the credibility of the assay result, including:

[0569] a) using images of the sample from the imager as input;

[0570] b) applying a machine learning model trained for defects (e.g., Fast RCNN) to the images of the sample for assay and detecting defects in the bounding boxes;

[0571] c) determining their segmentation contour masks in the fine image segmentation algorithm following the present application;

[0572] d) determining the total area occupied by defects in the area of interest (AoI) in the images of the sample for assay by summing up all the areas of the detected defects based on the segmentation contour masks covering the defects from (c);

[0573] e) determining the area ratio between the defect area in the AoI and the area of the AoI in the images of the sample:

[0574] Defect area ratio in the AoI = defect area in the AoI / area of the AoI; and

[0575] f) if the defect ratio in the AoI exceeds a certain threshold, a credibility raised flag for the assay result, where in some embodiments of the method, the threshold is about 15%, where the threshold depends on the impact of the defect area on the final estimate, which can be estimated from the training and evaluation data.

[0576] Air bubbles in sample: Air bubbles in sample are a special type of defects that occur in assays. Their occurrence is random, which can come from the procedure of operation as well as the reaction between analytes and other reagents in the sample. Unlike solid dust, their occurrence is more random as their number, size and shape can vary over time.

[0577] In some embodiments of the present invention, a dedicated process is designed and applied to detect air bubbles in sample in image-based assays, where images of good samples without air bubbles and images of samples with various degrees of air bubbles are collected as training data, according to which air bubbles in images are only roughly labeled by bounding boxes regardless of their shape and shape contour details. A machine learning model (e.g. Fast RCNN) is selected and trained with the labeled training images to detect air bubbles in sample images in the bounding boxes. Then, following the fine image segmentation algorithm described in the present invention, segmentation contour masks covering them in images of samples are determined.

[0578] In image-based assay processes, air bubble detection and area determination are performed in some embodiments of the present invention to verify the credibility of assay results, including:

[0579] a) using images of samples from imagers as input;

[0580] b) applying a trained machine learning model for air bubbles (e.g. Fast RCNN) to the images of samples for assay and detecting their air bubbles in the bounding boxes;

[0581] c) determining their segmentation contour masks in applying the fine image segmentation algorithm in the present invention;

[0582] d) determining the total area occupied by air bubbles in the area of interest (AoI) in the images of samples for assay by summing up all areas of detected air bubbles based on the segmentation contour masks covering air bubbles from (c);

[0583] e) determining the area ratio between air bubble area in AoI and AoI area in the images of samples for assay:

[0584] Air bubble area ratio in AoI = air bubble area in AoI / area of AoI; and

[0585] f) if the air bubble ratio in AoI exceeds a certain threshold, a credibility raised flag for the assay result, where in some embodiments of the method, the threshold is about 10%, where the threshold depends on the impact of air bubble area on the final estimate, which can be estimated from training and evaluation data.

[0586] In some embodiments of the invention, a more stringent threshold is applied to bubbles because the large area occupied by bubbles is an indication of some chemical or biological reaction between components in the sample under assay or some defect / issue in the sample holding device.

[0587] Imaging-based assay using monitoring marks

[0588] In an image-based assay for assaying an analyte in a sample, an imager is used to produce an image of the sample on a sample holder, and the image is used to determine a characteristic of the analyte.

[0589] However, many factors can distort the image (i.e., different from the image under real sample or perfect conditions). The image distortion can lead to inaccuracy in the analyte characteristic determination. For example, one fact is poor focus because the biological sample itself does not have the preferred sharp edges in focus. When the focus is poor, the object size will be different from the real object, and other objects (e.g., blood cells) can become unrecognizable. Another example is the lens can be perfect, resulting in different distortions for different locations of the sample. Yet another example is the sample holder is not in the same plane as the optical imaging system, resulting in good focus in one area and poor focus in other areas.

[0590] The present invention relates to devices and methods that are capable of obtaining a “real” image from a distorted image, thus improving the accuracy of the assay.

[0591] One aspect of the present invention is devices and methods using monitoring marks with a flat optically observable surface that is parallel to the adjacent surface.

[0592] Another aspect of the present invention is devices and methods using a QMAX card to make at least a portion of the sample into a uniform layer and using the monitoring marks on the card to improve the assay accuracy.

[0593] Another aspect of the present invention is devices and methods using monitoring marks with computational imaging, artificial intelligence, and / or machine learning.

[0594] The term “lateral dimension” refers to a linear dimension in the plane of the thin sample layer being imaged.

[0595] The terms “true lateral dimension (TLD)” and “field of view (FoV)” are interchangeable.

[0596] The term "microfeature in a sample" can refer to an analyte, a microstructure, and / or a microvariation of a substance in a sample. An analyte refers to a particle, a cell, a macromolecule, such as a protein, a nucleic acid, and other moieties. A microstructure can refer to a microscopic scale difference of different materials. A microscopic variation refers to a microscopic change in a local property of a sample. Examples of a microvariation are a local optical index and / or a local mass variation. Examples of a cell are blood cells, such as white blood cells, red blood cells, and platelets.

[0597] A. Monitoring markers on a solid surface

[0598] A1-1. An apparatus for determining a microfeature in a sample using an imager, the apparatus comprising:

[0599] (a) a solid surface comprising a sample contact area for contacting a sample containing a microfeature; and

[0600] (B) one or more monitoring markers, wherein the monitoring markers:

[0601] i. are made of a material different from the sample;

[0602] ii. are located inside the sample during the determination of the microstructure, wherein the sample forms a thin layer of less than 200 pm in thickness on the sample contact area; and

[0603] iii. have their lateral linear dimensions of about 1 pm (micrometer) or more, and

[0604] iv. have at least one lateral linear dimension of 300 pm or less; and

[0605] wherein at least one monitoring marker is imaged by the imager during the determination;

[0606] wherein the determination of the analyte; and the geometric parameters (e.g., shape and size) of the monitoring markers and / or the spacing between the monitoring markers are (a) predetermined and known prior to the determination of the analyte and (b) used as parameters in an algorithm that determines a property related to the microfeature.

[0607] A1-2. An apparatus for determining a microfeature in a sample using an imager, the apparatus comprising:

[0608] a solid surface comprising a sample contact area for contacting a sample containing a microfeature; and

[0609] one or more monitoring markers, wherein each monitoring marker comprises a protrusion or a groove from the solid surface, wherein:

[0610] v. the protrusion or groove comprises a flat surface that is substantially parallel to an adjacent surface, which is a portion of the solid phase surface adjacent to the protrusion or groove;

[0611] vi. the distance between the flat surface and the adjacent surface is about 200 micrometers (pm) or less;

[0612] vii. the flat surface has (a) a linear dimension of at least about 1 pm or more, and (b) an area with at least one linear dimension of 150 pm or less;

[0613] viii. imaging the flat surface of the monitoring marker by an imager used during the assaying of the microfeature; and

[0614] ix. the shape of the flat surface, the dimensions of the flat surface, the distance between the flat surface and the adjacent surface, and / or the spacing between the monitoring markers are (a) predetermined and known prior to assaying the microfeature, and (b) used as parameters in an algorithm that determines a characteristic related to the microfeature;

[0615] B. Monitoring markers on QMAX cards

[0616] A2-1. An apparatus for analyzing a microfeature in a sample using an imager, the apparatus comprising:

[0617] a first plate, a second plate, spacers, and one or more monitoring markers, wherein:

[0618] i. the first plate and the second plate are movable relative to each other into different configurations;

[0619] ii. each of the first plate and the second plate comprises an inner surface comprising a sample contact area for contacting a sample containing the microfeature;

[0620] iii. one or both of the first plate and the second plate comprises spacers permanently affixed to the inner surface of the respective plate;

[0621] iv. the spacers have a substantially uniform height equal to or less than 200 micrometers and a fixed spacer spacing (ISD).

[0622] v. the monitoring markers are made of a material different from the sample;

[0623] vi. during the assaying of the microstructure, the monitoring markers are located inside the sample, wherein the sample forms a thin layer of less than 200 pm in thickness on the sample contact area; and

[0624] vii. the monitoring markers have a lateral linear dimension of about 1 pm (micrometer) or more, and have at least one lateral linear dimension of 300 pm or less.

[0625] wherein at least one monitoring marker is imaged by the imager during the assay;

[0626] wherein a microfeature is used during the assay; and the shape of the planar surface, the dimensions of the planar surface, the distance between the planar surface and the adjacent surface, and / or the spacing between the monitoring markers are (a) predetermined and known prior to the assay on the microfeature and (b) used as parameters in an algorithm that determines a characteristic related to the microfeature;

[0627] wherein one of the configurations is an open configuration in which the two plates are partially or completely separated, the spacing between the plates is not regulated by the spacers, and a sample is deposited on one or both of the plates; and

[0628] wherein the other of the configurations is a closed configuration that is configured after a sample is deposited in the open configuration and the plates are forced to the closed configuration by applying a pressing force on the force area; and in the closed configuration, at least a portion of the sample is compressed by the two plates into a layer of highly uniform thickness and is substantially stagnant relative to the plates, wherein the uniform thickness of the layer is limited by the sample contact areas of the two plates and regulated by the plates and the spacers.

[0629] wherein the monitoring markers are (i) a different structure from the spacers, or (ii) the same structure used as the spacers.

[0630] A2-2. An apparatus for analyzing a microfeature in a sample using an imager, the apparatus comprising:

[0631] a first plate, a second plate, spacers, and one or more monitoring markers, wherein:

[0632] viii. the first plate and the second plate are movable relative to each other into different configurations;

[0633] ix. each of the first plate and the second plate comprises an inner surface comprising a sample contact area for contacting a sample containing the microfeature;

[0634] x. one or both of the first plate and the second plate comprises a spacer permanently fixed on the inner surface of the respective plate;

[0635] xi. the spacers have a substantially uniform height equal to or less than 200 microns and a fixed spacer spacing (ISD).

[0636] xii. each monitoring marker comprises a protrusion or a groove on one or both sample contact areas;

[0637] xiii. The protrusion or trench includes a flat surface that is substantially parallel to an adjacent surface, which is a portion of the solid phase surface adjacent to the protrusion or trench;

[0638] xiv. The distance between the flat surface and the adjacent surface is about 200 micrometers (pm) or less;

[0639] xv. The flat surface has (a) a linear dimension of at least about 1 pm or more, and (b) an area with at least one linear dimension of 150 pm or less;

[0640] xvi. Imaging the flat surface of at least one monitoring marker by an imager used during the assaying of the microfeature; and

[0641] xvii. The shape of the flat surface, the dimensions of the flat surface, the distance between the flat surface and the adjacent surface, and / or the spacing between monitoring markers are (a) predetermined and known prior to assaying the microfeature and (b) used as parameters in an algorithm that determines a characteristic related to the microfeature;

[0642] wherein one of the configurations is an open configuration in which the two plates are partially or completely separated, the spacing between the plates is not regulated by the spacers, and a sample is deposited on one or both of the plates; and

[0643] wherein the other of the configurations is a closed configuration, which is configured after a sample is deposited in the open configuration and the plates are forced to the closed configuration by exerting a pressing force on the force area; and in the closed configuration, at least a portion of the sample is compressed by the two plates into a layer of highly uniform thickness and is substantially stagnant relative to the plates, wherein the uniform thickness of the layer is limited by the sample-contacting areas of the two plates and regulated by the plates and the spacers.

[0644] wherein the monitoring markers are (i) a different structure than the spacers, or (ii) the same structure used as the spacers.

[0645] A3. An apparatus for an image-based assay, comprising:

[0646] The apparatus of any preceding embodiment, wherein the apparatus has at least five monitoring markers, wherein at least three of the monitoring markers are not aligned on a straight line.

[0647] A4. An apparatus for assaying an analyte in a sample using an imager, the system comprising:

[0648] (a) the apparatus of any of the preceding apparatus embodiments; and

[0649] (b) an imager for assaying a sample containing an analyte.

[0650] A5. A system for performing an imaging-based assay, the system comprising:

[0651] (a) a device as in any of the preceding embodiments;

[0652] (b) an imager for assaying a sample containing an analyte; and

[0653] (c) an algorithm that utilizes the monitoring label of the device to determine a property related to the analyte.

[0654] In some embodiments, the thickness of the thin layer is configured such that for a given analyte concentration, there is a monolayer of analyte in the thin layer. The term “monolayer” means that in the thin sample layer, there is essentially no overlap between two adjacent analytes in the direction perpendicular to the plane of the sample layer.

[0655] C. Monitoring labels with computational imaging artificial intelligence and / or machine learning

[0656] Another aspect of the present invention is the combination of monitoring labels with computational imaging, artificial intelligence, and / or machine learning. It utilizes the process of forming an image from measurements, using algorithms to process the image and map objects in the image to their physical dimensions in the real world. Machine learning (ML) is applied in the present invention to learn the salient features of objects in the sample, which are embedded in a ML model that is built and trained from images of the sample taken by an imager. Intelligent decision logic is built and applied in the derivation process of the present invention in order to detect and classify target objects in the sample according to the knowledge embedded in the ML model. Computational imaging is the process of forming an image indirectly from measurements using algorithms that rely on a large amount of computation.

[0657] A6. A system for assaying an analyte in a sample using an imager, the system comprising:

[0658] (a) a device as in any of the preceding embodiments;

[0659] (b) an imager for assaying a sample containing an analyte; and

[0660] (c) an algorithm that utilizes the monitoring label of the device to determine a property related to the analyte, wherein the algorithm uses machine learning.

[0661] A7. A method for assaying an analyte in a sample using an imager, comprising:

[0662] (a) obtaining a device, apparatus, or system as in any of the preceding embodiments;

[0663] (b) obtaining a sample and depositing the sample on a sample contact area in the device, apparatus, or system of (a), wherein the sample contains an analyte; and

[0664] (c) determining the analyte.

[0665] 8. A method for determining an analyte in a sample using an imager, comprising:

[0666] (a) obtaining the device, apparatus, or system of any of the preceding embodiments;

[0667] (b) obtaining a sample and depositing the sample on a sample contact area in the device, apparatus, or system of (a), wherein the sample contains an analyte; and

[0668] (c) determining the analyte, wherein the determining comprises a step of using machine learning.

[0669] One key idea of the present invention is to use pillars as detectable anchors in a sample holding device (e.g. QMAX device) for calibration and improving accuracy of image-based assays. In a QMAX device, pillars are monitoring marks to keep the gap between the two plates holding the sample in the sample holding device uniform. However, accurately detecting pillars in a sample holding device as anchors for calibration and improving accuracy of assays is a challenge because pillars are infiltrated and surrounded by analytes within the sample holding device. Moreover, their images are distorted and blurred in microscope imaging due to spherical (barrel) distortion of the lens, light diffraction from the microscope object, defects in the microscope level, misalignment of the focus, noise in the sample image, etc. Also, it becomes more difficult if imaging is done through a commercial device (e.g. a camera from a smartphone) because these cameras have no dedicated hardware calibration once they are manufactured.

[0670] In the present invention, pillar detection is formulated into a machine learning framework to detect pillars in a sample holding device (e.g. QMAX device) with accuracy suitable for calibration and accuracy improvement in image-based assays. Since the distribution and physical configuration of pillars are known a priori and controlled by fine nanoscale fabrication (e.g. QMAX device), it makes this innovative approach of using detectable monitoring marks (e.g. pillars) as anchors in image-based assays not only feasible but also effective.

[0671] In some embodiments, the algorithm of any of the preceding embodiments comprises an algorithm that computes imaging, artificial intelligence, and / or machine learning.

[0672] In some embodiments, the algorithm of any of the preceding embodiments comprises a machine learning algorithm.

[0673] In some embodiments, the algorithm comprises an artificial intelligence and / or machine learning algorithm, as in any of the preceding embodiments.

[0674] In some embodiments, the algorithm comprises a computed imaging and / or machine learning algorithm, as in any of the preceding embodiments.

[0675] In some embodiments, it designs a machine learning based column detection to detect columns from images of the sample, and from that, it applies the known configuration and distribution of columns from card manufacturing to construct a good set of points, i.e., points at the image plane and points at the physical plane of the card that correspond to each other. Then, based on the detected good set of points, a corresponding (perspective) transformation is computed, which homography maps the distorted image plane to the physical plane of the sample with correct TLD (true lateral dimension). Figure 6 is a flowchart of true lateral dimension (TLD) estimation and correction using machine learning based on columns, and embodiments of the invention comprise:

[0676] (1) using a sample addition device, such as a QMAX device, in an image based assay, where there are monitoring markers with known configuration in the device that are not submerged in the sample and can be imaged from the top by an imager in the image based assay;

[0677] (2) taking images of the sample in the sample addition device including the analyte and the monitoring markers;

[0678] (3) establishing and training a machine learning (ML) model for detecting the monitoring markers in the sample holding device from the images taken by the imager;

[0679] (4) using the ML detection model in (3) to detect and locate the monitoring markers in the sample addition device from the sample images taken by the imager;

[0680] (5) generating a grid of markers from the monitoring markers detected in (4);

[0681] (6) computing a homography based on the generated grid of monitoring markers; and

[0682] (7) estimating TLD and determining the area, size, and concentration of the imaged analyte in the image based assay.

[0683] The invention can be further improved to perform region based TLD estimation and calibration to improve the accuracy of the image based assay. Embodiments of this approach comprise:

[0684] (1) using a sample addition device, such as a QMAX device, in an image-based assay, where there are monitoring markers that are not submerged in the sample and reside in the device, can be imaged from the top by an imager in the image-based assay;

[0685] (2) taking an image of the sample in the sample holding device that includes the analyte and the monitoring markers;

[0686] (3) establishing and training a machine learning (ML) model for detecting the monitoring markers in the sample holding device from the image taken by the imager;

[0687] (4) segmenting the image of the sample taken by the imager into non-overlapping regions;

[0688] (5) detecting and locating the monitoring markers from the image of the sample taken by the imager using the ML model of (3);

[0689] (6) generating one region-based marker grid for each of the regions in which more than 5 non-collinear monitoring markers are detected in a local region;

[0690] (7) generating marker grids for all regions not in (6) from the image of the sample taken by the imager based on the detected monitoring markers;

[0691] (8) computing region-specific homographies for each region in (6) based on its own region-based marker grid generated in (6);

[0692] (9) computing homographies for all other regions based on the marker grids generated in (7);

[0693] (10) estimating region-based TLDs for each region in (6) based on the region-based homographies generated in (8);

[0694] (11) estimating TLDs for other regions based on the homographies from (9); and

[0695] (12) applying the estimated TLDs from (10) and (11) to determine the area and concentration of imaged analyte in each partition in the image-based assay.

[0696] In some embodiments, the monitoring markers have sharp edges and flat surfaces.

[0697] In some embodiments, the monitoring markers are used to determine local properties of the image and / or local operating conditions (e.g. gap size, plate mass).

[0698] In some embodiments, the monitoring markers have the same shape as the spacers.

[0699] Monitoring assay operation using monitoring indicia

[0700] One aspect of the invention is to use monitoring indicia placed inside a thin sample to monitor the operating conditions of a QMAX card with two movable plates for assays performed with the QMAX card. The operating conditions can include whether the sample was added correctly, whether the two plates were closed correctly, whether the gap between the two plates is the same or approximately the same as a predetermined value.

[0701] In some embodiments, for a QMAX card comprising two movable plates and having a predetermined gap between the two plates in the closed configuration, the operating conditions of a QMAX assay are monitored by taking an image of the monitoring indicia in the closed configuration. For example, if the two plates were not closed correctly, the monitoring indicia will appear differently in the image compared to when the two plates are closed correctly. The monitoring indicia surrounded by the sample will have a different appearance than the monitoring indicia not surrounded by the sample. Thus, it can provide information about the sample addition conditions.

[0702] Z-1.1 A device for monitoring the operating conditions of the device using monitoring indicia, the device comprising:

[0703] a first plate, a second plate, a spacer, and one or more monitoring indicia, wherein:

[0704] i. the first plate and the second plate are movable relative to each other into different configurations;

[0705] ii. each of the first plate and the second plate comprises an inner surface comprising a sample contact area for contacting a sample being analyzed;

[0706] iii. one or both of the first plate and the second plate comprises a spacer permanently fixed on the inner surface of the respective plate;

[0707] iv. the monitoring indicia has (a) at least one of its dimensions predetermined and known, and (b) observable by an imager;

[0708] v. the monitoring indicia is at least one microstructure with a lateral linear dimension of 300 pm or less; and

[0709] vi. the monitoring indicia is inside the sample;

[0710] wherein one of the configurations is an open configuration in which the two plates are partially or completely separated, the spacing between the plates is not regulated by the spacer, and a sample is deposited on one or both of the plates; and

[0711] wherein the other of the configurations is a closed configuration, the closed configuration being configured after the sample is deposited in the open configuration and the plates are forced to the closed configuration by applying a pressing force on the force region; and in the closed configuration, at least a portion of the sample is compressed by the two plates into a layer of highly uniform thickness and is substantially stagnant relative to the plates, wherein the uniform thickness of the layer is limited by the sample contact regions of the two plates and is adjusted by the plates and the spacers.

[0712] wherein after forcing the two plates to the closed configuration with the force, the monitoring indicia are imaged to determine (i) whether the two plates have reached the expected closed configuration, thereby adjusting the sample thickness to be approximately the predetermined thickness, and / or (ii) whether the sample has been added as needed.

[0713] In some embodiments, the image of the monitoring indicia is used to determine whether the two plates have reached the expected closed configuration, wherein the sample is adjusted to have a thickness of approximately the predetermined thickness.

[0714] In some embodiments, the image of the monitoring indicia is used to determine whether the sample has been added as needed.

[0715] In some embodiments, the monitoring indicia are imaged to determine whether the two plates have reached the expected closed configuration, wherein the sample thickness is adjusted to be the predetermined thickness, and whether the sample has been added as needed.

[0716] In some embodiments, the spacers function as the monitoring indicia.

[0717] In some embodiments, the system includes a device, a computing device, and a non-transitory computer-readable medium having instructions that, when executed, perform the determining.

[0718] In some embodiments, a non-transitory computer-readable medium having instructions that, when executed, perform a method including using one or more images of a thin sample layer in conjunction with monitoring indicia to determine (i) whether the two plates have reached the expected closed configuration, thereby adjusting the sample thickness to be approximately the predetermined thickness, or (ii) whether the sample has been added as needed.

[0719] In some embodiments, the system includes a non-transitory computer-readable medium having instructions that, when executed, perform any of the methods of the present disclosure.

[0720] W-1. A method for monitoring operating conditions of a device using monitoring indicia, the method including:

[0721] (a) obtaining a device as described in any of the preceding embodiments, wherein the device includes two movable plates, spacers, and one or more monitoring indicia, wherein the monitoring indicia are in the sample contact regions;

[0722] (b) obtaining an imager;

[0723] (c) depositing a sample in the sample contact area of the device of (a) and forcing the two plates into a closed configuration;

[0724] (d) taking one or more images of the thin sample layer with the imager along with the monitoring indicia; and

[0725] (e) using the images of the monitoring indicia to determine (i) whether the two plates have reached an intended closed configuration, thereby adjusting the sample thickness to be approximately a predetermined thickness, or (ii) whether the sample has been added as needed.

[0726] In some embodiments, the images of the monitoring indicia are used to determine whether the two plates have reached an intended closed configuration, wherein the sample is adjusted to have a thickness of approximately a predetermined thickness.

[0727] In some embodiments, the images of the monitoring indicia are used to determine whether the sample has been added as needed.

[0728] In some embodiments, the monitoring indicia are imaged to determine whether the two plates have reached an intended closed configuration, wherein the sample thickness is adjusted to a predetermined thickness, and whether the sample has been added as needed.

[0729] In some embodiments, the system includes a device, a computing device, and a non-transitory computer readable medium having instructions that, when executed, perform the determining.

[0730] Selecting a region of interest and / or removing a defective image region

[0731] In some embodiments, a sample has a defect, a method of eliminating the effect of the defect on a determination includes identifying the defect in an image, taking an image of the defect or selecting an image good region of the image that is not caused by the defect.

[0732] In some embodiments, the area of the taken image removed from the image is greater than the area of the defective image region.

[0733] In some embodiments, the thickness of the sample is configured to be a thin thickness such that objects of interest (e.g., cells) form a monolayer (i.e., there is no significant overlap between objects in a direction perpendicular to the sample layer).

[0734] A method for determining manufacturing quality of a QMAX card using an imager, the method includes:

[0735] (f) obtaining a device of any preceding embodiment, wherein the device includes two movable plates, a spacer, and one or more monitoring indicia, wherein the monitoring indicia are in the sample contact area;

[0736] (g) obtaining an imager;

[0737] (h) depositing a sample in the sample contact area of the device of (a) and forcing the two plates into a closed configuration;

[0738] (i) taking one or more images of the thin sample layer using the imager; and

[0739] (j) using the images of the monitoring marks to determine the manufacturing quality of the QMAX card.

[0740] In the method of any one of the preceding embodiments, determining the manufacturing quality comprises measuring a feature (e.g., length, width, pitch, thick edge) of one or more monitoring marks and comparing the measured feature to a reference value to determine the manufacturing quality of the QMAX card.

[0741] In the method of any one of the preceding embodiments, determining the manufacturing quality comprises measuring a first feature (e.g., number, length, width, pitch, thick edge) of one or more first monitoring marks and comparing the measured first feature to a second feature (e.g., number, length, width, pitch, thick edge) of one or more second monitoring marks to determine the manufacturing quality of the QMAX card.

[0742] In the method of any one of the preceding embodiments, the determining is performed during analysis of the sample using the device of any one of the preceding embodiments.

[0743] Another aspect of the present invention is to have the monitoring marks in a fixed-spaced pattern in the sample holder device, e.g., in the QMAX device, such that they appear in fixed-spaced intervals with certain intervals in the images of the sample taken by the imager. Based on this fixed-spaced property, the monitoring mark detection can become very reliable because when the monitoring marks are positioned in fixed-spaced intervals in a predetermined configuration, all the monitoring marks can be identified and derived from only a few detected monitoring marks, in addition, this configuration can be made precise by nanofabrication techniques such as nanoimprinting. Thereby, both the sample image-based and the image region-based TLD estimation can become more accurate and robust due to the fixed-spaced pattern of the monitoring marks.

[0744] A. Sample holder with micro-marks

[0745] Single plate

[0746] AA-1.1 A device for determining micro-features in a thin sample using an imager, the device comprising:

[0747] (a) a solid surface comprising a sample contact area for contacting a thin sample, the thin sample having a thickness of 200 pm or less and comprising or suspected of comprising micro-features; and

[0748] (b) one or more markers, wherein the markers:

[0749] x. have a sharp edge that (i) has a predetermined and known shape and size and (ii) is observable by an imager that images the microfeature;

[0750] xi. are at least one microstructure having a lateral linear dimension of 300 pm or less; and

[0751] xii. are within the sample;

[0752] wherein at least one of the markers is imaged by the imager during the assay.

[0753] AA-1.2 An apparatus for assaying a microfeature in a thin sample using an imager, the apparatus comprising:

[0754] (a) a solid phase surface comprising a sample contact area for contacting a thin sample that (i) has a thickness of 200 pm or less and (ii) comprises or is suspected of comprising a microfeature; and

[0755] (b) one or more markers, wherein the markers:

[0756] i. comprise a protrusion or a groove from the solid phase surface

[0757] ii. have a sharp edge that is observable by an imager that images the microfeature;

[0758] iii. are at least one microstructure having a lateral linear dimension of 300 pm or less; and

[0759] iv. are within the sample;

[0760] wherein at least one of the markers is imaged by the imager during the assay.

[0761] Constant pitch two plates

[0762] AA-2.1 An apparatus for assaying a microfeature in a thin sample using an imager, the apparatus comprising:

[0763] a first plate, a second plate, and one or more spacers, wherein:

[0764] xviii. each of the first plate and the second plate comprises an inner surface comprising a sample contact area for contacting a sample comprising or suspected of comprising a microfeature;

[0765] xix. at least a portion of the sample is confined by the first and second plates into a thin layer having a substantially constant thickness of 200 pm or less;

[0766] xx. The monitoring markers have sharp edges that (a) have a predetermined and known shape and size, and (b) are observable by the imager imaging the microfeatures;

[0767] xxi. The monitoring markers are at least one microstructure having a lateral linear dimension of 300 pm or less; and

[0768] xxii. The monitoring markers are inside the sample;

[0769] wherein at least one of the markers is imaged by the imager during the assay.

[0770] two moving plates

[0771] AA-3. An apparatus for assaying microfeatures in a thin sample using an imager, the apparatus comprising:

[0772] a first plate, a second plate, a spacer, and one or more monitoring markers, wherein:

[0773] vii. The first plate and the second plate are movable relative to each other into different configurations;

[0774] viii. Each of the first plate and the second plate comprises an inner surface comprising a sample contact area for contacting a sample comprising or suspected of comprising microfeatures;

[0775] ix. One or both of the first plate and the second plate comprises a spacer permanently affixed to the inner surface of the respective plate;

[0776] x. The monitoring markers have sharp edges that (a) have a predetermined and known shape and size, and (b) are observable by the imager imaging the microfeatures;

[0777] xi. The monitoring markers are at least one microstructure having a lateral linear dimension of 300 pm or less; and

[0778] xii. The monitoring markers are inside the sample;

[0779] wherein at least one of the markers is imaged by the imager during the assay.

[0780] wherein one of the configurations is an open configuration in which the two plates are partially or completely separated, the spacing between the plates is not regulated by the spacer, and a sample is deposited on one or both of the plates; and

[0781] wherein the other of the configurations is a closed configuration, the closed configuration being configured after the sample is deposited in the open configuration and the plates are forced to the closed configuration by applying a pressing force on the force region; and in the closed configuration, at least a portion of the sample is compressed by the two plates into a layer of highly uniform thickness and is substantially at rest relative to the plates, wherein the uniform thickness of the layer is limited by the sample-contacting regions of the two plates and is adjusted by the plates and spacers.

[0782] wherein the monitoring mark is (i) a different structure than the spacers, or (ii) the same structure used as the spacers.

[0783] B. Image capture improvement using sample holder with micro-mark

[0784] BB-1. An apparatus for improving image capture of micro-features in a sample, the apparatus comprising:

[0785] (c) the device of any of the preceding embodiments; and

[0786] (d) an imager for analyzing the sample containing or suspected of containing micro-features;

[0787] wherein the imager captures images, wherein at least one image contains both a portion of the sample and the monitoring mark.

[0788] BB-2. A system for improving image capture of micro-features in a sample, the system comprising:

[0789] (a) the device of any of the preceding device embodiments;

[0790] (e) an imager for analyzing the sample containing or suspected of containing micro-features; and

[0791] (f) a non-transitory computer-readable medium having instructions that, when executed, use the mark as a parameter with an image processing method to adjust settings of the imager for the next image.

[0792] C. Image analysis using sample holder with micro-mark

[0793] CC-1: An apparatus for improving analysis of images of micro-features in a sample, the apparatus comprising:

[0794] (a) the device of any of the preceding device embodiments; and

[0795] (B) a computing device for receiving the mark and images of the sample containing or suspected of containing micro-features;

[0796] wherein the computing device runs an algorithm that uses the mark as a parameter with an image processing method to improve image quality in the images.

[0797] CC-2: A system for improving analysis of images of microfeatures in a sample, the system comprising:

[0798] (a) the apparatus of any of the preceding embodiments;

[0799] (b) an imager for assaying a sample containing or suspected of containing microfeatures by taking one or more images of the sample and the markers; and

[0800] (c) a non-transitory computer-readable medium having instructions that, when executed, use the markers as a parameter with an imaging processing method to improve image quality in at least one image taken in (c).

[0801] CC-3: A computer program product for assaying microfeatures in a sample, the program comprising computer program code means applied and adapted for, in at least one image:

[0802] (a) receiving a sample and one or more images of the monitoring markers, wherein the sample is added to the apparatus of any of the preceding apparatus claims, and wherein the image is taken by the imager; and

[0803] (b) processing and analyzing the image to calculate the quantity of microfeatures, wherein the analysis uses a detection model based on machine learning and information of the monitoring markers provided by the image.

[0804] CC-4: A computing device for assaying microfeatures in a sample, the computing device comprising a computing device that operates the algorithm in any of the embodiments of the present invention.

[0805] CC-5: The method, apparatus, computer program product, or system of any of the preceding embodiments, wherein the improvement in image quality comprises at least one selected from the group consisting of: de-noising, image normalization, image sharpening, image scaling, alignment (e.g., for face detection), super-resolution, de-blurring, and any combination thereof.

[0806] CC-6: The method, apparatus, computer program product, or system of any of the preceding embodiments, wherein the imaging processing method comprises at least one selected from the group consisting of: histogram-based operations, math-based operations, convolution-based operations, smoothing operations, derivative-based operations, morphology-based operations, shadow correction, image enhancement and / or restoration, segmentation, feature extraction and / or matching, object detection and / or classification and / or localization, image understanding, and any combination thereof.

[0807] CC-6.1: The method, apparatus, computer program product, or system of any of the preceding embodiments, wherein the histogram-based operation includes at least one selected from the group consisting of: contrast stretch, equalization, minimum filter, median filter, maximum filter, and any combination thereof.

[0808] CC-6.2: The method, apparatus, computer program product, or system of any of the preceding embodiments, wherein the math-based operation includes at least one selected from the group consisting of: binary operations (e.g., NOT, OR, AND, XOR, and SUB), arithmetic-based operations (e.g., ADD, SUB, MUL, DIV, LOG, EXP, SQRT, TRIG, and INVERT), and any combination thereof.

[0809] CC-6.3: The method, apparatus, computer program product, or system of any of the preceding embodiments, wherein the convolution-based operation includes at least one selected from the group consisting of: operations in spatial domain, Fourier transform, DCT, integer transform, operations in frequency domain, and any combination thereof.

[0810] CC-6.4: The method, apparatus, computer program product, or system of any of the preceding embodiments, wherein the smoothing operation includes at least one selected from the group consisting of: linear filter, uniform filter, triangle filter, Gaussian filter, non-linear filter, median filter, kuwahara filter, and any combination thereof.

[0811] CC-6.5: The method, apparatus, computer program product, or system of any of the preceding embodiments, wherein the derivative-based operation includes at least one selected from the group consisting of: first derivative operation, gradient filter, basic derivative filter, Prewitt gradient filter, Sobel gradient filter, variational gradient filter, Gaussian gradient filter, second derivative filter, basic second derivative filter, frequency domain Laplacian, Gaussian second derivative filter, variational Laplacian filter, gradient direction second derivative (SDGD) filter, third derivative filter, higher order derivative filter (e.g., greater than third derivative filter), and any combination thereof.

[0812] CC-6.6: The method, apparatus, computer program product, or system of any of the preceding embodiments, wherein the morphology-based operation includes at least one selected from the group consisting of: dilation, erosion, Boolean convolution, opening and / or closing, hit-miss operation, skeletonization, thinning, propagation, gray value morphological processing, gray dilation, gray erosion, gray opening, gray closing, morphological smoothing, morphological gradient, morphological Laplacian, and any combination thereof.

[0813] CC-6.7: The method, apparatus, computer program product, or system of any of the preceding embodiments, wherein the image enhancement and / or restoration comprises at least one selected from the group consisting of: sharpening, non-sharpening, noise suppression, distortion suppression, and any combination thereof.

[0814] CC-6.8: The method, apparatus, computer program product, or system of any of the preceding embodiments, wherein the segmentation comprises at least one selected from the group consisting of: thresholding, fixed thresholding, histogram-derived thresholding, Isodata algorithm, background symmetry algorithm, triangle algorithm, edge finding, gradient-based procedure, zero-crossing-based procedure, PLUS-based procedure, binary mathematical morphology, salt-and-pepper filtering, separating objects with holes, filling holes in objects, removing edge-touching objects, outer skeleton, touching objects, gray value mathematical morphology, top-hat transformation, thresholding, local contrast stretch, and any combination thereof.

[0815] CC-6.9: The method, apparatus, computer program product, or system of any of the preceding embodiments, wherein the feature extraction and / or matching comprises at least one selected from the group consisting of: independent component analysis, contour plots, kernel principal component analysis, latent semantic analysis, partial least squares, principal component analysis, multi-factor dimensionality reduction, nonlinear dimensionality reduction, multilinear principal component analysis, multilinear subspace learning, semi-definite embedding, auto-encoder, and any combination thereof.

[0816] NN1. An apparatus for analyzing microfeatures in a sample using an imager, the apparatus comprising:

[0817] (a) a solid phase surface comprising a sample contact area for contacting a sample comprising microfeatures; and

[0818] (B) one or more monitoring labels, wherein the monitoring labels:

[0819] v. are made of a different material than the sample;

[0820] vi. are located inside the sample during the assay on the microstructure, wherein the sample forms a thin layer of less than 200 pm in thickness on the sample contact area;

[0821] vii. have their lateral linear dimensions of about 1 pm (micrometer) or greater, and

[0822] viii. have their lateral linear dimensions of about 1 pm (micrometer) or greater, and

[0823] wherein at least one monitoring label is imaged by the imager during the assay;

[0824] wherein the geometric parameters of the monitoring markers (e.g., shape and size) and / or the spacing between the monitoring markers are (a) predetermined and known prior to the assay of the microfeatures and (b) used as parameters in an algorithm that determines properties related to the microfeatures.

[0825] NN2. An apparatus for analyzing microfeatures in a sample using an imager, the apparatus comprising:

[0826] a solid phase surface comprising a sample contact area for contacting a sample comprising microfeatures; and

[0827] one or more monitoring markers, wherein each monitoring marker comprises a protrusion or a trench from the solid phase surface, wherein:

[0828] ix. the protrusion or trench comprises a flat surface that is substantially parallel to an adjacent surface that is a portion of the solid phase surface adjacent to the protrusion or trench;

[0829] x. a distance between the flat surface and the adjacent surface is about 200 micrometers (pm) or less;

[0830] xi. the flat surface has (a) a linear dimension of at least about 1 pm or more and (b) an area with at least one linear dimension of 150 pm or less;

[0831] xii. imaging the flat surface of at least one monitoring marker by an imager used during the assay of the microfeatures; and

[0832] xiii. the shape of the flat surface, the size of the flat surface, the distance between the flat surface and the adjacent surface, and / or the spacing between the monitoring markers are (a) predetermined and known prior to the assay of the microfeatures and (b) used as parameters in an algorithm that determines properties related to the microfeatures;

[0833] NN3. An apparatus for analyzing microfeatures in a sample using an imager, the apparatus comprising:

[0834] a first plate, a second plate, a spacer, and one or more detection markers, wherein:

[0835] xiii. the first plate and the second plate are movable relative to each other into different configurations;

[0836] xiv. each of the first plate and the second plate comprises an inner surface comprising a sample contact area for contacting a sample comprising microfeatures;

[0837] xv. one or both of the first plate and the second plate comprises the spacer permanently affixed to the inner surface of the respective plate;

[0838] xvi. Each of the first and second plates comprises an inner surface comprising a sample contact area for contacting a sample comprising microfeatures;

[0839] xvii. The monitoring markers are made of a material different from the sample;

[0840] xviii. During the assay of the microstructure, the monitoring markers are located inside the sample, wherein the sample forms a thin layer of less than 200 pm in thickness on the sample contact area; and

[0841] xix. The monitoring markers have a lateral linear dimension of about 1 pm (micrometer) or more and have at least one lateral linear dimension of 300 pm or less;

[0842] wherein at least one monitoring marker is imaged by the imager during the assay;

[0843] wherein the microfeatures are used during the assay; and the shape of the flat surface, the dimensions of the flat surface, the distance between the flat surface and the adjacent surface, and / or the spacing between the monitoring markers are (a) predetermined and known prior to the assay of the microfeatures and (b) used as parameters in an algorithm that determines a property related to the microfeatures;

[0844] wherein one of the configurations is an open configuration in which the two plates are partially or completely separated, the spacing between the plates is not adjusted by the spacers, and the sample is deposited on one or both of the plates; and

[0845] wherein the other of the configurations is a closed configuration, which is configured after the sample is deposited in the open configuration and the plates are forced to the closed configuration by applying a pressing force on the force area; and in the closed configuration, at least a portion of the sample is compressed by the two plates into a layer of a highly uniform thickness and is substantially stagnant relative to the plates, wherein the uniform thickness of the layer is confined by the sample contact areas of the two plates and is adjusted by the plates and the spacers.

[0846] wherein the monitoring markers are (i) a different structure from the spacers, or (ii) the same structure used as the spacers.

[0847] NN4. An apparatus for analyzing microfeatures in a sample using an imager, the apparatus comprising:

[0848] a first plate, a second plate, spacers, and one or more detection markers, wherein:

[0849] xx. The first and second plates are movable relative to each other into different configurations;

[0850] xxi. Each of the first and second plates comprises an inner surface comprising a sample contact area for contacting a sample comprising microfeatures;

[0851] xxii. One or both of the first and second plates comprises spacers permanently affixed to the inner surface of the respective plate;

[0852] xxiii. The spacers have a substantially uniform height equal to or less than 200 microns and a fixed spacer spacing (ISD).

[0853] xxiv. Each monitoring mark comprises a protrusion or a groove on one or both sample contact areas;

[0854] xxv. The protrusion or groove comprises a flat surface that is substantially parallel to an adjacent surface that is part of a solid surface adjacent to the protrusion or groove;

[0855] xxvi. The distance between the flat surface and the adjacent surface is about 200 microns (pm) or less;

[0856] xxvii. The flat surface has (a) a linear dimension of at least about 1 pm or more, and (b) an area with at least one linear dimension of 150 pm or less;

[0857] xxviii. Imaging the flat surface of at least one monitoring mark by an imager used during assaying the microfeatures; and

[0858] xxix. The shape of the flat surface, the dimensions of the flat surface, the distance between the flat surface and the adjacent surface, and / or the spacing between monitoring marks are (a) predetermined and known prior to assaying the microfeatures and (b) used as parameters in an algorithm that determines a property related to the microfeatures;

[0859] where one of the configurations is an open configuration in which the two plates are partially or completely separated, the spacing between the plates is not regulated by the spacers, and a sample is deposited on one or both of the plates; and

[0860] where the other of the configurations is a closed configuration, the closed configuration being configured after the sample is deposited in the open configuration and the plates are forced to the closed configuration by exerting a pressing force on the force area; and in the closed configuration, at least a portion of the sample is compressed by the two plates into a layer of substantially uniform height and is substantially stagnant relative to the plates, where the uniform height of the layer is limited by the sample contact areas of the two plates and regulated by the plates and the spacers.

[0861] where the monitoring marks are (i) a different structure than the spacers, or (ii) the same structure used as the spacers.

[0862] NN5. An apparatus for an image-based assay, comprising:

[0863] The apparatus of any preceding embodiment, wherein the apparatus has at least five monitoring marks, wherein at least three of the monitoring marks are not aligned on a straight line.

[0864] NN6. An apparatus for analyzing microfeatures in a sample using an imager, the apparatus comprising:

[0865] (c) the apparatus of any of the preceding embodiments; and

[0866] (d) an imager for assaying a sample comprising microfeatures.

[0867] NN7. A system for performing an imaging-based assay, the system comprising:

[0868] (d) the apparatus of any of the preceding apparatus embodiments;

[0869] (e) an imager for assaying a sample comprising microfeatures; and

[0870] (f) a non-transitory computer-readable medium comprising instructions that, when executed, utilize the monitoring marks of the apparatus to determine a property related to the microfeatures.

[0871] NN8. A system for analyzing microfeatures in a sample using an imager, the system comprising:

[0872] (a) the apparatus of any of the preceding apparatus embodiments;

[0873] (h) an imager for assaying a sample comprising microfeatures; and

[0874] (i) a non-transitory computer-readable medium comprising instructions that, when executed, utilize the monitoring marks of the apparatus to assay a property related to the microfeatures, wherein the instructions comprise machine learning.

[0875] NN9. A method for assaying microfeatures in a sample using an imager, comprising:

[0876] (d) obtaining the apparatus, device, or system of any of the preceding embodiments;

[0877] (e) obtaining a sample and depositing the sample on a sample contact area in the apparatus, device, or system of (a), wherein the sample comprises microfeatures; and

[0878] (f) assaying the microfeatures.

[0879] NN10. A method for determining microfeatures in a sample using an imager, comprising:

[0880] (d) obtaining the device, apparatus, or system of any of the preceding embodiments;

[0881] (e) obtaining a sample and depositing the sample on a sample contact area in the device, apparatus, or system of (a), wherein the sample comprises microfeatures; and

[0882] (f) determining the microfeatures, wherein the determining comprises a step of using machine learning.

[0883] T1. A method for determining a true lateral dimension (TLD) of a sample on a sample holder from a distorted image, the method comprising:

[0884] (a) obtaining the device of any preceding embodiment, wherein the device comprises one or more monitoring marks in a sample contact area;

[0885] (b) obtaining an imager, computing hardware, and a non-transitory computer- readable medium comprising an algorithm;

[0886] (c) depositing a thin sample layer comprising microfeatures in the sample contact area of the device of (a);

[0887] (d) taking one or more images of the thin sample layer along with the monitoring marks using the imager, wherein the imager is positioned above the thin sample layer; and

[0888] (e) determining the true lateral dimension of the sample using the algorithm;

[0889] wherein,

[0890] (i) the algorithm is computer code executed on a computer system; and

[0891] (ii) the algorithm uses the images of the monitoring marks as parameters.

[0892] T2. A method for determining a true lateral dimension (TLD) of a sample on a sample holder from a distorted image, the method comprising:

[0893] (a) obtaining the device of any preceding embodiment, wherein the device comprises one or more monitoring marks in a sample contact area;

[0894] (b) obtaining an imager, computing hardware, and a non-transitory computer- readable medium comprising an algorithm;

[0895] (c) depositing a thin sample layer comprising microfeatures in the sample contact area of the device of (a);

[0896] (d) taking one or more images of the thin sample layer along with the monitoring markers using an imager, wherein the imager is positioned above the thin sample layer; and

[0897] (e) determining the real lateral dimensions and coordinates of the imaged sample in the real world by physical measurements (e.g. microns) using the algorithm;

[0898] wherein,

[0899] (i) the algorithm is computer code that is executed on a computer system; and

[0900] (ii) the algorithm uses the images of the monitoring markers as parameters.

[0901] T3. The apparatus, method or system of any of the preceding embodiments, wherein the microfeatures and monitoring markers from the sample are disposed within a sample holding apparatus.

[0902] T4. The apparatus, method or system of any of the preceding embodiments, wherein the determining comprises detecting and locating the monitoring markers in the images of the sample taken by the imager.

[0903] T5. The apparatus, method or system of any of the preceding embodiments, wherein the determining comprises generating a grid of monitoring markers based on the monitoring markers detected from the images of the sample taken by the imager.

[0904] T6. The apparatus, method or system of any of the preceding embodiments, wherein the determining comprises computing a homographic transformation from the generated grid of monitoring markers.

[0905] T7. The apparatus, method or system of any of the preceding embodiments, wherein the determining comprises estimating a TLD from the homographic transformation, and determining the area, size and concentration of the detected microfeatures in the image-based assay.

[0906] T8. The method, apparatus or system of any of the preceding embodiments, wherein the TLD estimation is based on the area in the images of the sample taken by the imager, comprising:

[0907] (a) obtaining a sample;

[0908] (b) adding the sample to a sample holding apparatus, e.g. a QMAX apparatus, wherein there are monitoring markers, wherein the monitoring markers are not submerged in the sample and reside in the apparatus, which can be imaged from the top by an imager in the image-based assay;

[0909] (c) taking images of the sample in the sample adding apparatus including the microfeatures and monitoring markers;

[0910] (d) detecting the monitoring markers in the sample image taken by the imager;

[0911] (e) segmenting the sample image into non-overlapping regions;

[0912] (f) generating a region-based marker grid for each of the non-overlapping regions in which more than 5 non-collinear monitoring markers are detected in the local region;

[0913] (g) generating a marker grid for all other regions not in (f) based on the monitoring markers detected from the sample image taken by the imager;

[0914] (h) computing a region-specific homography for each region in (f) based on its own region-based marker grid generated by (f);

[0915] (i) computing a homography for all other regions not in (f) based on the marker grids generated in (g);

[0916] (j) estimating a region-based TLD for each region in (f) based on the region-based homographies of (g);

[0917] (k) estimating a TLD for other regions not in (f) based on the homographies of (i); and

[0918] (L) applying the estimated TLDs from (j) and (k) to determine the area and concentration of the imaged microfeatures in each image partition in the image-based assay.

[0919] T9. The method, device or system of any of the preceding embodiments, wherein the monitoring markers in the sample holding device are distributed according to a fixed spacing pattern having a defined pitch period.

[0920] T10. The method, device or system of any of the preceding embodiments, wherein the monitoring markers are detected and used as detectable anchor points for calibration and improving measurement accuracy in the image-based assay.

[0921] T11. The method, device or system of any of the preceding embodiments, wherein the detection of the monitoring markers in the sample image taken by the imager utilizes the fixed spacing of the monitoring marker distribution in the sample holding device for error correction and / or reliability of detection.

[0922] T12. The method, device or system of any of the preceding embodiments, wherein the detection, identification, area and / or shape profile estimation of the monitoring marker in the image-based assay is performed by machine learning (ML) using a ML-based monitoring marker detection model and device constructed or trained from images taken by an imager on the device in the image-based assay.

[0923] T13. The method, device or system of any of the preceding embodiments, wherein the detection, identification, area and / or shape profile estimation of the monitoring marker in the image-based assay is performed by image processing or image processing in conjunction with machine learning.

[0924] T14. The method, device or system of any of the preceding embodiments, wherein the detected monitoring marker is applied to TLD estimation in the image-based assay to calibrate the system and / or improve measurement accuracy in the image-based assay.

[0925] T15. The method, device or system of any of the preceding embodiments, wherein the detected monitoring marker is applied in the image-based assay to calibrate the system and / or improve measurement accuracy without limitation to microfeature size, volume and / or concentration estimation.

[0926] T16. The method, device or system of any of the preceding embodiments, wherein the detection of the monitoring marker and / or TLD estimation is applied to fault detection in the image-based assay, including but not limited to detecting defects in the sample holding device, mispositioning of the sample holding device in the imager, and / or focus failure of the imager.

[0927] T17. The method, device or system of any of the preceding embodiments, wherein the monitoring marker is detected as an anchor point to be applied in a system to estimate the area of an object in the image-based assay, comprising:

[0928] i. adding a sample in the image-based assay to a sample holding device with a monitoring marker residing in the device;

[0929] ii. taking an image of the sample in the sample holding device including the microfeature and the monitoring marker; and

[0930] iii. detecting the monitoring marker in the image of the sample taken by an imager on the sample holding device, determining the TLD and calculating an area estimation in the image-based assay to determine the size of the imaged object in terms of physical dimensions of microns in the real world from pixels in the image.

[0931] T18. The method, device or system of any of the preceding embodiments, wherein the system comprises:

[0932] i. Detecting monitoring markers in digital images;

[0933] ii. Generate monitoring mark grid;

[0934] iii. Calculating image transformation based on the monitoring marker grid; and

[0935] iv. Estimating the area of ​​objects in sample images and their physical size in the real world in image-based measurements.

[0936] T19. A method, apparatus or system as described in any of the preceding embodiments, wherein a monitoring marker grid generated based on the detected monitoring markers is used to calculate a homography transformation to estimate the TLD, the area of ​​the object in the image of the sample taken by the imager, and the physical size of the object in the real world.

[0937] T20. The method, apparatus, or system of any preceding embodiment, wherein the method comprises:

[0938] i. segmenting an image of a sample captured by an imager in an image-based assay into non-overlapping regions;

[0939] ii. Detect local monitoring markers in the image;

[0940] iii. If more than 5 non-collinear monitoring markers are detected in the region, a region-based marker grid is generated for the region;

[0941] iv. generating a marker grid for all other regions based on the monitoring markers detected in the image of the sample taken by the imager;

[0942] v. For each region in (iii), calculate the region-based homography transformation based on the generated region-based label grid;

[0943] vi. Based on the marker grid generated in (iv), calculate the homography transformation for all other regions in (iii); and

[0944] vii. Estimate the TLD of each region based on the homography transformation generated by (v) and (vi), and determine the object area of ​​each region in the sample image and its size in the real world in the image-based measurement.

[0945] T21. The method, device or system of any preceding embodiment, wherein the assay is a medical, diagnostic, chemical or biological test.

[0946] T22. The method, device or system of any preceding embodiment, wherein the microfeatures are cells.

[0947] T23. The method, device, or system of any of the preceding embodiments, wherein the microfeature is a blood cell.

[0948] T24. The method, device, or system of any of the preceding embodiments, wherein the microfeature is a protein, a peptide, DNA, RNA, a nucleic acid, a small molecule, a cell, or a nanoparticle.

[0949] T25. The method, device, or system of any of the preceding embodiments, wherein the microfeature comprises a label.

[0950] T26. The method, device, or system of any of the preceding embodiments, wherein the algorithm comprises a computer program product comprising computer program code means adapted to, in at least one image:

[0951] (a) receive an image of a sample, wherein the sample is added to a QMAX device and the image is taken by an imager connected to the QMAX device, wherein the image comprises the sample and a monitoring label;

[0952] (b) analyze the image using a detection model and generate a 2D data array of the image, wherein the 2D data array comprises probability data of the microfeature at each location in the image, and the detection model is built through a training process comprising:

[0953] i. feeding an annotated dataset to a convolutional neural network, wherein the annotated dataset is from a sample of the same type as the test sample and for the same microfeature; and

[0954] ii. training and building the detection model through convolution; and

[0955] (c) analyze the 2D data array to detect local signal peaks by:

[0956] i. a signal list process, or

[0957] ii. a local search process; and

[0958] (g) calculate the quantity of the microfeature based on the local signal peak information.

[0959] T27. The method, device, or system of any of the preceding embodiments, wherein the algorithm comprises a computer program product comprising computer program code means applying and adapted to, in at least one image:

[0960] (a) represent a derived pattern between objects in a sample and pixel contour maps of objects in an image of the sample, the image being taken by an imager on a sample holding device,

[0961] (b) numerically reconstructing images of at least one object detected from the derived pattern in the image of the sample and generating a contour mask encompassing the object identified by the derivation module, said object being focused in the derivation module,

[0962] (c) identifying at least one portion of the image of the sample for at least one object in the selected portion of the image of the sample, and

[0963] (d) calculating at least one feature unique from the at least one portion to identify the object in the selected portion of the image of the sample taken by the imager,

[0964] (e) calculating a count of the detected object in the selected portion and its concentration from the selected portion of the image of the sample,

[0965] At this time the program is running on a computing device or in a computing cloud through a network connection.

[0966] T28. The method, device or system of any of the preceding embodiments, wherein the algorithm comprises a computer program product comprising computer program code means adapted to, in at least one image:

[0967] (a) receiving an image of a sample, wherein the sample is added to a QMAX device and the image is taken by an imager connected to the QMAX device, wherein the image comprises the sample and a monitoring marker; and

[0968] (b) analyzing the image to calculate the amount of microfeatures, wherein the analysis uses a detection model based on machine learning and information of the monitoring marker provided by the image.

[0969] The method, device or system of any of the preceding embodiments further comprises a computer readable storage medium or memory storage unit comprising the computer program of any of the preceding embodiments.

[0970] The method, device or system of any of the preceding embodiments further comprises a computing facility or mobile device of a computing device of any of the preceding embodiments.

[0971] The method, device or system of any of the preceding embodiments further comprises a computing facility or mobile device comprising the computer program product of any of the preceding embodiments.

[0972] The method, device or system of any of the preceding embodiments further comprises a computing facility or mobile device comprising the computer readable storage medium or memory storage unit of any of the preceding embodiments.

[0973] A device for analyzing a sample, comprising:

[0974] a first plate, a second plate, a surface amplification layer, and a capture agent, wherein

[0975] (a) the first and second plates are movable relative to each other into different configurations and have sample contact regions on their respective surfaces for contacting a sample containing a target analyte,

[0976] (b) the surface amplification layer is on one of the sample contact regions,

[0977] (c) a capture agent is immobilized on the surface amplification layer, wherein the capture agent specifically binds to the target analyte,

[0978] wherein the surface amplification layer amplifies an optical signal from a label attached to the target analyte more intensely when the target analyte is in proximity to the surface amplification layer than when the target analyte is at a distance of microns or more.

[0979] wherein one configuration is an open configuration in which the average separation between the interior surfaces of the two plates is at least 200 μm; and

[0980] wherein another configuration is a closed configuration in which at least a portion of the sample is between the two plates and the average separation between the interior surfaces of the plates is less than 200 μm.

[0981] An apparatus for analyzing a sample, comprising:

[0982] a first plate, a second plate, a surface amplification layer, and a capture agent, wherein

[0983] (d) the first and second plates are movable relative to each other into different configurations and have sample contact regions on their respective surfaces for contacting a sample containing a target analyte,

[0984] (e) the surface amplification layer is on one of the sample contact regions,

[0985] (f) a capture agent is immobilized on the surface amplification layer, wherein the capture agent specifically binds to the target analyte,

[0986] wherein the surface amplification layer amplifies an optical signal from a label attached to the target analyte more intensely when the target analyte is in proximity to the surface amplification layer than when the target analyte is at a distance of microns or more.

[0987] wherein one configuration is an open configuration in which the average separation between the interior surfaces of the two plates is at least 200 μm;

[0988] wherein another configuration is a closed configuration in which at least a portion of the sample is between the two plates and the average separation between the interior surfaces of the plates is less than 200 μm;

[0989] wherein the thickness of the sample in the closed configuration, the concentration of the label dissolved in the sample in the closed configuration, and the magnification factor of the surface amplification layer are configured such that any label bound directly or indirectly to the capture agent is visible in the closed configuration without washing away unbound label.

[0990] An apparatus comprising the device of any of the preceding embodiments and a reader for reading the device.

[0991] A homogeneous assay method using the device of any of the preceding embodiments, wherein the thickness of the sample in the closed configuration, the concentration of the label, and the magnification factor of the amplification surface are configured such that label bound to the amplification surface is visible without washing away unbound label.

[0992] The method of any of the preceding embodiments, wherein the method is performed by:

[0993] obtaining the device of any of the embodiments;

[0994] depositing a sample on one or both of the plates when the plates are in the open configuration,

[0995] closing the plates to a closed configuration; and

[0996] reading the sample contact area with a reading device to produce a signal image.

[0997] The device or method of any of the preceding embodiments, wherein label bound to the amplification surface is visible within 60 seconds.

[0998] The device or method of any of the preceding embodiments, wherein the method is a homogeneous assay, wherein the signal is read without using a washing step to remove any biological material or label that is not bound to the amplification surface.

[0999] The device or method of any of the preceding embodiments, wherein label bound to the amplification surface is read by a pixelated reading method.

[1000] The device or method of any of the preceding embodiments, wherein label bound to the amplification surface is read by a one-time lump-sum reading method.

[1001] The device or method of any of the preceding embodiments, wherein the assay has a detection sensitivity of 0.1 nM or less.

[1002] The device or method of any of the preceding embodiments, wherein the method removes biological material or label that is not bound to the amplification surface by a sponge prior to reading.

[1003] The device or method of any of the preceding embodiments, wherein the signal amplification layer comprises a D2PA.

[1004] The device or method of any of the preceding embodiments, wherein the signal amplification layer comprises a layer of metallic material.

[1005] The device or method of any of the preceding embodiments, wherein the signal amplification layer comprises a continuous metallic film made of a material selected from the group consisting of gold, silver, copper, aluminum, alloys thereof, and combinations thereof.

[1006] The device or method of any of the preceding embodiments, wherein the different metallic layers locally enhance or act as reflectors, or both, to enhance the optical signal.

[1007] The device or method of any of the preceding embodiments, wherein the signal amplification layer comprises a layer of metallic material and a dielectric material over the layer of metallic material, wherein the capture agent is on the dielectric material.

[1008] The device or method of any of the preceding embodiments, wherein the layer of metallic material is a uniform metallic layer, a nanostructured metallic layer, or a combination thereof.

[1009] The device or method of any of the preceding embodiments, wherein the signal is amplified by plasmonic enhancement.

[1010] The device or method of any of the preceding embodiments, wherein the assay comprises detecting a label by Raman scattering.

[1011] The device or method of any of the preceding embodiments, wherein the capture agent is an antibody.

[1012] The device or method of any of the preceding embodiments, wherein the capture agent is a polynucleotide.

[1013] The device or method of any of the preceding embodiments, wherein the device further comprises a spacer immobilized on one of the plates, wherein the spacer adjusts the spacing between the first plate and the second plate in the closed configuration.

[1014] The device or method of any of the preceding embodiments, wherein the amplification factor of the surface amplification layer is adjusted to make the optical signal from a single label directly or indirectly bound to the capture agent visible.

[1015] The device or method of any of the preceding embodiments, wherein the amplification factor of the surface amplification layer is adjusted to make the optical signal from a single label directly or indirectly bound to the capture agent visible, wherein the visible single label bound to the capture agent is counted individually.

[1016] The device or method of any of the preceding embodiments, wherein the spacing between the first plate and the second plate in the closed configuration is configured such that the saturation binding time of the target analyte to the capture agent is 300 seconds or less.

[1017] The device or method of any of the preceding embodiments, wherein the spacing between the first plate and the second plate in the closed configuration is configured such that the saturation binding time of the target analyte to the capture agent is 60 seconds or less.

[1018] The device or method of any of the preceding embodiments, wherein the magnification factor of the surface magnification layer is adjusted such that the optical signal from a single label is visible.

[1019] The method and device of any of the preceding embodiments, wherein the capture agent is a nucleic acid.

[1020] The device or method of any of the preceding embodiments, wherein the capture agent is a protein.

[1021] The device or method of any of the preceding embodiments, wherein the capture agent is an antibody.

[1022] The device or method of any of the preceding embodiments, wherein the sample contact area of the second plate has reagent storage sites, and in the closed configuration, the storage sites are approximately above the binding sites on the first plate.

[1023] The device or method of any of the preceding embodiments, wherein the reagent storage sites comprise detection agents that bind to the target analyte.

[1024] The device or method of any of the preceding embodiments, wherein the detection agents comprise labels.

[1025] The device or method of any of the preceding embodiments, wherein both the capture agent and the detection agent bind to the target analyte to form a sandwich comprising the label.

[1026] The device or method of any of the preceding embodiments, wherein the signal magnification layer comprises a layer of metallic material.

[1027] The device or method of any of the preceding embodiments, wherein the signal magnification layer comprises a layer of metallic material and a dielectric material over the layer of metallic material, wherein the capture agent is on the dielectric material.

[1028] The device or method of any of the preceding embodiments, wherein the layer of metallic material is a uniform metal layer, a nanostructured metal layer, or a combination thereof.

[1029] The device or method of any of the preceding embodiments, wherein the amplification layer comprises a layer of metallic material and a dielectric material over the layer of metallic material, wherein the capture agent is on the dielectric material, and the layer of dielectric material has a thickness of 0.5 nm, 1 nm, 5 nm, 10 nm, 20 nm, 50 nm, 100 nm, 200 nm, 500 nm, 1000 nm, 2 μιη, 3 μιη, 5 μιη, 10 μιη, 20 μιη, 30 μιη, 50 μιη, 100 μιη, 200 μιη, 500 μιη, or a range between any two of the recited values.

[1030] The device or method of any of the preceding embodiments, wherein the method further comprises quantifying the signal in the image region to provide an estimate of the amount of one or more analytes in the sample.

[1031] The device or method of any of the preceding embodiments, wherein the method comprises identifying and counting individual binding events between the analyte and the capture agent in the image region, thereby providing an estimate of the amount of one or more analytes in the sample.

[1032] The device or method of any of the preceding embodiments, wherein the identifying and counting step comprises: (1) determining the local intensity of the background signal, (2) determining the local signal intensity of one label, two labels, three labels, and four or more labels; and (3) determining the total number of labels in the imaged region.

[1033] The device or method of any of the preceding embodiments, wherein the identifying and counting step comprises: (1) determining the local spectrum of the background signal, (2) determining the local signal spectrum of one label, two labels, three labels, and four or more labels; and (3) determining the total number of labels in the imaged region.

[1034] The device or method of any of the preceding embodiments, wherein the identifying and counting step comprises: (1) determining the local Raman signature of the background signal, (2) determining the local signal Raman signature of one label, two labels, three labels, and four or more labels; and (3) determining the total number of labels in the imaged region.

[1035] The device or method of any of the preceding embodiments, wherein the identifying and counting step comprises determining one or more of the local intensity, spectrum, and Raman signature.

[1036] The device or method of any of the preceding embodiments, wherein the method comprises quantifying the one-time lumped signal in the image region, thereby providing an estimate of the amount of one or more analytes in the sample.

[1037] The device or method of any of the preceding embodiments, wherein the sample contact area of the second plate has reagent storage sites, and in the closed configuration, the storage sites are positioned approximately above the binding sites on the first plate.

[1038] The device or method of any of the preceding embodiments, wherein the method further comprises the step of labeling the target analyte with a detection agent.

[1039] The device or method of any of the preceding embodiments, wherein the detection agent comprises a label.

[1040] The device or method of any of the preceding embodiments, wherein both the capture agent and the detection agent bind to the target analyte to form a sandwich.

[1041] The device or method of any of the preceding embodiments, wherein the method further comprises measuring the volume of sample in the area imaged by the reading device.

[1042] The device or method of any of the preceding embodiments, wherein the target analyte is a protein, a peptide, DNA, RNA, a nucleic acid, a small molecule, a cell, or a nanoparticle.

[1043] The device or method of any of the preceding embodiments, wherein the image shows the location, local intensity, and local spectrum of the signal.

[1044] The device or method of any of the preceding embodiments, wherein the signal is an optical signal selected from the group consisting of a fluorescent, electroluminescent, chemiluminescent, and electrochemiluminescent signal.

[1045] The device or method of any of the preceding embodiments, wherein the signal is a Raman scattering signal.

[1046] The device or method of any of the preceding embodiments, wherein the signal is a force due to a local electrical, local mechanical, local biological, or local optical interaction between the plate and the reading device.

[1047] The method and device of any of the preceding embodiments, wherein the spacers have a cylindrical shape and a nearly uniform cross-section.

[1048] The method or device of any of the preceding embodiments, wherein the spacer spacing (SD) is equal to or less than about 120 μm (micrometers).

[1049] The method or device of any of the preceding embodiments, wherein the spacer spacing (SD) is equal to or less than about 100 μm (micrometers).

[1050] The method and apparatus of any of the above embodiments, wherein the fourth power of the spacer pitch (ISD4) divided by the thickness (h) and Young's modulus (E) of the flexible plate (ISD4 / (hE)) is 5 x 106μm3 / GPa or less.

[1051] The method and apparatus of any of the above embodiments, wherein the fourth power of the spacer pitch (ISD4) divided by the thickness (h) and Young's modulus (E) of the flexible plate (ISD4 / (hE)) is 5 x 105μm3 / GPa or less.

[1052] The method and apparatus of any of the above embodiments, wherein the spacer has a columnar shape, a substantially planar top surface, a predetermined substantially uniform height, and a predetermined constant spacer pitch that is at least about 2 times larger than the size of the analyte, wherein the Young's modulus of the spacer multiplied by the fill factor of the spacer is equal to or greater than 2 MPa, wherein the fill factor is the ratio of the spacer contact area to the total plate area, and wherein for each spacer, the ratio of the lateral dimension of the spacer to its height is at least 1 (one).

[1053] The method and apparatus of any of the above embodiments, wherein the spacer has a columnar shape, a substantially planar top surface, a predetermined substantially uniform height, and a predetermined constant spacer pitch that is at least about 2 times larger than the size of the analyte, wherein the Young's modulus of the spacer multiplied by the fill factor of the spacer is equal to or greater than 2 MPa, wherein the fill factor is the ratio of the spacer contact area to the total plate area, and wherein for each spacer, the ratio of the lateral dimension of the spacer to its height is at least 1 (one), wherein the fourth power of the spacer pitch (ISD4) divided by the thickness (h) and Young's modulus (E) of the flexible plate (ISD4 / (hE)) is 5 x 106μm3 / GPa or less.

[1054] The method and apparatus of any of the above embodiments, wherein the ratio of the spacer pitch of the spacer to the average width of the spacer is 2 or greater, and the fill factor of the spacer multiplied by the Young's modulus of the spacer is 2 MPa or greater.

[1055] The method and apparatus of any of the above embodiments, wherein the analyte is a protein, a peptide, a nucleic acid, a synthetic compound, or an inorganic compound.

[1056] The method and apparatus of any of the above embodiments, wherein the sample is selected from the group of biological samples consisting of amniotic fluid, aqueous humor, vitreous fluid, blood (e.g., whole blood, fractionated blood, plasma, or serum), breast milk, cerebrospinal fluid (CSF), cerumen (earwax), chyle, chymus, endolymph, perilymph, fecal matter, breath, gastric acid, gastric juice, lymphatic fluid, mucus (including nasal drainage and sputum), pericardial fluid, peritoneal fluid, pleural fluid, pus, rheum, saliva, exhaled condensate, sebum, semen, sputum, sweat, synovial fluid, tears, vomit, and urine.

[1057] The method and apparatus of any of the above embodiments, wherein the spacer has the shape of a column, and the ratio of the width to the height of the column is equal to or greater than 1.

[1058] The method and apparatus of any of the above embodiments, wherein the sample deposited on one or both plates has an unknown volume.

[1059] The method and apparatus of any of the above embodiments, wherein the spacer has the shape of a column, and the column has a substantially uniform cross-section.

[1060] The method and apparatus of any of the above embodiments, wherein the sample is used for detecting, purifying, and quantifying chemical compounds or biological molecules associated with stages of certain diseases.

[1061] The method and apparatus of any of the above embodiments, wherein the sample is related to infectious and parasitic diseases, injuries, cardiovascular diseases, cancer, mental disorders, neuropsychiatric disorders, pulmonary diseases, renal diseases, and other and organic diseases.

[1062] The method and apparatus of any of the above embodiments, wherein the sample is related to the detection, purification, and quantification of microorganisms.

[1063] The method and apparatus of any of the above embodiments, wherein the sample is related to viruses, fungi, and bacteria from the environment (e.g., food, water, soil).

[1064] The method and apparatus of any of the above embodiments, wherein the sample is related to the detection, quantification of chemical compounds or biological samples that pose a threat to food safety or national security (e.g., toxic waste, anthrax).

[1065] The method and apparatus of any of the above embodiments, wherein the sample is related to the quantification of vital parameters in medical or physiological monitors.

[1066] The method and apparatus of any of the above embodiments, wherein the sample is related to glucose, blood, oxygen levels, complete blood count.

[1067] The method and apparatus of any of the above embodiments, wherein the sample involves detecting and quantifying specific DNA or RNA from a biological sample.

[1068] The method and apparatus of any of the above embodiments, wherein the sample involves sequencing and comparing genetic sequences in DNA in chromosomes and mitochondria for genomic analysis.

[1069] The method or apparatus of any of the above embodiments, wherein the sample involves detecting reaction products, for example, during drug synthesis or purification.

[1070] The method and apparatus of any of the above embodiments, wherein the sample is a cell, tissue, bodily fluid, and fecal.

[1071] The method and apparatus of any of the above embodiments, wherein the sample is a sample in the fields of human, veterinary, agricultural, food, environmental, and pharmaceutical testing.

[1072] The method and apparatus of any of the above embodiments, wherein the sample is a biological sample selected from hair, nail, cerumen, breath, connective tissue, muscle tissue, neural tissue, epithelial tissue, cartilage, cancerous sample, or bone.

[1073] The method and apparatus of any of the above embodiments, wherein the spacer pitch is in the range of 5 pm to 120 pm.

[1074] The method and apparatus of any of the above embodiments, wherein the spacer pitch is in the range of 120 pm to 200 pm.

[1075] The method and apparatus of any of the above embodiments, wherein the flexible plate has a thickness in the range of 20 pm to 250 pm and a Young's modulus in the range of 0.1 GPa to 5 GPa.

[1076] The method and apparatus of any of the above embodiments, wherein for the flexible plate, the thickness of the flexible plate multiplied by the Young's modulus of the flexible plate is in the range of 60 to 750 GPa-pm.

[1077] The method and apparatus of any of the above embodiments, wherein the uniform thickness sample layer is uniform over a lateral area of at least 1 mm2.

[1078] The method and apparatus of any of the above embodiments, wherein the uniform thickness sample layer is uniform over a lateral area of at least 3 mm2.

[1079] The method and apparatus of any of the above embodiments, wherein the uniform thickness sample layer is uniform over a lateral area of at least 5 mm2.

[1080] The method and apparatus of any of the preceding embodiments, wherein the uniform thickness sample layer is uniform over a lateral area of at least 10 mm2.

[1081] The method and apparatus of any of the preceding embodiments, wherein the uniform thickness sample layer is uniform over a lateral area of at least 20 mm2.

[1082] The method and apparatus of any of the preceding embodiments, wherein the uniform thickness sample layer is uniform over a lateral area of at least 20-100 mm2.

[1083] The method or apparatus of any of the preceding embodiments, wherein the uniform thickness sample layer has a thickness uniformity of up to + / - 5% or better.

[1084] The method or apparatus of any of the preceding embodiments, wherein the uniform thickness sample layer has a thickness uniformity of up to + / - 10% or better.

[1085] The method or apparatus of any of the preceding embodiments, wherein the uniform thickness sample layer has a thickness uniformity of up to + / - 20% or better.

[1086] The method or apparatus of any of the preceding embodiments, wherein the uniform thickness sample layer has a thickness uniformity of up to + / - 30% or better.

[1087] The method, apparatus, computer program product, or system of any of the preceding embodiments, having five or more monitoring marks, wherein at least three of the monitoring marks are not on a straight line.

[1088] The method, apparatus, computer program product, or system of any of the preceding embodiments, wherein each of the plates comprises a force zone on its respective outer surface for applying an imprecise pressing force that presses the plates together.

[1089] The method, apparatus, computer program product, or system of any of the preceding embodiments, wherein one or both plates are flexible;

[1090] The method, apparatus, computer program product, or system of any of the preceding embodiments, wherein the fourth power of the spacer pitch (ISD4) divided by the thickness (h) and Young’s modulus (E) of the flexible plate (ISD4 / (hE)) is 5 x 106μm3 / GPa or less.

[1091] The method, apparatus, computer program product, or system of any of the preceding embodiments, wherein at least one of the spacers is within the sample contact area;

[1092] The method, apparatus, computer program product, or system of any of the preceding embodiments, wherein the wide specification of analyte.

[1093] The method, apparatus, computer program product, or system of any of the preceding embodiments, wherein the wide specification of algorithm.

[1094] The method, apparatus, computer program product, or system of any of the preceding embodiments, wherein the wide specification of imprint is by hand forced and pressed.

[1095] The apparatus, system, or method of any of the preceding embodiments, wherein the algorithm is stored on a non-transitory computer readable medium, and wherein the algorithm comprises instructions that, when executed, perform a method of determining a characteristic corresponding to an analyte using a monitoring mark of the apparatus.

[1096] Some examples of marks

[1097] In the present invention, in some embodiments, the marks have the same shape as the spacers.

[1098] In certain embodiments, the marks are fixedly spaced or non-fixedly spaced.

[1099] In some embodiments, the distance between two marks is predetermined and known, but the absolute coordinates on the plate are unknown.

[1100] In some embodiments, the marks have a predetermined and known shape.

[1101] In some embodiments, the marks are configured to have a distribution in the plate such that there is always a mark in the field of view of the imaging optics regardless of the position of the plate.

[1102] In some embodiments, the marks are configured to have a distribution in the plate such that there is always a mark in the field of view of the imaging optics regardless of the position of the plate, and the number of marks is sufficient for local optical information.

[1103] In some embodiments, the marks are used to control the optical properties of a local region of the sample, and the region size is 1 pm2, 5 pm2, 10 pm2, 20 pm2, 50 pm2, 100 pm2, 200 pm2, 500 pm2, 1000 pm2, 2000 pm2, 5000 pm2, 10000 pm2, 100000 pm2, 500000 pm2, or in a range between any two of said values.

[1104] Using "limited imaging optics"

[1105] In the present invention, in some embodiments, the optical system used to image the assay has "limited imaging optics".

[1106] Some embodiments of limited imaging optics include, but are not limited to:

[1107] 1. A limited imaging optical system comprising:

[1108] an imaging lens;

[1109] an imaging sensor;

[1110] wherein the imaging sensor is part of a camera of a smartphone;

[1111] wherein at least one of the imaging lens is part of a camera of a smartphone;

[1112] 2. The limited imaging optical system of any of the preceding embodiments, wherein the physical optical resolution is worse than 1 pm, 2 pm, 3 pm, 5 pm, 10 pm, 50 pm, or in a range between any two of said values.

[1113] 3. The limited imaging optical system of any of the preceding embodiments, wherein each physical optical resolution is worse than 1 pm, 2 pm, 3 pm, 5 pm, 10 pm, 50 pm, or in a range between any two of said values.

[1114] 4. The limited imaging optical system of any of the preceding embodiments, wherein the preferred physical optical resolution is between 1 pm and 3 pm.

[1115] 5. The limited imaging optical system of any of the preceding embodiments, wherein the numerical aperture is less than 0.1, 0.15, 0.2, 0.2, 0.25, 0.3, 0.35, 0.4, 0.45, 0.5, or in a range between any two of said values.

[1116] 6. The limited imaging optical system of any of the preceding embodiments, wherein the preferred numerical aperture is between 0.2 and 0.25.

[1117] 7. The limited imaging optical system of any of the preceding embodiments, wherein the working distance is 0.2 mm, 0.5 mm, 1 mm, 2 mm, 5 mm, 10 mm, 20 mm, or in a range between any two of said values.

[1118] 8. The limited imaging optical system of any of the preceding embodiments, wherein the working distance is 0.2 mm, 0.5 mm, 1 mm, 2 mm, 5 mm, 10 mm, 20 mm, or in a range between any two of said values.

[1119] 9. The limited imaging optical system of any of the preceding embodiments, wherein the preferred working distance is between 0.5 mm and 1 mm.

[1120] 10. The limited imaging optical system of any of the preceding embodiments, wherein the depth of focus is 100 nm, 500 nm, 1 pm, 2 pm, 10 pm, 100 pm, 1 mm, or in a range between any two of the values.

[1121] 11. The limited imaging optical system of any of the preceding embodiments, wherein the depth of focus is 100 nm, 500 nm, 1 pm, 2 pm, 10 pm, 100 pm, 1 mm, or in a range between any two of the values.

[1122] 12. The limited imaging optical system of any of the preceding embodiments, wherein the image sensor is part of a smartphone camera module.

[1123] 13. The limited imaging optical system of any of the preceding embodiments, wherein the diagonal length of the image sensor is less than 1 inch, ½ inch, 1 / 3 inch, ¼ inch, or in a range between any two of the values.

[1124] 14. The limited imaging optical system of any of the preceding embodiments, wherein the imaging lens comprises at least two lenses, and one lens is part of a smartphone camera module.

[1125] 15. The limited imaging optical system of any of the preceding embodiments, wherein at least one external lens is paired with an internal lens of a smartphone.

[1126] 16. The limited imaging optical system of any of the preceding embodiments, wherein the optical axis of the external lens is aligned with an internal lens of a smartphone, with an alignment tolerance of less than 0.1 mm, 0.2 mm, 0.5 mm, 1 mm, or in a range between any two of the values.

[1127] 17. The limited imaging optical system of any of the preceding embodiments, wherein the height of the external lens is less than 2 mm, 5 mm, 10 mm, 15 mm, 20 m, or in a range between any two of the values.

[1128] 18. The limited imaging optical system of any of the preceding embodiments, wherein the preferred height of the external lens is between 3 mm and 8 mm.

[1129] 19. The limited imaging optical system of any of the preceding embodiments, wherein the preferred height of the external lens is between 3 mm and 8 mm.

[1130] 20. The limited imaging optical system of any of the preceding embodiments, wherein the outer lens has a diameter less than 2 mm, 4 mm, 8 mm, 10 mm, 15 mm, 20 mm, or in a range between any two of the recited values.

[1131] 21. The limited imaging optical system of any of the preceding embodiments, wherein the physical optical magnification is less than 0.1X, 0.5X, IX, 2X, 4X, 5X, 10X, or in a range between any two of the recited values.

[1132] 22. The limited imaging optical system of any of the preceding embodiments, wherein the physical preferred optical magnification is less than 0.1X, 0.5X, IX, 2X, 4X, 5X, 10X, or in a range between any two of the recited values.

[1133] The term “image-based assay” refers to an assay procedure that utilizes images of a sample taken by an imager, where the sample can be, but is not limited to, medical, biological, and chemical samples.

[1134] The term “imager” refers to any device capable of taking images of an object. It includes, but is not limited to, a microscope, a camera in a smartphone, or a special device that can take images at various wavelengths.

[1135] The term “sample feature” refers to some property of a sample that represents a condition of potential interest. In certain embodiments, a sample feature is a feature that appears in an image of a sample and can be segmented and classified by a machine learning model. Examples of sample features include, but are not limited to, types of analytes in a sample, e.g., red blood cells, white blood cells, and tumor cells, and include analyte counts, sizes, volumes, concentrations, etc.

[1136] The term “machine learning” refers to algorithms, systems, and devices in the field of artificial intelligence that generally use statistical techniques and artificial neural networks to provide computers with the ability to “learn” from data (i.e., progressively improve performance on a particular task) without being explicitly programmed.

[1137] The term “artificial neural network” refers to a hierarchical, connected system inspired by biological networks that can “learn” to perform a task by considering examples, generally without being programmed with any task-specific rules.

[1138] The term “convolutional neural network” refers to a class of multi-layer, feed-forward artificial neural networks most commonly applied to the analysis of visual images.

[1139] The term “deep learning” refers to a broad class of machine learning methods in artificial intelligence (AI) that learn from data with a certain depth of network structure.

[1140] The term“machine learning model” refers to a trained computational model built from the training process in machine learning from data. Machine learning models trained by computer applications during the derivation phase, which gives the computer the ability to perform specific tasks (e.g., detecting and classifying objects). Examples of machine learning models include ResNet, DenseNet, etc., which are also referred to as“deep learning models” due to the hierarchical depth in their network structure.

[1141] The term“image segmentation” refers to an image analysis process that segments a digital image into multiple segments (sets of pixels, typically with a set of bit-mask masks that cover the image segments enclosed by their segment boundary contours). Image segmentation can be achieved by image segmentation algorithms in image processing such as watershed, iterative graph cuts, mean shift, etc., or by machine learning algorithms such as MaskRCNN.

[1142] The term“defects in a sample” refers to phenomena that should not exist under ideal sample conditions or should not be considered in the characteristics of a sample. They can come from, but are not limited to, contaminants such as dust, bubbles, etc., and from extraneous objects in the sample such as monitoring markers (e.g., pillars) in the sample holding device. Defects can have significant size and occupy a significant amount of volume in the sample, such as bubbles. In addition to their distribution and amount in the sample, which are sample-dependent, they can have different shapes.

[1143] The term“threshold” herein refers to any number used as a cutoff value, for example, to classify a sample feature as a particular type of analyte or the ratio of abnormal cells to normal cells in a sample. The threshold can be identified empirically or analytically.

[1144] Using“limited sample manipulation”

[1145] In the present invention, in some embodiments, the sample positioning system used to image the assay has“limited sample manipulation”. Some embodiments of limited sample manipulation include, but are not limited to,

[1146] Limited sample manipulation system description:

[1147] 1. A limited sample manipulation system, comprising:

[1148] a sample holder;

[1149] wherein the sample holder has a receptacle for receiving a sample card.

[1150] 2. The limited sample manipulation system of any of the preceding embodiments, wherein the accuracy of positioning a sample in a direction along an optical axis is worse than 0.1 pm, 1 pm, 10 pm, 100 pm, 1 mm, or a range between any two of the values.

[1151] 3. The limited sample manipulation system of any of the preceding embodiments, wherein the preferred accuracy of positioning the sample in the direction along the optical axis is between 50 pm and 200 pm.

[1152] 4. The limited sample manipulation system of any of the preceding embodiments, wherein the accuracy of positioning the sample in the plane perpendicular to the optical axis is worse than 0.01 pm, 0.1 pm, 1 pm, 10 pm, 100 pm, 1 mm, or in a range between any two of the values.

[1153] 5. The limited sample manipulation system of any of the preceding embodiments, wherein the preferred accuracy of positioning the sample in the plane perpendicular to the optical axis is between 100 pm and 1 mm.

[1154] 6. The limited sample manipulation system of any of the preceding embodiments, wherein the horizontal error of positioning the sample card is worse than 0.01 degrees, 0.1 degrees, 0.5 degrees, 1 degree, 10 degrees, or in a range between any two of the values.

[1155] 7. The limited sample manipulation system of any of the preceding embodiments, wherein the preferred horizontal error of positioning the sample card is between 0.5 degrees and 10 degrees.

[1156] 8. The limited sample manipulation system of any of the preceding embodiments, wherein the preferred horizontal error of positioning the sample card is between 0.5 degrees and 10 degrees.

[1157] TLD and volume estimation using monitoring markers

[1158] Figure 2 Embodiments of sample holding devices, QMAX devices, and their monitoring markers, columns used in some embodiments of the invention are shown. The columns in the QMAX device make the gap between the two parallel plates of the sample holding device uniform. The gap is narrow and related to the size of the analyte that forms a monolayer of the analyte in the gap. In addition, the monitoring markers in the QMAX device are a special form of the columns, therefore, they are not submerged by the sample and can be imaged together with the sample by an imager in an image-based assay.

[1159] TLD (true lateral dimension) estimation example using monitoring markers

[1160] In some embodiments of the invention, estimation is performed for TLD and true volume. The monitoring markers (posts) are used as detectable anchor points. However, it is difficult to detect the monitoring markers with the accuracy suitable for TLD estimation in image-based assays. This is because these monitoring markers are penetrated and surrounded by the analyte within the sample holding device, and they are distorted and blurred in the image due to distortion from the lens, light diffraction from microscopic objects, imperfections at the microscopic level, misalignment of the focus, noise in the sample image, etc. And it becomes even more difficult if the imager is a camera from a commercial device (e.g., a camera from a smartphone), because these cameras are no longer calibrated with specialized hardware once they leave the manufacturing.

[1161] In the present invention, the detection and localization of the monitoring markers as the detectable anchor points for TLD estimation are formulated in a machine learning framework, and a specialized machine learning model is built / trained to detect them in microscopic imaging. Furthermore, in some embodiments of the present invention, the distribution of the monitoring markers is intentionally made fixed-spaced, and distributed in a predetermined pattern. This makes the method in the present invention more robust and reliable.

[1162] In particular, embodiments of the present invention include:

[1163] (8) adding a sample to a sample holding device, e.g., a QMAX device, where there are monitoring markers with known configurations in the device, which are not submerged in the sample and can be imaged by an imager;

[1164] (9) taking an image of the sample in the sample holding device including the analyte and the monitoring markers;

[1165] (10) building and training a machine learning (ML) model to detect the monitoring markers in the sample image;

[1166] (11) detecting and localizing the monitoring markers in the sample holding device from the image of the sample using the ML detection model from (3);

[1167] (12) generating a marker grid from the monitoring markers detected in (4);

[1168] (13) computing a homography based on the generated monitoring marker grid;

[1169] (14) estimating and saving the true lateral dimensions of the image of the sample from the homography from (6); and

[1170] (15) applying the estimated TLD from (7) in subsequent image-based assays to determine the area, size, volume, and concentration of the analyte.

[1171] In some embodiments of the invention, region-based TLD estimation and calibration are employed in image-based assays. This includes:

[1172] (13) adding the sample to a sample holding device, such as a QMAX device, where there are monitoring markers present in the device that are not submerged in the sample and can be imaged by an imager in an image-based assay;

[1173] (14) taking an image of the sample in the sample holding device including the analyte and monitoring markers;

[1174] (15) establishing and training a machine learning (ML) model for detecting monitoring markers from images of the sample taken by the imager;

[1175] (16) segmenting the images of the sample taken by the imager into non-overlapping regions;

[1176] (17) detecting and localizing monitoring markers from the images of the sample taken by the imager using the ML model of (3);

[1177] (18) generating one region-based marker grid for each of the regions with more than 5 non-collinear monitoring markers detected in the local region;

[1178] (19) generating marker grids for all regions not in (6) from the images of the sample taken by the imager based on the detected monitoring markers;

[1179] (20) computing region-specific homographies for each region in (6) based on its own region-based marker grid generated in (6);

[1180] (21) computing homographies for all other regions based on the marker grids generated in (7);

[1181] (22) estimating region-based TLD for each region in (6) based on the region-based homographies generated in (8);

[1182] (23) estimating TLD for other regions based on the homographies from (9); and

[1183] (24) saving and applying the estimated TLDs in (10) and (11) to subsequent image-based detection of segmented regions.

[1184] When the monitoring markers are distributed in a predefined fixed interval pattern, for example in a QMA device, they appear and are distributed at a certain interval fixedly, as a result, the detection of the monitoring markers in the above process becomes more robust and reliable. This is because in the case of fixed interval, if the detected position and configuration do not follow the predefined fixed interval pattern, all monitoring markers can be identified and determined from only a few detected monitoring markers, and detection errors can be corrected and eliminated.

[1185] Volume estimation with monitoring markers

[1186] In order to estimate the true volume of the sample in the assay, it is necessary to remove the volume from the defects and peripheral objects in the sample, embodiments of the present application include:

[1187] (1) Add the sample to the sample holding device with monitoring markers, such as QMAX device, and take an image of the sample by the imager;

[1188] (2) Estimate the true lateral dimension (TLD) of the image by detecting the monitoring markers in the sample image as anchor points using the above method and device;

[1189] (3) Detect and locate defects in the sample image, such as bubbles, dust, monitoring markers, etc., using a machine learning (ML) detection model constructed and trained according to the sample image;

[1190] (4) Determine the coverage mask of the detected defects in the image of the sample using a machine learning (ML) segmentation model constructed and trained according to the image of the sample;

[1191] (5) Determine the margin distance Δ according to the size of the analyte / object in the sample and sample holding device (e.g. 2x maximum analyte diameter);

[1192] (6) Determine the Δ+ mask of all detected defects, i.e. the mask of the coverage mask of (4) extended by an additional margin Δ;

[1193] (7) Remove the detected defects according to the Δ+ mask in the sample image (6);

[1194] (8) Save the image in (7), and save the corresponding volume after removing the volume with the corresponding Δ+ mask in (7) in the subsequent image-based assay; and

[1195] (9) If the area of the removed Δ+ mask or the remaining true volume of the sample exceeds a certain preset threshold, reject the sample.

[1196] In the present invention, additional margin Δ+ masks are used to remove defects from sample images. This is important because defects can affect their environment. For example, some defects can change the height of the gap in the sample holding device, and the local volume or concentration distribution around the defects can become different.

[1197] The terms "monitoring mark" and "mark" are interchangeable in the description of the present invention.

[1198] The terms "imager" and "camera" are interchangeable in the description of the present invention.

[1199] The term "denoising" refers to the process of removing noise from received signals. One example is removing noise from sample images, as images from imagers / cameras can pick up noise from various sources, including but not limited to white noise, salt and pepper noise, Gaussian noise, etc. Denoising methods include but are not limited to: linear and non-linear filtering, wavelet transform, statistical methods, deep learning, etc.

[1200] The term "image normalization" refers to algorithms, methods, and devices that change the range of pixel intensity values in a processed image. For example, it includes but is not limited to increasing contrast by histogram stretching, subtracting the average pixel value from each image, etc.

[1201] The term "image sharpening" refers to the process of enhancing the edge contrast and edge content of an image.

[1202] The term "image scaling" refers to the process of adjusting the size of an image. For example, if an object is too small in the image of a sample, image scaling can be applied to enlarge the image to help detection. In some embodiments of the present invention, for the purpose of training or derivation, images need to be adjusted to a specified size before input to a deep learning model.

[1203] The term "alignment" refers to transforming different data sets into one common coordinate system so that they can be compared and combined. For example, in image processing, different data sets come from but are not limited to images from multiple imager sensors and images from the same sensor but at different times, focus depths, etc.

[1204] The term "super-resolution" refers to the process of obtaining a higher resolution image from one or more low resolution images.

[1205] The term "deblurring" refers to the process of removing blur artifacts from an image, such as removing blur caused by defocus, shake, motion, etc. in the imaging process.

[1206] In some embodiments of the present invention, methods and algorithms are designed to take advantage of monitoring marks in sample holding devices (e.g., QMAX devices). This includes but is not limited to the estimation and adjustment of the following parameters in imaging devices:

[1207] 1. Shutter speed,

[1208] 2. Sensitivity

[1209] 3. Focus (lens position),

[1210] 4. Exposure compensation,

[1211] 5. White balance: temperature, hue, and

[1212] 6. Zoom (scale factor).

[1213] Examples of image processing / analysis algorithms used with markers

[1214] In some embodiments of the invention, image processing / analysis is applied and enhanced using the monitoring markers in the invention. They include but are not limited to the following image processing algorithms and methods:

[1215] 1. Histogram-based operations include but are not limited to:

[1216] a. Contrast stretching;

[1217] b. Equalization;

[1218] c. Min filtering;

[1219] d. Median filtering; and

[1220] e. Max filtering.

[1221] 2. Math-based operations include but are not limited to:

[1222] a. Binary operations: NOT, OR, AND, XOR, SUB, etc., and

[1223] b. Arithmetic operations: ADD, SUB, MUL, DIV, LOG, EXP, SQRT, TRIG, INVERT, etc.

[1224] 3. Convolution-based operations in spatial and frequency domains include but are not limited to Fourier transform, DCT, integer transform, wavelet transform, etc.

[1225] 4. Smoothing operations include but are not limited to:

[1226] a. Linear filtering: uniform filter, triangular filter, Gaussian filter, etc., and

[1227] b. Non-linear filtering: median filter, kuwahara filter, etc.

[1228] 5. Derivative-based operations include but are not limited to:

[1229] a. First derivative: Gradient filter, Basic derivative filter, Prewitt gradient filter, Sobel gradient filter, Alternate gradient filter, Gaussian gradient filter, etc.

[1230] b. Second derivative: Basic second derivative filter, Frequency domain Laplacian, Gaussian second derivative filter, Vari-Laplacian filter, Gradient Directional Second Derivative (SDGD) filter, etc., and

[1231] c. Other filters with higher derivatives, etc.

[1232] 6. Morphology-based operations include but are not limited to:

[1233] a. Dilation and Erosion;

[1234] b. Boolean convolution;

[1235] c. Opening and Closing;

[1236] d. Hit-Miss operation;

[1237] e. Split and Skeleton;

[1238] f. Skeleton;

[1239] g. Thinning;

[1240] h. Gray value morphological processing: Gray dilation, Gray erosion, Gray opening, Gray closing, etc.; and

[1241] i. Morphological smoothing, morphological gradient, morphological Laplacian, etc.

[1242] Other examples of image processing / analysis techniques

[1243] In some embodiments of the present application, image processing / analysis algorithms are used with and enhanced by the monitoring markers described in the present disclosure. They include but are not limited to the following:

[1244] 1. Image enhancement and restoration include but are not limited to

[1245] a. Sharpening and un-sharpening,

[1246] b. Noise suppression, and

[1247] c. Distortion suppression.

[1248] 2. Image segmentation include but are not limited to:

[1249] a. Thresholding - Fixed threshold, Histogram-derived threshold, Contouring algorithm, Background symmetry algorithm, Triangular algorithm, etc.

[1250] b. Edge finding - gradient-based procedures, zero-crossing-based procedures, PLUS-based procedures, etc.

[1251] c. Binary mathematical morphology - salt and pepper filtering, separating objects with holes, filling holes in objects, removing objects touching the border, outer skeleton, touching objects, etc.; and

[1252] d. Gray value mathematical morphology - top-hat transformation, adaptive thresholding, local contrast stretching, etc.

[1253] 3. Feature extraction and matching including, but not limited to:

[1254] a. Independent component analysis;

[1255] b. Contour plots;

[1256] c. Principal component analysis and kernel principal component analysis;

[1257] d. Latent semantic analysis;

[1258] e. Least squares and partial least squares;

[1259] f. Multifactorial and nonlinear dimensionality reduction;

[1260] g. Multilinear principal component analysis;

[1261] h. Multilinear subspace learning;

[1262] i. Semi-definite embedding; and

[1263] J. Autoencoders / decoders.

[1264] 4. Object detection, classification, and localization

[1265] 5. Image understanding

[1266] Improvements using monitoring markers for microscopy

[1267] Monitoring markers can be used to improve focusing in microscopy. In particular, markers with sharp edges will provide detectable (visible features) for focus evaluation algorithms to analyze the focus condition for certain focus settings, especially in low light environments and microscopy imaging. In some embodiments of the invention, the monitoring markers on the card are used to perform microscopy image correction and enhancement. For example, the focus evaluation algorithm is a core part of the autofocus implementation as shown in Figure 11

[1268] ​For some diagnostic applications (e.g. colorimetric, absorption-based hemoglobin tests, and for CBC of samples with very low cell concentration), the detectable features provided by the analytes in the image of the sample are often insufficient for the focus evaluation algorithm to run accurately and smoothly. Markers with sharp edges, such as the monitoring markers in the QMAX device, provide additional detectable features for the focus evaluation procedure, enabling the accuracy and reliability required in image-based assays.

[1269] For some diagnostic applications, the analytes in the sample are not uniformly distributed. Purely relying on the features provided by the analytes tends to produce some unfair focus settings that give high weight to focusing on some local high-concentration areas, while low-analyte-concentration areas are off-target. In some embodiments of the invention, the effect is controlled by focus adjustment using information from the monitoring markers, which have strong edges and are uniformly distributed in a fixed-spaced pattern of precise processing.

[1270] Higher resolution images are generated from single images using super-resolution.

[1271] Each imager has an imaging resolution that is partially limited by the number of pixels in its sensor, which varies from one million to several million. For some microscopic imaging applications, the analytes have small or microscopic dimensions in the sample, such as the dimensions of platelets in human blood, which have a diameter of about 1.4 pm. When the target detection procedure requires a certain number of pixels, in addition to the available size of the FOV, the limited resolution in the image sensor imposes a significant limitation on the capabilities of the device in image-based assays.

[1272] Single-image super-resolution (SISR) is a technique that uses image processing and / or machine learning to upsample the original source image to a higher resolution and remove as much blur as possible caused by interpolation, so that the object detection procedure can also run on the newly generated image. This will significantly reduce the above-mentioned limitation and allow some otherwise impossible applications. Markers with known shape and structure, such as the monitoring markers in QMAX, can be used as local references to evaluate the SISR algorithm, avoiding the over-sharpening effect produced by most existing technology algorithms.

[1273] In some embodiments of the invention, image fusion is performed to break the physical SNR (signal-to-noise ratio) limit in image-based assays.

[1274] Signal to noise ratio measures the quality of the image of a sample taken by an imager in microscopy. There are practical limitations on the imaging device due to cost, technology, manufacturing, etc. In some cases, such as in mobile health care, the application requires higher SNR than the imaging device can provide. In some embodiments of the present invention, multiple images are taken and processed (with the same and / or different imaging settings, such as 3D fusion of multiple images focused at different depths of focus merged into one super-focused image) to produce an output image with higher SNR, thus making such applications possible.

[1275] However, the images taken by one imager or multiple imagers tend to have some defects and imperfections caused by physical limitations and implementation constraints. The situation becomes acute in the microscopic imaging of a sample in image-based assays, since the analytes in the sample have tiny dimensions and often have no distinct edge features. In some embodiments of the present invention, the monitoring marks in the sample holding device (such as the QMAX device) are used for enhanced solutions.

[1276] One such embodiment is to handle the distortions in the images of a sample taken by an imager. Figure 12 is an improvement over the general camera model. When the distortion parameters are known, the situation is relatively simple (most manufacturers give curves / tables of their lenses to describe the ratio distortion, other distortions can be measured in well-defined experiments). However, when the distortion parameters are unknown for the monitoring marks, it can estimate the distortion parameters iteratively using the regularly, even fixedly, spaced monitoring marks of the sample holding device (such as the QMAX device) without the need for a single coordinate reference as Figure 13 shown.

[1277] Some additional embodiments

[1278] In the present invention, in some embodiments, for the analysis of micro-features in image-based assays, the sample holding device has a flat surface with some special monitoring marks. Some embodiments of the present invention are listed as follows:

[1279] A1: Estimation of true lateral dimension (TLD) of micro images of samples in image based assays. True lateral dimension (TLD) determines the physical dimension of imaged analytes in the real world and it also determines the image coordinates of samples in the real world that are relevant for concentration estimation in image based assays. Monitoring markers can be used as detectable anchor points, determine TLD and improve accuracy of image based assays. In one embodiment of the invention, a machine learning model using monitoring markers is used to detect monitoring markers from which TLD of sample images is derived. Further, if monitoring markers have a fixed interval distribution pattern on the flat surface of sample holding devices, detection of monitoring markers and TLD estimation per sample can become more robust and robust in image based assays.

[1280] A2: Analysis of analytes using measured response from analyte compounds at a specific wavelength of light or at multiple wavelengths of light to predict analyte concentration. Monitoring markers not immersed in the sample can be used to determine light absorption corresponding to background of no analyte compounds in order to determine analyte concentration by light absorption, for example HgB test in complete blood test. Further, each monitoring marker can act as an independent detector of background absorption to make concentration estimation robust and reliable.

[1281] A3: Focusing in micro images of image based assays. Uniformly distributed monitoring markers can be used to improve focusing accuracy. (a) It can be used to provide minimum amount of visual features for samples that have no / less than necessary number of features for reliable focusing and this can be done in low light due to edge content of monitoring markers. (b) It can be used to provide visual features when features in the sample are not uniformly distributed to make focus decision more accurate. (c) It can provide reference for local lighting conditions that are not affected / less affected / different from sample content to adjust weights in focusing evaluation algorithms.

[1282] A4: Monitoring markers can be used as reference to detect and / or correct image defects caused by but not limited to: non-uniformly distributed lighting, various types of image distortions, noise and imperfect image pre-processing operations. (a) For example, as shown in Figure 14 , when straight lines in 3D world are mapped to curves in images, positions of markers in sample images can be used to detect and / or correct ratio distortion. Ratio distribution parameters for the whole image can be estimated based on the change in positions of markers. And ratio distortion parameter values can be iteratively estimated by linear testing of horizontal / vertical lines in the rendered image with distortion elimination based on assumed ratio distortion parameters.

[1283] Examples of machine learning (ML) computations

[1284] B1: One way to use machine learning is to detect analytes in a sample image and compute bounding boxes covering their locations, and use a trained machine learning model to perform in the course of processing the derivation. Another way to use machine learning method to detect and locate analytes in a sample image is to build and train a detection and segmentation model, which involves annotating analytes in a sample image at pixel level. In this method, analytes in a sample image can be detected and located with sealed binary pixel masks covering them in an image-based assay.

[1285] B2: When testing hemoglobin in human blood, an image is taken at a given narrow band wavelength, then the average energy through the analyte region and reference region is analyzed. Based on the known absorbance of the analyte at the given wavelength and the height of the analyte sample region, the concentration can be estimated. However, this measurement has noise. To eliminate the noise, multiple images can be taken using light of different wavelengths, and machine learning regression is used to achieve more accurate and robust estimation. In some embodiments of the invention, a machine learning based derivation takes multiple input images of a sample taken at different wavelengths and outputs a single concentration number.

[1286] Examples of identifying error risks to improve measurement reliability

[1287] In some embodiments, a method for improving reliability of an assay, the method comprising:

[1288] (a) imaging a sample on a QMAX card;

[1289] (b) analyzing error risk factors; and

[1290] (c) rejecting the card to report the measurement result of the card if the error risk factor is higher than a threshold;

[1291] wherein the error risk factor is one or any combination of the following factors. These factors are, but not limited to, (1) blood origin, (2) air bubbles in blood, (3) too small or too large blood volume, (4) blood cells under spacers, (5) aggregated blood cells, (6) lysed blood cells, (7) overexposed image of sample, (8) underexposed image of sample, (8) poor focus of sample, (10) optical system error with lever position error, (11) card not closed, (12) wrong card with no spacer in card, (13) dust in card, (14) oil in card, (15) smudge on the card out of the focal plane on the card, (16) card not in correct position inside the reader, (17) empty card, (18) manufacturing error in card, (19) wrong card for other applications, (20) dry blood, (21) expired card, (22) large variation of blood cell distribution, (23) non-blood sample or non-target blood sample, etc.

[1292] In some embodiments, the error risk analyzer is capable of detecting, differentiating, classifying, correcting, and / or rectifying the following conditions in biological and chemical applications in the device: (1) sample edge, (2) air bubble in sample, (3) sample volume too small or sample volume too large, (4) sample under spacer, (5) clumped sample, (6) lysed sample, (7) overexposed image of sample, (8) underexposed image of sample, (8) poor focus of sample, (10) optical system error with lever position error, (11) card not closed, (12) wrong card due to no spacer in card, (13) dust in card, (14) oil in card, (15) smudge on the card out of the focal plane, (16) card not in correct position in reader, (17) empty card, (18) manufacturing error in card, (19) wrong card for other applications, (20) dry sample, (21) expired card, (22) large variation in blood cell distribution, (23) wrong sample, etc.

[1293] wherein the threshold is determined by a panel test.

[1294] wherein the threshold is determined by machine learning.

[1295] wherein the monitoring marker is used as a control to identify error risk factors.

[1296] wherein the monitoring marker is used as a comparison to evaluate threshold of error risk factors.

[1297] The following brief summary is not intended to include all features and aspects of the present application.

[1298] Methods and systems for image-based assays that analyze morphological, molecular, physical, colorimetric, and pathological features of samples in biological, chemical, and medical diagnostics are disclosed herein. For example, in CBC (complete blood count) tests, it is to characterize blood cells such as red blood cells, white blood cells, etc. from images of samples; molecular and nucleic acid tests are to capture binding of antigens and antibodies or particles in images of samples taken under various conditions along with reagents; and colorimetric tests are to determine sample features and particle concentrations from captured light absorption at various wavelengths in images of samples.

[1299] However, all these tests are conducted under many unknown factors from sample preparation to device operation during testing, which can affect the reliability of test results. For example, in blood tests, common unknown factors include but are not limited to cell clumping, dust, air bubbles, uneven distribution, dry texture structure, etc. In addition to the conventional test procedures, there is an urgent need for methods and systems that can control these unknown factors.

[1300] One aspect of the methods and systems disclosed herein is a framework of trust in image-based assays. It runs in parallel to conventional image-based assays but it analyzes meta-features of the tested sample from the same images used for the image-based assays. These meta-features about the tested sample provide key information to determine the trustworthiness of the test results. Moreover, in some embodiments, these meta-features about the image-based assays are captured and characterized by a dedicated machine learning model that can perform in noisy and diverse environments.

[1301] In particular, key methods and systems are disclosed for verifying the trustworthiness of test results, including:

[1302] (a) In methods and systems for detecting non-uniform distribution of samples in image-based assays, non-uniformly distributed (smear) samples are a cause of inaccuracy and errors.

[1303] (b) In methods and systems for detecting dry texture structure of samples in image-based assays, partially dried (e.g. blood) samples can affect the accuracy of the test results.

[1304] (c) In methods and systems for detecting sample aggregation in image-based assays, the degree of aggregation in the test sample is a strong indicator of sample quality, reagents used for the test, etc.

[1305] (d) In methods and systems for detecting defects in samples for image-based assays, defects in the sample include but are not limited to dust, bubbles, fibers, etc.

[1306] The proposed framework of trust for image-based assays is powered by machine learning, where a dedicated machine learning model is built for fast and robust estimation of the trustworthiness of the assay results. It has been applied to real-world applications, such as blood and colorimetric tests.

[1307] Moreover, a dedicated segmentation method is disclosed herein for obtaining fine contour mask level segmentation of objects in image-based assays. It is applied to cell detection and characterization, but also to trustworthiness assays to detect and characterize defects and abnormalities in the sample.

[1308] The proposed method applies machine learning based segmentation to bounding box level segmentation, which is fast and easy to build. Then, for each object in the bounding box, a dedicated morphological analysis method is designed to obtain fine contour mask segmentation. This method is widely applied in image-based validity analysis.

[1309] Another aspect of the methods and systems disclosed herein is the framework of image-based assays with specially designed sample holders, where the sample holders have a monitoring marker structure in the form of a post that is visible from the image of the sample in the sample holder during the image-based assay. This specially designed sample holder exhibits many new capabilities and features in the image-based assays, including:

[1310] (a) In the methods and systems for true lateral dimension (TLD) correction for image-based assays, the TLD determines the size of the cells in the image as their true size in the physical world, which is a key parameter that affects the accuracy of the assay results, because of the distortion in the image of the sample taken by the imager.

[1311] (b) Methods and systems for spectrophotometer in the image system based on the image of the sample with the sample holder, where the image of the sample is divided into two sub-regions using a machine learning model. The first sub-region corresponds to the sample, and the second sub-region corresponds to the post. The two types of sub-regions are subjected to spectrochemical measurements based on one or more images taken at a single or multiple wavelengths of light, and these data are collected and applied to derive the chemical and physical properties of the sample, such as the concentration of a compound.

[1312] (c) Methods and systems for determining the volume of a sample based on a monitoring marker post with a uniform height, where the volume of the sample can be determined by the surface area of the sample in the image, since the sample is sandwiched between two parallel plates of the sample holder with a uniform gap controlled by the post. This proposed structure is able to actually remove any visible defects, posts, dust, bubbles, etc. from the sample and still have precise control over the volume of the sample after removal for the assay.

[1313] (d) Methods and systems for improving the quality of microscopic imaging in image-based assays using monitoring marker posts, where the local illumination, uniformity of illumination, and ratio distortion control can be based on the image of the post detected in the image-based assay. These parameters are difficult to control in microscopic imaging, especially for different types of samples imaged for the assay.

[1314] In general, detecting, locating, and segmenting the post in the image of the sample is a challenging problem, especially in microscopic images, such as those from image-based assays. In the described methods and embodiments, a specialized machine learning model for post detection is also constructed, and the above segmentation method is combined with true lateral dimension (TLD) correction for high-accuracy estimation of the post location, shape profile, and size, making the above method effective for image-based assays.

Claims

1. A method for improving the accuracy of an assay device for detecting an analyte in a sample, wherein the sample, the assay device, or operation of the assay device has one or more imperfect conditions, the method comprising: (a) detecting the analyte in the sample using the assay device to generate a detection result; (b) determining the confidence level of the test result by (i) imaging the sample in the assay device and (ii) processing the image using an algorithm; as well as (c) reporting the detection result only when the confidence level meets a predetermined threshold.

2. An apparatus for improving the accuracy of an assay device for detecting an analyte in a sample, wherein the sample, the assay device, or operation of the assay device has one or more imperfect conditions, the apparatus comprising: (a) an assay device for detecting the analyte in the sample to produce a test result, wherein the assay device has a sample holder; as well as (B) an imager that images the sample in the sample holder; as well as (c) A non-transitory storage medium storing an algorithm that uses the image to determine the confidence level of the detection result.

3. A method for monitoring imperfect conditions in an operational detection device, comprising: (a) Place the Q card with the monitoring mark on the plate (inside the sample), (b) performing sample deposition, (c) imaging the monitoring marker using an imager during the measurement process; (d) determining an operation error by comparing the image of the monitoring mark with an ideal image of the monitoring mark; in, Therefore, the ideal image of the monitoring mark is the image of the monitoring mark when operating correctly; The monitoring mark is prefabricated.

4. A method for improving the accuracy of a monitoring device having one or more imperfect conditions during operation of the detection device, wherein the imperfect conditions are unpredictable or random, comprising: (a) detecting an analyte in a sample containing or suspected of containing the analyte; (i) placing the sample in a detection instrument, and (ii) measuring the sample using the monitoring device to detect the analyte and generating a detection result of the detection; (b) determining the reliability of the test result in step (a), comprising: (i) imaging (1) a portion of the sample and / or (2) a portion of the detection device surrounding the portion of the sample, wherein the image substantially represents the conditions under which the portion of the sample is measured during the process of generating the detection result in step (a); and (ii) using a computing device having an algorithm to analyze the image formed in step (b)(i) to determine the reliability of the detection result in step (a); and (c) reporting the test results and the degree of confidence; wherein said step (a) has one or more unpredictable or random operating conditions.

5. A method for improving the accuracy of an image-based assay device for detecting an analyte in a sample, wherein the assay has an optical system with distortion, the method comprising: (a) having a sample holder having a sample contacting surface, wherein (i) a sample forms a thin layer 200 nm thick or less on the sample contacting surface, and (ii) one or more monitoring marks on the sample contacting surface of the sample, wherein the monitoring marks have a first set of parameters predetermined during manufacture of the sample holder; (b) capturing one or more images of the sample in the sample holder together with the monitoring marker using the optical system of the assay device, wherein the monitoring marker has a second set of parameters in the images; (c) processing the one or more images using a processor, wherein the processor detects distortion of the optical system by using the algorithm and the first set of parameters and the second set of parameters.

6. An apparatus for improving the accuracy of an image-based assay device for detecting an analyte in a sample, wherein the assay has an optical system with distortion, the apparatus comprising: (a) a sample holder having a sample contacting surface, wherein (i) a sample forms a thin layer 200 nm thick or less on the sample contacting surface, and (ii) one or more monitoring marks on the sample contacting surface of the sample, wherein the monitoring marks have a first set of parameters predetermined during manufacture of the sample holder; (b) an optical system of the assay device for capturing one or more images of the sample in the sample holder together with the monitoring marker, wherein the monitoring marker has a second set of parameters in the images; (c) a processor having a non-transitory storage medium storing an algorithm that processes the one or more images and corrects distortion of the optical system by using the algorithm and the first set of parameters and the second set of parameters.

Images