Intelligent measuring method based on image processing and machine learning and image processing system

Abstract

The invention relates to an intelligent determination method based on image processing and machine learning and an image processing system, belonging to biochemical detection technology, the image processing system comprises a sample card, an optical sensor, an adapter and a computing device comprising a processing device, and is used for improving the accuracy of image-based biological / chemical determination in a sample.

Description

[0001] This application is a divisional application of the Chinese national phase application with the application number 201980068655.3 and the invention title of Image-based Determination Using an Intelligent Monitoring Structure, and enjoys the same priority as the parent case.

[0002] Cross-reference to related applications

[0003] This application claims the priority benefits of U.S. Provisional Patent Application No. 62 / 719,129, filed on August 16, 2018, and U.S. Provisional Patent Application No. 62 / 764,886, filed on August 16, 2018, the entire contents of which are incorporated herein by reference. The entire disclosure of any publication or patent document mentioned herein is incorporated by reference in its entirety. Technical field

[0004] In particular, the present invention relates to devices and methods for performing biological and chemical assays, computational imaging, artificial intelligence, and machine learning. Background art

[0005] In biological and chemical assays, including complete blood counts, accuracy is necessary. However, under conditions of using low-cost instruments and limited resources, errors in the instruments and operations are inevitable. The present invention particularly provides devices and methods for improving the accuracy of image-based biological / chemical assays in samples. Summary of the invention

[0006] The following brief overview is not intended to include all features and aspects of the present invention.

[0007] One aspect of the present invention is a method for improving imaging-based assays using a sample holding device having a specially designed monitoring structure for assay accuracy, efficiency, error detection, monitoring, or any combination thereof.

[0008] Another aspect of the present invention is a method of micro-selective image assay (MSIA) that can perform multiple assay applications on a single image of a sample captured by an imager.

[0009] Another aspect of the present invention is a method for detecting monitoring structures in an image of a sample for assay and locating their centers, wherein the method combines machine learning with error correction using predetermined structural characteristics associated with multiple monitoring structures.

[0010] Another aspect of the present invention is a method for estimating the true lateral dimension (TLD) in image-based assays, whereby the actual size or morphological characteristics of an analyte in a sample image can be determined.

[0011] Another aspect of the present invention is a method for determining the actual area of a region of interest in a sample image and the associated sample volume, by which image-based measurements can be made on any selected sub-region to obtain flexibility and granularity.

[0012] Another aspect of the present invention is a method for monitoring the quality of a sample holding device and the quality of sample preparation from an image of a monitoring structure in the sample holding device in an image-based measurement.

[0013] Another aspect of the present invention is a method for using a pre-designed monitoring structure in a sample holding device to adjust the operation of an imager in an image-based measurement.

[0014] Another aspect of the present invention is a method for removing defects or foreign objects (such as bubbles, dust, etc.) from an image of a sample taken by an imager in an image-based measurement.

[0015] Another aspect of the present invention is a method for constructing a machine learning model for image-based measurement using an intelligent monitoring structure, in which two methods based on the original image of the sample and the transformed sample image are described.

[0016] Another aspect of the present invention is to use the methods and algorithms described herein to determine the actual usage of red blood cells based on images in a complete blood count (CBC).

[0017] A method for improving image-based measurement using a device, comprising:

[0018] (a) having a sample holder that includes a first plate and a second plate facing each other and a plurality of monitoring structures on the sample contact surface of one or both plates, wherein the monitoring structures have at least one pre-designed and predetermined parameter of geometric shape and / or optical property, and wherein the sample contact surface contacts the sample;

[0019] (b) clamping a sample containing or suspected of containing an analyte between the corresponding sample contact surfaces of the two plates to form a thin layer 200 um thick or thinner, wherein the sample on the sample contact surface is mixed and reacted with a reagent, and wherein the clamping, mixing, or reaction is susceptible to errors;

[0020] (c) using an imager to image the sample and the monitoring structures on the sample contact area together, wherein the imager is susceptible to defects in imaging, and wherein the imaging operation is susceptible to errors; and

[0021] (d) Analyze the image captured in step (c) using an algorithm to detect parameters related to the analyte, wherein the analysis includes comparing the image with the at least one pre-designed and predetermined parameter, and detecting and / or correcting the defects and errors using the at least one pre-designed and predetermined parameter. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Those skilled in the art will understand that the drawings described below are for illustrative purposes only. The drawings are not intended to limit the scope of the present teachings in any way. The drawings are not drawn to scale. In the figures presenting experimental data points, the lines connecting the data points are for guiding the observation of the data only and have no other meaning.

[0023] Figure 1 and Figure 2 shows a side view of an apparatus for imaging-based assays. Figure 1 shows a solid-phase surface with protruding-type monitoring markers. Figure 2 shows a solid-phase surface with trench-type monitoring markers. In the algorithm, characteristics corresponding to the monitoring markers (e.g., spacing and distance) can be used to determine the nature of the analyte in the sample.

[0024] Figure 3 and Figure 4 shows a side view of an apparatus for imaging-based assays. Figure 3 shows how a monitoring marker (e.g., protruding type) can be a structure separated from the spacer. Figure 4 shows how a monitoring marker can be the same structure as the spacer. In the algorithm, characteristics corresponding to the monitoring markers can be used to determine the nature of the analyte in the sample.

[0025] Figure 5 shows a side view of an apparatus for image-based assays. Figure 5 shows how a monitoring marker (e.g., trench type) can be a structure separated from the spacer. In the algorithm, characteristics corresponding to the monitoring markers can be used to determine the nature of the analyte in the sample.

[0026] Figure 6 and Figure 7 shows a side view of an apparatus for image-based assays. Figure 6 and Figure 7 shows how a monitoring marker can be a structure separated from the spacer and disposed on two sample contact regions of the apparatus. In the algorithm, characteristics corresponding to the monitoring markers can be used to determine the nature of the analyte in the sample.

[0027] Figure 8 is an exemplary diagram and workflow for using monitoring markers (e.g., pillars), an imaging process, and / or a machine learning algorithm.

[0028] Figure 9 Shows an analyte detection and localization workflow according to some embodiments of the present invention, which includes two phases, training and prediction.

[0029] Figure 10 Shows a process of removing an item from an ordered list according to some embodiments of the present invention.

[0030] Figure 11 Shows an embodiment of a QMAX card used in an image-based assay.

[0031] Figure 12 Shows a flowchart of an image-based assay of a sample holding device using a QMAX card.

[0032] Figure 13 Shows a flowchart of column or monitor marker-based TLD / FoV estimation in an image-based assay.

[0033] Figure 14 Shows a flowchart of training a machine learning model for column or monitor marker detection.

[0034] Figure 15 Shows a flowchart of generating a training database for a machine learning model based on image transformation.

[0035] Figure 16 Shows a flowchart of training an image transformation model for column or monitor marker detection based on image transformation.

[0036] Figure 17 Is a sample image of an image-based assay with columns in a sample holding device.

[0037] Figure 18 Shows a column detected in a transformed image for column or monitor marker detection.

[0038] Figure 19 Shows defects of bubbles and dust in a sample for an assay.

[0039] Figure 20 Is a three-side view of a sample holding device of a QMAX card with columns or monitor markers in an image-based assay using an imager.

[0040] Figure 21 Is an image of a sample with large bubbles.

[0041] Figure 22 Shows a column detected in an image of a sample.

[0042] Figure 23 Is a graph of light intensity curve versus position. Figure 24 The homography transformation described by the transformation matrix H for characterizing perspective projection.

[0043] Detailed description of exemplary embodiments

[0044] The following detailed description shows some embodiments of the present invention by way of example and not limitation. The section headings and subheadings used herein are for organizational purposes only and should not be construed as limiting the subject matter described in any way. The content under the section headings and / or subtitles is not limited to the section headings and / or subtitles, but applies to the entire description of the present invention.

[0045] Any reference to a publication is due to its publication date being earlier than the filing date and should not be construed as an admission that the claims of the present invention are not entitled to antedate such publications by virtue of a prior invention. In addition, the provided publication date may be different from the actual publication date, and the actual publication date may need to be independently confirmed.

[0046] In an image - based assay for determining an analyte in a sample, an imager is used to create an image of the sample in a sample holder, and the image is used to determine the properties of the analyte or the sample for the assay.

[0047] However, many factors can distort the image (i.e., be different from the true sample or an image under non - ideal conditions). Image distortion can lead to inaccurate determination of analyte properties. For example, the fact that the focus is poor because biological samples themselves do not have sharp edges that are preferred for focusing. When the focus is poor, the object size will be different from the true object size, and other objects (e.g., blood cells) may become unrecognizable. Another example is that the lens may not be perfect, resulting in different degrees of distortion at different positions of the sample. Another example is that the sample holder is not in the same plane as the optical imaging system, resulting in good focus in one area and poor focus in another area.

[0048] The present invention relates to devices and methods capable of obtaining a "true" image from a distorted image in an image - based assay, thus improving the accuracy of the assay.

[0049] One aspect of the present invention is a device and method using a monitoring marker having an optically observable flat surface parallel to an adjacent surface.

[0050] Another aspect of the present invention is a device and method using a QMAX card to form a layer of uniform thickness of at least a portion of the sample in the sample holding area of the QMAX card and using the monitoring markers on the card to improve the assay accuracy.

[0051] Another aspect of the present invention is apparatuses and methods for using monitoring markers and computational imaging, artificial intelligence, and / or machine learning in image-based assays.

[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. Although any methods and materials similar or equivalent to those described herein can also be used to practice or test the present teachings, some exemplary methods and materials are now described.

[0053] The term "lateral dimension" refers to a linear dimension in the plane of a thin sample layer being imaged.

[0054] The terms "true lateral dimension (TLD)" and "field of view (FoV)" are interchangeable.

[0055] The term "microfeatures in a sample" can refer to analytes, microstructures, and / or microscale variations in a substance in the sample. An analyte refers to particles, cells, macromolecules such as proteins, nucleic acids, and other moieties. A microstructure can refer to microscale differences in different materials. A microscale variation refers to a microscale change in the local properties of the sample. Examples of microscale variations are changes in local optical index and / or local mass. Examples of cells are blood cells such as white blood cells, red blood cells, and platelets.

[0056] As used herein, the term "sample" refers to a material or mixture of materials containing one or more analytes or entities of interest. In certain embodiments, the sample is a body fluid sample from a subject. In some cases, a solid or semi-solid sample may be provided. The sample may include tissue and / or cells collected from a subject. The sample may be a biological sample. Examples of biological samples may include but are not limited to blood, serum, plasma, nasal swab, nasopharyngeal wash, saliva, urine, gastric fluid, spinal fluid, tears, feces, mucus, sweat, earwax, sebum, glandular secretions, cerebrospinal fluid, tissue, semen, vaginal secretions, interstitial fluid derived from tumor tissue, ocular fluid, spinal fluid, throat swab, exhaled condensate (e.g., breath), hair, nails, skin, biopsy, placental fluid, amniotic fluid, cord blood, lymph fluid, cavity fluid, sputum, pus, microbiota, meconium, breast milk, and / or other excretions. The sample may include a nasopharyngeal wash. Nasal swabs, throat swabs, fecal samples, hair, nails, earwax, breath, and other solid, semi-solid, or gaseous samples may be processed (e.g., fixed or for variable times) in an extraction buffer prior to their analysis. If desired, the extraction buffer or an aliquot thereof may then be processed similar to other fluid samples. Examples of tissue samples from a subject may include but are not limited to connective tissue, muscle tissue, nerve tissue, epithelial tissue, cartilage, cancer samples, or bone. In specific embodiments, a sample may be obtained from a subject such as a human and may be processed prior to use in a subject assay. For example, prior to analysis, proteins / nucleic acids may be extracted from a tissue sample by methods known in the art. In certain embodiments, the sample may be a clinical sample, e.g., a sample collected from a patient. The sample may also be a sample of food, environment, etc. Some samples have a deformable but non-free-flowing shape (e.g., sputum).

[0057] The term "analyte" refers to any substance suitable for testing in the present invention. Analytes include but are not limited to atoms, molecules (e.g., proteins, peptides, DNA, RNA, nucleic acids, or other molecules), cells, tissues, viruses, bacteria, and nanoparticles of different shapes. Biomarkers are analytes.

[0058] As used herein, the terms "determine," "measure," "evaluate," and "assay" are used interchangeably and include both quantitative and qualitative determinations.

[0059] As used herein, the term "luminescent label" refers to a label that is capable of emitting light when externally excited. This may be luminescent. Fluorescent labels (which include dye molecules or quantum dots) and luminescent labels (e.g., electrochemiluminescent or chemiluminescent labels) are types of luminescent labels. The external excitation is light (photons) for fluorescence, current for electrochemiluminescence, and chemical reaction for chemiluminescence. The external excitation may be a combination of the above.

[0060] The phrase "labeled analyte" refers to an analyte detectably labeled with a luminescent tag such that the analyte can be detected by assessing the presence of the tag. The labeled analyte can be directly labeled (i.e., the analyte itself can be directly conjugated to the tag, e.g., by a strong bond such as a covalent or non-covalent bond), or the labeled analyte can be indirectly labeled (i.e., the analyte is bound by a second capture agent that is directly labeled).

[0061] With respect to nucleic acids, the terms "hybridization" and "binding" are used interchangeably.

[0062] The term "hybridization" refers to the reaction of one or more polynucleotides to form a complex that is stabilized by hydrogen bonds between the bases of the nucleotide residues. The hydrogen bonds can occur through Watson-Crick base pairing, Hoogstein binding, or in any other sequence-specific manner. The complex can comprise two strands that form a duplex structure, three or more strands that form a multi-stranded complex, a single self-hybridizing strand, or any combination of these.

[0063] As is known to those skilled in the art, hybridization can be carried out under conditions of various stringencies. Suitable hybridization conditions are such that the recognition interaction between the capture sequence and the target nucleic acid is sufficiently specific and sufficiently stable. Conditions that increase the stringency of the hybridization reaction are well known and published in the art. See, e.g., Green et al. (2012), infra.

[0064] The terms "spacer" and "optical calibration marker" and "optical calibration marker" and "post" are interchangeable.

[0065] The term "assay" refers to a research (analytical) procedure in but not limited to laboratories, medicine, pharmacology, environmental biology, healthcare, and molecular biology - for but not limited to qualitatively evaluating or quantitatively measuring the presence, amount, concentration, or functional activity of a target entity (i.e., an analyte). The analyte can be a drug, a biochemical, or a cell in an organism or an organic sample (e.g., human blood).

[0066] The term "image-based assay" refers to an assay procedure that utilizes an image of a sample taken by an imager, where the sample can be but not limited to medical, biological, and chemical samples.

[0067] The term "imager" refers to any device capable of taking an image of an object. It includes but not limited to a camera in a microscope, a smart phone, or a special device that can take images at various wavelengths.

[0068] The term "sample feature" refers to some properties of a sample that represent conditions of potential interest. In some embodiments, a sample feature is a feature that appears in an image of the sample and can be segmented and classified by a machine learning model or some other algorithm. Examples of sample features include, but are not limited to, the type of analyte in the sample, such as red blood cells, white blood cells, and tumor cells, and include analyte shape, count, size, volume, concentration, etc.

[0069] The term "defects in the sample" refers to foreign objects and artifacts that should not be present under ideal sample conditions or should not be considered in the sample. They can come from, but are not limited to, contaminants such as dust, bubbles, etc., and from peripheral objects, including structural objects in the sample, such as monitoring markers (e.g., columns) in a sample holding device. Defects in the sample can have significant dimensions and occupy a significant amount of volume in the sample for determination, such as bubbles, where they can appear in the sample in different shapes, sizes, amounts, and concentrations, and they also have sample dependence depending on the sample.

[0070] The term "morphological features" of an analyte refers to the appearance (e.g., shape, color, size, etc.) and structure of the analyte.

[0071] The term "homography transformation" refers to a class of collinear transformations caused by an isomorphism of projective spaces. It is known in the field of image processing and is applied in camera models to characterize the image plane in the real world and the corresponding physical plane.

[0072] The term "machine learning" refers to algorithms, systems, and devices in the field of artificial intelligence that typically use statistical techniques and artificial neural networks to provide a computer with the ability to "learn" (i.e., gradually improve performance on a specific task) from data without being explicitly programmed.

[0073] The term "artificial neural network" refers to a hierarchically connected system inspired by biological networks that can "learn" by considering examples to perform tasks, usually without being programmed with any task-specific rules.

[0074] The term "convolutional neural network" refers to a class of multi-layer feedforward artificial neural networks most commonly applied to analyze visual images.

[0075] The term "deep learning" refers to a large class of machine learning methods in artificial intelligence (AI) that learn from data with a certain depth of network structure.

[0076] The term "machine learning model" refers to a trained computational model constructed from the training process in machine learning of data. The trained machine learning model is applied by a computer during the inference phase, which gives the computer the ability to perform a specific task (e.g., detecting and classifying objects). Examples of machine learning models include ResNet, DenseNet, etc., which are also referred to as "deep learning models" due to the depth of their hierarchical network structures.

[0077] The term "image segmentation" refers to an image analysis process of dividing a digital image into multiple image patch segments (groups of pixels, usually with a set of bitmap masks covering the image segments surrounded by their segment boundary contours). Image segmentation can be achieved by image segmentation algorithms in image processing, such as watershed, grabcut, mean shift, etc., and it can also be achieved by dedicated machine learning algorithms, such as MaskRCNN, etc.

[0078] A. Monitoring markers on a solid phase surface

[0079] A1-1. An apparatus for determining microfeatures in a sample using an imager, the apparatus comprising:

[0080] (a) A solid phase surface that includes a sample contact area for contacting a sample containing microfeatures; and

[0081] (b) One or more monitoring markers, where the monitoring markers:

[0082] i. Are made of a material different from the sample;

[0083] ii. Are located inside the sample during the determination of the microstructure, where the sample forms a thin layer with a thickness less than 200 um on the sample contact area;

[0084] iii. Have a lateral linear dimension of about 1 um (micrometer) or greater, and

[0085] iv. Have at least one lateral linear dimension of 300 um or less; and

[0086] where during the determination process, the imager images at least one monitoring marker

[0087] where it is used during the determination of the analyte; and the geometric parameters (e.g., shape and size) of the monitoring markers and / or the spacing between the monitoring markers are (a) predetermined and known before the analyte determination, and (b) used as parameters in an algorithm for determining the characteristics related to the microfeatures.

[0088] A1-2. An apparatus for determining microfeatures in a sample using an imager, the apparatus comprising:

[0089] A solid-phase surface that includes a sample contact area for contacting a sample that includes microfeatures; and

[0090] One or more monitoring markers, where each monitoring marker includes a protrusion or groove from the solid-phase surface, where:

[0091] i. The protrusion or groove includes a flat surface that is substantially parallel to an adjacent surface, which is the portion of the solid-phase surface adjacent to the protrusion or groove;

[0092] ii. The distance between the flat surface and the adjacent surface is about 200 micrometers (μm) or less;

[0093] iii. The flat surface has (a) a linear dimension of at least about 1 μm or greater, and (b) a region with at least one linear dimension of 150 μm or less;

[0094] iv. The flat surface of at least one monitoring marker is imaged by an imager used in determining the microfeatures; and

[0095] v. The shape of the flat surface, the dimensions of the flat surface, the distance between the flat surface and the adjacent surface, and / or the spacing between the monitoring markers are (a) predetermined and known prior to the determination of the microfeatures, and (b) used as parameters in an algorithm for determining properties associated with the microfeatures.

[0096] B. Monitoring Markers on the QMAX Card

[0097] A2-1. An apparatus for determining microfeatures in a sample using an imager, the apparatus comprising:

[0098] A first plate, a second plate, spacers, and one or more monitoring markers, where:

[0099] i. The first plate and the second plate are movable relative to each other into different configurations;

[0100] ii. Each of the first plate and the second plate includes an inner surface that includes a sample contact area for contacting a sample that includes microfeatures;

[0101] iii. One or both of the first plate and the second plate include spacers permanently fixed to the inner surface of the respective plate,

[0102] iv. The spacers have a substantially uniform height and a fixed spacing distance (ISD) that is equal to or less than 200 micrometers;

[0103] v. The monitoring markers are made of a material different from the sample;

[0104] vi. Monitoring the marker during the determination of the microstructure, where the sample forms a thin layer with a thickness less than 200 um on the sample contact area; and

[0105] vii. The monitoring marker has a lateral linear dimension of about 1 um (micrometer) or greater and has at least one lateral linear dimension of 300 um or less;

[0106] Wherein during the determination process, the imager images at least one monitoring marker

[0107] Which is used during the determination of microfeatures; and the shape, size of the flat surface, the distance between the flat surface and the adjacent surface, and / or the spacing between the monitoring markers (a) are predetermined and known before the determination of the microfeatures, and (b) are used as parameters in the algorithm for determining the characteristics related to the microfeatures;

[0108] Wherein one configuration is an open configuration, in which: the two plates are partially or completely separated, the spacing between the plates is not adjusted by spacers, and the sample is deposited on one or both plates;

[0109] Wherein the other of the configurations is a closed configuration, which is configured after the sample is deposited in the open configuration and the plates are forced to reach the closed configuration by applying a pressing force on the stressed area; and in the closed configuration: at least a part of the sample is compressed by the two plates into a layer with a very uniform thickness and is substantially stationary relative to the plates, wherein the uniform thickness of the layer is defined by the sample contact areas of the two plates and is adjusted by the plates and spacers; and

[0110] Wherein the monitoring marker is (i) a structure different from the spacer, or (ii) the same structure used as the spacer.

[0111] A2-2. An apparatus for determining microfeatures in a sample using an imager, the apparatus comprising:

[0112] A first plate, a second plate, a spacer, and one or more monitoring markers, wherein:

[0113] i. The first plate and the second plate are movable relative to each other into different configurations;

[0114] ii. Each of the first plate and the second plate includes an inner surface, and the inner surface includes a sample contact area for contacting a sample containing microfeatures;

[0115] iii. One or both of the first plate and the second plate include spacers permanently fixed on the inner surfaces of the respective plates;

[0116] iv. The spacer has a substantially uniform height equal to or less than 200 micrometers and a fixed spacing distance (ISD);

[0117] v. Each monitoring mark includes a protrusion or a groove on one or two sample contact areas;

[0118] vi. The protrusion or groove includes a flat surface substantially parallel to an adjacent surface, which is the part of the solid phase surface adjacent to the protrusion or groove;

[0119] vii. The distance between the flat surface and the adjacent surface is about 200 micrometers (μm) or less;

[0120] viii. The flat surface has (a) a linear dimension of at least about 1 μm or greater, and (b) an area with at least one linear dimension of 150 μm or less;

[0121] ix. The flat surface of at least one monitoring mark is imaged by an imager used in determining microfeatures; and

[0122] x. The shape of the flat surface, the size of the flat surface, the distance between the flat surface and the adjacent surface, and / or the spacing between the monitoring marks (a) are predetermined and known before the determination of the microfeatures, and (b) are used as parameters in an algorithm for determining the characteristics associated with the microfeatures.

[0123] One of the configurations is an open configuration, in which: the two plates are partially or completely separated, the spacing between the plates is not adjusted by spacers, and the sample is deposited on one or two plates;

[0124] Another of the configurations is a closed configuration, which is configured after the sample is deposited in the open configuration and the plates are forced to the closed configuration by applying a pressing force on the stressed area; and in the closed configuration: at least a part of the sample is compressed by the two plates into a layer with a very uniform thickness and is substantially stationary relative to the plates, where the uniform thickness of the layer is defined by the sample contact areas of the two plates and is adjusted by the plates and the spacers; and

[0125] Wherein the monitoring mark is (i) a structure different from the spacer, or (ii) the same structure used as the spacer.

[0126] A3. An apparatus for image-based determination, comprising:

[0127] An apparatus as in any of the previous apparatus embodiments, wherein the apparatus has at least five monitoring marks, and at least three of the monitoring marks are not aligned on a straight line.

[0128] A4. An apparatus for determining an analyte in a sample using an imager, the system comprising:

[0129] (a) An apparatus as in any of the previous apparatus embodiments; and

[0130] (b) An imager for measuring a sample containing an analyte.

[0131] A5. A system for performing an imaging-based assay, the system comprising:

[0132] (a) A device as in any preceding device embodiment;

[0133] (b) an imager for measuring a sample containing an analyte; and

[0134] (c) an algorithm for utilizing the monitoring markers of the device to determine a property associated with the analyte.

[0135] In some embodiments, the thickness of the thin layer is configured so that the analytes form a monolayer in the sample holder. The term "monolayer" means that in the thin sample layer inside the sample holder, there is substantially no overlap between two adjacent analytes in a direction perpendicular to the plane of the sample layer.

[0136] C. Monitoring markers with computational imaging artificial intelligence and / or machine learning

[0137] Another aspect of the invention is to utilize a column or monitoring tag of a sample holding device with computational imaging, artificial intelligence and / or machine learning. It has a process of forming an image based on measurements, using algorithms to process the image and map the objects in the image to their actual size in the real world. Machine learning (ML) is applied in the present invention to learn the salient features of objects in the sample captured by the ML model, constructed and trained from images of the sample taken by the imager. Intelligent decision logic is constructed and applied in the inference process of the present invention to detect and classify target objects in the sample based on the knowledge embedded in the ML model.

[0138] Figure 9 A block diagram of process 100 for TLD (true lateral dimension) / FoV (field of view) estimation in the present invention is shown. In various implementations of process 100, actions may be removed, combined, or decomposed into sub-actions. The process starts at action module 101, where the process captures an image of a sample for determination, where the image is taken by an imager such as a QMAX card above the sample holding device, which has a column or monitoring mark and the structure described in the previous section of the monitoring mark on the QMAX card. The captured image from action module 101 is fed as input to action module 102, where the image for determination is processed and searched to detect and locate the center of the column or monitoring mark. In a typical implementation, process 102 has a pre-trained machine learning model M_column to detect the column or monitoring mark in the image for determination.

[0139] As described above, the pillars or monitoring markers in the present invention have known shapes and dimensions and are fabricated by a high-precision nanoimprint manufacturing process. Additionally, these pillars or monitoring markers are distributed in the sample holding device in a predefined periodic pattern and have a known spacing between them. These features in the present invention become critical for process 100 because the pillars or monitoring markers are small, approximately ~30 microns, surrounded by the sample in the sample holding device, and subject to strong light scattering and diffraction among the particles in the sample. Moreover, they are generally not in the focus of the imager because the imager is focused on the analyte in the sample for image-based assays rather than on the structure of the sample holder. Therefore, errors occur even when using a machine learning model for detection, where some pillars are not detected, are detected erroneously, or are detected at an incorrect or inaccurate center position, making the direct use of the detected pillars and center positions unreliable and introducing errors for TLD / FoV estimation.

[0140] As Figure 13 illustrated, action module 103 in process 100 obtains the detected pillars and their centers from action module 102 and performs post-error correction and estimation refinement based on the characteristics of the pillars or monitoring markers described in the present invention. In particular, the detected pillars should be aligned in a predefined known periodic pattern, and the distance between the centers of two adjacent pillars should be subject to a known spacing, etc. The actions taken by action module 103 are to eliminate false detections, align the detected pillars in a known design pattern during manufacturing, use the periodicity of their distribution to find others, and adjust the positions of the pillar centers according to the known periodicity and the spacing between them. The combination of the actions taken by action modules 102 and 103 in the present invention makes the pillar or monitoring marker detection robust to the accuracy of TLD / FoV estimation.

[0141] In action module 103 of process 100, it estimates a homography transformation based on the detected pillar centers from action modules 102 and 103. In the field of image processing, a homography transformation is known, where the image captured by the imager on the sample is modeled by a perspective projection between the object in the image being captured and the actual object in the real world on which it is imaged. This perspective projection can be characterized by a homography transformation (also known as a perspective transformation) described by the transformation matrix H as follows.

[0143] Represented by x′ = Hx, where H is a 3×3 non-singular homogeneous matrix that maps the object in the image captured by the imager to the object in the actual sample plane being imaged. According to this matrix, the length or area of the object in the image captured by the imager is mapped to its actual size in the real world, and through this matrix, the TLD / FoV of the object and its actual size in the image can be determined.

[0144] However, to determine the transformation matrix H, at least 4 pairs of points corresponding to the mapping of H from the image captured by the imager to the actual sample plan to be imaged are required. And these 4 pairs of points are used as anchors, which combine the image captured by the imager and the actual sample plan imaged through the homography transformation between them, and which require at least 4 non-collinear column centers to make the H matrix non-singular. By the actions in action module 102 and by applying the structural features of the columns or monitoring markers in action module 103, more than 4 non-collinear column center points can be detected, and their corresponding positions in the actual image plane can be determined based on the structural features of the columns or monitoring markers in the sample holding device. In this way, action module 104 of process 100 estimates the homography transformation matrix H for TLD / FoV estimation based on the detected column centers from action module 103. In addition, in action module 104, the estimated transformation matrix H is used for TLD / FoV estimation in subsequent image-based determination processes.

[0145] In the present invention, a dedicated machine learning model for column or monitoring marker detection is constructed from training data. In some embodiments of the present invention, column and monitoring marker detection is performed directly on the image for determination. Figure 14 A block diagram of process 200 for constructing a machine learning model for column or monitoring marker detection on the original input image for determination is shown. The process starts in action module 201, where the process obtains a set of images for determination from the imager in training database DB0. These images are collected by taking images of the sample in a sample holding device (such as a QMAX card). In action module 202, each training image is retrieved from DB0, and the columns or monitoring markers in each image are marked. The marked images are saved in a second training database DB1 for machine learning model training. This training dataset DB1 is dedicated to training a machine learning model to detect columns or monitoring markers, which is different from the typical training database for detecting analytes in a sample.

[0146] In action module 203, the new training database DB1 is obtained from action module 202, and a machine learning model structure in the form of a deep neural network is selected to train the model for training database DB1. In some embodiments of the present invention, a machine learning model of RetinaNet is used, and in some other embodiments, a machine learning model of Fast-RCNN is selected. Using training database DB1 for column or monitoring marker detection, Tensorflow and PyTourch are used to train the machine learning model. Process 200 ends in action module 205, in which the machine learning model obtained from action module 204 is verified and saved for determination applications.

[0147] Figure 15 It is a block diagram for creating a specialized training database for an image transformation-based machine learning method for column and monitoring marker detection. Process 300 starts with action module 301, which acquires a training image database DB1A consisting of images taken by an imager for determination. In action module 302, the images from DB1A are labeled for columns or monitoring markers. The labeled images in DB1A are the input to action module 303, where the labeled images are transformed by covering the labeled columns or monitoring markers with a white mask along their contours. The transformed images from action module 303 are the input to action module 304, where the images are further transformed by applying a black mask to the areas in the images that are not covered by the white mask from action module 303. In the present invention, using a white mask for columns and a black mask for the areas not covered by the white mask has two advantages. One is to maximize the contrast between the columns and other areas, and the second is to suppress the noise in the background, making the subsequent detection of columns or monitoring markers and their centers more robust. In action module 304, the images transformed by 302 and 303 are verified and saved in a new target image training database DB2A to train an image transformation model in column or detection marker detection.

[0148] Figure 16 It shows a block diagram for constructing an image transformation-based machine learning model for column and monitoring marker detection. Process 400 starts with action module 401. Action module 401 loads a training image database DB1A consisting of images taken by an imager for determination. Action module 402 loads a target image training database DB2A obtained by transforming the images in DB1A using the previously described process 300. In action module 403, it takes the training image database DB1A and the paired target image training database DB2A from action module 402 as training targets. It selects a machine learning framework to train a machine learning model that transforms the original images taken by the imager for determination into new images with white columns or monitoring masks and black backgrounds in the target image training database. In some embodiments of the present invention, a machine learning framework for pixel-to-pixel (PP) transformation is selected, while in some other embodiments of the present invention, a machine learning framework of CycleGAN is used. Action module 403 trains a machine learning model TModel using the training image database DB1A and the paired target image transformation database DB2A from process 300. In an embodiment of the present invention, column and monitoring marker detection includes the following actions:

[0149] 1. Taking the images for determination as input;

[0150] 2. Apply the image transformation model TModel to transform the input image to measure an image with a white mask covering columns or monitoring markers, and cover the remaining area with the black mask described herein;

[0151] 3. Detect the columns or monitoring markers in the transformed image;

[0152] 4. Remap the centers of the detected columns or monitoring markers back to the original input image;

[0153] 5. Use the constraints from the pre-designed column or monitoring marker structure in the sample holding device to correct the error in the position of the detected columns or monitoring markers relative to the original input image;

[0154] 6. Determine the centers of the detected columns or monitoring markers and refine their positions in the sample image according to the constraints of the known distribution structure in the sample holding device; and

[0155] 7. Estimate the holomorphic transformation on the image of the sample based on the centers of the detected columns or monitoring markers to determine the TLD / FoV for subsequent image-based measurements.

[0156] Figure 21 is the actual image of the sample taken by the imager for measuring red blood cells. Apparently, the noise level in the image of the sample is quite high, and in the image of the sample, the focus on the monitoring structure (such as columns or monitoring markers) in the sample holder is poor because in image-based measurements, the focus is on the analyte rather than on the prereferal. This makes the detection and localization of columns or monitoring markers in the sample image challenging. Figure 22 Shows the detection results of the columns of the monitoring markers in the image of the sample for measuring red blood cells based on the transformed image using the systems and methods described herein.

[0157] In some embodiments of the present invention, the image for measurement is divided into non-overlapping patches, and if there are at least 4 detected non-collinear column centers in the patch, a patch-dependent holomorphic transformation is estimated for each image patch. Otherwise, the holomorphic transformation estimated from all the detected column centers in the image is used to determine the TLD / FoV in the image-based measurement.

[0158] Microselective image measurement using region of interest selection, volume estimation, and defect removal

[0159] The TLD / FoV estimation method described in this paper opens up possibilities for other applications in image-based measurements. In some embodiments of the present invention, it is used to remove defects in image-based measurements, such as bubbles, dust, etc. Depending on the environment, measurement operation, and type of sample used for measurement, bubbles and dust may appear in the sample. To better measurement accuracy, these defects in the sample used for measurement should be removed from the sample. However, once these defects occur and are trapped in the closed space of the sample holding device, this can be extremely difficult. Moreover, in many cases, the device is pressure-sealed and should not be opened again.

[0160] Figure 19 The situation where bubbles and dust appear in the sample is depicted. In some embodiments of the present invention, it performs micro-selective image measurement, including:

[0161] 1. Taking an image of the sample used for measurement as input;

[0162] 2. Estimating the TLD / FoV of the sample image from (1) to estimate the area, size, and subsequent sample volume;

[0163] 3. Detecting defects, such as bubbles, dust, etc., in the image of the sample through a trained machine learning model for measuring and segmenting these defects in the image of the sample;

[0164] 4. Estimating the total area of the segmented defects in the sample image and calculating their actual area size by using the estimated TLD / FoV from (2), which utilizes the column of the sample holding device and the monitoring markers;

[0165] 5. Estimating the actual volume of the sample corresponding to the total surface area of the detected defects in the image of the sample used for measurement based on the area estimation from (4) and the known height of the sample in the sample holding device;

[0166] 6. Removing the surface area of the detected defects in the image of the sample for measurement and updating the total volume of the sample by subtracting the defect volume estimation, where the defect volume estimation corresponds to the total volume under the surface area of the detected defects in the image of the sample; and

[0167] 7. Performing image-based measurement on the selected region in the updated sample image using the updated sample volume from (6).

[0168] In some embodiments of the present invention, the area removed from the image of the sample is larger than the area of the detected defect with a margin Δ, and thus, a larger volume is actually removed based on the enlarged defect area in the image-based determination. One benefit of this operation is to further reduce the negative impact of the defect on the determination result, because the analyte can attach to the defect and can affect the local uniformity of the analyte distribution.

[0169] As described herein, the present invention critically uses the structure of a sample holding device such as a QMAX card, and columns or monitoring markers therein. In particular, the image of the sample taken by the imager on the sample holding device (e.g., QMAX card) is a pseudo 3D image because the height of the sample in the sample holding device is a priori known and uniform. Thus, once the actual area size in the original sample plan can be obtained, the sample volume corresponding to the surface area of the object in the image of the sample can be determined.

[0170] To obtain an estimate of the actual area in the original sample plan, the structure of the columns or monitoring markers in the sample holding device is utilized to perform a more reliable TLD / FoV estimate. This leads to the paradigm of microselective image determination in image-based assays, by which the assay process can select specific regions / volumes from the image of the sample taken by the imager for determination.

[0171] For example, in some embodiments of the present invention, it selects specific regions where the analyte does not form clusters, and the sample volume of the selected region is determined by the image of the sample and the uniform height of the sample in the sample holding device.

[0172] In some embodiments, it selects regions in the image of the sample based on fewer defects, better signal-to-noise ratio, focusing conditions, etc. to obtain better determination accuracy. The combination of machine learning-based column and monitoring marker detection for FoV estimation and machine learning-based defect detection and segmentation to reduce variations in the sample makes the method described in the present invention flexible and resilient in image-based assays.

[0173] Monitoring the assay operation using monitoring markers

[0174] In some embodiments, the thickness of the sample is configured to be a thin thickness such that the objects of interest (e.g., cells) form a single layer (i.e., there is no significant overlap between the objects in the direction perpendicular to the sample layer).

[0175] Similarly, a method for determining the manufacturing quality of a QMAX card using an imager includes:

[0176] (a) Obtain a device as in any of the foregoing embodiments, wherein the device comprises two movable plates, spacers, and one or more monitoring markers, and wherein the monitoring markers are in the sample contact area;

[0177] (b) Obtain an imager;

[0178] (c) Deposit a sample in the sample contact area of the device of (a), and force the two plates into a closed configuration;

[0179] (d) Use the imager to take one or more images of the thin sample layer; and

[0180] (e) Use the image of the monitoring markers to determine the manufacturing quality of the QMAX card.

[0181] In addition, a method for determining the manufacturing quality of a QMAX card using an imager, the method comprising:

[0182] (a) Obtain a device as in the previous embodiments, wherein the device has two movable plates, spacers, and one or more monitoring markers, and wherein the monitoring markers are in the sample contact area;

[0183] (b) Obtain an imager;

[0184] (c) Deposit a sample in the sample contact area of the device of (a), and force the two plates into a closed configuration;

[0185] (d) Use the imager to take one or more images of the thin sample layer and the monitoring markers; and

[0186] (e) Use the image of the monitoring markers to determine the manufacturing quality of the QMAX card.

[0187] A method as in any of the foregoing embodiments, wherein determining the manufacturing quality comprises measuring one or more characteristics of the monitoring markers (e.g., length, width, spacing, thick edge), and comparing the measured characteristics with a reference value to determine the manufacturing quality of the QMAX card.

[0188] A method as in any of the foregoing embodiments, wherein determining the manufacturing quality comprises measuring a first characteristic of one or more first monitoring markers (e.g., quantity, length, width, spacing, thick edge), and comparing the measured first characteristic with a second characteristic of one or more second monitoring markers (e.g., number, length, width, spacing, thick edge) to determine the manufacturing quality of the QMAX card.

[0189] A method as in any of the foregoing embodiments, wherein the determination is performed during the analysis of a sample using a device as in any of the foregoing embodiments.

[0190] One aspect of the present invention is for performing assays using a QMAX card having two movable plates, and monitoring markers placed within a thin sample can be used to monitor the operating conditions of the QMAX card. The operating conditions may include whether the sample loading is correct, whether the two plates are properly closed, and whether the gap between the two plates is the same as or approximately the same as a predetermined value.

[0191] In some embodiments, for a QMAX card that includes two movable plates and has a predetermined gap between the two plates in a closed configuration, the operating conditions of an assay based on the QMAX card are monitored by taking an image of the monitoring marker in the closed configuration. For example, if the two plates are not properly closed, the monitoring marker will appear differently in the image than when the two plates are properly closed. A monitoring marker surrounded by the sample (properly closed) will have a different appearance than a monitoring marker not surrounded by the sample (not properly closed). Thus, it can provide information about the sample loading conditions. An apparatus for monitoring the operating conditions of a device using a monitoring marker includes a first plate, a second plate, a spacer, and one or more monitoring markers, wherein:

[0192] i. The first plate and the second plate are movable relative to each other into different configurations;

[0193] ii. Each of the first plate and the second plate includes an inner surface that includes a sample contact area for contacting the sample to be analyzed;

[0194] iii. One or both of the first plate and the second plate include a spacer permanently fixed to the inner surface of the respective plate;

[0195] iv. The monitoring marker has at least one of (a) a predetermined and known, and (b) a size observable by an imager;

[0196] v. The monitoring marker is a microstructure having at least one lateral linear dimension of 300 um or less; and

[0197] vi. The monitoring marker is located within the sample;

[0198] One of the configurations is an open configuration, in which: the two plates are partially or fully separated, the spacing between the plates is not adjusted by the spacer, and the sample is deposited on one or both plates;

[0199] The other configuration is a closed configuration, which is configured after the sample is deposited in the open configuration and the plates are forced into the closed configuration by applying an imprecise pressing force on the force application area; and in the closed configuration: at least a portion of the sample is compressed by the two plates into a layer with a very uniform thickness and is substantially stagnant relative to the plates, wherein the uniform thickness of the layer is defined by the sample contact areas of the two plates and is adjusted by the plates and the spacer; and

[0200] After a force is used to bring the two plates into a closed configuration, the monitoring marker is imaged to determine (i) whether the two plates have reached the desired closed configuration, thereby adjusting the sample thickness to a generally predetermined thickness, and / or (ii) determining whether the sample has been loaded as required.

[0201] In some embodiments, an image of the monitoring marker is used to determine whether the two plates have reached the desired closed configuration, where the sample is adjusted to have a thickness of approximately the predetermined thickness.

[0202] In some embodiments, an image of the monitoring marker is used to determine whether the sample has been loaded as required.

[0203] In some embodiments, the monitoring marker is imaged to determine whether the two plates have reached the desired closed configuration in which the sample thickness is adjusted to the predetermined thickness, and to determine whether the sample has been loaded as required.

[0204] In some embodiments, the spacer serves as the monitoring marker.

[0205] In some embodiments, the system includes a device, a computing device, and a non-transitory computer-readable medium having instructions that, when executed, perform the determination process in the determination.

[0206] In some embodiments, a non-transitory computer-readable medium having instructions that, when executed, perform a method that includes using one or more images of a thin sample layer and a monitoring marker to determine (i) whether the two plates have reached the desired closed configuration, thereby adjusting the sample thickness to a generally predetermined thickness, or (ii) whether the sample has been loaded as required.

[0207] In some embodiments, the system includes a non-transitory computer-readable medium having instructions that, when executed, perform any method of the present disclosure.

[0208] W-1. A method for monitoring operating conditions of a device using a monitoring marker, the method comprising:

[0209] a) Obtaining a device as in any of the foregoing embodiments, wherein the device includes two movable plates, a spacer, and one or more monitoring markers, wherein the monitoring marker is in the sample contact area;

[0210] b) Obtaining an imager;

[0211] c) Depositing a sample in the sample contact area of the device of (a) and forcing the two plates into a closed configuration;

[0212] d) Taking one or more images of the thin sample layer using an imager and monitoring the markers; and

[0213] e) Using the images of the monitored markers to determine (i) whether the two plates have reached the desired closed configuration, thereby adjusting the sample thickness to a substantially predetermined thickness, or (ii) whether the sample has been loaded as required.

[0214] Some examples

[0215] Single plate

[0216] AA-1.1 An apparatus for determining microfeatures in a sample using an imager, the apparatus comprising:

[0217] (a) A solid-phase surface that includes a sample contact area for contacting a thin sample having a thickness of 200 um or less and that includes or is suspected of including microfeatures; and

[0218] (b) One or more markers, wherein the markers:

[0219] I. Have sharp edges that (i) have a predetermined and known shape and size and (ii) can be observed by an imager that images the microfeatures;

[0220] II. Are microstructures having at least one lateral linear dimension of 300 um or less; and

[0221] III. Are located inside the sample;

[0222] wherein at least one of the markers is imaged by the imager during the determination.

[0223] AA-1.2 An apparatus for determining microfeatures in a sample using an imager, the apparatus comprising:

[0224] (a) A solid-phase surface that includes a sample contact area for contacting a thin sample having a thickness of 200 um or less and that includes or is suspected of including microfeatures; and

[0225] (b) One or more markers, wherein the markers:

[0226] i. Include protrusions or grooves from the solid-phase surface

[0227] ii. Have sharp edges that (i) have a predetermined and known shape and size and (ii) can be observed by an imager that images the microfeatures;

[0228] iii. Are microstructures having at least one lateral linear dimension of 300 um or less; and

[0229] iv. Are located inside the sample;

[0230] At least one of the markers is imaged by the imager during the measurement.

[0231] Two plates with a constant spacing

[0232] AA-2.1 An apparatus for measuring microfeatures in a sample using an imager, the apparatus comprising:

[0233] A first plate, a second plate, and one or more monitoring markers, wherein:

[0234] i. Each of the first and second plates includes an inner surface that includes a sample contact region for contacting a sample that contains or is suspected of containing microfeatures;

[0235] ii. At least a portion of the sample is defined by the first and second plates as a thin layer having a substantially constant thickness of 200 µm or less;

[0236] iii. The monitoring marker has a sharp edge that (a) has a predetermined and known shape and size and (b) is observable by an imager that images the microfeatures;

[0237] iv. The monitoring marker is a microstructure having at least one lateral linear dimension of 300 µm or less; and

[0238] v. The monitoring marker is located inside the sample;

[0239] At least one of the markers is imaged by the imager during the measurement.

[0240] Two movable plates

[0241] AA-3 An apparatus for measuring microfeatures in a sample using an imager, the apparatus comprising:

[0242] a. A first plate, a second plate, a spacer, and one or more monitoring markers, wherein:

[0243] ii. The first and second plates are movable relative to each other into different configurations;

[0244] iii. Each of the first and second plates includes an inner surface that includes a sample contact region for contacting a sample that contains or is suspected of containing microfeatures;

[0245] iv. One or both of the first and second plates include spacers permanently fixed to the inner surfaces of the respective plates,

[0246] v. The monitoring marker has a sharp edge that (a) has a predetermined and known shape and size and (b) is observable by an imager that images the microfeatures;

[0247] vi. The monitoring marker is a microstructure with at least one lateral linear dimension of 300 um or less; and

[0248] vii. The monitoring marker is located inside the sample;

[0249] wherein at least one of the markers is imaged by the imager during the determination;

[0250] One configuration is an open configuration, in which the two plates are partially or fully separated, the spacing between the plates is not adjusted by spacers, and the sample is deposited on one or both plates;

[0251] The other configuration is a closed configuration, which is configured after the sample is deposited in the open configuration and the plates are forced to reach the closed configuration by applying an imprecise pressing force on the stressed area; and in the closed configuration: at least a part of the sample is compressed by the two plates into a layer with a very uniform thickness and is substantially stationary relative to the plates, wherein the uniform thickness of the layer is defined by the sample contact area of the two plates and is adjusted by the plates and spacers; and

[0252] wherein the monitoring marker is (i) a structure different from the spacer, or (ii) the same structure used as the spacer.

[0253] Improvement of image capture using a sample holder with micro - markers

[0254] BB - 1. An apparatus for improving the image capture of micro - features in a sample, the apparatus comprising:

[0255] (c) An apparatus as in any of the preceding apparatus embodiments; and

[0256] (d) An imager for determining a sample containing or suspected of containing micro - features;

[0257] wherein the imager captures an image, and at least one image contains a part of the sample and the monitoring marker both.

[0258] CB - 2. A system for improving the image capture of micro - features in a sample, the system comprising:

[0259] (a) An apparatus as in any of the preceding apparatus embodiments;

[0260] (b) An imager for determining a sample containing or suspected of containing micro - features; and

[0261] (c) An algorithm that uses the marker as a parameter to adjust the settings of the imager for the next image together with an imaging processing method.

[0262] C. Imaging analysis using a sample holder with micro - markers

[0263] CC-1 An apparatus for improving the analysis of an image of microfeatures in a sample, the apparatus comprising:

[0264] (a) An apparatus as in any of the preceding apparatus embodiments;

[0265] (b) A computing device configured to receive an image of a sample and a marker that contains or is suspected of containing microfeatures; and

[0266] wherein the computing device runs an algorithm that uses the marker as a parameter together with an imaging processing method to improve the image quality in the image.

[0267] CC-2. A system for improving the analysis of an image of microfeatures in a sample, the system comprising:

[0268] (a) An apparatus as in any of the preceding apparatus embodiments;

[0269] (b) An imager configured to determine a sample that contains or is suspected of containing microfeatures by taking one or more images of the sample and the marker; and

[0270] (c) An algorithm that uses the marker as a parameter together with an imaging processing method to improve the image quality of at least one image taken in (c).

[0271] CC-3 A computer program product for determining microfeatures in a sample, the program comprising computer program code that is applied to and adapted for at least one image:

[0272] (a) Receiving an image of a sample and a monitoring marker, wherein the sample is loaded into the apparatus of any of the preceding apparatus claims, and wherein the image is taken by an imager; and

[0273] (b) Processing and analyzing the image to calculate the amount of the microfeatures, wherein the analysis uses a machine learning-based detection model and information provided by the image of the monitoring marker.

[0274] CC-4 A computing device for analyzing microfeatures in a sample, the computing device comprising a computing device that operates the algorithm in any of the embodiments of the present invention.

[0275] CC-5 A method, apparatus, computer program product, or system as in any of the preceding embodiments, wherein the improvement of the image quality comprises at least one selected from the group consisting of: denoising, image normalization, image sharpening, image scaling, alignment, super-resolution, deblurring, and any combination thereof.

[0276] CC-6 is a method, apparatus, computer program product, or system as in any of the foregoing embodiments, wherein the imaging processing method includes at least one selected from the group consisting of: histogram-based operations, mathematics-based operations, convolution-based operations, smoothing operations, derivative-based operations, morphology-based operations, shadow correction, image enhancement and / or restoration, segmentation, feature extraction and / or matching, object detection and / or classification and / or localization, image understanding, and any combination thereof.

[0277] CC-6.1 is a method, apparatus, computer program product, or system as in any of the foregoing embodiments, wherein the histogram-based operations include at least one selected from the group consisting of: contrast stretching, equalization, minimum filtering, median filtering, maximum filtering, and any combination thereof.

[0278] CC-6.2 is a method, apparatus, computer program product, or system as in any of the foregoing embodiments, wherein the mathematics-based operations include at least one selected from the group consisting of: binary operations (e.g., NOT, OR, AND, XOR, and SUB), arithmetic-based operations (e.g., ADD, SUB, MUL, DIV, LOG, EXP, SQRT, TRIG, and INVERT), and any combination thereof.

[0279] CC-6.3 is a method, apparatus, computer program product, or system as in any of the foregoing embodiments, wherein the convolution-based operations include at least one selected from the group consisting of: operations in the spatial domain, Fourier transform, DCT, integer transform, operations in the frequency domain, and any combination thereof.

[0280] CC-6.4 is a method, apparatus, computer program product, or system as in any of the foregoing embodiments, wherein the smoothing operations include at least one selected from the group consisting of: linear filtering, uniform filtering, triangular filtering, Gaussian filtering, non-linear filtering, median filtering, kuwahara filtering, and any combination thereof.

[0281] CC-6.5 is a method, apparatus, computer program product, or system as in any of the foregoing embodiments, wherein the derivative-based operations are selected from at least one of the group consisting of: first-order derivative operations, gradient filtering, basic derivative filtering, Prewitt gradient filtering, Sobel gradient filtering, alternative gradient filtering, Gaussian gradient filtering, second-order derivative filtering, basic second-order derivative filtering, frequency domain Laplacian, Gaussian second-order derivative filtering, alternative Laplacian filtering, second-order derivative of gradient direction (SDGD) filtering, third-order derivative filtering, higher-order derivative filtering (e.g., filtering greater than the third-order derivative), and any combination thereof.

[0282] CC-6.6 A method, apparatus, computer program product, or system as in any of the foregoing embodiments, wherein the morphology-based operation comprises at least one selected from the group consisting of: dilation, erosion, Boolean convolution, opening and / or closing, hit-or-miss operations, contour, skeleton, propagation, gray-value morphology processing, gray-level dilation, gray-level erosion, gray-level opening, gray-level closing, morphological smoothing, morphological gradient, morphological Laplacian, and any combination thereof.

[0283] CC-6.7 A method, apparatus, computer program product, or system as in any of the foregoing embodiments, wherein the image enhancement and / or restoration comprises at least one selected from the group consisting of: sharpening, de-sharpening, noise suppression, distortion suppression, and any combination thereof.

[0284] CC-6.8 A method, apparatus, computer program product, or system as in any of the foregoing embodiments, wherein the segmentation comprises at least one selected from the group consisting of: thresholding, fixed thresholding, histogram-derived thresholding, Isodata algorithm, background symmetry algorithm, triangle algorithm, edge finding, gradient-based process, zero-crossing-based process, PLUS-based process, binary mathematical morphology, salt-and-pepper filtering, separating objects with holes, filling holes in objects, removing boundary-touching objects, outer skeleton, touching objects, gray-value mathematical morphology, Top-hat transform, thresholding, local contrast stretching, and any combination thereof.

[0285] CC-6.9 A method, apparatus, computer program product, or system as in any of the foregoing embodiments, wherein the feature extraction and / or matching comprises at least one selected from the group consisting of: independent component analysis, isometric mapping, kernel principal component analysis, latent semantic analysis, partial least squares, principal component analysis, multi-factor dimensionality reduction, non-linear dimensionality reduction, multilinear principal component analysis, multilinear subspace learning, semidefinite embedding, autoencoder, and any combination thereof.

[0286] A. Sample holder with micro-markings

[0287] Single board

[0288] AA-1.1 An apparatus for determining micro-features in a sample using an imager, the apparatus comprising:

[0289] (a) A solid-phase surface that includes a sample contact area for contacting a thin sample having a thickness of 200 um or less and that includes or is suspected of including micro-features; and

[0290] (b) One or more markings, wherein the markings:

[0291] i. having a sharp edge, which (i) has a predetermined and known shape and size, and (ii) is observable by an imager for imaging microfeatures;

[0292] ii. being a microstructure with at least one transverse linear dimension of 300 um or less; and

[0293] iii. being located inside the sample; and

[0294] wherein at least one of the markers is imaged by the imager during the determination.

[0295] AA-1.2 An apparatus for determining microfeatures in a sample using an imager, the apparatus comprising:

[0296] (a) a solid-phase surface, which comprises a sample contact area for contacting a thin sample with a thickness of 200 um or less and comprises or is suspected of including microfeatures; and

[0297] (b) one or more markers, wherein the markers:

[0298] i. comprise protrusions or grooves from the solid-phase surface

[0299] ii. have a sharp edge, which (i) has a predetermined and known shape and size, and (ii) is observable by an imager for imaging microfeatures;

[0300] iii. are microstructures with at least one transverse linear dimension of 300 um or less; and

[0301] iv. are located inside the sample;

[0302] wherein at least one of the markers is imaged by the imager during the determination.

[0303] Two plates with a constant spacing

[0304] AA-2.1 An apparatus for determining microfeatures in a sample using an imager, the apparatus comprising:

[0305] a first plate, a second plate and one or more monitoring markers, wherein:

[0306] i. each of the first plate and the second plate comprises an inner surface, which comprises a sample contact area for contacting a sample comprising or suspected of comprising microfeatures;

[0307] ii. at least a part of the sample is defined by the first and second plates into a thin layer with a substantially constant thickness of 200 um or less;

[0308] iii. The monitoring marker has a sharp edge, which (a) has a predetermined and known shape and size, and (b) can be observed by an imager for imaging microfeatures;

[0309] iv. The monitoring marker is a microstructure with at least one transverse linear dimension of 300 um or less; and

[0310] v. The monitoring marker is located inside the sample;

[0311] wherein at least one of the markers is imaged by the imager during the assay.

[0312] Two movable plates

[0313] AA-3 An apparatus for assaying microfeatures in a sample using an imager, the apparatus comprising:

[0314] A first plate, a second plate, a spacer, and one or more monitoring markers, wherein:

[0315] I. The first plate and the second plate are movable relative to each other into different configurations;

[0316] i. Each of the first plate and the second plate includes an inner surface that includes a sample contact area for contacting a sample that contains or is suspected of containing microfeatures;

[0317] ii. One or both of the first plate and the second plate include spacers permanently fixed to the inner surface of the respective plate,

[0318] iii. The monitoring marker has a sharp edge, which (a) has a predetermined and known shape and size, and (b) can be observed by an imager for imaging microfeatures;

[0319] iv. The monitoring marker is a microstructure with at least one transverse linear dimension of 300 um or less; and

[0320] v. The monitoring marker is located inside the sample;

[0321] wherein at least one of the markers is imaged by the imager during the assay.

[0322] One of the configurations is an open configuration, in which: the two plates are partially or fully separated, the spacing between the plates is not adjusted by the spacer, and the sample is deposited on one or both of the plates;

[0323] Another in the configuration is a closed configuration, which is configured after the sample is deposited in the open configuration and the plate is forced to reach the closed configuration by applying an imprecise pressing force on the stressed area; and in the closed configuration: at least a part of the sample is compressed by two plates into a layer with a very uniform thickness and is substantially stationary relative to the plates, wherein the uniform thickness of the layer is defined by the sample contact areas of the two plates and is adjusted by the plates and the spacers; and

[0324] wherein the monitoring mark is (a) a structure different from the spacer, or (b) the same structure used as the spacer.

[0325] B. Improvement of imaging using a sample holder with micro marks

[0326] BB-1. An apparatus for improving the imaging of micro features in a sample, the apparatus comprising:

[0327] (e) an apparatus as in any of the foregoing apparatus embodiments; and

[0328] (f) an imager for determining a sample containing or suspected of containing micro features; and

[0329] wherein the imager captures an image, wherein at least one image includes a part of the sample and the monitoring both.

[0330] CB-2. A system for improving the imaging of micro features in a sample, the system comprising:

[0331] i. an apparatus as in any of the foregoing apparatus embodiments;

[0332] ii. an imager for determining a sample containing or suspected of containing micro features; and

[0333] iii. a non-transitory computer-readable medium having instructions that, when executed, use the mark as a parameter to adjust the settings of the imager for the next image together with an imaging processing method.

[0334] C. Imaging analysis using a sample holder with micro marks

[0335] CC-1 An apparatus for improving the analysis of an image of micro features in a sample, the apparatus comprising:

[0336] a) an apparatus as in any of the foregoing apparatus embodiments; and

[0337] b) a computing device for receiving an image of a mark and a sample containing or suspected of containing micro features;

[0338] wherein the computing device runs an algorithm that uses the mark as a parameter together with an imaging processing method to improve the image quality in the image.

[0339] CC-2. A system for improving the analysis of an image of microfeatures in a sample, the system comprising:

[0340] (a) An apparatus as in any of the preceding apparatus embodiments;

[0341] (b) An imager for determining a sample containing or suspected of containing microfeatures by taking one or more images of the sample and a marker; and

[0342] (c) A non-transitory computer-readable medium having instructions which, when executed, improve the image quality of at least one image taken in (c) using the marker as a parameter together with an imaging processing method.

[0343] CC-3 A computer program product for determining microfeatures in a sample, the program comprising computer program code means for applying to and adapted for at least one image:

[0344] (a) Receiving an image of the sample and a monitoring marker, wherein the sample is loaded into the apparatus of any of the preceding apparatus claims, and wherein the image is taken by an imager; and

[0345] (b) Processing and analyzing the image to calculate the amount of the microfeatures, wherein the analysis uses a machine learning-based detection model and information provided by the image of the monitoring marker.

[0346] CC-4 A computing device for analyzing microfeatures in a sample, the computing device comprising a computing device operating an algorithm in any embodiment of the present invention.

[0347] CC-5 A method, apparatus, computer program product or system as in any of the preceding embodiments, wherein the improvement of the image quality comprises at least one selected from the group consisting of: denoising, image normalization, image sharpening, image scaling, alignment (e.g., for face detection), super-resolution, deblurring, and any combination thereof.

[0348] CC-6 A method, apparatus, computer program product or system as in any of the preceding embodiments, wherein the imaging processing method comprises at least one selected from the group consisting of: histogram-based operations, mathematical-based operations, convolution-based operations, smoothing operations, derivative-based operations, morphology-based operations, shadow correction, image enhancement and / or restoration, segmentation, feature extraction and / or matching, object detection and / or classification and / or localization, image understanding, and any combination thereof.

[0349] CC-6.1 A method, apparatus, computer program product, or system as in any of the foregoing embodiments, wherein the histogram-based operation includes at least one selected from the group consisting of: contrast stretching, equalization, minimum filtering, median filtering, maximum filtering, and any combination thereof.

[0350] CC-6.2 A method, apparatus, computer program product, or system as in any of the foregoing embodiments, wherein the mathematics-based operation includes at least one selected from the group consisting of: binary operations (e.g., NOT, OR, AND, XOR, and SUB), arithmetic-based operations (e.g., ADD, SUB, MUL, DIV, LOG, EXP, SQRT, TRIG, and INVERT), and any combination thereof.

[0351] CC-6.3 A method, apparatus, computer program product, or system as in any of the foregoing embodiments, wherein the convolution-based operation includes at least one selected from the group consisting of: operations in the spatial domain, Fourier transform, DCT, integer transform, operations in the frequency domain, and any combination thereof.

[0352] CC-6.4 A method, apparatus, computer program product, or system as in any of the foregoing embodiments, wherein the smoothing operation includes at least one selected from the group consisting of: linear filtering, uniform filtering, triangular filtering, Gaussian filtering, non-linear filtering, median filtering, kuwahara-filtering, and any combination thereof.

[0353] CC-6.5 A method, apparatus, computer program product, or system as in any of the foregoing embodiments, wherein the derivative-based operation is selected from at least one of the group consisting of: first-order derivative operation, gradient filtering, basic derivative filtering, Prewitt gradient filtering, Sobel gradient filtering, alternative gradient filtering, Gaussian gradient filtering, second-order derivative filtering, basic second-order derivative filtering, frequency-domain Laplacian, Gaussian second-order derivative filtering, alternative Laplacian filtering, second-order derivative of gradient direction (SDGD) filtering, third-order derivative filtering, higher-order derivative filtering (e.g., filtering greater than the third-order derivative), and any combination thereof.

[0354] CC-6.6 A method, apparatus, computer program product, or system as in any of the foregoing embodiments, wherein the morphology-based operation includes at least one selected from the group consisting of: dilation, erosion, Boolean convolution, opening and / or closing, hit-or-miss operation, contour, skeleton, propagation, gray-value morphology, gray-level dilation, gray-level erosion, gray-level opening, gray-level closing, morphological smoothing, morphological gradient, morphological Laplacian, and any combination thereof.

[0355] CC-6.7 A method, apparatus, computer program product, or system as in any of the foregoing embodiments, wherein the image enhancement and / or restoration includes at least one selected from the group consisting of sharpening, noise suppression, distortion suppression, and any combination thereof.

[0356] CC-6.8 A method, apparatus, computer program product, or system as in any of the foregoing embodiments, wherein the segmentation includes at least one selected from the group consisting of thresholding, fixed thresholding, histogram-derived thresholding, ISO data algorithm, background symmetry algorithm, triangle algorithm, edge finding, gradient-based process, zero-crossing-based process, PLUS-based process, binary mathematical morphology, salt-and-pepper filtering, separating objects with holes, filling holes in objects, removing boundary-touching objects, outer skeleton, touching objects, gray-value mathematical morphology, Top-hat transform, thresholding, local contrast stretching, and any combination thereof.

[0357] CC-6.9 A method, apparatus, computer program product, or system as in any of the foregoing embodiments, wherein the feature extraction and / or matching includes at least one selected from the group consisting of independent component analysis, isometric mapping, kernel principal component analysis, latent semantic analysis, partial least squares, principal component analysis, multi-factor dimensionality reduction, non-linear dimensionality reduction, multilinear principal component analysis, multilinear subspace learning, semidefinite embedding, autoencoder, and any combination thereof.

[0358] T1. A method for determining a true lateral dimension (TLD) of a sample on a sample holder from a distorted image, the method comprising:

[0359] (a) Obtaining an apparatus as in any of the foregoing embodiments, wherein the apparatus includes one or more monitoring marks in the sample contact area;

[0360] (b) Obtaining an imager, computing hardware, and a non-transitory computer-readable medium containing an algorithm;

[0361] (c) Depositing a thin sample layer containing microfeatures in the sample contact area of the apparatus of (a);

[0362] (d) Using the imager to take one or more images of the thin sample layer and the monitoring marks, wherein the imager is located above the thin sample layer; and

[0363] (e) Using the algorithm to determine the true lateral dimension of the sample;

[0364] wherein

[0365] (i) the algorithm is computer code executed on a computer system; and

[0366] (ii) the algorithm uses the image of the monitoring marks as a parameter.

[0367] T2. A method, apparatus, computer program product, or system as in any of the foregoing embodiments, wherein each monitoring marker comprises a protrusion or groove from a solid surface.

[0368] T3. A method, apparatus, computer program product, or system as in any of the foregoing embodiments, wherein the microstructure does not have sharp edges.

[0369] T4. A method, apparatus, computer program product, or system as in any of the foregoing embodiments, wherein the sample comprises a group selected from the group consisting of: biological samples, chemical samples, and samples without sharp edges.

[0370] T5. A method, apparatus, computer program product, or system as in any of the foregoing embodiments, wherein the monitoring marker is used as a parameter in an algorithm together with an imaging processing method, which (i) adjusts the image, (ii) processes the image of the sample, (iii) determines the properties related to the microfeatures, or (iv) any combination of the above.

[0371] T6. A method, apparatus, computer program product, or system as in any of the foregoing embodiments, wherein the spacer has a substantially uniform height equal to or less than 200 microns and a fixed spacer distance (ISD);

[0372] AA. A method and apparatus for improving the imaging of a thin layer of a sample.

[0373] AA1. A method for improving the imaging of a thin layer of a sample, the method comprising:

[0374] (a) obtaining a labeled sample holder, wherein the sample comprises one or more monitoring markers in the sample contact area;

[0375] (b) obtaining an imager, computing hardware, and a non - transitory computer - readable medium comprising an algorithm;

[0376] (c) depositing a thin sample layer comprising microfeatures in the sample contact area of the apparatus of (a);

[0377] (d) using the imager to take one or more images of the thin sample layer and the monitoring markers, wherein the imager is located above the thin sample layer; and

[0378] (e) using the algorithm to determine the true lateral dimensions of the sample;

[0379] wherein:

[0380] (i) the algorithm is computer code executed on a computer system; and

[0381] (ii) the algorithm uses the image of the monitoring marker as a parameter.

[0382] A-1

[0383] T1. A method for determining the true lateral dimension (TLD) of a sample on a sample holder from a distorted image, the method comprising:

[0384] i. Obtaining a device as in any of the foregoing embodiments, wherein the device includes one or more monitoring marks in the sample contact area;

[0385] ii. Obtaining an imager, computing hardware, and a non-transitory computer-readable medium containing an algorithm;

[0386] iii. Depositing a thin sample layer containing microfeatures in the sample contact area of the device of (a);

[0387] iv. Using the imager to take one or more images of the thin sample layer and the monitoring marks, wherein the imager is located above the thin sample layer; and

[0388] v. Using the algorithm to determine the true lateral dimension of the sample;

[0389] wherein

[0390] (a) the algorithm is computer code executed on a computer system; and

[0391] (b) the algorithm uses the image of the monitoring mark as a parameter.

[0392] NN1. A device for determining microfeatures in a sample using an imager, the device comprising:

[0393] (a) a solid-phase surface including a sample contact area for contacting a sample containing microfeatures; and

[0394] (b) one or more monitoring marks, wherein the monitoring marks:

[0395] i. are made of a material different from the sample;

[0396] ii. are located inside the sample during the determination of the microstructure, wherein the sample forms a thin layer with a thickness less than 200 um on the sample contact area;

[0397] iii. have a lateral linear dimension of about 1 um (micrometer) or greater, and

[0398] iv. have at least one lateral linear dimension of 300 um or less; and

[0399] wherein during the determination process, the imager images at least one monitoring mark

[0400] which is used during the determination of the analyte; and monitoring the geometric parameters (e.g., shape and size) of the markers and / or monitoring the spacing between the markers is (a) predetermined and known prior to the analyte determination, and (b) used as a parameter in an algorithm for determining properties associated with the microfeatures.

[0401] NN2. An apparatus for using an imager to determine microfeatures in a sample, the apparatus comprising:

[0402] A solid-phase surface comprising a sample contact region for contacting a sample containing microfeatures; and

[0403] One or more monitoring markers, each monitoring marker comprising a protrusion or groove from the solid-phase surface, wherein:

[0404] (a) The protrusion or groove includes a flat surface substantially parallel to an adjacent surface, which is the portion of the solid-phase surface adjacent to the protrusion or groove;

[0405] (b) The distance between the flat surface and the adjacent surface is about 200 micrometers (μm) or less;

[0406] (c) The flat surface has (a) a linear dimension of at least about 1 μm or greater, and (b) a region with at least one linear dimension of 150 μm or less;

[0407] (d) The flat surface of at least one monitoring marker is imaged by an imager used in determining the microfeatures; and

[0408] (e) The shape of the flat surface, the size of the flat surface, the distance between the flat surface and the adjacent surface, and / or the spacing between the monitoring markers (a) are predetermined and known prior to the determination of the microfeatures, and (b) are used as parameters in an algorithm for determining properties associated with the microfeatures.

[0409] NN3. An apparatus for using an imager to determine microfeatures in a sample, the apparatus comprising:

[0410] A first plate, a second plate, a spacer, and one or more monitoring markers, wherein:

[0411] (a) The first plate and the second plate are movable relative to each other into different configurations;

[0412] (b) Each of the first plate and the second plate includes an inner surface comprising a sample contact region for contacting a sample containing microfeatures;

[0413] (c) One or both of the first plate and the second plate include spacers permanently fixed to the inner surface of the respective plate,

[0414] (d) The spacer has a substantially uniform height equal to or less than 200 micrometers and a fixed spacer distance (ISD);

[0415] (e) The monitoring marker is made of a material different from that of the sample;

[0416] (f) During the determination of the microstructure, the monitoring marker is located inside the sample, where the sample forms a thin layer with a thickness less than 200 um on the sample contact area; and

[0417] (g) The monitoring marker has a lateral linear dimension of about 1 um (micrometer) or greater and has at least one lateral linear dimension of 300 um or less;

[0418] Wherein during the determination process, the imager images at least one monitoring marker

[0419] Wherein it is used during the determination of microfeatures; and the shape, size, distance between the flat surface and the adjacent surface, and / or the spacing between the monitoring markers of the flat surface (a) are predetermined and known before the determination of the microfeatures, and (b) are used as parameters in the algorithm for determining the characteristics related to the microfeatures;

[0420] Wherein one configuration is an open configuration, in which: the two plates are partially or completely separated, the spacing between the plates is not adjusted by the spacer, and the sample is deposited on one or both of the plates;

[0421] Wherein the other of the configurations is a closed configuration, which is configured after the sample is deposited in the open configuration and the plates are forced to reach the closed configuration by applying an imprecise pressing force on the force application area; and in the closed configuration: at least a part of the sample is compressed by the two plates into a layer with a very uniform thickness and is substantially stationary relative to the plates, wherein the uniform thickness of the layer is defined by the sample contact areas of the two plates and is adjusted by the plates and the spacer; and

[0422] Wherein the monitoring marker is (i) a structure different from the spacer, or (ii) the same structure as the spacer used.

[0423] NN4. An apparatus for determining microfeatures in a sample using an imager, the apparatus comprising:

[0424] A first plate, a second plate, a spacer, and one or more monitoring markers, wherein:

[0425] (a) The first plate and the second plate are movable relative to each other into different configurations;

[0426] (b) Each of the first plate and the second plate includes an inner surface, and the inner surface includes a sample contact area for contacting a sample containing microfeatures;

[0427] (c) One or both of the first plate and the second plate include spacers permanently fixed to the inner surfaces of the respective plates,

[0428] (d) The spacers have a substantially uniform height and a fixed spacing distance (ISD) equal to or less than 200 micrometers;

[0429] (e) Each monitoring mark includes a protrusion or a groove on one or both of the sample contact areas;

[0430] (f) The protrusion or the groove includes a flat surface substantially parallel to an adjacent surface, which is the part of the solid phase surface adjacent to the protrusion or the groove;

[0431] (g) The distance between the flat surface and the adjacent surface is about 200 micrometers (μm) or less;

[0432] (h) The flat surface has (a) a linear dimension of at least about 1 μm or greater, and (b) a region with at least one linear dimension of 150 μm or less;

[0433] (i) The flat surface of at least one monitoring mark is imaged by an imager used in determining microfeatures; and

[0434] (j) The shape of the flat surface, the size of the flat surface, the distance between the flat surface and the adjacent surface, and / or the spacing between the monitoring marks (a) are predetermined and known before the determination of the microfeatures, and (b) are used as parameters in an algorithm for determining the characteristics related to the microfeatures.

[0435] a. One of the configurations is an open configuration, in which: the two plates are partially or completely separated, the spacing between the plates is not adjusted by the spacers, and the sample is deposited on one or both of the plates;

[0436] The other of the configurations is a closed configuration, which is configured after the sample is deposited in the open configuration and the plates are forced to reach the closed configuration by applying an imprecise pressing force on the stress area; and in the closed configuration: at least a part of the sample is compressed by the two plates into a layer with a very uniform thickness and is substantially stagnant relative to the plates, where the uniform thickness of the layer is defined by the sample contact areas of the two plates and is adjusted by the plates and the spacers; and

[0437] The monitoring marks are (i) structures different from the spacers, or (ii) the same structures used as spacers.

[0438] NN5. A device for imaging-based determination, comprising:

[0439] A device as in any of the previous device embodiments, wherein the device has at least four monitoring marks that are not aligned on a linear line.

[0440] NN6. An apparatus for determining microfeatures in a sample using an imager, the system comprising:

[0441] (a) An apparatus as in any of the preceding apparatus embodiments; and

[0442] (b) An imager for determining a sample containing microfeatures.

[0443] NN7. A system for performing an imaging-based determination, the system comprising:

[0444] (a) An apparatus as in any of the preceding apparatus embodiments;

[0445] (b) An imager for determining a sample containing microfeatures; and

[0446] (c) A non-transitory computer-readable medium containing instructions that, when executed, utilize monitoring marks of the apparatus to determine characteristics related to the microfeatures.

[0447] NN8. A system for determining microfeatures in a sample using an imager, the system comprising:

[0448] (a) An apparatus as in any of the preceding apparatus embodiments;

[0449] (b) An imager for determining a sample containing microfeatures; and

[0450] (c) A non-transitory computer-readable medium containing instructions that, when executed, utilize monitoring marks of the apparatus to determine characteristics related to the microfeatures, wherein the instructions include machine learning.

[0451] NN9. A method for determining microfeatures in a sample using an imager, comprising:

[0452] (a) Obtaining an apparatus, device or system as in any of the preceding embodiments;

[0453] (b) Obtaining a sample and depositing the sample on a sample contact area of the apparatus, device or system in (a), wherein the sample contains microfeatures; and

[0454] (c) Determining the microfeatures.

[0455] NN10. A method for determining microfeatures in a sample using an imager, comprising:

[0456] (a) Obtaining an apparatus, device or system as in any of the preceding embodiments;

[0457] (b) Obtaining a sample and depositing the sample on a sample contact area of the apparatus, device or system in (a), wherein the sample contains microfeatures;

[0458] (c) Measure the micro-features, where the measurement includes the step of using machine learning.

[0459] T1. A method for determining the true lateral dimension (TLD) of a sample on a sample holder from a distorted image, the method comprising:

[0460] (a) Obtain a device as in any of the foregoing embodiments, where the device includes one or more monitoring marks in the sample contact area;

[0461] (b) Obtain an imager, computing hardware, and a non-transitory computer-readable medium containing an algorithm;

[0462] (c) Deposit a thin sample layer containing micro-features in the sample contact area of the device in (a);

[0463] (d) Use the imager to take one or more images of the thin sample layer and the monitoring marks, where the imager is located above the thin sample layer; and

[0464] (e) Use the algorithm to determine the true lateral dimension of the sample;

[0465] where

[0466] (i) The algorithm is computer code executed on a computer system; and

[0467] (ii) The algorithm uses the images of the monitoring marks as parameters.

[0468] T2. A method for determining the true lateral dimension (TLD) of a sample on a sample holder from a distorted image, the method comprising:

[0469] (a) Obtain a device as in any of the foregoing embodiments, where the device includes one or more monitoring marks in the sample contact area;

[0470] (b) Obtain an imager, computing hardware, and a non-transitory computer-readable medium containing an algorithm;

[0471] (c) Deposit a thin sample layer containing micro-features in the sample contact area of the device in (a);

[0472] (d) Use the imager to take one or more images of the thin sample layer and the monitoring marks, where the imager is located above the thin sample layer; and

[0473] (e) Use the algorithm to determine the true lateral dimension and coordinates of the imaged sample in the real world by physical metrics (e.g., microns);

[0474] where

[0475] (i) The algorithm is computer code executed on a computer system; and

[0476] (ii) The algorithm uses the monitored marked image as a parameter.

[0477] T3. An apparatus, method or system as in any of the preceding embodiments, wherein the microfeatures and the monitored markers from the sample are disposed within the sample holding device.

[0478] T4. An apparatus, method or system as in any of the preceding embodiments, wherein the determination comprises detecting and locating the monitored marker in an image of the sample taken by the imager.

[0479] T5. An apparatus, method or system as in any of the preceding embodiments, wherein the determination comprises generating a monitored marker grid based on the monitored marker detected from an image of the sample taken by the imager.

[0480] T6. An apparatus, method or system as in any of the preceding embodiments, wherein the determination comprises calculating a homography transformation from the generated monitored marker grid.

[0481] T7. An apparatus, method or system as in any of the preceding embodiments, wherein the determination comprises estimating the TLD from the homography transformation and determining the area, size and concentration of the microfeatures detected in the image-based determination.

[0482] T8. A method, apparatus or system as in any of the preceding embodiments, wherein the TLD estimation is based on a region in a sample image taken by the imager, comprising:

[0483] (a) Obtaining a sample;

[0484] (b) Loading the sample into a sample holding device, such as a QMAX device, wherein there are monitored markers, wherein the monitored markers are not submerged in the sample and are located in the device, which can be imaged from the top by an imager in the image-based determination;

[0485] (c) Taking an image of the sample in the sample loading device including the microfeatures and the monitored markers;

[0486] (d) Detecting the monitored markers in the sample image taken by the imager;

[0487] (e) Dividing the sample image into non-overlapping regions;

[0488] (f) Generating a region-based marker grid for each non-overlapping region, wherein more than 4 non-collinear monitored markers are detected in the local region;

[0489] (g) Generate a marker grid for all other regions not in (f) based on the monitoring markers detected from the sample image captured by the imager;

[0490] (h) Calculate a region-specific homography transformation for each region in (f) based on its own region-based marker grid generated by (f);

[0491] (i) Calculate the homography transformation for all other regions not in (f) based on the marker grid generated in (g);

[0492] (j) Estimate the region-based TLD for each region in (f) based on the region-based homography transformation of (g);

[0493] (k) Estimate the TLD for other regions not included in (f) based on the homography transformation of (i); and

[0494] (l) Apply the estimated TLDs from (j) and (k) to determine the area and concentration of the microfeatures imaged in each image partition in the image-based assay.

[0495] T9. The method, apparatus or system of any of the preceding embodiments, wherein the monitoring markers in the sample holding device are distributed according to a periodic pattern having a defined pitch period.

[0496] T10. The method, apparatus or system of any of the preceding embodiments, wherein the monitoring markers are detected and used as detectable anchors for calibrating and improving the measurement accuracy in the image-based assay.

[0497] T11. The method, apparatus or system of any of the preceding embodiments, wherein the detection of the monitoring markers in the sample image captured by the imager utilizes the periodicity of the distribution of the monitoring markers in the sample holding device for error correction and / or detection reliability.

[0498] T12. The method, apparatus or system of any of the preceding embodiments, wherein the detection, identification, area and / or shape contour estimation of the monitoring markers in the image-based assay is performed by machine learning (ML) having an ML-based monitoring marker detection model and a device constructed or trained from the images captured by the imager on the device in the image-based assay.

[0499] T13. The method, apparatus or system of any of the preceding embodiments, wherein the detection, identification, area and / or shape contour estimation of the monitoring markers in the image-based assay is performed by image processing or image processing combined with machine learning.

[0500] T14. A method, apparatus, or system as in any of the foregoing embodiments, wherein the detected monitoring marker is applied to TLD estimation in an image-based assay to calibrate the system and / or improve measurement accuracy in the imaging-based assay.

[0501] T15. A method, apparatus, or system as in any of the foregoing embodiments, wherein the detected monitoring marker is applied in an image-based assay not limited to microfeature size, volume, and / or concentration estimation to calibrate the system and / or improve measurement accuracy.

[0502] T16. A method, apparatus, or system as in any of the foregoing embodiments, wherein the detection of the monitoring marker and / or TLD estimation is applied to fault detection in an image-based assay, including but not limited to detecting defects in the sample holding device, misalignment of the sample holding device in the imager, and / or focusing faults of the imager.

[0503] T17. A method, apparatus, or system as in any of the foregoing embodiments, wherein the monitoring marker is detected as an anchor for application to the system to estimate the area of an object in an image-based assay, including:

[0504] i. Loading the sample into a sample holding device having a monitoring marker located in the device in an image-based assay;

[0505] ii. Taking an image of the sample in the sample holding device including microfeatures and the monitoring marker; and

[0506] iii. Detecting the monitoring marker in the image of the sample taken by the imager on the sample holding device, determining the TLD, and calculating an area estimate in the image-based assay to determine the size of the imaged object from pixels in the image to its physical size in microns in the real world.

[0507] T18. A method, apparatus, or system as in any of the foregoing embodiments, wherein the system includes:

[0508] i. Detecting a monitoring marker in a digital image;

[0509] ii. Generating a monitoring marker grid;

[0510] iii. Calculating an image transformation based on the monitoring marker grid; and

[0511] iv. Estimating the area of an object in the image of the sample and its physical size in the real world in an image-based assay.

[0512] T19. A method, apparatus, or system as in any of the foregoing embodiments, wherein a homography transformation is calculated using a monitoring marker grid generated from detected monitoring markers to estimate the TLD, the area of an object in an image of a sample taken by an imager, and the physical dimensions of the object in the real world.

[0513] T20. A method, apparatus, or system as in any of the foregoing embodiments, wherein the method comprises:

[0514] i. Dividing an image of a sample taken by an imager in an image-based determination into non-overlapping regions;

[0515] ii. Detecting local monitoring markers in the image;

[0516] iii. If more than 4 non-collinear monitoring markers are detected in the region, generating a region-based marker grid for the region;

[0517] iv. Generating a marker grid for all other regions based on the monitoring markers detected in the image of the sample taken by the imager;

[0518] v. Calculating a region-based homography transformation from the region-based marker grid generated for each region in (iii);

[0519] vi. Calculating the homography transformation for all other regions not in (iii) based on the marker grid generated in (iv); and

[0520] vii. Estimating the TLD for each region based on the homography transformations resulting from (v) and (vi), and then determining the area of the object in the image of the sample for each region in the image-based determination and their dimensions in the real world.

[0521] T21. A method, apparatus, or system as in any of the foregoing embodiments, wherein the determination is a medical, diagnostic, chemical, or biological test.

[0522] T22. A method, apparatus, or system as in any of the foregoing embodiments, wherein the microfeatures are cells.

[0523] T23. A method, apparatus, or system as in any of the foregoing embodiments, wherein the microfeatures are blood cells.

[0524] T24. A method, apparatus, or system as in any of the foregoing embodiments, wherein the microfeatures are proteins, peptides, DNA, RNA, nucleic acids, small molecules, cells, or nanoparticles.

[0525] T25. A method, apparatus, or system as in any of the foregoing embodiments, wherein the microcomponents comprise tags.

[0526] T26. A method, apparatus, or system as in any of the foregoing embodiments, wherein the algorithm comprises a computer program product, the computer program product comprising computer program code means adapted in at least one image to:

[0527] (a) Receive an image of a sample, wherein the sample is loaded into a QMAX device and the image is taken by an imager connected to the QMAX device, and wherein the image includes both the sample and a monitoring marker; and

[0528] (b) Analyze the image using a detection model and generate a two-dimensional data array of the image, wherein the two-dimensional data array includes probability data of the microfeatures at each position in the image, and the detection model is established through a training process, the training process comprising:

[0529] i. Feeding an annotated data set into a convolutional neural network, wherein the annotated data set is from samples of the same type as the test sample and for the same microfeatures; and

[0530] ii. Training and establishing the detection model through convolution; and

[0531] (c) Analyze the two-dimensional data array to detect local signal peaks by:

[0532] i. A signal list process, or

[0533] ii. A local search process; and

[0534] (d) Calculate the quantity of the microfeatures based on the local signal peak information.

[0535] T27. A method, apparatus, or system as in any of the foregoing embodiments, wherein the algorithm comprises a computer program product, the computer program product comprising computer program code means applied to and adapted in at least one image to:

[0536] (a) Represent an inference pattern between an object in a sample and a pixel isogram of the object in an image of the sample taken by an imager on a sample holding device,

[0537] (b) Digitally reconstruct an image of at least one object detected from the inference pattern in the image of the sample and generate a contour mask surrounding the object identified by the inference module, wherein the object is in focus,

[0538] (c) Identify at least a part of the image of the sample for at least one object in a selected part of the sample image, and

[0539] (d) Calculate at least one feature of the object from the at least one part to identify the object in the selected part of the image of the sample taken by the imager.

[0540] (e) Calculate the count and concentration of the detection object in the selected part from the selected part of the image of the sample.

[0541] When the program runs on a computing device or in a computing cloud connected via a network.

[0542] T28. A method, apparatus or system as in any of the preceding embodiments, wherein the algorithm comprises a computer program product, the computer program product comprising computer program code means adapted in at least one image to:

[0543] (a) Receive an image of a sample, wherein the sample is loaded into a QMAX device and the image is taken by an imager connected to the QMAX device, wherein the image includes both the sample and a monitoring marker; and

[0544] (b) Analyze the image to calculate the amount of the microfeatures, wherein the analysis uses a machine learning-based detection model and information provided by the image of the monitoring marker.

[0545] A method, apparatus or system as in any of the preceding embodiments further comprises a computer-readable storage medium or memory storage unit comprising a computer program as in any of the preceding embodiments.

[0546] A method, apparatus or system as in any of the preceding embodiments further includes a computing device or a mobile device comprising a computing device as in any of the preceding embodiments.

[0547] A method, apparatus or system as in any of the preceding embodiments further includes a computing device or a mobile device comprising a computer program product as in any of the preceding embodiments.

[0548] A method, apparatus or system as in any of the preceding embodiments further includes a computing device or a mobile device comprising a computer-readable storage medium or storage unit as in any of the preceding embodiments.

[0549] A system for analyzing a sample, comprising:

[0550] A first plate, a second plate, a surface amplification layer, a capture agent, wherein

[0551] (a) The first plate and the second plate are movable relative to each other into different configurations and have sample contact areas on their respective inner surfaces for contacting a sample containing a target analyte;

[0552] (b) The surface amplification layer is on one of the sample contact areas.

[0553] (c) The capture agent is immobilized on the surface amplification layer, wherein the capture agent specifically binds the target analyte.

[0554] When the surface amplification layer is adjacent to the surface amplification layer, the surface amplification layer amplifies the optical signal from the target analyte or the label attached to the target analyte, and the optical signal is much stronger than the optical signal when the surface amplification layers are separated by micrometers or more.

[0555] One configuration is an open configuration where the average spacing between the inner surfaces of the two plates is at least 200 μm; and

[0556] Another configuration is a closed configuration where at least a portion of the sample is located between the two plates and the average spacing between the inner surfaces of the plates is less than 200 μm.

[0557] A system for analyzing a sample, comprising:

[0558] A first plate, a second plate, a surface amplification layer, a capture agent, where

[0559] (a) The first plate and the second plate are movable relative to each other into different configurations and have sample contact regions on their respective inner surfaces for contacting a sample containing a target analyte;

[0560] (b) The surface amplification layer is on one of the sample contact regions,

[0561] (c) The capture agent is immobilized on the surface amplification layer, where the capture agent specifically binds to the target analyte,

[0562] When the surface amplification layer is adjacent to the surface amplification layer, the surface amplification layer amplifies the optical signal from the label attached to the target analyte, and the optical signal is much stronger than the optical signal when the surface amplification layers are separated by micrometers or more.

[0563] One configuration is an open configuration where the average spacing between the inner surfaces of the two plates is at least 200 μm;

[0564] Another configuration is a closed configuration where at least a portion of the sample is located between the two plates and the average spacing between the inner surfaces of the plates is less than 200 μm;

[0565] Wherein the thickness of the sample in the closed configuration, the concentration of the label dissolved in the sample in the closed configuration, and the amplification factor of the surface amplification layer are configured such that any label directly or indirectly bound to the capture agent in the closed configuration is visible without washing away unbound labels.

[0566] An apparatus comprising the device of any of the preceding embodiments and a reader for reading the device.

[0567] A homogeneous assay method using the device of any of the foregoing embodiments, wherein the thickness of the sample in the closed configuration, the concentration of the labels, and the amplification factor of the amplification surface are configured such that the labels bound to the amplification surface are visible without washing away the unbound labels.

[0568] A method of any of the foregoing embodiments, wherein the method is performed by:

[0569] Obtaining a device of any of the foregoing embodiments;

[0570] Depositing a sample on one or both plates when the plates are in the open configuration;

[0571] Closing the plates to the closed configuration; and

[0572] Reading the sample contact area with a reading device to generate an image of the signal.

[0573] A device or method of any of the foregoing embodiments, wherein the labels bound to the amplification surface are visible in less than 60 seconds.

[0574] A device or method of any of the foregoing embodiments, wherein the method is a homogeneous assay, wherein the signal is read without using a washing step to remove any biomaterial or label not bound to the amplification surface.

[0575] A device or method of any of the foregoing embodiments, wherein the labels bound to the amplification surface are read by a pixelated reading method.

[0576] A device or method of any of the foregoing embodiments, wherein the labels bound to the amplification surface are read by an aggregated reading method.

[0577] A device or method of any of the foregoing embodiments, wherein the assay has a detection sensitivity of 0.1 nM or less.

[0578] A device or method of any of the foregoing embodiments, wherein unbound biomaterial or labels are removed from the amplification surface by a sponge before reading.

[0579] A device or method of any of the foregoing embodiments, wherein the signal amplification layer comprises D2PA.

[0580] A device or method of any of the foregoing embodiments, wherein the signal amplification layer comprises a metal material layer.

[0581] A device or method of any of the foregoing embodiments, wherein the signal amplification layer comprises a continuous metal film made of a material selected from the group consisting of gold, silver, copper, aluminum, alloys thereof, and combinations thereof.

[0582] The device or method of any of the foregoing embodiments, wherein different metal layers are locally enhanced or used as reflectors, or both, to enhance the optical signal.

[0583] The device or method of any of the foregoing embodiments, wherein the signal amplification layer comprises a metal material layer and a dielectric material on top of the metal material layer, and wherein the capture agent is on the dielectric material.

[0584] The device or method of any of the foregoing embodiments, wherein the metal material layer is a uniform metal layer, a nanostructured metal layer, or a combination.

[0585] The device or method of any of the foregoing embodiments, wherein the signal is amplified by plasmon enhancement.

[0586] The device or method of any of the foregoing embodiments, wherein the assay comprises detecting the tag by Raman scattering.

[0587] The device or method of any of the foregoing embodiments, wherein the capture agent is an antibody.

[0588] The device or method of any of the foregoing embodiments, wherein the capture agent is a polynucleotide.

[0589] The device or method of any of the foregoing embodiments, wherein the device further comprises a spacer fixed to a plate, and wherein the spacer adjusts the spacing between the first plate and the second plate in a closed configuration.

[0590] The device or method of any of the foregoing embodiments, wherein the amplification factor of the surface amplification layer is adjusted such that the optical signal from a single tag that is directly or indirectly bound to the capture agent is visible.

[0591] The device or method of any of the foregoing embodiments, wherein the amplification factor of the surface amplification layer is adjusted such that the optical signal from a single tag that is directly or indirectly bound to the capture agent is visible, and wherein the visible single tags bound to the capture agent are individually counted.

[0592] The device or method of any of the foregoing embodiments, wherein the spacing between the first plate and the second plate in the closed configuration is configured such that the saturation binding time of the target analyte to the capture agent is 300 seconds or less.

[0593] The device or method of any of the foregoing embodiments, wherein the spacing between the first plate and the second plate in the closed configuration is configured such that the saturation binding time of the target analyte to the capture agent is 60 seconds or less.

[0594] The device or method of any of the foregoing embodiments, wherein the amplification factor of the surface amplification layer is adjusted such that the optical signal from a single tag is visible.

[0595] The device or method of any of the foregoing embodiments, wherein the capture agent is a nucleic acid.

[0596] The device or method of any of the foregoing embodiments, wherein the capture agent is a protein.

[0597] The device or method of any of the foregoing embodiments, wherein the capture agent is an antibody.

[0598] The device or method of any of the foregoing embodiments, wherein the sample contact region of the second plate has a reagent storage site, and in the closed configuration the storage site is approximately above the binding site on the first plate.

[0599] The device or method of any of the foregoing embodiments, wherein the reagent storage site contains a detection agent that binds to the target analyte.

[0600] The device or method of any of the foregoing embodiments, wherein the detection agent contains a label.

[0601] The device or method of any of the foregoing embodiments, wherein both the capture agent and the detection agent bind to the target analyte to form a sandwich that includes a label.

[0602] The device or method of any of the foregoing embodiments, wherein the signal amplification layer comprises a metal material layer.

[0603] The device or method of any of the foregoing embodiments, wherein the signal amplification layer comprises a metal material layer and a dielectric material on top of the metal material layer, wherein the capture agent is on the dielectric material.

[0604] The device or method of any of the foregoing embodiments, wherein the metal material layer is a uniform metal layer, a nanostructured metal layer, or a combination.

[0605] The device or method of any of the foregoing embodiments, wherein the amplification layer comprises a metal material layer and a dielectric material on top of the metal material layer, wherein the capture agent is on the dielectric material, and the dielectric material layer has a thickness of 0.5 nm, 1 nm, 5 nm, 10 nm, 20 nm, 50 nm, 100 nm, 200 nm, 500 nm, 1000 nm, 2 μm, 3 μm, 5 μm, 10 μm, 20 μm, 30 μm, 50 μm, 100 μm, 200 μm, 500 μm, or in a range between any two values.

[0606] The device or method of any of the foregoing embodiments, wherein the method further comprises quantifying the signal in a region of the image to provide an estimate of the amount of one or more analytes in the sample.

[0607] The device or method of any of the foregoing embodiments, wherein the method comprises identifying and counting individual binding events between the analyte and the capture agent in the image region, thereby providing an estimate of the amount of one or more analytes in the sample.

[0608] The apparatus or method of any of the foregoing embodiments, wherein the identifying and counting steps comprise: (1) determining the local intensity of the background signal; (2) determining the local signal intensity of one label, two labels, three labels, four or more labels; and (3) determining the total number of labels within the imaging area.

[0609] The apparatus or method of any of the foregoing embodiments, wherein the identifying and counting steps comprise: (1) determining the local spectrum of the background signal; (2) determining the local signal spectrum of one label, two labels, three labels, four or more labels; and (3) determining the total number of labels within the imaging area.

[0610] The apparatus or method of any of the foregoing embodiments, wherein the identifying and counting steps comprise: (1) determining the local Raman signature of the background signal; (2) determining the local signal Raman signature of one label, two labels, three labels, four or more labels; and (3) determining the total number of labels within the imaging area.

[0611] The apparatus or method of any of the foregoing embodiments, wherein the identifying and counting steps comprise determining one or more of the local intensity, spectrum, and Raman signature.

[0612] The apparatus or method of any of the foregoing embodiments, wherein the method comprises quantifying the integrated signal in a region of the image to provide an estimate of the amount of one or more analytes in the sample.

[0613] The apparatus or method of any of the foregoing embodiments, wherein the sample contact region of the second plate has a reagent storage site, and in the closed configuration the storage site is approximately above the binding site on the first plate.

[0614] The apparatus or method of any of the foregoing embodiments, wherein the method further comprises the step of labeling the target analyte with a detection agent.

[0615] The apparatus or method of any of the foregoing embodiments, wherein the detection agent comprises a label.

[0616] The apparatus or method of any of the foregoing embodiments, wherein both the capture agent and the detection agent bind to the target analyte to form a sandwich.

[0617] The apparatus or method of any of the foregoing embodiments, wherein the method further comprises measuring the volume of the sample in the region imaged by the reading device.

[0618] The apparatus or method of any of the foregoing embodiments, wherein the target analyte is a protein, peptide, DNA, RNA, nucleic acid, small molecule, cell, or nanoparticle.

[0619] The apparatus or method of any of the foregoing embodiments, wherein the image shows the position, local intensity, and local spectrum of the signal.

[0620] An apparatus or method as in any of the preceding embodiments, wherein the signal is an optical emission signal selected from the group consisting of: fluorescence, electroluminescence, chemiluminescence, and electrochemiluminescence signals.

[0621] An apparatus or method as in any of the preceding embodiments, wherein the signal is a Raman scattering signal.

[0622] An apparatus or method as in any of the preceding embodiments, wherein the signal is a force generated by local electrical interaction, local mechanical interaction, local biological interaction, or local optical interaction between the plate and the reading device.

[0623] An apparatus or method as in any of the preceding embodiments, wherein the spacer has a columnar shape and a substantially uniform cross-section.

[0624] An apparatus or method as in any of the preceding embodiments, wherein the spacer distance (SD) is equal to or less than about 120 μm (micrometers).

[0625] An apparatus or method as in any of the preceding embodiments, wherein the spacer distance (SD) is equal to or less than about 100 μm (micrometers).

[0626] An apparatus or method as in any of the preceding embodiments, wherein the fourth power of the spacer distance (ISD) divided by the thickness (h) and Young's modulus (E) of the flexible plate (ISD 4 / (hE)) is 5×10 6 μm 3 / GPa or less.

[0627] An apparatus or method as in any of the preceding embodiments, wherein the fourth power of the spacer distance (ISD) divided by the thickness (h) and Young's modulus (E) of the flexible plate (ISD 4 / (hE)) is 5×10 5 μm 3 / GPa or less.

[0628] An apparatus or method as in any of the preceding embodiments, wherein the spacer has a columnar shape, a substantially flat top surface, a predetermined substantially uniform height, and a predetermined constant spacer distance, the spacer distance being at least about 2 times greater than the size of the analyte, wherein the Young's modulus of the spacer multiplied by the fill factor of the spacer is equal to or greater than 2 MPa, wherein the fill factor is the ratio of the spacer contact area to the total plate area, and wherein for each spacer, the ratio of the lateral dimension of the spacer to its height is at least 1 (1).

[0629] A method or apparatus as in any of the foregoing embodiments, wherein the spacer has a columnar shape, a substantially flat top surface, a predetermined substantially uniform height, and a predetermined constant spacing distance, the spacing distance being at least about 2 times greater than the size of the analyte, wherein the Young's modulus of the spacer multiplied by the fill factor of the spacer is equal to or greater than 2 MPa, where the fill factor is the ratio of the spacer contact area to the total plate area, and wherein for each spacer, the ratio of the lateral dimension of the spacer to its height is at least 1(1), where the fourth power of the spacing distance (ISD) divided by the thickness (h) and Young's modulus (E) of the flexible plate (ISD 4 / (hE)) is 5 x 10 6 um 3 / GPa or less.

[0630] A method or apparatus as in any of the foregoing embodiments, wherein the ratio of the spacing distance of the spacer to the average width of the spacer is 2 or greater, and the fill factor of the spacer multiplied by the Young's modulus of the spacer is 2 MPa or greater.

[0631] A method or apparatus as in any of the foregoing embodiments, wherein the analyte is a protein, peptide, nucleic acid, synthetic compound, or inorganic compound.

[0632] A method or apparatus as in any of the foregoing embodiments, wherein the sample is a biological sample selected from amniotic fluid, aqueous humor, vitreous humor, blood (e.g., whole blood, fractionated blood, plasma, or serum), breast milk, cerebrospinal fluid (CSF), earwax (cerumen), chyle, chyme, endolymph, perilymph, feces, breath, gastric acid, gastric juice, lymph, mucus (including nasal drainage and sputum), pericardial fluid, peritoneal fluid, pleural fluid, pus, rheumatic fluid, saliva, exhaled condensate, sebum, semen, sputum, sweat, synovial fluid, tears, vomit, and urine.

[0633] A method or apparatus as in any of the foregoing embodiments, wherein the spacer has a columnar shape, and the ratio of the width to the height of the column is equal to or greater than 1.

[0634] A method or apparatus as in any of the foregoing embodiments, wherein the sample deposited on one or both of the plates has an unknown volume.

[0635] A method or apparatus as in any of the foregoing embodiments, wherein the spacer has a columnar shape, and the column has a substantially uniform cross-section.

[0636] A method or apparatus as in any of the foregoing embodiments, wherein the sample is used to detect, purify, and quantify compounds or biomolecules associated with the stages of certain diseases.

[0637] The method or device of any of the foregoing embodiments, wherein the sample is related to infectious and parasitic diseases, injuries, cardiovascular diseases, cancers, mental disorders, neuropsychiatric disorders, lung diseases, kidney diseases, and other organic diseases.

[0638] The method or device of any of the foregoing embodiments, wherein the sample is related to the detection, purification, and quantification of microorganisms.

[0639] The method or device of any of the foregoing embodiments, wherein the sample is related to viruses, fungi, and bacteria from the environment (e.g., water, soil, or biological samples).

[0640] The method or device of any of the foregoing embodiments, wherein the sample is related to the detection and quantification of compounds or biological samples (e.g., toxic waste, anthrax) that pose a hazard to food safety or national security.

[0641] The method or device of any of the foregoing embodiments, wherein the sample is related to the quantification of vital parameters in medical or physiological monitoring.

[0642] The method or device of any of the foregoing embodiments, wherein the sample is related to glucose, blood, oxygen level, and total blood cell count.

[0643] The method or device of any of the foregoing embodiments, wherein the sample is related to the detection and quantification of specific DNA or RNA from biological samples.

[0644] The method or device of any of the foregoing embodiments, wherein the sample is related to the sequencing and comparison of genetic sequences of DNA in chromosomes and mitochondria for genomic analysis.

[0645] The method or device of any of the foregoing embodiments, wherein the sample is related to the detection of reaction products, for example, during drug synthesis or purification.

[0646] The method or device of any of the foregoing embodiments, wherein the sample is cells, tissues, body fluids, and feces.

[0647] The method or device of any of the foregoing embodiments, wherein the sample is a sample in the fields of human, veterinary, agricultural, food, environmental, and drug testing.

[0648] The method or device of any of the foregoing embodiments, wherein the sample is a biological sample selected from hair, nails, ear wax, breath, connective tissue, muscle tissue, nerve tissue, epithelial tissue, cartilage, cancer samples, or bone.

[0649] The method or device of any of the foregoing embodiments, wherein the spacing distance is in the range of 5 um to 120 um.

[0650] The method or device of any of the foregoing embodiments, wherein the spacing distance is in the range of 120 um to 200 um.

[0651] A method or apparatus according to any of the preceding embodiments, wherein the thickness of the flexible plate is in the range of 20 um to 250 um and the Young's modulus is in the range of 0.1 to 5 GPa.

[0652] A method or apparatus according to any of the preceding embodiments, wherein for the flexible plate, the product of the thickness of the flexible plate and the Young's modulus of the flexible plate is in the range of 60 to 750 GPa-um.

[0653] A method or apparatus according to any of the preceding embodiments, wherein the uniform thickness sample layer is uniform over a lateral area of at least 1 mm 2 thereof.

[0654] A method or apparatus according to any of the preceding embodiments, wherein the uniform thickness sample layer is uniform over a lateral area of at least 3 mm 2 thereof.

[0655] A method or apparatus according to any of the preceding embodiments, wherein the uniform thickness sample layer is uniform over a lateral area of at least 5 mm 2 thereof.

[0656] A method or apparatus according to any of the preceding embodiments, wherein the uniform thickness sample layer is uniform over a lateral area of at least 10 mm 2 thereof.

[0657] A method or apparatus according to any of the preceding embodiments, wherein the uniform thickness sample layer is uniform over a lateral area of at least 20 mm 2 thereof.

[0658] A method or apparatus according to any of the preceding embodiments, wherein the uniform thickness sample layer is uniform over a lateral area in the range of 20 mm 2 to 100 mm 2 thereof.

[0659] A method or apparatus according to any of the preceding embodiments, wherein the uniform thickness sample layer has a thickness uniformity of up to + / - 5% or better.

[0660] A method or apparatus according to any of the preceding embodiments, wherein the uniform thickness sample layer has a thickness uniformity of up to + / - 10% or better.

[0661] A method or apparatus according to any of the preceding embodiments, wherein the uniform thickness sample layer has a thickness uniformity of up to + / - 20% or better.

[0662] A method or apparatus according to any of the preceding embodiments, wherein the uniform thickness sample layer has a thickness uniformity of up to + / - 30% or better.

[0663] The method, apparatus, computer program product, or system of any of the preceding embodiments has five or more monitoring markers, where at least three of the monitoring markers are not straight lines.

[0664] The method, apparatus, computer program product, or system of any of the preceding embodiments, wherein each of the plates includes a force-receiving area on its respective outer surface for applying an imprecise pressing force that forces the plates together;

[0665] The method, apparatus, computer program product, or system of any of the preceding embodiments, wherein one or two of the plates are flexible;

[0666] The method, apparatus, computer program product, or system of any of the preceding embodiments, wherein the fourth power of the spacing distance (IDS) divided by the thickness (h) and Young's modulus (E) of the flexible plate (ISD4 / (hE)) is 5 x 106 um3 / GPa or less.

[0667] The method, apparatus, computer program product, or system of any of the preceding embodiments, wherein at least one of the spacers is within the sample contact area;

[0668] The method, apparatus, computer program product, or system of any of the preceding embodiments, wherein fat specification of analyte

[0669] The method, apparatus, computer program product, or system of any of the preceding embodiments, wherein fat specification of algorithms

[0670] The method, apparatus, computer program product, or system of any of the preceding embodiments, wherein fat specification of algorithms

[0671] The apparatus, system, or method of any of the preceding embodiments, wherein the algorithm is stored on a non-transitory computer-readable medium, and wherein the algorithm includes instructions that, when executed, perform a method of determining a property corresponding to the analyte using the monitoring markers of the apparatus.

[0672] Some examples of markers

[0673] In the present invention, in some embodiments, the markers have the same shape as the spacers.

[0674] In some embodiments, the markers are periodic or non-periodic.

[0675] In some embodiments, the distance between two markers is predetermined and known, but the absolute coordinates on the plate are unknown.

[0676] In some embodiments, the markers have a predetermined and known shape.

[0677] In some embodiments, the markers are configured to have a distribution in the plate such that regardless of the position of the plate, there is always a marker in the field of view of the imaging optics.

[0678] In some embodiments, the markers are configured to have a distribution in the plate such that regardless of the position of the plate, there is always a marker in the field of view of the imaging optics, and the number of markers is sufficient for local optical information.

[0679] In some embodiments, the markers are used to control the optical properties of a local region of the sample, and the region size is 1 um^2, 5 um^2, 10 um^2, 20 um^2, 50 um^2, 100 um^2, 200 um^2, 500 um^2, 1000 um^2, 2000 um^2, 5000 um^2, 10000 um^2, 100000 um^2, 500000 um^2, or within a range between any two values.

[0680] Use of "finite imaging optics"

[0681] In the present invention, in some embodiments, the optical system for imaging the assay has "finite imaging optics". Some embodiments of finite imaging optics include, but are not limited to:

[0682] 1. A finite imaging optical system comprising:

[0683] An imaging lens;

[0684] An imaging sensor;

[0685] Wherein the imaging sensor is part of a camera of a smartphone;

[0686] Wherein at least one of the imaging lenses is part of a camera of a smartphone;

[0687] 2. The finite imaging optical system according to any of the preceding embodiments, wherein: the physical optical resolution is less than 1 um, 2 um, 3 um, 5 um, 10 um, 50 um, or within a range between any two values.

[0688] 3. The finite imaging optical system according to any of the preceding embodiments, wherein: the per physical optical resolution is less than 1 um, 2 um, 3 um, 5 um, 10 um, 50 um, or within a range between any two values.

[0689] 4. A finite imaging optical system according to any of the foregoing embodiments, wherein: the preferred physical optical resolution is between about 1 um and 3 um;

[0690] 5. A finite imaging optical system according to any of the foregoing embodiments, wherein: the numerical aperture is less than 0.1, 0.15, 0.2, 0.2, 0.25, 0.3, 0.35, 0.4, 0.45, 0.5, or within the range between any two values.

[0691] 6. A finite imaging optical system according to any of the foregoing embodiments, wherein: the preferred numerical aperture is between 0.2 and 0.25.

[0692] 7. A finite imaging optical system according to any of the foregoing embodiments, wherein: the working distance is 0.2 mm, 0.5 mm, 1 mm, 2 mm, 5 mm, 10 mm, 20 mm, or within the range between any two values.

[0693] 8. A finite imaging optical system according to any of the foregoing embodiments, wherein: the working distance is 0.2 mm, 0.5 mm, 1 mm, 2 mm, 5 mm, 10 mm, 20 mm, or within the range between any two values.

[0694] 9. A finite imaging optical system according to any of the foregoing embodiments, wherein: the preferred working distance is between 0.5 mm and 1 mm.

[0695] 10. A finite imaging optical system according to any of the foregoing embodiments, wherein: the depth of focus is 100 nm, 500 nm, 1 um, 2 um, 10 um, 100 um, 1 mm, or within the range between any two values.

[0696] 11. A finite imaging optical system according to any of the foregoing embodiments, wherein: the depth of focus is 100 nm, 500 nm, 1 um, 2 um, 10 um, 100 um, 1 mm, or within the range between any two values.

[0697] 12. A finite imaging optical system according to any of the foregoing embodiments, wherein: the image sensor is part of a smartphone camera module.

[0698] 13. A finite imaging optical system according to any of the foregoing embodiments, wherein: the diagonal length of the image sensor is less than 1 inch, 1 / 2 inch, 1 / 3 inch, 1 / 4 inch, or within the range between any two values;

[0699] 14. A finite imaging optical system according to any of the foregoing embodiments, wherein: the imaging lens comprises at least two lenses, and one lens is part of the camera module of a smartphone.

[0700] 15. A finite imaging optical system as in any of the foregoing embodiments, wherein: at least one external lens is paired with an internal lens of a smartphone.

[0701] 16. A finite imaging optical system as in any of the foregoing embodiments, wherein: an optical axis of the external lens is aligned with the internal lens of the smartphone, and the alignment tolerance is less than 0.1 mm, 0.2 mm, 0.5 mm, 1 mm, or within a range between any two values.

[0702] 17. A finite imaging optical system as in any of the foregoing embodiments, wherein: a height of the external lens is less than 2 mm, 5 mm, 10 mm, 15 mm, 20 mm, or within a range between any two values.

[0703] 18. A finite imaging optical system as in any of the foregoing embodiments, wherein: a preferred height of the external lens is between 3 mm and 8 mm.

[0704] 19. A finite imaging optical system as in any of the foregoing embodiments, wherein: a preferred height of the external lens is between 3 mm and 8 mm.

[0705] 20. A finite imaging optical system as in any of the foregoing embodiments, wherein: a diameter of the external lens is less than 2 mm, 4 mm, 8 mm, 10 mm, 15 mm, 20 mm, or within a range between any two values.

[0706] 21. A finite imaging optical system as in any of the foregoing embodiments, wherein: each physical optical magnification is less than 0.1X, 0.5X, 1X, 2X, 4X, 5X, 10X, or within a range between any two values.

[0707] 22. A finite imaging optical system as in any of the foregoing embodiments, wherein: a preferred each physical optical magnification is less than 0.1X, 0.5X, 1X, 2X, 4X, 5X, 10X, or within a range between any two values.

[0708] Use of "finite sample operation"

[0709] In the present invention, in some embodiments, a sample positioning system for imaging an assay has "finite sample operation". Some embodiments of finite sample operation include, but are not limited to:

[0710] Description of the finite sample operation system:

[0711] 1. A finite sample operation system, comprising:

[0712] A sample holder;

[0713] wherein the sample holder has a socket for receiving a sample card.

[0714] 2. The limited sample operating system according to any one of the foregoing embodiments, wherein: the accuracy of positioning the sample in the direction along the optical axis is worse than 0.1 um, 1 um, 10 um, 100 um, 1 mm, or within the range between any two values.

[0715] 3. The limited sample operating system according to any one of the foregoing embodiments, wherein: the preferred accuracy of positioning the sample in the direction along the optical axis is between 50 um and 200 um.

[0716] 4. The limited sample operating system according to any one of the foregoing embodiments, wherein: the accuracy of positioning the sample in a plane perpendicular to the optical axis is worse than 0.01 um, 0.1 um, 1 um, 10 um, 100 um, 1 mm, or within the range between any two values.

[0717] 5. The limited sample operating system according to any one of the foregoing embodiments, wherein: the preferred accuracy of positioning the sample in a plane perpendicular to the optical axis is between 100 um and 1 mm.

[0718] 6. The limited sample operating system according to any one of the foregoing embodiments, wherein the horizontal error of positioning the sample card is worse than 0.01 degree, 0.1 degree, 0.5 degree, 1 degree, 10 degrees, or within the range between any two values.

[0719] 7. The limited sample operating system according to any one of the foregoing embodiments, wherein the preferred horizontal error of positioning the sample card is between 0.5 degrees and 10 degrees.

[0720] 8. The limited sample operating system according to any one of the foregoing embodiments, wherein the preferred horizontal error of positioning the sample card is between 0.5 degrees and 10 degrees.

[0721] Monitoring the sharp edge of the mark

[0722] The term "sharp optical edge" refers to the edge or boundary of an object in an image captured by a specific optical system having the following properties: at the edge or boundary of the object in the image, the light intensity (including but not limited to R, G, B, gray, hue, brightness) changes sharply with respect to position. For example, quantitatively, as Figure 23 shown in, at the boundary, the distance (X 90％ ) where the normalized intensity decreases from 90% to 10% should be less than 5% of the length of the object in the image by such an optical system. In other words, the boundary gradient = 90% / X 90％ > 1800% / object length. An example of a sharp edge is the edge of a columnar spacer having a flat top and sidewalls approaching 90 degrees. An example of an object without a sharp edge is a sphere.

[0723] Example of using monitored markers for TLD and volume estimation

[0724] Figure 12 Examples of the sample holding device, the QMAX device, and its monitored markers and columns used in some embodiments of the present invention are shown. The column in the QMAX device makes the gap between the two parallel plates of the sample holding device uniform. This gap is narrow and related to the size of the analyte, where the analyte forms a monolayer in this gap. In addition, the monitored markers in the QMAX device are a special form of the column, so they are not submerged by the sample and can be imaged together with the sample by an imager in an image-based assay.

[0725] Example of estimating TLD (True Lateral Dimension) using monitored markers

[0726] For TLD and true volume estimation in some embodiments of the present invention. The monitored markers (columns) are used as detectable anchors. However, it is difficult to detect the monitored markers in an image-based assay with an accuracy suitable for TLD estimation. This is because these monitored markers are penetrated and surrounded by the analyte within the sample holding device. They are distorted and blurred in the image due to lens distortion, light diffraction of micro-objects in the sample, defects at the micro level, misalignment of focus, noise in the sample image, etc. If the imager is a camera from a commercial device (e.g., a camera from a smartphone), it becomes even more difficult because these cameras are not calibrated by dedicated hardware once they leave the manufacturing process.

[0727] In the present invention, the detection and localization of the monitored markers, which are used as detectable anchors for TLD estimation, are formulated in a machine learning framework, and a dedicated machine learning model is constructed / trained to detect them in microscopic imaging. In addition, in some embodiments of the present invention, the distribution of the monitored markers is intentionally set to be periodic and distributed in a predetermined pattern. This makes the method of the present invention more robust and reliable.

[0728] Embodiments of the present invention include:

[0729] (1) Loading a sample into a sample holding device, such as a QMAX device, where there are monitored markers with a known structure in the device that are not submerged in the sample and can be imaged by an imager;

[0730] (2) Taking an image of the sample in the sample holding device including the analyte and the monitored markers;

[0731] (3) Constructing and training a machine learning (ML) model to detect the monitored markers in the image of the sample;

[0732] (4) Using the ML detection model from (3) to detect and locate the monitored markers in the sample holding device from the image of the sample;

[0733] (5) Generate a marker grid based on the monitoring markers detected in (4);

[0734] (6) Calculate a homography transformation based on the generated monitoring marker grid;

[0735] (7) Estimate and save the true lateral dimension of the sample image based on the homography transformation from (6); and

[0736] (8) Apply the estimated TLD from (7) in subsequent image-based determinations to determine the area, dimension, volume, and concentration of the analyte.

[0737] In some embodiments of the present invention, region-based TLD estimation and calibration are employed in image-based determinations. It includes:

[0738] (1) Load the sample into a sample holding device, such as a QMAX device, where monitoring markers are present in the device - not submerged in the sample and capable of being imaged by an imager in the image-based determination;

[0739] (2) Take an image of the sample in the sample holding device including the analyte and the monitoring markers;

[0740] (3) Construct and train a machine learning (ML) model for detecting monitoring markers from the image of the sample taken by the imager;

[0741] (4) Divide the sample image taken by the imager into non-overlapping regions;

[0742] (5) Use the ML model of (3) to detect and locate the monitoring markers from the image of the sample taken by the imager;

[0743] (6) Generate a region-based marker grid for each region in which more than 4 non-collinear monitoring markers are detected in the local region;

[0744] (7) Generate a marker grid for all regions not in (6) based on the monitoring markers detected from the image of the sample taken by the imager;

[0745] (8) Calculate a region-specific homography transformation for each region in (6) based on its own region-based marker grid generated in (6);

[0746] (9) Calculate a homography transformation for all other regions based on the marker grid generated in (7);

[0747] (10) Estimate the region-based TLD of each region in (6) based on the region-based homography transformation generated in (8);

[0748] (11)Estimate the TLD of other regions based on the homography transformation from (9); and

[0749] (12)In subsequent image-based measurements, save and apply the estimated TLD from (10) and (11) on the partition

[0750] When the monitoring marks are distributed in a predefined periodic pattern, such as in a QMAX device, they appear and are distributed periodically at a certain spacing. As a result, in the above process, the detection of the monitoring marks becomes more robust and reliable. This is because, if the detected positions and configurations do not follow the predefined periodic pattern, by using periodicity, all the monitoring marks can be identified and determined from only a small number of detected monitoring marks, and detection errors can be corrected and eliminated.

[0751] The terms "monitoring mark", "monitor mark", and "mark" are interchangeable in the description of the present invention.

[0752] The terms "imager" and "camera" are interchangeable in the description of the present invention.

[0753] The term "denoising" refers to the process of removing noise from a received signal. An example is removing noise from an image of a sample, because an image from an imager / camera can pick up noise from various sources, including but not limited to white noise, salt and pepper noise, Gaussian noise, etc. Denoising methods include but are not limited to: linear and nonlinear filtering, wavelet transform, statistical methods, deep learning, etc.

[0754] The term "image normalization" refers to algorithms, methods, and devices for changing the range of pixel intensity values in a processed image. For example, it includes but is not limited to increasing contrast by histogram stretching, subtracting the average pixel value from each image, etc.

[0755] In some embodiments of the present invention, methods and algorithms are designed to utilize monitoring marks in, for example, a sample holding device (such as a QMAX device). This includes but is not limited to the estimation and adjustment of the following parameters in an imaging device:

[0756] 1. Shutter speed,

[0757] 2. ISO,

[0758] 3. Focus (lens position),

[0759] 4. Exposure compensation,

[0760] 5. White balance: temperature, color, and

[0761] 6. Zoom (scale factor).

[0762] Examples of image processing / analysis algorithms used with markers

[0763] In some embodiments of the present invention, the monitoring markers in the present invention are used to apply and enhance image processing / analysis. They include, but are not limited to, the following image processing algorithms and methods:

[0764] 1. Histogram-based operations include, but are not limited to:

[0765] a. Contrast stretching;

[0766] b. Equalization;

[0767] c. Minimum filtering;

[0768] d. Median filtering; and

[0769] e. Maximum filtering.

[0770] 2. Math-based operations include, but are not limited to:

[0771] a. Binary operations: NOT, OR, AND, XOR, SUB, etc., and

[0772] b. Arithmetic-based operations: ADD, SUB, MUL, DIV, LOG, EXP, SQRT, TRIG, INVERT, etc.

[0773] 3. Convolution-based operations in the spatial and frequency domains include, but are not limited to, Fourier transform, DCT, integer transform, wavelet transform, etc.

[0774] 4. Smoothing operations include, but are not limited to:

[0775] a. Linear filtering: uniform filtering, triangular filtering, Gaussian filtering, etc., and

[0776] b. Nonlinear filtering: median filtering, kuwahara filtering, etc.

[0777] 5. Derivative-based operations include, but are not limited to:

[0778] a. First-order derivatives: gradient filtering, basic derivative filtering, Prewitt gradient filtering, Sobel gradient filtering, alternative gradient filtering, Gaussian gradient filtering, etc.;

[0779] b. Second-order derivatives: basic second-order derivative filtering, Laplacian in the frequency domain, Gaussian second-order derivative filtering, alternative Laplacian filtering, second-order derivative of gradient direction (SDGD) filtering, etc., and

[0780] c. Other filtering with higher-order derivatives, etc.

[0781] 6. Morphology-based operations include, but are not limited to:

[0782] a. Dilation and erosion;

[0783] b. Boolean convolution;

[0784] c. Opening and closing;

[0785] d. Hit and miss operations;

[0786] e. Segmentation and contour;

[0787] f. Skeleton;

[0788] g. Propagation;

[0789] h. Gray-scale value morphological processing: gray-scale dilation, gray-scale erosion, gray-scale opening, gray-scale closing, etc.; and i. Morphological smoothing, morphological gradient, morphological Laplacian, etc.

[0790] Other examples of image processing / analysis techniques

[0791] In some embodiments of the present invention, the image processing / analysis algorithm is used together with and enhanced by the monitoring marker. They include, but are not limited to, the following:

[0792] 1. Image enhancement and restoration include, but are not limited to

[0793] a. Sharpening and unsharp masking,

[0794] b. Noise suppression, and

[0795] c. Distortion suppression.

[0796] 2. Image segmentation includes, but is not limited to:

[0797] a. Thresholding - fixed threshold, histogram-derived threshold, Isodata algorithm, background symmetry algorithm, triangle algorithm, etc.;

[0798] b. Edge finding - gradient-based process, zero-crossing-based process, PLUS-based process, etc.;

[0799] c. Binary mathematical morphology - salt and pepper filtering, separating objects with holes, filling holes in objects, removing boundary-touching objects, outer skeleton, touching objects, etc.; and

[0800] d. Gray-scale value mathematical morphology - top-hat transform, adaptive thresholding, local contrast stretching, etc.

[0801] 3. Feature extraction and matching include, but are not limited to:

[0802] a. Independent component analysis;

[0803] b. Isometric mapping;

[0804] c. Principal component analysis and kernel principal component analysis;

[0805] d. Latent semantic analysis;

[0806] e. Least squares and partial least squares;

[0807] f. Multi-factor dimensionality reduction and non-linear dimensionality reduction;

[0808] g. Multilinear principal component analysis;

[0809] h. Multilinear subspace learning;

[0810] i. Semidefinite embedding; and

[0811] j. Autoencoder / decoder.

[0812] 4. Object detection, classification, and localization

[0813] 5. Image understanding

[0814] Example E1. Improvement of microscopic imaging using a monitoring marker

[0815] In the present invention, a monitoring marker is used to improve focusing in microscopic imaging. In particular, a monitoring marker with sharp edges will provide a detectable (visible feature) for a focus evaluation algorithm to analyze the focusing conditions of a specific focusing setting, especially in low-light environments and microscopic imaging. In an embodiment of an image-based assay, the focus evaluation algorithm is a core part in an autofocus implementation.

[0816] For some diagnostic applications (e.g., colorimetry, absorption-based hemoglobin tests, and CBC for samples with very low cell concentrations), the detectable features provided by the analyte in the image of the sample are usually not sufficient to enable the focus evaluation algorithm to operate accurately and smoothly. Monitoring a marker with sharp edges, such as the monitoring marker in a QMAX device, provides additional detectable features for the focus evaluation procedure to achieve the accuracy and reliability required in an image-based assay.

[0817] For some diagnostic applications, the analyte distribution in the sample is non-uniform. Simply relying on the features provided by the analyte often results in some unfair focusing settings, which give high weights to the focusing of some locally high-concentration regions, while the low analyte concentration regions deviate from the target. In some embodiments of the present invention, this effect is controlled by a focus adjustment according to the information of a monitoring marker with strong edges and evenly distributed in a precisely processed periodic pattern.

[0818] In addition, each imager has an imaging resolution that is limited in part by the number of pixels in its imager sensor, which varies from one million to several million pixels. For some microscopy imaging applications, the analyte has a small or minute size in the sample, such as the size of platelets in human blood being approximately 1.4 µm. When the target detection procedure requires a certain number of pixels, in addition to the available size of the FOV, the limited resolution in the image sensor poses a significant constraint on the device's ability in image-based assays.

[0819] Single-image super-resolution (SISR) is a technique that uses image processing and / or machine learning techniques to upsample an original source image to a higher resolution and remove as much of the blur caused by interpolation as possible, such that object detection procedures can also operate on the newly generated image. This will significantly reduce the above constraints and enable some other applications that were not possible before. Monitoring markers with known shapes and structures (such as the monitoring markers in a QMAX card) can be used as local references to evaluate the SISR algorithm to avoid the oversharpening effect produced by most prior art algorithms.

[0820] In some embodiments of the present invention, image fusion is performed to break the physical SNR (signal-to-noise ratio) limit in image-based assays.

[0821] The signal-to-noise ratio measures the quality of an image of a sample taken by an imager in microscopy imaging. Due to cost, technology, manufacturing, etc., there are practical limitations in imaging devices. In some cases, such as in mobile healthcare, the application requires a higher SNR than what conventional imaging devices can provide. In some embodiments of the present invention, multiple images (with the same and / or different imaging settings, such as in an embodiment of 3D fusion for combining multiple images focused at different depths of focus into one super-focused image) are taken and processed to generate an output image with a higher SNR, thus making these applications possible.

[0822] However, the images taken by one imager or multiple imagers tend to have some drawbacks and defects caused by physical limitations and implementation constraints. In image-based assays, this situation becomes severe in the microscopy imaging of samples because the analytes in the sample have minute sizes and often do not have distinct edge features. In some embodiments of the present invention, monitoring markers in a sample holding device (such as a QMAX card) are used for enhanced solutions.

[0823] One such embodiment is to process the distortion in the image of a sample taken by an imager. If the distortion parameters are known, the situation is relatively straightforward (most are manufactured to give curves / tables for their lenses to describe scale distortion, and other distortions can be measured in well-defined experiments). However, when the distortion parameters are unknown (which can vary with the focus position and even with the sample), using monitoring marks, a new algorithm can iteratively estimate the distortion parameters using the regularly or even periodically placed monitoring marks on a sample holding device (e.g., a QMAX card) without the need for a single coordinate reference.

[0824] Other examples

[0825] In the present invention, in some embodiments, the sample holding device has a flat surface that has some special monitoring marks for analyzing microfeatures in image-based assays. Some exemplary embodiments are listed below:

[0826] Estimation of the true lateral dimension (TLD) of a microscopic image of a sample in an image-based assay. The true lateral dimension (TLD) determines the physical size of the imaged analyte in the real world and also determines the image coordinates of the sample in the real world related to the concentration estimation in the image-based assay. The monitoring marks can be used as detectable anchors to determine the TLD and improve the accuracy of the image-based assay. In an embodiment of the present invention, a machine learning model is used to detect the monitoring marks and derive the TLD / FoV of the image of the sample therefrom. Additionally, if the monitoring marks have a periodic distribution pattern on the flat surface of the sample holding device, the detection of the monitoring marks and the TLD / FoV estimation based on each sample can become more reliable and robust in the image-based assay.

[0827] Analyzing the analyte using the response of the measured analyte compound at a specific optical wavelength or multiple optical wavelengths to predict the analyte concentration. Monitoring marks not immersed in the sample can be used to determine the light absorption of the background corresponding to zero concentration - to determine the analyte concentration by light absorption, and this method is used for the HgB test in the present invention. Additionally, each monitoring mark can act as an independent detector of background absorption to make the concentration estimation robust and reliable.

[0828] Focusing on the microscopic image of an image-based assay. Uniformly distributed monitoring marks can be used to improve the focusing accuracy. (a) It can be used to provide a minimum amount of visual features for samples without / fewer than the necessary features for reliable focusing. Due to the edge content of the monitoring marks, this can be performed in low light. (b) It can be used to provide visual features when the feature distribution in the sample is uneven to make the focusing determination fair. (c) It can provide a reference for local illumination conditions that have no / less / different impact on the content of the sample to adjust the weights in the focus evaluation algorithm.

[0829] The monitoring markers can be used as references for detecting and / or correcting image defects, which are caused by, but not limited to, unevenly distributed illumination, various types of image distortion, noise, and defective image preprocessing operations. For example, as Figure 22 shown, when a straight line in the 3D world is mapped as a curve in the image of the sample, the position of the marker can be used to detect and / or correct scale distortion. The scale distribution parameters of the entire image can be estimated based on the position changes of the monitoring markers of the sample holding device described herein. And by linearly testing the horizontal / vertical lines in the reproduced image through distortion elimination based on the assumed scale distortion parameters, the value of the scale distortion parameters can be iteratively estimated.

[0830] Examples of machine learning (ML) calculations

[0831] One way to use machine learning is to use a trained machine learning model during the inference process of processing to perform the detection of analytes in the image of the sample and calculate the bounding boxes covering their positions. Another way to use machine learning methods to detect and locate analytes in the sample image is to build and train a detection and segmentation model, which involves annotating the analytes in the sample image at the pixel level. In this method, in an image-based assay, a tight binary pixel mask covering the analytes in the sample image can be used to detect and locate the analytes in the sample image.

[0832] When measuring hemoglobin in human blood, an image is taken at a given narrowband wavelength, and then the average energy passing through the analyte region and the reference region is analyzed. Based on the known absorption rate of the analyte at the given wavelength and the height of the analyte sample region, the concentration can be estimated. However, this measurement has noise. To eliminate the noise, multiple images can be taken using light of different wavelengths and machine learning regression can be used to achieve a more accurate and robust estimate. The machine learning-based inference takes multiple input images of the sample, acquired at different wavelengths, and outputs a single concentration value.

[0833] E. Machine learning algorithms

[0834] E-1. QMAX device for assay and imaging

[0835] According to the present invention, a device for bioanalyte detection and localization is disclosed, comprising a QMAX device, an imager, and a computing unit. A biological sample is suspected to be on the QMAX device. The count and position of the analyte contained in the sample are obtained through the present disclosure.

[0836] According to the present invention, the imager captures an image of the biological sample. The image is submitted to the computing unit. The computing unit can be physically directly connected to the imager, through a network connection, or directly through image transmission.

[0837] E-2. Workflow

[0838] The disclosed analyte detection and localization employ machine learning and deep learning. Machine learning algorithms are algorithms that can learn from data. A more rigorous definition of machine learning is "a computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E." It explores the research and construction of algorithms that can learn and predict data - such algorithms overcome the following strictly static program instructions by building models based on sample inputs and making data-driven predictions or decisions.

[0839] Deep learning is a specific type of machine learning based on a set of algorithms that attempt to model high-level abstractions in data. In a simple case, there may be two sets of neurons: neurons that receive input signals and neurons that send output signals. When the input layer receives an input, it passes a modified version of the input to the next layer. In a deep network, there are many layers between the input and the output (and these layers are not composed of neurons, but this can help to think of it in that way), allowing the algorithm to use multiple processing layers composed of multiple linear and non-linear transformations.

[0840] The disclosed analyte detection and localization workflow includes two phases, training and prediction, as Figure 9 shown. We describe the training and prediction phases in the following paragraphs.

[0841] Training

[0842] In the training phase, annotated training data is fed into a convolutional neural network. A convolutional neural network is a neural network specialized for processing data, with a known grid-like topology. Examples include time series data and image data. Time series data can be considered as a 1D grid sampled at regular time intervals, and image data can be considered as a 2D grid of pixels. Convolutional networks are very successful in practical applications. The name "convolutional neural network" indicates that the network employs a mathematical operation called convolution. Convolution is a specialized linear operation. A convolutional network is simply a neural network that uses convolution to replace general matrix multiplication in at least one of its layers.

[0843] Annotate the training data for the analyte to be detected. The annotation indicates whether the analyte is present in the training data. The annotation can be in the form of a bounding box that completely contains the analyte or the central position of the analyte. In the latter case, the central position is further transformed into a circle that covers the analyte.

[0844] When the training data is large, it poses two challenges: annotation (usually done manually) is time-consuming, and training is computationally expensive. To overcome these challenges, the training data can be split into small-sized patches, which are then annotated and trained, either these patches or a portion of these patches.

[0845] The annotated training data is fed into a convolutional neural network for model training. The output is a model that can be used for pixel-level prediction of images. We use the Caffe library with a fully convolutional network (FCN). Other convolutional neural network architectures, such as TensorFlow, can also be used.

[0846] The training phase produces a model that will be used in the prediction phase. This model can be reused in the prediction phase of input images. Therefore, the computing unit only needs to access the generated model. It does not need to access the training data, nor does it have to run the training phase on the computing unit.

[0847] Prediction

[0848] In the prediction phase, the detection component is applied to the input image, followed by the localization component. The output of the prediction phase is the count of the analyte contained in the sample, as well as the location of each analyte.

[0849] In the detection component, the input image is fed into a convolutional neural network together with the model generated from the training phase. The output at the detection level is a pixel-level prediction in the form of a heatmap. The heatmap can have the same size as the input image, or it can be a scaled-down version of the input image. Each pixel in the heatmap has a value from 0 to 1, which can be considered the probability (confidence) that the pixel belongs to the analyte. The higher the value, the greater the chance that it belongs to the analyte.

[0850] This heatmap is the input to the localization component. We disclose an algorithm for locating the center of the analyte. The main idea is to iteratively detect local peaks based on the heatmap. After a peak is found, a local region around the peak but with smaller values is calculated. We remove this region from the heatmap and find the next peak from the remaining pixels. This process is repeated until all pixels are removed from the heatmap.

[0851] One embodiment of the localization algorithm is to sort the heatmap values from the highest to the lowest into a one-dimensional ordered list. Then the pixel with the highest value is selected, and this pixel together with its neighbors is removed from the list. This process is repeated to select the pixel with the highest value in the list until all pixels are removed from the list.

[0852] Algorithm global search (heatmap)

[0853]

[0854] After sorting, the heatmap is a one-dimensional ordered list where the heatmap values are sorted from high to low. Each heatmap value is associated with its corresponding pixel coordinates. The first item in the heatmap is the item with the highest value, i.e., the output of the pop (heatmap) function. Create a disk where the center is the pixel coordinates of the disk with the highest heatmap value. Then remove all heatmap values from the heatmap whose pixel coordinates are within the disk. The algorithm repeatedly pops the highest value in the current heatmap, removes the disk around it, until the item is removed from the heatmap.

[0855] In the ordered list heatmap, each item knows about the proceeding item and the following item. When removing an item from the ordered list, we make the following changes as Figure 10 shown:

[0856] · Assume the item to be removed is x r , whose proceeding item is x p , and whose following item is x f .

[0857] · For the proceeding item x p , redefine its following item as the following item of the removed item. Thus, the following item of x p is now x f .

[0858] · For the removed item x r , undefine its proceeding item and following item, and remove it from the ordered list.

[0859] · For the following item x f , redefine its proceeding item as the proceeding item of the removed item. Thus, the proceeding item of x f is now x p .

[0860] After removing all items from the ordered list, the localization algorithm ends. The number of elements in the set locus will be the count of the analyte, and the position information is the pixel coordinates of each s in the set locus.

[0861] Another embodiment searches for local peaks, which are not necessarily the local peaks with the highest heatmap values. To detect each local peak, we start from a random starting point and search for local maxima. After finding a peak, calculate the local region around the peak but with smaller values. We remove this region from the heatmap and find the next peak from the remaining pixels. Repeat the process until all pixels are removed from the heatmap.

[0862] Algorithm local_search(s, heatmap)

[0863]

[0864] This is a breadth - first search algorithm starting from s, with a modified access point condition: add the neighbor p of the current position q for coverage only when the heatmap[p]>0 and the heatmap[p] <= heatmap[q]. Thus, each pixel in the coverage has a non - decreasing path leading to the local peak s.

[0865] Algorithm localization (heatmap)

[0866]

[0867] E - 3. Example of the present invention

[0868] EA1. A deep - learning method for data analysis, comprising:

[0869] (f) Receiving an image of a test sample, where the sample is loaded into a QMAX device, and the image is taken by an imager connected to the QMAX device, and the image includes a detectable signal from an analyte in the test sample;

[0870] (g) Analyzing the image using a detection model and generating a two - dimensional data array of the image, where the two - dimensional data array includes probability data of the analyte at each position in the image, and the detection model is established through a training process, and the training process comprises:

[0871] iii. Feeding an annotated data set into a convolutional neural network, where the annotated data set is from a sample of the same type as the test sample and for the same analyte; and

[0872] iv. Establishing the detection model through convolutional training; and

[0873] (h) Analyzing the two - dimensional data array to detect local signal peaks by:

[0874] iii. A signal list process, or

[0875] iv. A local search process; and

[0876] (i) Calculating the amount of the analyte based on the local signal peak information.

[0877] EB1. A system for data analysis, comprising:

[0878] A QMAX device, an imager, and a computing unit, where:

[0879] (a) The QMAX device is configured to compress at least part of the test sample into a layer with a highly uniform thickness;

[0880] (b) The imager is configured to generate an image of the sample at the layer of uniform thickness, wherein the image includes a detectable signal from the analyte in the test sample;

[0881] (c) The calculation unit is configured to:

[0882] i. Receive the image from the imager;

[0883] ii. Analyze the image using a detection model and generate a two-dimensional data array of the image, wherein the two-dimensional data array includes probability data of the analyte at each position in the image, and the detection model is established through a training process, and the training process includes:

[0884] Feeding an annotated data set into a convolutional neural network, wherein the annotated data set is from a sample of the same type as the test sample and for the same analyte; and

[0885] Establishing the detection model through convolutional training; and

[0886] iii. (C) Analyze the two-dimensional data array using a signal list process or a local search process to detect local signal peaks; and

[0887] iv. Calculate the amount of the analyte based on the local signal peak information.

[0888] EA2. The method according to embodiment EA1, wherein the signal list process includes:

[0889] i. Iteratively detect local peaks from the two-dimensional data array to establish a signal list, calculate the local region around the detected local peaks, and sequentially remove the detected peaks and local region data into the signal list; and

[0890] ii. Sequentially and repeatedly remove the highest signal from the signal list and remove signals around the highest signal, thereby detecting local signal peaks.

[0891] EA3. The method according to any one of the EA embodiments, wherein the local search process includes:

[0892] i. Start from a random point and find the local maximum in the two-dimensional data array;

[0893] ii. Calculate the local region around the peak but with smaller values;

[0894] iii. Remove the local maximum and the surrounding smaller values from the two-dimensional data array; and

[0895] iv. Repeat steps i-iii to detect local signal peaks.

[0896] EA4. A method as in any of the foregoing EA embodiments, wherein the annotated data set is partitioned before annotation.

[0897] EB2. A system as in embodiment EB1, wherein the imager comprises a camera.

[0898] EB3. A system as in embodiment EB2, wherein the camera is part of a mobile communication device.

[0899] EB4. A system as in any of the foregoing EB embodiments, wherein the computing unit is part of a mobile communication device.

[0900] Examples of identifying error risks to improve measurement reliability

[0901] In some embodiments, a method for improving the reliability of an assay is provided, the method comprising:

[0902] (a) Imaging a sample on a QMAX card;

[0903] (b) Analyzing error risk factors; and

[0904] (c) If the error risk factor is higher than a threshold, rejecting the measurement result of the card and reporting the card;

[0905] wherein the error risk factor is one or any combination of the following factors. These factors are, but not limited to, (1) the edge of blood, (2) air bubbles in blood, (3) too small or too large a blood volume, (4) blood cells under a spacer, (5) aggregated blood cells, (6) lysed blood cells, (7) overexposed images of the sample, (8) underexposed images of the sample, (8) poor focus of the sample, (9) optical system errors such as incorrect lever position, (10) the card not being closed, (11) an incorrect card for a spacerless card (12) dust in the card, (13) oil in the card, (14) dirt outside the focus plane of the card, (15) the card not being in the correct position inside the reader, (16) the card being empty, (17) manufacturing errors in the card, (18) an incorrect card for other applications, (19) dried blood, (20) an expired card, (21) large variations in blood cell distribution, (22) no blood sample or no target blood sample, and others.

[0906] In some embodiments, the error risk analyzer is capable of detecting, differentiating, classifying, modifying, and / or correcting the following in biological and chemical applications in the device: (1) at the edge of the sample, (2) air bubbles in the sample, (3) too small or too large sample volume, (4) sample under the spacer, (5) aggregated sample, (6) lysed sample, (7) overexposed image of the sample, (8) underexposed image of the sample, (8) poor sample focus, (9) optical system errors such as misaligned levers, (10) card not closed, (11) wrong card for a spacerless card (12) dust in the card, (13) oil in the card, (14) dirt outside the focus plane of the card, (15) card not in the correct position inside the reader, (16) card empty, (17) manufacturing errors in the card, (18) wrong card for other applications, (19) dry sample, (20) expired card, (21) large variations in blood cell distribution, (22) wrong sample, and others.

[0907] Wherein the threshold is determined from a group of tests.

[0908] Wherein the threshold is determined from machine learning.

[0909] Wherein the monitoring markers are used as a comparison to identify the error risk factors.

[0910] Wherein the monitoring markers are used as a comparison to evaluate the threshold of the error risk factors.

[0911] More examples

[0912] Example A1 is a method of using a device to improve an imaging-based assay. The method may include receiving a sample image of a sample holder that includes a plurality of monitoring structures integrated on a contact surface of at least one plate of the sample holder, wherein the plurality of monitoring structures are placed according to a pattern, and wherein the contact surface contacts a sample containing a plurality of analytes; detecting the plurality of monitoring structures in the sample image using a machine learning model; optionally performing error correction on the detected plurality of monitoring structures using predetermined structural characteristics associated with the plurality of monitoring structures; determining a true lateral dimension value associated with the sample image based on the detected plurality of monitoring structures; determining a homography transformation between the position of the detected plurality of monitoring structures in the sample image and a predetermined distribution pattern of the plurality of monitoring structures in an actual image plane associated with the sample holder based on the true lateral dimension value; transforming the sample image containing a plurality of analytes into a corresponding perspective view in the actual image plane based on the homography transformation; and calculating at least one morphological characteristic associated with at least one of the plurality of analytes in the sample image.

[0913] In Example A2, the method of Example A1 may further include detecting the center of each of the plurality of monitoring structures; and determining a homography transformation based on the detected centers of the plurality of monitoring structures that include at least 4 non-collinear points.

[0914] In Example A3, the method of Example A1 may further provide dividing the sample image into non-overlapping image patches, each of the non-overlapping image patches including at least 4 non-collinear detected centers of the plurality of monitoring structures, wherein a patch-specific homography transformation is estimated and applied to compensate the image patch, and for other image patches, a global homography transformation is estimated and applied based on the detected centers of the plurality of monitoring structures in the entire image.

[0915] In Example A4, the method of Example A1 may further provide that the monitoring structure is a pillar or a monitoring mark. In some implementations, the pillar may be a nanostructure integrated with one or two contact surfaces that are substantially perpendicular to at least one plate of the sample holder. In other implementations, the monitoring marks are marked regions on the one or two contact surfaces. These marked regions may have different optical properties (e.g., transparency) on the one or two contact surfaces compared to the unmarked regions. For example, the marked region may be a surface region coated or engraved with a thin layer of nanomaterial, while the unmarked region is not covered by any nanomaterial.

[0916] In Example A5, the method of Example A1 may further provide: training a machine learning model using a set of labeled training images, and wherein the machine learning model may be a RetinerNet or a convolutional neural network (CNN) whose parameters are trained using the set of labeled training images.

[0917] In Example A6, the method of Example A1 may further provide: the predetermined structural characteristics include at least one of periodicity, shape, or size associated with the plurality of monitoring structures. The monitoring structures may be arranged in an organized pattern. The periodicity may refer to the number of monitoring structures within a measurement unit (e.g., a linear measurement such as inches or millimeters, or an area measurement such as square inches or square millimeters). The shape may refer to the geometric configuration of each monitoring structure. The monitoring structure may be a cylindrical structure with a cross-section that is triangular, rectangular, square, circular, polygonal, or any suitable two-dimensional shape. The size may refer to the cross-sectional area of the monitoring structure. In one implementation, the monitoring structures may have substantially the same shape and size. In another implementation, the monitoring structures may have various shapes and sizes, and the positions of different monitoring structures are predetermined during the manufacture of the sample holder.

[0918] In Example A7, the method of Example A1 can further provide that the at least one morphological characteristic includes at least one of the size or length of one of the plurality of analytes. The size can be an area measurement. The length can be a linear measurement along an axis. For example, for a circle, the length can be the diameter; for a rectangle, the length can be the height, width, or diagonal length.

[0919] Example A8 is a method of microselective image assay (MSIA) in an image-based assay. The method includes capturing an image of a sample for assay in the sample holding device, where the sample holding device can be the sample holder described in Example A1, and the sample in the sample holding device containing a known uniform height and analytes forms a monolayer in the region of interest; estimating the TLD or FoV of the image of the sample to determine the estimates of area, size, and volume in the image-based assay; detecting defects including bubbles or dust in the image of the sample by a trained machine learning model to assay and segment these defects in the image of the sample; using the estimated TLD / FoV from (b) to estimate the total area of the segmented defects in the sample image and calculate their actual area sizes; estimating the actual volume of the sample corresponding to the total surface area of the detected defects in the image of the sample for assay based on the area estimate from (d) and the known height of the sample in the sample holding device; subtracting the defect volume estimate from the total volume corresponding to the surface area under the detected defects in (e), removing the surface area of the detected defects in the image of the sample for assay, and updating the total volume of the remaining sample; and performing an image-based assay on the selected region and the updated sample volume of the updated sample image from (f) for microselective image assay.

[0920] In Example A9, the method of Example A8 can further provide that the microselective image assay (MSIA) uses the monitoring structure in the sample holding device of A4 to estimate the TLD / FoV of the image of the sample and maps the image of the sample to its actual size by the embodiment of A1.

[0921] In Example A10, the method of Example A8 can further provide that the microselective image assay (MSIA) is based on other selection criteria, including: a) the distribution of defects in the sample, including bubbles and dust; b) the positions of columns, monitoring markers, and other artifacts in the image of the sample for assay; and c) the distribution and conditions of analytes in the image of the sample for assay, including the conditions of analyte clustering and focusing conditions.

[0922] In Example A11, the method of Example A8 can further provide for performing a multi-target assay from an image of the sample by microselective image assay (MSIA), where the multi-target assay is based on region / zone selection having regions defined by different reagents or sample heights for multiple assay applications from a single image of the sample.

[0923] In Example A12, the method of either Example A8 or A11 can further provide for training a machine learning model on a labeled training image sample captured by an imager for defect detection and segmentation in image-based assays.

[0924] In Example A13, the method of either Example A8 or A11 can further provide for training a machine learning model to detect analytes in a sample image and determining the size of the detected analytes using the method of Example A1 and the structure of the sample holding device described in Example A8.

[0925] Example A14 is a method of monitoring an image-based assay using a column or monitoring an image of a column or a monitoring marker in an image of a sample used for the assay to determine the quality of an image holding device and the quality of sample preparation, the method including detecting regions corresponding to missing or broken columns in the sample holding device that affect the effective sample volume for the assay; and detecting bubbles and dust in the image of the sample, indicating flaws in the assay operation or defects in the sample holding device.

[0926] In Example A15, the method of Example A14 can further include performing an assay using the image of the column or the monitoring marker in the image of the sample to detect and adjust the operation of the imager in the image-based assay, including: focusing, contrast stretching, iso adjustment, and filtering.

[0927] Example BA-1 is an intelligent assay monitoring method that includes receiving, by a processing device, an image encoding first information about a biological sample deposited in a sample card and second information about a plurality of monitoring markers; performing, by the processing device, a first machine learning model on the image to determine measurements of geometric features associated with the plurality of monitoring markers; determining, by the processing device, a change between the measurements of the geometric features and a ground truth of geometric features provided with the sample card; correcting, by the processing device, the image encoding the first information and the second information based on the change; and determining, by the processing device, a biological characteristic of the biological sample using the corrected image.

[0928] In Example BA-2, the method of Example BA-1 can further provide that the sample card includes a first plate, a plurality of columns integrally formed substantially perpendicular to a surface of the first plate, and a second plate capable of surrounding the first plate to form a thin layer in which the biological sample is deposited.

[0929] In Example BA-3, the method of either Example BA-1 or BA-2 may further provide that a plurality of monitoring markers correspond to a plurality of columns.

[0930] In Example BA-4, the method of Example BA-3 may further provide that at least two of the plurality of columns are separated by a True Lateral Dimension (TLD), and wherein performing a first machine learning model on the image by the processing device to determine measurements of geometric features associated with the plurality of monitoring markers includes performing the first machine learning model on the image by the processing device to determine the TLD.

[0931] Example BB-1 is an image system that includes a sample card that includes a first plate, a plurality of columns integrally formed substantially perpendicular to a surface of the first plate, and a second plate that can surround the first plate to form a thin layer in which the biological sample is deposited; and a computing device that includes: a processing device communicatively coupled to an optical sensor to receive an image encoding first information about a biological sample deposited in the sample card and second information about a plurality of monitoring markers, use a first machine learning model on the image to determine measurements of geometric features associated with the plurality of monitoring markers, determine a variation between the measurements of the geometric features and a ground truth of geometric features of the provided sample card, correct the image encoding the first information and the second information based on the variation, and determine a biological characteristic of the biological sample based on the corrected image.

[0932] Example DA-1 is a method for measuring the volume of a sample in a thin layer sample card, the method including receiving, by a processing device of an image system, an image of a sample card that includes a sample and a monitoring standard, wherein the monitoring standard includes a plurality of columns integrally formed perpendicular to a first plate of the sample, and each of the plurality of columns has a substantially same height (H); determining, by the processing device, a plurality of non-sample sub-regions using a machine learning model, wherein the plurality of non-sample sub-regions correspond to at least one of columns, bubbles, or impurity elements; calculating, by the processing device, an area occupied by the sample by removing the plurality of non-sample sub-regions from the image; calculating, by the processing device, the volume of the sample based on the calculated area and the height (H); and determining, by the processing device, a biological characteristic of the sample based on the volume.

[0933] A1: A method of using a device to improve an imaging-based assay, including:

[0934] a) Receive a sample image of a sample holder that includes a plurality of monitoring structures integrated on a contact surface of at least one plate of the sample holder, where the plurality of monitoring structures are placed according to a pattern, and where the contact surface contacts a sample that includes a plurality of analytes;

[0935] b) Detect the plurality of monitoring structures in the sample image using a machine learning model;

[0936] c) Perform error correction on the detected plurality of monitoring structures using predetermined structural characteristics associated with the plurality of monitoring structures;

[0937] d) Determine a true lateral dimension value associated with the sample image based on the detected plurality of monitoring structures;

[0938] e) Determine a homography transformation between the positions of the detected plurality of monitoring structures in the sample image and a predetermined distribution pattern of the plurality of monitoring structures in an actual image plane associated with the sample holder based on the true lateral dimension value;

[0939] f) Based on the homography transformation, transform the sample image that includes a plurality of analytes into a corresponding perspective view in the actual image plane; and

[0940] g) Calculate at least one morphological characteristic associated with at least one of the plurality of analytes in the sample image.

[0941] A2: The method as in A1, further comprising:

[0942] Detect the center of each of the plurality of monitoring structures; and

[0943] Determine the homography transformation based on the detected centers of the plurality of monitoring structures that include at least 4 non - collinear points.

[0944] A3: The method as in A1, where the sample image is divided into non - overlapping image patches, each of the non - overlapping image patches includes at least 4 non - collinear detected centers of the plurality of monitoring structures, where a patch - specific homography transformation is estimated and applied to compensate the image patch, and for other image patches, a global homography transformation is estimated and applied based on the detected centers of the plurality of monitoring structures in the entire image.

[0945] D1: The method as in A1, where the monitoring structure is a pillar or a monitoring mark.

[0946] D2: The method as in A1, where a labeled training image set is used to train the machine learning model, and where the machine learning model is a RetinerNet whose parameters are trained using the labeled training image set.

[0947] D3: The method as in A1, wherein the predetermined structural characteristic includes at least one of periodicity, shape or size associated with a plurality of monitoring structures.

[0948] D4: The method as in A1, wherein the at least one morphological characteristic includes at least one of the size or length of one of the plurality of analytes.

[0949] A4: A method for microselective image assay (MSIA) in an image-based assay, comprising:

[0950] a) Capturing an image of a sample for assay in the sample holding device, wherein the sample holding device is as in A1(a), and the sample in the sample holding device containing a known uniform height and analytes forms a monolayer in the region of interest;

[0951] b) Estimating the TLD / FoV of the image of the sample according to (a) for the estimation of area, size and volume in subsequent image-based assays;

[0952] c) Detecting defects including bubbles or dust in the image of the sample through a trained machine learning model for inspection and segmentation of these defects in the image of the sample;

[0953] d) Estimating the total area of the segmented defects in the sample image using the estimated TLD / FoV from (b) and calculating their actual area size;

[0954] e) Estimating the actual volume of the sample corresponding to the total surface area of the detected defects in the image of the sample for assay according to the area estimation from (d) and the known height of the sample in the sample holding device;

[0955] f) Subtracting the defect volume estimation from (e) corresponding to the total volume under the surface area of the detected defects, removing the surface area of the detected defects in the image of the sample for assay, and updating the total volume of the remaining sample; and

[0956] g) Performing an image-based assay on the selected region of the updated sample image from (f) and the updated sample volume for microselective image assay.

[0957] A5: The method as in A4, wherein the microselective image assay (MSIA) utilizes the monitoring structures in the sample holding device of A4 to estimate the TLD / FoV of the sample image and maps the sample image to its actual size through the embodiments of A1.

[0958] A6: The method as in A4, wherein the microselective image assay (MSIA) is based on other selection criteria, including:

[0959] a) Defect distribution in the sample, including bubbles and dust;

[0960] b) Positions of columns and monitoring markers and other artifacts in the image of the sample for determination;

[0961] c) Distribution and conditions of the analyte in the image of the sample for determination, including conditions for analyte clustering and focusing conditions.

[0962] A7: A method as in A4, wherein microselective image assay (MSIA) performs multi-target determination from an image of the sample, wherein the multi-target determination is based on region / zone selection having regions defined by different reagents or sample heights for multiple assay applications from a single image of the sample.

[0963] A8: A method as in any one of A4 or A7, wherein a machine learning model is trained on a labeled training image sample captured by an imager for defect detection and segmentation in image-based determination.

[0964] A9: A method as in any one of A4 or A7, wherein a machine learning model is trained to detect analytes in a sample image and the size of the detected analytes is determined using the structure of the sample holding device described in A1 and A4.

[0965] A10: A method of monitoring an image-based determination using columns or monitoring markers to determine the quality of an image holding device and the quality of sample preparation, the columns or monitoring markers using an image of columns or monitoring markers in an image of the sample for determination, the method comprising:

[0966] a) Detecting missing columns or broken columns in the sample holding device that affect the effective sample volume for determination;

[0967] b) Detecting objects or analytes on the columns or monitoring markers that affect the height of the sample in the sample holding device and thus affect the effective volume of the sample for determination; and

[0968] c) Detecting bubbles and dust in the sample image, indicating flaws in the assay operation or defects in the sample holding device.

[0969] A12: A method as in A10, wherein the labeled image is assayed to detect and adjust the operation of the imager in the image-based determination, including: focusing, contrast stretching, iso adjustment, and filtering.

[0970] Definition

[0971] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. Although any methods and materials similar or equivalent to those described herein can also be used in the practice or testing of the present teachings, some exemplary methods and materials are now described.

[0972] The “QMAX” (Q: quantification; M: amplification; A: addition of reagent; X: acceleration; also known as self-calibrated compressed open flow (SCOF)) device, assay, method, kit, and system are described in: U.S. Provisional Patent Application No. 62 / 202,989, filed Aug. 10, 2015; U.S. Provisional Patent Application No. 62 / 218,455, filed Sep. 14, 2015; U.S. Provisional Patent Application No. 62 / 293,188, filed Feb. 9, 2016; U.S. Provisional Patent Application No. 62 / 305,123, filed Mar. 8, 2016; and U.S. Provisional Patent Application No. 62 / 369,181, filed Jul. 31, 2016; U.S. Provisional Patent Application No. 62 / 394,753, filed Sep. 15, 2016; PCT Application (designating the United States) No. PCT / US2016 / 045437, filed Aug. 10, 2016; PCT Application (designating the United States) No. PCT / US2016 / 051775, filed Sep. 14, 2016; PCT Application (designating the United States) No. PCT / US2016 / 051794, filed Sep. 15, 2016; and PCT Application (designating the United States) No. PCT / US2016 / 054025, filed Sep. 27, 2016, all of which disclosures are hereby incorporated by reference in their entirety and for all purposes.

[0973] As used herein, the term “sample” refers to a material or mixture of materials containing one or more analytes or entities of interest. In some embodiments, the sample can be obtained from a biological sample, such as cells, tissues, body fluids, and feces. Body fluids of interest include, but are not limited to, amniotic fluid, aqueous humor, vitreous humor, blood (e.g., whole blood, fractionated blood, plasma, serum, etc.), breast milk, cerebrospinal fluid (CSF), cerumen (earwax), chyle, chyme, endolymph, perilymph, feces, gastric acid, gastric juice, lymph fluid, mucus (including nasal drainage and sputum), pericardial fluid, peritoneal fluid, pleural fluid, pus, inflammatory exudate, saliva, sebum (skin oil), semen, sputum, sweat, synovial fluid, tears, vomit, urine, and exhaled condensate. In specific embodiments, a sample can be obtained from a subject, such as a human, and can be processed before being used for subject assays. For example, prior to analysis, proteins / nucleic acids can be extracted from tissue samples, and the methods are known. In certain embodiments, the sample can be a clinical sample, e.g., a sample collected from a patient.

[0974] The term "analyte" refers to molecules (such as proteins, peptides, DNA, RNA, nucleic acids, or other molecules), cells, tissues, viruses, and nanoparticles having different shapes. In some embodiments, the "analyte" as used herein is any substance suitable for testing in the present method.

[0975] As used herein, a "diagnostic sample" refers to any biological sample obtained from a subject as a bodily byproduct, such as a body fluid. A diagnostic sample can be obtained directly from the subject in liquid form or can be obtained from the subject by first placing the bodily byproduct in a solution, such as a buffer. Exemplary diagnostic samples include, but are not limited to, saliva, serum, blood, sputum, urine, sweat, tears, semen, feces, breath, biopsy, mucus, and the like.

[0976] As used herein, an "environmental sample" refers to any sample obtained from the environment. Environmental samples can include liquid samples from rivers, lakes, ponds, oceans, glaciers, icebergs, rain, snow, sewage, reservoirs, tap water, drinking water, and the like; solid samples from soil, compost, sand, rock, concrete, wood, brick, sewage, and the like; and gas samples from air, underwater thermal vents, industrial exhaust, vehicle exhaust, and the like. Typically, samples in non-liquid form are converted to liquid form before being analyzed by the methods of the present invention.

[0977] As used herein, a "food sample" refers to any sample suitable for animal consumption (e.g., human consumption). Food samples can include raw materials, cooked foods, plant and animal sources of food, pre-treated foods, and partially or fully processed foods, among others. Typically, samples in non-liquid form are converted to liquid form before being analyzed by the methods of the present invention.

[0978] As used herein, the term "diagnosis" refers to a method or analyte used to identify, predict the outcome of a disease or condition of interest, and / or predict the therapeutic response to a disease or condition of interest. Diagnosis can include predicting the likelihood or propensity of having a disease or condition, estimating the severity of a disease or condition, determining the risk of disease or condition progression, assessing the clinical response to treatment, and / or predicting the response to treatment.

[0979] A "biomarker" as used herein is any molecule or compound found in a sample of interest and known to be diagnostic or associated with the presence or propensity of a disease or condition of interest in the subject from which the sample was derived. Biomarkers include, but are not limited to, polypeptides or their complexes (e.g., antigens, antibodies) known to be associated with a disease or condition of interest, nucleic acids (e.g., DNA, miRNA, mRNA), drug metabolites, lipids, carbohydrates, hormones, vitamins, and the like.

[0980] As used herein, the "condition" for diagnosing a health condition refers to a physical or mental state that is distinct from other physiological states. In some cases, a health condition may not be diagnosed as a disease. Exemplary health conditions of interest include, but are not limited to, nutritional health; aging; exposure to environmental toxins, pesticides, herbicides, synthetic hormone analogs; pregnancy; menopause; climacteric; sleep; stress; prediabetes; exercise; fatigue; chemical balance; etc.

[0981] It must be noted that, as used herein and in the appended claims, the singular forms "a", "an", and "the" include plural referents unless the context clearly dictates otherwise, such as when the term "single" is used. For example, reference to an "analyte" includes a single analyte and multiple analytes, reference to a "capture agent" includes a single capture agent and multiple capture agents, reference to a "detection agent" includes a single detection agent and multiple detection agents, and reference to a "reagent" includes a single reagent and multiple reagents.

[0982] As used herein, the terms "adapted" and "configured" mean that an element, component, or other subject is designed and / or intended to perform a given function. Thus, the use of the terms "adapted" and "configured" should not be construed to mean that a given element, component, or other subject simply "can" perform a given function. Similarly, a subject that is stated to be configured to perform a particular function may additionally or alternatively be described as operable to perform that function.

[0983] As used herein, the phrases "at least one" and "one or more" with respect to a list of more than one entity refer to any one or more of the entities in the list and are not limited to at least one of each and every entity specifically listed in the list. For example, "at least one of A and B" (or equivalently, "at least one of A or B", or equivalently, "at least one of A and / or B") can refer to A alone, B alone, or a combination of A and B.

[0984] As used herein, the term "and / or" placed between a first entity and a second entity refers to (1) the first entity, (2) the second entity, and (3) one of the first entity and the second entity. Multiple entities listed using "and / or" shall be construed in the same manner, i.e., "one or more" of the entities so combined. There may optionally be other entities in addition to those specifically identified by the "and / or" clause, whether or not they are related to those specifically identified.

[0985] When the present invention mentions a numerical range, the present invention includes embodiments that include the endpoints, embodiments that exclude both endpoints, and embodiments that include one endpoint and exclude the other endpoint. It should be assumed that both endpoints are included, unless otherwise stated. In addition, unless otherwise stated or it is obvious to those of ordinary skill in the art from the context and understanding.

Claims

1. An image system, comprising: A sample card, comprising a first plate, a plurality of columns integrally formed substantially perpendicular to a surface of the first plate, and a second plate capable of surrounding the first plate to form a thin layer in which the biological sample is deposited; A computing device, comprising: A processing device communicatively coupled to an optical sensor to: Receive an image encoding first information of a biological sample deposited in the sample card and second information of a plurality of monitoring markers from the optical sensor; Use a first machine learning model on the image to determine measurements of geometric features associated with the plurality of monitoring markers; Determine a variation between the measurements of the geometric features and a ground truth of the geometric features provided by the sample card; Correct the image encoding the first information and the second information based on the variation; and Determine a biological characteristic of the biological sample based on the corrected image.

2. An image system, comprising: An adapter for holding a sample card comprising a first plate, a second plate, and a plurality of marker elements; A mobile computing device coupled to the adapter, the mobile computing device comprising: An optical sensor for capturing an image of the plurality of marker elements and the sample card, which comprises a substantially uniform sample layer deposited between the first plate and the second plate of the sample card; A processing device, communicatively coupled to the optical sensor, to: Receive the image captured by the optical sensor; Detect the plurality of marker elements in the image; Compare the detected plurality of marker elements with a monitoring standard associated with the sample card to determine a geometric mapping between the plurality of marker elements and the monitoring standard; Determine a non-ideal factor of the imaging system based on the geometric mapping; And Process the image of the sample card to correct the non-ideal factor.

3. An image system, comprising: An adapter for holding a sample card comprising a first plate, a second plate, and a plurality of marker elements; A mobile computing device coupled to the adapter, the mobile computing device comprising: An optical sensor for capturing an image of the plurality of marker elements and the sample card, which comprises a substantially uniform sample layer deposited between the first plate and the second plate of the sample card; A processing device, communicatively coupled to the optical sensor, to: Receive the image captured by the optical sensor; Divide the image into a plurality of sub-regions; Use a machine learning model and a plurality of marker elements to determine whether each of the plurality of sub-regions meets the requirements of the imaging system; In response to determining that a sub-region does not meet the requirements, mark the first sub-region as non-compliant; In response to determining that the sub-region meets the requirements, mark the first sub-region as compliant; And Perform an assay analysis using the compliant sub-regions of the image.

4. An image system, comprising: An adapter for holding a sample card containing monitoring criteria, wherein the monitoring criteria includes a plurality of nanostructures on a plate of the sample card, and wherein a sample is deposited on the plate; A mobile computing device coupled to the adapter, the mobile device comprising: An optical sensor; and A processing device communicatively coupled to the optical sensor to: Receive an image of the sample card; Determine a non-ideal factor of the image system by comparing the image of the sample card containing the sample deposited on the plate with a plurality of geometric values of the monitoring criteria determined during the manufacture of the sample card; and Correct the image of the sample card considering the non-ideal factor.

5. An intelligent determination method based on image processing and machine learning, comprising: Receiving, by a processing device in the graphic system according to claim 1, an image encoding first information of a biological sample deposited in a sample card and second information of a plurality of monitoring markers; The processing device executes a first machine learning model on the image to determine measurements of geometric features associated with a plurality of monitoring markers; The processing device determines a variation between the measurements of the geometric features and the ground truth of the geometric features provided by the sample card; The processing device corrects the image encoding the first information and the second information based on the variation; And The processing device uses the corrected image to determine the biological characteristics of the biological sample.

6. The method according to claim 5, wherein the sample card comprises a first plate, a plurality of columns integrally formed substantially perpendicular to a surface of the first plate, and a second plate capable of surrounding the first plate to form a thin layer in which the biological sample is deposited.

7. The method according to claim 5, wherein the plurality of monitoring markers correspond to the plurality of columns.

8. The method according to claim 7, wherein at least two of the plurality of columns are separated by a true lateral dimension (TLD), and wherein the processing device executes a first machine learning model on the image to determine measurements of geometric features associated with the plurality of monitoring markers, including executing the first machine learning model on the image by the processing device to determine the TLD.

9. A method for correcting manual operation errors recorded in a determination image of a thin-layer sample, the method comprising: The processing device of the image system according to claim 3 receives an image of a sample card containing a sample and a monitoring standard, wherein the monitoring standard includes a plurality of nanostructures integrated on a first plate of the sample card, and wherein the sample is deposited on the first plate in an open configuration of the sample card and is surrounded by a second plate of the sample card in a closed configuration of the sample card; The processing device segments the image into a first sub-region corresponding to the sample and a second sub-region corresponding to a plurality of nanostructures; The processing device compares the second sub-region with the monitoring standard provided during the manufacture of the sample to determine whether at least one of the second sub-regions contains foreign matter other than the nanostructures; In response to determining that at least one of the second sub-regions contains foreign matter other than the nanostructures, determine an error associated with operating the sample card; and Correct the image of the sample card by removing the at least one sub-region from the image.

10. The method according to claim 9, wherein the foreign object is one of a part of the sample, a bubble, or an impurity.

11. A method for measuring the volume of a sample in a thin-layer sample card, the method comprising: The processing device of the image system according to claim 1 receives an image of a sample card containing a sample and a monitoring standard, wherein the monitoring standard includes a plurality of columns perpendicularly integrated with a first plate of the sample, and each of the plurality of columns has a substantially same height (H); The processing device determines a plurality of non-sample sub-regions using a machine learning model, wherein the plurality of non-sample sub-regions correspond to at least one of columns, bubbles, or impurity elements; The processing device calculates the area occupied by the sample by removing a plurality of non-sample sub-regions from the image; The processing device calculates the volume of the sample based on the calculated area and height (H), and The processing device determines the biological characteristics of the sample based on the volume.

12. A method for determining a credibility measure associated with an image determination result, the method comprising: The processing device of the image system according to claim 3 receives an image of a sample card containing a sample and a monitoring standard, the monitoring standard including a plurality of nanostructures integrated into a first plate of the sample; The processing device segments the image into a first sub-region corresponding to the sample and a second sub-region corresponding to a plurality of nanostructures; The processing device uses a first machine learning model to determine non-compliant elements in at least one of the first sub-region or the second sub-region; The processing device determines the biological characteristics of the sample based on the first sub-region and the second sub-region; The processing device calculates a credibility measurement associated with the biological characteristics based on a statistical analysis of the non-compliant elements; And The processing device determines a further action on the sample based on the credibility measurement.

13. The method according to claim 12, further comprising: The processing device determines that the biometric characteristic is reliable based on the credibility measure; and The processing device provides the biometric characteristic to a display device.

14. The method according to claim 12, further comprising: determining, by the processing device, that the biometric is not reliable based on the credibility measurement; and providing, by the processing device, the biometric and the corresponding credibility measurement to a display device to allow a user to determine whether to accept or discard the biometric.

15. The method according to claim 12, wherein the processing device segments the image into a first sub-region corresponding to the sample and a second sub-region corresponding to the plurality of nanostructures: performing, by the processing device, a first image segmentation on the image using an image processing method to produce a first segmentation result; The second image segmentation is performed on the image by the processing device using a second machine learning model to generate a second segmentation result; and the first segmentation result and the second segmentation result are combined by the processing device to segment the image into the first sub-region corresponding to the sample and the second sub-region corresponding to the plurality of nanostructures.

16. The method according to claim 12, wherein determining, by the processing device, non-compliant elements in at least one of the first sub-region or the second sub-region using a first machine learning model comprises at least one of the following: determining, by the processing device, the non-compliant elements based on non-uniform distribution of at least one analyte in the sample; The non-compliant element is determined by the processing device based on the detection of aggregated analytes in the sample; The non-compliant element is determined by the processing device based on the detection of dry texture in the sample; The non-compliant element is determined by the processing device based on the detection of impurities in the sample; or The non-compliant element is determined by the processing device based on the detection of bubbles in the sample.

17. A method for determining measurements of multiple analytes using a single-sample card, the method comprising: receiving, by a processing device of the image system according to claim 3, an image of a sample card comprising a sample and a monitoring standard, wherein the monitoring standard comprises a plurality of nanostructures integrated into a first plate of the sample; segmenting, by the processing device, the image into a first sub-region associated with a first analyte contained in the sample, a second region associated with a second analyte contained in the sample, and a third sub-region corresponding to the plurality of nanostructures using a first machine learning model; The true lateral distance (TLD) between two adjacent nanostructures is determined by the processing device based on the third sub-region corresponding to the plurality of nanostructures; The first cumulative area of the first sub-region is determined by the processing device based on the TLD, and a first volume is further determined based on the first cumulative area and the height associated with the plurality of nanostructures; The second cumulative area of the sub-region is determined by the processing device based on the TLD, and a second volume is further determined based on the second cumulative area and the height associated with the plurality of nanostructures; The first measurement of the first analyte is determined by the processing device based on the count of the first analyte in the first volume; and The second measurement of the second analyte is determined by the processing device based on the count of the second analyte in the second volume.

18. A method for measuring the biological properties of a sample provided in a sample card, the sample card including a plurality of nanocolumns, the method comprising: Receiving, by a processing device of the image system according to claim 3, an image of a sample card including a sample and a monitoring standard, the monitoring standard including a plurality of nanostructures integrated into a first plate of the sample; The image is segmented by the processing device into a first sub-region corresponding to the sample and a second sub-region corresponding to the plurality of nanocolumns; The first spectrophotometric measurement of the first sub-region is determined by the processing device; The second spectrophotometric measurement of the second sub-region is determined by the processing device; and The biological characteristics of the sample are determined by the processing device based on the ratio between the first spectrophotometric measurement and the second spectrophotometric measurement.

19. A method for labeling a plurality of objects in an image to prepare training data, the method comprising: Receiving, by a processing device of the image system according to claim 1, an image in a graphical user interface; The selection of a position in the image is received by the processing device through the graphical user interface; The bounding box around the position is calculated by the processing device based on a plurality of pixels local to the position; The display of the bounding box superimposed on the image in the graphical user interface is provided by the processing device; and In response to receiving user confirmation, the region within the bounding box is marked as training data.

20. A method for preparing a measurement image, the method comprising: Providing a sample card including a plurality of marker elements; Depositing a sample on a first plate of the sample card in an open configuration; Closing the sample card to press a second plate of the sample card against the first plate to a closed configuration, wherein in the closed configuration the first plate and the second plate form a thin layer of substantially uniform thickness including the sample and the plurality of marker elements; and Providing the image system according to claim 2, which, when instructions are executed by the processing device, is configured to: Capture an image of the sample card, which includes a sample of substantially uniform thickness and a plurality of marker elements; Detect the plurality of marker elements in the image; Compare the detected plurality of marker elements with a monitoring standard associated with the sample card to determine a geometric mapping between the plurality of marker elements and the monitoring standard; The non-ideal factor of the image system is determined based on the geometric mapping; and The image of the sample card is processed to correct the non-ideal factor.

21. A method for preparing a measurement image, the method comprising: Providing a sample card including a plurality of marker elements; Deposit a sample on a first plate of the sample card in an open configuration; Close the sample card to press a second plate of the sample card against the first plate to a closed configuration, wherein in the closed configuration the first plate and the second plate form a thin layer of substantially uniform thickness containing the sample and the plurality of marker elements; and Provide the image system of claim 3, for use when instructions are executed by the processing device to: Capture an image of the sample card, which contains a sample of substantially uniform thickness and a plurality of marker elements; Divide the image into a plurality of sub-regions; A machine learning model and a plurality of marked elements are used to determine whether each of the plurality of sub-regions meets the requirements of the image system; In response to determining that the sub-region does not meet the requirements, the first sub-region is marked as non-compliant; In response to determining that the sub-region meets the requirements, the first sub-region is marked as compliant; and The assay analysis is performed using the compliant sub-region of the image.

22. A method for correcting non-ideal factors of an image system, the method comprising: Receive, by a processing device of the image system of claim 4, an image of a sample card that contains a substantially uniform sample layer deposited on a plate of the sample card and the plurality of marker elements associated with the sample card; Detect, by the processing device, the plurality of marker elements in the image; Compare the detected multiple marker elements with the monitoring criteria associated with the sample card to determine the geometric mapping between the multiple marker elements and the monitoring criteria; Determine the non-ideal factor of the imaging system based on the geometric mapping; And Process the image of the sample card to correct the non-ideal factor.

23. A method for correcting non-ideal factors of an image system, the method comprising: receiving, by a processing device of the image system of claim 3, an image of a sample card that contains a substantially uniform sample layer deposited on a plate of the sample card and the plurality of marker elements associated with the sample card; Divide, by the processing device, the image into a plurality of sub-regions; Use a machine learning model and multiple marker elements by the processing device to determine whether each of the multiple sub-regions meets the requirements of the imaging system; In response to determining that the sub-region does not meet the requirements, mark the first sub-region as non-compliant by the processing device; In response to determining that the sub-region meets the requirements, mark the first sub-region as compliant by the processing device; And Perform assay analysis using the compliant sub-regions of the image by the processing device.

24. An intelligent determination monitoring method, comprising: Receive, by a processing device of the image system of claim 1, an image encoding first information about a biological sample deposited in a sample card and second information about a plurality of monitoring markers; Perform a first machine learning model on the image by the processing device to determine the measurement of geometric features associated with multiple monitoring markers; Determine the variation between the measurement of the geometric feature and the ground truth of the geometric feature of the provided sample card by the processing device; Correct the image encoding the first information and the second information by the processing device based on the variation; And Determine the biological characteristics of the biological sample by the processing device using the corrected image.

25. The method according to claim 24, wherein the sample card comprises a first plate, a plurality of columns integrally formed substantially perpendicular to the surface of the first plate, and a second plate capable of surrounding the first plate to form a thin layer in which the biological sample is deposited.

26. The method according to claim 25, wherein a plurality of monitoring markers correspond to the plurality of columns.

27. The method according to claim 26, wherein the plurality of monitoring markers are provided in at least one of the first plate or the second plate.

28. The method according to claim 24, wherein the performing, by the processing device, of a first machine learning model on the image to determine a measurement of the geometric feature associated with the plurality of monitoring markers further comprises: identifying, by the processing device executing the first machine learning model, the plurality of monitoring markers from the image; and determining, by the processing device, a measurement of the geometric feature based on the identified plurality of monitoring markers.

29. The method according to claim 24, wherein the determining, by the processing device, of a variation between the measurement of the geometric feature and a ground truth of the geometric feature of the sample card provided further comprises: determining, by the processing device, one of a systematic error or an operator error; and presenting, on a display device associated with the processing device, one of the determined systematic error or operator error.

30. A method for correcting non-ideal factors in a determination image of a thin-layer sample, the method comprising: providing a sample card comprising a monitoring standard, the sample card comprising a plurality of nanostructures on a plate of the sample card; depositing a sample on a plate of the sample card; and providing the image system according to claim 4, which, when instructions are executed by the processing device, is configured to: capture an image of the sample card comprising the sample and the monitoring standard; determine a non-ideal factor of the image system by comparing the image of the sample card with a plurality of geometric values of the monitoring standard determined during the manufacture of the sample card; and correct the image of the sample card in consideration of the non-ideal factor.

31. The method according to claim 30, wherein the machine learning model is further trained by comparing an image of the sample card with geometric values of a monitoring standard determined during the manufacture of the sample card.

32. The method according to claim 29, comprising: receiving, by a processing device of the image system, an image and a first parameter associated with a sample card comprising a sample and a monitoring standard, wherein the monitoring standard comprises a plurality of nanostructures on a plate of the sample card, and wherein the sample is deposited on the plate; determining, by the processing device, a system error of the image system by comparing the first parameter with a second parameter associated with the monitoring standard determined during the manufacture of the sample card; and correcting the image of the sample card in consideration of the system error.

33. The method according to claim 29, further comprising: Receive an image of a sample card containing a sample and monitoring criteria by a processing device of an imaging system, wherein the monitoring criteria includes multiple nanostructures on the plate of the sample card, and wherein the sample is deposited on the plate; Use a machine learning model by the processing device to determine the manual operation error reflected in the image by comparing the image of the sample card with multiple geometric values of the monitoring criteria determined during the manufacture of the sample card, wherein the manual operation error includes incorrect processing of the imaging system; And correct the image of the sample card by removing the manual operation error reflected in the image.

Images