Efficient training and accuracy improvement of imaging-based assays

Abstract

The present disclosure relates to apparatus, devices, and methods for improving the accuracy of image-based assays using imaging systems that have uncertainties or deviations (defects) compared to an ideal imaging system. One aspect of the invention is to add monitoring markers to a sample holder, wherein at least one of the geometric and / or optical properties of the monitoring markers is predetermined and known, and to capture an image of the sample with the monitoring markers, and to use the images with the monitoring markers to train a machine learning model.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application No. 62 / 940,242, filed on November 25, 2019, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to apparatus, devices, and methods for improving the accuracy of image-based determinations using imaging systems that have uncertainties or deviations (imperfections) compared to an ideal imaging system. Background Art

[0004] In image-based biological / chemical sensing and assays (e.g., immunoassays, nucleotide assays, blood cell counting, etc.), low-quality imaging systems are often used to achieve low-cost and / or portable systems. However, low-quality imaging systems may have defects (deviations from ideality) in the optical components, mechanical systems, and / or electrical systems. Such defects can significantly affect assay accuracy.

[0005] To improve the accuracy of image-based assays using low-quality imaging systems, machine learning methods can be used. However, with machine learning alone, imperfections in the imaging system create many variables, requiring long training times, large training samples, and artifacts in the imaging (thus creating new errors). One aspect of the present invention is to add monitoring markers to a sample holder, wherein at least one of the geometric and / or optical properties of the monitoring markers is predetermined and known, and to capture images of the sample with the monitoring markers, and to use the images with the monitoring markers to train a machine learning model. Summary of the Invention

[0006] One aspect of the present invention is to add monitoring marks on the sample holder, wherein at least one of the geometric and / or optical properties of the monitoring marks is predetermined and known, and to take images of the sample with the monitoring marks, and to use the images with the monitoring marks to train a machine learning model.

[0007] In some embodiments, the present invention provides a method for training a machine learning model for an image-based assay, wherein the assay is imaged by a low-quality imaging system during testing, the method comprising the steps of: forming a thin layer of a first sample on an imaging area of ​​a first sample holder, wherein the first sample holder is a labeled sample holder, and the labeled sample holder includes one or more monitoring marks on the imaging area; forming a thin layer of a second sample on an imaging area of ​​a second sample holder, wherein the second sample holder is a labeled sample holder, and the labeled sample holder includes one or more monitoring marks on the imaging area of ​​the second sample holder that are identical to the one or more monitoring marks on the first sample holder; using the low-quality imaging system Imaging a first image of the sample on an imaging area of ​​the first sample holder; imaging a second image of the sample on an imaging area of ​​the second sample holder using a high-quality imaging system; correcting defects in the first image using the monitoring mark to generate a first corrected image; correcting defects in the second image using the monitoring mark, and if the second image has defects, generating a second corrected image; and training a machine learning model using the first corrected image, the second corrected image, and the monitoring mark to generate a trained model, wherein the geometric characteristics and optional optical characteristics of the one or more monitoring marks imaged under an ideal imaging system are predetermined and known; wherein the low-quality imaging system contains more defects than the high-quality imaging system.

[0008] In some embodiments, the method includes forming a thin layer of a third sample on an imaging area of ​​a third sample holder, wherein the third sample holder is a marked sample holder comprising one or more monitoring marks on the imaging area of ​​the third sample holder that are identical to the one or more monitoring marks on the first sample holder; imaging a third image of the sample on the imaging area of ​​the third sample holder using a low-quality imaging system; correcting defects in the third image using the monitoring marks to generate a third corrected image; and analyzing the transformed third corrected image using the machine learning model trained in claim 1 and generating a measurement result.

[0009] In some embodiments, the machine learning model comprises a Cycle Generative Adversarial Network (CycleGAN).

[0010] In some embodiments, the machine learning model includes a cyclic generative adversarial network (CycleGAN), the cyclic generative adversarial network (CycleGAN) includes a forward generative adversarial network (forward GAN) and a backward GAN, wherein the forward GAN includes a first generator and a first discriminator, and the backward GAN includes a second generator and a second discriminator, and wherein, training the machine learning model using each transformed region in the first image and each transformed region in the second image includes: training the CycleGAN using each transformed region in the first image and each transformed region in the second image aligned at four structural elements at the four corners of the corresponding region.

[0011] In some embodiments, the first sample and the second sample are the same sample, and the first sample holder and the second sample holder are identical.

[0012] In some embodiments, the present invention provides a method for training a machine learning model for image-based determination, the method comprising the following steps: receiving a first image of a sample holder containing a sample captured by a first optical sensor, wherein the sample holder is manufactured with standard patterned structural elements at predetermined positions; identifying a first region in the first image based on the positions of one or more structural elements in the patterned structural elements in the first image; determining a spatial transformation associated with the first region based on a mapping between the positions of the one or more structural elements in the first image and the predetermined positions of one or more structural elements in the sample holder; applying the spatial transformation to the first region in the first image to calculate the transformed first region; and training the machine learning model using the transformed first image.

[0013] In some embodiments, the sample holder comprises a first plate, a second plate, and the patterned structural element, and wherein the patterned structural element comprises posts embedded at predetermined locations on at least one of the first plate or the second plate.

[0014] In some embodiments, the method further includes the following steps: detecting the position of the patterned structural element in the first image; dividing the first image into regions including the first area, wherein each of the regions is bounded by four structural elements at the four corners of the corresponding area; determining a corresponding spatial transformation associated with each of the regions in the first image based on a mapping between the positions of the four structural elements at the four corners of the corresponding area and four predetermined positions of the four structural elements in the sample holder; applying the corresponding spatial transformation to each region in the first image, thereby calculating a corresponding transformed region in the first image; and using each transformed region in the first image to train the machine learning model, wherein the trained machine learning model is used to transform the measurement image from low resolution to high resolution.

[0015] In some embodiments, the predetermined positions of the patterned structure elements are periodically distributed with at least one periodic value, and wherein detecting the positions of the patterned structure elements in the first image comprises: detecting the positions of the patterned structure elements in the first image using a second machine learning model; and correcting an error at the detected positions of the patterned structure elements in the first image based on the at least one periodic value.

[0016] In some embodiments, the method further includes the following steps: receiving a second image of the sample holder captured by a second optical sensor, wherein the first image is captured at a first quality level and the second image is captured at a second quality level higher than the first quality level; dividing the second image into a plurality of regions, wherein each of the regions in the second image is bounded by four structural elements at four corners of the corresponding region in the second image and matches the corresponding region in the first image; determining a second spatial transformation associated with the region in the second image based on a mapping between the positions of the four structural elements at the four corners of the corresponding region in the second image and four predetermined positions of the four structural elements in the sample holder; applying the second spatial transformation to each of the regions in the second image to calculate a corresponding transformed region in the second image; and using each transformed region in the first image and each transformed region in the second image to train the machine learning model to transform the first quality level image into the second quality level image.

[0017] In some embodiments, the machine learning model includes a cyclic generative adversarial network (CycleGAN), the cyclic generative adversarial network (CycleGAN) includes a forward generative adversarial network (forward GAN) and a backward GAN, wherein the forward GAN includes a first generator and a first discriminator, and the backward GAN includes a second generator and a second discriminator, and wherein training the machine learning model using each transformed region in the first image and each transformed region in the second image includes: training the CycleGAN using each transformed region in the first image and each transformed region in the second image aligned at four structural elements at the four corners of the corresponding region.

[0018] In some embodiments, training the machine learning model using each transformed region in the first image and each transformed region in the second image includes: training the first generator and the first discriminator by providing each transformed region in the first image to the forward GAN; training the second generator and the second discriminator by providing each transformed region in the second image to the backward GAN; and optimizing the forward and backward GAN training under cycle consistency constraints.

[0019] In some embodiments, the present invention provides a method for converting a measurement image using a machine learning model, the method comprising the following steps: receiving a first image of a sample holder containing a sample captured by a first optical sensor, wherein the sample holder is manufactured with standard patterned structural elements at predetermined positions; identifying a first region in the first image based on the positions of one or more structural elements in the patterned structural elements in the first image; determining a spatial transformation associated with the first region based on a mapping between the positions of the one or more structural elements in the first image and the predetermined positions of one or more structural elements in the sample holder; applying the spatial transformation to the first region in the first image to calculate a transformed first region; and applying the machine learning model to the transformed first region in the first image to produce a second region.

[0020] In some embodiments, the method further includes the following steps: dividing the first image into multiple regions based on the positions of the one or more structural elements in the patterned structural elements in the first image, wherein the multiple regions include the first region; determining a corresponding spatial transformation associated with each of the multiple regions; applying the corresponding spatial transformation to each of the multiple regions in the first image to calculate a transformed region; applying the machine learning model to each of the transformed regions in the first image to produce a transformed region of a second quality level; and combining the transformed regions to form a second image.

[0021] In some embodiments, the present invention provides an image-based measurement system comprising: a database system for storing images; and a processing device communicatively coupled to the database system to: receive a first image of a sample holder containing a sample captured by a first optical sensor, wherein the sample holder is fabricated with standard patterned structural elements at predetermined positions; identify a first region in the first image based on the positions of one or more of the patterned structural elements in the first image; determine a spatial transformation associated with the first region based on a mapping between the positions of the one or more structural elements in the first image and the predetermined positions of one or more structural elements in the sample holder; apply the spatial transformation to the first region in the first image to calculate the transformed first region; and train the machine learning model using the transformed first image.

[0022] In some embodiments, the sample holder comprises a first plate, a second plate, and the patterned structural element, and wherein the patterned structural element comprises posts embedded at predetermined locations on at least one of the first plate or the second plate.

[0023] In some embodiments, the processing device is further used to: detect the position of the patterned structural element in the first image; divide the first image into regions including the first area, wherein each of the regions is defined by four structural elements at the four corners of the corresponding area; determine the corresponding spatial transformation associated with each of the regions in the first image based on a mapping between the positions of the four structural elements at the four corners of the corresponding area and the four predetermined positions of the four structural elements in the sample holder; apply the corresponding spatial transformation to each region in the first image, thereby calculating the corresponding transformed region in the first image; and use each transformed region in the first image to train the machine learning model, wherein the trained machine learning model is used to transform the measurement image from low resolution to high resolution.

[0024] In some embodiments, the predetermined positions of the patterned structure elements are periodically distributed with at least one periodic value, and wherein the positions of the patterned structure elements in the first image are detected, and the processing device is further used to: detect the positions of the patterned structure elements in the first image using a second machine learning model; and correct errors in the detected positions of the patterned structure elements in the first image based on the at least one periodic value.

[0025] In some embodiments, the processing device is further used to: receive a second image of the sample holder captured by a second optical sensor, wherein the first image is captured at a first quality level and the second image is captured at a second quality level higher than the first quality level; separate the second image into a plurality of regions, wherein each of the regions in the second image is defined by four structural elements at four corners of the corresponding region in the second image and matches the corresponding region in the first image; determine a second spatial transformation associated with the region in the second image based on a mapping between the positions of the four structural elements at the four corners of the corresponding region in the second image and four predetermined positions of the four structural elements in the sample holder; apply the second spatial transformation to each of the regions in the second image to calculate a corresponding transformed region in the second image; and use each transformed region in the first image and each transformed region in the second image to train the machine learning model to transform the first quality level image into the second quality level image.

[0026] In some embodiments, the machine learning model includes a cyclic generative adversarial network (CycleGAN), the cyclic generative adversarial network (CycleGAN) includes a forward generative adversarial network (forward GAN) and a backward GAN, wherein the forward GAN includes a first generator and a first discriminator, and the backward GAN includes a second generator and a second discriminator, and wherein training the machine learning model using each transformed region in the first image and each transformed region in the second image includes: training the CycleGAN using each transformed region in the first image and each transformed region in the second image aligned at four structural elements at the four corners of the corresponding region.

[0027] In some embodiments, in order to train the machine learning model using each transformed region in the first image and each transformed region in the second image, the processing device is also used to: train the first generator and the first discriminator by providing each transformed region in the first image to the forward GAN; and train the second generator and the second discriminator by providing each transformed region in the second image to the backward GAN.

[0028] In some embodiments, the present invention provides an image-based measurement system for converting measurement images using a machine learning model, comprising: a database system for storing images; and a processing device communicatively coupled to the database system to: receive a first image of a sample holder containing a sample captured by a first optical sensor, wherein the sample holder is manufactured with standard patterned structural elements at predetermined positions; identify a first region in the first image based on the position of one or more structural elements in the patterned structural elements in the first image; determine a spatial transformation associated with the first region based on a mapping between the position of the one or more structural elements in the first image and the predetermined positions of one or more structural elements in the sample holder; apply the spatial transformation to the first region in the first image to calculate the transformed first region; and use the transformed first image to train the machine learning model.

[0029] In some embodiments, the processing device is further used to: separate the first image into multiple regions based on the positions of the one or more structural elements in the patterned structural elements in the first image, wherein the multiple regions include the first region; determine a corresponding spatial transformation associated with each of the multiple regions; apply the corresponding spatial transformation to each of the multiple regions in the first image to calculate a transformed region; apply the machine learning model to each of the transformed regions in the first image to generate a transformed region of a second quality level; and combine the transformed regions to form a second image.

[0030] In some embodiments, the sample holder has a plate, and the sample contacts a surface of the plate.

[0031] In some embodiments, the sample holder has two plates, and the sample is located between the two plates.

[0032] In some embodiments, the sample holder has two plates that are movable relative to each other, and the sample is located between the two plates.

[0033] In some embodiments, the sample holder has two plates that are movable relative to each other, and the sample is positioned between the two plates. A plurality of spacers are attached to at least one interior opposing surface of at least one or both of the plates, and the plurality of spacers are positioned between the opposing plates. The sample thickness is adjusted by the spacers.

[0034] In some embodiments, there is at least one spacer within the sample.

[0035] In some embodiments, the spacer is a monitoring marker.

[0036] In certain embodiments, the two plates of the device are initially on top of each other and need to be separated to enter an open configuration for sample deposition.

[0037] In some embodiments, the two plates of the device are already in a closed configuration before sample deposition.The sample enters the device from the gap between the two plates.

[0038] In some embodiments, the thickness of the sample layer is 0.1 μm, 0.5 μm, 1 μm, 2 μm, 3 μm, 5 μm, 10 μm, 50 μm, 100 μm, 200 μm, 500 μm, 1000 μm, 5000 μm, or a range between any two of the values.

[0039] In some embodiments, the preferred thickness of the sample layer is 1 μm, 2 μm, 5 μm, 10 μm, 30 μm, 50 μm, 100 μm, 200 μm, or a range between any two of the values.

[0040] In some embodiments, the spacing between two monitoring markers is 1 μm, 2 μm, 3 μm, 5 μm, 10 μm, 50 μm, 100 μm, 200 μm, 500 μm, 1000 μm, 5000 μm, or a range between any two of the values.

[0041] In certain embodiments, the preferred spacing between the two monitoring markers is 10 μm, 50 μm, 100 μm, 200 μm, or a range between any two of the aforementioned values.

[0042] In some embodiments, the spacing between two monitoring markers is 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, 90%, 100% of the lateral dimension of the imaging area, or within a range between any two of the stated values.

[0043] In certain embodiments, the preferred spacing between two monitoring markers is 30%, 50%, 80%, 90% of the lateral dimension of the imaging region, or a range between any two of these values.

[0044] In certain embodiments, the average size of the monitoring markers is 1 μm, 2 μm, 3 μm, 5 μm, 10 μm, 50 μm, 100 μm, 200 μm, 500 μm, 1000 μm, or a range between any two of these values.

[0045] In certain embodiments, the preferred average size of monitoring markers is 5 μm, 10 μm, 50 μm, 100 μm, or a range between any two of these values.

[0046] In certain embodiments, the average size of the monitoring markers is 1%, 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80% of the size of the imaging area, or a range between any two of these values.

[0047] In certain embodiments, the preferred average size of the monitoring markers is 1%, 10%, 20%, 30% of the size of the imaging area, or a range between any two of these values.

[0048] In certain embodiments, the spacer is a monitoring marker having a height of 0.1 μm, 0.5 μm, 1 μm, 2 μm, 3 μm, 5 μm, 10 μm, 50 μm, 100 μm, 200 μm, 500 μm, 1000 μm, 5000 μm, or a range between any two of the recited values.

[0049] In certain embodiments, the spacers are monitoring markers preferably having a height of 1 μm, 2 μm, 5 μm, 10 μm, 30 μm, 50 μm, 100 μm, 200 μm or a range between any two of these values.

[0050] In certain embodiments, the spacer is a monitoring marker having a height that is 1%, 5%, 10%, 20%, 40%, 60%, 80%, 100% of the height of the sample layer, or a range between any two of the stated values.

[0051] In certain embodiments, the spacer is a monitoring marker having a preferred height of 50%, 60%, 80%, 100% of the sample layer height or a range between any two of said values.

[0052] In certain embodiments, the shape of the monitoring marker is selected from the group consisting of circle, polygon, perfect circle, square, rectangle, oval, ellipse, or any combination thereof.

[0053] In certain embodiments, the monitoring marker has a pillar shape and has a substantially flat top surface covering at least 10% of the top projected area of ​​the marker.

[0054] In certain embodiments, where the distance between monitoring markers is periodic, conventional image-based assays can employ high-precision microscopes equipped with imaging systems to capture assay images. These types of optical microscopes may adhere to the objective zoom rule, according to which the field of view (FoV) is inversely proportional to magnification. According to this rule, images used for assays with higher magnification have smaller FoVs, while images with larger FoVs have smaller magnifications. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The present disclosure will be more fully understood from the detailed description given below and from the accompanying drawings of various embodiments of the present disclosure. However, the accompanying drawings should not be used to limit the present disclosure to specific embodiments, but are only for explanation and understanding.

[0056] Figure 1 The training phase is shown: the sample is placed on the same type of labeled sample holder (MSH), and low-quality (LQ) and high-quality (HQ) imaging systems are used to capture LQ and HQ images of the sample in the MMH. The images in the two domains are corrected using the monitor marker (MM) to finalize the training data. The machine learning model is trained based on the training data of the image transformation model G.

[0057] Figure 2 The same process as in the training phase Route A is followed to prepare the monitor-marked corrected low-quality image (MMC-LQI). The MMC-LQI is then fed to the model G to obtain the final result.

[0058] Figure 3 Shown Figure 2 A detailed description of the training phase in Box 5 is given in the figure.

[0059] Figure 4 An image-based assay system according to an embodiment of the present disclosure is shown that can capture assay images and process the images to train a machine learning model.

[0060] Figure 5 A flowchart of a method for preparing high-quality measurement images in a training dataset according to an embodiment of the present disclosure is depicted.

[0061] Figure 6 The construction of a Cycle Generative Adversarial Network (CycleGAN) model for determining image-to-image translation according to an embodiment of the present disclosure is shown.

[0062] Figure 7 A flow chart of a method of enhancing low-quality measurement images according to an embodiment of the present disclosure is depicted.

[0063] Figure 8A flowchart of a method for preparing a training dataset for a machine learning model according to an embodiment of the present disclosure is depicted.

[0064] Figure 9 A block diagram depicts a computer system operating in accordance with one or more aspects of the present invention.

[0065] Figure 10 (A) shows that the monitoring mark in the present invention actually adds the known meta-structure to the grid. When the grid is distorted, people can always use the knowledge of the meta-structure to restore it to a near-perfect structure, thus greatly improving the accuracy of the machine learning model and the accuracy of the measurement.

[0066] Figure 11 shows the results of training machine learning without and with monitoring labels ( Figure 10 The difference between (a0) is that without using monitoring markers, many samples and a long time are required to train, and the test produces artifacts (missing cells and generating non-existent cells), while when using monitoring markers, the number of training samples and time are significantly reduced, and the test does not produce artifacts. DETAILED DESCRIPTION

[0067] definition

[0068] The term "labeled sample holder" or "MSH" refers to a sample holder having a monitoring label.

[0069] The term "monitoring mark" or "MM" refers to one or more structures on a sample holder, wherein at least one geometric property of the one or more structures is predetermined and known when viewed in an ideal imaging system, wherein the geometric property comprises the size and / or orientation of the one or more structures, and / or the position between two structures. In some embodiments, in addition to the predetermined and known geometric property, the one or more structures also have predetermined and known optical properties, wherein the optical properties comprise light transmission, absorption, reflection, and / or scattering.

[0070] The term "identical labeled sample holder" refers to a second labeled sample holder that is precisely manufactured to have the same labeling structure as the first labeled sample holder.

[0071] The term "imaging system" refers to a system for capturing images. An imaging system comprises optical components, including an imager (which captures an image), light illumination, lenses, filters, and / or mirrors; a mechanical system, including a mechanical stage, a scanner, and / or a mechanical holder; and an electrical system, including a power supply, wiring, and / or electronics.

[0072] The term "defect" refers to a deviation from the ideal, where the defect may be random or non-random, and / or time-dependent.

[0073] The term "corrected monitoring mark" or "MC" refers to a processed image from an original image in which one or more defects (if present) in the original image are corrected using:

[0074] The term "correction" refers to calculation using an algorithm.

[0075] The terms "machine learning model" and "algorithm" are used interchangeably.

[0076] The terms "transformed image using monitoring markers" and "corrected image using monitoring markers" are interchangeable.

[0077] The terms "optical sensor" and "imaging system" in the figures are interchangeable.

[0078] In certain embodiments, the "standard patterned structural element at a predetermined position" is a monitoring mark.

[0079] Imaging system deficiencies are defined as imperfections in the following elements of the system:

[0080] Optical components and conditions, including but not limited to optical attenuators, beam splitters, depolarizers, apertures, diffraction beam splitters, diffusers, ground glass, lenses, Littrow prisms, multifocal diffraction lenses, nanophotonic resonators, nullers, optical circulators, optical isolators, optical microcavities, photonic integrated circuits, pinholes (optical devices), polarizers, primary mirrors, prisms, q-plates, retroreflectors, spatial filters, spatial light modulators, virtual image phased arrays, waveguides (optical devices), wave plates, and zone plates;

[0081] Lighting conditions, including but not limited to light source intensity, light source spectrum, light source color, light source direction, light source brightness, light source contrast, light source wavelength bandwidth, light source coherence, light source phase, and light source polarization;

[0082] Image sensor and photodetector components and conditions, including but not limited to exposure time, color separation, resolution, ISO, noise level, and sensitivity of CCD (charge-coupled device) and CMOS (complementary metal oxide semiconductor) image sensors;

[0083] Mechanical components and conditions, including but not limited to sample holder (flatness, parallelism, surface roughness, distance to lens), scanning system (flatness, parallelism, stiffness, resolution, travel), material stability, material thermal expansion;

[0084] Imagery from imaging system conditions, including but not limited to spherical distortion, noise level, resolution, brightness distribution, contrast distribution, color distribution, temperature distribution, hue distribution, saturation, brightness, rotation, artifacts;

[0085] All of the above factors and conditions are time dependent.

[0086] The term "imaging area" of a sample holder refers to the area of ​​the sample holder that is to be imaged by the imager.

[0087] In some embodiments, the first image and the second image are more than one image.

[0088] The term "geometric characteristics" of one or more monitoring markers refers to shape, size, distance, relative position, total number, number density, area density, rotation, symmetry, periodicity, etc.

[0089] The term "optical property" of one or more monitoring markers refers to transmission, absorption, reflection, fluorescence, scattering, phase contrast, polarization, chromatography, diffusion, phase change, brightness, intensity contrast, Raman scattering, nonlinear harmonic light generation, electroluminescence, radiation, IR spectroscopy, spontaneous emission, stimulated emission, etc.

[0090] The term "imaging area" of a sample holder refers to the area of ​​the sample holder that is to be imaged by the imager.

[0091] In some embodiments, the spacing between two monitoring markers is 1 μm, 2 μm, 3 μm, 5 μm, 10 μm, 50 μm, 100 μm, 200 μm, 500 μm, 1000 μm, 5000 μm, or a range between any two of the values.

[0092] In certain embodiments, the preferred spacing between two monitoring markers is 10 μm, 50 μm, 100 μm, 200 μm, or a range between any two of the aforementioned values.

[0093] In some embodiments, the spacing between two monitoring markers is 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, 90%, 100% of the lateral dimension of the imaging area, or within a range between any two of the stated values.

[0094] In certain embodiments, the preferred spacing between two monitoring markers is 30%, 50%, 80%, 90% of the lateral dimension of the imaging region, or a range between any two of these values.

[0095] In certain embodiments, the average size of the monitoring markers is 1 μm, 2 μm, 3 μm, 5 μm, 10 μm, 50 μm, 100 μm, 200 μm, 500 μm, 1000 μm, or a range between any two of these values.

[0096] In certain embodiments, the preferred average size of monitoring markers is 5 μm, 10 μm, 50 μm, 100 μm, or a range between any two of these values.

[0097] In certain embodiments, the average size of the monitoring markers is 1%, 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80% of the size of the imaging area, or a range between any two of these values.

[0098] In certain embodiments, the preferred average size of the monitoring markers is 1%, 10%, 20%, 30% of the size of the imaging area, or a range between any two of these values.

[0099] Traditional image-based assays employ high-precision microscopes equipped with imaging systems to capture assay images. These types of optical microscopes are subject to the objective zoom rule, according to which the field of view (FoV) is inversely proportional to magnification. According to this rule, images with higher magnifications used for assays have smaller FoVs, while images with larger FoVs have smaller magnifications.

[0100] These high-precision microscopes are typically expensive and bulky. Therefore, they are typically used in laboratory environments handled by specialized operators (e.g., pathologists). In contrast, image-based assays, in which images are captured by cameras already present in large mobile devices (e.g., smartphones), can provide a low-cost solution for areas such as the low-cost collection and analysis of blood samples from the public. This can be particularly useful in certain healthcare situations, such as point-of-care (POC) settings where a large number of assays need to be processed quickly and economically. However, assay images captured using low-cost mobile devices are often of low quality, containing variations caused by the wide range of imaging systems used to capture the images. Factors such as the resolution and magnification of different cameras, as well as distortion from the imaging system or lens behind the camera, can vary widely, severely limiting the accuracy and reliability of image-based assays based on these mobile devices.

[0101] Therefore, there is a need to improve the low-quality images in image-based assays captured by such devices to a level comparable to images captured by high-precision imaging systems, such as microscopes used in laboratory environments. Improved image-based assays using such devices can overcome the limitations of objective lens scaling rules and provide accurate image-based assays, particularly using low-cost commodity optical components commonly used in medicine, healthcare, POC (point-of-care), chemistry, or biology.

[0102] In this disclosure, the term "system L" may refer to a low-quality image system (e.g., due to inconsistencies in illumination / camera optics / imaging system sensor array (e.g., photodetector array), spherical distortion, high noise levels in captured images, lower magnification, lower resolution, etc.).

[0103] The term "System H" refers to high-quality imaging systems that meet regulations currently deployed in commercial environments. System H typically has better image quality (e.g., lower noise levels in captured images, higher magnification, higher resolution, etc.) than System L (e.g., in terms of top-of-the-line optical systems with specialized lighting / camera optics / imaging system sensor arrays).

[0104] In some embodiments, each of the factors such as the light source, optics, and imaging system may also vary across different systems L. Such variations may be caused by a lack of calibration between different types and / or individual systems L. The statistical distribution of these variations in different systems L may also vary. In order to improve the performance of the system L, a machine learning model may be employed to compensate for variations in a particular system L. A machine learning model, such as a neural network model, may be custom trained for each individual system L. The customized machine learning model may then be used to process images captured by the corresponding individual system L during an assay. While the customized machine learning model may improve the performance of each individual system L, it is not suitable for deployment in the mass market because such training of each individual system L (also referred to as a "device" or "assay device") is inefficient, time consuming, and expensive, and is therefore not practical in real-world applications.

[0105] To overcome the above and other deficiencies in current implementations, embodiments of the present disclosure provide technical solutions that may include a sample holder fabricated using standard patterned structural elements at predetermined locations, and an image enhancement method that can mitigate variations caused by individual systems L based on the standard patterned structural elements fabricated in the sample holder, thereby producing high-quality image-based measurements. The image enhancement method can include employing a machine learning model and training the machine learning model using standard characteristics in an environment containing measurement images of a sample in such a sample holder. The standard characteristics can include geometric characteristics and / or material characteristics of the patterned structural elements.

[0106] Embodiments of the present disclosure may provide a method for training a machine learning model for image-based determination. The method may include: receiving a first image of a sample holder containing a sample captured by a first imaging sensor, wherein the sample holder is manufactured with standard patterned structural elements at predetermined positions; identifying a first region in the first image based on the position of one or more structural elements in the patterned structural elements in the first image; determining a spatial transformation associated with the first region based on a mapping between the position of the one or more structural elements in the first image and the predetermined position of the one or more structural elements in the sample holder; applying the spatial transformation to the first region in the first image to calculate a transformed first region; and training the machine learning model using the transformed first image.

[0107] Embodiments of the present disclosure may also include a method for processing an assay image using a machine learning model trained using the above method. The method for converting an assay image using a machine learning model may include: receiving a first image of a sample holder containing a sample captured by a first imaging system, wherein the sample holder is fabricated with standard patterned structural elements at predetermined positions; identifying a first region in the first image based on the position of one or more structural elements in the patterned structural elements in the first image; determining a spatial transformation associated with the first region based on a mapping between the position of the one or more structural elements in the first image and the predetermined position of the one or more structural elements in the sample holder; applying the spatial transformation to the first region in the first image to calculate the transformed first region; and applying the machine learning model to the first image to produce a second image.

[0108] Thus, embodiments of the present disclosure provide a system and method that can eliminate or substantially eliminate variations in measurement images captured by system L using a standard of a sample holder represented in the measurement image as a reference point. These embodiments can include converting the image into regions in a real-dimensional space based on patterned structural elements in the standard, and further training a machine learning model using these regions in the real-dimensional space. In this way, embodiments of the present disclosure can use such a trained machine learning model to enhance images produced by system L to a quality level comparable to images produced by system H, with little to no impact on variations caused by the respective systems L.

[0109] Figure 4 An image-based measurement system according to an embodiment of the present disclosure is shown, which can capture measurement images and process the images to train a machine learning model. Figure 4An image-based assay system 1 may include a computing system 2, an imaging system 3 (e.g., a smartphone camera with an adapter), and a sample holder device 4. The imaging system 3 may include a built-in light source, a lens, and an imaging system (e.g., a photodetector array). The imaging system 3 may be associated with a computing device (e.g., a mobile smartphone) for capturing assay images. Each imaging system 3 associated with a separate computing device may have its own individual variations compared to other imaging systems.

[0110] like Figure 4 The illustrated computing system 2 may be a standalone computer or a networked computing resource implemented in a computing cloud. The computing system 2 may include one or more processing devices 102 , storage devices 104 , and interface devices 106 , wherein the storage devices 104 and the interface devices 106 are communicatively coupled to the processing device 102 .

[0111] The processing device 102 may be a hardware processor, such as a central processing unit (CPU), a graphics processing unit (GPU), or an accelerator circuit. The interface device 106 may be a display such as a touch screen of a desktop, laptop, or smartphone. The storage device 104 may be a storage device, a hard disk, or cloud storage 110 connected to the computing system 2 via a network interface card (not shown). The processing device 102 may be a programmable device that can be programmed to implement the machine learning model 108. Implementation of the machine learning model 108 may include training the model and / or applying the trained model to image-based measurement data.

[0112] The imaging system 3 in the image-based measurement system 1 can be an imaging system of a system L. For example, the imaging system 3 can include a built-in image sensing photodetector array of a smart phone available in the consumer market. The image-based measurement system 1 can also include a sample holder device 4 for holding a sample 6 (e.g., a biological sample or a chemical sample) therein. The sample holder device 4 can be a QMAX card described in detail in International Application No. PCT / US2016 / 046437. The sample holder device 4 can include two plates that are parallel to each other when in a closed configuration, wherein at least one of the two plates is transparent. The imaging system 3 can be used to capture a measurement image of the sample 6 contained in the sample holder device 4 through the transparent plate, so that the computing system 2 can analyze the sample based on the measurement image.

[0113] Measurement images captured by the imaging system 3 of system L are typically of low quality and unsuitable for further analysis by expert operators or computer analysis programs. This low quality can be reflected in high levels of noise, distortion, and low resolution in the captured images. A machine learning model 108 can be used to enhance the quality of the measurement images. However, as described above, the machine learning model 108 can be ineffective when applied directly to the measurement images because the characterizing parameters of the imaging system 3 used to capture the measurement images (e.g., Field of View, magnification, or resolution) can vary widely. To mitigate variations between imaging systems 3 of different systems L, in one embodiment, the sample holder assembly 4 can be fabricated using patterned structural elements of a standard 5. These patterned structural elements of the standard 5 are precisely fabricated on the inner surface of at least one plate of the sample holder assembly 4. Therefore, the positions of these patterned structural elements on the inner surface are precise and consistent in real-dimensional space. In one embodiment, the structural elements can have different optical properties than the sample 6. Therefore, when captured together with the sample 6, these structural elements can provide reference points for correcting for variations caused by different imaging systems 3.

[0114] In one embodiment, the structural elements can be pillars fabricated vertically on the inner surface of the sample holder assembly 4. Fabrication of these pillars can include growing nanomaterials on the inner surface. Each pillar can have a three-dimensional shape comprising a cylinder of a certain height, and a cross-section having a certain area and a certain two-dimensional shape, such that the cross-section can be detected from an image of the sample holder assembly. The sample holder assembly 4 can include a first plate and a second plate. The pillars can be precisely fabricated at predetermined locations on the inner surface of either the first plate or the second plate, wherein the inner surfaces of the first and second plates are the inner surfaces facing each other when the sample holder assembly 4 is in the closed configuration. When the sample holder assembly 4 is in the open configuration, the sample 6 can be placed on the inner surface of the first plate or the inner surface of the second plate. After the sample holder assembly 4 is closed, enclosing the sample 6 within the holder and sandwiched between the first and second plates, the sample holder assembly 4 can be inserted into an adapter assembly (not shown) mounted on a computing device associated with the imaging system 3. The adapter assembly can hold the sample holder assembly 4 in place to stabilize the relative position between the sample holder assembly 4 and the imaging system 3, allowing the imaging system 3 to be activated to capture the measurement image 7. The captured assay image 7 in digital form may include pixels representing the sample and the cross-sectional area of ​​the column. The captured assay image 7 may be uploaded to a database system 110 for use in training the machine learning model 108 or for analysis using the trained machine learning model 108.

[0115] The first or second plate may be made of a transparent material. Thus, the imaging system 3, with the aid of a light source provided in the adapter, can capture a measurement image 7 containing pixels representing a cross-section of the column 5 and the sample 6. The measurement image 7 may include distortions (linear and nonlinear) caused by a number of factors associated with the system L. These factors may include inaccurate coupling between the sample holder device 4 and the adapter, as well as characteristic variations inherent in the imaging system 3. These distortions, if not corrected, may adversely affect the performance of the machine learning model 108.

[0116] Embodiments of the present invention can use the position of the posts 5 to determine and correct these distortions. In one embodiment, the posts 5 are manufactured at predetermined positions on the inner surface. The positions of these posts can be located according to a pattern. For example, the positions of these posts can form a rectangular array with horizontal periodicity and vertical periodicity, which means that in real-dimensional space, the horizontal distance (dx) between any two adjacent posts is the same, and the distance (dy) between any two adjacent posts is the same. Because the positions of these posts are predetermined during the manufacturing process of the sample holder device 4, the detected positions of these posts in the measurement image 7 can be used to determine and correct the distortions therein.

[0117] Embodiments of the present disclosure may include a method 10 for training a machine learning model 108 to enhance an image-based assay. The processing device 102 may be configured to perform the method 10. At 112, the processing device 102 may receive a first image of a sample holder containing a sample captured by a first imaging system, wherein the sample holder is fabricated with standard patterned structural elements at predetermined positions; the first image may be an assay image 7 captured by the imaging system 3, the first image including pixels representing a post 5 and a sample 6. The sample 5 may be a biological sample, such as a drop of blood or a chemical sample. As described above, the standard patterned structural elements may include posts fabricated vertically on the inner surface of a plate of the sample holder device 4. The predetermined positions of these posts may form a rectangular array, wherein the post regions are spaced uniformly apart in the horizontal and vertical directions.

[0118] At 114, the processing device 102 may identify a first region in the first image based on the position of one or more structural elements of the patterned structural element in the first image. In this regard, the processing device 102 may first detect the position of the patterned structural element in the first image in the measurement image 7. Because the optical properties (e.g., transparency) and cross-sectional shape of the pillars are designed to be different from the optical properties and cross-sectional shape of the analyte in the sample, the pillars can be distinguished from the sample in the measurement image 7. Embodiments of the present invention may include any suitable image analysis method for detecting the position of the pillars. The method for detecting the position of the pillars may be based on pixel intensity values ​​or morphological characteristics of the region.

[0119] In one embodiment, a second machine learning model can be trained to detect the position of pillars in the measurement image 7. The second machine learning model can be a neural network such as RetinaNet, which is a first-level object detection model suitable for detecting dense and small-scale objects. In the forward propagation of training, the training measurement image is fed into the second machine model to generate a result image including a detection area of ​​the pillar. The result image including the detected pillar area can be compared with the pillar area marked by a human operator. In the backward propagation of training, the parameters of the second machine learning model can be adjusted based on the difference between the detection area and the marked area. In this way, the second machine learning can be trained to detect pillars in the measurement image.

[0120] Embodiments of the present disclosure may include using a trained second machine learning model to detect areas corresponding to columns in the measurement image 7. Although applying the trained second machine learning model to the measurement image 7 may produce better column detection results than other methods, the detection results may still include missed detections and false detections of columns. Embodiments of the present disclosure may also include a detection correction step to identify missed detections of columns and remove false detections. In one embodiment, the processing device 102 may perform the detection correction step based on a periodic distribution pattern of the columns. For any position in the periodic distribution pattern where a corresponding column is missed, the processing device 102 may insert the corresponding column into the detection result based on the horizontal and / or vertical periodicity of the pattern; for any column at a position that is not located in the periodic distribution pattern, the processing device 102 may determine the column as a false alarm based on the horizontal and / or vertical periodicity of the pattern and remove it from the detection result.

[0121] Based on the positions of the pillars in the measurement image 7, the processing device 102 can further divide the first image into several regions, each of which is defined by the detected pillars in the measurement image 7. When the pillars are arranged in a rectangular array in the real-dimensional space of the sample holder device 4, the measurement image 7 can be divided into regions 8, each of which is defined by four adjacent pillars at four corners. For example, a region can be defined by four lines drawn from the center of the pillar region. However, due to distortions in the inaccurate coupling between the sample holder device 4 and the adapter and in the imaging system 3, each region may not be a rectangular region corresponding to its physical shape in the real-dimensional space manufactured on the inner surface of the sample holder device 4. Instead of being rectangular, each region may be warped into a quadrilateral due to these deformations. In addition, the distortion may be non-uniform across the entire measurement image 7 (e.g., due to a limited field of view), resulting in different warping effects in different regions.

[0122] Embodiments of the present disclosure can mitigate distortion by projecting each region back to real dimensional space. Figure 4At 116, processing device 102 may determine a spatial transformation associated with the first region based on a mapping between the positions of the one or more structural elements in the first image and the predetermined positions of the one or more structural elements in the sample holder; each quadrilateral region defined by the four pillar regions at the four corners may correspond to a rectangle in the real-dimensional space of sample holder device 4 defined by the four corresponding pillars. Therefore, parameters of the spatial transformation may be determined based on the mapping between the four detected pillar regions in measurement image 7 and the four pillars in real-dimensional space. In one embodiment, the spatial transformation may be a homography (perspective transformation) that maps the quadrilateral plane defined by the four detected pillar regions to a rectangular plane of its real physical shape defined by the four corresponding pillars, wherein each pillar region may be represented by the center of the region, and each pillar may be represented by the center of its cross-section. Processing device 102 may determine a corresponding homography for each region in measurement image 7. Thus, processing device 102 may use the homography associated with each region to mitigate distortion associated with that region.

[0123] At 118, processing device 102 may apply a spatial transformation to the first region in the first image to calculate a transformed first region. Processing device 102 may apply the determined spatial transformation to pixels in each region to transform the region into a real-dimensional space, thereby substantially eliminating distortion associated with the region. In this manner, embodiments of the present disclosure may utilize predetermined positions of the posts within the real-dimensional space defined on the inner surface of the sample holder device to mitigate or substantially eliminate distortion of system L.

[0124] At 120, processing device 102 may use the transformed first image to train machine learning model 108. Machine learning model 108 may be trained to enhance measurement images captured by system L to a quality level comparable to those captured by system H. Quality may be reflected in terms of noise level, distortion, and / or resolution. The training dataset may include partitioned regions that have been transformed into real-dimensional space. Thus, the training dataset is less affected by distortion caused by system L. In one embodiment, the training dataset may include transformed regions from a plurality of measurement images captured by the imaging system of system L.

[0125] For training purposes, the training dataset can also include corresponding regions in assay images captured by system H. The assay images captured by system H are of high quality and suitable for analysis by an expert operator (e.g., a clinical pathologist) or by a computer assay analysis program. To construct the training dataset, system H can also capture a high-quality assay image for each assay image 7 captured by system L of a sample holder device 4 containing a sample 6. Processing device 102 can similarly separate the high-quality assay images into regions defined by columns.

[0126] Figure 5 A flow chart depicts a method 200 for preparing high-quality measurement images in a training dataset, according to an embodiment of the present disclosure. Method 200 may be performed by processing device 102. At 202, processing device 102 may receive a second image of a sample holder captured by a second imaging system, wherein the first image was captured at a first quality level and the second image was captured at a second quality level higher than the first quality level. As described above, after capturing a first measurement image of sample holder arrangement 4 using imaging system 3 of system L, a second measurement image of the same sample holder arrangement 4 may be captured using system H. The second measurement image may have a higher quality than the corresponding first measurement image. This higher quality may be reflected in higher resolution, lower noise levels, and / or less distortion. The second measurement image may be of a quality that can be directly used by an expert operator in a laboratory environment to analyze sample content using the second measurement image. In one embodiment, the second measurement image may be captured using a microscope in a controlled environment with appropriate lighting and calibrated optics, such that the captured second measurement image may be less affected by distortion than the first measurement image captured by system L. After being captured, the high-quality second measurement image has been stored in the database system 110 , so that the processing device 102 can retrieve the second measurement image from the database.

[0127] At 204, the processing device 204 may segment the second image into a plurality of regions, wherein each of the regions in the second image is bounded by four structural elements at four corners of the corresponding region in the second image. The second measurement image may be segmented in a manner similar to segmenting the corresponding first measurement image. Segmenting the second measurement image may include detecting pillar regions in the second measurement image and correcting any missing pillar regions and / or erroneous pillar regions based on the periodicity of the pillar positions. Each region in the second measurement image may be a quadrilateral defined by four adjacent pillar regions at four corners.

[0128] Based on the positions of the detected pillars in the measurement image, the processing device 102 can further separate the first image into regions, each of which is defined by a detected pillar in the measurement image. When the pillars are arranged in a rectangular array in the real-dimensional space of the sample holder device 4, the second measurement image can also be separated into regions, each defined by four adjacent pillar regions at four corners. However, due to the inaccurate coupling between the sample holder device 4 and the adapter, as well as distortion in the imaging system 3, each region may not be a rectangular region corresponding to its physical shape in the real-dimensional space as created on the inner surface of the sample holder device 4. Similar to the processing of the first measurement image, a spatial transformation can be used to correct the distortion associated with each region in the second measurement image.

[0129] At 206, processing device 102 may determine a second spatial transformation associated with the region in the second image based on a mapping between the positions of the four structural elements at the four corners of the corresponding region in the second image and the four predetermined positions of the four structural elements in the sample holder. In one embodiment, each quadrilateral region defined by the four detected pillar regions in the second measurement image may correspond to a rectangle in the real-dimensional space of sample holder device 4 defined by the four corresponding pillars. Therefore, parameters of the second spatial transformation may be determined based on the mapping between the four detected pillar regions in the second measurement image and the four pillars in the real-dimensional space. In one embodiment, the spatial transformation may be a homography (perspective transformation) that maps the quadrilateral plane defined by the four detected pillar regions in the second measurement image to a rectangular plane defined by the four corresponding pillars in the sample holder. In another embodiment, instead of determining a specific second spatial transformation for each region in the second measurement image, processing device 102 may determine a global second spatial transformation for all regions in the second measurement image. This is possible because, due to the high quality of system H's imaging system, the second measurement image captured by system H may include more uniform distortion across all regions. Therefore, a global second spatial transformation is sufficient to correct for uniform distortion. The global second spatial transformation may be determined using the column area associated with a region in the second measurement image.Alternatively, the global second spatial transformation may be determined as an average of the second spatial transformations associated with a plurality of regions in the second measurement image.

[0130] At 208, processing device 102 may apply a second spatial transform to each of the regions in the second image to calculate a corresponding transformed region in the second image. The second spatial transform may be a global transform or a local transform. Application of the second spatial transform may transform the region in the second measurement image to a real-dimensional space on the inner surface of the sample holder device, thereby reducing distortion associated with the second measurement image. For each first measurement image in the training dataset, processing device 102 may prepare a corresponding second measurement image. Processing device 102 may place the transformed region in the second measurement image in the training dataset.

[0131] At 210, the processing device 102 may train a machine learning model using each transformed region in the first image and each transformed region in the second image. The machine learning model 108 may be trained using a training dataset that includes the transformed regions of the first measured image and their corresponding regions of the second measured image, wherein the transformed regions of the first measured image and their corresponding regions of the second measured image are mapped into a real-dimensional space to mitigate distortions present in system L and system H. In this way, individual system variations are substantially eliminated, and regions of the first measured image and regions of the second measured image are mapped into a common real-dimensional space using standard information embedded in the measured images.

[0132] Machine learning model 108 is a model constructed from training data comprising examples. In the present disclosure, examples include regions of the first measurement image in the training dataset and their corresponding regions in the second measurement image. These regions in the training dataset have been corrected for distortion by transforming them into a real-dimensional space defining the positions of the posts on the inner surface of sample holder device 4. Embodiments can use trained machine learning model 108 to enhance measurement images captured by system L to a quality level comparable to measurement images captured by system H.

[0133] The machine learning model 108 can be any suitable model, including but not limited to a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), a graph neural network (GNN), etc. All of these models can include parameters that can be adjusted during training based on a training dataset or examples.

[0134] The mapping from the measurement image of system L to the measurement image of system H can fall into the category of image-to-image transformation problems. Training of a machine learning model for image-to-image transformation may require a large number of perfectly matched paired instances from the source image (the measurement image captured by system L) to the target image (the measurement image captured by system H). In the context of measurement images, the sample contained in the sample holder is typically some type of liquid, in which tiny analytes can constantly move between images taken by different imaging systems (e.g., system L and then system H). Therefore, it is impractical, if not impossible, to construct a sufficiently large training dataset containing perfectly matched pairs of low-quality and high-quality measurement image instances to train a machine learning model for image transformation. To overcome this practical problem in the transformation of measurement images, embodiments of the present disclosure employ a Cycle Generative Adversarial Network (CycleGAN) model that can be trained using unpaired measurement images. Although the regions separated from the low-quality assay image are not paired with the corresponding regions separated from the high-quality assay image due to the movement of analytes in the sample and other factors, the pillar regions at the corners of the regions in the low-quality assay image match the pillar regions at the corners of the high-quality assay image because they are physically fixed in real-dimensional space during the precise manufacturing of the sample holder. The matched pillar regions embedded in the assay image in the present disclosure provide registered landmark information and additional constraints, which help further improve the fidelity of training and transforming images for assay purposes using CycleGAN.

[0135] The CycleGAN model consists of a forward GAN and a backward GAN. Figure 6 FIG. 3 shows the construction of a CycleGAN model 300 for determining image-to-image transformation according to an embodiment of the present disclosure. Figure 6 , the CycleGAN model 300 may include a forward GAN 302 and a backward GAN 304. The forward GAN 302, similar to a typical GAN ​​model, may include a generator 306A and a discriminator 306B; similarly, the backward GAN 304 may also include a generator 308A and a discriminator 308B. Each of the generators 306A, 308A and the discriminators 306B, 308B may be a neural network (e.g., a multi-layer convolutional neural network). The generators 306A, 308A may convert an input measurement image in a first domain (e.g., having a first quality or a first resolution) into an output measurement image in a second domain (e.g., having a second quality or a second resolution).

[0136] Generator 306A may convert a measurement image 310 in a first image domain (Domain L) into a measurement image generated in a second image domain (Domain H). For example, generator 306A may convert a low-resolution measurement image captured by system L into a generated high-resolution measurement image having the same resolution as that captured by system H. Generator 308A may convert a measurement image 312 in Domain H into a measurement image generated in Domain L. For example, generator 308A may convert a high-resolution measurement image captured by system H into a low-resolution measurement image generated in Domain L. Discriminator 306B may compare the measurement image generated in Domain H with a true measurement image in Domain H to output a first generator loss function and a first discriminator loss function. The first generator loss function may indicate whether the measurement image generated in Domain H belongs to Domain H (or "true") (or "false"). A first discriminator loss function (not shown) may indicate whether discriminator 306B correctly classified "true" or "false." The discriminator 308B may compare the measurement image generated in domain L with the true measurement image in domain L to output a second generator loss function and a second discriminator loss function. The second generator loss function may indicate whether the measurement image generated in domain L belongs to domain L (or "true") (or "false"). The second discriminator loss function (not shown) may indicate whether the discriminator 308B correctly classifies "true" or "false."

[0137] During training, the discriminator 306B can be trained using a first discriminator loss function in backpropagation, and the discriminator 308B can be trained using a first discriminator loss function in backpropagation. The parameters of the generator 306 can be adjusted in backpropagation based on the first generator loss function from the discriminator 306B, so that the generator 306A can produce generated measurement images in the domain H that the discriminator 306B considers to be "real." Similarly, the parameters of the generator 308A can be adjusted in backpropagation based on the second generator loss function from the discriminator 308B, so that the generator 308A can produce generated measurement images in the domain L that the discriminator 306B considers to be "real." The GANs 302 and 304 containing the discriminators 306B and 308B and the generators 306A and 308A can be trained in alternative time periods. For example, in step 1, the discriminators 306B, 308B may be trained for a few epochs, and then in step 2, the generators 306A, 308A may be trained for a few subsequent epochs. Steps 1 and 2 may be alternately repeated during training until each of the GANs 302, 304 converges.

[0138] In an optional embodiment, CycleGAN 300 may require cycle consistency. Cycle consistency requires that the combined transformations of generator 306A and generator 308A produce an identity cycle map. The identity map means that generator 306A can transform the input measurement image into the generated measurement image, and generator 308A can transform the generated measurement image back into the original input measurement image. The cycle consistency requirement can allow CycleGAN 300 to train the image transformation model using unpaired images. In addition, embodiments of the present disclosure provide additional constraints, such as landmark information for registration of column regions, to output high-fidelity transformed measurement images and positional correspondence of the transformed measurement images for measurement purposes.

[0139] In the context of the present disclosure, low-resolution image 310 may include regions separated from a measurement image captured by imaging system 3 based on detected pillar regions, and high-resolution image 312 may include corresponding regions separated from a measurement image captured by a high-quality imaging system built into a microscope based on detected pillar regions, wherein all measurement image regions have been transformed into real-dimensional space to mitigate distortion. During training of forward GAN 302, a first measurement image region from low-resolution image 310 may be provided to generator 306A to generate a generated first region in the high-resolution domain. Discriminator 306B may compare the generated first region with a corresponding first region measurement image in high-resolution measurement image 312 to generate a first generator loss function and a first discriminator loss function. The first discriminator loss function may be used in backpropagation to train discriminator 306B, while the first generator loss function may be used in backpropagation to train generator 306A. Each measurement image region in low-resolution image 310 may be similarly used to train generator 306A and discriminator 306B of forward GAN 302. In the training of the backward GAN 304, the second measurement image region from the high-resolution image 312 can be provided to the generator 308A to produce a generated second region in the low-resolution domain. The discriminator 308B can compare the generated second region with the corresponding second region measurement image in the low-resolution measurement image 310 to generate a second generator loss function and a second discriminator loss function. The second discriminator loss function can be used in backpropagation to train the discriminator 308B, while the second generator loss function can be used in backpropagation to train the generator 308A. Each measurement image region in the high-resolution image 312 can be similarly used to train the generator 308A and the discriminator 308B of the backward GAN 304.

[0140] In application, the trained CycleGAN 300 can be used to convert measurement images captured by the imaging system 3 into measurement images generated with a quality level comparable to measurement images captured by a microscope. Figure 7 A flow chart of a method 400 for enhancing low-quality metrology images according to an embodiment of the present disclosure is depicted. One or more processing devices (eg, processing device 102) may perform the operations of method 400.

[0141] At 402, a processing device may receive a first image of a sample holder containing a sample captured by a first imaging system, wherein the sample holder is fabricated with standard patterned structural elements at predetermined locations. The first image may be a first assay image that includes pixels representing the sample and structural elements such as pillars.

[0142] At 404, the processing device may identify a first region in the first image based on the positions of one or more structural elements of the patterned structural element in the first image. Identifying the first region may include detecting a pillar region in the first image and identifying the first region based on four adjacent pillar regions at four corners. Due to distortion associated with the first image, the first region may be a quadrilateral.

[0143] At 406, the processing device can determine a spatial transformation associated with the first region based on a mapping between the positions of the one or more structural elements in the first image and the predetermined positions of the one or more structural elements in the sample holder. The spatial transformation can be a homography. Parameters of the spatial transformation can be determined based on a mapping of the positions (e.g., centers) of four pillar regions at the four corners of the first region to the positions (e.g., centers of the cross-sections) of the four pillars in a real dimensional space on the inner surface of the sample holder device.

[0144] At 408, the processing device may apply a spatial transformation to the first region in the first image to calculate a transformed first region. Applying the spatial transformation to the first region may help eliminate distortion based on the position of the pillars.

[0145] At 410, the processing device may apply a machine learning model to the transformed first region in the first image to generate a second region. The machine learning model may be Figure 6 Generator 306A is shown. In one embodiment, the second image region may have a higher resolution than the first image region. For example, the second image region may have a resolution of a microscopic image.

[0146] The processing device may further process each identified region in the first image according to steps 402-404 to generate a corresponding high-resolution region. Furthermore, the processing device may reassemble these generated high-resolution regions to form a second image, which is a high-resolution version of the first image. The second image may be analyzed for content by a professional human operator or by another intelligent computer system.

[0147] Instead of applying a spatial transformation to each region of an assay image captured by system L, some embodiments may apply a global spatial transformation to the entire image when preparing the training dataset. Figure 8 Depicted is a flow chart of a method 500 for preparing a training dataset for a machine learning model according to an embodiment of the present disclosure.

[0148] At 502, a processing device may receive a low-quality assay image captured by an imaging system of system L of a sample holder device containing a sample and a standard patterned post.

[0149] At 504, the processing device may detect pillar positions in the low-quality measurement image and, optionally, an orientation of a pattern of pillar positions. A machine learning model, such as that described above, may be used to detect pillar positions in the low-quality measurement image. The orientation of the pattern may be detected based on horizontal inter-pillar distance, vertical inter-pillar distance, horizontal pillar count, and / or vertical pillar count.

[0150] At 506, the processing device may determine a global spatial transformation based on a mapping between the post positions in the low-quality assay image and the post positions in the real-dimensional space of the sample holder device. The processing device may optionally perform other pre-processing operations, such as estimating a field of view (FoV) based on the post positions in the low-quality image and correcting the low-quality image based on the estimated FoV.

[0151] At 508 , the processing device may separate the low-quality image into a number of regions, each of the regions being defined by four pillars at four corners.

[0152] The processing device may optionally resize or scale each region to a regular size (e.g., 256×256 pixels) at 510. This resizing or scaling may prepare data for calculations using a later machine learning model.

[0153] At 512, the processing device may rotate the low-quality image (and the region therein) according to the orientation of the pattern of pillars. This operation is to ensure that all images are compared at the same pattern orientation.

[0154] At 514, the processing device may optionally convert each pixel into a grayscale pixel (e.g., from RGB color to grayscale). This operation may further reduce computations of the machine learning model.

[0155] At 516 , the processing device may store the regions of the low-quality images thus processed as examples of the domain L in a training dataset.

[0156] High quality assay images can be processed similarly.

[0157] At 518, the processing device may receive a high-quality assay image captured by the imaging system of system H of a sample holder device containing the sample and the standard patterned column.

[0158] At 520, the processing device may detect pillar positions in the high-quality measurement image and, optionally, an orientation of a pattern of pillar positions. A machine learning model, such as that described above, may be used to detect pillar positions in the high-quality measurement image. The orientation of the pattern may be detected based on horizontal inter-pillar distance, vertical inter-pillar distance, horizontal pillar count, and / or vertical pillar count.

[0159] At 522, the processing device may determine a global spatial transformation based on a mapping between the post positions in the high-quality assay image and the post positions in the real-dimensional space of the sample holder device. The processing device may optionally perform other pre-processing operations, such as estimating the field of view (FoV) based on the post positions in the high-quality image and correcting the high-quality image based on the estimated FoV.

[0160] At 524 , the processing device may separate the high-quality image into a number of regions, each of the regions being defined by four pillars at four corners.

[0161] The processing device may optionally resize or scale each region to a regular size (e.g., 256×256 pixels) at 526. This resizing or scaling may prepare data for calculations using a later machine learning model.

[0162] At 528, the processing device may rotate the high-quality image (and the region therein) according to the orientation of the pattern of pillars. This is done to ensure that all images are compared at the same pattern orientation.

[0163] At 530, the processing device may optionally convert each pixel into a grayscale pixel (e.g., from RGB color to grayscale). This operation may further reduce computations of the machine learning model.

[0164] At 532 , the processing device may store such processed regions of the high-quality image in a training dataset as examples of the domain H. Thus, a training dataset may be constructed.

[0165] While embodiments of the present disclosure are described in the context of image-based measurements using a sample holder device fabricated with standard patterned structural elements as landmark references in the measurement image, the machine learning systems and methods can be readily applied to imaging of other types of materials, where the imaged material can transform from a well-defined shape (e.g., crystalline) to an amorphous state, and the imaging system used to capture images of the amorphous material is imperfect. In some implementations, a machine learning model can be trained to learn a mapping from a material image of an amorphous material captured by an imperfect imaging system to a material image of a crystalline material captured by a perfect imaging system. However, such direct training of a machine learning model requires a large number of training examples (e.g., on the order of 10,000 or more), which is costly and impractical.

[0166] Figure 10 Figure A shows the mapping between a crystal structure 702 and an amorphous structure 704. The crystal structure 702 represents a material structure with a specific pattern (e.g., a periodic atomic arrangement). In contrast, the amorphous structure 704 represents a material structure without a pattern. In a traditional test paradigm (e.g., the Precision Protocol Paradigm (PPP)), the image of the crystal structure 702 can be captured by a professional operator using a high-precision instrument (e.g., an electron microscope), while the crystal structure 702 is a perfect shape. In the smart test paradigm, the image of the amorphous structure 704 can be captured by a non-professional operator using an imprecise instrument (e.g., an imaging system of a smartphone). The purpose of the smart test paradigm is to map the amorphous structure 704 to the crystal structure 702 using a machine learning model. As described above, the training of the machine learning model requires a large amount of training data that is not easily available. In addition, the direct mapping from the amorphous structure 704 to the crystal structure 702 using such a trained machine learning model may result in machine learning artifacts in the material image. Therefore, it is necessary to reduce the demand for a large amount of training data and improve image fidelity.

[0167] Instead of trying to train a machine learning model that maps directly from the amorphous structure 704 to the crystalline structure 702, embodiments of the present disclosure can introduce intermediate structures to the crystalline and amorphous structures during the construction of the training data. The intermediate structures can be used to train the machine learning model, thereby reducing the need for training data and improving image fidelity. Figure 10B shows a mapping between a crystalline structure 704 injected with a metastructure and an amorphous structure 706 injected with a metastructure according to an embodiment of the present disclosure. The metastructure may include elements with significant physical properties (such as shape and optical properties), which can easily distinguish these elements from the carrier crystalline / amorphous structure in the material image. Therefore, when the training data contains material images of the crystalline structure 704 injected with the metastructure and material images of the amorphous structure 706 injected with the metastructure, the metastructure can be first extracted from these material images and can be used as training data to train the machine learning model. Because the physical properties and characteristics of the metastructure are known in advance, the metastructure can be reliably extracted from the material images captured in the smart test example by a non-professional operator using an imprecise imaging system. Also due to the known physical properties and characteristics, the training of the machine learning model may require less training data and result in high-fidelity images.

[0168] Embodiments of the present disclosure may include a method for training a machine learning model that maps an image of a first material having a first structure to an image of a second material having a second structure. The method includes injecting a metastructure into the first material and capturing a first image of the first material using a first imaging system, injecting the metastructure into the second material and capturing a second image using a second imaging system, extracting a first position of the metastructure from the first image, extracting a second position of the metastructure from the second image, and training the machine learning model using the first position and the second position. The first material includes an amorphous structure and the second material includes a crystalline structure. The method also includes applying the trained machine learning model to map an image of a third material having the first structure to an image of a fourth material having the second structure.

[0169] The sample holder device described in the present disclosure can be a QMAX card. Technical details are described in International Application No. PCT / US2016 / 046437. The QMAX card can include two boards.

[0170] I.Board

[0171] In the present disclosure, generally, compression regulated open flow (CROF) plates are made of any material that: (i) can be used with spacers to adjust the thickness of a portion or the entire volume of a sample; and (ii) has no significant adverse effects on the sample, the assay, or the purpose for which the plate is intended to be used. However, in certain embodiments, specific materials (and therefore their properties) are used for the plate to achieve certain purposes.

[0172] In certain embodiments, the two panels have the same or different parameters for each of the following parameters: panel material, panel thickness, panel shape, panel area, panel flexibility, panel surface properties, and panel optical transparency.

[0173] (i) Sheet material. The sheet material is made of a single material, a composite material, multiple materials, a multilayer material, an alloy, or a combination thereof. Each material used for the sheet material is an inorganic material, an organic material, or a mixture thereof. Examples of the materials are given in the above-mentioned Mat-1 and Mat-2 sections.

[0174] Mat-1: Inorganic materials used for the plate include, but are not limited to, for example, glass, quartz, oxides, silicon dioxide, silicon nitride, hafnium oxide (HfO), aluminum oxide (AlO), semiconductors: (silicon, gallium arsenide, gallium nitride, etc.), metals (e.g., gold, silver, copper, aluminum, titanium, nickel, etc.), ceramics, or any combination thereof.

[0175] Mat-2: Organic materials for spacers include, but are not limited to, polymers (e.g., plastics) or amorphous organic materials. Polymer materials for spacers include, but are not limited to, acrylate polymers, vinyl polymers, olefin polymers, cellulosic polymers, non-cellulosic polymers, polyester polymers, nylon, cyclic olefin copolymer (COC), poly(methyl methacrylate) (PMMA), polycarbonate (PC), cyclic olefin polymer (COP), liquid crystal polymer (LCP), polyamide (PA), polyethylene (PE), polyimide (PI), polypropylene (PP), polyphenylene ether (PPE), polystyrene (PS), polyoxymethylene (POM), polyetheretherketone (PEEK), polyethersulfone (PES), polyethylene phthalate (PET), polytetrafluoroethylene (PTFE), polyvinyl chloride (PVC), polyvinylidene fluoride (PVDF), polybutylene terephthalate (PBT), fluorinated ethylene propylene (FEP), perfluoroalkoxyalkane (PFA), polydimethylsiloxane (PDMS), rubber, or any combination thereof.

[0176] In some embodiments, the plates are each independently made of at least one of glass, plastic, ceramic, and metal. In some embodiments, each plate independently comprises at least one of glass, plastic, ceramic, and metal.

[0177] In some embodiments, one plate is different from another plate in lateral area, thickness, shape, material or surface treatment. In some embodiments, one plate is the same as another plate in lateral area, thickness, shape, material or surface treatment.

[0178] The material used for the panels is rigid, flexible, or anything in between. Rigidity (ie, hard) or flexibility is relative to a given pressure used in bringing the panels into the closed configuration.

[0179] In certain embodiments, the choice of a rigid or flexible plate may be determined by the need to control the uniformity of sample thickness in the closed configuration.

[0180] In some embodiments, at least one of the two plates is transparent (to light). In some embodiments, at least a portion or portions of one or both plates are transparent. In some embodiments, the plates are opaque.

[0181] (ii) Plate Thickness. In certain embodiments, the average thickness of at least one plate is, for example, 2 nm or less, 10 nm or less, 100 nm or less, 500 nm or less, 1000 nm or less, 2 μm (micrometers) or less, 5 μm or less, 10 μm or less, 20 μm or less, 50 μm or less, 100 μm or less, 150 μm or less, 200 μm or less, 300 μm or less, 500 μm or less, 800 μm or less, 1 mm (millimeter) or less, 2 mm or less, 3 mm or less, or a range between any two of these values.

[0182] In some embodiments, the average thickness of at least one of the plates is, for example, at most 3 mm (millimeters), at most 5 mm, at most 10 mm, at most 20 mm, at most 50 mm, at most 100 mm, at most 500 mm, or in a range between any two of the stated values.

[0183] In some embodiments, the thickness of the plate is non-uniform across the plate. Using different plate thicknesses at different locations can be used to control plate bending, folding, specimen thickness adjustment, and other.

[0184] (iii) Plate shape and area. Generally, the plate can have any shape that allows for compressed, open flow of the sample and adjustment of the sample thickness. However, in certain embodiments, a specific shape may be advantageous. The plate shape can be circular, oval, rectangular, triangular, polygonal, annular, or any combination of these shapes.

[0185] In certain embodiments, two plates can have the same size or shape, or different sizes or shapes.The area of ​​plate depends on application.The area of ​​plate is at most 1mm2 (square millimeter), at most 10mm2, at most 100mm2, at most 1cm2 (square centimeters), at most 5cm2, at most 10cm2, at most 100cm2, at most 500cm2, at most 1000cm2, at most 5000cm2, at most 10000cm2 or more than 10000cm2, or in the scope between any two said values.The shape of plate can be rectangle, square, circle or other shape.

[0186] In certain embodiments, at least one panel is in the form of a strip (or ribbon) having a width, thickness, and length. The width is at most 0.1 cm (centimeter), at most 0.5 cm, at most 1 cm, at most 5 cm, at most 10 cm, at most 50 cm, at most 100 cm, at most 500 cm, at most 1000 cm, or within a range between any two of these values. The length can be any desired length. The strip can be rolled into a roll.

[0187] (iv) Plate surface flatness. In many embodiments, the inner surface of the plate is flat or significantly flat, or planar. In certain embodiments, the two inner surfaces are in a closed configuration, parallel to each other. A flat inner surface helps to quantify and / or control sample thickness by simply using a predetermined spacer height in a closed configuration. For non-flat inner surfaces of a plate, it is necessary to know not only the spacer height, but also the exact topology of the inner surface to quantify and / or control sample thickness in a closed configuration. In order to know the surface topology, additional measurements and / or corrections are required, which may be complicated, time-consuming, and expensive.

[0188] The flatness of the plate surface is relative to the final sample thickness (the final thickness being the thickness in the closed configuration) and is often characterized by the term "relative surface flatness," which is the ratio of the variation in plate surface flatness to the final sample thickness.

[0189] In certain embodiments, the relative surface area is less than 0.01%, 0.1%, less than 0.5%, less than 1%, less than 2%, less than 5%, less than 10%, less than 20%, less than 30%, less than 50%, less than 70%, less than 80%, less than 100%, or a range between any two of the stated values.

[0190] (v) Plate Surface Parallelism. In some embodiments, the two surfaces of the plate are substantially parallel to each other. In some embodiments, the two surfaces of the plate are not parallel to each other.

[0191] (vi) Plate Flexibility. In some embodiments, the plates are flexible under compression during the CROF process. In some embodiments, both plates are flexible under compression during the CROF process. In some embodiments, one plate is rigid and the other plate is flexible under compression during the CROF process. In some embodiments, both plates are rigid. In some embodiments, both plates are flexible but have different degrees of flexibility.

[0192] (vii) Optical transparency of the panels. In some embodiments, the panels are optically transparent. In some embodiments, both panels are optically transparent. In some embodiments, one panel is optically transparent while the other is opaque. In some embodiments, both panels are opaque. In some embodiments, both panels are optically transparent but have different optical transparencies. The optical transparency of a panel can refer to a portion or the entire area of ​​the panel.

[0193] (viii) Surface Wetting Properties. In some embodiments, a plate has an inner surface that wets (e.g., has a contact angle less than 90 degrees) the sample, the transferred liquid, or both. In some embodiments, both plates have inner surfaces that wet the sample, the transferred liquid, or both; and have the same or different wettabilities. In some embodiments, a plate has an inner surface that wets the sample, the transferred liquid, or both; and another plate has an inner surface that does not wet (e.g., has a contact angle equal to or greater than 90 degrees). Wetting of the inner surface of a plate can refer to a portion or the entire area of ​​the plate.

[0194] In certain embodiments, the inner surface of the plate has other nanostructures or microstructures to control the lateral flow of the sample during the CROF process. Nanostructures or microstructures include, but are not limited to, channels, pumps, etc. Nano and micro structures are also used to control the wetting properties of the inner surface.

[0195] II. Spacer

[0196] (i) Function of the spacer. In the present invention, the spacer is configured to have one or any combination of the following functions and characteristics: (1) control the thickness of the sample or the relevant volume of the sample in conjunction with the plate (preferably, the thickness control is precise, uniform, or both over the relevant area); (2) enable the sample to have a compressed regulated open flow (CROF) on the plate surface; (3) not take up a significant surface area (volume) within a given sample area (volume); (4) reduce or increase the sedimentation effect of particles or analytes in the sample; (5) change and / or control the wetting properties of the inner surface of the plate; (6) identify the position, size scale, and / or information related to the plate, or (7) any combination of the above.

[0197] (ii) Spacer structure and shape. To achieve the desired sample thickness reduction and control, in certain embodiments, spacers secure their corresponding plates. Generally, spacers can have any shape as long as they can adjust sample thickness during the CROF process, but certain shapes are preferred to achieve certain functions, such as better uniformity, less overshoot during pressing, etc.

[0198] The spacer can be a single spacer or a plurality of spacers (e.g., an array). Certain embodiments of the plurality of spacers are spacer arrays (e.g., a pillar array) where the spacer spacing is periodic or aperiodic, or periodic or aperiodic in certain regions of the plate, or has different distances in different regions of the plate.

[0199] There are two types of spacers: open spacers and closed spacers. An open spacer is one that allows sample flow through the spacer (e.g., the sample flows around and through the spacer. For example, a column acts as a spacer), and a closed spacer is one that blocks sample flow (e.g., the sample cannot flow past the spacer. For example, a ring-shaped spacer with the sample inside the ring). Both types of spacers use their height to adjust the final sample thickness in the closed configuration.

[0200] In some embodiments, the spacer is only an open spacer. In some embodiments, the spacer is only a closed spacer. In some embodiments, the spacer is a combination of open spacers and closed spacers.

[0201] The term "column spacer" means that the spacer has a columnar shape, and the columnar shape may refer to an object having a height and lateral shape that allows a sample to flow around it during compressed open flow.

[0202] In certain embodiments, the lateral shape of the column spacer is selected from the following shapes: (i) a circle, an ellipse, a rectangle, a triangle, a polygon, a ring, a star, a letter shape (e.g., an L shape, a C shape, i.e., letters from A to Z), a number shape (e.g., shapes of 0, 1, 2, 3, 4 to 9, ...); (ii) a shape from group (i) having at least one rounded corner; (iii) a shape from group (i) having a jagged or rough edge; and (iv) any combination of (i), (ii), and (iii). For multiple spacers, different spacers can have different lateral shapes and sizes, as well as different distances from adjacent spacers.

[0203] In some embodiments, the spacers can be and / or include rods, posts, beads, spheres, and / or other suitable geometric shapes. The lateral shape and size of the spacers (e.g., transverse to the corresponding plate surface) can be any shape and size, except for the following limitations in some embodiments: (i) the spacer geometry does not cause significant errors in measuring sample thickness and volume; or (ii) the spacer geometry does not prevent the flow of sample between the plates (e.g., it is not a closed form). However, in some embodiments, they require some spacers to act as closed spacers to restrict sample flow.

[0204] In some embodiments, the shape of the spacer has rounded corners. For example, a rectangular spacer has one, several, or all rounded corners (similar to a circle rather than a 90-degree angle). Rounded corners generally make the spacer easier to manufacture and, in some cases, less damaging to the biomaterial.

[0205] The sidewalls of the pillars can be straight, curved, angled, or of varying shapes at different portions of the sidewalls. In certain embodiments, the spacers are pillars of varying transverse shapes, sidewalls, and ratios of pillar height to pillar transverse area. In preferred embodiments, the spacers are pillars shaped to allow for open flow.

[0206] (iii) Spacer Material. In the present invention, the spacer is generally made of any material that can be used with the two plates to adjust the thickness of the relevant volume of the sample. In some embodiments, the material used for the spacer is different from the material used for the plates. In some embodiments, the material used for the spacer is at least partially the same as the material used for at least one of the plates.

[0207] These spacers are made of a single material, a composite material, multiple materials, a multilayer material, an alloy, or a combination thereof. Each material used for the spacers is an inorganic material, an organic material, or a mixture thereof, wherein examples of the materials are given in the above-mentioned Mat-1 and Mat-2 paragraphs. In a preferred embodiment, the spacers are made of the same material as the plates used in the CROF.

[0208] (iv) Mechanical strength and flexibility of the spacer. In certain embodiments, the mechanical strength of the spacer is sufficient so that during compression and in the closed configuration of the panels, the height of the spacer is the same or significantly the same as when the panels are in the open configuration. In certain embodiments, the difference in the spacer between the open and closed configurations can be characterized and predetermined.

[0209] The material used for the spacer can be rigid, flexible, or anywhere in between. The rigidity is relative to the given compression force used to force the panels into the closed configuration: if the spacer does not deform more than 1% of its height under compression, the spacer material is considered rigid; otherwise, it is flexible. When the spacer is made of a flexible material, the final sample thickness in the closed configuration can still be predetermined by the compression force and mechanical properties of the spacer.

[0210] (v) Spacers within the sample. To achieve the desired reduction and control of sample thickness, and in particular to achieve good sample thickness uniformity, in certain embodiments, a spacer is placed within the sample, or within the relevant volume of the sample. In certain embodiments, one or more spacers are present within the sample or within the relevant volume of the sample, with appropriate spacer spacing. In certain embodiments, at least one spacer is within the sample, at least two spacers are within the sample or within the relevant volume of the sample, or at least "n" spacers are within the sample or within the relevant volume of the sample, where "n" can be determined by the desired sample thickness uniformity or sample flow properties during the CROF process.

[0211] (vi) Spacer height. In some embodiments, all spacers have the same predetermined height. In some embodiments, the spacers have different predetermined heights. In some embodiments, the spacers can be divided into groups or regions, where each group or region has its own spacer height. In some embodiments, the predetermined height of the spacers is the average height of the spacers. In some embodiments, the spacers have approximately the same height. In some embodiments, a percentage of the spacers have the same height.

[0212] The height of the spacer is selected by the desired adjusted final sample thickness and the residual sample thickness between the plates. The spacer height (predetermined spacer height) and / or sample thickness is 3 nm or less, 10 nm or less, 50 nm or less, 100 nm or less, 200 nm or less, 500 nm or less, 800 nm or less, 1000 nm or less, 1 μm or less, 2 μm or less, 3 μm or less, 5 μm or less, 10 μm or less, 20 μm or less, 30 μm or less, 50 μm or less, 100 μm or less, 150 μm or less, 200 μm or less, 300 μm or less, 500 μm or less, 800 μm or less, 1 mm or less, 2 mm or less, 4 mm or less, or in a range between any two of the stated values.

[0213] The spacer height and / or sample thickness is 1 nm to 100 nm in one preferred embodiment, 100 nm to 500 nm in another preferred embodiment, 500 nm to 1000 nm in a separate preferred embodiment, 1 μm (e.g., 1000 nm) to 2 μm in another preferred embodiment, 2 μm to 3 μm in a separate preferred embodiment, 3 μm to 5 μm in another preferred embodiment, 5 μm to 10 μm in a separate preferred embodiment, and 10 μm to 50 μm in another preferred embodiment, 50 μm to 100 μm in a separate preferred embodiment.

[0214] In certain preferred embodiments, the spacer height and / or sample thickness is (i) equal to or slightly larger than the smallest dimension of the analyte, or (ii) equal to or slightly larger than the largest dimension of the analyte. "Slightly larger" means approximately 1% to 5% larger, and any value in between.

[0215] In certain embodiments, the spacer height and / or sample thickness is greater than the smallest dimension of the analyte (eg, the analyte has an anisotropic shape), but less than the largest dimension of the analyte.

[0216] For example, red blood cells have a disc shape with a minimum dimension of 2 μm (disc thickness) and a maximum dimension of 11 μm (disc diameter). In embodiments of the present invention, the spacers are selected so that the spacing between the inner surfaces of the plates in the relevant area is 2 μm (equal to the minimum dimension) in one embodiment, 2.2 μm in another embodiment, or 3 μm in another embodiment.

[0217] (50% larger than the minimum dimension), but smaller than the maximum dimension of a red blood cell. This embodiment has certain advantages in blood cell counting. In one embodiment, for red blood cell counting, by setting the inner surface spacing to 2 μm or 3 μm and any value in between, an undiluted whole blood sample is confined to this spacing; on average, each red blood cell (RBC) does not overlap with another, allowing for accurate visual counting of the red blood cells. (Excessive overlap between RBCs can lead to significant errors in counting).

[0218] In the present invention, the plates and spacers used in the present invention are used not only to adjust the thickness of the sample, but also to adjust the orientation and / or surface density of the analytes / entities in the sample when the plate is in a closed configuration. When the plate is in a closed configuration, a thinner thickness of the sample results in fewer analytes / entities per surface area (e.g., a smaller surface concentration).

[0219] (vii) Spacer Transverse Dimensions. For open spacers, the transverse dimensions can be characterized by their transverse dimensions in two orthogonal directions, x and y (sometimes referred to as width). The transverse dimensions of the spacer in each direction may be the same or different.

[0220] In some embodiments, the ratio of the lateral dimensions in the x and y directions is 1, 1.5, 2, 5, 10, 100, 500, 1000, 10000, or in a range between any two of these values. In some embodiments, different ratios are used to adjust the direction of sample flow; the larger the ratio, the flow is along one direction (the direction of the larger dimension).

[0221] In some embodiments, the different lateral dimensions of the spacers in the x and y directions serve to (a) use the spacers as scale markers to indicate plate orientation, (b) use the spacers to generate more sample flow in a preferred direction, or both.

[0222] In a preferred embodiment, period, width and height.

[0223] In some embodiments, all spacers have the same shape and size. In some embodiments, each spacer has a different lateral dimension.

[0224] For the enclosing spacer, in some embodiments, the inner lateral shape and dimensions are selected based on the total volume of the sample to be enclosed by the enclosing spacer, where the volume dimensions have been described in the present disclosure; and in some embodiments, the outer shape and dimensions are selected based on the required strength to support the pressure of the liquid against the spacer and the compression pressure of the pressing plate.

[0225] (viii) Aspect ratio of the height of the column spacer to the average lateral dimension. In certain embodiments, the aspect ratio of the height of the column spacer to the average lateral dimension is 100,000, 10,000, 1,000, 100, 10, 1, 0.1, 0.01, 0.001, 0.0001, 0,00001, or a range between any two of these values.

[0226] (ix) Spacer Height Accuracy. The spacer height should be precisely controlled. The relative accuracy of the spacers (e.g., the ratio of the deviation to the desired spacer height) is 0.001% or less, 0.01% or less, 0.1% or less; 0.5% or less, 1% or less, 2% or less, 5% or less, 8% or less, 10% or less, 15% or less, 20% or less, 30% or less, 40% or less, 50% or less, 60% or less, 70% or less, 80% or less, 90% or less, 99.9% or less, or a range between any two of the stated values.

[0227] (x) Spacer spacing. The spacer can be a single spacer or a plurality of spacers on the plate or in the sample-related area. In some embodiments, the spacers on the plate are configured and / or arranged in an array, and the array is periodic, aperiodic, or periodic at some locations on the plate and aperiodic at other locations.

[0228] In some embodiments, the periodic array of spacers is arranged in a lattice of squares, rectangles, triangles, hexagons, polygons, or any combination thereof, where combination means that different locations of the plate have different spacer lattices.

[0229] In some embodiments, the spacer spacing of the spacer array is periodic in at least one direction of the array (eg, uniform spacer spacing). In certain embodiments, the spacer spacing is configured to improve uniformity between plate spacing in a closed configuration.

[0230] The distance between adjacent spacers (e.g., spacer pitch) is 1 μm or less, 5 μm or less, 10 μm or less, 20 μm or less, 30 μm or less, 40 μm or less, 50 μm or less, 60 μm or less, 70 μm or less, 80 μm or less, 90 μm or less, 100 μm or less, 200 μm or less, 300 μm or less, 400 μm or less, or in a range between any two of the values.

[0231] In certain embodiments, the spacer pitch is 400 μm or less, 500 μm or less, 1 mm or less, 2 mm or less, 3 mm or less, 5 mm or less, 7 mm or less, 10 mm or less, or any range therebetween. In certain embodiments, the spacer pitch is 10 mm or less, 20 mm or less, 30 mm or less, 50 mm or less, 70 mm or less, 100 mm or less, or any range therebetween.

[0232] The distance between adjacent spacers (e.g., spacer spacing) is selected so that for given properties of the plate and sample, in the closed configuration of the plate, in some embodiments, the sample thickness between two adjacent spacers varies by at most 0.5%, 1%, 5%, 10%, 20%, 30%, 50%, 80%, or any range between the stated values; or in some embodiments, by at most 80%, 100%, 200%, 400%, or within a range between any two of the stated values.

[0233] Obviously, in order to maintain a given sample thickness variation between two adjacent spacers, a closer spacer spacing is required when using a more flexible plate.

[0234] Specifies the accuracy of spacer spacing.

[0235] In a preferred embodiment, the spacers are a periodic square array, wherein the spacers are pillars having a height of 2 to 4 μm, an average lateral dimension of 5 to 20 μm, and a spacer pitch of 1 to 100 μm.

[0236] In a preferred embodiment, the spacers are a periodic square array, wherein the spacers are pillars having a height of 2 to 4 μm, an average lateral dimension of 5 to 20 μm, and a spacer pitch of 100 to 250 μm.

[0237] In a preferred embodiment, the spacers are a periodic square array, wherein the spacers are pillars having a height of 4 to 50 μm, an average lateral dimension of 5 to 20 μm, and a spacer pitch of 1 to 100 μm.

[0238] In a preferred embodiment, the spacers are a periodic square array, wherein the spacers are pillars having a height of 4 to 50 μm, an average lateral dimension of 5 to 20 μm, and a spacer pitch of 100 to 250 μm.

[0239] The spacing of the spacer array is 1 nm to 100 nm in one preferred embodiment, 100 nm to 500 nm in another preferred embodiment, 500 nm to 1000 nm in a separate preferred embodiment, 1 μm (e.g., 1000 nm) to 2 μm in another preferred embodiment, 2 μm to 3 μm in a separate preferred embodiment, 3 μm to 5 μm in another preferred embodiment, 5 μm to 10 μm in a separate preferred embodiment, 10 μm to 50 μm in another preferred embodiment, 50 μm to 100 μm in a separate preferred embodiment, 100 μm to 175 μm in a separate preferred embodiment, and 175 μm to 300 μm in a separate preferred embodiment.

[0240] (xi) Spacer Density. The spacers are arranged on the respective panels at a surface density of greater than 1 per μm², greater than 1 per 10 μm², greater than 1 per 100 μm², greater than 1 per 500 μm², greater than 1 per 1000 μm², greater than 1 per 5000 μm², greater than 1 per 0.01 mm², greater than 1 per 0.1 mm², greater than 1 per 1 mm², greater than 1 per 5 mm², greater than 1 per 10 mm², greater than 1 per 100 mm², greater than 1 per 1000 mm², greater than 1 per 10000 mm², or within a range between any two of these values.

[0241] The spacer is configured so as not to occupy a significant surface area (volume) within a given sample area (volume).

[0242] (xii) Spacer Volume to Sample Volume Ratio. In many embodiments, the ratio of the spacer volume (e.g., the volume of the spacer) to the sample volume (i.e., the volume of the sample), and / or the ratio of the volume of the spacer within the relevant volume of the sample to the relevant volume of the sample, is controlled to achieve certain advantages. Advantages include, but are not limited to, uniformity of sample thickness control, uniformity of analyte, and sample flow properties (e.g., flow rate, flow direction, etc.).

[0243] In certain embodiments, the ratio of the spacer volume (r) to the sample volume and / or the ratio of the volume of the spacer within the relevant volume of the sample to the relevant volume of the sample is less than 100%, at most 99%, at most 70%, at most 50%, at most 30%, at most 10%, at most 5%, at most 3%, at most 1%, at most 0.1%, at most 0.01%, at most 0.001%, or within a range between any two of the stated values.

[0244] (xiii) Spacers fixed to the panels. The spacing of the spacers and the orientation of the spacers, which are critical in the present invention, are preferably maintained during the process of moving the panels from the open configuration to the closed configuration, and / or are preferably predetermined prior to the process from the open configuration to the closed configuration.

[0245] In certain embodiments of the present disclosure, a spacer is secured to one of the panels prior to bringing the panels into a closed configuration. The term "a spacer is secured to its respective panel" means that the spacer is attached to the panel and remains attached during use of the panel. An example of a "spacer is secured to its respective panel" is when the spacer is made integrally from one piece of material of the panel and the position of the spacer relative to the panel surface does not change. An example of a "spacer is not secured to its respective panel" is when the spacer is bonded to the panel by an adhesive, but during use of the panel, the adhesive is unable to maintain the spacer in its original position on the panel surface (e.g., the spacer moves away from its original position on the panel surface).

[0246] In some embodiments, at least one of the spacers is secured to its corresponding plate. In some embodiments, two spacers are secured to their corresponding plates. In some embodiments, a majority of the spacers are secured to their corresponding plates. In some embodiments, all spacers are secured to their corresponding plates.

[0247] In certain embodiments, the spacers are integrally secured to the plates.

[0248] In certain embodiments, the spacers are secured to their respective plates by one or any combination of the following methods and / or configurations: attaching, bonding, fusing, stamping, and etching.

[0249] The term "embossing" refers to integrally securing the spacer and the plate by embossing (eg, stamping) a piece of material to form the spacer on the plate surface. The material may be a single layer or multiple layers.

[0250] The term "etching" means that the spacers and the board are integrally fixed by etching a piece of material to form the spacers on the board surface. The material can be a single layer of material or multiple layers of material.

[0251] The term “fused” means that the spacer and the plate are integrally fixed by attaching the spacer and the plate together, original materials of the spacer and the plate are fused to each other, and a clear material boundary exists between the two materials after the fusion.

[0252] The term "bonding" means bonding the spacer and the plate by adhesion, thereby integrally fixing the spacer and the plate.

[0253] The term "attached" means that the spacer and the plate are connected together.

[0254] In some embodiments, the spacers and the plates are made of the same material. In other embodiments, the spacers and the plates are made of different materials. In other embodiments, the spacers and the plates are integrally formed. In yet another embodiment, the spacers have one end secured to their respective plates, while the other end is open to accommodate different configurations of the two plates.

[0255] In other embodiments, each of the spacers is at least one of: attached, bonded, fused, stamped, or etched into the corresponding plate. The term "independently" means that one spacer is secured to its corresponding plate by the same or different methods selected from the group consisting of attaching, bonding, fusing, stamping, and etching into the corresponding plate.

[0256] In certain embodiments, the distance between at least two spacers is predetermined ("predetermined spacer spacing" means that the distance is known to a user when the board is used).

[0257] In certain embodiments of all methods and devices described herein, additional spacers are present in addition to the fixed spacers.

[0258] (xiv) Specific Sample Thickness. In the present invention, it was observed that by using a smaller plate spacing (e.g., for a given sample area), or a larger sample area (for a given plate spacing), or both, a larger plate holding force (i.e., the force holding the two plates together) can be obtained.

[0259] In some embodiments, at least one of the plates is transparent in a region surrounding the relevant area, and each plate has an inner surface that is configured to: contact the sample in a closed configuration; the inner surfaces of the plates are substantially parallel to each other in the closed configuration; the inner surfaces of the plates are substantially planar except for locations with spacers; or any combination thereof.

[0260] Spacers can be manufactured on the plate in various ways using photolithography, etching, molding (nanoimprinting), deposition, peeling, fusion or a combination thereof. In certain embodiments, the spacers are directly molded or embossed on the plate. In certain embodiments, the spacers are embossed into the material (e.g., plastic) deposited on the plate. In certain embodiments, the spacers are manufactured by directly molding the surface of a CROF plate. Nanoimprinting can be performed using a roller imprinter or roller-to-plane nanoimprinting by roll-to-roll technology. This method has great economic advantages, thus reducing costs.

[0261] In some embodiments, the spacers are deposited on the plate. Deposition can be by evaporation, pasting, or stripping. In pasting, the spacers are first made on a carrier, and then the spacers are transferred from the carrier to the plate. In stripping, a removable material is first deposited on the plate and a hole is formed in the material; the bottom of the hole exposes the plate surface, and then the spacer material is deposited into the hole, after which the removable material is removed, leaving only the spacer on the plate surface. In some embodiments, the spacers deposited on the plate are fused to the plate. In some embodiments, the spacers and the plate are made in a single process. The single process includes embossing (e.g., embossing, molding) or synthesis.

[0262] In certain embodiments, at least two of the spacers are secured to the respective plates by different manufacturing methods, and optionally wherein the different manufacturing methods include at least one of deposition, bonding, fusing, embossing, and etching.

[0263] In certain embodiments, one or more of the spacers are secured to the respective plates by the following manufacturing methods: gluing, fusing, stamping, or etching, or any combination thereof.

[0264] In certain embodiments, the manufacturing method for forming such monolithic spacers on the board includes methods of bonding, fusing, embossing, or etching, or any combination thereof.

[0265] B) Adapter

[0266] The details of the adapter have been described in detail in various publications including International Application No. PCT / US2018 / 017504.

[0267] The invention described herein addresses this problem by providing a system comprising an optical adapter and a smartphone. The optical adapter device is mounted on the smartphone, transforming it into a microscope capable of acquiring both fluorescence and brightfield images of a sample. The system can be conveniently and reliably operated by ordinary personnel in any location. The optical adapter utilizes existing smartphone resources, including a camera, light source, processor, and display, providing a low-cost solution for users to perform both brightfield and fluorescence microscopy.

[0268] In the present invention, the optical adapter device includes a holder frame that fits onto the top of a smartphone and an optical box attached to the holder, which has a sample receptacle slot and illumination optics. In some references (e.g., U.S. Patent Nos. 2016 / 029091 and 2011 / 0292198), the optical adapter design is a single unit, including mechanical components that clamp onto the smartphone and the functional optical elements. The problem with this design is that the entire optical adapter must be redesigned for each specific smartphone model. However, in the present invention, the optical adapter is separated into a holder frame that only fits the smartphone and a universal optical box that contains all the functional components. For smartphones of different sizes, as long as the relative positions of the camera and light source remain the same, only the holder frame needs to be redesigned, saving significant design and manufacturing costs.

[0269] The optics box of the optical adapter contains: a receptacle slot that receives the sample within the field of view and focal length of the smartphone camera and positions the sample in the sample slide; brightfield illumination optics for capturing brightfield microscopic images of the sample; fluorescence illumination optics for capturing fluorescence microscopic images of the sample; and a lever for switching between brightfield illumination optics and fluorescence illumination optics by sliding the lever inward and outward in the optics box.

[0270] The receptacle slot has a rubber door attached to it that completely covers the slot, preventing ambient light from entering the optical box and being collected by the camera. In US Patent 2016 / 0290916, the sample slot is always exposed to ambient light, which is not a major problem because it only performs brightfield microscopy. However, the present invention can utilize this rubber door when performing fluorescence microscopy, as ambient light can introduce significant noise to the camera's image sensor.

[0271] In order to capture good fluorescence microscopy images, it is desirable that almost no excitation light enters the camera and only the fluorescence emitted by the sample is collected by the camera. However, for all common smartphones, due to the large divergence angle of the light beam emitted by the light source and the optical filter that does not work well for non-collimated beams, the optical filter placed in front of the camera cannot well block the light of the undesired wavelength range emitted from the smartphone's light source. Collimating optical devices can be designed to collimate the light beam emitted by the smartphone light source to solve this problem, but this approach increases the size and cost of the adapter. In contrast, in the present invention, the fluorescence illumination optical device enables the excitation light to illuminate the sample at a large oblique incident angle, partially from the waveguide inside the sample slide and partially from the back of the sample slide, so that the excitation light is almost not collected by the camera to reduce the noise signal entering the camera.

[0272] Brightfield illumination optics in the adapter receive and rotate the beam emitted by the light source to back-illuminate the sample at normal incidence.

[0273] Typically, the optical box also includes a magnifying lens mounted in the box, which is aligned with the smartphone's camera to magnify the image captured by the camera. The image captured by the camera can be further processed by the smartphone's processor and the analysis results can be output on the smartphone's screen.

[0274] To implement both brightfield and fluorescence illumination optics in the same optical adapter, as in the present invention, a slidable rod can be used. The optical elements of the fluorescence illumination optics are mounted on the rod. When the rod is fully slid into the optical cartridge, the fluorescence illumination optics block the optical path of the brightfield illumination optics, switching the illumination optics to fluorescence. Furthermore, when the rod is slid out, the fluorescence illumination optics mounted on the rod move out of the optical path, switching the illumination optics to brightfield illumination. This rod design allows the optical adapter to operate in both brightfield and fluorescence illumination modes without requiring the design of two separate single-mode optical cartridges.

[0275] The rod comprises two flat surfaces at different levels at different heights.

[0276] In some embodiments, the two planes can be connected together by a vertical rod and moved in or out of the optical box together. In some embodiments, the two planes can be separated and each plane can be moved in or out of the optical box separately.

[0277] The upper rod plane contains at least one optical element, which may be, but is not limited to, an optical filter. The upper rod plane moves below the light source, and a preferred distance between the upper rod plane and the light source is in the range of 0 to 5 mm.

[0278] A portion of the bottom rod plane is non-parallel to the image plane. The surface of the non-parallel portion of the bottom rod plane has a mirror finish with a high reflectivity greater than 95%. The non-parallel portion of the bottom rod plane moves beneath the light source and deflects light emitted from the light source to illuminate the sample area directly below the camera. The preferred inclination angle of the non-parallel portion of the bottom rod plane is in the range of 45 to 65 degrees, and this inclination angle is defined as the angle between the non-parallel bottom plane and a vertical plane.

[0279] A portion of the bottom rod plane is, for example, parallel to the image plane and can be located below the sample at a distance of 1 to 10 mm. The surface of this portion of the bottom rod plane is highly absorptive, absorbing greater than 95% of light. This absorptive surface eliminates reflected light that strikes the sample backward at low angles of incidence.

[0280] To allow the rod to slide in and out to switch illumination optics, a stopper design consisting of a detent ball and a groove on the rod is used to stop the rod at a predetermined position as it is pulled outward from the adapter. This allows the user to pull the rod with any force, but it stops in a fixed position, where the optical adapter switches to brightfield illumination.

[0281] The sample slider fits into the socket slot, accepts the QMAX device, and positions the sample in the QMAX device within the field of view and focal length of the smartphone camera.

[0282] The sample slide consists of a fixed track frame and a movable arm:

[0283] The frame rails are fixedly mounted in the receptacle slots of the optical box. The rail frame features sliding rail slots that are tailored to the width and thickness of the QMAX device, allowing it to slide along the rails. The width and height of the rail slots are carefully configured to ensure that the QMAX device can move within the sliding plane perpendicular to the sliding direction to less than 0.1 mm, and within the thickness of the QMAX device to less than 0.2 mm.

[0284] The frame rail has an open window under the field of view of the smartphone's camera to allow light to illuminate the sample backward.

[0285] The movable arm is pre-placed in the slide rail groove of the track frame and moves together with the QMAX device to guide the movement of the QMAX device in the track frame.

[0286] The movable arm is equipped with a stop mechanism with two predefined stop positions. In one position, the arm stops the QMAX device directly below the fixed sample area on the QMAX device. In the other position, the arm stops the QMAX device at a position where the sample area on the QMAX device is out of the field of view of the smartphone, and the QMAX device can be easily removed from the track slot.

[0287] The movable arm is switched between the two stop positions by being pressed to the end of the track groove by the QMAX device and the movable arm and then released.

[0288] A movable arm indicates whether the QMAX device is inserted in the correct orientation. One corner of the QMAX device is configured differently than the other three right-angle corners. The movable arm aligns the corner with the specific geometry, allowing the QMAX device to slide into the correct position in the track slot only when inserted in the correct orientation.

[0289] C) Smartphone / Detection System

[0290] Details of the smartphone / detection system are described in detail in multiple publications, including International Application (IA) No. PCT / US2016 / 046437, filed on August 10, 2016, IA No. PCT / US2016 / 051775, filed on September 14, 2016, U.S. Provisional Application No. 62 / 456,065, filed on February 7, 2017, U.S. Provisional Application Nos. 62 / 456,287 and 62 / 456,590, filed on February 8, 2017, U.S. Provisional Application No. 62 / 456,504, filed on February 8, 2017, U.S. Provisional Application No. 62 / 459,544, filed on February 15, 2017, and U.S. Provisional Application Nos. 62 / 460,075 and 62 / 459,920, filed on February 16, 2017.

[0291] The devices / apparatus, systems and methods disclosed herein may include or use a Q card for sample detection, analysis and quantification. In certain embodiments, the Q card is used in conjunction with an adapter that can connect the Q card to a smartphone detection system. In certain embodiments, the smartphone includes a camera and / or an illumination source. In certain embodiments, the smartphone includes a camera that can be used to capture an image or sample when the sample is in the camera's field of view (e.g., through an adapter). In certain embodiments, the camera includes a set of lenses (e.g., iPhone™ 6). In certain embodiments, the camera includes at least two sets of lenses (e.g., iPhone™ 7). In certain embodiments, the smartphone includes a camera, but the camera is not used for image capture.

[0292] In some embodiments, the smartphone includes a light source, such as, but not limited to, an LED (light emitting diode). In some embodiments, the light source is used to illuminate the sample when the sample is in the field of view of the camera (e.g., via an adapter). In some embodiments, the light from the light source is enhanced, amplified, modified, and / or optimized by the optical components of the adapter.

[0293] In certain embodiments, the smartphone includes a processor configured to process information from the sample. The smartphone includes software instructions that, when executed by the processor, can enhance, amplify, and / or optimize the signal (e.g., image) from the sample. The processor may include one or more hardware components, such as a central processing unit (CPU), an application specific integrated circuit (ASIC), an application specific instruction set processor (ASIP), a graphics processing unit (GPU), a physical processing unit (PPU), a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic device (PLD), a controller, a microcontroller unit, a reduced instruction set computer (RISC), a microprocessor, etc., or any combination thereof.

[0294] In some embodiments, the smartphone includes a communication unit configured and / or used to transmit data and / or images related to the sample to another device. By way of example only, the communication unit can use a cable network, a wired network, a fiber optic network, a telecommunications network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a wide area network (WAN), a public switched telephone network (PSTN), a Bluetooth network, a ZigBee network, a near field communication (NFC) network, or the like, or any combination thereof. In some embodiments, the smartphone is an iPhone™, an Android™ phone, or a Windows™ phone.

[0295] D) Manufacturing method

[0296] Details of the manufacturing methods are described in detail in various publications, including International Application No. PCT / US2018 / 057873, filed on October 26, 2018, which is incorporated herein by reference for all purposes.

[0297] The device of the present disclosure can be manufactured using techniques known in the art. The selection of manufacturing technology will depend on the material used for the device and the size of the spacer array and / or the size of the spacer. Exemplary materials for manufacturing the device of the present invention include glass, silicon, steel, nickel, polymers, such as polymethyl methacrylate (PMMA), polycarbonate, polystyrene, polyethylene, polyolefins, silicones (e.g., poly(dimethylsiloxane)), polypropylene, cis-polyisoprene (rubber), poly(vinyl chloride) (PVC), poly(vinyl acetate) (PVAc), polychloroprene (chloroprene rubber), polytetrafluoroethylene (Teflon), poly(vinylidene chloride) (Salon), and cycloolefin polymers (COP) and cycloolefin copolymers (COC), and combinations thereof. Other materials are known in the art. For example, deep reactive ion etching (DRIE) is used to manufacture silicon-based devices with small gaps, small spacers, and large aspect ratios (the ratio of spacer height to lateral dimension). Thermoforming (compression molding, injection molding) of plastic devices can also be used when the smallest lateral features are >20 microns and the aspect ratio of these features is ≤10.

[0298] Additional methods include photolithography (e.g., stereolithography or x-ray lithography), molding, embossing, silicon micromachining, wet or dry chemical etching, milling, diamond cutting, deep electroforming molding (LIGA), and electroplating. For example, for glass, conventional silicon manufacturing techniques of photolithography followed by wet (KOH) or dry etching (reactive ion etching with fluorine or other reactive gases) can be used. Techniques such as laser micromachining can be applied to plastic materials with high photon absorption efficiency. Due to the serial nature of the process, the technology is suitable for lower-volume manufacturing. For large-scale production of plastic devices, thermoplastic injection molding and compression molding are suitable. Conventional thermoplastic injection molding used for large-scale production of optical discs (which maintain the fidelity of submicron features) can also be used to manufacture the devices of the present invention. For example, device features are replicated on a glass master disc by conventional photolithography. The glass master disc is electroformed to produce a tough, thermal shock resistant, thermally conductive, hard mold. The mold serves as a master template for injection molding or compression molding the features into the plastic device. Depending on the plastic material used to manufacture the device and the requirements for optical quality and finished product yield, injection molding or compression molding can be selected as the manufacturing method. Compression molding (also known as hot stamping or letterpress embossing) has the advantage of being compatible with high molecular weight polymers, which is excellent for small structures and can replicate high aspect ratio structures, but has longer cycle times. Injection molding works well for low aspect ratio structures and is best suited for low molecular weight polymers.

[0299] The device can be made into one or more parts that are subsequently assembled. The layers of the device can be bonded together by clamps, adhesives, heat, anodic bonding, or reactions between surface groups (e.g., wafer bonding). Alternatively, a device having channels or gaps in more than one plane can be made as a single piece using stereolithography or other three-dimensional manufacturing techniques.

[0300] In order to reduce the non-specific adsorption of cells or compounds released by lysed cells on the device surface, one or more surfaces of the device can be chemically modified to be non-adhesive or repellent. The surface can be coated with a thin film coating (e.g., a monolayer) of a commercially available non-adhesive reagent (e.g., a reagent for forming a hydrogel). Other exemplary chemical substances that can be used to modify the device surface include oligoethylene glycols, fluorinated polymers, organosilanes, thiols, polyethylene glycols, hyaluronic acid, bovine serum albumin, polyvinyl alcohol, mucin, polyhydroxyethyl methacrylate, methacrylated PEG, and agarose. Charged polymers can also be used to repel substances with opposite charges. The type of chemical species used to repel and the method of attaching to the device surface can depend on the properties of the substance being repelled and the properties of the surface as well as the substance being attached. Such surface modification techniques are known in the art. The surface can be functionalized before or after assembling the device. The surface of the device can also be coated to capture certain materials in the sample, such as membrane fragments or proteins.

[0301] In certain embodiments of the present disclosure, a method for manufacturing any Q card of the present disclosure may include injection molding a first plate. In certain embodiments of the present disclosure, a method for manufacturing any Q card of the present disclosure may include nanoimprinting or extrusion printing a second plate. In certain embodiments of the present disclosure, a method for manufacturing any Q card of the present disclosure may include laser cutting a first plate. In certain embodiments of the present disclosure, a method for manufacturing any Q card of the present disclosure may include nanoimprinting or extrusion printing a second plate. In certain embodiments of the present disclosure, a method for manufacturing any Q card of the present disclosure may include injection molding and laser cutting a first plate. In certain embodiments of the present disclosure, a method for manufacturing any Q card of the present disclosure may include nanoimprinting or extrusion printing a second plate. In certain embodiments of the present disclosure, a method for manufacturing any Q card of the present disclosure may include nanoimprinting or extrusion printing to manufacture both the first and second plates. In certain embodiments of the present disclosure, a method for manufacturing any Q card of the present disclosure may include using injection molding, laser cutting a first plate, nanoimprinting, extrusion printing, or a combination thereof to manufacture the first plate or the second plate. In certain embodiments of the present disclosure, a method for manufacturing any Q-card of the present disclosure may include the step of attaching a hinge to the first and second panels after manufacturing the first and second panels.

[0302] E) Sample type and subjects

[0303] Details of the samples and subjects are described in detail in multiple publications, including International Application (IA) No. PCT / US2016 / 046437 filed on August 10, 2016, IA No. PCT / US2016 / 051775 filed on September 14, 2016, IA No. PCT / US201 / 017307 filed on February 7, 2018, and PCT / US2017 / 065440 filed on December 8, 2017.

[0304] The sample can be obtained from a subject. The subjects described herein can be of any age and can be adults, infants, or children. In some cases, the subject is 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57 , 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, or 99 years old, or in the range therein (for example, between 2 years old and 20 years old, between 20 years old and 40 years old, or between 40 years old and 90 years old). The subject of a particular category that can benefit is a subject suffering from or suspected of having an infection (for example, a bacterial infection and / or a viral infection). The subject of another particular category that can benefit is a subject that may be at a higher risk of infection. In addition, the subject treated by any method or composition as described herein can be male or female. Any method, device or kit disclosed herein can also be performed on non-human subjects, such as laboratory or farm animals. Non-limiting examples of non-human subjects include dogs, goats, guinea pigs, hamsters, mice, pigs, non-human primates (e.g., gorillas, apes, orangutans, lemurs or baboons), rats, sheep, cows or zebrafish.

[0305] The devices, apparatus, systems and methods disclosed herein can be used with samples, such as, but not limited to, diagnostic samples, clinical samples, environmental samples, and food samples.

[0306] For example, in certain embodiments, the devices, apparatus, systems, and methods disclosed herein are used with samples comprising cells, tissues, body fluids, and / or mixtures thereof. In certain embodiments, the sample comprises a human body fluid. In certain embodiments, the sample comprises at least one of the following: cells, tissues, body fluids, feces, amniotic fluid, aqueous humor, vitreous humor, blood, whole blood, fractionated blood, plasma, serum, breast milk, cerebrospinal fluid, cerumen, chyle, chyme, endolymph, perilymph, feces, gastric acid, gastric juice, lymph, mucus, nasal drainage, sputum, pericardial fluid, peritoneal fluid, pleural fluid, pus, rheumatic fluid, saliva, sebum, semen, sputum, sweat, synovial fluid, tears, vomitus, urine, and exhaled breath condensate.

[0307] In some embodiments, the devices, apparatuses, systems and methods disclosed herein are used for environmental samples obtained from any suitable source, such as, but not limited to, liquid samples from rivers, lakes, ponds, oceans, glaciers, icebergs, rainwater, snow, sewage, reservoirs, tap water, drinking water, etc.; solid samples from soil, compost, sand, rock, concrete, wood, brick, dirt, etc.; and gaseous samples from air, underwater heat dissipation, industrial exhaust, vehicle exhaust, etc. In certain embodiments, the environmental sample is a fresh sample obtained from a source; in certain embodiments, the environmental sample is processed. For example, a sample in a non-liquid form is converted to a liquid form before applying the subject apparatus, apparatus, system and method.

[0308] In certain embodiments, the devices, apparatuses, systems, and methods disclosed herein are used with food samples that are suitable or can be made suitable for animal consumption, such as human consumption. In certain embodiments, food samples can include raw materials, cooked or processed foods, foods of plant and animal origin, pre-processed foods, and partially or fully processed foods. In certain embodiments, samples in non-liquid form are converted to liquid form prior to application of the subject devices, apparatuses, systems, and methods.

[0309] The devices, apparatus, systems and methods of the present invention can be used to analyze samples of any volume. Examples of volumes include, but are not limited to, about 10 mL or less, 5 mL or less, 3 mL or less, 1 microliter ("uL") or less, 500 μL or less, 300 μL or less, 250 μL or less, 200 μL or less, 170 μL or less, 150 μL or less, 125 μL or less, 100 μL or less, 75 μL or less, 50 μL or less, 25 μL or less, 20 μL or less, 15 μL or less, 10 μL or less, 5 μL or less, 3 μL or less, 1 μL or less, 0.5 μL or less, 0.1 μL or less, 0.05 μL or less, 0.001 μL or less, 0.0005 μL or less, 0.0001 μL or less, 10 pL or less, 1 pL or less, or a range between any two of said values.

[0310] In certain embodiments, the volume of the sample includes, but is not limited to, about 100 μL or less, 75 μL or less, 50 μL or less, 25 μL or less, 20 μL or less, 15 μL or less, 10 μL or less, 5 μL or less, 3 μL or less, 1 μL or less, 0.5 μL or less, 0.1 μL or less, 0.05 μL or less, 0.001 μL or less, 0.0005 μL or less, 0.0001 μL or less, 10 pL or less, 1 pL or less, or a range between any two of said values. In certain embodiments, the volume of the sample includes, but is not limited to, about 10 μL or less, 5 μL or less, 3 μL or less, 1 μL or less, 0.5 μL or less, 0.1 μL or less, 0.05 μL or less, 0.001 μL or less, 0.0005 μL or less, 0.0001 μL or less, 10 pL or less, 1 pL or less, or a range between any two of said values.

[0311] In some embodiments, the sample is in an amount of about a drop of liquid. In some embodiments, the sample is in an amount collected from a finger prick or fingerstick. In some embodiments, the sample is in an amount collected from a microneedle, micropipette, or intravenous aspiration.

[0312] F) Machine Learning

[0313] Details of the network are described in various publications, including International Application Nos. PCT / US2018 / 017504, filed February 8, 2018, and PCT / US2018 / 057877, filed October 26, 2018, each of which is incorporated herein by reference for all purposes.

[0314] One aspect of the present invention provides a framework for machine learning and deep learning for analyte detection and localization. Machine learning algorithms are algorithms that can learn from data. A more rigorous definition of machine learning is "a computer program is said to learn from experience E about some class of tasks T and performance measure P if performance at tasks in T, as measured by P, improves with experience E." The exploration and construction of algorithms that can learn and predict from data—such algorithms overcome static program instructions by building models based on sample inputs and making data-driven predictions or decisions.

[0315] Deep learning is a specific type of machine learning based on a set of algorithms that attempt to model high-level abstractions in data. In a simple case, there might be two groups of neurons: those that receive input signals and those that send output signals. When an input layer receives input, it passes a modified version of the input to the next layer. In a deep network, there are many layers between the input and output (and these layers are not made up of neurons, but it can be helpful to think of it that way), allowing the algorithm to use multiple processing layers composed of multiple linear and nonlinear transformations.

[0316] One aspect of the present invention is to provide two methods for analyte detection and localization. The first method is a deep learning method, and the second method is a combination of deep learning and computer vision methods.

[0317] (i) Deep Learning Approach. In the first approach, the disclosed analyte detection and localization workflow consists of two phases: training and prediction. We describe the training and prediction phases in the following paragraphs.

[0318] (a) Training phase

[0319] During the training phase, the annotated training data is fed into a convolutional neural network. A convolutional neural network is a specialized neural network with a grid-like, feed-forward, layered network topology for processing data. Examples of data include time series data, which can be thought of as a 1D grid sampled at regular intervals, and image data, which can be thought of as a 2D grid of pixels. Convolutional networks have achieved success in practical applications. The name "convolutional neural network" indicates that the network employs a mathematical operation called convolution. Convolution is a specialized linear operation. A convolutional network is simply a neural network that uses convolution instead of conventional matrix multiplication in at least one of its layers.

[0320] The machine learning model receives as training data one or more images of a sample containing an analyte, acquired by an imaging system above a sample-holding QMAX device. The training data is annotated for the analyte to be measured, where the annotation indicates whether the analyte is present in the training data and its location in the image. Annotation can be in the form of a tight bounding box that completely encompasses the analyte or the center location of the analyte. In the latter case, the center location is further converted into a Gaussian kernel in a circle or dot plot that covers the analyte.

[0321] When the training data is large, training machine learning models faces two challenges: annotation (usually done by humans) is time-consuming, and training is computationally expensive. To overcome these challenges, the training data can be split into small chunks, and then these chunks or parts of these chunks are annotated and trained. The term "machine learning" can refer to algorithms, systems, and devices in the field of artificial intelligence, which typically use statistical techniques and artificial neural networks trained on data without explicit programming.

[0322] The annotated images can be fed into a machine learning (ML) training module, and a model trainer in the machine learning module can train an ML model based on the training data (annotated sample images). The input data is fed into the model trainer multiple times in iterations until a specific stopping criterion is met. The output of the ML training module is an ML model—a computational model built from the data according to a training process in machine learning that gives computers the ability to independently perform certain tasks (e.g., detecting and classifying objects).

[0323] A machine learning model trained by a computer application during the prediction (or inference) phase. Examples of machine learning models include ResNet, DenseNet, etc., which are also referred to as "deep learning models" due to the depth of the associated layers in their network structure. In some embodiments, the Caffe library with a fully convolutional network (FCN) is used for model training and prediction, and other convolutional neural network architectures and libraries, such as TensorFlow, may also be used.

[0324] The training phase generates a model to be used in the prediction phase. This model can be reused in the prediction phase to measure inputs. Therefore, the compute unit only needs access to the generated model. It does not require access to the training data, nor does it need to run the training phase again on the compute unit.

[0325] (b) Prediction stage

[0326] In the prediction / inference phase, the detection component is applied to the input image, and the input image is fed into the prediction (inference) module pre-loaded with the training model generated from the training phase. The output of the prediction phase can be a bounding box containing the detected analytes, with a center position or a dot map indicating the location of each analyte, or a heat map containing information about the detected analytes.

[0327] When the output of the prediction phase is a list of bounding boxes, the number of analytes in the image of the sample being assayed is characterized by the number of bounding boxes detected. When the output of the prediction phase is a dot map, the number of analytes in the image of the sample being assayed is characterized by the integral of the dot map. When the output of the prediction phase is a heat map, a localization component is used to identify the locations, and the number of analytes detected is characterized by the entries in the heat map.

[0328] One embodiment of the localization algorithm is to sort the heatmap values ​​into a one-dimensional ordered list from highest to lowest. Then, the pixel with the highest value is picked and removed from the list along with its neighboring pixels. This process is repeated to pick the pixel with the highest value in the list until all pixels are removed from the list.

[0329] In the detection component using heatmaps, the input image is fed to a convolutional neural network along with the model generated from the training phase, and the output of the detection phase is a pixel-level prediction in the form of a heatmap. The heatmap can have the same size as the input image, or it can be a scaled-down version of the input image, and it is the input to the localization component. We disclose an algorithm for locating the center of the analyte. The main idea is to iteratively detect local peaks based on the heatmap. After the peak is located, we calculate the local area around the peak but with smaller values. We remove this area from the heatmap and find the next peak from the remaining pixels. The process is repeated, only to remove all pixels from the heatmap.

[0330] In certain embodiments, the present invention provides a localization algorithm to sort the heatmap values ​​from highest to lowest into a one-dimensional ordered list. The pixel with the highest value is then picked and removed from the list along with its neighboring pixels. This process is repeated to pick the pixel with the highest value in the list until all pixels have been removed from the list.

[0331] Algorithm GlobalSearch(heatmap)

[0332] Input:

[0333] heatmap

[0334] Output:

[0335] loci

[0336] loci←{}

[0337] sort(heatmap)

[0338] while(heatmap is not empty){

[0339] s←pop(heatmap)

[0340] D←{disk center as s with radius R}

[0341] heatmap=heatmap\D / / remove D from the heatmap

[0342] add s to loci

[0343] }

[0344] The sorted heatmap is a one-dimensional ordered list where the heatmap values ​​are sorted from highest to lowest. Each heatmap value is associated with its corresponding pixel coordinate. The first item in the heatmap is the item with the highest value, which is the output of the pop(heatmap) function. A disk is created where the center is the pixel coordinate of the first item with the highest heatmap value. All heatmap values ​​whose pixel coordinates fall within the disk are then removed from the heatmap. The algorithm repeatedly pops the highest value in the current heatmap, removing the disks around it, until the item is removed from the heatmap.

[0345] In an ordered list heatmap, each item is aware of the item that follows it and the item that follows it. When an item is removed from an ordered list, we make the following changes:

[0346] Assume the removed item is x r , its continuation item is x p , its successor item is x f .

[0347] For ongoing projects x p , redefine its successor to be the successor of the removed item. Therefore, x p The subsequent item is now x f .

[0348] For removing item x r , undefines its continuation and successor items, removing them from the ordered list.

[0349] For the following items x f , redefines its continuation as the continuation of the removed item. So now x f The next project is X p .

[0350] After all items are removed from the sorted list, the localization algorithm ends. The number of elements in the set site will be the count of analytes, and the position information is the pixel coordinates of each s in the set site.

[0351] Another embodiment searches for local peaks that are not necessarily the local peak with the highest heatmap value. To detect each local peak, we start from a random starting point and search for the local maximum. Once a peak is found, the local area around the peak is calculated, but the value is smaller. We remove this area from the heatmap and find the next peak from the remaining pixels. This process is repeated, only to remove all pixels from the heatmap.

[0352] Algorithm LocalSearch(s,heatmap)

[0353] Input:

[0354] s:starting location(x,y)

[0355] heatmap

[0356] Output:

[0357] s:location of local peak.

[0358] We only consider pixels of value>0.

[0359] Algorithm Cover(s,heatmap)

[0360] Input:

[0361] s:location of local peak.

[0362] heatmap:

[0363] Output:

[0364] cover:a set of pixels covered by peak:

[0365] This is a breadth-first search algorithm starting from s, with a changing condition for visiting points: if heatmap[p]>0 and heatmap[p]<=heatmap[q], then only the neighbors p of the current position q are added to the cover. Therefore, every pixel in the cover has a non-descending path to the local peak s.

[0366] Algorithm Localization(heatmap)

[0367] Input:

[0368] heatmap

[0369] Output:

[0370] loci

[0371] loci←{}

[0372] pixels←{all pixels from heatmap}

[0373] while pixels is not empty

[0374] s←any pixel from pixels

[0375] s←LocalSearch(s,heatmap) / / s is now local peak

[0376] probe local region of radius R surrounding s for better local peak

[0377] r←Cover(s,heatmap)

[0378] pixels←pixels\r / / remove all pixels in cover

[0379] add s to loci

[0380] (ii) A method combining deep learning and computer vision. In the second method, detection and localization are achieved by a computer vision algorithm, and classification is achieved by a deep learning algorithm, wherein the computer vision algorithm detects and localizes possible candidates for the analyte, and the deep learning algorithm classifies each possible candidate as a true analyte or a false analyte. The positions of all true analytes (along with the total count of true analytes) will be recorded as output.

[0381] (a) Detection. Computer vision algorithms detect possible candidates based on the characteristics of the analyte (including but not limited to intensity, color, size, shape, distribution, etc.). Preprocessing schemes can improve the detection efficiency. Preprocessing schemes include contrast enhancement, histogram adjustment, color enhancement, denoising, smoothing, defocusing, etc. After preprocessing, the input image is fed into the detector. The detector tells the presence of possible candidates for the analyte and gives an estimate of its location. Using schemes such as adaptive thresholding, detection can be based on analyte structure (e.g., edge detection, line detection, circle detection, etc.), connectivity (e.g., spot detection, connected components, contour detection, etc.), intensity, color, shape, etc.

[0382] (b) Localization. After detection, a computer vision algorithm locates each possible candidate for the analyte by providing its boundaries or a tight bounding box containing it. This can be achieved through object segmentation algorithms such as adaptive thresholding, background subtraction, fill color, mean shift, watershed, etc. Typically, localization can be combined with detection to produce a detection result and the location of each possible candidate for the analyte.

[0383] (c) Classification. Deep learning algorithms such as convolutional neural networks have enabled the development of visual classification techniques. We employ deep learning algorithms to classify each possible candidate for an analyte. Various convolutional neural networks can be used for analyte classification, such as VGGNet, ResNet, MobileNet, and DenseNet.

[0384] Given each possible candidate for an analyte, a deep learning algorithm computes through layers of neurons via convolutional and nonlinear filters to extract high-level features that distinguish the analyte from non-analytes. A layer of fully convolutional networks combines the high-level features into a classification result, which tells whether it is a true analyte, or the probability of it being an analyte.

[0385] G) Applications, biological / chemical biomarkers and health conditions

[0386] Applications of the present invention include, but are not limited to (a) detection, purification and quantification of compounds or biomolecules associated with certain disease stages, such as infectious and parasitic diseases, injuries, cardiovascular diseases, cancers, mental disorders, neuropsychiatric disorders and organic diseases, such as lung disease and kidney disease, (b) detection, purification and quantification of microorganisms such as viruses, fungi and bacteria from the environment, such as water, soil or biological samples, such as tissues and body fluids, (c) detection and quantification of chemical compounds or biological samples that pose a threat to food safety or national security, such as toxic waste, anthrax, (d) quantification of vital parameters in medical or physiological monitors, such as glucose, blood oxygen levels, total blood cell counts, (e) detection and quantification of specific DNA or RNA from biological samples, such as cells, viruses, body fluids, (f) sequencing and comparison of genetic sequences of chromosomal and mitochondrial DNA for genomic analysis, or (g) detection of reaction products, such as during drug synthesis or purification.

[0387] Detection can be carried out in various sample matrices, such as cells, tissues, body fluids and feces. Target body fluids include, but are not limited to, amniotic fluid, aqueous humor, vitreous humor, blood (e.g., whole blood, fractionated blood, plasma, serum), breast milk, cerebrospinal fluid (CSF), cerumen (earwax), chyle, chyme, endolymph, perilymph, excreta, gastric acid, gastric juice, lymph, mucus (including nasal drainage and sputum), pericardial fluid, peritoneal fluid, pleural fluid, pus, rheumatic fluid, saliva, sebum (skin oil), semen, sputum, sweat, synovial fluid, tears, vomitus, urine and exhaled condensate. In certain embodiments, the sample comprises human body fluids. In certain embodiments, the sample comprises at least one of: cells, tissue, body fluid, feces, amniotic fluid, aqueous humor, vitreous humor, blood, whole blood, fractionated blood, plasma, serum, breast milk, cerebrospinal fluid, cerumen, chyle, chyme, endolymph, perilymph, feces, gastric acid, gastric juice, lymph, mucus, nasal drainage, sputum, pericardial fluid, peritoneal fluid, pleural fluid, pus, rheumatic fluid, saliva, sebum, semen, sputum, sweat, synovial fluid, tears, vomitus, urine, and exhaled breath condensate.

[0388] In some embodiments, the sample is at least one of a biological sample, an environmental sample, and a biochemical sample.

[0389] The devices, systems, and methods of the present invention are useful in a variety of applications across a wide range of fields where it is desirable to determine the presence or absence and / or quantify one or more analytes in a sample. For example, the methods can be used to detect proteins, peptides, nucleic acids, synthetic compounds, inorganic compounds, and the like. These various fields include, but are not limited to, human, veterinary, agricultural, food, environmental, and pharmaceutical testing.

[0390] In certain embodiments, the methods of the present invention can be used to detect nucleic acids, proteins or other biomolecules in a sample. The methods may include detecting groups of biomarkers in a sample, such as two or more different protein or nucleic acid biomarkers. For example, the methods can be used for rapid clinical detection of two or more disease biomarkers in a biological sample, for example, which can be used to diagnose a disease condition in a subject, or for ongoing management or treatment of a disease condition in a subject. As described above, communication with a physician or other health care provider can better ensure that the physician or other health care provider is aware and aware that they may be concerned and therefore more likely to take appropriate measures.

[0391] For example, applications of the devices, systems, and methods of the present invention using CROF devices include, but are not limited to, (a) detection, purification, and quantification of compounds or biomolecules associated with certain disease stages, such as infectious and parasitic diseases, injuries, cardiovascular diseases, cancers, psychiatric disorders, neuropsychiatric disorders, and organic diseases, such as lung disease and kidney disease, (b) detection, purification, and quantification of microorganisms, such as viruses, fungi, and bacteria, from the environment, such as water, soil, or biological samples, such as tissues and body fluids, (c) detection and quantification of chemical compounds or biological samples that pose a threat to food safety or national security, such as toxic waste and anthrax, (d) quantification of vital parameters, such as glucose, blood oxygen levels, and total blood cell counts, in medical or physiological monitors, (e) detection and quantification of specific DNA or RNA from biological samples, such as cells, viruses, and body fluids, (f) sequencing and comparison of genetic sequences of chromosomal and mitochondrial DNA for genomic analysis, or (g) detection of reaction products, such as during drug synthesis or purification. Some specific applications of the devices, systems, and methods of the present invention will now be described in more detail.

[0392] For example, applications of the present invention include, but are not limited to, (a) detection, purification, and quantification of compounds or biomolecules associated with certain disease stages, such as infectious and parasitic diseases, injuries, cardiovascular diseases, cancers, mental disorders, neuropsychiatric disorders, and organic diseases, such as lung disease and kidney disease; (b) detection, purification, and quantification of microorganisms, such as viruses, fungi, and bacteria, from the environment, such as water, soil, or biological samples, such as tissues and body fluids; (c) detection and quantification of chemical compounds or biological samples that pose a threat to food safety or national security, such as toxic waste and anthrax;

[0393] (d) quantification of vital parameters in medical or physiological monitors, such as glucose, blood oxygen levels, total blood cell count, (e) detection and quantification of specific DNA or RNA from biological samples, such as cells, viruses, body fluids, (f) sequencing and comparison of genetic sequences of chromosomal and mitochondrial DNA for genomic analysis, or (g) detection of reaction products, such as during drug synthesis or purification.

[0394] Implementation of the devices, systems, and methods of the present invention may include a) obtaining a sample, b) applying the sample to a CROF device containing a capture agent bound to an analyte of interest under conditions suitable for binding of the analyte in the sample to the capture agent, c) washing the CROF device, and d) reading the CROF device to obtain a measurement of the amount of analyte in the sample. In certain embodiments, the analyte may be a biomarker, an environmental marker, or a food marker. In some cases, the sample is a liquid sample and may be a diagnostic sample (e.g., saliva, serum, blood, sputum, urine, sweat, tears, semen, or mucus); an environmental sample obtained from a river, ocean, lake, rainwater, snow, sewage, sewage runoff, agricultural runoff, industrial runoff, tap water, or drinking water; or a food sample obtained from tap water, drinking water, prepared food, processed food, or unprocessed food.

[0395] In any embodiment, the CROF device can be placed in a microfluidic device, and applying step b) can include applying the sample to the microfluidic device containing the CROF device.

[0396] In any embodiment, reading step d) may include detecting a fluorescent or luminescent signal from the CROF device.

[0397] In any embodiment, the reading step d) may comprise reading the CROF device with a handheld device configured to read the CROF device. The handheld device may be a mobile phone, such as a smartphone.

[0398] In any embodiment, the CROF device can include a labeling reagent that can bind to the analyte-capture reagent complex on the CROF device.

[0399] In any embodiment, between steps c) and d), the devices, systems, and methods of the present invention may further comprise the steps of applying a labeling reagent that binds to the analyte-capture agent complex on the CROF device to the CROF device, and washing the CROF device.

[0400] In any embodiment, the reading step d) may include reading an identifier of the CROF device. The identifier may be an optical barcode, a radio frequency ID tag, or a combination thereof.

[0401] In any embodiment, the devices, systems, and methods of the present invention may further comprise applying a control sample to a control CROF device comprising a capture agent that binds the analyte, wherein the control sample comprises a known detectable amount of the analyte, and reading the control CROF device to obtain a control measurement of the known detectable amount of the analyte in the sample.

[0402] In any embodiment, the sample can be a diagnostic sample obtained from a subject, the analyte can be a biomarker, and the amount of the analyte in the sample measured can be diagnostic of a disease or condition.

[0403] In any embodiment, the devices, systems, and methods of the present invention may further include receiving or providing a report to the subject, the report indicating the measured amounts of the biomarkers and the range of measured values ​​of the biomarkers in individuals who do not have or are at low risk of having a disease or condition, wherein the measured amounts of the biomarkers relative to the range of measured values ​​are used to diagnose the disease or condition.

[0404] In any embodiments, the devices, systems, and methods of the present invention may further include diagnosing the subject based on information including the measured amounts of the biomarkers in the sample. In some cases, the diagnosing step includes sending data including the measured amounts of the biomarkers to a remote location and receiving a diagnosis based on the information including the measurements from the remote location.

[0405] In any embodiment, applying step b) can include isolating miRNA from the sample to produce an isolated miRNA sample, and applying the isolated miRNA sample to a disk-coupled rod antenna (CROF device) array.

[0406] In any embodiment, the method can include receiving or providing a report indicating the safety or harmfulness to the subject of exposure to the environment from which the sample was obtained.

[0407] In any embodiment, the method can include transmitting data comprising the measured amount of the environmental marker to a remote location and receiving a report indicating the safety or hazardousness to the subject of exposure to the environment from which the sample was obtained.

[0408] In any embodiment, the CROF device array can include a plurality of capture agents, each capture agent binding an environmental marker, and wherein reading step d) can include obtaining a measurement of the amount of the plurality of environmental markers in the sample.

[0409] In any embodiment, the sample can be a food sample, wherein the analyte can be a food marker, and wherein the amount of the food marker in the sample can be correlated with the safety of the food being consumed.

[0410] In any embodiment, the method can include receiving or providing a report indicating the safety or hazards of consuming by the subject a food from which the sample was obtained.

[0411] In any embodiment, the method can include transmitting data containing the measured amounts of the food markers to a remote location and receiving a report indicating the safety or harmfulness of the subject consuming the food from which the sample was obtained.

[0412] In any embodiment, the CROF device array can include a plurality of capture agents, each capture agent bound to a food marker, wherein the obtaining can include obtaining measurements of the amounts of the plurality of food markers in the sample, and wherein the amounts of the plurality of food markers in the sample can be correlated to the safety of the food for consumption.

[0413] Also provided herein are kits for practicing the devices, systems, and methods of the present invention.

[0414] The sample size can be about one drop of sample. The sample size can be the amount collected from a finger prick or fingertip. The sample size can be the amount collected from a microneedle or intravenous aspiration.

[0415] The sample can be used without further processing after being obtained from the source, or can be processed, for example, to enrich for an analyte of interest, remove large particulate matter, dissolve or resuspend a solid sample, etc.

[0416] Any suitable method for applying the sample to the CROF device can be used. Suitable methods may include using a pipette, dropper, syringe, etc. In certain embodiments, when the CROF device is on a holder in the form of a dipstick, as described below, the sample can be applied to the CROF device by immersing the sample receiving area of ​​the dipstick into the sample.

[0417] Samples can be collected one or more times. Samples collected over time can be aggregated and / or processed individually (as described herein, by applying to a CROF device and obtaining measurements of the amount of analyte in the sample). In some cases, the measurements obtained over time can be aggregated and used for longitudinal analysis over time to facilitate screening, diagnosis, treatment, and / or disease prevention.

[0418] Washing the CROF device to remove unbound sample components can be performed in any convenient manner, as described above. In certain embodiments, the surface of the CROF device is washed with a binding buffer to remove unbound sample components.

[0419] The detectable labeling of the analyte can be carried out by any convenient method. The analyte can be directly or indirectly labeled. In direct labeling, the analyte in the sample is labeled before the sample is applied to the CROF device. In indirect labeling, the unlabeled analyte in the sample is labeled after the sample is applied to the CROF device to capture the unlabeled analyte, as described below.

[0420] Samples from a subject, the subject's health, and other applications of the invention are further described below. Exemplary samples, health conditions, and applications are also disclosed, for example, in U.S. Publication Nos. 2014 / 0154668 and 2014 / 0045209, which are incorporated herein by reference.

[0421] The present invention can be used in a variety of applications, among which these applications are generally analyte detection applications, in which the presence of a specific analyte in a given sample is detected at least qualitatively (if not quantitatively). The protocols for conducting analyte detection assays are well known to those skilled in the art and do not need to be described in detail here. Typically, a sample suspected of containing the analyte of interest is contacted with the surface of a target nanosensor under conditions sufficient to allow the analyte to bind to its respective capture agent, which is bound to the sensor. The capture agent has a highly specific affinity for the target molecule of interest. This affinity can be an antigen-antibody reaction, in which the antibody binds to a specific epitope on the antigen, or a DNA / RNA or DNA / RNA hybridization reaction, which is sequence-specific between two or more complementary strands of nucleic acid. Therefore, if the analyte of interest is present in the sample, it may bind to the sensor at the site of the capture agent and form a complex on the sensor surface. That is, the captured analyte is immobilized on the sensor surface. After removing unbound analyte, the presence of the bound complex (e.g., the immobilized analyte of interest) on the sensor surface is then detected, for example, using a labeled second capture agent.

[0422] Specific analyte detection applications of interest include hybridization assays in which nucleic acid capture agents are used and protein binding assays in which polypeptides (e.g., antibodies) are used. In these assays, a sample is first prepared, and after sample preparation, the sample is contacted with the target nanosensor under specific binding conditions, thereby forming a complex between the target nucleic acid or polypeptide (or other molecule) that is complementary to the capture agent attached to the sensor surface.

[0423] In one embodiment, the capture oligonucleotide is a synthetic single-stranded DNA of 20-100 base length, one end of which is thiolated. These molecules are fixed on the surface of the nanodevice to capture the target single-stranded DNA (which can be at least 50bp long) with a sequence complementary to the fixed capture DNA. After the hybridization reaction, a detection single-stranded DNA (length can be 20-100bp) whose sequence is complementary to the unoccupied nucleic acid of the target DNA is added to hybridize with the target. One end of the detection DNA is conjugated with a fluorescent marker, and the emission wavelength of the fluorescent marker is within the plasma resonance of the nanodevice. Therefore, by detecting the fluorescent emission emitted from the nanodevice surface, the target single-stranded DNA can be accurately detected and quantified. The length of the capture and detection DNA determines the melting temperature (nucleotide chains will separate above the melting temperature), and the degree of mismatch (the longer the chain, the lower the mismatch).

[0424] One of the concerns in selecting the length of complementary binding is the need to minimize mismatches while keeping the melting temperature as high as possible. In addition, the total length of the hybridization length is determined to obtain optimal signal amplification.

[0425] The subject sensor can be used in a method for diagnosing a disease or condition, comprising: (a) obtaining a liquid sample from a patient suspected of having the disease or condition, (b) contacting the sample with a test nanosensor, wherein the capture agent of the nanosensor specifically binds to a biomarker of the disease, and wherein the contact is performed under conditions suitable for specific binding of the biomarker to the capture agent; (c) removing any biomarker not bound to the capture agent; and (d) reading a light signal from the biomarker that remains bound to the nanosensor, wherein the light signal indicates that the patient has the disease or condition, wherein the method further comprises labeling the biomarker with a luminescent label before or after the biomarker binds to the capture agent. As will be described in more detail below, the patient may be suspected of having cancer, and the antibody binds to a cancer biomarker. In other embodiments, the patient is suspected of having a neurological disorder, and the antibody binds to a biomarker of a neurological disorder.

[0426] For example, applications of the subject sensors include, but are not limited to, (a) detection, purification, and quantification of compounds or biomolecules associated with certain disease stages, such as infectious and parasitic diseases, injuries, cardiovascular diseases, cancers, psychiatric disorders, neuropsychiatric disorders, and organic diseases, such as lung disease and kidney disease; (b) detection, purification, and quantification of microorganisms, such as viruses, fungi, and bacteria, from the environment, such as water, soil, or biological samples, such as tissues and body fluids; (c) detection and quantification of chemical compounds or biological samples that pose a threat to food safety or national security, such as toxic waste and anthrax; (d) quantification of vital parameters, such as glucose, blood oxygen levels, and total blood cell counts, in medical or physiological monitors;

[0427] (e) detection and quantification of specific DNA or RNA from biological samples, such as cells, viruses, body fluids, (f) sequencing and comparison of genetic sequences of chromosomal and mitochondrial DNA for genomic analysis, or (g) detection of reaction products, for example, during drug synthesis or purification.

[0428] Detection can be performed in a variety of sample matrices, such as cells, tissues, body fluids, and feces. Target body fluids include, but are not limited to, amniotic fluid, aqueous humor, vitreous humor, blood (e.g., whole blood, fractionated blood, plasma, serum), breast milk, cerebrospinal fluid (CSF), cerumen (earwax), chyle, chyme, endolymph, perilymph, feces, gastric acid, gastric juice, lymph, mucus (including nasal drainage and sputum), pericardial fluid, peritoneal fluid, pleural fluid, pus, rheumatic fluid, saliva, sebum (skin oil), semen, sputum, sweat, synovial fluid, tears, vomitus, urine, and exhaled condensate.

[0429] In certain embodiments, the subject biosensors can be used to diagnose pathogen infection by detecting a target nucleic acid of the pathogen in a sample. The target nucleic acid can be, for example, selected from the group consisting of human immunodeficiency viruses 1 and 2 (HIV-1 and HIV-2), human T-cell leukemia virus 2 (HTLV-1 and HTLV-2), respiratory syncytial virus (RSV), adenovirus, hepatitis B virus (HBV), hepatitis C virus (HCV), Epstein-Barr virus (EBV), human papillomavirus (HPV), varicella-zoster virus (VZV), cytomegalovirus (CMV), herpes simplex virus 1 and 2 (HSV-1 and HSV-2), human herpesvirus 8 (HHV-8, also known as Kaposi's sarcoma herpesvirus), and flaviviruses, including yellow fever virus, dengue virus, Japanese encephalitis virus, West Nile virus, and Ebola virus. However, the invention is not limited to the detection of nucleic acids, such as DNA or RNA sequences, from the abovementioned viruses, but can be applied without problems to other pathogens of importance in veterinary and / or human medicine.

[0430] Human papillomavirus (HPV) is further subdivided into over 70 different types based on the homology of their DNA sequences. These types cause different diseases. HPV types 1, 2, 3, 4, 7, 10, and 26-29 cause benign warts. HPV types 5, 8, 9, 12, 14, 15, 17, 19-25, and 46-50 cause lesions in patients with weakened immune systems. Types 6, 11, 34, 39, 41-44, and 51-55 cause benign condylar warts on the genital area and respiratory mucosa. HPV types 16 and 18 are of particular medical significance because they cause epithelial dysplasia in the genital mucosa and are associated with a high rate of invasive cancers of the cervix, vagina, vulva, and anal canal. Integration of HPV DNA is considered a decisive factor in the development of cervical cancer. Human papillomaviruses can be detected, for example, from the DNA sequences of their capsid proteins L1 and L2. Therefore, the method of the present invention is particularly suitable for detecting DNA sequences of HPV type 16 and / or HPV type 18 in tissue samples for assessing the risk of cancer development.

[0431] In some cases, nanosensors can be used to detect biomarkers that are present in low concentrations. For example, nanosensors can be used to detect cancer antigens in readily accessible body fluids (e.g., blood, saliva, urine, tears, etc.), detect biomarkers of tissue-specific diseases (e.g., biomarkers of neurological disorders (e.g., Alzheimer's disease antigens)) in readily accessible body fluids, detect infections (particularly detecting low titers of latent viruses, such as HIV), detect fetal antigens in maternal blood, and detect exogenous compounds (e.g., drugs or pollutants) in a subject's bloodstream.

[0432] The following table provides a list of protein biomarkers and their associated diseases that can be detected using the subject nanosensors (when used in conjunction with appropriate monoclonal antibodies). A potential source of the biomarker (e.g., "CSF"; cerebrospinal fluid) is also shown in the table. In many cases, the subject biosensors can detect biomarkers in a body fluid different from the indicated body fluid. For example, a biomarker found in CSF can be identified in urine, blood, or saliva.

[0433] H) Practicality

[0434] The present method can be used in a variety of applications where it is desired to determine the presence or absence and / or quantification of one or more analytes in a sample. For example, the present method can be used to detect proteins, peptides, nucleic acids, synthetic compounds, inorganic compounds, and the like.

[0435] In certain embodiments, the methods of the present invention can be used to detect nucleic acids, proteins or other biomolecules in a sample. The methods may include detecting groups of biomarkers in a sample, such as two or more different protein or nucleic acid biomarkers. For example, the methods can be used for rapid clinical detection of two or more disease biomarkers in a biological sample, for example, which can be used to diagnose a disease condition in a subject, or for ongoing management or treatment of a disease condition in a subject. As described above, communication with a physician or other health care provider can better ensure that the physician or other health care provider is aware and aware that they may be concerned and therefore more likely to take appropriate measures.

[0436] For example, applications of the devices, systems, and methods of the present invention using CROF devices include, but are not limited to, (a) detection, purification, and quantification of compounds or biomolecules associated with certain disease stages, such as infectious and parasitic diseases, injuries, cardiovascular diseases, cancers, psychiatric disorders, neuropsychiatric disorders, and organic diseases, such as lung disease and kidney disease, (b) detection, purification, and quantification of microorganisms, such as viruses, fungi, and bacteria, from the environment, such as water, soil, or biological samples, such as tissues and body fluids, (c) detection and quantification of chemical compounds or biological samples that pose a threat to food safety or national security, such as toxic waste and anthrax, (d) quantification of vital parameters, such as glucose, blood oxygen levels, and total blood cell counts, in medical or physiological monitors, (e) detection and quantification of specific DNA or RNA from biological samples, such as cells, viruses, and body fluids, (f) sequencing and comparison of genetic sequences of chromosomal and mitochondrial DNA for genomic analysis, or (g) detection of reaction products, such as during drug synthesis or purification. Some specific applications of the devices, systems, and methods of the present invention will now be described in more detail.

[0437] I) Diagnostic methods

[0438] In certain embodiments, the methods of the present invention can be used to detect biomarkers. In certain embodiments, the devices, systems, and methods of the present invention using CROF are used to detect the presence or absence of specific biomarkers, as well as increases or decreases in the concentration of specific biomarkers in blood, plasma, serum, or other body fluids or excretions, such as, but not limited to, urine, blood, serum, plasma, saliva, semen, prostatic fluid, nipple aspirate, tears, sweat, feces, cheek swabs, cerebrospinal fluid, cell lysate samples, amniotic fluid, gastrointestinal fluid, biopsy tissue, etc. Thus, samples, such as diagnostic samples, can include various fluids or solid samples.

[0439] In some cases, the sample can be a body fluid sample from a subject to be diagnosed. In some cases, a solid or semi-solid sample can be provided. The sample can include tissues and / or cells collected from a subject. The sample can be a biological sample. Examples of biological samples can include, but are not limited to, blood, serum, plasma, nasal swabs, nasopharyngeal washes, saliva, urine, gastric juice, spinal fluid, tears, feces, mucus, sweat, earwax, oil, glandular secretions, cerebrospinal fluid, tissue, semen, vaginal fluid, interstitial fluid from tumor tissue, eye fluid, spinal fluid, throat swabs, breath, hair, fingernails, skin, biopsy, placental fluid, amniotic fluid, umbilical cord blood, lymph, cavity fluid, sputum, pus, microbial flora, meconium, breast milk, exhaled condensate and / or other excreta. The sample can include nasopharyngeal washes. Nasal swabs, throat swabs, stool samples, hair, fingernails, earwax, breath, and other solid, semisolid, or gaseous samples can be processed in an extraction buffer, e.g., for a fixed or variable amount of time prior to analysis. If desired, the extraction buffer or an aliquot thereof can then be processed similarly to other fluid samples. Examples of tissue samples from a subject can include, but are not limited to, connective tissue, muscle tissue, neural tissue, epithelial tissue, cartilage, cancerous samples, or bone.

[0440] In some cases, the subject from which the diagnostic sample is obtained can be a healthy individual, or can be an individual who is at least suspected of having a disease or health condition. In some cases, the subject can be a patient.

[0441] In certain embodiments, the CROF device includes a capture agent configured to specifically bind to a biomarker in a sample provided by a subject. In certain embodiments, the biomarker can be a protein. In certain embodiments, the biomarker protein is specifically bound by an antibody capture agent present in the CROF device. In certain embodiments, the biomarker is an antibody that specifically binds to an antigen capture agent present in the CROF device. In certain embodiments, the biomarker is a nucleic acid specifically bound by a nucleic acid capture agent that is complementary to one or both strands of a double-stranded nucleic acid biomarker, or complementary to a single-stranded biomarker. In certain embodiments, the biomarker is a nucleic acid specifically bound by a nucleic acid binding protein. In certain embodiments, the biomarker is specifically bound by an aptamer.

[0442] The presence or absence of a biomarker or a significant change in the concentration of a biomarker can be used to diagnose disease risk in an individual, the presence of a disease, or to customize the treatment of a disease in an individual. For example, the presence of a specific biomarker or biomarker panel can affect the choice of drug treatment or dosing regimen given to an individual. In evaluating potential drug treatments, biomarkers can be used as surrogates for natural endpoints such as survival or irreversible morbidity. If a treatment changes a biomarker that is directly related to improved health, the biomarker can be used as an alternative endpoint to evaluate the clinical benefit of a specific treatment or dosing regimen. Therefore, the present method promotes personalized diagnosis and treatment based on a specific biomarker or biomarker panel detected in an individual. In addition, as described above, the high sensitivity of the devices, systems and methods of the present invention facilitates early detection of biomarkers associated with a disease. Due to the ability to detect multiple biomarkers using mobile devices such as smart phones, combined with sensitivity, scalability and ease of use, the methods of the present disclosure can be used for portable and point-of-care or near-patient molecular diagnostics.

[0443] In certain embodiments, the methods of the present invention can be used to detect biomarkers of diseases or disease states. In some cases, the methods of the present invention can be used to detect biomarkers that characterize cell signaling pathways and intracellular communication for drug discovery and vaccine development. For example, the subject methods can be used to detect and / or quantify the amount of biomarkers in diseased, healthy, or benign samples. In certain embodiments, the methods of the present invention can be used to detect biomarkers of infectious diseases or disease states. In some cases, the biomarkers can be molecular biomarkers, such as, but not limited to, proteins, nucleic acids, carbohydrates, small molecules, and the like.

[0444] The present methods can be used in diagnostic assays, such as, but not limited to, the following: detection and / or quantification of biomarkers, as described above; screening assays, in which samples are tested at regular intervals on asymptomatic subjects; prognostic assays, in which the presence and / or amount of biomarkers is used to predict the likely course of disease; stratification assays, in which the response of a subject to different drug treatments can be predicted; efficacy assays, in which the efficacy of a drug treatment is monitored; and the like.

[0445] In certain embodiments, the subject biosensors can be used to diagnose pathogen infection by detecting a target nucleic acid of the pathogen in a sample. The target nucleic acid can be, for example, selected from the group consisting of human immunodeficiency viruses 1 and 2 (HIV-1 and HIV-2), human T-cell leukemia virus 2 (HTLV-1 and HTLV-2), respiratory syncytial virus (RSV), adenovirus, hepatitis B virus (HBV), hepatitis C virus (HCV), Epstein-Barr virus (EBV), human papillomavirus (HPV), varicella-zoster virus (VZV), cytomegalovirus (CMV), herpes simplex virus 1 and 2 (HSV-1 and HSV-2), human herpesvirus 8 (HHV-8, also known as Kaposi's sarcoma herpesvirus), and flaviviruses, including yellow fever virus, dengue virus, Japanese encephalitis virus, West Nile virus, and Ebola virus. However, the invention is not limited to the detection of nucleic acids, such as DNA or RNA sequences, from the abovementioned viruses, but can be applied without problems to other pathogens of importance in veterinary and / or human medicine.

[0446] Human papillomavirus (HPV) is further subdivided into over 70 different types based on the homology of their DNA sequences. These types cause different diseases. HPV types 1, 2, 3, 4, 7, 10, and 26-29 cause benign warts. HPV types 5, 8, 9, 12, 14, 15, 17, 19-25, and 46-50 cause lesions in patients with weakened immune systems. Types 6, 11, 34, 39, 41-44, and 51-55 cause benign condylar warts on the genital area and respiratory mucosa. HPV types 16 and 18 are of particular medical significance because they cause epithelial dysplasia in the genital mucosa and are associated with a high rate of invasive cancers of the cervix, vagina, vulva, and anal canal. Integration of HPV DNA is considered a decisive factor in the development of cervical cancer. Human papillomaviruses can be detected, for example, from the DNA sequences of their capsid proteins L1 and L2. Therefore, the method of the present invention is particularly suitable for detecting DNA sequences of HPV type 16 and / or HPV type 18 in tissue samples for assessing the risk of cancer development.

[0447] Other pathogens that can be detected in diagnostic samples using the devices, systems, and methods of the present invention include, but are not limited to: varicella zoster; Staphylococcus epidermidis, Escherichia coli, methicillin-resistant Staphylococcus aureus (MSRA), Staphylococcus aureus, Staphylococcus hominis, Enterococcus faecalis, Pseudomonas aeruginosa, Staphylococcus capitis, Staphylococcus waldenshii, Klebsiella pneumoniae, Haemophilus influenzae, Staphylococcus spp., Streptococcus pneumoniae, and Candida albicans; Neisseria gorrhoeae, Treponena pallidum, Clamydia tracomitis, Ureaplasm urealyticum, Haemophilus influenzae, and Streptococcus pneumoniae. ducreyi), trichomoniasis (Trichomonas vaginalis); Pseudomonas aeruginosa, methicillin-resistant Staphylococcus aureus (MSRA), Klebsiella pneumoniae, Haemophilus influenzae, Staphylococcus aureus, Stenotrophomonas maltophilia, Haemophilus parainfluenzae, Escherichia coli, Enterococcus faecalis, Serratia marcescens, Haemophilus parahaemolyticus, Enterococcus cloacae, Candida albicans, Moraxella catarrhalis, Streptococcus pneumoniae, Citrobacter freundii, Enterococcus faecium, Klebsiella oxytoca, Pseudomonas fluorescens, Neisseria meningitidis, Streptococcus pyogenes, Pneumocystis carinii, Klebsiella pneumoniae, Legionella pneumophila, Mycoplasma pneumoniae and Mycobacterium tuberculosis, etc.

[0448] In some cases, CROF devices can be used to detect biomarkers that are present at low concentrations. For example, CROF devices can be used to detect cancer antigens in readily accessible body fluids (e.g., blood, saliva, urine, tears, etc.), detect biomarkers of tissue-specific diseases (e.g., biomarkers of neurological disorders (e.g., Alzheimer's disease antigens)) in readily accessible body fluids, detect infections (particularly detecting low titers of latent viruses, such as HIV), detect fetal antigens in maternal blood, and detect exogenous compounds (e.g., drugs or pollutants) in a subject's bloodstream.

[0449] One potential source of biomarkers (e.g., "CSF"; cerebrospinal fluid) is also shown in the table. In many cases, the subject biosensors can detect biomarkers in a body fluid different from the indicated body fluid. For example, a biomarker found in CSF can be identified in urine, blood, or saliva. It will also be clear to one of ordinary skill in the art that the subject CROF device can be configured to capture and detect many more biomarkers known in the art for diagnosing diseases or health conditions.

[0450] Biomarkers can be protein or nucleic acid (e.g., mRNA) biomarkers, unless otherwise specified. Diagnosis can be associated with an increase or decrease in the level of a biomarker in a sample, unless otherwise specified. A list of biomarkers, diseases for which they can be diagnosed, and samples for which they can be detected are described in Tables 1 and 2 of U.S. Provisional Application Serial No. 62 / 234,538, filed September 29, 2015, which is incorporated herein by reference.

[0451] In some cases, the devices, systems, and methods of the present invention are used to inform the subject from whom the sample was obtained of their health condition. Health conditions that can be diagnosed or measured by the methods, devices, and systems of the present invention include, for example, chemical balance; nutritional health; exercise; fatigue; sleep; stress; prediabetes; allergies; aging; exposure to environmental toxins, pesticides, herbicides, synthetic hormone analogs; pregnancy; female menopause; and male menopause. Table 3 of U.S. Provisional Application Serial No. 62 / 234,538, filed September 29, 2015, provides a list of biomarkers that can be detected using the present CROF device (when used in conjunction with appropriate monoclonal antibodies, nucleic acids, or other capture agents) and their associated health conditions, which application is incorporated herein by reference.

[0452] J) Kit

[0453] Aspects of the present disclosure include test kits for implementing the apparatus, system and method of the present invention as described above. In certain embodiments, the test kit includes instructions for implementing the subject method using a handheld device (e.g., a mobile phone). These instructions can be present in the subject test kit in various forms, one or more of which can be present in the test kit. One form in which these instructions can exist is information printed on a suitable medium or substrate, such as one or more sheets of paper on which information is printed, in the packaging of the test kit, in a package insert, etc. Another way is a computer-readable medium for recording or storing information thereon, such as a disk, CD, DVD, Blu-ray disc, computer-readable memory, etc. Another means that can exist is a website address, which can be used via the Internet to access information at a removal site. The test kit may further include software for implementing the method for the analyte on the measuring device as described herein, which is provided on a computer-readable medium. Any convenient means can be present in the test kit.

[0454] In certain embodiments, the kit includes a detection agent containing a detectable label, such as a fluorescently labeled antibody or oligonucleotide that specifically binds to the analyte of interest, for labeling the analyte of interest. The detection agent can be provided in a separate container as the CROF device, or can be provided in the CROF device.

[0455] In certain embodiments, the kit includes a control sample comprising a known detectable amount of an analyte to be detected in the sample. The control sample can be provided in a container and can be in a solution of known concentration, or can be provided in a dry form, such as lyophilized or freeze-dried. If provided in a dry form, the kit can also include a buffer for dissolving the control sample.

[0456] Figure 9 A block diagram depicts a computer system operating in accordance with one or more aspects of the present invention. In various illustrative examples, computer system 600 may be Figure 4 System 2.

[0457] In some implementations, the computer system 600 can be connected (e.g., via a network such as a local area network (LAN), an intranet, an extranet, or the Internet) to other computer systems. The computer system 600 can operate as a server or client computer in a client-server environment, or as a peer computer in a peer-to-peer or distributed network environment. The computer system 600 can be provided by a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular phone, a web device, a server, a network router, a switch, or a bridge, or any device capable of executing a set of instructions (sequential or otherwise) specifying the actions to be taken by the device. In addition, the term "computer" should include any computer collection that executes a set (or multiple sets) of instructions to perform any one or more methods described herein, either individually or jointly.

[0458] On the other hand, the computer system 600 may include a processing device 602, a volatile memory 604 (e.g., random access memory (RAM)), a non-volatile memory 606 (e.g., read-only memory (ROM) or electrically erasable programmable ROM (EEPROM)), and a data storage device 616, which may communicate with each other via a bus 608.

[0459] The processing device 602 may be provided by one or more processors, for example, a general-purpose processor (e.g., a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a microprocessor that implements other types of instruction sets, or a microprocessor that implements a combination of various types of instruction sets), or a special-purpose processor, such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), or a network processor).

[0460] The computer system 600 may also include a network interface device 622. The computer system 600 may also include a video display unit 610 (eg, LCD), an alphanumeric input device 612 (eg, keyboard), a cursor control device 614

[0461] (eg, a mouse) and a signal generating device 620.

[0462] The data storage device 616 may include a non-transitory computer-readable storage medium 624 on which may be stored instructions 626 encoding any one or more of the methods or functions described herein, including instructions such as Figure 4 Instructions for training and applying the machine learning model 108 are shown.

[0463] The instructions 626 may also reside, completely or partially, within the volatile memory 604 and / or within the processing device 602 during execution of the instructions 626 by the computer system 600; thus, the volatile memory 604 and the processing device 602 may also constitute machine-readable storage media.

[0464] Although the computer-readable storage medium 624 is shown as a single medium in the illustrative example, the term "computer-readable storage medium" shall include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) that store one or more sets of executable instructions. The term "computer-readable storage medium" shall also include any tangible medium that can store or encode a set of instructions for execution by a computer, the instructions causing the computer to perform any one or more of the methods described herein. The term "computer-readable storage medium" shall include, but is not limited to, solid-state memories, optical media, and magnetic media.

[0465] The methods, components, and features described herein may be implemented by discrete hardware components or may be integrated into the functionality of other hardware components such as ASICs, FPGAs, DSPs, or similar devices. Furthermore, the methods, components, and features may be implemented by firmware modules or functional circuits within a hardware device. Furthermore, the methods, components, and features may be implemented in any combination of hardware devices and computer program components or in a computer program.

[0466] Figure 10 (A) shows that the monitoring mark in the present invention actually adds the known meta-structure to the grid. When the grid is distorted, people can always use the knowledge of the meta-structure to restore it to a near-perfect structure, thus greatly improving the accuracy of the machine learning model and the accuracy of the measurement.

[0467] Mapping between crystal and amorphous structures. Figure 10(B) shows a mapping between a crystalline structure implanted with a metastructure and an amorphous structure implanted with a metastructure according to an embodiment of the present disclosure.

[0468] Figure 11 shows the results of training machine learning without and with monitoring labels ( Figure 10 The difference between (a0) is that without using monitoring markers, many samples and a long time are required to train, and the test produces artifacts (missing cells and generating non-existent cells), while when using monitoring markers, the number of training samples and time are significantly reduced, and the test does not produce artifacts.

[0469] Unless otherwise specifically stated, terms such as "receiving," "associating," "determining," "updating," and the like refer to actions and processes performed or implemented by a computer system that manipulate and transform data represented as physical (electronic) quantities within computer system registers and memories into other data similarly represented as physical quantities within computer system memories or registers or other such information storage, transmission, or display devices. In addition, the terms "first," "second," "third," "fourth," and the like as used herein are intended to be labels for distinguishing between different elements and may not have an ordinal meaning based on their numerical designation.

[0470] The examples described herein also relate to apparatus for performing the methods described herein. The apparatus may be specially constructed to perform the methods described herein, or it may comprise a general-purpose computer system selectively programmed by a computer program stored in the computer system. Such a computer program may be stored in a computer-readable tangible storage medium.

[0471] The methods and illustrative examples described herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems may be used according to the teachings described herein, or it may prove convenient to construct more specialized apparatus to perform the method 300 and / or each of its various functions, routines, subroutines, or operations. Examples of the configuration of various such systems are set forth in the foregoing description.

[0472] The above description is intended to be illustrative rather than limiting. Although the present disclosure has been described with reference to specific illustrative examples and embodiments, it will be appreciated that the present disclosure is not limited to the described examples and embodiments. The scope of the present disclosure should be determined with reference to the following claims and the full scope of equivalents to which the claims are entitled.

Claims

1. A method for training a machine learning model for image-based determination, wherein the assay is imaged by a low-quality imaging system during testing, comprising: forming a thin layer of a first sample on an imaging area of ​​a first sample holder, wherein the first sample holder comprises one or more monitoring marks on the imaging area; forming a thin layer of a second sample on an imaging area of ​​a second sample holder, wherein the second sample holder comprises one or more monitoring marks on the imaging area of ​​the second sample holder that are identical to the one or more monitoring marks comprised on the first sample holder; imaging a first image of the first sample onto an imaging area of ​​the first sample holder using the low-mass imaging system; imaging a second image of the second sample onto an imaging area of ​​the second sample holder using a high quality imaging system; Correcting defects in the first image using the one or more monitoring marks to generate a first corrected image; correcting defects in the second image using the one or more monitoring marks, and generating a second corrected image if the second image has defects; as well as training a machine learning model using the first correction image, the second correction image, and the one or more monitoring marks to generate a trained model, wherein the one or more monitoring marks imaged under an ideal imaging system are one or more structures of the first and second sample holders, and geometric and optical properties of the one or more monitoring marks are predetermined and known; Among them, low-quality imaging systems contain more defects than high-quality imaging systems.

2. The method of claim 1, further comprising: forming a thin layer of a third sample on an imaging area of ​​a third sample holder, wherein the third sample holder comprises one or more monitoring marks on the imaging area of ​​the third sample holder that are identical to the one or more monitoring marks on the first sample holder; imaging a third image of the third sample on an imaging area of ​​the third sample holder using a low-mass imaging system; Correcting defects in the third image using the monitoring mark to generate a third corrected image; and The trained machine learning model is used to analyze the transformed third corrected image and generate a measurement result.

3. The method of claim 1, wherein the machine learning model comprises a Cycle Generative Adversarial Network (CycleGAN).

4. The method of claim 1 , wherein the machine learning model comprises a Cycle Generative Adversarial Network (CycleGAN), the Cycle Generative Adversarial Network (CycleGAN) comprising a forward generative adversarial network (forward GAN) and a backward GAN, wherein the forward GAN comprises a first generator and a first discriminator, and the backward GAN comprises a second generator and a second discriminator, and wherein, Training the machine learning model using each transformed region in the first image and each transformed region in the second image includes training the CycleGAN using each transformed region in the first image and each transformed region in the second image aligned at four monitoring markers at four corners of the corresponding region.

5. The method of claim 1, wherein the first sample and the second sample are the same sample, and the first sample holder and the second sample holder are the same.

6. The method of claim 1 , further comprising identifying a first region in the first image based on a location of the one or more monitoring markers in the first image; determining a spatial transformation associated with the first region based on a mapping between positions of the one or more monitoring marks in the first image and predetermined positions of the one or more monitoring marks in the first sample holder; applying the spatial transformation to the first region in the first image to calculate a transformed first region; as well as The machine learning model is trained using the transformed first image.

7. A method as claimed in claim 6, wherein the first sample holder comprises a first plate, a second plate and the one or more monitoring marks, and wherein the one or more monitoring marks comprise posts embedded at predetermined positions on at least one of the first plate or the second plate, wherein the spacing between the first plate and the second plate is 200 microns or less, and the sample is sandwiched between the first plate and the second plate.

8. The method of claim 6, further comprising: detecting positions of the one or more monitoring markers in the first image; dividing the first image into regions containing the first area, wherein each of the regions is defined by four or more monitoring marks at four corners of the corresponding region; determining a spatial transformation corresponding to each of the regions in the first image based on a mapping between positions of four monitoring marks at four corners of the corresponding region and four predetermined positions of four monitoring marks in the first sample holder; applying the corresponding spatial transformation to each region in the first image, thereby computing a corresponding transformed region in the first image; as well as The machine learning model is trained using each transformed region in the first image, wherein the trained machine learning model is used to transform the assay image from low resolution to high resolution.

9. The method of claim 8, wherein the one or more monitoring marks are periodically distributed at predetermined positions in the first image with at least one period value, and wherein detecting the positions of the one or more monitoring marks in the first image comprises: detecting positions of the one or more monitoring markers in the first image using a second machine learning model; and Errors in detected positions of the one or more monitoring marks in the first image are corrected based on the at least one period value.

10. The method of claim 8, further comprising: receiving a second image of the second sample holder captured by a second optical sensor, wherein the first image was captured at a first quality level and the second image was captured at a second quality level higher than the first quality level; dividing the second image into a plurality of regions, wherein each of the regions in the second image is defined by four monitoring marks at four corners of the corresponding region in the second image and matches a corresponding region in the first image; determining a second spatial transformation of the region in the second image based on a mapping between positions of the four monitoring marks at four corners of the corresponding region in the second image and four predetermined positions of the four monitoring marks in the sample holder; applying the second spatial transform to each of the regions in the second image to compute a corresponding transformed region in the second image; as well as The machine learning model is trained using each transformed region in the first image and each transformed region in the second image to transform the first quality level image into the second quality level image.

11. The method of claim 10, wherein the machine learning model comprises a Cycle Generative Adversarial Network (CycleGAN), the Cycle Generative Adversarial Network (CycleGAN) comprising a forward generative adversarial network (forward GAN) and a backward GAN, wherein the forward GAN comprises a first generator and a first discriminator, and the backward GAN comprises a second generator and a second discriminator, and wherein, Training the machine learning model using each transformed region in the first image and each transformed region in the second image includes: training the CycleGAN using each transformed region in the first image and each transformed region in the second image aligned at four monitoring markers at the four corners of the corresponding region.

12. The method of claim 11 , wherein training the machine learning model using each transformed region in the first image and each transformed region in the second image comprises: training the first generator and the first discriminator by providing each transformed region in the first image to the forward GAN; training the second generator and the second discriminator by providing each transformed region in the second image to the backward GAN, and The forward and backward GAN training are optimized under cycle consistency constraints.

13. The method according to claim 1, further comprising: identifying a first region in the first image based on positions of the one or more monitoring markers in the first image; determining a spatial transformation associated with the first region based on a mapping between positions of the one or more monitoring marks in the first image and predetermined positions of one or more monitoring marks in the first sample holder; applying the spatial transformation to a first region in the first image to calculate a transformed first region; as well as The machine learning model is applied to the transformed first region in the first image to produce a second region.

14. The method of claim 13, wherein the machine learning model is trained as follows: receiving a first training image of a sample holder containing a sample captured by a first optical sensor, wherein the sample holder includes one or more monitoring marks located at predetermined positions; identifying a first region in the first training image using the positions of the one or more monitoring markers in the first training image; determining a spatial transformation of the first region using a mapping between positions of the one or more monitoring marks in the first training image and predetermined positions of the one or more monitoring marks in the sample holder; applying a spatial transformation to a first region in a first training image to calculate a transformed first region; as well as The machine learning model is trained using the transformed first region of the first training image.

15. The method of claim 13, further comprising: dividing the first image into a plurality of regions based on positions of the one or more monitoring markers in the first image, wherein the plurality of regions include the first region; determining a corresponding spatial transformation for each of the plurality of regions; applying the corresponding spatial transform to each of the plurality of regions in the first image to compute a transformed region; applying the machine learning model to each of the transformed regions in the first image to produce transformed regions at a second level of quality; as well as The transformed regions are combined to form a second image.

16. The method of claim 1, wherein each of the first and second sample holders has a plate, and the sample contacts a surface of the plate.

17. The method of claim 1, wherein each of the first and second sample holders has a first plate and a second plate, wherein the one or more monitoring marks are located on at least one of the first or second plates, and the sample is sandwiched between the first and second plates.

18. The method of claim 1, wherein each of the first and second sample holders has two plates that are movable relative to each other, and the sample is located between the two plates.

19. The method of claim 1, wherein Each of the first and second sample holders has two plates movable relative to each other, wherein the sample is located between the two plates; A plurality of spacers are on at least one interior opposing surface of at least one or both of the plates, and the plurality of spacers are located between the opposing surfaces; the sample thickness is adjusted by the spacers.

20. The method of claim 19, wherein at least one spacer is present within the sample.

21. The method of claim 19, wherein the spacers are the monitoring marks and the geometrical and optionally optical properties of the spacers imaged under an ideal imaging system are predetermined and known.

22. The method of claim 19, wherein the first and second plates of the first and second sample holders are initially on top of each other and need to be separated to enter an open configuration for sample deposition.

23. The method of claim 19, wherein the first and second plates of the first and second sample holders are in a closed configuration prior to depositing the sample, and the sample enters the first and second sample holders from a gap between the two plates.

24. The method of claim 1, wherein the thin layer has a thickness of 0.1 μm, 0.5 μm, 1 μm, 2 μm, 3 μm, 5 μm, 10 μm, 50 μm, 100 μm, 200 μm, or a range between any two of the values.

25. The method of claim 1, wherein the thin layer has a thickness of 1 μm, 2 μm, 5 μm, 10 μm, 30 μm, 50 μm, 100 μm, 200 μm, or a range between any two of the values.

26. The method of claim 1, wherein the monitoring mark has a columnar shape, and the spacing between two monitoring marks is 1 μm, 2 μm, 3 μm, 5 μm, 10 μm, 50 μm, 100 μm, 200 μm, 500 μm, or within a range between any two of the values.

27. The method according to claim 1, wherein The monitoring mark has a columnar shape and has a substantially flat top surface, the top surface covering at least 10% of a top projection area of ​​the monitoring mark.

28. The method according to claim 1, wherein The monitoring marks have a columnar shape and are periodically arranged.

29. The method according to claim 13, wherein The sample holder includes a first plate, a second plate, and one or more monitoring marks, wherein the one or more monitoring marks include a column located at a predetermined position on at least one of the first plate and the second plate, wherein the spacing between the first plate and the second plate is 200 μm or less, and the sample holder is between the first plate and the second plate.

30. The method of claim 13, wherein: The monitoring marks have a columnar shape and are periodically arranged.

31. The method of claim 1, wherein the monitoring marker has a pillar-like shape and comprises at least four pillars.

32. The method of claim 6, wherein the monitoring marker has a columnar shape and comprises at least four columns.

33. The method of claim 13, wherein the monitoring marker has a pillar-like shape and comprises at least four pillars.

34. An image-based measurement system comprising: a database system for storing images; and A processing device, communicatively coupled to the database system, for: receiving a first image of a sample holder containing a sample captured by a first optical sensor, wherein the sample holder includes a monitoring mark at a predetermined location; identifying a first region in the first image based on positions of one or more monitoring markers in the first image; determining a spatial transformation associated with the first region based on a mapping between positions of the one or more monitoring marks in the first image and predetermined positions of one or more monitoring marks in the sample holder; applying the spatial transformation to a first region in the first image to calculate a transformed first region; training a machine learning model using the transformed first image; detecting positions of the one or more monitoring markers in the first image; dividing the first image into regions containing the first area, wherein each of the regions is defined by four monitoring marks at four corners of the corresponding region; determining a corresponding spatial transformation for each of the regions in the first image based on a mapping between the positions of four monitoring marks at four corners of the corresponding region and four predetermined positions of four monitoring marks in the sample holder; applying the corresponding spatial transformation to each region in the first image, thereby computing a corresponding transformed region in the first image; as well as The machine learning model is trained using each transformed region in the first image, wherein the trained machine learning model is used to transform the assay image from low resolution to high resolution.

35. The system of claim 34, wherein: The sample holder includes a first plate and a second plate, wherein the monitoring mark includes a post embedded at a predetermined position on at least one of the first plate and the second plate, and the sample is sandwiched between the first plate and the second plate.

36. The system of claim 34, wherein the predetermined positions of the one or more monitoring marks are periodically distributed with at least one period value, and wherein to detect the positions of the one or more monitoring marks in the first image, the processing device is further configured to: detecting a position of the monitoring marker in the first image using a second machine learning model; and Errors in detected positions of the one or more monitoring marks in the first image are corrected based on the at least one period value.

37. The system of claim 34, wherein the processing device is further configured to: receiving a second image of the sample holder captured by a second optical sensor, wherein the first image was captured at a first quality level and the second image was captured at a second quality level higher than the first quality level; dividing the second image into a plurality of regions, wherein each of the regions in the second image is defined by four monitoring marks at four corners of the corresponding region in the second image and matches a corresponding region in the first image; determining a second spatial transformation associated with the region in the second image based on a mapping between positions of the four monitoring marks at four corners of the corresponding region in the second image and four predetermined positions of the four monitoring marks in the sample holder; applying the second spatial transform to each of the regions in the second image to compute a corresponding transformed region in the second image; as well as The machine learning model is trained using each transformed region in the first image and each transformed region in the second image to transform the first quality level image into the second quality level image.

38. The system of claim 37, wherein the machine learning model comprises a Cycle Generative Adversarial Network (CycleGAN), the Cycle Generative Adversarial Network (CycleGAN) comprising a forward generative adversarial network (forward GAN) and a backward GAN, wherein the forward GAN comprises a first generator and a first discriminator, and the backward GAN comprises a second generator and a second discriminator, and wherein, Using each transformed region in the first image and each transformed region in the second image to train the machine learning model includes: using each transformed region in the first image and each transformed region in the second image aligned at four monitoring markers at four corners of the corresponding region to train the CycleGAN.

39. The system of claim 38, wherein to train the machine learning model using each transformed region in the first image and each transformed region in the second image, the processing device is further configured to: training the first generator and the first discriminator by providing each transformed region in the first image to the forward GAN; The second generator and the second discriminator are trained by providing each transformed region in the second image to the backward GAN.

40. The system of claim 34, wherein the monitoring marker has a pillar-like shape and comprises at least four pillars.

41. An image-based assay system for converting an assay image using a machine learning model, comprising: a database system for storing images; and A processing device, communicatively coupled to the database system, for: receiving a first image of a sample holder containing a sample captured by a first optical sensor, wherein the sample holder is manufactured with standardized monitoring marks at predetermined locations; identifying a first region in the first image based on positions of one or more monitoring markers in the first image; determining a spatial transformation associated with the first region based on a mapping between positions of the one or more monitoring marks in the first image and predetermined positions of one or more monitoring marks in the sample holder; applying the spatial transformation to a first region in the first image to calculate a transformed first region; as well as applying the machine learning model to the transformed first region in the first image to produce a second region; detecting positions of one or more monitoring markers in the first image; dividing the first image into regions including the first area, wherein each region is defined by four monitoring marks at four corners of the corresponding region; determining a corresponding spatial transformation for each region in the first image using a mapping between the positions of the four monitoring marks at the four corners of the corresponding region and the four predetermined positions of the four monitoring marks in the sample holder; applying the corresponding spatial transformation to each region in the first image to calculate a corresponding transformed region in the first image; and A machine learning model is trained using each transformed region in the first image, wherein the trained machine learning model is used to convert the analysis image from low resolution to high resolution.

42. The system of claim 41 , wherein the machine learning model is trained by: receiving a first training image of a sample holder containing a sample captured by a first optical sensor, wherein the sample holder includes a monitoring mark located at a predetermined position; identifying a first region in the first image using the positions of the one or more monitoring markers in the first training image; determining a spatial transformation of the first region using a mapping between positions of the one or more monitoring marks in the first training image and predetermined positions of the one or more monitoring marks in the sample holder; applying a spatial transformation to a first region in a first training image to calculate a transformed first region; as well as The machine learning model is trained using the transformed first region of the first training image.

43. The system of claim 41, wherein the processing device is further configured to: dividing the first image into a plurality of regions based on positions of the one or more monitoring marks among the monitoring marks in the first image, wherein the plurality of regions include the first region; determining a corresponding spatial transformation for each of the plurality of regions; applying the corresponding spatial transform to each of the plurality of regions in the first image to compute a transformed region; applying the machine learning model to each of the transformed regions in the first image to produce transformed regions at a second level of quality; as well as The transformed regions are combined to form a second image.

44. The system of claim 41, wherein the monitoring marks have a columnar shape, and a spacing between two monitoring marks is 10 μm, 50 μm, 100 μm, 200 μm, 500 μm, or a range between any two of said values.

45. The system of claim 41, wherein the monitoring marker has a columnar shape and comprises at least four columns.

46. ​​The system of claim 41, wherein: The monitoring mark has a columnar shape and has a substantially flat top surface, the top surface covering at least 10% of a top projection area of ​​the monitoring mark.

47. The system of claim 41, wherein the monitoring marks have a columnar shape and are arranged periodically.

48. The system of claim 41, wherein the sample holder comprises a first plate and a second plate, wherein the monitoring mark comprises a post located at a predetermined position on at least one of the first plate and the second plate, and the sample is sandwiched between the first and second plates.

49. The system of claim 41 , wherein the one or more monitoring markers are periodically distributed with at least one period value, and wherein upon detecting the position of the one or more monitoring markers in the first image, the processing device further: detecting the locations of one or more surveillance markers in the first image using a second machine learning model; and Errors in the detected positions of the one or more monitoring marks in the first image are corrected using the at least one period value.

Images