Systems and methods for detection of sample detachment

Abstract

A method for determining sample detachment from a substrate is provided. A plurality of images of a sample on a substrate are received where the plurality of images includes a plurality of fields of view of the sample. At least one focus score is determined for each image in the plurality of images. Based on the determined focus scores, one or more portions of the sample are determined as being detached from the substrate.

Description

Attorney Docket No. GEI-05025 SYSTEMS AND METHODS FOR DETECTION OF SAMPLE DETACHMENT Cross-reference to related applications

[0001] This application claims the benefit of U.S. Provisional App. No. 63 / 726,987, filed on December 2, 2024, and U.S. Provisional App. No. 63 / 664484, filed on June 26, 2024. The entire contents of each are hereby incorporated by reference in their entirety. Field of the Invention

[0002] The present disclosure is directed to imaging techniques for samples, e.g., biological samples. More specifically, the present disclosure describes identifying regions of samples that are detached from a substrate based on stains, fluorescence, and / or inherent contrast properties of the sample. Background

[0003] For automated, high-throughput tissue imaging applications, automatically identifying regions of a sample where the sample has detached from a substrate can be challenging as many biological samples are substantially translucent or transparent. Sample detachment causes many issues with downstream in situ analysis processes, such as in situ decoding, because the detached regions may experience more drift during multiple cycles of probing and imaging, have problems during image alignment and registration, have low-quality scores, among other issues. Processing data obtained from regions of a sample that are detached can have negative consequences to throughput – specifically, due to the increased computational resources required to store the volumetric images and any additional processing of those images.

[0004] Accordingly, there exists a need for a fast and accurate method for detecting sample detachment from a substrate for use in automated and high-throughput imaging systems. Summary

[0005] In various embodiments, a method for determining sample detachment from a substrate is provided. A plurality of images of a sample on a substrate are received where the plurality of images includes a plurality of fields of view of the sample. At least one focus score is determined for each image in the plurality of images. Based on the determined focusAttorney Docket No. GEI-05025 scores, one or more portions of the sample are determined as being detached from the substrate.

[0006] In various embodiments, a computer program product is provided for determining sample detachment from a substrate. The computer program product includes a computer readable storage medium having program instructions embodied therewith, and the program instructions are executable by a processor to cause the processor to perform a method where a plurality of images of a sample on a substrate are received. The plurality of images includes a plurality of fields of view of the sample. At least one focus score is determined for each image in the plurality of images. Based on the determined focus scores, one or more portions of the sample are determined as being detached from the substrate.

[0007] In various embodiments, a system is provided for determining sample detachment from a substrate. The system includes an image database and a computing node including a computer readable storage medium having program instructions embodied therewith. The program instructions are executable by a processor to cause the processor to perform a method where a plurality of images of a sample on a substrate are received where the plurality of images includes a plurality of fields of view of the sample. At least one focus score is determined for each image in the plurality of images. Based on the determined focus scores, one or more portions of the sample are determined as being detached from the substrate.

[0008] In various embodiments, a method is provided for determining sample detachment from a substrate. In performing the method, a z-stack of images of a biological sample on a substrate is received. The z-stack of images corresponds to a field of view. A focus map is determined based on the z-stack of images indicating, for each of a plurality of patches of the field of view, one of the images of the z-stack bringing into focus that patch of the field of view. Based on the focus map, one or more portions of the sample are determined as being detached from the substrate.

[0009] In various embodiments, a computer program product is provided for determining sample detachment from a substrate. The computer program product includes a computer readable storage medium having program instructions embodied therewith, and the program instructions are executable by a processor to cause the processor to perform a method where a z-stack of images of a biological sample on a substrate is received. The z-stack of images corresponds to a field of view. A focus map is determined based on the z-stack of images indicating, for each of a plurality of patches of the field of view, one of the images of the z-Attorney Docket No. GEI-05025 stack bringing into focus that patch of the field of view. Based on the focus map, one or more portions of the sample are determined as being detached from the substrate.

[0010] In various embodiments, a system is provided for determining sample detachment from a substrate. The system includes an image database and a computing node including a computer readable storage medium having program instructions embodied therewith. The program instructions are executable by a processor to cause the processor to perform a method where a z-stack of images of a biological sample on a substrate is received. The z- stack of images corresponds to a field of view. A focus map is determined based on the z- stack of images indicating, for each of a plurality of patches of the field of view, one of the images of the z-stack bringing into focus that patch of the field of view. Based on the focus map, one or more portions of the sample are determined as being detached from the substrate. Brief Description of the Drawings

[0011] FIG. 1 is an example workflow of analysis of a biological sample (e.g., a cell or tissue sample) using an opto-fluidic instrument, according to various embodiments.

[0012] FIGS. 2A-2B illustrate cross-sectional views of an optics module in an imaging system, according to some embodiments.

[0013] FIG. 3A illustrates a dark field image of a tissue sample, according to some embodiments. FIG. 3B illustrates estimated local thicknesses at a plurality of FOVs, according to some embodiments. FIG. 3C illustrates an overlay of the local thicknesses on top of the dark field image of the tissue sample, according to some embodiments. FIG. 3D illustrates a histogram of the local tissue thicknesses, according to some embodiments. FIG. 3E illustrates an image of a tissue sample, according to some embodiments. FIG. 3F illustrates a window of view of the dark field image of the tissue sample in FIG. 3E, according to some embodiments. FIG. 3G illustrates a Tenengrad map of the dark field image of the tissue sample in FIG. 3E, according to some embodiments. FIG. 3H illustrates an overlay of the Tenengrad map on top of the dark field image of the tissue sample in FIG. 3E, according to some embodiments. FIG. 3I illustrates an overlay of the Tenengrad map generated from a window of view of the dark field image on top of the same window of view of the dark field image of the tissue sample in FIG. 3E, according to some embodiments.

[0014] FIG. 4 illustrates a graph of focus scores obtained from a plurality of slices in a single z-stack of images, according to some embodiments.

[0015] FIGS. 5A-5H illustrate graphs of focus scores determined using various methods, according to some embodiments.Attorney Docket No. GEI-05025

[0016] FIG. 6A illustrates estimates of local heights at a plurality of FOVs at the top of the tissue sample, according to some embodiments. FIG. 6B illustrates estimates of local heights at a plurality of FOVs at the bottom of the tissue sample, according to some embodiments. FIG. 6C illustrates estimates of thicknesses of the top and bottom surfaces along a YZ plane of the tissue sample, according to some embodiments. FIG. 6D illustrates estimates of thicknesses of the top and bottom surfaces along a XZ plane of the tissue sample, according to some embodiments.

[0017] FIG. 7 illustrates a flow chart of a method for tissue bounds detection based on inherent contrast, according to some embodiments.

[0018] FIG. 8 illustrates a flow chart of a method for determining axial bounds of a tissue sample based on inherent contrast, according to some embodiments.

[0019] FIG. 9 illustrates a z-slice image exhibiting distortion at the edges of the FOV due to field curvature, according to some embodiments.

[0020] FIG. 10 illustrates various cycles of an optofluidic instrument used to generate a final stain image using multi-focus image fusion, according to some embodiments.

[0021] FIG. 11 depicts a visualization of a z-slice of a multi-channel z-stack of images of an unstained biological sample, according to various embodiments.

[0022] FIG. 12 depicts a visualization of a z-slice of a multi-channel z-stack of images of a stained biological sample, according to various embodiments.

[0023] FIGS. 13A-13H depict aspects of steps of a multi-focus image fusion algorithm, according to various embodiments.

[0024] FIG. 14 depicts a technique for determining a subtracted image by subtracting a convex combination of fused unstained images from the stained fused image, according to various embodiments.

[0025] FIG. 15 depicts a visualization of a finalized subtracted focused image, according to various embodiments.

[0026] FIG. 16 is a flowchart illustrating a method of image fusion, according to embodiments of the present disclosure.

[0027] FIG. 17 is a flowchart illustrating a method of image fusion, according to embodiments of the present disclosure.

[0028] FIG. 18A depicts a cross-sectional image of a mouse ileum with at least one region exhibiting tissue detachment from a substrate and at least one region exhibiting tissue folding, according to embodiments of the present disclosure. FIGS. 18B-18E depict visualizations ofAttorney Docket No. GEI-05025 the specific detached and folded regions with detected transcripts overlaid confirming detachment or folding, according to embodiments of the present disclosure.

[0029] FIGS. 19A-19B depict plots of focus score across the entire image of FIG. 18A.

[0030] FIGS. 20A-20D depict a process of removing empty regions from the analysis of focus scores, according to embodiments of the present disclosure.

[0031] FIGS. 21A-21B depict the difference between plots of focus score including the empty regions (FIG. 21A) and excluding the empty regions (FIG. 21B), according to embodiments of the present disclosure.

[0032] FIGS. 22A-22B depict the effects of removing empty regions on the analysis of focus scores in the folded region of the image of FIG. 18A.

[0033] FIGS. 23A-23B depict the effects of removing empty regions on the analysis of focus scores in the detached region of the image of FIG. 18A.

[0034] FIG. 24 depicts histogram plots of patches in-focus z-index per FOV, according to embodiments of the present disclosure.

[0035] FIG. 25 depicts a panel provided on an instrument for presenting a visualization of tissue detachment to a user, according to embodiments of the present disclosure.

[0036] FIGS. 26A-26B depict visualizations of a tissue detachment, according to embodiments of the present disclosure.

[0037] FIG. 27 depicts a computing node according to some embodiments disclosed herein.

[0038] In the figures, elements and steps having the same or similar reference numeral have the same or similar attributes or description, unless explicitly stated otherwise. Detailed Description

[0039] As explained above, current technologies are insufficient for detecting detachment of samples from substrates in automated and high-throughput volumetric imaging systems. Detachment of samples from substrates causes issues in downstream in situ analysis (e.g., during in situ decoding), such as decoding failures due to low quality data. Accordingly, there is a need for a system and method to detect detachment of a sample so that the detached regions can be excluded from in situ analysis, thereby reducing computational burden and improving throughput. Moreover, experiment success rates can be improved by excluding low-quality, detached regions of a sample from in situ analysis.

[0040] The present disclosure resolves the above technical problems by providing systems, methods, and computer program products to determine regions of detachment of a sample (e.g., a tissue sample, hydrogel, etc.) using stains, fluorescence, and / or inherent contrastAttorney Docket No. GEI-05025 properties of the sample (e.g., using dark field imaging). In general, the systems and methods described herein use any suitable method to generate contrast of a sample against a background (e.g., illumination of a sample via bright field imaging, illumination of a sample via fluorescence imaging to illuminate fluorescent probes, inducing autofluorescence within the sample, adding contrast to the sample with one or more stains, etc.). In various embodiments, the one or more stains are imaged via bright field imaging or fluorescence imaging (e.g., epi-illumination fluorescence, transillumination fluorescence), etc. For example, a hematoxylin and eosin (H&E) stain can be imaged

[0041] Target molecules (e.g., nucleic acids, proteins, antibodies, etc.) can be detected in biological samples (e.g., one or more cells or a tissue sample) using an instrument having integrated optics and fluidics modules (an “opto-fluidic instrument” or “opto-fluidic system”). In an opto-fluidic instrument, the fluidics module is configured to deliver one or more reagents (e.g., fluorescent probes) to the biological sample and / or remove spent reagents therefrom. Additionally, the optics module is configured to illuminate the biological sample with light having one or more spectral emission curves (over a range of wavelengths) and subsequently capture one or more images of emitted light signals from the biological sample during one or more probing cycles. In various embodiments, the captured images may be processed in real time and / or at a later time to determine the presence of the one or more target molecules in the biological sample, as well as three-dimensional position information associated with each detected target molecule. Additionally, the opto-fluidics instrument includes a sample module configured to receive (and, optionally, secure) one or more biological samples. In some instances, the sample module includes an X-Y stage configured to move the biological sample along an X-Y plane (e.g., perpendicular to an objective lens of the optics module).

[0042] In various embodiments, the opto-fluidic instrument is configured to analyze one or more target molecules in their naturally occurring place (i.e., in situ) within the biological sample. For example, an opto-fluidic instrument may be an in-situ analysis system used to analyze a biological sample and detect target molecules including but not limited to DNA, RNA, proteins, antibodies, and / or the like.

[0043] A sample disclosed herein can be or be derived from any biological sample. Biological samples may be obtained from any suitable source using any of a variety of techniques including, but not limited to, biopsy, surgery, and laser capture microscopy (LCM), and generally includes cells, tissues, and / or other biological material from the subject. A biological sample can be obtained from a prokaryote such as a bacterium, anAttorney Docket No. GEI-05025 archaea, a virus, or a viroid. A biological sample can also be obtained from non-mammalian organisms (e.g., a plant, an insect, an arachnid, a nematode, a fungus, or an amphibian). A biological sample can also be obtained from a eukaryote, such as a tissue sample from a mammal. A biological sample from an organism may comprise one or more other organisms or components therefrom. For example, a mammalian tissue section may comprise a prion, a viroid, a virus, a bacterium, a fungus, or components from other organisms, in addition to mammalian cells and non-cellular tissue components. Subjects from which biological samples can be obtained can be healthy or asymptomatic subjects, subjects that have or are suspected of having a disease (e.g., an individual with a disease such as cancer) or a pre- disposition to a disease, and / or subjects in need of therapy or suspected of needing therapy.

[0044] The biological sample can include any number of macromolecules, for example, cellular macromolecules and organelles (e.g., mitochondria and nuclei). The biological sample can be obtained as a tissue sample, such as a tissue section, biopsy, a core biopsy, needle aspirate, or fine needle aspirate. The sample can be a fluid sample, such as a blood sample, urine sample, or saliva sample. The sample can be a skin sample, a colon sample, a cheek swab, a histology sample, a histopathology sample, a plasma or serum sample, a tumor sample, living cells, cultured cells, a clinical sample such as, for example, whole blood or blood-derived products, blood cells, or cultured tissues or cells, including cell suspensions.

[0045] In some embodiments, the biological sample may comprise cells or a tissue sample which are deposited on a substrate. As described herein, a substrate can be any support that is insoluble in aqueous liquid and allows for positioning of biological samples, analytes, features, and / or reagents on the support. In some embodiments, a biological sample is attached to a substrate. In some embodiments, the substrate is optically transparent to facilitate analysis on the opto-fluidic instruments disclosed herein. For example, in some instances, the substrate is a glass substrate (e.g., a microscopy slide, cover slip, or other glass substrate). Attachment of the biological sample can be irreversible or reversible, depending upon the nature of the sample and subsequent steps in the analytical method. In certain embodiments, the sample can be attached to the substrate reversibly by applying a suitable polymer coating to the substrate and contacting the sample to the polymer coating. The sample can then be detached from the substrate, e.g., using an organic solvent that at least partially dissolves the polymer coating. Hydrogels are examples of polymers that are suitable for this purpose. In some embodiments, the substrate can be coated or functionalized with one or more substances to facilitate attachment of the sample to the substrate. SuitableAttorney Docket No. GEI-05025 substances that can be used to coat or functionalize the substrate include, but are not limited to, lectins, poly-lysine, antibodies, and polysaccharides.

[0046] It is to be noted that, although the above discussion relates to an opto-fluidic instrument that can be used for in situ target molecule detection via probe hybridization, the discussion herein equally applies to any opto-fluidic instrument that employs any imaging or target molecule detection technique. That is, for example, an opto-fluidic instrument may include a fluidics module that includes fluids needed for establishing the experimental conditions required for the probing of target molecules in the sample. Further, such an opto- fluidic instrument may also include a sample module configured to receive the sample, and an optics module including an imaging system for illuminating (e.g., exciting one or more fluorescent probes within the sample) and / or imaging light signals received from the probed sample. The in-situ analysis system may also include other ancillary modules configured to facilitate the operation of the opto-fluidic instrument, such as, but not limited to, cooling systems, motion calibration systems, etc.

[0047] FIG. 1 shows an example workflow of analysis of a biological sample 110 (e.g., cell or tissue sample) using an opto-fluidic instrument 120, according to various embodiments. In various embodiments, the sample 110 can be a biological sample (e.g., a tissue) that includes molecules such as DNA, RNA, proteins, antibodies, etc. For example, the sample 110 can be a sectioned tissue that is treated to access the RNA thereof for labeling with circularizable DNA probes. Ligation of the probes may generate a circular DNA probe which can be enzymatically amplified and bound with fluorescent oligonucleotides, which can create bright signal that is convenient to image and has a high signal-to-noise ratio.

[0048] In various embodiments, the sample 110 may be placed in the opto-fluidic instrument 120 for analysis and detection of the molecules in the sample 110. In various embodiments, the opto-fluidic instrument 120 can be a system configured to facilitate the experimental conditions conducive for the detection of the target molecules. For example, the opto-fluidic instrument 120 can include a fluidics module 140, an optics module 150, a sample module 160, and an ancillary module 170, and these modules may be operated by a system controller 130 to create the experimental conditions for the probing of the molecules in the sample 110 by selected probes (e.g., circularizable DNA probes), as well as to facilitate the imaging of the probed sample (e.g., by an imaging system of the optics module 150). In various embodiments, the various modules of the opto-fluidic instrument 120 may be separate components in communication with each other, or at least some of them may be integrated together.Attorney Docket No. GEI-05025

[0049] In various embodiments, the sample module 160 may be configured to receive the sample 110 into the opto-fluidic instrument 120. For instance, the sample module 160 may include a sample interface module (SIM) that is configured to receive a sample device (e.g., cassette) onto which the sample 110 can be deposited. That is, the sample 110 may be placed in the opto-fluidic instrument 120 by depositing the sample 110 (e.g., the sectioned tissue) on a sample device that is then inserted into the SIM of the sample module 160. In some instances, the sample module 160 may also include an X-Y stage onto which the SIM is mounted. The X-Y stage may be configured to move the SIM mounted thereon (e.g., and as such the sample device containing the sample 110 inserted therein) in perpendicular directions along the two-dimensional (2D) plane of the opto-fluidic instrument 120.

[0050] The experimental conditions that are conducive for the detection of the molecules in the sample 110 may depend on the target molecule detection technique that is employed by the opto-fluidic instrument 120. For example, in various embodiments, the opto-fluidic instrument 120 can be a system that is configured to detect molecules in the sample 110 via hybridization of probes. In such cases, the experimental conditions can include molecule hybridization conditions that result in the intensity of hybridization of the target molecule (e.g., nucleic acid) to a probe (e.g., oligonucleotide) being significantly higher when the probe sequence is complementary to the target molecule than when there is a single-base mismatch. The hybridization conditions include the preparation of the sample 110 using reagents such as washing / stripping reagents, hybridizing reagents, etc., and such reagents may be provided by the fluidics module 140.

[0051] In various embodiments, the fluidics module 140 may include one or more components that may be used for storing the reagents, as well as for transporting said reagents to and from the sample device containing the sample 110. For example, the fluidics module 140 may include reservoirs configured to store the reagents, as well as a waste container configured for collecting the reagents (e.g., and other waste) after use by the opto- fluidic instrument 120 to analyze and detect the molecules of the sample 110. Further, the fluidics module 140 may also include pumps, tubes, pipettes, etc., that are configured to facilitate the transport of the reagent to the sample device (e.g., and as such the sample 110). For instance, the fluidics module 140 may include pumps (“reagent pumps”) that are configured to pump washing / stripping reagents to the sample device for use in washing / stripping the sample 110 (e.g., as well as other washing functions such as washing an objective lens of the imaging system of the optics module 150).Attorney Docket No. GEI-05025

[0052] In various embodiments, the ancillary module 170 can be a cooling system of the opto-fluidic instrument 120, and the cooling system may include a network of coolant- carrying tubes that are configured to transport coolants to various modules of the opto-fluidic instrument 120 for regulating the temperatures thereof. In such cases, the fluidics module 140 may include coolant reservoirs for storing the coolants and pumps (e.g., “coolant pumps”) for generating a pressure differential, thereby forcing the coolants to flow from the reservoirs to the various modules of the opto-fluidic instrument 120 via the coolant-carrying tubes. In some instances, the fluidics module 140 may include returning coolant reservoirs that may be configured to receive and store returning coolants, i.e., heated coolants flowing back into the returning coolant reservoirs after absorbing heat discharged by the various modules of the opto-fluidic instrument 120. In such cases, the fluidics module 140 may also include cooling fans that are configured to force air (e.g., cool and / or ambient air) into the returning coolant reservoirs to cool the heated coolants stored therein. In some instance, the fluidics module 140 may also include cooling fans that are configured to force air directly into a component of the opto-fluidic instrument 120 so as to cool said component. For example, the fluidics module 140 may include cooling fans that are configured to direct cool or ambient air into the system controller 130 to cool the same.

[0053] As discussed above, the opto-fluidic instrument 120 may include an optics module 150 which include the various optical components of the opto-fluidic instrument 120, such as but not limited to a camera, an illumination module (e.g., light source such as LEDs), an objective lens, and / or the like. The optics module 150 may include a fluorescence imaging system that is configured to image the fluorescence emitted by the probes (e.g., oligonucleotides) in the sample 110 after the probes are excited by light from the illumination module of the optics module 150.

[0054] In some instances, the optics module 150 may also include an optical frame onto which the camera, the illumination module, and / or the X-Y stage of the sample module 160 may be mounted.

[0055] In various embodiments, the system controller 130 may be configured to control the operations of the opto-fluidic instrument 120 (e.g., and the operations of one or more modules thereof). In some instances, the system controller 130 may take various forms, including a processor, a single computer (or computer system), or multiple computers in communication with each other. In various embodiments, the system controller 130 may be communicatively coupled with data storage, set of input devices, display system, or a combination thereof. In some cases, some or all of these components may be considered to beAttorney Docket No. GEI-05025 part of or otherwise integrated with the system controller 130, may be separate components in communication with each other, or may be integrated together. In other examples, the system controller 130 can be, or may be in communication with, a cloud computing platform.

[0056] In various embodiments, the opto-fluidic instrument 120 may analyze the sample 110 and may generate the output 190 that includes indications of the presence of the target molecules in the sample 110. For instance, with respect to the example embodiment discussed above where the opto-fluidic instrument 120 employs a hybridization technique for detecting molecules, the opto-fluidic instrument 120 may cause the sample 110 to undergo successive rounds of fluorescent probe hybridization (using two or more sets of fluorescent probes, where each set of fluorescent probes is excited by a different color channel) and be imaged to detect target molecules in the probed sample 110. In such cases, the output 190 may include optical signatures (e.g., a codeword) specific to each gene, which allow the identification of the target molecules.

[0057] FIG. 2A illustrates a cross-sectional view of an optics module 200 in an imaging system. One or more illumination sources 210, e.g., one or more light emitting diodes (LEDs), provides light through one or more optical components and an objective lens 220 to thereby illuminate a sample 250. In various embodiments, the optical components include a collimator 211. In various embodiments, the optical components include a field stop 212. In various embodiments, the optical components include one or more excitation filters 213. In various embodiments, the one or more excitation filters 213 are configured to filter light from the illumination source(s) 210 for a predetermined range of wavelengths (e.g., each filter has one or more blocking band(s) and / or transmission band(s) that may be different or may overlap at least in part) and each excitation filter 213 is aligned with appropriate illumination sources (e.g., blue LEDs, green LEDs, yellow LEDs, red LEDs, ultraviolet LEDs, etc.). In various embodiments, the optical components include a condenser 214. In various embodiments, the optical components include a beam splitter 215. An optical axis 251 is illustrated extending through the center of the optical surfaces in the objective lens 220 and its path includes an image plane, a focal plane, and input / output pupils (illustrated in FIG. 2B).

[0058] A sensor array 260 (e.g., CMOS sensor) receives light signals from the sample 250. In various embodiments, the optical components include one or more emission filters 265. In various embodiments, the one or more emission filters 265 are configured to filter light from the sample (e.g., emitted from one or more fluorophores, autofluorescence, etc.) for a predetermined range of wavelengths (e.g., each filter has one or more blocking band(s) and / orAttorney Docket No. GEI-05025 transmission band(s) that may be different or may overlap at least in part). In various embodiments, the emission filters 265 align (e.g., via motorized translation) with optics and / or the sensor array. In various embodiments, the sample 250 is probed with fluorescent probes configured to bind to a target (e.g., DNA or RNA) that, when illuminated with a particular wavelength (or range of wavelengths) of light, emit light signals that can be detected by the sensor array 260. In various embodiments, the sample 250 is repeatedly probed with two or more (e.g., two, three, four, five, six, etc.) different sets of probes. In various embodiments, each set of probes corresponds to a specific color (e.g., blue, green, yellow, or red) such that, when illuminated by that color, probes bound to a target emit light signals. In some embodiments, the sensor array 260 is aligned with the optical axis 251 of the objective lens 220 (i.e., the optical axis of the camera is coincident with and parallel to the optical axis of the objective lens 220). In various embodiments, the sensor array 260 is positioned perpendicularly to the objective lens 220 (i.e., the optical axis of the camera is perpendicular to and intersects the optical axis of the objective lens 220). In various embodiments, a tube lens 261 is mounted in the optical path to focus light on the sensor array 260 thereby allowing for image formation with infinity-corrected objectives. Descriptions of optical modules and illumination assemblies for use in opto-fluidic instruments can be found in U.S. provisional patent application no. 63 / 427,282, filed on November 22, 2022, titled “Systems and Methods for Illuminating a Sample” and U.S. provisional patent application no. 63 / 427,360, file on November 22, 2022, titled “Systems and Methods for Imaging Samples,” each of which is incorporated by reference in its entirety.

[0059] In various embodiments, the sample is illuminated with one or more wavelengths configured to induce fluorescence in the sample. In various embodiments, the sample is probed during one or more probing cycles with one or more fluorescent probes configured to bind to one or more target analytes. In various embodiments, the one or more wavelengths are selected to induce fluorescence in a subset of the one or more fluorescent probes. In various embodiments, each probing cycle includes illumination with two or more (e.g., four) colors of light. In various embodiments, the sample is treated with a fluorescent stain configured to illuminate one or more structures within the sample. In various embodiments, the sample is contacted with a nuclear stain. In various embodiments, the sample is contacted with 4′,6-diamidino-2-phenylindole (“DAPI”) configured to bind to adenine–thymine-rich regions in DNA. In various embodiments, illumination of the sample causes autofluorescence of the sample. In various embodiments, autofluorescence is the natural emission of light by biological structures when they have absorbed light, and may be used toAttorney Docket No. GEI-05025 distinguish the light originating from artificially added fluorescent markers. In various embodiments, fluorescence of the sample through fluorescent probes, autofluorescence, and / or a fluorescent stain can be used with the methods described herein to determine one or more focus metrics of a tissue sample.

[0060] In various embodiments, the sample is illuminated via edge lighting or transillumination along one or more edges of the sample and / or sample substrate. In various embodiments, the edge lighting provides dark-field illumination of the sample. In various embodiments, edge lighting is provided by one or more light sources positioned to provide light substantially perpendicular to a normal of the substrate surface on which the sample is disposed. In various embodiments, the substrate is a glass slide. In various embodiments, the substrate is configured as a wave guide to thereby guide light emitted from the edge lighting towards the sample. In various embodiments, illumination of the sample via edge lighting can be used with the methods described herein to determine one or more focus metrics of a tissue sample.

[0061] FIG. 3A illustrates a dark field image 300A of a tissue sample 302. In various embodiments, the tissue sample 302 is imaged by an optics module as described above. In various embodiments, the optics module images the tissue sample 302 as a plurality of fields- of-view (FOV). In various embodiments, each FOV is imaged as a z-stack having a plurality of z-slices (e.g., two or more z-slices). In various embodiments, each z-stack of images represents (e.g., approximates) a volume of the respective FOV being imaged. In various embodiments, the images of each FOV are stitched into a combined image of the tissue sample 302. In various embodiments, each FOV of the tissue sample 302 is imaged independently of other FOVs (e.g., the FOVs may not be stitched together into a single image). In various embodiments, the plurality of FOVs have a grid-like pattern 304, where each region (e.g., approximately square region) formed by the grid-like pattern 304 is imaged by the objective independently of the other areas. In various embodiments, the optics module images a predetermined volume. In various embodiments, the predetermined volume is defined by a maximum width, a maximum length, and / or a maximum thickness of the sample. In various embodiments, a buffer factor may be applied to any of the dimensions (e.g., max length, max width, and / or max thickness) to thereby add a predetermined buffer to the known dimension(s) and ensure that the tissue sample volume is fully imaged.

[0062] In various embodiments, the buffer factor is over 100%. In various embodiments, the buffer factor is about 100%. In various embodiments, the buffer factor is about 95%. In various embodiments, the buffer factor is about 90%. In various embodiments, the bufferAttorney Docket No. GEI-05025 factor is about 85%. In various embodiments, the buffer factor is about 80%. In various embodiments, the buffer factor is about 75%. In various embodiments, the buffer factor is about 70%. In various embodiments, the buffer factor is about 65%. In various embodiments, the buffer factor is about 60%. In various embodiments, the buffer factor is about 55%. In various embodiments, the buffer factor is about 50%. In various embodiments, the buffer factor is about 45%. In various embodiments, the buffer factor is about 40%. In various embodiments, the buffer factor is about 35%. In various embodiments, the buffer factor is about 30%. In various embodiments, the buffer factor is about 25%. In various embodiments, the buffer factor is about 20%. In various embodiments, the buffer factor is about 15%. In various embodiments, the buffer factor is about 10%. In various embodiments, the buffer factor is about 5%. In various embodiments, the buffer factor is about 1%. In various embodiments, the buffer factor is about 0.1%. In various embodiments, the buffer factor is about 0.1% to about 100%. In various embodiments, the buffer factor is about 0.1% to about 90%. In various embodiments, the buffer factor is about 0.1% to about 80%. In various embodiments, the buffer factor is about 0.1% to about 70%. In various embodiments, the buffer factor is about 0.1% to about 60%. In various embodiments, the buffer factor is about 0.1% to about 50%. In various embodiments, the buffer factor is about 0.1% to about 40%. In various embodiments, the buffer factor is about 0.1% to about 30%. In various embodiments, the buffer factor is about 0.1% to about 20%. In various embodiments, the buffer factor is about 0.1% to about 10%. In various embodiments, the buffer factor is about 0.1% to about 5%. In various embodiments, the buffer factor is about 1% to about 100%. In various embodiments, the buffer factor is about 1% to about 90%. In various embodiments, the buffer factor is about 1% to about 80%. In various embodiments, the buffer factor is about 1% to about 70%. In various embodiments, the buffer factor is about 1% to about 60%. In various embodiments, the buffer factor is about 1% to about 50%. In various embodiments, the buffer factor is about 1% to about 40%. In various embodiments, the buffer factor is about 1% to about 30%. In various embodiments, the buffer factor is about 1% to about 20%. In various embodiments, the buffer factor may be about 1% to about 10%. In various embodiments, the buffer factor is about 1% to about 5%. For example, if a maximum thickness (from the axial bounds) is determined to be 10 µm, the maximum length is 30mm and the maximum width is 30mm (from the lateral bounds), and a 10% buffer is added to each measurement, the imageable volume would be 11 µm x 33mm x 33mm. In various embodiments, a buffer factor applied to the thickness is larger than the buffer factor applied to the lateral bounds toAttorney Docket No. GEI-05025 ensure that all thickness of the tissue sample is imaged, as complete data in the z-direction (i.e., thickness) throughout the tissue sample may be more valuable than data in along the perimeter (e.g., at the peripheries) of the tissue sample.

[0063] In various embodiments, an imageable volume of the tissue sample is defined. In various embodiments, the optics module divides the imageable volume into a plurality of fields of view (FOVs) and images the sample volume by way of imaging a z-stack of images for each FOV. In various embodiments, the z-stack from each FOV represents (e.g., approximates) a volume (e.g., a sub-volume of the total volume) of the sample. In various embodiments, the imageable volume is defined based on the maximum length, maximum width, and maximum thickness determined via the methods described herein. In various embodiments, the imageable volume includes the buffer factor applied to the determined dimensions as described above.

[0064] In various embodiments, one or more FOV (e.g., all FOVs of the sample) is imaged using a first objective lens having first optical properties (e.g., a first magnification and / or a first numerical aperture). In various embodiments, one or more FOV (e.g., all FOVs of the sample) is subsequently imaged using a second objective lens having second optical properties (e.g., a second magnification and / or a second numerical aperture). In various embodiments, the second magnification is a higher magnification than the first magnification. In various embodiments, the first FOV is divided into a plurality of sub-FOVs and each sub- FOV is imaged using the second objective lens having the second optical properties (e.g., a higher magnification than the first objective lens). In various embodiments, measuring focus scores (and thus, thicknesses) at the sub-FOV level provides more accurate thickness measurements of the sample as more data points are obtained across the sample surface. In various embodiments, sub-FOVs are determined for two or more additional objective lenses (e.g., each additional objective lens having higher magnification than the previous) and each set of sub-FOVs is imaged using the appropriate objective lens. In various embodiments, subsequently higher magnifications may require more sub-FOVs to be defined within the first FOV). In various embodiments, sub-FOVs are imaged for each FOV using one, two, three, four, five, etc. additional objective lenses.

[0065] In various embodiments, once tissue bounds (e.g., lateral tissue bounds) are obtained for the sample, images are registered to one another across imaging cycles. In various embodiments, for cyclic biochemical imaging (e.g., in situ analysis) the same volume is imaged over and over again. In various embodiments, the tissue itself is used for image registration. In various embodiments, a reference imaging volume is determined. In variousAttorney Docket No. GEI-05025 embodiments, to register an image in cyclical biochemical imaging, an image or set of images (e.g., a z-stack) is obtained and the image or set of images are registered to the reference volume to thereby determine physical offsets that register the cycles together. In various embodiments, image registration includes feature-based algorithms, such as scale-invariant feature transform (SIFT). In various embodiments, image registration includes an intensity- based algorithm, such as phase correlation.

[0066] FIG. 3B illustrates estimated local thicknesses at a plurality of FOVs illustrated in FIG. 3A. Each dot in FIG. 3B is colored based on the thickness determined based on the focus scores of a z-stack of images obtained at that FOV location. In various embodiments, a filter may be applied to the raw thickness data. In various embodiments, the filter may be applied only within the lateral bounds of the tissue (e.g., because tissue thicknesses drop off significantly at the lateral bounds, but may be similar within the bounds). In various embodiments, the filter may be a smoothing filter configured to smooth out the variations in the thickness data. In various embodiments, the filter may be a high pass filter, a low pass filter, and / or a band pass filter. In various embodiments, the filter may be a rolling ball filter.

[0067] FIG. 3C illustrates an overlay 300C of the local thicknesses on top of the dark field image 300A of the tissue sample 302. In particular, FIG. 3C illustrates the thickness plot of FIG. 3B overlaid on the image 300A of the tissue sample 302 such that each thickness measurement (i.e., colored dot) corresponds to an FOV of the dark field image 300A.

[0068] FIG. 3D illustrates a histogram of the local tissue thicknesses from the plot shown in FIG. 3B. As shown in FIG. 3D, the nominal thickness is about 10 µm. In various embodiments, the nominal thickness is greater than about 10 µm. In various embodiments, the nominal thickness is less than about 15 µm. In various embodiments, the nominal thickness is less than about 20 µm. In various embodiments, the nominal thickness is less than about 30 µm. In various embodiments, the nominal thickness is less than about 40 µm. In various embodiments, the nominal thickness is less than about 50 µm. In various embodiments, the nominal thickness is about 5 µm to about 20 µm. In various embodiments, the nominal thickness is about 10 µm to about 20 µm. In various embodiments, the nominal thickness is about 10 µm to about 15 µm.

[0069] FIG. 3E illustrates an image 300E of a tissue sample 306. As shown in FIG. 3E, the tissue sample 306 is stained with DAPI and illuminated with near ultraviolet (nUV) light using, for example, an epifluorescence imaging system. In various embodiments, the optics module images the tissue sample 306 as a plurality of fields of view (FOVs) 308. One highlighted FOV 308 is shown as an image 300F in FIG. 3F. In various embodiments, a z-Attorney Docket No. GEI-05025 stack of images is obtained at each FOV 308 for use in determining detachment of the sample. In various embodiments, a single image is obtained at each FOV for use in determining detachment of the sample. In various embodiments, the single image is a best in-focus image for that FOV, which may be determined using one of the methods described above. In various embodiments, the single image for each FOV is taken at a set z-height (e.g., a z-height set by a user).

[0070] In various embodiments, as shown in FIG. 3F, the FOV 308 is subdivided into a plurality of windows 310. The window size illustrated in FIG. 3F is for illustrative purposes only and the actual window size used to determined focus scores may be smaller or larger. In various embodiments, the windows 310 are non-overlapping. In various embodiments, each window is about 10 pixels by 10 pixels to about 1000 pixels by 1000 pixels in size. In various embodiments, each window is about 10 pixels by 10 pixels to about 500 pixels by 500 pixels in size. In various embodiments, each window is about 10 pixels by 10 pixels to about 200 pixels by 200 pixels in size. In various embodiments, each window is about 25 pixels by 25 pixels to about 100 pixels by 100 pixels in size. In various embodiments, each window is 25 pixels by 55 pixels in size. In various embodiments, the size of the window is dependent on the resolution of the imaging sensor (e.g., a larger pixel area can be used for higher resolution sensors and obtain substantially accurate results). In various embodiments, a focus score (e.g., Tenengrad, Vollath’s F4, etc.) is determined for each window 310 of the FOV 308. In various embodiments, the focus score is compared to a predetermined threshold to determine whether the window contains a region of detached sample. In various embodiments, a low focus score (representing a blurry image) corresponds to sample detachment while a high focus score (representing a sharp image) corresponds to attached tissue. In various embodiments, the predetermined threshold is set at a percentage of the maximum focus score across all FOVs. In various embodiments, the predetermined threshold is set at a percentage of the maximum focus score for each respective FOV. In various embodiments, the predetermined threshold is about 5% to about 95% of the maximum focus score. In various embodiments, the predetermined threshold is about 5%, 10%, 15%, 20%, 25%, 30%, 35%, 40%, 45%, 50%, 55%, 60%, 65%, 70%, 75%, 80%, 85%, 90%, or 95%. For example, the predetermined threshold can be set at 50% of the maximum focus score (e.g., Tenengrad) for that FOV of about 130,000. In this example, any window 310 of the FOV 308 having a focus score equal to or below 65,000 is considered detached while any window 310 of the FOV 308 having a focus score above 65,000 is considered attached. In various embodiments, a region of detached tissue is identified based on the focus scores ofAttorney Docket No. GEI-05025 the windows 310, and the region of detached tissue is automatically excluded from further in situ analysis, such as decoding of observed fluorescent punctae.

[0071] In various embodiments, a detachment metric is determined for each FOV 308 based on the windows 310 that are indicated as having detached tissue. In various embodiments, the detachment metric is a percentage of pixels having detached tissue across the FOV 308. For example, the total number of pixels having detached tissue may be calculated as the window size (e.g., 25 pixels x 50 pixels) multiplied by the number of windows determined to have detached tissue (i.e., windows having focus scores below the predetermined threshold). In various embodiments, the total pixels representing detached tissue is divided by the total pixels in an FOV 308 and multiplied by 100 to obtain a percentage. In various embodiments, if the percentage of detached tissue for the FOV 308 is above a predetermined threshold, the entire FOV is identified as low quality. In various embodiments, when a FOV 308 is identified as low quality, the FOV 308 is automatically excluded from further in situ analysis, such as further imaging cycles and / or decoding of observed fluorescent punctae. In various embodiments, the predetermined threshold is about 5% to about 95%. In various embodiments, the predetermined threshold is about 5%, 10%, 15%, 20%, 25%, 30%, 35%, 40%, 45%, 50%, 55%, 60%, 65%, 70%, 75%, 80%, 85%, 90%, or 95%. For example, if 25% of pixels in an FOV are determined to contain detached tissue, the FOV may be automatically excluded from imaging by the optofluidic instrument to thereby improve throughput of the analysis of the sample.

[0072] In various embodiments, the sample is segmented to determine pixels that are part of the sample and pixels that are not part of the sample. In various embodiments, the sample is segmented to determine windows that are a part of the sample and windows that are not a part of the sample. In various embodiments, a detachment metric is a percentage of pixels from windows identified as having sample detachment, where the pixels also have a label indicating that the pixels correspond to the sample. In various embodiments, a detachment metric is a percentage of windows identified as having sample detachment, where the windows also have a label indicating that the windows correspond to the sample (e.g., contain pixels of the sample). In various embodiments, when the detachment metric is above a predetermined threshold, the entire sample is determined to have poor quality and / or has high detachment. In various embodiments, the predetermined threshold is about 5%, 10%, 15%, 20%, 25%, 30%, 35%, 40%, 45%, 50%, 55%, 60%, 65%, 70%, 75%, 80%, 85%, 90%, or 95%. For example, if 20% of pixels having the sample label are determined to haveAttorney Docket No. GEI-05025 detachment, the entire sample may be identified as having poor quality and / or the sample is rejected by the instrument.

[0073] FIG. 3G shows a map (e.g., a heatmap) 300G generated from the determined focus scores for each FOV of the sample. As shown in FIG. 3G, the focus scores are overlaid on an image (e.g., DAPI image) of the sample using a color bar (yellow-gold coloring indicates high focus score and purple coloring indicates low focus score). In various embodiments, the map 300G is generated for display to a user. In various embodiments, the map 300G is presented to a user on a display, e.g., a display screen of the optofluidic instrument. In various embodiments, input is received from the user that includes a selection of one or more regions of the sample to exclude from further in situ analysis. In various embodiments, input is received from the user that includes a selection of one or more regions of the sample to include for further in situ analysis. In various embodiments, the regions of the sample determined to have detachment from the substrate are highlighted to the user, for example, through brightness, shading, outlines, colors, blinking, etc. In various embodiments, recommendations are provided to the user for regions to exclude from further in situ analysis. In various embodiments, the recommendations are determined from the determined focus scores, for example, the regions having the lowest quartile of focus scores.

[0074] FIG. 3H illustrates a map (e.g., a heatmap) 300H with the FOV 308 identified in the same location as in FIG. 3E. The FOV 308 is shown in greater detail in FIG. 3I with the focus values overlaid on the image of the sample. In FIG. 3I, the windows are smaller than those windows illustrated in FIG. 3F. In the zoomed-in image 300I of the FOV 308, areas with high focus scores (i.e., sharp focus) are indicated as yellow-gold color while areas with low focus scores (e.g., defocused) as purple color. In an example, high focus scores may be near or above 100,000, whereas low focus scores (i.e., defocused regions) may be near or below 25,000. In various embodiments, based on the map 300H, one or more regions of the sample are determined to have detachment based on a predetermined threshold, as described above.

[0075] FIG. 4 illustrates a graph of focus scores obtained from a plurality of slices in a single z-stack of images. In various embodiments, because a focused image has high contents of higher frequencies, the focus score may be based on a measure of high frequencies (e.g., edges) within the image. As shown in FIG. 4, a Tenenbaum Gradient (Tenengrad) focus measure is determined at each of a plurality of images (e.g., image slices) within a z-stack. The Tenengrad is a convolution of an image with vertical (Sx) and horizontal (Sy) Sobel operators. In various embodiments, the Tenengrad sums the square of all the magnitudesAttorney Docket No. GEI-05025 greater than a predetermined threshold. In various embodiments, all pixels are included in the summation. To determine a global measure over the whole image, the square of the gradient vector components are summed, as follows:(1) In various embodiments, a normalization factor is applied to the pixel values while determining the Tenengrad. In various embodiments, the normalization factor includes dividing each pixel value by a square root of the sum of the squared pixel values. In various embodiments, the normalization factor includes an intensity of the pixels (e.g., mean intensity, standard deviation, maximum intensity, minimum intensity, etc.) In various embodiments, a curve 402 may be interpolated from discrete Tenengrad focus scores determined for each image in the z-stack. In various embodiments, a same method of determining focus scores is used for each FOV. For example, a Tenengrad focus score may be determined for each image within the z-stacks across all FOVs. As shown in FIG. 4, a distance, d1, may be determined between two sides of the resulting curve interpolated from the discrete focus scores. In various embodiments, the distance, d1, is determined between a first inflection point 404a and a second inflection point 404b of the resulting focus score (e.g., Tenengrad) curve.

[0076] In various embodiments, an entropy-based focus score may be determined for one or more (e.g., all) images within the z-stacks.^^^^(2)

[0077] In various embodiments, a Sum of Modified Laplace (SML) focus score may be determined for one or more (e.g., all) images within the z-stacks. An example of a SML is as follows:(3)

[0078] In various embodiments, a focus score is based on the contrast of an image as the absolute difference of a pixel with its eight neighbors, summed over all the pixels of the image.(4.1)Attorney Docket No. GEI-05025 where the contrast C(x,y) for each pixel in the gray image I(x,y) is determined as:(4.2)

[0079] In various embodiments, a focus score is based on the coefficients of the discrete cosine transform obtained after dividing the image into 8x8 non overlapped windows and then averaging over all the 8x8 windows.(5.1)where :<;1 is determined from the DCT coefficients ^^=, >^ as:(5.2)

[0080] In various embodiments, a focus score is based on Vollath’s F4, where g(i, j) represented the gray level intensity of a pixel (i, j) in an image of size M x N:^D^EEF = ∑#0- / - ∑H. / - G^^, +^. G^^ + 1, +^ − ∑#0^ / - ∑H. / - G^^, +^. G^^ + 2, +^ (6)

[0081] FIGS. 5A-5H illustrate graphs of focus scores determined using various methods. As shown in FIG. 5A, a Tenengrad may be determined for one or more (e.g., all) images in a z- stack representing a volume at a particular FOV. As shown in FIG. 5B, a minimum pixel value (e.g., intensity value) may be determined for one or more (e.g., all) images in a z-stack representing a volume at a particular FOV. As shown in FIG. 5C, a maximum pixel value may be determined for one or more (e.g., all) images in a z-stack representing a volume at a particular FOV. As shown in FIG. 5D, a mean pixel value may be determined for one or more (e.g., all) images in a z-stack representing a volume at a particular FOV. As shown in FIG. 5E, a standard deviation of pixel values may be determined for one or more (e.g., all) images in a z-stack representing a volume at a particular FOV. As shown in FIG. 5F, a 10thpercentile (p10) of pixel values may be determined for one or more (e.g., all) images in a z- stack representing a volume at a particular FOV. As shown in FIG. 5G, a 50thpercentile (p50) of pixel values may be determined for one or more (e.g., all) images in a z-stack representing a volume at a particular FOV. As shown in FIG. 5H, a 90thpercentile of pixel values (p90) may be determined for one or more (e.g., all) images in a z-stack representing a volume at a particular FOV.Attorney Docket No. GEI-05025

[0082] FIG. 6A illustrates estimates of local heights at a plurality of FOVs at the top of the tissue sample. The z-heights represents as colored dots in the plot shown in FIG. 6A correspond to the right-side inflection point 404b in the Tenengrad plot of FIG. 4.

[0083] FIG. 6B illustrates estimates of local heights at a plurality of FOVs at the bottom of the tissue sample. The z-heights represents as colored dots in the plot shown in FIG. 6B correspond to the left-side inflection point 404a in the Tenengrad plot of FIG. 4.

[0084] FIG. 6C illustrates estimates of thicknesses of the top and bottom surfaces along a YZ plane of the tissue sample. In particular, FIG. 6C plots the top surface heights and the bottom surface heights of all FOVs (looking at the tissue sample from the YZ plane). As can be seen from FIG 6C, the tissue sample is substantially flat on the bottom and has some variation in height on the top surface.

[0085] FIG. 6D illustrates estimates of thicknesses of the top and bottom surfaces along a XZ plane of the tissue sample. In particular, FIG. 6D plots the top surface heights and the bottom surface heights of all FOVs (looking at the tissue sample from the XZ plane). As can be seen from FIG. 6D, the tissue sample is substantially angled on the bottom and top surfaces, which may suggest that the sample slide is tilted about the Y axis.

[0086] FIG. 7 illustrates a flow chart of a method for tissue bounds detection based on inherent contrast. At step 1, a coarse focus is determined at an FOV with an objective lens. At step 2a, an axial image z-stack is obtained (representing an estimated image volume) at the FOV with coarse steps to obtain a rough estimate of sample thickness. At step 2b, an axial image z-stack is obtained (representing an estimated image volume) at the FOV with fine steps to obtain a more-accurate estimate of sample thickness. At step 3, focus scores are determined for each image in each z-stack and an axial tissue bound (i.e., thickness) is determined for the FOV. If axial bounds are not determined, the process returns to step 2a. If axial bounds are able to be determined (e.g., within a predetermined confidence metric), the process proceeds to step 4. At step 4, the objective is moved to the next FOV. At step 5, when the entire sample has been scanned, lateral bounds are determined based on the determined thicknesses at each FOV.

[0087] FIG. 8 illustrates a flow chart of a method 800 for determining detachment of a sample from a substrate. In various embodiments, 802 a plurality of images of a sample on a substrate are received. The plurality of images include a plurality of fields of view of the sample. At 804, for each image in the plurality of images, at least one focus score is determined. At 806, based on the focus scores, one or more portions of the sample are determined as being detached from the substrate.Attorney Docket No. GEI-05025

[0088] In various embodiments, determining the one or more portions of the sample as being detached from the substrate includes: for each focus score, comparing the focus score to a predetermined threshold and, when the focus score is below the predetermined threshold, indicating that the one or more portions of the sample corresponding to the focus score are detached from the substrate. Multi-Focus Image Fusion With Background Removal

[0089] Described herein are methods for and systems for performing multi-focus image fusion with background removal. In particular, multi-focus image fusion generates a single best-focused image from one or more z-stacks of images (e.g., a plurality of z-stacks of images where each z-stack represents a different FOV of a plurality of FOVs). In some embodiments, a best in-focus image is generated for each color channel (e.g., red, yellow, green, blue, nUV) of an imaging instrument (e.g., an optofluidic instrument). Due to variations in the thickness, composition, and / or surface contours of a sample, the most in- focus z-slice (which can be represented as a z-slice index, e.g., an integer) of a z-stack of images taken of the sample may differ across FOVs of the sample (e.g., two adjacent FOVs have most in-focus z-slices that differ by one, two, three, four, or five z-slice indices). In some embodiments, the most in-focus z-slice may differ across a single FOV of the sample (e.g., if the FOV is divided into patches or sub-portions, two adjacent patches or sub-portions within the FOV may differ by one, two, three, four, or five z-slice indices). Because of this variation in focus across a sample, selecting a single z-slice to represent a FOV of the sample (or a plurality of FOVs of the sample) may result in diminished performance of downstream image processing where the most in-focus images (e.g., images with high contrast and well- defined edges) will produce better results, e.g., for nucleus and / or cell segmentation tasks. Moreover, images of biological samples (e.g., tissue samples, hydrogels having analytes immobilized therein, etc.) may include background signal (e.g., autofluorescence, reflected excitation light, etc.) that can be removed to further improve performance of downstream image processing. The systems and methods described herein advantageously divide each FOV into smaller patches, determine the best in-focus z-slice index for each individual patch, and fuse the most in-focus patches together into a single most in-focus image that has high focus (e.g., high focus scores) across the entirety of the image. Additionally, or optionally, the systems and methods described herein advantageously remove background signal from the single most in-focus image to produce a high quality, in-focus, and background-removed image of a sample that can be used in downstream image processing, such as nucleus and cellAttorney Docket No. GEI-05025 segmentation tasks or to present to a user in a graphical user interface (e.g., a display). In some embodiments, the single most in-focus and background-removed image is generated for each color channel in an imaging instrument.

[0090] FIG. 9 illustrates a z-slice image 900 exhibiting distortion at the edges of the FOV due to field curvature. Field curvature (also called Petzval field curvature) is an optical aberration where light from a flat object is focused onto a curved surface instead of a flat plane, meaning that the sharpest image is formed on a curved focal plane, causing the edges of an image to appear blurry while the center remains sharp. Field curvature may be particularly noticeable in wide-angle lenses. As shown in FIG. 9, the z-slice image 900 includes a plurality of cells scattered across the FOV. Due to the field curvature of the objective lens used in the optofluidic instrument, regions at the edges of the FOV become defocused (because the focal plane is not perfectly flat across the FOV). In various embodiments, the field curvature can cause up to 3 z-slices of shift to the focal plane at various portions (e.g., the edges) of the FOV (assuming a 0.75 µm step size between z- slices), thereby causing the cells at those portions of the FOV to appear out-of-focus. In one example of simple field curvature, a lens focuses rays of light from a curved field that passes through at least image plane A and image plane B. Thus, some portions of the resulting image will be most in-focus at image plane A and some portions of the resulting image will be most in-focus at image plane B.

[0091] FIG. 10 illustrates various cycles of an optofluidic instrument used to generate a final stain image using multi-focus image fusion. In various embodiments, the optofluidic instrument has a total of 17 cycles, which includes 15 decoding cycles, a blank cycle to image background, and a cell segmentation cycle to image various stains (e.g., antibody stains, cytoplasmic RNA stains and / or nuclear stains) needed for downstream cell segmentation tasks. Decoding cycles may also be referred to as probing cycles. In various embodiments, the optofluidic instrument has over 30 decoding cycles for detecting analytes (e.g., for an analysis of a 5000 gene panel). In various embodiments, the optofluidic instrument performs over 100 decoding cycles for detecting analytes (e.g., for a whole transcriptome analysis). In various embodiments, a first probing cycle 1002 (e.g., an RNA decoding cycle) includes a nuclear stain (e.g., DAPI) cycle imaged in a nuclear stain color channel (e.g., a nUV channel), and four channels for imaging RCPs (e.g., red, yellow, green, and blue channels). In various embodiments, additional probing cycles 1004 (e.g., cycles 2 through 15) are imaged in four color channels (e.g., red, yellow, green, and blue channels) for imaging analytes (e.g., RCPs associated with a RNA transcript). In various embodiments, a blankAttorney Docket No. GEI-05025 cycle 1006 is performed after RNA decoding cycles have been performed (e.g., after all RNA decoding cycles have been performed). In various embodiments, the blank cycle 1006 is performed after a predetermined number of decoding cycles has been performed such that the background signal is determined to be stable. For example, the predetermined number of decoding cycles may be 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, or 15 cycles. In various embodiments, the predetermined number of decoding cycles is at least 8 cycles. In various embodiments, the blank cycle 1006 is imaged in the nuclear stain color channel (e.g., nUV channel) and one or more other color channels to determine nuclear stain signal, autofluorescence signal, and / or background light (e.g., excitation light reflected at the sample and received at the image sensor through the emission filter). For example, the one or more channels includes the red channel, green channel, and blue channel (no yellow channel). One skilled in the art will recognize that any suitable number of color channels can be imaged to obtain suitable estimates of the nuclear stain signal, autofluorescence signal, and / or background light. In another example, the one or more channels includes the red channel, yellow channel, green channel, and blue channel. In various embodiments, a cell segmentation cycle 1008 is performed. In various embodiments, the cell segmentation cycle 1008 is performed as the last cycle (i.e., after all RNA decoding cycles and the blank cycle). In various embodiments, the cell segmentation cycle 1008 is imaged in the nuclear stain channel (e.g., nUV channel) and one or more other channels to thereby image one or more stains (e.g., membrane stain, cytoplasmic RNA stain, cytoplasmic protein stain, and / or nuclear stain) for downstream cell segmentation tasks. Exemplary methods of image segmentation and multi-modal cell segmentation can be found in U.S. patent application publication no. 2025-0061732 and U.S. patent application no. 19 / 037,114, each of which is incorporated by reference herein in its entirety. For example, the one or more channels includes the red channel, green channel, and blue channel (no yellow channel). In another example, the one or more channels includes the red channel, yellow channel, green channel, and blue channel. In various embodiments, when imaging proteins, the blank cycle and the cell segmentation cycle include DAPI imaging (e.g., in the nUV channel) and all color channels (e.g., in red, yellow, green, and blue channels). In various embodiments, the cell segmentation cycle 1008 may be performed before the blank cycle 1006. In various embodiments, a final stain image 1010 is generated using the multi-focus image fusion techniques described below.

[0001] In various embodiments, a first z-stack of images of a biological sample is received. In various embodiments, the first z-stack of images corresponds to a field of view (FOV). InAttorney Docket No. GEI-05025 various embodiments, the first FOV comprises a plurality of (i.e., two or more) patches. In various embodiments, the z-stack first images is based on at least one probing cycle of a biological sample in an imaging instrument. In various embodiments, the biological sample includes at least one cellular structure (e.g., one or more nuclei). In various embodiments, the first z-stack of images includes the cellular structure (e.g., the one or more nucleus).

[0092] In various embodiments, a first z-stack of images of a biological sample is acquired or received from an imaging instrument or other source. In various embodiments, the first z- stack of images includes background images of the biological sample, which may or may not include a background stain (e.g., DAPI). In various embodiments, the nuclear stain is applied and background signal is obtained in non-nuclear stain color channels (i.e., non-nUV color channels) to determine background signal that results from the nuclear stain. In various embodiments, the first z-stack of images is of a biological sample that lacks cytoplasmic, membrane, and / or nuclear staining. Preferably, the first z-stack of images is of a biological sample that lacks cytoplasmic and membrane staining. In various embodiments, the first z- stack of images includes images of a biological sample stained with nuclear staining, such as DAPI staining. In various embodiments, the background stain is only DAPI staining (i.e., no other stains emit light in the background stain image). In various embodiments, this first z- stack of images is referred to as a z-stack of background images.

[0093] In various embodiments, the first z-stack of images is captured after a predetermined number of probing cycles of an imaging instrument (where emitted light from fluorescently labelled rolling circle products is imaged). For example, the predetermined number of probing cycles may be 15 probing cycles. In various embodiments, the background signal of the sample may change during sample analysis. For example, the background signal after the first probing cycle may be different than the background cycle after the tenth probing cycle, but the background signal may not change (i.e., stabilizes) after a set number of cycles are completed. In various embodiments, the first z-stack of images representing the background is captured after the background signal of the sample is stable. In various embodiments, when the background signal (e.g., autofluorescence) becomes stable is empirically determined. For example, imaging can be performed of the biological sample with no staining to measured autofluorescence over 20 cycles and average intensity change per cycle can be determined. In various embodiments, the average intensity change decreases over time, and the sample has a stable background after ~8 cycles of imaging. In various embodiments, only one color channel (e.g., near UV) is imaged for the background z-stack ofAttorney Docket No. GEI-05025 images. In various embodiments, multiple channels are imaged for the background z-stack of images (e.g., nUV, red, green, and blue).

[0094] In various embodiments, a ZCYX (or ZCXY) imaging procedure is followed. Specifically, a z-stack of images is obtained at a fixed FOV (with fixed x- and y-values) for each color channel of a plurality of color channels. Then the fixed x- and / or y-values are changed to obtain images at a new FOV, repeating the z-stack imaging for each color channel, and this process continues for the entire tissue sample (or for a selected subset of FOVs). One benefit of this technique is that no image registration may be needed. Also, moving first in the z-direction, without changing the x- or y-values, can help minimize error since moving in the x- and y-directions can produce more error than moving in the z- direction. This imaging approach may be used to avoid computational image registration between channels, as the objective remains in the same XY position relative to the stage during capture of the z-stack across multiple color channels.

[0095] In various embodiments, a ZYXC (or ZXYC) imaging procedure is followed. Specifically, a z-stack of images is obtained at a fixed FOV (with fixed x- and y-values) for a single color channel before changing the x- and / or y-values to obtain images at a new FOV in the same color channel. After all FOVs are imaged for the entire tissue sample (or for a selected subset of FOVs) in the same color channel, the color channel is switched to the next color channel and a z-stack of images is obtained at all FOVs in the next color channel. One benefit of this method may be that the reduced number of switches between color channels results in less time elapsed for the entire procedure. Switching from one color channel to another color channel can take longer than moving in the z-direction through the sample. Also, moving first in the z-direction, without changing the x- or y-values, can help minimize error since moving in the x- and y-directions can produce more error than moving in the z- direction. Image registration may be required for FOVs across all color channels due to thermal drift in the sample and / or mechanical drift in the motion systems (e.g., z-stage, xy stage, etc.).

[0096] In various embodiments, the first z-stack of images is obtained using a ZXY imaging order where a z-stack is imaged, and the imaging instrument moves in X and / or Y to the next FOV for imaging of the next z-stack. In various embodiments, the images are captured using a ZCYX (or ZCXY) imaging order. In particular, an imaging order of ZCYX (or ZCXY) images a z-stack of a single FOV in all color channels before moving in X and / or Y to the next FOV to image the next FOV in all color channels, and so on and so forth until all FOVs are imaged. This imaging approach may be used to avoid computational image registrationAttorney Docket No. GEI-05025 between channels, as the objective remains in the same XY position relative to the stage during capture of the z-stack across multiple channels.

[0097] In some embodiments, the first imaging cycle is imaged using a ZCYX imaging procedure while subsequent imaging cycles for detection of analytes are imaged using a ZYXC imaging procedure. In some embodiments, imaging cycles for cell segmentation are imaged using a ZCYX imaging procedure. In some embodiments, imaging cycles for cell segmentation are imaged using a ZYXC imaging procedure. In some embodiments, a DAPI color channel (e.g., nUV) is imaged during (e.g., at the beginning of) each imaging cycle. In some embodiments, a DAPI color channel is imaged during the first imaging cycle for detection of analytes (but not subsequent imaging cycles for detection of analytes) and also during cell segmentation imaging cycles after the analyte detection imaging cycles are completed.

[0098] In various embodiments, the biological sample has no other staining, other than nuclear (e.g., DAPI) staining, during the blank cycle where a background z-stack of images (e.g., a multichannel z-stack of images) is obtained. For example, cytoplasmic staining (e.g., proteins or 18S ribosomal RNA) and / or membrane staining may not be applied to the sample and, thus, cytoplasmic and / or membrane staining is not imaged. Acquisition of multiple images, such as in the first z-stack of images representing the background signal of the biological sample, may be performed because the background of the images may change over cycles (but may stabilize after a predetermined number of cycles have been performed). In various embodiments, a base level of background signal is determined from one blank cycle and the background signal may be determined again from an addition, subsequent blank cycle to determine the extent to which the background signal changes over time (e.g., over one or more cycles of staining and imaging for cell segmentation).

[0099] In various embodiments, after the first z-stack of images is received or captured, a number of well-focused z-slices (See FIG. 11) of the z-stack are determined. FIG. 11 depicts a visualization of a z-slice of a multi-channel z-stack of images of a biological sample (e.g., a background image with only DAPI staining). In particular, z-slice 30 (slice located at 22.5µm above the substrate surface) is being shown for a first background cycle. In various embodiments, distortion correction is performed on the number of well-focused z-slices of the z-stack. For example, field curvature aberration correction is computationally performed. In another example, other optical aberrations can be computationally corrected for, such as, spherical aberration, coma, astigmatism, distortion, chromatic aberration. In various embodiments, deconvolution may be performed on the image to sharpen the image based onAttorney Docket No. GEI-05025 a point spread function that can be determined empirically for a particular optical system. In various embodiments, the first z-stack of images comprises multi-channel images (e.g., a z- stack of images in one or more color channels, such as, nUV, red, yellow, green, and / or blue channels). In various embodiments, Scale-Invariant Feature Transform (SIFT) features are extracted from these z-slices. In various embodiments, z-slices of the first z-stack of images are scored according to a focus score, such as a Vollath’s F4 or Tenengrad. In various embodiments, a predetermined number (e.g., 3, 4, 5, 6, 7, 8, 9, 10, 11, etc.) of the highest scoring z-slices are determined. For example, the predetermined number may be three or more z-slices. In various embodiments, distortion correction may be performed based on information, such as distortion parameters, from the imaging instrument. For example, distortion correction may be performed to correct for distortion such as pincushion distortion as well as other types of distortion. In various embodiments, SIFT features from the determined / selected z-slices are then extracted. In various embodiments, the SIFT features are associated with features, textures, or points of at least one cellular structure shown in the z-slices, such as a morphology of the nuclei of one or more cells shown in the z-slices.

[0100] In various embodiments, a second z-stack of images of the biological sample is received. In various embodiments, the second z-stack corresponds to a second FOV, which may be the same FOV as the FOV of the first z-stack. In various embodiments, the second FOV includes a plurality of (i.e., two or more) patches, e.g., the same number of patches as the first FOV. In various embodiments, the second z-stack of images is based on at least one probing cycle of the biological sample in an imaging instrument. In various embodiments, the biological sample includes at least one cellular structure (e.g., one or more nucleus). In various embodiments, the second z-stack of images includes the cellular structure (e.g., the one or more nucleus), and this may be the same cellular structure in the first z-stack of images.

[0101] In various embodiments, the second z-stack of images of a biological sample is acquired or received from an imaging instrument or other source. In various embodiments, the second z-stack of images comprises multi-channel images (e.g., a z-stack of images in one or more color channels, such as, nUV, red, yellow, green, and / or blue channels). In various embodiments, the second z-stack of images includes images of the biological sample stained with at least one stain such that the at least one stain is illuminated in at least one illumination channel. In various embodiments, the at least one stain includes one or more fluorescent stains. The fluorescent stains may include one or more nuclear stain, such as DAPI, and / or one or more cytoplasmic stain, such as one or more ribosomal RNA stain (e.g.,Attorney Docket No. GEI-05025 18S), one or more lectin stain, one or more antibody stain, and / or the like. In various embodiments, this second z-stack of images is referred to as a z-stack of stained images.

[0102] In various embodiments, the second z-stack of images is captured after a predetermined number of probing cycles of an imaging instrument (where emitted light from fluorescently labelled rolling circle products is imaged). In various embodiments, the images are captured using a ZCYX (or ZCXY) imaging order. In particular, an imaging order of ZCYX (or ZCXY) images a z-stack of a single FOV in all color channels before moving in X and / or Y to the next FOV to image the next FOV in all color channels, and so on and so forth until all FOVs are imaged. This imaging approach may be used to avoid computational image registration between channels, as the objective remains in the same XY position relative to the stage during capture of the z-stack across multiple channels.

[0103] In various embodiments, after the second z-stack of images is received or captured, a number of well-focused stained z-slices (See FIG. 12) of the z-stack are determined. FIG. 12 depicts a visualization of a z-slice of a multi-channel z-stack of images of a stained biological sample. In particular, z-slice 30 (slice located at 22.5µm above the substrate surface) is being shown for a first cell segmentation cycle where one or more stains are imaged for downstream cell segmentation purposes (e.g., one or more cell membrane stain, one or more cytoplasmic stain, one or more nuclear stain, etc.). In various embodiments, distortion correction is performed on the number of well-focused z-slices of the z-stack. For example, field curvature aberration correction is computationally performed. In another example, other optical aberrations can be computationally corrected for, such as, spherical aberration, coma, astigmatism, distortion, chromatic aberration. In various embodiments, deconvolution may be performed on the image to sharpen the image based on a point spread function that can be determined empirically for a particular optical system. In various embodiments, the first z-stack of images comprises multi-channel images (e.g., a z-stack of images in one or more color channels, such as, nUV, red, yellow, green, and / or blue channels). In various embodiments, SIFT features are extracted from these stained z-slices. In various embodiments, z-slices of the z-stack of images are scored according to a focus score, such as a Vollath’s F4 or Tenengrad. In various embodiments , a predetermined number (e.g., 3, 4, 5, 6, 7, 8, 9, 10, 11, etc.) of the highest scoring z-slices are determined. For example, the predetermined number may be three or more z-slices. In various embodiments, distortion correction is performed based on information, such as distortion parameters, from the imaging instrument. For example, distortion correction may be performed to correct for distortion such as pincushion distortion as well as other types ofAttorney Docket No. GEI-05025 distortion. In various embodiments, SIFT features from the determined / selected stained z- slices are then extracted. In various embodiments, the SIFT features are associated with features or points of at least one cellular structure shown in the z-slices, such as a morphology of the nuclei of one or more cells shown in the z-slices, which may include the same or similar SIFT features associated with the first z-stack of images.

[0104] In various embodiments, the first z-stack of images and the second z-stack of images are registered to each other based on at least one cellular structure (e.g., at least one nucleus, at least one cell membrane, etc.) within the biological sample. In various embodiments, XYZ registration of the first z-stack of images and the second Z-stack of images is performed only in the nUV (DAPI) channel assuming that other channels are aligned with the nUV channel due to the ZCYX imaging order. In various embodiments, where ZYXC imaging is performed, registration is performed between the different channels imaged in the same cycle. At least one of the SIFT features associated with the cellular structure from the z- slices from the first z-stack, as described above, are matched with the SIFT features from the stained z-slices from the second z-stack, as described above. Based on this matching, an XYZ translation between the first z-stack and the second z-stack is determined using an image registration algorithm, such as the random sample consensus (RANSAC) algorithm. For example, the matching may produce a registered coordinate space, which includes an estimate of a translation of points in space between the first and the second z-stacks. In various embodiments, an image registration algorithm such as RANSAC produces a registered coordinate space for the first and the second z-stacks. For example, in various embodiments, the nucleus in the first z-stack of images may be registered to the nucleus in the second z-stack of images.

[0105] In various embodiments, a focus map is determined based on the second z-stack of images. In various embodiments, a focus map is determined for each color channel. For example, a focus map may be determined for a nUV channel, a blue channel, a green channel, a yellow channel, and / or a red channel. In various embodiments, the focus map indicates, for each of a plurality of patches of the selected FOV (which may be the first FOV, for example), one of the images (i.e., z-slices) of the second z-stack bringing into focus that patch of the FOV. In various embodiments, each patch is a block or group of a predetermined number of pixels (e.g., 8 by 8 pixel block, 16 by 16 pixel block, 32 by 32 pixel block, or 64 by 64 pixel block). In various embodiments, all patches will be square (have equal number of pixels for each side). In various embodiments, all patches will have the same size of block (e.g., all patches will be 64 by 64 pixels).Attorney Docket No. GEI-05025

[0106] In various embodiments, a multi-focus image fusion algorithm is applied to the second z-stack of images (e.g., using the selected subset of slices from the second z-stack of images). In various embodiments, the multi-focus image fusion algorithm is applied for each FOV. In various embodiments, the multi-focus image fusion algorithm is applied for each color channel. For example, with four color channels of red, yellow, green, and blue, the multi-focus image fusion algorithm may generate four focus maps for each FOV that is imaged. At a high-level, a multi-focus image fusion algorithm is an algorithm to computationally synthesize an all-in-focus image and / or a fused image from images taken at different focal planes (i.e., a z-stack of images). In particular, the resulting fused, most in- focus image includes a focused image synthesized from images taken with different focuses (i.e., images taken at different z-slices at a FOV). Multi-focus image fusion algorithms may have applications to volumetric imaging and, in particular, to microscopy applications where three-dimensional volumes are imaged (e.g., in situ analysis of three-dimensional samples). Any such multi-focus image fusion algorithm may have variations (e.g., based on the sample type, analytes being detected / quantified, etc.). The application of the multi-focus image fusion algorithm may be used to determine a focus map and generate an all-in-focus image and / or a fused image.

[0107] In various embodiments, a multi-focus image fusion algorithm is applied to the second z-stack of images using the following process.

[0108] As one step, in various embodiments, the most focused z-slice (i.e., image plane) of z- slices in the second z-stack of images is determined. For example, the most focused z-slice out of all (e.g., 40) z-slices in the second z-stack of images may be determined by scoring each z-slice according to a focus score, such as Vollath’s F4 or Tenengrad, and the highest scoring z-slice is determined as the most focused z-slice. FIG. 13A shows a grayscale image of a z-slice having a z-index of 10 (i.e.,the 11th z-slice in the z-stack from a reference slice having an index of 0, such as the glass slide). Where 20 adjacent z-slices are selected, the z- index of 10 may represent the most in-focus z-slice within the z-stack and 10 slices below the 11th z-slice and 10 slices above the 11th z-slice are selected for further processing, as explained in more detail below. In various embodiments, the Vollath’s F4 focus score for an image is a single, unitless number representing how in-focus the image is across the entire image. The equation for Vollath’s F4 is below:Attorney Docket No. GEI-05025 where g(i, j) is the gray level intensity of I pixel (i, j) in an image of size M x N.

[0109] In various embodiments, because a focused image has high contents of higher frequencies, the focus score may be based on a measure of high frequencies (e.g., edges) within the image. In various embodiments, a Tenenbaum Gradient (Tenengrad) focus score is determined for a plurality of images (e.g., image slices) within a z-stack. The Tenengrad is a convolution of an image with vertical (Sx) and horizontal (Sy) Sobel operators. In various embodiments, the Tenengrad sums the square of all the magnitudes greater than a predetermined threshold. In various embodiments, all pixels are included in the summation. To determine a global measure over the whole image, the square of the gradient vector components are summed, as follows:In various embodiments, a normalization factor is applied to the pixel values while determining the Tenengrad. In various embodiments, the normalization factor includes dividing each pixel value by a square root of the sum of the squared pixel values. In various embodiments, the normalization factor includes an intensity of the pixels (e.g., mean intensity, standard deviation, maximum intensity, minimum intensity, etc.).

[0110] In various embodiments, other suitable focus scores may be determined. In various embodiments, an entropy-based focus score may be determined for one or more (e.g., all) images within the z-stacks. ^^^^

[0111] In various embodiments, a Sum of Modified Laplace (SML) focus score may be determined for one or more (e.g., all) images within the z-stacks. An example of a SML is as follows:

[0112] In various embodiments, a focus score is based on the contrast of an image as the absolute difference of a pixel with its eight neighbors, summed over all the pixels of the image.where the contrast C(x,y) for each pixel in the gray image I(x,y) is determined as:Attorney Docket No. GEI-05025

[0113] In various embodiments, a focus score is based on the coefficients of the discrete cosine transform obtained after dividing the image into 8x8 non overlapped windows and then averaging over all the 8x8 Windows.where :<;1 is determined from the DCT coefficients ^^=, >^ as:

[0114] As another step, in various embodiments, a predetermined number of z-slices in the second z-stack adjacent to the most focused plane are selected. For example, 10 z-slices above and 10 z-slices below a most in-focus z-slice may be selected, for a total of 21 slices in the predetermined number of slices. In various embodiments, the predetermined number of z-slices together with the most in-focus plane is referred to as a selected set of adjacent images. In various embodiments, the selected set of adjacent images has a predetermined size, such as a size between 2 images and 41 images. For example, a predetermined number of z-slices above and below (e.g., ±10) the most focused plane / z-slice may be determined and extracted to form the selected set of adjacent images. In various embodiments, the upper bound on the predetermined size depends on the tissue thickness. For example, thicker tissue samples may have a larger predetermined size of the selected set of images. Each of these images in the selected set of adjacent images may include associated corresponding z-index indicating its position within the second z-stack. In various embodiments, the z-index ranges from 0 to a maximum number of adjacent images (e.g., 0 to 20). Alternatively, the z-index ranges from -max adjacent images / 2 to a + max adjacent images / 2 (e.g., -10 to +10) where the zeroth index is the most in-focus z-slice. As explained above, this process of determining a most in-focus z-slice based on a focus score and then selecting a predetermined number of adjacent slices can be performed for each color channel of a plurality of color channels (e.g., nUV, blue, green, yellow, and / or red).Attorney Docket No. GEI-05025

[0115] As another step, optionally, a denoising filter, such as a Gaussian filter, may be applied to each image of the selected set of adjacent images determined at the previous step. For example, the filter may be applied to denoise each z-slice image in the set.

[0116] As another step, in various embodiments, each of the selected set of adjacent images is divided into a number of image patches (2-D x-y dimension patches). In various embodiments, a most focused z-slice (which may be represented as a z-index, which is an incrementing integer assigned to each z-slice in the z-stack) is selected for each image patch. As used herein, a “patch” is a group of pixels, such as a square of 64x64 pixels, in an image. In various embodiments, each of the images corresponding to the selected set of adjacent images is divided into the same plurality of patches. Each patch may be a portion of an image (i.e., a z-slice of a z-stack taken at a FOV) in the x-y plane that also corresponds to a z- index (varying in the z direction) of the selected set of adjacent images. In various embodiments, the z-index for each patch corresponds to a z-index of a z-slice within the second z-stack. In various embodiments, each patch may have the same shape, such as a square shape, and may be about 4x4 to 128x128 pixels in size. For example, the number of image patches may include 16 x 16 pixel denoised image patches, 32 x 32 pixel denoised image patches, 64 x 64 pixel denoised image patches, or 96 x 96 pixel denoised image patches. In various embodiments, 64 x 64 pixel patches was selected because this patch size captures the texture of a single cell (a single nucleus is about 20 µm x 20 µm, which is about 100 pixels by 100 pixels).

[0117] In various embodiments, the focus map is generated having the same size and number of patches as the selected set of adjacent images. For each patch in the focus map, a most- focused patch is selected from the patches in the same position across the selected set of adjacent images and a z-index of the most-focused patch is assigned to the respective patch in the focus map. Each patch in each image of the selected set of adjacent images (with varying z-indices) is scored according to a focus score, such as Tenengrad or a Vollath’s F4, and the z-index of the highest scoring patch is determined. Performing such a technique for all patches, thereby assigning z-indices from the most in-focus patches from the selected set of images to a map, generates a raw focus index map, as shown in FIG. 13B. In various embodiments, the raw focus index map indicates the index of the z-slice associated with the selected image for each patch.

[0118] As another step, in various embodiments, outliers in the raw focus index map are removed to generate an outlier-removed focus index map. In various embodiments, for each patch (as a target patch) in the raw focus index map, z-indices of one or more neighboringAttorney Docket No. GEI-05025 patches are compared with the z-index of the target patch. That is, the z-index of a patch will be compared to the z-indices of neighboring patches, and this comparison will be performed for each patch in the raw focus map. In various embodiments, when the patch has more than a predetermined disparity (e.g., difference) between its associated z-index and a z-index associated with one or more neighboring patches (See FIG. 13C), the patch is determined to be a disparate patch. FIG. 13C illustrates a heat map of number of neighbors with an abrupt change. That is, for each patch, the color represents the number of neighbors (0, 1, 2, 3, or 4 neighbors) having z-indices that are above a predetermined disparity (e.g., greater than 5 z- index difference). In various embodiments, disparate patches are removed, as shown in FIG. 13D. In various embodiments, after removal, disparate patches are replaced with replacement values for the indices, as shown in FIG. 13E. In particular, the replacement values may be interpolated z-indices between neighboring patches (e.g., interpolated based on the z-indices of all 4 adjacent patches) such that the interpolated value is within the predetermined disparity. In various embodiments, the predetermined disparity is about 2 to about 10 z-indices. In various embodiments, the predetermined disparity is about 5 z-indices. In various embodiments, the predetermined disparity is about 10% to about 50% of the total number of well-focused stained z-slices selected. In various embodiments, the predetermined disparity is about 20% to about 30% of the total number of well-focused stained z-slices selected. In various embodiments, the predetermined disparity is about 25% of the total number of well-focused stained z-slices selected (e.g., predetermined disparity of 5 for a selection of 20 well-focused stained z-slices). In various embodiments, a z-index of a patch is removed and replaced when two or more (e.g., two) neighboring patches have z-indices with a predetermined disparity with the z-index of the patch. For example, if a patch has a z- index of 10, two neighboring patches with z-indices of 11 and two neighboring patches with z-indices of 16, then the patch having the z-index of 10 will be removed and replaced (e.g., replaced with an average of the pixel values, 13). In various embodiments, when the z-index of a patch is determined to disparate (after comparison with one or more z-indices of one or more neighboring patches), the patch is replaced by a z-index value that is based on a nearest neighbor z-index value. In various embodiments, if the z-index of the current patch is greater than a predetermined disparity from the z-index of a neighboring patch, and the number of such neighboring patches is greater than a threshold number of disparate neighbors, then the z-index of the current patch in the raw focus index map is replaced. For example, the replacement z-index value may be interpolated based on two or more neighboring patch z- index values (See FIG. 13E). In various embodiments, the replacement z-index value is anAttorney Docket No. GEI-05025 average of a number of (e.g., all four) neighboring patch z-index value(s). In various embodiments, the replacement z-index value is assigned one of the neighboring patch z-index values (rather than interpolating). After such outlier indices are removed (and replaced) from the raw focus index map, the resulting map is an outlier-removed focus index map.

[0119] As another step, in various embodiments, a filter (e.g., a median filter) is applied to the raw and / or outlier-removed focus index map(s) to produce a filtered focus index map (See FIG. 13F). As shown in FIG. 13, a median filter is applied to the focus index map (after having outliers removed and replaced). In various embodiments, the filtered index map includes floating point values for the z-indices (rather than integers).

[0120] As another step, in various embodiments, the focus index map (e.g., the raw, outlier- removed or filtered focus index map) is upsampled to a full pixel resolution. As shown in 13G, the median filtered focus map is upsampled to a full pixel resolution such that each pixel is assigned a z-index value (e.g., a floating-point value). For example, the raw, outlier- removed or filtered focus index map may be upsampled using linear interpolation. In particular, the per pixel value of the z-index may be interpolated based on the raw, outlier- removed or filtered focus index map. In various embodiments, the resulting focus index map, after upsampling the raw, outlier-removed or filtered focus index map, is referred to herein as an upsampled focus map generated by the application of a multi-focus image fusion algorithm. In various embodiments, the upsampled focus map is used to sample a z-stack of images. In various embodiments, the upsampled focus map includes floating point values. In various embodiments, a function, such as a round, floor, or ceiling function, is applied to the upsampled focus map before it is used to sample a z-stack, in order to change any floating- point values of the z-indices in the focus map to integer values of the z-indices. For example, a floor function can be applied to the upsampled focus map to sample the background fuse_bg_0, and then the background can be sampled again for fuse_bg_1 by adding one to each z-index in the focus map. In another example, a ceiling function can be applied to the upsampled focus map to sample the background fuse_bg_1, and then the background can be sampled again for fuse_bg_0 by subtracting one to each z-index in the focus map.

[0121] In various embodiments, the second z-stack of images is sampled using the upsampled focus map generated by the application of the multi-focus image fusion algorithm. Thus, the upsampled focus map is applied to the second z-stack of images. The result of applying the upsampled focus map to the z-stack of images may be referred to herein as a stained fused FOV image or an all-in-focus FOV image generated by applying the upsampled focus map generated by the multi-focus image fusion algorithm to the second z-stack ofAttorney Docket No. GEI-05025 images. FIG. 13H shows an example of a stained fused image (e.g., in a specific color channel) generated by applying the upsampled focus map to a plurality of z-stacks of images (representing a plurality of FOVs of a sample). As explained above, a plurality of upsampled focus index maps may be generated, where each upsampled focus index map represents a specific color channel. In summary, the multi-focus image fusion algorithm may assist in determining, for the second z-stack of images, for each pixel patch, which z-slice of the second z-stack of images to include in an all-in-focus / fused FOV image based on the upsampled focus map.

[0122] In various embodiments, the multi-focus image fusion algorithm is applied to the second z-stack of images using the following steps: 1. Find the most focused plane out of the 40 z-slices. (Focus score: Vollath’s F4). 2. Extract ±10 z-slices around the most focused plane. 3. Apply a Gaussian filter to each slice as a mild denoising step. 4. Make 64 x 64 denoised image patches and find the most focused plane at each image patch. (Focus score: Tenengrad). This step creates a raw focus index map. 5. Find an abrupt change of focus index and replace them with nearest neighbors to generate an outlier-removed focus index map. 6. Apply a median filter to the outlier-removed focus index map to generate a filtered focus index map. 7. Upsample the filtered focus index map with linear interpolation to the full resolution to generate an upsampled focus index map. 8. Sample the original z-stack with the upsampled focus index map.

[0123] In various embodiments, the first z-stack images is sampled with the upsampled focus map to generate an unstained fused image. In particular, the upsampled focus index map may be applied to the first z-stack of images. The result of this may be referred to herein as an unstained fused image generated by applying the upsampled focus index map generated by the multi-focus image fusion algorithm to the first z-stack of images.

[0124] Referring now to FIG. 14, in various embodiments, the first z-stack of images is sampled with the upsampled focus index map to generate a first unstained intermediate image 1404. In various embodiments, the first z-stack of images is sampled with a shifted version of the upsampled focus index map to determine a second unstained intermediate image 1406. In various embodiments, a convex combination of the first unstained intermediate image 1404 and the second unstained intermediate image 1406 may be determined to generate an unstained fused image 1407. In various embodiments, the unstained fused image 1407 isAttorney Docket No. GEI-05025 subtracted from the stained fused image 1402 to produce a subtracted image 1408, such as a background-subtracted image.

[0125] In various embodiments, the upsampled focus map, generated by the application of a multi-focus image fusion algorithm, is used to sample the first z-stack / z-stack of unstained images to produce a first unstained intermediate image. In various embodiments, a shifted version of this upsampled focus index map is used to sample the first z-stack / z-stack of unstained images to produce a second unstained intermediate image. For example, the upsampled focus index map with each z-index shifted up or down by a z-index of 1, may be used to sample the first z-stack of images. In various embodiments, a function, such as a round, floor, or ceiling function, is applied to the upsampled focus index map before it is used to sample a z-stack or before the upsampled focus index map is shifted, in order to change any floating-point values in the upsampled focus index map to integer values.

[0126] In various embodiments, a convex combination of the first unstained intermediate image and the second unstained intermediate image is determined by sampling the first z- stack of images using the upsampled focus index map and a shifted version of the upsampled focus index map. In various embodiments, a convex combination of b and c is determined using the formula–y(a) = (1 − a)b + ac, where 0 ≤ a ≤ 1. This formula may be applied to images, where b and c are images, each with multiple pixel points. In various embodiments, an algorithm, such as the random sample consensus (RANSAC) algorithm, is used to determine the convex combination of the first and second intermediate images. For example, multiple convex combinations of the first and the second intermediate images may be determined, using the algorithm, and each may be scored according to a focus metric or focus score. The highest scoring convex combination of the intermediate images may be determined as the best convex combination and used as the unstained fused image.

[0127] In various embodiments, the unstained fused image is subtracted from the stained fused image produced by applying the multi-focus image fusion algorithm. For example, pixel-by-pixel subtraction may be performed, whereby each pixel of the unstained fused image may be subtracted from each corresponding pixel of the stained fused image. A subtracted image, such as a background-subtracted image, may be produced as a result of this subtraction being performed.

[0128] In various embodiments, each subtracted image of a plurality of subtracted images (representing all FOVs of a sample in a particular color channel) may be stitched together based on a FOV of that subtracted image to produce a finalized subtracted focused image. Additionally, or optionally, the resulting background-subtracted images for each colorAttorney Docket No. GEI-05025 channel are combined with background-subtracted images from all other color channels to generate a multicolor, background-subtracted, multi-focus, fused image. FIG. 15 illustrated a resulting multicolor, background subtracted, multi-focus, fused image that can be provided to downstream image processing tasks (e.g., nucleus segmentation, cell membrane segmentation, etc.) and / or presented to a user via a graphical user interface (e.g., a display on the optofluidic instrument).

[0129] As discussed above, images may be captured by the imaging instrument using a ZCYX (or ZCXY) imaging order, in which the instrument images a z-stack of a single FOV in all color channels before moving in X or Y to the next FOV to image the next FOV in all color channels, and so on and so forth until all FOVs are imaged. For each captured FOV of the biological tissue sample, and for each channel, a subtracted image may be produced, as described herein, and it may be stitched together with other of the subtracted images produced for the other FOVs, to produce a finalized subtracted all-in-focus image.

[0130] FIG. 16 is a flowchart 1600 illustrating a method of image fusion, according to embodiments of the present disclosure. At 1602, a first z-stack of images of a biological sample may be received. The first z-stack of images may correspond to a first field of view. The first field of view may comprise a plurality of patches. At 1604, a second z-stack of images of the biological sample may be received. The second z-stack of images may correspond to the first field of view. At 1606, a focus map may be determined based on the second z-stack. The focus map may indicate, for each of a plurality of patches of the first field of view, one of the images of the second z-stack bringing into focus that patch of the first field of view. At 1608, the focus map may be applied to the first z-stack to generate a first fused image. At 1610, the focus map may be applied to the second z-stack to generate a second fused image. At 1612, the first fused image may be subtracted from the second fused image to produce a subtracted image.

[0131] In various embodiments, the first z-stack of images may comprise background images of the biological sample, the second z-stack of images may comprise images of the biological sample stained with at least one stain, and the subtracted image may comprise a background- subtracted image. In various embodiments, the background images of the biological sample that may include nuclear staining, such as a DAPI stain. The first z-stack of images may comprise images of the biological sample that lacks cytoplasmic, membrane, and / or nuclear staining. The first z-stack of images may comprise images of the biological sample stained with a nuclear stain. The nuclear stain may comprise DAPI. The first z-stack of images may comprise images of the biological sample stained solely with DAPI. The second z-stack ofAttorney Docket No. GEI-05025 images may comprise images of the biological sample stained with one or more fluorescent stain. The one or more fluorescent stains may comprise a nuclear stain. The one or more fluorescent stains may comprise one or more cytoplasmic stain. The one or more cytoplasmic stain may comprise at least one of: one or more ribosomal RNA stain, one or more lectin stain, and one or more antibody stain. The first z-stack of images and second z- stack of images may be acquired based on at least one probing cycle of the biological sample in an imaging instrument. The biological sample may comprise at least one cellular structure. The first z-stack of images and the second z-stack of images may be registered to each other based on the at least one cellular structure. The at least one cellular structure may comprise a nucleus. The first z-stack of images and the second z-stack of images each may comprise the nucleus. The registering may comprise registering the nucleus of the first z-stack to the nucleus of the second z-stack. The registering may comprise a scale-invariant feature transform (SIFT). The registering may comprise applying random sampling consensus (RANSAC).

[0132] In various embodiments, determining the focus map may include: determining a first focus metric for each image of the second z-stack; selecting one image of the second z-stack having a highest value of the first focus metric; selecting a set of adjacent images of the second z-stack including the selected image; determining a second focus metric for each of the plurality of patches of each of the selected set of images; and for each patch of the plurality of patches, selecting one of the selected set of images having a highest value of the second focus metric for that patch. The first focus metric may comprise Vollath’s F4. The first focus metric may comprise a Tenengrad. The second focus metric may comprise a Tenengrad. The second focus metric may comprise Vollath’s F4. Each patch in the plurality of patches may comprise a same shape. The shape may comprise a square shape. The shape may be about 16x16 pixels to about 128x128 pixels in size. Each selected image of the selected set of images may have an associated index indicating a position within the second z-stack of images. The focus map may indicate the index associated with the selected image for each patch. One or more patch having more than a predetermined disparity between its associated index and the indices associated with two or more neighboring patches may be identified. The index associated with the identified one or more patch may be replaced with an interpolated index based on the indices of the two or more neighboring patches. The predetermined disparity may be 2, 3, 4, 5, 6, 7, 8, 9, or 10. A median filter may be applied to the indices of the focus map. The focus map may be upsampled to pixel resolution. The upsampling may comprises linear interpolation. The set of adjacent images may be of aAttorney Docket No. GEI-05025 predetermined size. The predetermined size may be 2 images to 31 images. The predetermined size may be 21 images. A denoising filter may be applied to each image of the selected set of images. Applying the focus map to the first z-stack of images may comprise: sampling the first z-stack of images according to the focus map to generate a first intermediate image; generating a shifted focus map from the focus map; sampling the first z- stack of images according to the shifted focus map to generate a second intermediate image; and determining a convex combination of the first intermediate image and the second intermediate image to generate the first fused image. Determining the convex combination may comprise a random sampling consensus. The subtracted image may be stitched together with one or more additional subtracted images based on the field of view and a color channel of the subtracted image.

[0133] FIG. 17 is a flowchart 1700 illustrating a method of image fusion, according to embodiments of the present disclosure. At 1702, a z-stack of images of a biological sample may be received. The z-stack of images may correspond to a field of view. At 1704, a focus map may be determined based on the z-stack of images, wherein determining the focus map may include: determining a focus metric for each image of the z-stack of images; selecting one image of the z-stack having a highest value of the focus metric; selecting a set of adjacent images of the z-stack including the selected image; dividing each image in the selected set of images into a plurality of patches; determining a second focus metric for each of the plurality of patches of each of the selected set of images; and for each patch of the plurality of patches, selecting one of the selected set of images having a highest value of the second focus metric for that patch. At 1706, the focus map may be applied to the z-stack to generate a fused image.

[0134] The z-stack of images may comprise images of the biological sample stained with one or more fluorescent stain. The one or more fluorescent stains may comprise a nuclear stain. The one or more fluorescent stains may comprise one or more cytoplasmic stain. The one or more cytoplasmic stain may comprise at least one of: one or more ribosomal RNA stain, one or more lectin stain, and one or more antibody stain. In some embodiments, the first focus metric is Vollath’s F4 and the second focus metric is Tenengrad. In other embodiments, the first focus metric is Tenengrad and the second focus metric is Vollath’s F4. Each patch in the plurality of patches may have a same shape. The shape may comprise a square shape. The shape may be about 16x16 pixels to about 128x128 pixels in size. Each selected image of the set of selected images may have an associated index indicating a position within the z-stack of images. The focus map may indicate the index associated with the selected image forAttorney Docket No. GEI-05025 each patch. One or more patch having more than a predetermined disparity between its associated index and the indices associated with two or more neighboring patches may be identified. The index associated with the identified one or more patch may be replaced with an interpolated index based on the indices of the two or more neighboring patches. The predetermined disparity may be 2, 3, 4, 5, 6, 7, 8, 9, or 10. A median filter may be applied to the indices of the focus map. The focus map may be upsampled to pixel resolution. The upsampling may comprise linear interpolation. The set of adjacent images may be of a predetermined size. The predetermined size may be 2 images to 31 images. The predetermined size may be 21 images. A denoising filter may be applied to each image of the selected set of images. Detection of Tissue Detachment using Focus Scores

[0135] Tissue detachment is a common error mode during in situ analysis (specifically, during sample preparation) where part of the tissue detaches from the substrate (e.g., glass slide), causing lower data quality or complete loss of usable data. Tissue detachment impacts detection, registration, decoding, and all downstream analyses during in situ analysis. Moreover, presenting a user with feedback and / or a visualization of tissue detachment may better inform a user of the quality of the sample (and subsequent analysis) prior to starting a run on an in situ analysis instrument (e.g., an optofluidic instrument).

[0136] FIG. 18A depicts a cross-sectional image 1800 of a mouse ileum with at least one region 1802 exhibiting tissue detachment from a substrate and at least one region 1804 exhibiting tissue folding. In particular, FIG. 18A shows a composite image having a DAPI stain image and one or more of: at least one cytoplasmic stain image (e.g., using 18S RNA stain) and a membrane stain image (e.g., using an antibody stain targeting one or more targets on a cell membrane). FIGS. 18B-18E depict visualizations of the specific detached and folded regions with detected transcripts overlaid confirming detachment or folding. In particular, FIG. 18B illustrates a zoomed-in view of 1802 in addition to a depth profile view showing the detected DAPI signal through the z-direction. FIG. 18C further illustrates detected RNA transcripts (multi-colored spots) on top of the DAPI signal. As shown in FIG. 18C, detected transcripts throughout the thickness suddenly disappear in the detached region (illustrating a complete loss of data for that detached region). FIG. 18D illustrates a zoomed- in view of 1804 in addition to a depth profile view showing the detected DAPI signal through the z-direction. FIG. 18E further illustrates detected RNA transcripts (multi-colored spots) on top of the DAPI signal. As shown in FIG. 18E, the detected transcripts throughout theAttorney Docket No. GEI-05025 thickness exhibit a wavy profile in the folded tissue region (illustrating low quality data for that folded region).

[0137] In various embodiments, sharpness (e.g., focus score) increases towards the top of the Z-stack for detached and / or folded areas of tissue. Because of this increase in sharpness towards the top of the z-stack of images, the detached and / or folded areas are assigned focus indices towards the top of the z-stack of images. For example, if a z-stack of images includes 40 images taken with 0.75um spacing, detached and / or folded regions of tissue may be assigned z-indices of 25 to 40 (meaning the detached portion is in focus at the 25thslice, 26thslice, 27thslice, 28thslice, 29thslice, 30thslice,,…, 39thslice, 40thslice). In various embodiments, when a patch of an FOV is assigned a z-index that is above a predetermined threshold, the patch is determined to be a detached and / or folded region of tissue on a substrate. In some embodiments, a median value of z-indices is determined across all patches of the FOV and the predetermined threshold is set based on the median. In some embodiments, a mode value of z-indices is determined across all patches of the FOV and the predetermined threshold is set based on the mode. In some embodiments, a minimum value of z-indices is determined across all patches of the FOV and the predetermined threshold is set based on the minimum. In some embodiments, a maximum value of z-indices is determined across all patches of the FOV and the predetermined threshold is set based on the maximum. In some embodiments, the predetermined threshold is set as a fixed value above the median (e.g., 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, etc. slices above the median slice).

[0138] FIGS. 19A-19B depict plots of focus score across the entire image of FIG. 18A. In particular, FIG. 19A shows an exemplary analysis where a focus score is not reliable in empty regions around the FOV (i.e., regions without tissue). FIG. 19B illustrates a histogram of z-slice index across the entire FOV including the z-slice indices of the empty regions.

[0139] FIGS. 20A-20D depict a process of removing empty regions from the analysis of focus scores. In various embodiments, pixel intensity is used to determine empty regions. In various embodiments, low pixel intensity is associated with empty regions and, thus, patches having a low maximum pixel intensity are filtered out (i.e., removed) from the data set to generate a filtered image representing only the regions of the image having tissue. FIG. 20A illustrates the effects of removing empty regions from the z-slice index data such that only z- slice index data is retained where tissue is present. FIG. 20B illustrates a top-down view of FIG. 20A. FIG. 20C illustrates a histogram of maximum pixel intensity on a per patchAttorney Docket No. GEI-05025 within the FOV. The vertical red line represents a threshold below which all patches will be removed from the dataset (e.g., patches with a maximum pixel intensity below the red line are associated with empty regions). In various embodiments, the threshold is determined empirically by determining maximum pixel intensity of patches definitively having tissue. FIG. 20D illustrates another top-down view of FIG. 20A with the dark color assigned to the empty regions removed.

[0140] FIGS. 21A-21B depict the difference between plots of focus score including the empty regions (FIG. 21A) and excluding the empty regions (FIG. 21B). As shown by comparing FIGS. 21A and 21B, the histogram of patches in-focus z-index in FIG. 21B has significantly fewer instances of high z-indices due to the removal of the empty regions from the analysis.

[0141] FIGS. 22A-22B depict the effects of removing empty regions on the analysis of focus scores in the folded region 1804 of the image 1800 of FIG. 18A. In particular, FIG. 22A illustrates a first FOV of image 1800 that includes the folded region 1804. As shown in FIG. 22B, filtering out empty patches results in fewer instances of high z-indices in addition to more instances of lower z-indices.

[0142] FIGS. 23A-23B depict the effects of removing empty regions on the analysis of focus scores in the detached region 1802 of the image 1800 of FIG. 18A. In particular, FIG. 23A illustrates a second FOV of image 1800 that includes the detached region 1802. As shown in FIG. 23B, filtering out empty patches results in significantly fewer instances of high z- indices.

[0143] FIG. 24 depicts histogram plots of patches in-focus z-index per FOV. As illustrated in FIG. 24, the FOVs having larger frequencies of higher z-indices (e.g., z-indices above 25 in a 40 slice z-stack).

[0144] FIG. 25 depicts a panel provided on an instrument for presenting a visualization of tissue detachment to a user. As shown in FIG. 25, the panel simplifies the presentation of the image of the tissue by providing a colored box per FOV indicating how detached the tissue is in that particular FOV. In various embodiments, a detachment score is determined per FOV. In various embodiments, the detachment score is determined based on a maximum z-index of the patches in the FOV. In various embodiments, the detachment score is determined based on a median or average of the z-indices of the patches in the FOV. For example, darker colored regions represent a higher detachment score and thus indicate more likelihood of detached tissue in that FOV. In various embodiments, a user is presented with an option to exclude one or more FOVs from in situ analysis. In various embodiments, the user selectsAttorney Docket No. GEI-05025 the one or more FOVs from within the panel (e.g., by touching the FOV to select it or by clicking the FOV with a cursor). In some embodiments, an in situ analysis instrument automatically excludes regions having high detachment scores from analysis (e.g., when the detachment score is above a predetermined threshold that causes analyte data to be low quality and / or entirely missed).

[0145] FIGS. 26A-26B depict visualizations of a tissue detachment. In FIG. 26A, a plot of FOVs (not stiched) is shown represents a spatial view of the imaged tissue section divided in 64 x 64 pixel bins. For each bin, the index of its most in-focus Z-slice is determined in a 20 slices range centered around the most in-focus Z-slice for the FOV that bin belongs to. Then, each bin is assigned the distance in Z between its most in-focus Z-slice and the most in-focus Z-slice for the entire FOV, so the values of the heatmap are always in a range of [-10;10], where 0 indicates the bin's in-focus Z-slice coincides with the FOV's z-slice. In some embodiments, when a bin is above a predetermined value, that bin is determined to be detached. In some embodiments, when a bin is assigned a value of 10 (i.e., the top of the search range), the bin is determined to be detached. In some embodiments, any bin with a low maximum intensity (below a certain threshold) is excluded from the plot. FIG. 26B illustrates substantially the same plot as in FIG. 26A, but with no color bar, a different colormap, and all bins having a detachment score of 6 or lower are colored dark blue. In particular, white is not tissue (based on DAPI signal intensity of pixels), dark blue is not detached, and red is detached (or wrinkled / folded / thicker).

[0146] Referring now to FIG. 27, a schematic of an example of a computing node is shown. Computing node 10 is only one example of a suitable computing node and is not intended to suggest any limitation as to the scope of use or functionality of embodiments of the invention described herein. Regardless, computing node 10 is capable of being implemented and / or performing any of the functionality set forth hereinabove.

[0147] In computing node 10 there is a computer system / server 12, which is operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations that may be suitable for use with computer system / server 12 include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputer systems,Attorney Docket No. GEI-05025 mainframe computer systems, and distributed cloud computing environments that include any of the above systems or devices, and the like.

[0148] Computer system / server 12 may be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, and so on that perform particular tasks or implement particular abstract data types. Computer system / server 12 may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media including memory storage devices.

[0149] As shown in FIG. 27, computer system / server 12 in computing node 10 is shown in the form of a general-purpose computing device. The components of computer system / server 12 may include, but are not limited to, one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including system memory 28 to processor 16.

[0150] Bus 18 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

[0151] Computer system / server 12 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by computer system / server 12, and it includes both volatile and non-volatile media, removable and non-removable media.

[0152] System memory 28 can include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Computer system / server 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 can be provided for reading from and writing to a non-removable, non-volatile magnetic media (not shown and typically called a “hard drive”). Although not shown, a magnetic disk drive for reading from and writing to a removable, non-volatile magnetic disk (e.g., a “floppy disk”), and an optical disk drive for reading from or writing to a removable,Attorney Docket No. GEI-05025 non-volatile optical disk such as a CD-ROM, DVD-ROM or other optical media can be provided. In such instances, each can be connected to bus 18 by one or more data media interfaces. As will be further depicted and described below, memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.

[0153] Program / utility 40, having a set (at least one) of program modules 42, may be stored in memory 28 by way of example, and not limitation, as well as an operating system, one or more application programs, other program modules, and program data. Each of the operating system, one or more application programs, other program modules, and program data or some combination thereof, may include an implementation of a networking environment. Program modules 42 generally carry out the functions and / or methodologies of embodiments of the invention as described herein.

[0154] Computer system / server 12 may also communicate with one or more external devices 14 such as a keyboard, a pointing device, a display 24, etc.; one or more devices that enable a user to interact with computer system / server 12; and / or any devices (e.g., network card, modem, etc.) that enable computer system / server 12 to communicate with one or more other computing devices. Such communication can occur via Input / Output (I / O) interfaces 22. Still yet, computer system / server 12 can communicate with one or more networks such as a local area network (LAN), a general wide area network (WAN), and / or a public network (e.g., the Internet) via network adapter 20. As depicted, network adapter 20 communicates with the other components of computer system / server 12 via bus 18. It should be understood that although not shown, other hardware and / or software components could be used in conjunction with computer system / server 12. Examples, include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0155] The present invention may be a system, a method, and / or a computer program product. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention.

[0156] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of moreAttorney Docket No. GEI-05025 specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD- ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber- optic cable), or electrical signals transmitted through a wire.

[0157] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0158] Computer readable program instructions for carrying out operations of the present invention may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet ServiceAttorney Docket No. GEI-05025 Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present invention.

[0159] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0160] These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function / act specified in the flowchart and / or block diagram block or blocks.

[0161] The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0162] The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in theAttorney Docket No. GEI-05025 block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustration, and combinations of blocks in the block diagrams and / or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.

[0163] The descriptions of the various embodiments of the present invention have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

Attorney Docket No. GEI-05025 What is claimed is:

1. A method comprising: receiving a plurality of images of a sample on a substrate, wherein the plurality of images comprises a plurality of fields of view of the sample; for each image in the plurality of images, determining at least one focus score; and based on the focus scores, determining one or more portions of the sample as being detached from the substrate.

2. The method of claim 1, wherein determining the one or more portions of the sample as being detached from the substrate comprises: for each focus score, comparing the focus score to a predetermined threshold; and when the focus score is below the predetermined threshold, indicating that the one or more portions of the sample corresponding to the focus score are detached from the substrate.

3. The method of claim 2, further comprising: when the focus score is above the predetermined threshold, indicating that the one or more portions of the sample corresponding to the focus score are attached to the substrate.

4. The method of any one of claims 1 to 3, wherein determining the at least one focus score for each image in the plurality of images comprises: dividing the image into a plurality of windows; for each window, determining a window focus score.

5. The method of claim 4, wherein each window of the plurality of windows has a size of 10 pixels by 10 pixels to 200 pixels by 200 pixels.

6. The method of claim 5, wherein each window of the plurality of windows has a size of about 25 pixels by 25 pixels to about 100 pixels by about 100 pixels.

7. The method of any one of claims 4 to 6, wherein each window is non-overlapping with adjacent windows.Attorney Docket No. GEI-05025 8. The method of any one of claims 1 to 7, further comprising generating, for display, a map of the sample comprising the determined focus scores.

9. The method of claim 8, further comprising displaying the map to a user.

10. The method of claim 9, further comprising receiving input from the user comprising a selection of one or more regions of the sample to exclude during in situ analysis.

11. The method of any one of claims 1 to 9, further comprising automatically excluding the detached one or more portions of the sample from in situ analysis.

12. The method of any one of claims 1 to 11, wherein the plurality of images comprises a single image for each field of view.

13. The method of any one of claims 1 to 11, wherein the plurality of images comprises a plurality of z-stacks, wherein each z-stack in the plurality of z-stacks represents at least a portion of a volume of the sample.

14. The method of any one of claims 1 to 13, further comprising: for each field of view: determining a detachment metric; when the detachment metric is above a predetermined threshold, indicating that the field of view has sample detachment.

15. The method of claim 14, wherein the detachment metric comprises a percent of pixels in the field of view that are detached.

16. The method of claim 14 or claim 15, further comprising: when a field of view of the plurality of fields of view is indicated as having detachment, excluding the field of view from future imaging cycles.

17. The method of any one of claims 1 to 16, wherein the plurality of images is received from an optics module comprising a camera and an objective.Attorney Docket No. GEI-05025 18. The method of any one of claims 1 to 16, wherein the plurality of images is received from a remote database.

19. The method of any one of claims 1 to 16, wherein the plurality of images is received from a local database.

20. The method of any one of claims 1 to 19, wherein the at least one focus score comprises Tenengrad.

21. The method of any one of claims 1 to 19, wherein the at least one focus score comprises Vollath’s F4.

22. The method of any one of claims 1 to 21, wherein the sample comprises a tissue sample.

23. The method of any one of claims 1 to 21, wherein the sample comprises a hydrogel.

24. The method of any one of claims 1 to 23, wherein the plurality of images comprises dark field images of the sample.

25. The method of any one of claims 1 to 23, wherein the plurality of images comprises fluorescent images of the sample.

26. The method of claim 25, wherein the fluorescent images comprise DAPI images.

27. The method of any one of claims 1 to 23, wherein the plurality of images comprises H&E images.

28. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform the method of any one of claims 1 to 27.

29. A system comprising: an image database; and a computing node comprising a computer readable storage medium having programAttorney Docket No. GEI-05025 instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform a method comprising: receiving, from the image database, a plurality of images of a sample on a substrate, wherein the plurality of images comprises a plurality of fields of view of the sample; for each image in the plurality of images, determining at least one focus score; and based on the focus scores, determining one or more portions of the sample as being detached from the substrate.

30. The system of claim 29, wherein determining the one or more portions of the sample as being detached from the substrate comprises: for each focus score, comparing the focus score to a predetermined threshold; and when the focus score is below the predetermined threshold, indicating that the one or more portions of the sample corresponding to the focus score are detached from the substrate.

31. The system of claim 30, further comprising: when the focus score is above the predetermined threshold, indicating that the one or more portions of the sample corresponding to the focus score are attached to the substrate.

32. The system of any one of claims 29 to 31, wherein determining the at least one focus score for each image in the plurality of images comprises: dividing the image into a plurality of windows; for each window, determining a window focus score; 33. The system of claim 32, wherein each window of the plurality of windows has a size of 10 pixels by 10 pixels to 200 pixels by 200 pixels.

34. The system of claim 33, wherein each window of the plurality of windows has a size of about 25 pixels by 25 pixels to about 100 pixels by about 100 pixels.

35. The system of any one of claims 32 to 34, wherein each window is non-overlapping with adjacent windows.Attorney Docket No. GEI-05025 36. The system of any one of claims 29 to 35, further comprising generating, for display, a map of the sample comprising the determined focus scores.

37. The system of claim 36, further comprising displaying the map to a user.

38. The system of claim 37, further comprising receiving input from the user comprising a selection of one or more regions of the sample to exclude during in situ analysis.

39. The system of any one of claims 1 to 37, further comprising automatically excluding the detached one or more portions of the sample from in situ analysis.

40. The system of any one of claims 1 to 39, wherein the plurality of images comprises a single image for each field of view.

41. The system of any one of claims 1 to 39, wherein the plurality of images comprises a plurality of z-stacks, wherein each z-stack in the plurality of z-stacks represents at least a portion of a volume of the sample.

42. The system of any one of claims 1 to 41, further comprising: for each field of view: determining a detachment metric; when the detachment metric is above a predetermined threshold, indicating that the field of view has sample detachment.

43. The system of claim 42, wherein the detachment metric comprises a percent of pixels in the field of view that are detached.

44. The system of claim 42 or claim 43, further comprising: when a field of view of the plurality of fields of view is indicated as having detachment, excluding the field of view from future imaging cycles.

45. The system of any one of claims 29 to 44, wherein the plurality of images is received from an optics module comprising a camera and an objective.Attorney Docket No. GEI-05025 46. The system of any one of claims 29 to 44, wherein the plurality of images is received from a remote database.

47. The system of any one of claims 29 to 44, wherein the plurality of images is received from a local database.

48. The system of any one of claims 29 to 47, wherein the at least one focus score comprises Tenengrad.

49. The system of any one of claims 29 to 47, wherein the at least one focus score comprises Vollath’s F4.

50. The system of any one of claims 29 to 49, wherein the sample comprises a tissue sample.

51. The system of any one of claims 29 to 49, wherein the sample comprises a hydrogel.

52. The system of any one of claims 29 to 51, wherein the plurality of images comprises dark field images of the sample.

53. The system of any one of claims 29 to 51, wherein the plurality of images comprises fluorescent images of the sample.

54. The system of claim 53, wherein the fluorescent images comprise DAPI images.

55. The system of any one of claims 29 to 51, wherein the plurality of images comprises H&E images.

56. A method comprising: receiving a z-stack of images of a biological sample on a substrate, wherein the z- stack of images corresponds to a field of view; determining a focus map based on the z-stack of images indicating, for each of a plurality of patches of the field of view, one of the images of the z-stack bringing into focus that patch of the field of view;Attorney Docket No. GEI-05025 based on the focus map, determining one or more portions of the sample as being detached from the substrate.

57. The method of claim 56, wherein determining the focus map comprises: determining a focus metric for each image of the z-stack of images; selecting one image of the z-stack having a highest value of the focus metric; selecting a set of adjacent images of the z-stack including the selected image; dividing each image in the selected set of images into the plurality of patches; determining a second focus metric for each of the plurality of patches of each of the selected set of images; and for each patch of the plurality of patches, selecting one of the selected set of images having a highest value of the second focus metric for that patch.

58. The method of claim 57, further comprising, applying the focus map to the z-stack to generate a fused image.

59. The method of claim 57, wherein the first focus metric comprises Vollath’s F4.

60. The method of claim 59, wherein the second focus metric comprises Tenengrad.

61. The method of claim 57, wherein the first focus metric comprises Tenengrad.

62. The method of claim 61, wherein the second focus metric comprises Vollath’s F4.

63. The method of any one of claims 57 to 62, wherein each selected image of the set of selected images has an associated index indicating a position within the z-stack of images, and wherein the focus map indicates the index associated with the selected image for each patch.

64. The method of claim 63, further comprising: identifying one or more patch having more than a predetermined disparity between its associated index and the indices associated with two or more neighboring patches; and replacing the index associated with the identified one or more patch with an interpolated index based on the indices of the two or more neighboring patches.Attorney Docket No. GEI-05025 65. The method of claim 64, wherein the predetermined disparity is 2, 3, 4, 5, 6, 7, 8, 9, or 10.

66. The method of any one of claims 63 to 65, further comprising: applying a median filter to the indices of the focus map.

67. The method of any one of claims 63 to 66, further comprising: upsampling the focus map to pixel resolution.

68. The method of claim 67, wherein said upsampling comprises linear interpolation.

69. The method of any one of claims 63 to 68, wherein determining one or more portions of the sample as being detached from the substrate comprises: determining one or more patches of the focus map having an index that is above a predetermined threshold.

70. The method of claim 69, wherein the predetermined threshold is determined based on a mean index of all patches in the focus map.

71. The method of claim 69, wherein the predetermined threshold is determined based on a minimum or a maximum index in the focus map.

72. The method of any one of claims 57 to 71, wherein the set of adjacent images is of a predetermined size.

73. The method of claim 72, wherein the predetermined size is 2 images to 31 images.

74. The method of claim 73, wherein the predetermined size is 21 images.

75. The method of any one of claims 56 to 74, further comprising: applying a denoising filter to each image of the selected set of images.Attorney Docket No. GEI-05025 76. The method of any one of claims 56 to 75, wherein the z-stack of images comprises images of the biological sample stained with one or more fluorescent stain.

77. The method of claim 76, wherein the one or more fluorescent stains comprises a nuclear stain.

78. The method of claim 76 or claim 77, wherein the one or more fluorescent stains comprises one or more cytoplasmic stain.

79. The method of claim 78, wherein the one or more cytoplasmic stain comprises at least one of: one or more ribosomal RNA stain, one or more lectin stain, and one or more antibody stain.

80. The method of any one of claims 56 to 79, wherein each patch in the plurality of patches comprises a same shape.

81. The method of claim 80, wherein the shape comprises a square shape.

82. The method of claim 80 or claim 81, wherein the shape is about 16x16 pixels to about 128x128 pixels in size.

83. The method of claim 82, wherein the shape is 64x64 pixels in size.

84. The method of any one of claims 56 to 83, wherein the z-stack of images is acquired based on at least one probing cycle of the biological sample in an imaging instrument.

85. The method of claim 84, wherein the imaging instrument is an optofluidic instrument.

86. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform a method according to any one of claims 56-Attorney Docket No. GEI-05025 87. A system comprising: an imaging instrument; and a computing node operatively coupled to the imaging instrument and comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform a method according to any one of claims 56-85.

Images