Method and system for enhanced interference detection and characterization of single particles via three-dimensional image registration - Patents.com

Abstract

Presented herein are systems and methods for analyzing images of target particles bound to a surface of a substrate. In an embodiment, the systems and methods described herein utilize techniques that remove background noise from interference images of small particles, allowing increased accuracy in the detection and / or characterization of individual particles. Such techniques, as described herein, may allow individual particles having a size below about 100 nm (e.g., below about 50 nm, e.g., down to 20 nm) to be accurately detected in the image and, among other things, their characteristic size to be measured, e.g., by utilizing their contrast.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Application No. 63 / 303,888, filed January 27, 2022, and U.S. Provisional Application No. 63 / 345,200, filed May 24, 2022, each of which is incorporated by reference in its entirety herein.

[0002] This application generally relates to a method and system for analyzing images. In an embodiment, the image is a coherent scattering-based image of small particles. The small particles may be extracellular vesicles, exosomes, viruses, virus-like particles, or lipid nanoparticles. Detection of small particles in an image may be used to classify, diagnose, and / or monitor a disease in a patient, or to select a treatment option for the patient. [Background technology]

[0003] The ability to detect biological target molecules is essential for our understanding of both cellular physiology and disease progression, as well as for use in a variety of applications such as early and rapid assessment and diagnosis of disease.

[0004] Thus, a need exists for systems and methods that provide for the detection and characterization of biological molecules, and in particular, for improved detection of small biological particles. Summary of the Invention [Means for solving the problem]

[0005] Described herein are systems and methods for analyzing images of particles (e.g., small particles and target particles described herein) bound on a surface of a substrate. In certain embodiments, the systems and methods may utilize techniques that remove background noise from interference images of particles, which may increase the accuracy of detection and / or characterization of individual particles. Such techniques, which may include affine transformations, may allow individual particles having sizes below about 100 nm (e.g., below about 75 nm, below about 50 nm, below about 25 nm, down to about 20 nm, inclusive of all values ​​and subranges therein) to be accurately detected in the images. In other words, the sizes of individual particles that may be detected may range from about 20 nm to about 100 nm, inclusive of all values ​​and subranges therein. For example, the size of the individual particles may be about 20 nm, about 25 nm, about 30 nm, about 35 nm, about 40 nm, about 45 nm, about 50 nm, about 55 nm, about 60 nm, about 65 nm, about 70 nm, about 75 nm, about 80 nm, about 85 nm, about 90 nm, about 95 nm, or about 100 nm. The size of the individual particles may be measured, for example, using their contrast.

[0006] In an embodiment, the systems and methods described herein utilize a pre-scanned image of an area before the sample is contacted with the sensor surface to remove background from a post-scanned image of the area that is taken of the same or substantially the same area (e.g., within about ±5 microns in the X and Y directions, within about 100 nm in the Z direction, and / or within about ±2 degrees of rotation) after the sensor is contacted with the sample. In this way, features in the post-scanned image that are not altered, such as background gain and loss and substrate roughness and imperfections, can be removed, leaving only features of interest that represent target particles in the sample that have been captured at the sensor surface. Images may be taken using an imaging system that includes an interferometric biosensor, e.g., SP-IRIS (Single Particle Interferometric Reflectance Imaging Sensor).

[0007] Examples of samples that may be placed on the sensor surface and imaged include, but are not limited to, cells, microorganisms, lysed cells, cell extracts, nuclear extracts, components of cells or microorganisms, extracellular fluids, culture media in which cells are cultured, blood, plasma, serum, gastrointestinal secretions, tissue or tumor homogenates, synovial fluid, stool, saliva, sputum, cyst fluid, amniotic fluid, cerebrospinal fluid, peritoneal fluid, lung lavage fluid, semen, lymphatic fluid, tears, and prostatic fluid. In addition, samples may be viral or bacterial samples, samples obtained from environmental sources such as polluted bodies of water, air samples, or soil samples, as well as food industry samples. Target particles that may be detected by imaging include, but are not limited to, extracellular vesicles, exosomes, viruses, virus-like particles, or lipid nanoparticles.

[0008] In an embodiment, the disclosed approach utilizes a three-dimensional registration approach that may allow the pre-scan image to be precisely aligned with the post-scan image. The approach not only considers translation, rotation, etc. in a lateral plane (e.g., parallel to the sensor surface), but also leverages stacks of images collected at different axial distances (e.g., between the sensor surface and the objective lens of the imaging system) to precisely identify and match the pre-scan and post-scan focal planes. In an embodiment, a reference particle is centered on the sensor surface to provide a feature that can be detected in the image, and the axial position is varied and used to identify the focal plane.

[0009] In this manner, the systems and methods described herein remove background features that would otherwise reduce the ability to detect nanoscale particles (e.g., particles having a size below 50 nm) in coherent scattering images, thereby opening the door for imaging applications to previously impractically small particles such as extracellular vesicles having a size of 50 nm or smaller (e.g., exomeres and supermeres), viral vectors for gene delivery (e.g., adeno-associated virus (AAV) vectors), lipid nanoparticles, and the like.

[0010] In one aspect, a method for reducing background in an interference image (e.g., an SP-IRIS image) via three-dimensional (3D) image registration and background correction includes the steps of: (a) receiving and / or accessing, by a processor of a computing device, a pre-scan stack including a plurality of images of a first region of a sensor chip surface, the images of the pre-scan stack having been acquired prior to contacting a sample, the sample including one or more target particles, with the sensor chip surface, each image of the pre-scan stack corresponding to an image of the first region of the sensor chip surface taken at a different axial distance from an objective lens of an imaging system (e.g., an SP-IRIS imaging system); and (b) receiving and / or accessing, by the processor, a post-scan stack including a plurality of images of a second region of the sensor chip surface, the second region being identical to or substantially overlapping with the first region (e.g., within about ±5 microns in X and Y directions, within about 100 nm in Z direction, and / or within about ±2 degrees of rotation of the first region). ), images of the post-scan stack have been acquired after contacting the sensor chip surface with the sample (e.g., thereby capturing at least a portion of one or more target particles on the sample surface), each image of the post-scan stack corresponding to an image of a second region of the sensor chip surface taken at a different axial distance from an objective lens of an imaging system (e.g., an SP-IRIS imaging system); (c) determining (e.g., automatically), by the processor, a pre-scan focal plane position for the pre-scan stack and using the pre-scan focal plane position to acquire a focused pre-scan image from the pre-scan stack (e.g., a particular image of the pre-scan stack identified as being at and / or closest to the focal plane position); (d) determining (e.g., automatically), by the processor, a post-scan focal plane position for the post-scan stack and using the post-scan focal plane position to acquire a focused post-scan image from the post-scan stack (e.g., a particular image of the post-scan stack identified as being at and / or closest to the focal plane position); and (e) determining (e.g., automatically), by the processor, a post-scan focal plane position for the post-scan stack and using the post-scan focal plane position to acquire a focused post-scan image from the post-scan stack (e.g., a particular image of the post-scan stack identified as being at and / or closest to the focal plane position).(e.g., automatically) aligning the focused pre-scan image and the focused post-scan image; and (f) (e.g., following step (e)) using the focused pre-scan image and the focused post-scan image, which are aligned with one another, by a processor to generate one or more final particle images with background removed. In some variations of the method, imaging may be performed in a liquid medium.

[0011] In one embodiment, a first region of the sensor chip surface that is imaged in the pre-scan stack includes a plurality of reference particles, each of which corresponds to (e.g., represents) an individual reference particle, such that at least a portion of the pre-scan images of the pre-scan stack each include one or more reference particle features (e.g., spots).

[0012] In one embodiment, step (c) includes detecting, within each of at least a portion of the images of the pre-scan stack, a set of one or more reference particle features (e.g., localized areas of relatively high or low contrast relative to their surrounding areas (e.g., bright and / or dark spots)), each of which corresponds to (e.g., represents) an individual reference particle, and using the set of reference particle features within the images of the pre-scan stack to determine a pre-scan focal plane position.

[0013] In an embodiment, the method includes: (A) determining, for each of at least a portion of images of a pre-scan stack, one or more focus metrics using the detected set of detected reference particle features for the image, where determining the one or more focus metrics includes: (i) determining a number of reference particle features in each of the portion of the pre-scan images (e.g., counting the number of detected reference particles); (ii) determining, for each of the portion of the pre-scan images, a measure of reference particle contrast (e.g., a statistical measure (e.g., a median, mean, mode, etc.)); and (iii) determining a number of reference particle features in each of the portion of the pre-scan images using the detected set of detected reference particle features for the image. (B) determining a pre-scan focal plane position using one or more focus metrics for each of a portion of the scan images; and (B) determining a pre-scan focal plane position using the one or more focus metrics. In some cases, determining the pre-scan focal plane position further includes determining an axial position within the pre-scan stack that maximizes a particular focus metric, for example by comparing the particular focus metric to a reference value and / or range (e.g., based on known values ​​of the reference particle size).

[0014] In one embodiment, a second region of the sensor chip surface imaged in the post-scan stack includes a plurality of reference particles, and at least some of the reference particles in the first and second regions match (e.g., are the same particle), such that at least some of the images of the post-scan stack each include one or more reference particle features (e.g., spots), each of which corresponds to (e.g., represents) an individual reference particle.

[0015] In one embodiment, step (d) includes detecting, within each of at least a portion of the images of the post-scan stack, a set of one or more reference particle features (e.g., localized areas of relatively high or low contrast relative to their surrounding areas (e.g., bright and / or dark spots)), each of which corresponds to (e.g., represents) an individual reference particle, and using the set of reference particle features within the images of the post-scan stack to determine the post-scan focal plane position.

[0016] In one embodiment, the method includes the steps of: (A) determining, for each of at least a portion of images of the post-scan stack, one or more focus metrics using the detected set of detected reference particle features for the image, where determining the one or more focus metrics includes one or more of: (i) determining a number of reference particle features in each of the portion of the pre-scan images (e.g., counting the number of detected reference particles); (ii) determining, for each of the portion of the pre-scan images, a measure of reference particle contrast (e.g., a statistical measure (e.g., typical, mean, median, mode, etc.)); and (iii) determining, for each of the portion of the pre-scan images, a measure of reference particle size (e.g., a statistical measure (e.g., typical, mean, median, mode, etc.)); and (B) determining a post-scan focal plane position using the one or more focus metrics. In some cases, determining the post-scan focal plane position further includes determining an axial position within the post-scan stack that maximizes a particular focus metric, for example, by comparing the particular focus metric to a reference value and / or range (e.g., based on a known value of a reference particle size).

[0017] In some embodiments, the reference particles are spherical or nearly spherical particles having a refractive index ranging from about 1.2 to about 2.0, including all values ​​and subranges therein. For example, the refractive index may be about 1.2, about 1.3, about 1.4, about 1.5, about 1.6, about 1.7, about 1.8, about 1.9, or about 2.0. In some variations, the refractive index may be less than or about equal to 2.0. In one variation, the refractive index may be about 1.45. When the reference particles are comprised of beads, the beads may be one or more members selected from the group consisting of polystyrene beads, silica beads, latex beads, and polystyrene sulfate beads. The reference particles may have a diameter of about 2.5 microns or less, about 2.25 microns or less, about 2.0 microns or less, about 1.75 microns or less, about 1.5 microns or less, about 1.25 microns or less, about 1.0 microns or less, about 0.75 microns or less, about 0.5 microns or less, about 0.2 microns or less, or about 0.1 microns or less.

[0018] In an embodiment, the first and / or second regions of the sensor chip surface include at least a portion (e.g., the same portion) of a portion, including a spot, that includes one or more target binding agents that bind to a target particle (e.g., bind to one or more markers on the surface of the target particle). In this embodiment, the target particle may be an antibody.

[0019] In certain embodiments, the reference particles comprise one or more molecules on their surface to which one or more target binding agents of the spots also bind.

[0020] In certain embodiments, the spots also contain one or more reference binding agents that bind to the reference particles.

[0021] The number of reference particles in a spot may range from about 25 to about 100, inclusive of all values ​​and subranges therein. For example, the number of reference particles in a spot may be about 25, about 30, about 40, about 45, about 50, about 55, about 60, about 65, about 70, about 75, about 80, about 85, about 90, about 95, or about 100.

[0022] In one embodiment, step (e) includes using the set of detected reference particle features in the focused pre-scanned image and the set of detected reference particle features in the focused post-scanned image (e.g., as fiducial markers) to align the focused pre-scanned image and the focused post-scanned image with each other.

[0023] In some embodiments, the sensor chip surface comprises fiducial markers, and step (e) includes using the fiducial markers for image registration. In some variations, the fiducial markers may be reference particles. In other variations, the fiducial markers may be lithographically defined fiducial markers. Other types of fiducial markers may also be employed.

[0024] In an embodiment, step (c) includes determining a measure of background intensity variation (e.g., standard deviation) for each of at least a portion of the images in the pre-scan stack, and using the measure of background intensity variation to determine the pre-scan focal plane position.

[0025] In an embodiment, step (d) includes determining a measure of background intensity variation (e.g., standard deviation) for each of at least a portion of the images in the post-scan stack, and using the measure of background intensity variation to determine the post-scan focal plane position.

[0026] In an embodiment, step (f) includes subtracting (eg, pixel-by-pixel subtraction) the focused pre-scan image from the focused post-scan image (eg, to generate a final particle image).

[0027] In one embodiment, the method further includes detecting, by the processor, one or more target particle features within at least a portion (e.g., a particular one, e.g., a subset) of the one or more final particle images, each of which corresponds to (e.g., represents) an individual target particle, and using the detected one or more target particles to enumerate and / or characterize (e.g., determine measures of size, weight, etc.) the number of individual target particles bound to the sensor chip surface.

[0028] In certain embodiments, the method further includes receiving and / or accessing, by the processor, a fluorescent image of a third region of the sensor chip surface, the third region substantially overlapping with the second region, and using at least a portion (e.g., a particular one, e.g., a subset) of the fluorescent image and one or more final particle images to quantify and / or characterize a population of target particles that contain a particular biomarker (e.g., at or within their surface).

[0029] In some embodiments, the size of the target particle is about 50 nm or less in diameter. For example, the size of the target particle may be about 45 nm or less, about 40 nm or less, about 35 nm or less, or about 30 nm or less. In some embodiments, the size of the target particle may be about 20 nm to about 50 nm, including all values ​​and subranges therein. For example, the size of the target particle may be about 20 nm, about 25 nm, about 30 nm, about 35 nm, about 40 nm, about 45 nm, or about 50 nm.

[0030] In certain embodiments, the targeted particle comprises an extracellular vesicle (e.g., an exosome, e.g., an exomere and / or a supermere).

[0031] In certain embodiments, the targeted particle comprises a virus and / or a virus-like particle (eg, a lentivirus, such as an adeno-associated virus (AAV)).

[0032] In certain embodiments, the targeted particle comprises a lipid nanoparticle.

[0033] In some embodiments, detection of target particles in the background-subtracted image may be used to classify, diagnose, and / or monitor disease in a patient, or to select treatment options for the patient.

[0034] In one embodiment, the method includes the steps of: prior to contacting the sensor chip surface with a sample, using an imaging system to image the sensor chip surface at a plurality of different axial distances from an objective lens of the imaging system, thereby obtaining an image of a pre-scan stack; contacting the sensor chip surface with a sample, thereby capturing at least a portion of the target particles in the sample at the sensor surface; and using the imaging system to image the sensor chip surface along with a portion of the target particles captured thereon, at a plurality of different axial distances from the objective lens of the imaging system, thereby obtaining an image of a post-scan stack.

[0035] In one embodiment, the sensor chip surface is the top surface of a thin partially transparent layer (eg, an oxide layer) on a base substrate (eg, silicon).

[0036] In an embodiment, the sensor chip surface comprises one or more fiducial markers, which may be reference particles, i.e., lithographically defined fiducial markers (e.g., made of deposited material such as metal and / or dielectric, e.g., etched into the sensor chip surface).

[0037] In one embodiment, the method includes determining a pre-scan focal plane position and / or a post-scan focal plane position using images of the fiducial markers and calculating one or more focus metrics, e.g., by calculating a score, such as a DoG score (Gaussian difference score), and measuring high spatial frequency components based on areas in each image of the pre-scan stack and surrounding one or more of the fiducial markers, and / or calculating one or more focus metrics, e.g., by calculating a score, such as a DoG score, and measuring high spatial frequency components based on areas in each image of the post-scan stack and surrounding one or more of the fiducial markers.

[0038] In one embodiment, the method includes aligning the focused pre-scanned image and the focused post-scanned image using images of the fiducial markers, e.g., by identifying, for each of the one or more fiducial markers in the focused pre-scanned image, a set of coordinates (e.g., representing a center, e.g., one or more vertices, e.g., one or more edges) that locates the fiducial marker, thereby identifying a first set of fiducial coordinates in the focused pre-scanned image; identifying, for each of the one or more fiducial markers in the focused post-scanned image, a set of coordinates (e.g., representing a center, e.g., one or more vertices, e.g., one or more edges) that locates the fiducial marker, thereby identifying a second set of fiducial coordinates in the focused post-scanned image; and determining an in-plane shift using the first and second sets of fiducial markers.

[0039] In an embodiment, step (e) includes identifying a pre-scanned background region in the focused pre-scanned image, identifying a post-scanned background region in the focused post-scanned image, and using the pre-scanned background region and the post-scanned background region to determine a fine in-plane shift. The fine in-plane shift may be a lateral (e.g., XY) plane and / or a fine in-plane rotation, e.g., a rotation in the lateral (e.g., XY) plane.

[0040] In one embodiment, the method includes repeatedly applying a varied 2D in-plane shift and / or rotation to one background region (e.g., a pre-scanned background region) and / or image (e.g., a focused pre-scanned image) relative to another (e.g., a post-scanned background region and / or a focused post-scanned image); subtracting (e.g., subsequently, after the shift) the pre-scanned background region from the post-scanned background region, thereby obtaining a subtracted image; and calculating a measure of variation (e.g., a standard deviation) using the subtracted image (e.g., determining an optimal 2D shift and / or rotation, such as that which minimizes the measure of variation).

[0041] In some embodiments, the method includes identifying a first pre-scanned background region and a first post-scanned background region and using the first pre-scanned background region to determine a fine in-plane shift, e.g., a shift in the lateral (e.g., XY) plane, and identifying a second pre-scanned background region and a second post-scanned background region and using the second pre-scanned background region to determine a fine in-plane rotation, e.g., a rotation in the lateral (e.g., XY) plane. In these embodiments, the second pre-scanned background region may be larger than the first pre-scanned background region, and the second post-scanned background region may be larger than the first post-scanned background region.

[0042] In one embodiment, step (f) includes dividing the imaged first region of the pre-scan stack and the imaged second region of the post-scan stack into a plurality of sub-regions (e.g., quadrants) and determining, for each of the plurality of sub-regions, a corresponding optimal image pair consisting of an optimal pre-scan image and an optimal post-scan image (e.g., thereby taking into account out-of-plane tilt).

[0043] In one embodiment, the step of determining a corresponding optimal image pair for each particular sub-region of the plurality of sub-regions includes the steps of iteratively acquiring inspection image pairs, where each inspection image pair consists of a pre-scan image selected from the pre-scan stack and a post-scan image selected from the post-scan stack, calculating a measure of background variation between the pre-scan image and the post-scan image within the particular sub-region for the image pairs, and selecting the optimal image pair for the particular one based on the calculated measure of background variation.

[0044] In one embodiment, step (f) includes, for each particular sub-region, using the corresponding optimal image pair to generate a corresponding final sub-region image for the particular sub-region.

[0045] A system for removing background from an interference image is also described herein and may include a processor of a computing device and a memory having instructions stored thereon, the instructions, when executed by the processor, causing the processor to perform a method according to any one of the aspects and embodiments described herein.

[0046] In an embodiment, the system comprises an imaging device and / or a sensor chip.

[0047] In one embodiment, the sensor chip surface is the top surface of a thin partially transparent layer (eg, an oxide layer) on a base substrate (eg, silicon).

[0048] Elements (eg, methods) of embodiments involving one aspect of the invention can also be applied to embodiments involving other aspects of the invention, and vice versa. [Brief description of the drawings]

[0049] [Figure 1] FIG. 1 is a schematic diagram illustrating an exemplary method for acquiring a pre-scan stack.

[0050] [Diagram 2] FIG. 2 is a schematic diagram illustrating an exemplary method for acquiring a post-scan stack.

[0051] [Diagram 3] FIG. 3 is a schematic diagram illustrating the method used to acquire images in which a focused pre-scan image and a focused post-scan image are aligned with each other and with background removal.

[0052] [Figure 4] FIG. 4 illustrates an exemplary method for sample analysis.

[0053] [Figure 5A] FIG. 5A depicts an exemplary method for image registration and subtraction. [Figure 5B] FIG. 5B is a flow diagram illustrating the alignment process of FIG. 5A.

[0054] [Figure 6] Figures 6A and 6B show two example DoG focus curves from pre-scan and post-scan image stacks obtained from a reference spot on a region of a substrate, and Figure 6C depicts an example image of the reference spot on a substrate.

[0055] [Figure 7] Figure 7A is an example image of a reference area where pre-scan and post-scan images of the same focal plane have been subtracted without alignment in the XY plane, and Figure 7B is an example image of the same reference spot as Figure 7A where the pre-scan and post-scan images have been roughly aligned using an XY offset determined prior to subtraction.

[0056] [Figure 8] FIG. 8 is an example image of a reference spot and a particle with a mask applied.

[0057] [Figure 9A]FIG. 9A depicts an example image of a chip including an alignment marker as an inverted "L."

[0058] [Figure 9B] FIG. 9B shows the standard deviation of the difference between two example images over a 20×20 pixel search area.

[0059] [Figure 10] 10A and 10B show an exemplary process for rotational alignment between pre-scan and post-scan images. Fig. 10B is a graph of the standard deviation of the difference between the two images depicted in Fig. 10A as one image is rotated over the other.

[0060] [Figure 11] FIG. 11 is an exemplary plot showing the optimization of four quadrants of a pre-scan image as it is scanned along corresponding quadrants in the images of the post-scan stack.

[0061] [Figure 12] FIG. 12 depicts an example image of a chip, including multiple sub-regions, that is used to generate an affine transformation that accounts for magnification and / or scaling due to environmental factors. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0062] Detailed Description Presented herein are methods and systems for analyzing images to remove or minimize background noise in the images, thus improving the visibility of particles (e.g., small particles, target particles). The methods and systems may use a stack of images and an affine transformation that accounts for rotation and translation in the images to remove or reduce background noise in images taken after placement of a sample on the sensor surface. In addition to rotation and translation, the affine transformation may account for magnification and / or scaling factors during the alignment process.

[0063] Particles that can be detected by imaging include, but are not limited to, extracellular vesicles, exosomes, viruses, virus-like particles, or lipid nanoparticles. The particles may be small particles having diameters ranging from about 20 nm to about 100 nm, including all values ​​and subranges therein. For example, the diameter of the small particles may be about 20 nm, about 25 nm, about 30 nm, about 35 nm, about 40 nm, about 45 nm, about 50 nm, about 55 nm, about 60 nm, about 65 nm, about 70 nm, about 75 nm, about 80 nm, about 85 nm, about 90 nm, about 95 nm, or about 100 nm.

[0064] The particles may be contained within various types of samples, such as samples of cells, microorganisms, lysed cells, cell extracts, nuclear extracts, components of cells or microorganisms, extracellular fluids, culture media in which cells are cultured, blood, plasma, serum, gastrointestinal secretions, tissue or tumor homogenates, synovial fluid, stool, saliva, sputum, cyst fluid, amniotic fluid, cerebrospinal fluid, peritoneal fluid, lung lavage fluid, semen, lymph, tears, or prostatic fluid. In addition, the samples may be viral or bacterial samples, samples obtained from environmental sources such as polluted bodies of water, air samples, or soil samples, as well as food industry samples. Detection of particles within an image may be used to classify, diagnose, and / or monitor disease in a patient, or to select treatment options for the patient.

[0065] The disclosed methods and systems may utilize a stack of images, including multiple images taken of the sensor chip surface at different axial positions. In some embodiments, the disclosed methods and systems may acquire images of particles while they are in a liquid medium. As illustrated in FIG. 1, a pre-scan stack 100 may be acquired prior to contacting a particular sample with the surface of the sensor chip. Turning to FIG. 2, after contacting (e.g., incubating) the sample with the surface of the sensor chip, a post-scan stack 200 may be acquired. As described herein, the pre-scan stack and post-scan stack (100, 200) may be used to identify a focused pre-scan image 102 and a focused post-scan image 202.

[0066] In one embodiment, reference particles, such as polystyrene beads, are intentionally attached to the surface of the sensor to create artificial reference features on the surface of the sensor chip. These features can then be detected in images of the pre-scan stack and / or post-scan stack and used to identify the focus position. The reference particles may be bound to the sensor surface in a variety of ways, for example, using antibodies (e.g., IgG, e.g., mouse IgG), biotin, streptavidin, etc. In one embodiment, polystyrene sulfate particles (e.g., OptiBind TM ) may be used as the reference particle. In addition to polystyrene particles or beads, other types of beads that may be used include silica beads or latex beads.

[0067] Turning to FIG. 3, in an embodiment, the focused pre-scan 302 image and the focused post-scan 304 image are mutually aligned to match features in the sample (e.g., lateral) plane as shown in the aligned subtraction image 306. In an embodiment, the alignment is performed using an affine transformation. In an embodiment, the reference particle may also be used as a fiducial mark for image alignment. In an embodiment, other fiducial marks on the sensor chip surface may be used for alignment. In an embodiment, sub-pixel alignment is achieved.

[0068] FIG. 4 is an illustrative embodiment of a method 400 for sample analysis according to various embodiments described herein. In an embodiment, the sensor surface may have one or more spots (e.g., predetermined spots) on the sensor surface. In an embodiment, the predetermined spots may have one or more binding agents (e.g., proteins, antigens, antibodies) for binding to particles (e.g., biological particles, e.g., extracellular vesicles, viruses). As shown in FIG. 4, a pre-scan stack is acquired (402), including multiple images (e.g., at different axial positions) of the sensor chip surface. The pre-scan stack may be acquired prior to incubating the sample on the sensor surface (404). In an embodiment, the pre-scan is performed using a suitable interferometric imaging platform (e.g., for imaging at a sufficiently high resolution and / or magnification) for imaging, for example, single particles having a submicron size (e.g., a diameter less than or equal to 1 micron). In certain embodiments, an interferometric imaging platform may be used, such as a single particle interferometric imaging sensor system suitable for imaging particles having diameters of about 100 nm and below, as described, for example, in PCT Application Nos. PCT / US2017 / 016434, filed February 3, 2017, PCT / US2019 / 034831, filed May 31, 2019, and PCT / US2020 / 016098, filed January 31, 2020, each of which is incorporated herein by reference in their entirety. In certain embodiments, the pre-scan is performed using another imaging platform (e.g., a microscope).

[0069] After obtaining the pre-scan stack, a sample (e.g., a biological sample) containing particles such as extracellular vesicles, viruses, virus-like particles, lipid nanoparticles, and the like to be imaged is then incubated on the surface of the sensor chip surface 404 to capture one or more particles on the surface of the sensor. In an embodiment, the particles are attached to spots (e.g., predetermined spots) on the surface of the sensor using one or more binding agents. In an embodiment, the particles may be labeled with a secondary marker (e.g., a fluorescent dye molecule).

[0070] After sample incubation on the sensor (404), a post-scan stack is acquired (406). In an embodiment, the post-scan stack and the pre-scan image are both acquired using the same imaging device or platform (e.g., as described herein). One or more background processes (408) (e.g., as described herein) are used to process the pre-scan and / or post-scan images. In an embodiment, the background processes may include background subtraction, image registration (e.g., image alignment) (e.g., rotation, translation), image masking, compensation for magnification and / or scaling due to tilt, environmental factors (e.g., temperature and / or humidity), or another image processing method (e.g., as discussed herein).

[0071] Background processing (408) as described herein may be performed automatically, e.g., by a processor, e.g., without any user interaction and / or with substantially limited user interaction (e.g., limited to a click to initiate a final quality control review, etc.). In particular, background processing 408 may include processes described herein that align images of the pre-scan stack with images of the post-scan stack, identify appropriate focal plane locations, not just in-plane, and perform aligned subtraction, thereby allowing specific pre-scan images to be subtracted from specific post-scan images to minimize background intensity gain and loss and enable high sensitivity particle imaging.

[0072] The final particle image obtained via the registered subtraction approach described herein may be further analyzed to detect and characterize targeted particles. Further analysis may include identifying the number, type, and / or size (e.g., diameter, mass), etc., of particles bound to the sensor surface.

[0073] In an embodiment, the imaging is performed using an imaging system based on single particle reflectance imaging sensor (SP-IRIS) imaging technology. SP-IRIS technology, including its use in combination with fluorescence imaging methods for particle detection and characterization, is described in further detail in PCT Application Nos. PCT / US2017 / 016434, filed February 3, 2017, PCT / US2019 / 034831, filed May 31, 2019, and PCT / US2020 / 016098, filed January 31, 2020, each of which is incorporated herein by reference in its entirety.

[0074] A. Image stack In an embodiment, the image processing approach described herein utilizes and processes image stacks. As used herein, the term "stack" when used in reference to an image, such as "image stack", "pre-scan stack", "post-scan stack", etc., refers to multiple images of a sample recorded at different axial distances (of the sample) relative to an imaging lens, such as an objective lens of an imaging system. Thus, in an embodiment, each image of the image stack represents a different offset distance along the axial direction relative to the focal plane of the imaging system. For example, as shown in Figures 1 and 2, an image stack (100, 200) includes multiple image slices, each recorded at and corresponding to a different axial location. Each image slice has spatial coordinates in two dimensions, with pixels representing different spatial locations of the sample in the lateral (e.g., xy) plane, and a single axial (e.g., z-) coordinate or slice index corresponding to a spatial location of the sample along the axial direction.

[0075] In an embodiment, the images of the image stack are recorded at fixed intervals such that the distance between the axial locations at which each image and adjacent images are recorded is the same for all images. For example, an image stack may be taken by placing the sample at a minimum axial location and moving it upward in fixed steps, pausing at each step and recording an image. For example, an image stack may be taken by placing the sample at a maximum axial location and moving it downward in fixed steps, pausing at each step and recording an image. In an embodiment, the axial distance between adjacent images of the image stack is less than about 1 micron. In an embodiment, the axial distance between adjacent images of the image stack is less than about 500 nanometers. In an embodiment, the axial distance between adjacent images of the image stack is less than about 200 nanometers. In an embodiment, the axial distance between adjacent images of the image stack is less than about 100 nanometers. In an embodiment, the axial distance between adjacent images of the image stack is less than about 50 nanometers.

[0076] In an embodiment, the axial distance between adjacent images of the image stack varies between different adjacent pairs of images in the image stack. For example, a first increment (e.g., between steps for taking different images) may be used near the maximum and minimum boundaries of the stack, and a second increment (e.g., a smaller, finer increment) may be used closer to the center of the range of axial distances, e.g., within a certain vicinity of the expected focal plane position. In an embodiment, the axial step between adjacent images of the image stack is determined as a function of the step number, e.g., using a functional form such as a (e.g., quadratic) polynomial.

[0077] As described herein, image stacks may be recorded for sensor chips comprising a substantially planar surface onto which biomolecular capture probes may be spotted and, in turn, used to bind target particles such as extracellular vesicles, viruses, virus-like particles, lipid nanoparticles, and the like. Other features, such as lithographically defined fiducial markers, reference particles such as polystyrene beads, etc., as described herein, may also be present on the surface of the sensor chip.

[0078] In an embodiment, the sensor chip may comprise a SiO2 surface. In an embodiment, the sensor chip may be a substantially single material, such as a glass slide, or other substantially transparent material (e.g., in the visible and / or near infrared spectral regions). In an embodiment, the sensor chip comprises a multi-layer substrate, such as a thin semi-transparent layer on a base substrate. In an embodiment, the semi-transparent layer is or consists of SiO2. In an embodiment, the semi-transparent layer has a thickness of about 1 micron or less. In an embodiment, the semi-transparent layer has a thickness of about 500 nm or less. In an embodiment, the semi-transparent layer has a thickness of about 200 nm or less. In an embodiment, the semi-transparent layer has a thickness of about 100 nm or less. In an embodiment, the base substrate is or consists of silicon (Si).

[0079] In some embodiments, when an image of the sensor chip is recorded, the sensor chip is located at approximately a certain axial distance from the imaging (e.g., objective) lens, for example by a sample holder. In some embodiments, the surface of the sensor chip is approximately planar, but various target particles, biomolecules, and features of interest are physical three-dimensional objects and may be located at various distances above the sensor chip surface. For example, particles such as viruses and exosomes have diameters of about tens to hundreds of nanometers, and therefore their centers may be located above the chip surface. Additionally or alternatively, capture probes and various surface chemistries may also result in target particles being located at a non-negligible height above the sensor chip surface. Additionally or alternatively, when the sensor chip comprises multiple layers, additional, for example buried, layer surfaces may be located at axial positions just below the top surface of the sensor chip.

[0080] Thus, in certain embodiments, depending on the particular positioning of the sensor chip, various features such as the top surface of the sensor chip, various particles located thereon, and any additional (e.g., reflective) lower surfaces may be located at different precise axial positions and thus different offsets from the absolute focal plane of the imaging system. Depending on the precise positioning of, for example, the focal plane relative to the particle of interest, the visibility of the particle may be affected, such that optimization of the sample position relative to the focal plane is desired for detection of small particles. Additionally or alternatively, in certain embodiments, the ability to consistently align images axially when the sensor chip is removed and replaced in the sample holder allows for the processing of multiple images to be used, for example, for background correction, thereby allowing low visibility particles to be detected and quantitatively analyzed.

[0081] Without wishing to be bound by any particular theory, axial positioning relative to the focal plane is particularly important for interferometric imaging modalities where the imaged light intensity is, for example, not simply a function of scattering from the particle, but rather also results from phased interference between the scattered field from the particle and the light reflected from the sensor chip surface. In certain embodiments, small phase differences between the reflected and scattered fields (which may result, for example, from different distances from the focal plane of the imaging system) can lead to significant variations in image intensity.

[0082] Thus, in some embodiments, rather than acquiring a single image at a single axial location, the approaches described herein utilize image stacks, which allow images at multiple axial locations to be acquired and analyzed. Thus, for example, an optimal axial location can be identified, for example, in a processing step and / or after capturing multiple images in a stack, which in some embodiments are analyzed to improve detection sensitivity and accuracy.

[0083] The image stack may be analyzed to perform accurate background correction (e.g., subtraction). In particular, in an embodiment, a pre-scan (image) stack is recorded for the sensor chip prior to contacting it with a sample of interest. In this way, the pre-scan stack provides an image of the sensor chip surface before the particles of interest are bound. In an embodiment, the pre-scan stack is recorded after the sample has been spotted with the capture probe, but immediately prior to contacting the sample, including the particles of interest. In an embodiment, a subsequent post-scan stack is recorded after the sensor chip has been contacted with the sample of interest (e.g., bound to the target particles).

[0084] In some embodiments, the image of the post-scanned stack includes features representative of bound target particles in addition to background noise, which may result from features such as surface roughness and variations due to, for example, spotted capture probe material. Background noise, such as surface roughness and variations due to tip surface preparation, may also be present in the image of the pre-scanned stack, which can be substantially removed, for example, by subtraction or division.

[0085] In some embodiments, the sensor chip's, e.g., axial and / or lateral position may be changed between the assemblages of the pre-scan and post-scan stacks. For example, after the pre-scan stack is recorded, the chip may be removed from the imaging device so that it can be spotted with a sample, subjected to various processing steps, rinsed and dried, and replaced in the imaging device for post-scan imaging. Thus, in some embodiments, the pre-scan and post-scan image stacks are axially aligned and / or aligned in the lateral plane, allowing for accurate background correction.

[0086] B. Image stack alignment and focal plane position determination In some embodiments, the pre-scan image stack and the post-scan image stack are aligned to allow one or more particular post-scan images to be matched to corresponding pre-scan images recorded at substantially the same axial location. For example, in some embodiments, even if the same nominal z-position of the sample stage is used to collect the pre-scan stack and the post-scan stack, variations in stage accuracy and positioning of the sensor tip within the sample holder may produce an axial offset (e.g., a fixed offset) between the images of the pre-scan stack and those of the post-scan stack. For example, the pre-scan stack and the post-scan stack may each include 10 images, but with, for example, a fixed two-image offset, the actual axial position of the first post-scan image may match most closely with that of the third pre-scan image, the axial position of the second post-scan image may match most closely with that of the fourth pre-scan image, and so on. Thus, in one embodiment, the pre-scan and post-scan stacks are aligned such that the particular images from each stack that match most closely in terms of the axial position at which they were recorded can be compared to each other, e.g., for purposes of background correction.

[0087] In an embodiment, aligning the pre-scan stack and the post-scan stack includes determining one or more focus metrics for each image in the pre-scan stack and / or the post-scan stack. For example, in an embodiment, a focal plane metric may be determined for a particular image of the image stack based on one or more detected features in the image.

[0088] For example, in an embodiment, reference particles, such as polystyrene beads, are placed on the sensor chip surface before it is contacted with the sample of interest to provide a feature that can be used to calculate a focal plane metric. For example, in an embodiment, the focal plane metric is the number of particles in a particular area (e.g., the reference area). In an embodiment, the focal plane metric is a measure of a particular contrast of the reference. In an embodiment, the measure of the reference particle contrast may be a statistical measure, such as, for example, average contrast, median contrast, mode, etc., calculated based on measurements of a plurality of (e.g., 10 or more, 100 or more, 500 or more) reference particles located in a particular area (e.g., the reference area). In an embodiment, the focal plane metric is a measure of the reference particle size. In an embodiment, the measure of the reference particle size may be a statistical measure, such as, for example, average size, median size, mode, etc., calculated based on measurements of a plurality of (e.g., 10 or more, 100 or more, 500 or more) reference particles located in a particular area (e.g., the reference area).

[0089] Additionally or alternatively, in an embodiment, the sensor chip may include fiducial markers, such as lithographically fabricated (or otherwise micro / nano-fabricated) markers, examples of which are shown in FIGS. 6C, 7A, 7B, and 9A. The fabricated markers may be composed of deposited materials, such as metal or dielectric, and / or etched into the surface of the sensor chip. In an embodiment, the presence of the fabricated markers allows a focal plane metric to be calculated for a particular image through analysis of one or more portions (e.g., sub-regions) of the particular image that include one or more fiducial markers. For example, in an embodiment, a numerical score representing a measure of spatial frequency content may be calculated for a region of the image. For example, in an embodiment, a difference of Gaussians (DoG) score may be calculated for a particular image by generating two smeared versions of a reference region of a particular image, e.g., by convolving with differently sized (i.e., having different standard deviations) Gaussian functions and subtracting one from the other in a pixel-by-pixel manner to generate a DoG image. The present DoG approach emphasizes high spatial frequency edges, and the intensity values ​​of each pixel can be summed over a reference region to generate a single number as the DoG score for the image. Other approaches can also be used to score images based on their spatial frequency content within a particular reference region (e.g., based on Fourier transform calculations and the like).

[0090] In an embodiment, the focus metrics calculated for the images of the pre-scan stack and the post-scan stack can be used to align (e.g., determine offsets) the pre-scan image stack and the post-scan image stack by analyzing focus metric curves, for example, as shown in FIGS. 6A and 6B. For example, in an embodiment, a pre-scan focal plane location can be determined from the focus metrics calculated for the pre-scan stack. The pre-scan focal plane location can be used to identify a focused pre-scan image. As used herein, the term "pre-scan focal plane location" refers to a particular image slice index and / or axial (e.g., z-) coordinate in the pre-scan stack that is determined, for example, by a processor, to meet certain target criteria and accordingly represent a properly focused image location. For example, as shown in FIGS. 6A and 6B, the pre-scan focal plane location is determined by finding the peak DoG score and incrementing the slice index by one. A focused pre-scan image is an image at the pre-scan focal plane location. Similarly, a post-scan focal plane location and a focused post-scan image can be similarly identified. As shown in FIG. 6A, the pre-scan focal plane position 602 may match the post-scan 604 focal plane position, or they may be different, as shown in FIG. 6B.

[0091] In an embodiment, a pre-scan image may be matched to a post-scan image by calculating a comparison metric based on a pairwise comparison of a particular pre-scan image and a particular post-scan image. For example, in an embodiment, an image pair including a particular pre-scan image and a particular post-scan image is acquired, the two images are subtracted from each other, and a standard deviation is calculated. In an embodiment, matching sub-regions (e.g., representing the same physical area on the sensor chip) are identified, and the subtraction is performed within the matching sub-regions. A standard deviation may then be calculated for the particular sub-regions, e.g., rather than the entire image. These sub-regions may be identified, e.g., to reflect background variations. Additionally or alternatively, as described in more detail herein, the sub-regions may also be used to account for, e.g., the relative tilt of the sensor chip between the times the pre-scan and post-scan stacks are recorded. Various pairs of images may be compared to search for an optimal pair that minimizes the background standard deviation. In an embodiment, this approach may be used to adjust for additional offsets due to, e.g., tilt, as described in Example 2 below.

[0092] In addition to rotation and translation, the pre-scanned and post-scanned images may be aligned using an affine transformation that accounts for magnification and / or scaling factors. As discussed above, environmental factors such as temperature and humidity may affect the images during the alignment process. Thus, in some embodiments, a local translation shift in the xy plane between sub-regions of the pre-scanned and post-scanned images may be determined. The local translation shift may be determined by the steps described with respect to masking and fine xy alignment in Example 2 below. Next, multiple sub-regions may be analyzed for local translation shift to identify those with acceptable results. Two to twelve sub-regions may be analyzed for acceptable results. For example, two sub-regions, three sub-regions, four sub-regions, five sub-regions, six sub-regions, seven sub-regions, eight sub-regions, nine sub-regions, ten sub-regions, eleven sub-regions, or twelve sub-regions may be analyzed. An acceptable result may be, for example, when the standard deviation of the difference between sub-regions in the pre-scan and post-scan images is a minimum standard deviation (e.g., about 0.2% or less contrast percent difference). The number of acceptable sub-regions from which local translation shift information may be obtained and used to fit an affine transformation may range from 2 to 5. For example, the number of acceptable sub-regions from which local translation shift information may be obtained and used to fit an affine transformation may be 2, 3, 4, or 5. In some embodiments, the number of acceptable sub-regions from which local translation shift information may be obtained and used to fit an affine transformation may be 3. In other embodiments, the number of acceptable sub-regions from which local translation shift information may be obtained and used to fit an affine transformation may be 5. For example, as shown in FIG. 12, 12 sub-regions 1200 on a chip 1204 may be analyzed for local xy translations, as indicated by arrows 1202. Information from the five allowable sub-regions 1200a, 1200b, 1200c, 1200d, and 1200e may then be used in an affine transformation, which takes into account the scaling factor.The use of local translation shift information from five sub-regions can be beneficial when the magnification change is about 0.5% to about 1.0%. In some embodiments, the use of local translation shift information from five sub-regions can be beneficial when imaging particles smaller than about 40 nm.

[0093] C. In-plane image alignment In some embodiments, the images of the pre-scan stack and the images of the post-scan stack are also registered relative to one another in the lateral (e.g., xy) plane. In some embodiments, a particular pre-scan image and a particular post-scan image are analyzed together to determine a transformation to apply to align the images of the pre-scan stack and the post-scan stack. For example, in some embodiments, once the focused pre-scan image and the focused post-scan image have been acquired (e.g., as described herein), they are analyzed to calculate in-plane shifts and rotations.

[0094] In an embodiment, a coarse in-plane shift (e.g., in the xy plane) is determined, for example, by using fiducial particles and / or fiducial markers on the chip surface. In an embodiment, for example, the coordinates of the fiducial particles and / or fiducial markers may be identified in the focused pre-scanned image and the focused post-scanned image and matched to each other to determine the in-plane shift. For example, matching fiducial markers may be identified in the focused pre-scanned image and the focused post-scanned image, and the coordinates of the centroid, apex, edge, etc. may be aligned to calculate the in-plane (e.g., xy) shift. An exemplary coarse xy alignment result is shown in Figures 7A and 7B and described in more detail in Example 2.

[0095] In an embodiment, fine in-plane alignment may also be performed, for example, following the coarse alignment. The fine alignment may be performed by identifying a first background region in the focused pre-scanned image in addition to a second matching background region in the focused post-scanned image (e.g., corresponding to a substantially identical physical region on the sensor chip surface). The first and second background regions may be identified by locating matching regions (e.g., rectangular sub-regions) near the fiducial marker that appear in both the focused pre-scanned image and the focused post-scanned image. In an embodiment, the first and second background regions are located within the boundary of the fiducial marker. In an embodiment, the first and second background regions are located outside the boundary of the fiducial marker.

[0096] In an embodiment, a first background region is subtracted from a second background region, and a measure of variation across the resulting subtracted image, such as the standard deviation of pixel intensities, is calculated. In an embodiment, a mask is applied to remove particles and / or fiducial markers from the first and second background regions, e.g., to measure primarily pure background variation (e.g., not due to the presence of particles). In an embodiment, this process is repeated as a shift in the x and / or y directions (e.g., small, e.g., about 10 or less or 50 or less pixels) is applied to one image versus the other. As illustrated in FIG. 9B, an optimal shift can be identified as the shift that produces the smallest standard deviation (or other measure of variation).

[0097] In an embodiment, in addition or alternatively, the rotational shift is determined using the focused pre-scan image and the focused post-scan image. For example, in an embodiment, the focused pre-scan image is subtracted from the focused post-scan image, and a measure of variation across the resulting subtracted image, such as the standard deviation of pixel intensity, is calculated. In an embodiment, a mask is applied to remove particles and / or fiducial markers from the focused pre-scan image and the focused post-scan image, e.g., to measure primarily pure background variation (e.g., not due to the presence of particles). In an embodiment, this process is repeated as one image is rotated relative to the other. As illustrated in FIG. 10B, the optimal shift can be identified as the shift that produces the smallest standard deviation (or other measure of variation). In an embodiment, the entire or substantially the majority of the focused pre-scan image and the focused post-scan image are used to perform the subtraction and rotational shift determination. In an embodiment, a small sub-region is used (e.g., similar to the approach described above for fine xy alignment).

[0098] In an embodiment, the pre-scan stack and the post-scan stack may additionally or alternatively be used to account for tilt of the sensor chip during imaging. For example, in an embodiment, the image region may be divided into multiple sub-regions, such as, but not limited to, quadrants. As illustrated in FIG. 11, in an embodiment, each sub-region (e.g., quadrant) of the pre-scan image may be compared (e.g., in a pairwise manner as described herein, e.g., in Section B) with a matching sub-region (e.g., quadrant) of one or more post-scan images to determine a sub-region-specific offset, which may vary across different sub-regions due to, e.g., tilt.

[0099] D. Background subtraction and final particle image Thus, various image registration and alignment steps may be performed in various combinations, as described herein, to align the pre-scan and post-scan images in both the axial and in-plane (e.g., lateral) directions. With this precise three-dimensional (e.g., including axial position) alignment, matching pre-scan and post-scan images or portions thereof (e.g., subregions, such as to account for tilt, as described herein) may be identified and subtracted from each other to generate one or more final particle images. For example, in an embodiment, a single particle image may be generated from a single pre-scan and post-scan image of an image stack. In an embodiment, each image of at least a portion of the post-scan stack is matched to a pre-scan image of the pre-scan stack and used to determine multiple final particle images (e.g., at different levels of defocusing).

[0100] In an embodiment, subtraction may include weighting one or both images and / or dividing or subtracting it from the other, e.g., a reference image, to account for, e.g., dark current and / or illumination source variations. In an embodiment, division (e.g., of a post-scan image by a pre-scan image) may be used in addition to or as an alternative to subtraction.

[0101] F. Working Example Example 1: Exemplary extracellular vesicle detection using the ExoView platform This example demonstrates the detection of extracellular vesicle particles utilizing an embodiment of a SP-IRIS-based imaging device and an embodiment of the image processing approach described herein.

[0102] Introduction The ExoView platform allows researchers to capture extracellular vesicles (EVs) using surface antigen-specific antibodies (e.g., tetraspanins) and enumerate individual particles using two imaging techniques: single particle interferometric reflectance imaging sensor (SP-IRIS) imaging and fluorescence imaging. SP-IRIS imaging using the ExoView platform typically has a lower size limit of detection of EVs of about 50 nm. The development of a technique capable of measuring biological particles as small as 20 nm would allow the analysis of recently discovered exomere and supermere-EV subsets that are less than 50 nm in diameter. In this example, using the R200 ExoView platform, improvements engineered to the SP-IRIS technique, according to certain embodiments described herein, enabled the detection of EVs as small as 20 nm in diameter. When combined with fluorescence imaging, which does not have a size limit of detection, new insights can be discovered in the surface and cargo expression patterns of different size populations of EVs.

[0103] method Human cell culture medium was incubated overnight on the chip with printed CD9, CD63, and CD81 antibody capture probes. Immobilized EVs were fixed, permeabilized, and stained with a cocktail of fluorescent antibodies (CD9, CD63, CD81, Syntenin, ALIX, TSG101). The chip was scanned on an ExoView R200 platform, and image analysis was performed using ExoView Analyzer 3.1.

[0104] result An EV size histogram from 30 to 200 nm is presented, showing the modal size below 50 nm. When EVs are sorted into containers based on their size, clearly distinct four-color expression profiles of EV markers (CD9, CD63, CD81, Syntenin, ALIX, TSG101) are observed between size groups.

[0105] Summary / Conclusion In recent years, knowledge of the size and functional diversity of EVs has grown dramatically. Without wishing to be bound by any particular theory, few would have anticipated that particles as small as 25 nm could have biologically relevant effects on the human body. Using these findings, developing and utilizing techniques that can reliably measure phenotypes of EVs as small as 20 nm provides important functionality for the EV field.

[0106] Example 2: Exemplary 3D Image Registration and Subtraction Approach This example demonstrates an approach for image alignment and subtraction that utilizes lithographically defined fiducial markers, and compares performance to an alternative embodiment in which deposited beads are used.

[0107] Sample workflow FIG. 5A shows an exemplary workflow 500 used herein. As illustrated in FIG. 5A, a sensor chip may include a plurality of probes (e.g., antibody capture probes) attached to the sensor chip surface. The probes may be arranged into a plurality of spots, with each spot having a particular type of probe, for example, in a microarray format. The capture probes may be used to immobilize extracellular vesicles (EVs) on the surface of the chip.

[0108] Prior to contacting the surface of the sensor chip with a sample of interest, a pre-scan stack was acquired (502). The pre-scan stack was obtained as illustrated in FIG. 1 by imaging the sensor chip surface as the chip was moved axially (e.g., in a step-by-step manner) such that each image of the pre-scan stack was taken at a different distance from the objective lens of the imaging system. After pre-scanning, a biological sample containing particles (e.g., extracellular vesicles) was incubated (504) on the sensor chip to capture target particles on the surface of the sensor chip. In some embodiments, the incubation may include various processing steps, including, for example, labeling with secondary probes (e.g., fluorophores) and fixation and permeabilization steps (e.g., providing access to biomolecular cargo within particles such as extracellular vesicles, lipid nanoparticles, and viruses (e.g., encapsulated viruses)).

[0109] After incubation, a post-scan stack was acquired (506). The post-scan stack was acquired in a manner similar to the manner in which the pre-scan stack was acquired, for example, as illustrated in FIG. 2. An image registration and subtraction process was used to match focal plane positions in the pre-scan and post-scan stacks, identify and align the focused images, perform adjustments to account for tip tilt, and finally produce a final particle image, as described in more detail below. The final particle image was then analyzed (510). As shown in 510, the analysis may include the generation of a heat map, which shows the number of particles (e.g., EVs) immobilized on the printed spots. The analysis may further include the determination of particle size, shape, and type.

[0110] 3D Image Alignment Process The sensor chips used in this example were equipped with multiple fiducial fiducial markers on their surfaces, as shown, for example, in Figures 6C, 7A and 7B, and 9 A. The presence of the fiducial markers in the pre-scan stack and post-scan stack images was used in combination with a series of image processing steps to perform a registered subtraction to obtain a final particle image in which, among other things, background noise (e.g., gain and loss) due to sensor chip surface roughness was subtracted and minimized, allowing for improved particle detection.

[0111] FIG. 5B shows a flow diagram illustrating registered image subtraction as performed in this example, according to an embodiment described herein.

[0112] 1. Aligning Z-stacks using DoG curves over reference regions In a first step 552, the pre-scan stack and the post-scan stack are aligned. In particular, each image (e.g., slice) in the image stack is acquired at a different axial distance from the microscope objective. For convenience, in this embodiment, the XY plane in three-dimensional (3D) space is considered to be aligned with the surface of the sensor chip, while movements perpendicular to the plane are considered to be movements along the "Z-axis". To acquire an image stack, an image (e.g., slice) in the stack is taken at a first axial location along the Z-axis, and then the imaging device is adjusted (e.g., mechanically) to a second position to acquire a second image (e.g., second slice) in a second focal plane. The adjustment is made such that each image (e.g., slice) of the stack represents a different position along the Z-axis. This process can be repeated as many times as necessary to generate a series of images (e.g., slices) along the Z-axis, thus generating a Z-stack of images. In an embodiment, the distance between adjacent focal planes is substantially uniform.

[0113] To account for small offsets between the absolute z-position of the sensor chip before and after incubation, the pre-scan and post-scan image stacks are aligned to determine the pre-scan focal plane position that corresponds to a particular slice number in the pre-scan stack, and the post-scan focal plane position that corresponds to a particular slice number in the post-scan stack.

[0114] In particular, in this exemplary processing approach, a reference region was identified surrounding a particular fiducial marker image in the pre-scan stack, and a difference of Gaussian (DoG) score was calculated for each image of the pre-scan stack using the reference region. This was repeated for the post-scan image stack as well. Figures 6A and 6B show two exemplary DoG focus curves from a pre-scan image stack and a post-scan image stack (e.g., Figure 6C) obtained from a reference spot on an area of ​​the substrate. Figure 6C is an exemplary image of a reference spot on a substrate, where the bright, inverted "L" shaped area is the reference spot. In an embodiment, the images of the Z-stack are non-saturated images (e.g., 25% full good exposure). In an embodiment, there are few particles on the edge of the reference spot (e.g., on the inverted "L" shaped outline shown in Figure 6C).

[0115] The DoG focal curves were used to identify a particular slice of the prescan stack as the focused prescan image and a particular slice of the postscan stack as the focused postscan image.

[0116] 2. Coarse local matching of reference regions After alignment of the Z-stack images, a coarse local alignment of the fiducial area on the surface of the chip was performed using the focused pre-scan image and the focused post-scan image (554). FIG. 7A is an example image of the fiducial area where pre-scan and post-scan images of the same focal plane have been subtracted without alignment in the XY plane. After the coarse XY offset is found, for example, by identifying the location of the fiducial fiducial, an XY shift can be applied to the image prior to subtraction to coarsely align the images. FIG. 7B is an example image of the same fiducial spot as FIG. 7A where the pre-scan and post-scan images have been coarsely aligned using the determined XY offset prior to subtraction. Both the particle and fiducial marker shown in FIG. 7B have clear, well-defined edges (e.g., in contrast to the positive and negative contrast edges shown in the unaligned image of FIG. 7A).

[0117] 3. Masking particles on the reference area In the next step 556, a mask is applied to the particles, fiducial areas, and other features found in the images of the pre-scan and post-scan images. FIG. 8 is an example image of fiducial spots and particles with a mask applied. The mask can be used to determine small background areas that exclude particles, fiducial markers, and other small features on the sensor chip surface, leaving small background areas of the image that represent background roughness, i.e., intensity increases and decreases in the background areas are due (e.g., solely) to surface roughness. The background areas may be located within or outside the spots (of capture molecules) on the chip. Matching background areas are identified in the focused pre-scan and focused post-scan images (following coarse alignment).

[0118] 4. Fine XY alignment of local neighborhoods In the next step, to perform fine matching 558 in the xy plane, background regions of the pre-scanned and post-scanned images are iteratively subtracted from each other as small relative shifts in the xy plane are applied to search for local nearby locations, as illustrated in Figures 9A and 9B. Figure 9A is an image of a chip with matching markers 900 (like an inverted "L") shown thereon. One of the matching markers 900a is within the boundaries of a red box 902. The green and blue inset boxes 903, 904 represent small background regions. In an embodiment, a portion of the pre-scanned image without a reference spot (906 as shown in green) can be scanned over a small area of ​​the corresponding post-scanned image (e.g., as shown by the blue box 904 in Figure 9A). This allows for optimal matching between the two images. Figure 9B shows the standard deviation of the difference between the two images over a 20x20 pixel search area (i.e., when each image is shifted relative to each other by 0 to 20 pixels in the X and Y directions). If two images are perfectly aligned without any difference between them, the standard deviation of the differences is expected to be minimized, and therefore the minimum value shown on the 3D plot corresponds to the optimal shift that should be applied to align the two images.

[0119] 5. Rotation alignment on large images After the local alignment is achieved, rotational alignment of the two images is performed (560). The rotational alignment of the two images is performed over a larger area of ​​the images than the fine XY alignment. Figures 10A and 10B show the process for rotational alignment between pre-scanned and post-scanned images. The center of rotation is placed at the center of the previously aligned reference area used in the step of performing the fine XY alignment. Figure 10B is a graph of the standard deviation of the difference between the two images as one image is rotated over the other image. As larger images are rotated over each other, the optimal alignment between the two images corresponds to the minimum standard deviation of the difference between the images, which in this embodiment is about 0.002 radians. Thus, rotation can be applied to the images to achieve a better alignment between them.

[0120] 6. Chip Out-of-plane Tilt Search After the rotation, an out-of-plane tilt search can be performed to determine the optimal slice for image subtraction (562). As discussed herein, each slice of a pre-scan or post-scan image stack corresponds to a different axial plane along the Z-axis at which the imaging device acquired the images. In an embodiment, the optimal slice for image subtraction may be determined by taking an image from one stack (e.g., a pre-scan stack) and dividing it into four substantially equal square quadrants. For example, for a square or rectangular image, this may correspond to the top left (TL), top right (TR), bottom left (BL), and bottom right (BR) quadrants. The slice is then compared to the corresponding region of the image in a second stack (e.g., a post-scan stack) to minimize the difference between the images. The standard deviation of the difference between the images should decrease with better alignment between the slices. FIG. 11 is an exemplary plot showing the optimization of four quadrants, namely TL1100, TR1102, BL1104, and BR1106, as the pre-scan image is scanned along the corresponding quadrants in the images of the post-scan stack.

[0121] result After performing the registration as described herein, an optimal pair of registered and focused pre-scanned and post-scanned images was acquired and subtracted from each other to obtain the final particle image. The background standard deviation was calculated to evaluate the performance of the approach in minimizing background enhancement. Table 1 below compares the present approach with a bead-based approach using fiducial markers and background intensity enhancement. [Table 1]

[0122] The results show that the background standard deviation is comparable between the two methods for the beadless method using 32 frame averages. However, increasing the number of frames averaged to 256 shows a small decrease in the standard deviation for the beadless method compared to the bead-based method. The time required to perform the beadless method exceeds that of the bead-based method. It takes about 7 seconds per spot to process an image using the bead-based method compared to the beadless method, which takes about 14 seconds per spot to process an image. However, further optimization can be achieved to reduce the processing time for the beadless method.

[0123] Particles that can be detected by the imaging methods described herein include, but are not limited to, extracellular vesicles, exosomes, viruses, virus-like particles, or lipid nanoparticles. Detection of particles in an image can be used to classify, diagnose, and / or monitor a disease (e.g., involving a tumor, cancer, malignancy) in a patient, or to select a treatment option for the patient. In some embodiments, the method may include treating the patient.

[0124] In one embodiment, detection of particles by the imaging methods described herein may be used to classify, diagnose, and / or monitor tumors or cancer. In some embodiments, the cancer is an adenocarcinoma. In some embodiments, the cancer is selected from breast, lung, head or neck, prostate, esophagus, trachea, brain, liver, bladder, stomach, pancreas, ovary, uterus, cervix, testes, colon, rectum, and skin. In some embodiments, the cancer is an adenocarcinoma of the breast, lung, head or neck, prostate, esophagus, trachea, brain, liver, bladder, stomach, pancreas, ovary, cervix, testes, colon, rectum, or skin. In some embodiments, the cancer is selected from pancreas, lung (e.g., small cell or non-small cell), and breast.

[0125] Other examples of tumors, cancers, or malignancies include, but are not limited to, acute childhood lymphoblastic leukemia, acute lymphoblastic leukemia, acute lymphocytic leukemia, acute myeloid leukemia, adrenocortical carcinoma, adult (primary) hepatocellular carcinoma, adult (primary) liver cancer, adult acute lymphocytic leukemia, adult acute myeloid leukemia, adult Hodgkin's disease, adult Hodgkin's lymphoma, adult lymphocytic leukemia, adult non-Hodgkin's lymphoma, adult primary liver cancer, adult soft tissue sarcoma, AIDS-related lymphoma, AIDS-related malignancies, anal carcinoma, astrocytoma, cholangiocarcinoma, bladder cancer ... Bladder cancer, bone cancer, brain stem glioma, brain tumor, breast cancer, renal pelvis and ureter cancer, central nervous system (primary) lymphoma, central nervous system lymphoma, cerebellar astrocytoma, cerebral astrocytoma, cervical cancer, childhood (primary) hepatocellular carcinoma, childhood (primary) liver cancer, childhood acute lymphoblastic leukemia, childhood acute myeloid leukemia, childhood brain stem glioma, childhood cerebellar astrocytoma, childhood cerebral astrocytoma, childhood extracranial germ cell tumor, childhood Hodgkin's disease, childhood Hodgkin's lymphoma, childhood hypothalamic and visual pathway glioma, childhood lymphoblastic leukemia, childhood medulloblastoma, childhood non-Hodgkin's lymphoma, childhood pineal and supratentorial primitive neuroectodermal tumor, Childhood primary liver cancer, Childhood rhabdomyosarcoma, Childhood soft tissue sarcoma, Childhood visual pathway and hypothalamic glioma, Chronic lymphocytic leukemia, Chronic myeloid leukemia, Colon cancer, Cutaneous T-cell lymphoma, Endocrine islet cell carcinoma, Endometrial cancer, Ependymoma, Epithelial carcinoma, Esophageal cancer, Ewing's sarcoma and related tumors, Exocrine pancreatic cancer, Extracranial germ cell tumors, Extragonadal germ cell tumors, Extrahepatic bile duct cancer, Eye cancer, Female breast cancer, Gaucher disease, Gallbladder cancer, Gastric cancer, Gastrointestinal carcinoid tumors, Gastrointestinal tumors, Germ cell tumors, Gestational trophoblastic tumors, Hairy cell leukemia, Head and neck cancer, Hepatocellular carcinoma, Hodgkin's disease, Hodgkin's lymphoma, Hypergammaglobulinemia, Hypocalcemia, Pharyngeal cancer, Intestinal cancer, Intraocular melanoma, Islet cell carcinoma, Islet cell pancreatic cancer, Kaposi's sarcoma, Kidney cancer, Laryngeal cancer, Lip and oral cavity cancer, Liver cancer, Lung cancer, Lymphoproliferative disorders, Macroglobulinemia, Male breast cancer, Malignant mesothelioma, Malignant thymoma, Medulloblastoma, Melanoma, Mesothelioma, Metastatic occult primary squamous cell cervical cancer, Metastatic primary squamous cell cervical cancer, Metastatic squamous cell cervical cancer, Multiple myeloma, Multiple myeloma / plasma cell neoplasm, Myelodysplastic syndrome, Myeloid leukemia, Myeloproliferative disorders, Nasal cavity and paranasal sinus cancer, Nasopharyngeal carcinoma, Neuroblastoma, Non-Hodgkin's lymphoma during pregnancy, Non-melanoma skin cancer, Non-small cell lung cancer,Occult primary metastatic squamous cell neck cancer, oropharyngeal cancer, bone / malignant fibrous sarcoma, osteosarcoma / malignant fibrous histiocytoma, osteosarcoma / malignant fibrous histiocytoma of bone, ovarian epithelial cancer, ovarian germ cell tumor, ovarian low malignant potential tumor, pancreatic cancer, dysproteinemia, purpura, parathyroid cancer, penile cancer, pheochromocytoma, pituitary tumor, plasma cell neoplasm / multiple myeloma, primary central nervous system lymphoma, primary liver cancer, prostate cancer, rectal cancer, renal cell carcinoma, renal pelvis and ureter cancer, retinoblastoma, rhabdomyosarcoma, salivary gland cancer, sarcoidosis tumors, Sezary syndrome, skin cancer, small cell lung cancer, small intestine cancer, soft tissue sarcoma, squamous cell cervical cancer, gastric cancer, supratentorial primitive neuroectodermal tumor and pineal tumor, T-cell lymphoma, testicular cancer, thymoma, thyroid cancer, transitional cell carcinoma of the renal pelvis and ureter, transitional pelvic and ureteral carcinoma, trophoblastic tumor, ureteral and renal pelvic cell carcinoma, urethral cancer, uterine cancer, uterine sarcoma, vaginal cancer, visual pathway and hypothalamic glioma, vulvar cancer, Waldenstrom's macroglobulinemia, Wilms' tumor, and any other hyperproliferative disease.

[0126] Elements of different implementations described herein may be combined to form other implementations not specifically mentioned above. Elements may be removed from the processes, computer programs, databases, etc. described herein without adversely affecting their operation. Additionally, the logic flows depicted in the figures do not require the particular order or sequential order shown to achieve desirable results. Various separate elements may be combined into one or more individual elements to perform the functions described herein.

[0127] Throughout the description, where apparatus and systems are described as having, including, or comprising specific components, or processes and methods are described as having, including, or comprising specific steps, it is additionally assumed that there are apparatus and systems of the described technology that consist essentially of or consist of the recited components, and that there are processes and methods according to the described technology that consist essentially of or consist of the recited processing steps.

[0128] It should be understood that the order of steps or order for performing certain actions is immaterial so long as the technology being described remains operable, and two or more steps or actions may be conducted simultaneously.

[0129] Although the described technology has been particularly shown and described with reference to specific embodiments, it should be understood by those skilled in the art that various changes in form and details can be made therein without departing from the spirit and scope of the described technology as defined by the appended claims.

Claims

1. 1. A method for reducing background in an interference image through three-dimensional (3D) image registration and background correction, the method comprising: (a) receiving, by a processor of a computing device, a pre-scan stack including a plurality of images of a first region of a sensor chip surface, the plurality of images of the pre-scan stack being acquired prior to contacting the sensor chip surface with a sample including one or more target particles, each image of the pre-scan stack corresponding to an image of the first region of the sensor chip surface taken at a different axial distance from an objective lens of an imaging system; (b) the processor receiving a post-scan stack including a plurality of images of a second region of the sensor chip surface (e.g., a second region that substantially overlaps with the first region), the plurality of images of the post-scan stack being acquired after contacting the sensor chip surface with the sample, each image of the post-scan stack corresponding to an image of the second region of the sensor chip surface taken at a different axial distance from the objective lens of the imaging system; (c) the processor determining a prescan focal plane position relative to the prescan stack and acquiring a focused prescan image from the prescan stack using the prescan focal plane position; (d) the processor determining a post-scan focal plane position relative to the post-scan stack and acquiring a focused post-scan image from the post-scan stack using the post-scan focal plane position; (e) the processor registering the focused pre-scan image and the focused post-scan image; (f) following step (e), the processor uses the focused pre-scan image and the focused post-scan image, which are aligned with one another, to generate reduced background, one or more final particle images; A method comprising:

2. 2. The method of claim 1, wherein the first region of the sensor chip surface imaged in the pre-scan stack includes a plurality of reference particles such that at least some of the plurality of pre-scan images of the pre-scan stack each include one or more reference particle features.

3. Step (c) detecting a set of one or more reference particle features corresponding to individual reference particles in each of at least a portion of the plurality of images of the pre-scan stack; determining the pre-scan focal plane position using the set of reference particle features within the plurality of images of the pre-scan stack; The method of claim 2 , comprising:

4. 4. The method of claim 3, wherein the second region of the sensor chip surface imaged in the post-scan stack includes a plurality of reference particles such that at least some of the images of the post-scan stack each include one or more reference particle features.

5. Step (d) detecting a set of one or more reference particle features corresponding to individual reference particles in each of at least a portion of the plurality of images of the post-scan stack; determining the post-scan focal plane position within the plurality of images of the post-scan stack using the set of reference particle features; The method of claim 4, comprising:

6. The method of claim 2 , wherein the reference particles comprise one or more members selected from the group consisting of polystyrene beads, silica beads, latex beads, and polystyrene sulfate beads.

7. 3. The method of claim 2, wherein at least one of the first and second regions of the sensor chip surface comprises a spot, the spot comprising one or more target binding agents that bind to the target particle.

8. 8. The method of claim 7, wherein the reference particle comprises one or more molecules on a surface of the reference particle, and one or more target binding agents of the spot also bind to the surface of the reference particle.

9. The method of claim 7 , wherein the spots also include one or more reference binding agents that bind to the reference particles.

10. 3. The method of claim 2, wherein step (e) includes aligning the focused pre-scanned image and the focused post-scanned image with each other using a set of detected reference particle features in the focused pre-scanned image and a set of detected reference particle features in the focused post-scanned image.

11. The method of claim 1, wherein step (e) includes aligning the focused pre-scan image with the focused post-scan image using an affine transformation.

12. The method of claim 1 , wherein step (f) comprises subtracting the focused pre-scan image from the focused post-scan image.

13. The method of claim 1 , wherein the one or more target particles have a size of about 20 nm to about 50 nm.

14. 10. The method of claim 1, wherein the one or more targeted particles comprise extracellular vesicles.

15. The method of claim 1 , wherein the one or more target particles comprise a virus or virus-like particle.

16. 16. The method of claim 15, wherein the virus is a lentivirus or an adeno-associated virus (AAV).

17. The method of claim 1 , wherein the one or more targeted particles comprise lipid nanoparticles.

18. 1. A system comprising: a processor of a computing device; Memory and 18. A system comprising: said memory having instructions stored on said memory that, when executed by said processor, cause said processor to perform the method of any one of claims 1 to 17.

19. The system of claim 18 , wherein the system further comprises an imaging device or sensor chip.

20. 20. The system of claim 19, wherein the sensor chip surface is a top surface of a thin, partially transparent layer on a base substrate.

Images