Microscope system for imaging three-dimensional objects in three-dimensional matrices

By combining pre-scanning and high-resolution scanning, the problem of time-consuming 3D object imaging has been solved, achieving faster and more efficient imaging results. In particular, when imaging biological cells, it reduces invalid scans and data processing complexity.

CN114303172BActive Publication Date: 2024-11-26REVITI CELLULAR TECH GMBH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202080029181.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-06-30
Filing Date
2020-02-14
Publication Date
2024-11-26
Estimated Expiration
2040-02-14

AI Technical Summary

Technical Problem

Existing technologies are time-consuming when imaging three-dimensional objects and struggle to provide usable results within the required timeframe, especially when imaging biological cells, particularly due to the large amount of data and increased processing complexity caused by numerous invalid scans.

Method used

A combination of pre-scanning and scanning is employed. A low-magnification objective lens is used for pre-scanning to identify the xy position and z height of a 3D object. Subsequently, a high-magnification objective lens is used for high-resolution scanning in the z direction. Combined with image correction techniques, this reduces invalid scans and the amount of data.

Benefits of technology

It significantly reduces scanning time and data volume, achieving faster imaging speed and higher resolution, and supports real-time or near real-time 3D object imaging.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114303172B_ABST
    Figure CN114303172B_ABST
Patent Text Reader

Abstract

Certain configurations of methods and systems that can be used to image three-dimensional objects such as biological cells, biological tissues, or biological organisms are described. The methods and systems can image three-dimensional objects with reduced imaging time and reduced data volume.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Certain configurations relate to methods and systems that can be used to image three-dimensional objects. More specifically, certain methods and systems are described that can image three-dimensional objects, such as biological cells, in an automated and high-throughput manner. Background Art

[0002] Three-dimensional objects are often imaged to study their structure and function. Imaging three-dimensional objects can be time consuming and may not provide usable results within the required time frame, especially if the imaged objects contain biological cells. Summary of the invention

[0003] In one aspect, a method for imaging a three-dimensional object present in a matrix using a microscope is described. In some embodiments, the method includes pre-scanning a matrix using a first magnification objective lens to identify the xy position of the three-dimensional object in the matrix and identifying the z height of the three-dimensional object in the matrix. In other examples, the method also includes scanning the identified three-dimensional object in the z direction using the identified z height of the identified three-dimensional object using a second magnification objective lens to provide a three-dimensional image of at least some portion of the three-dimensional object present in the matrix, wherein the second magnification objective lens is greater than or equal to the first magnification objective lens.

[0004] In some examples, the pre-scanning includes obtaining a plurality of separate z-plane images using the first magnification objective of the microscope to provide a first image set of discrete z-plane images for identifying the z-height. In other examples, the scanning includes obtaining a plurality of discrete z-plane images of the identified three-dimensional object starting from the identified z-height of the three-dimensional object, and using the second magnification objective of the microscope to provide a second image set. In some embodiments, the plurality of discrete z-plane images of the second image set are used to provide a complete three-dimensional image of the three-dimensional object in the matrix. In certain examples, before scanning the identified three-dimensional object using the second magnification objective, the first image set may be image corrected to provide a corrected first image set. In other examples, the corrected xy position of the three-dimensional object and the corrected z-height of the three-dimensional object obtained from the corrected first image set are used to scan the three-dimensional object using the second magnification objective to provide the second image set. In some embodiments, the second image set may be image corrected to provide a corrected second image set for providing the three-dimensional image of the three-dimensional object.

[0005] In some instances, the pre-scan using the first magnification objective is performed using an air objective, such as a 5X or 10X air objective of a microscope, and the scan using the second magnification objective is performed using an immersion objective, such as a water immersion objective or an oil immersion objective of the microscope, which can be a 40X, 50X, 60X or higher magnification objective.

[0006] In some examples, both pre-scanning and scanning are performed using a laser confocal scanning microscope. In certain embodiments, the three-dimensional object is a biological organism, a biological organ, a biological tissue, a biological cell, or a component or organelle thereof. In other examples, the pre-scanning and scanning of the biological organism, the biological organ, the biological tissue, the biological cell, or a component or organelle thereof are performed in a matrix comprising a hydrogel or in a matrix comprising a three-dimensional scaffold.

[0007] In some instances, described method comprises using the first magnification object lens of described microscope to pre-scan described matrix to identify the xy position of each in a plurality of independent three-dimensional objects in matrix and identify the z height of each in a plurality of three-dimensional objects identified in described matrix.Described method can also comprise using the second magnification object lens to scan each identified three-dimensional object in z direction using the z height of each identified three-dimensional object separately, to provide the three-dimensional image of at least a certain part of each in the three-dimensional object existing in described matrix.When a plurality of objects are imaged in matrix, air objective lens (such as 5X or 10X air objective lens of microscope) can be used to pre-scan using the first magnification object lens, and immersion liquid objective lens (such as water immersion objective lens of microscope or oil immersion objective lens, can be 40X, 50X, 60X times or higher multiple object lens) can be used to scan using the second magnification object lens.When a plurality of objects are imaged in matrix, each described three-dimensional object is independently biological organism, biological organ, biological tissue, biological cell or its component or organelle, and wherein said matrix comprises hydrogel or three-dimensional support. The objects do not have to be the same type of cells, tissues, organisms, etc.

[0008] In some embodiments, the method includes using a pre-scan to identify a z-width of the identified three-dimensional object in the matrix, and using the identified z-width to select a scan time or select an amount of data obtained by scanning the three-dimensional object in the matrix.

[0009] In another aspect, a microscope system configured to image a three-dimensional object in a three-dimensional matrix is ​​described. In some examples, the system includes a sample holder configured to receive a sample including the three-dimensional object in a three-dimensional semi-solid or three-dimensional solid matrix; a first light source optically coupled to the sample holder and configured to illuminate at least some portion of the three-dimensional object in the three-dimensional matrix received by the sample holder, wherein a wavelength provided by the first light source is selected to excite at least one substance present in the three-dimensional object (or to excite the entire three-dimensional object); at least one objective lens optically coupled to the light source and configured to receive light emissions from at least one excited substance present in the three-dimensional object in the three-dimensional matrix; a detector optically coupled to the at least one objective lens and configured to receive light emissions from the objective lens; and a processor electrically coupled to the detector. In certain configurations, the microscope system is configured to pre-scan the three-dimensional semi-solid or three-dimensional solid substrate using a first magnification objective lens to identify the xy position of the three-dimensional object and to identify the z-height of the three-dimensional object, and wherein the microscope system is configured to scan the identified three-dimensional object in the z direction using the identified z-height of the identified three-dimensional object using a second magnification objective lens to provide a three-dimensional image of at least a portion of the three-dimensional object.

[0010] In some embodiments, the system includes a first objective lens and a second objective lens, wherein the first objective lens is used as a first magnification objective lens and the second objective lens is used as a second magnification objective lens, and wherein the second objective lens is different from the first objective lens and provides a magnification greater than or equal to the magnification of the first objective lens. In some examples, the first objective lens is an air objective lens and the second objective lens is a water immersion objective lens (or other immersion liquid objective lens).

[0011] In some instances, the detector comprises at least one camera and the first light source comprises at least one laser.For example, the system can be configured to perform laser scanning confocal microscopy using the camera and the laser during pre-scanning and scanning of the three-dimensional object.

[0012] In some embodiments, the processor is configured to execute instructions to construct a three-dimensional image of the three-dimensional object in the three-dimensional matrix based on the image set obtained from the scanning step. In other embodiments, the processor is configured to execute instructions to perform image correction on the image set before constructing the image of the three-dimensional object.

[0013] In some instances, the system uses a second image set for image correction, wherein the second image set is one or more images of a two-dimensional graphic pattern obtained using the detector in the same fixed position used to obtain the image set, wherein the two-dimensional graphic pattern includes points in a dot matrix, wherein the points are located at vertices defining one or more geometric shapes, wherein the dot matrix of the points is non-periodic, and wherein the absolute position of the imaging points can be determined, and wherein the images of the image set are corrected using the one or more images of the two-dimensional graphic pattern to correct geometric distortions in the images of the image set.

[0014] In some configurations, the system includes at least two separate light sources configured to provide light of different wavelengths to the sample on the sample holder.

[0015] In other configurations, the system includes a confocal unit optically coupled to each of the at least two separate light sources and located between the at least two separate light sources and the sample holder.

[0016] In some configurations, the system includes at least two separate detectors configured to receive light of different wavelengths emitted by the sample on the sample holder. In other configurations, each of the at least two separate detectors includes a camera.

[0017] In some embodiments, one of the at least two separate detectors is configured to detect light emissions during the pre-scanning step, and another of the at least two separate detectors is configured to detect light emissions during the scanning step.

[0018] In other embodiments, the system is configured to use the first magnification objective of the microscope to identify the xy position of each of the multiple individual three-dimensional objects in the matrix and to identify the z height of each of the multiple identified three-dimensional objects in the matrix, and wherein the system is configured to use the second magnification objective to scan each identified three-dimensional object in the z direction using the respective identified z height of each identified three-dimensional object to provide a three-dimensional image of at least some portion of each three-dimensional object present in the matrix.

[0019] In some examples, the system includes four independent light sources, each configured to provide light of a different wavelength to the sample on the sample holder. The system may also include four independent cameras, each configured to detect light emission from the sample.

[0020] In other examples, the system is further configured to identify a z-width of the identified three-dimensional object in the matrix and use the identified z-width to select a scan time or an amount of data obtained from a scan of the three-dimensional object in the matrix.

[0021] On the other hand, a non-transitory computer-readable medium storing instructions, wherein the instructions, when executed by a processor, cause the processor to: pre-scan a substrate including a three-dimensional object using a first magnification objective lens to identify an xy position of the three-dimensional object in the substrate and identify a z height of the three-dimensional object in the substrate, and scan the identified three-dimensional object in a z direction using a second magnification objective lens using the identified z height of the identified three-dimensional object to provide a three-dimensional image of at least a portion of the three-dimensional object present in the substrate.

[0022] In some cases, the instructions cause the processor to select a magnification for the second magnification objective lens that is greater than or equal to a magnification of the first magnification objective lens.

[0023] In an additional aspect, a method for imaging a three-dimensional object present in a matrix using a laser confocal scanning microscope includes pre-scanning the matrix using a first magnification objective lens to identify the xy position of the three-dimensional object in the matrix and identifying the z height of the three-dimensional object in the matrix, and recording the identified xy position and z height as a first image set, correcting imaging anomalies from the pre-scanning step to provide a corrected first image set, scanning the identified three-dimensional object in the z direction using a second magnification objective lens using the corrected z height and corrected xy position from the corrected first image set to provide a second image set, correcting imaging anomalies from the scanning step to provide a corrected second image set, and constructing an image of the three-dimensional object in the matrix using the corrected second image set.

[0024] In some instances, correcting the imaging anomalies from the pre-scan step includes obtaining one or more images of a two-dimensional graphical pattern using a detector in the same fixed position used to obtain the first set of images, wherein the graphical pattern is a lattice of points at vertices defining one or more geometric shapes, wherein the lattice of points is non-periodic, and wherein the absolute positions of the imaging points can be determined, and automatically adjusting, by a processor, images of the first set of images using the one or more images of the two-dimensional graphical pattern to correct for geometric distortions in the images of the first set of images. In other instances, the method includes automatically adjusting, by a processor, images of the second set of images using one or more images of the two-dimensional graphical pattern (which may be the same images used to correct pre-scan values) to correct for geometric distortions in the images of the second set of images.

[0025] In other instances, correcting the imaging anomalies from the scanning step includes obtaining one or more images of a two-dimensional graphic pattern using a detector in the same fixed position used to obtain the second image set, wherein the graphic pattern is a dot matrix including points at vertices defining one or more geometric shapes, wherein the dot matrix of points is non-periodic and wherein absolute positions of imaging points can be determined, and automatically adjusting images of the second image set using the one or more images of the two-dimensional graphic pattern by a processor to correct geometric distortions in the images of the second image set.

[0026] In some instances, the pre-scan includes obtaining a plurality of separate z-plane images using the first magnification objective of the microscope to provide the first image. In other instances, the scanning includes obtaining a plurality of discrete z-plane images of the identified three-dimensional object starting from the corrected identified z height of the three-dimensional object, and using the second magnification objective of the microscope to provide the second image set. In a further instance, the plurality of discrete z-plane images of the second image set are used to provide a complete three-dimensional image of the three-dimensional object in the matrix after image correction. In certain embodiments, the pre-scan using the first magnification objective is performed using a 10X air objective of the microscope, and the scan using the second magnification objective is performed using a 40X water immersion objective of the microscope.

[0027] In some instances, when imaging correction is also performed, both pre-scanning and scanning are performed using a laser confocal scanning microscope. In some embodiments, the three-dimensional object is a biological organism, a biological organ, a biological tissue, a biological cell, or a component or organelle thereof. In other embodiments, the pre-scanning and scanning of the biological organism, the biological organ, the biological tissue, the biological cell, or a component or organelle thereof are performed in a matrix comprising a hydrogel or in a matrix comprising a three-dimensional scaffold.

[0028] In some embodiments, the method includes using the pre-scan to identify the z-width of the identified three-dimensional object in the substrate and using the identified z-width to select a scan time or select the amount of data obtained from the scan of the three-dimensional object in the substrate. If necessary, the identified z-width can be image corrected to provide a corrected z-width.

[0029] In another aspect, a microscope system configured to image a three-dimensional object in a three-dimensional matrix includes a sample holder configured to receive a sample including the three-dimensional object in the three-dimensional matrix; a first light source optically coupled to the sample holder and configured to illuminate at least some portion of the three-dimensional object in the three-dimensional matrix received by the sample holder, wherein a wavelength provided by the first light source is selected to excite at least one substance present in the three-dimensional object; at least one objective lens optically coupled to the light source and configured to receive light emissions from at least one excited substance in the three-dimensional object present in the three-dimensional matrix; a detector optically coupled to the at least one objective lens and configured to receive light emissions from the objective lens; and a processor electrically coupled to the detector. In some configurations, the microscope system is configured to pre-scan the substrate using a first magnification objective lens to identify the xy position of the three-dimensional object in the substrate and to identify the z height of the three-dimensional object in the substrate and record the identified xy position and z height as a first image set, correct imaging anomalies from the pre-scan to provide a corrected first image set, scan the identified three-dimensional object in the z direction using the corrected z height and corrected xy position from the corrected first image set using a second magnification objective lens to provide a second image set, and correct imaging anomalies from the scan to provide a corrected second image set.

[0030] In some embodiments, the system includes a first objective lens and a second objective lens, and wherein the first objective lens is used as a first magnification objective lens and the second objective lens is used as a second magnification objective lens, and wherein the second objective lens is different from the first objective lens and provides a magnification greater than or equal to the magnification of the first objective lens. In some instances, the first objective lens is an air objective lens and the second objective lens is a water immersion objective lens. In other instances, the detector includes at least one camera and the first light source includes at least one laser, and wherein the system is configured to perform laser scanning confocal microscopy using the camera and the laser during the pre-scan and scan of the three-dimensional object.

[0031] In some embodiments, the processor is configured to execute instructions to construct a three-dimensional image of the three-dimensional object in the three-dimensional matrix from the image set obtained from the scanning step. In other embodiments, the first image set is image corrected using another image set, the other image set being one or more images obtained from a two-dimensional graphical pattern using the detector in the same fixed position used to obtain the image set, wherein the two-dimensional graphical pattern includes points in a lattice, wherein the points are at vertices defining one or more geometric shapes, wherein the lattice of the points is non-periodic, and wherein the absolute positions of the imaged points can be determined, and the images of the first image set are corrected using the one or more images of the two-dimensional graphical pattern to correct for geometric distortions in the images of the first image set and provide a corrected image set.

[0032] In other embodiments, the system includes at least two separate light sources configured to provide light of different wavelengths to the sample on the sample holder. In some instances, the microscope system includes a confocal unit optically coupled to each of the at least two separate light sources and located between the at least two separate light sources and the sample holder. In some instances, the system includes at least two separate detectors configured to receive light of different wavelengths emitted by the sample on the sample holder. In other instances, each of the at least two separate detectors includes a camera. In some configurations, one of the at least two separate detectors is configured to detect light emission during the pre-scanning step, and another of the at least two separate detectors is configured to detect light emission during the scanning step.

[0033] In some instances, the system is configured to use the first magnification objective of the microscope to identify the xy position of each of a plurality of individual three-dimensional objects in the matrix and to identify the z-height of each of the identified plurality of three-dimensional objects in the matrix, and wherein the system is configured to use the second magnification objective to scan each identified three-dimensional object in the z-direction using the respective identified z-height of each identified three-dimensional object to provide a three-dimensional image of at least some portion of each of the three-dimensional objects present in the matrix.

[0034] In other configurations, the system includes four independent light sources, each configured to provide light of a different wavelength to the sample on the sample holder, and four independent cameras, each configured to detect light emissions from the sample.

[0035] In other embodiments, the system is further configured to identify a z-width of the identified three-dimensional object in the substrate and use the identified z-width to select a scan time or an amount of data obtained from a scan of the three-dimensional object in the substrate.

[0036] In another aspect, a non-transitory computer readable medium storing instructions, wherein the instructions, when executed by a processor, cause the processor to: pre-scan a substrate using a first magnification objective lens to identify an xy position of a three-dimensional object in the substrate and to identify a z height of the three-dimensional object in the substrate and record the identified xy position and z height as a first image set, correct imaging anomalies from the pre-scan to provide a corrected first image set, scan the identified three-dimensional object in a z direction using a second magnification objective lens using the corrected z height and corrected xy position from the corrected first image set to provide a second image set, correct imaging anomalies from the scan to provide a corrected second image set, and construct an image of the three-dimensional object in the substrate using the corrected second image set. In some instances, the instructions cause the processor to select a magnification for the second magnification objective lens that is greater than or equal to the magnification of the first magnification objective lens.

[0037] Additional aspects, embodiments, examples, configurations and features are described in further detail below. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Certain examples of methods and systems are described with reference to the accompanying drawings, in which:

[0039] Figure 1 is an illustration of a three-dimensional object in a matrix according to some examples;

[0040] Figure 2 shows a diagram of a scan of a well including a substrate and a three-dimensional object according to some embodiments;

[0041] Figure 3 shows a diagram of a pre-scan of a well including a substrate and a three-dimensional object according to some examples;

[0042] Figure 4 shows a diagram of a scan of a three-dimensional object identified in a pre-scan according to some examples;

[0043] Figure 5 An illustration showing two three-dimensional objects of different sizes according to some embodiments;

[0044] Figure 6 According to some examples, after the pre-scan Figure 5 A graphical representation of the scan of each three-dimensional object;

[0045] Figure 7is an illustration of a system that may be used to image a three-dimensional object according to some embodiments;

[0046] Figure 8 is an illustration of another system that may be used to image a three-dimensional object according to certain embodiments;

[0047] Fig. 9 is an illustration of other systems that may be used to image three-dimensional objects according to certain embodiments;

[0048] Fig. 10A is a diagrammatic representation of three spherical objects present in a hydrogel matrix according to some examples;

[0049] Fig. 10B , 10C , 10D, 10E and 10F show some embodiments of the present invention. Fig. 10A Various image fields of the objects shown;

[0050] Fig.11A showing pre-scan measurements in various xy planes at different z-dimension values ​​to identify the x, y, and z coordinates (or z-heights) of the two objects according to some examples;

[0051] Fig. 11B shows the use of some examples from Fig.11A A higher resolution scan of the pre-scan measurement.

[0052] Fig. 12A , 12B , 12C, 12D, 12E, and 12F are illustrations of various samples and sample arrangements that may be imaged using the methods and systems described herein;

[0053] Fig.13 A flow chart illustrating a method of pre-scanning and scanning to image a three-dimensional object according to certain examples;

[0054] Fig.14 A flow chart showing a pre-scanning and scanning method according to some embodiments that also uses image correction to image a three-dimensional object;

[0055] Fig.15 shows a diagram of determining the z-width of an object in a matrix according to some examples;

[0056] Fig.16 shows a diagram of 225 fields of view when scanning an entire 384-well microplate according to some examples;

[0057] Fig.17 shows a diagram of 25 fields of view that may be scanned after pre-scanning a 384-well microplate according to some embodiments;

[0058] Fig.18A shows a z-stack for a sphere pre-scan according to some embodiments, and Fig.18B shows an image resulting from scanning a sphere using pre-scan information according to some embodiments;

[0059] Fig.19A shows a 3D visualization of a pre-scan obtained with a 10x air objective according to certain examples, Fig.19B shows an xyz view from a MDCK cyst scan according to certain examples, and Fig.19C shows box plots according to certain examples; and

[0060] Fig. 20A shows a 5x air objective pre-scan according to some examples, Fig. 20B shows segmented tumor cells on a global image according to certain examples, with coordinates for a higher power scan, Fig. 20C shows a maximum intensity projection of a global image obtained using a 20x water immersion objective (280 planes, 1 micron step size) according to certain examples, and Fig.20D , Fig.20E and 20F Each of the diagrams shows a 3D view from a higher power scan of a segmented tumor cell according to certain examples.

[0061] Those skilled in the art will recognize that the objects shown in the figures are provided as examples to facilitate discussion of some aspects and features of the technology. No particular shape, size, dimension, position, or material is intended to be implied or required unless clearly stated by the context describing the particular illustration or example. Specific embodiments

[0062] Some illustrative configurations of the method and system that can be used for imaging the three-dimensional object present in the matrix in an automated manner are described, to promote better understanding of some configurations and examples that this technology can adopt. In view of the benefit of the present disclosure, those of ordinary skill in the art can also produce and use other different configurations. The method described herein is usually implemented in a system comprising a microscope such as a confocal scanning microscope, to allow the three-dimensional object in the matrix to be automatically imaged. However, the method can be used for an imaging system comprising components other than the microscope, to provide similar attributes and benefits. The matrix can be a separate external material, or in the case of a cell imaged in a living organism, the matrix can be other parts of an organism. Although the matrix can take various forms, it is desirable that the matrix does not interfere with the imaging of selected cells, tissues, organisms or other three-dimensional objects.

[0063] In certain embodiments, rapid imaging of three-dimensional objects such as biological cells may be required to monitor the metabolic and / or physiological responses of cells to stimuli. For example, more physiologically relevant three-dimensional model systems are deployed in high-content screening assays in order to better predict the effects of candidate drugs during preclinical screening. In addition, imaging of nanosystems, nanostructures, or other complex three-dimensional structures present in semi-solid or solid matrices can provide information about the overall properties and structure of these materials.

[0064] In some instances, one disadvantage of imaging three-dimensional objects in a matrix is ​​the large amount of data that is acquired, especially if the three-dimensional objects are randomly distributed in the sample container, because a significant number of images will be empty, i.e., contain no portions of the three-dimensional object, or will be out of focus. This amount of data presents several problems, including the need to at least temporarily store all of the data, and the need to process all of the data to construct an image of the three-dimensional object in the matrix. A simplified illustration of this problem is shown in Figure 1 105 is shown in a spherical object 110 shown by a side view (e.g., in two dimensions) as being present in a flat-bottomed hole 105. Although the flat-bottomed hole 105 is used in various illustrations, other hole shapes may also be used, including but not limited to non-flat-bottomed holes such as circular holes. In addition, the sidewalls of the hole 105 do not need to be orthogonal to the flat bottom, but may be circular, conical, or otherwise configured in a manner such that the longitudinal axis of the sidewall is not oriented at 90 degrees to the flat bottom. The object 110 is in a matrix 115 in the hole 105, although as described herein, the object 110 may be adhered to or suspended in the hole 105, and the surrounding matrix is ​​air or liquid, or a matrix including a liquid layer on a solid or semi-solid layer. For example, in the case of biological cells, the matrix 115 may be solid or semi-solid, optionally with nutrient materials to support the survival of cells in the matrix 115 or other materials to promote the biological response of the cells in the matrix 115. For example, in order to image the object 110 in the hole 105 using a microscope, it is necessary to scan the hole 105 at many different points in all three dimensions of the hole 105 to provide an image map of the hole 105 and the three-dimensional object 110 therein. Figure 2, various scan lines (collectively referred to as 210) are shown indicating scan points in the xy dimension of different z-plane holes. For the purposes of this discussion, the z-dimension is the vertical dimension of the figure (up / down), the x-dimension is the horizontal dimension of the figure (left / right), and the y-dimension is the inside / outside of the page of the figure. Different scan lines are different scans in the z-dimension. By scanning over the hole 105 in the xy dimension and scanning at different heights in the z dimension, sufficient data can be obtained and used to construct a three-dimensional image of the object 110. However, the scanning step generates a large amount of data, most of which is irrelevant to constructing an image of the object 110, for example, a large amount of the scan data represents the matrix 115 without any object in it. For example, an image field 230 is shown that does not contain any data for any three-dimensional object in the hole 105. A large amount of time is spent scanning the image field 230, and scanning the field 230 produces significant data that is not used to image the object 110. Although scan lines 210 are shown for illustrative purposes, the exact number and spacing of scans in the z direction may vary depending on the desired resolution. For illustrative purposes, each data point is shown in FIG. Figure 2 , but for imaging three-dimensional objects such as biological cells, the z distance between scan lines is usually 1 micron or less, which will result in a ratio of Figure 2 The scan lines and data points shown are significantly more scan lines and data points. These sequential scanning processes can generate a large amount of data that needs to be at least temporarily stored and / or processed into an image of the object 110. The amount of data can significantly slow down the imaging time and can increase costs due to the additional data storage and processing capabilities required for imaging. Real-time imaging is also difficult due to the long scanning times.

[0065] In some embodiments, in order to provide images of desired resolution while increasing imaging speed, the methods and systems described herein may use a pre-scan to determine the z-height and xy-coordinates of the object to be imaged. For example, rather than scanning the entire hole to image any three-dimensional object in the matrix, a pre-scan may be performed to identify the xy-coordinates and z-height of the object in the matrix. A simplified diagram is shown in FIG. Figure 3305. The pre-scan step uses a lower resolution scan than the scanning step to be able to quickly distinguish between the object 310 in the matrix 315 and the space / region in the matrix 315 of the well 305 that does not contain any object to be imaged. For example, a low power objective lens, such as a 10X air objective lens, can be used to perform the pre-scan to provide a quick identification of the object in the well 305. The xy coordinates of the object 310 and the z height 320 can be identified by the pre-scan. In this illustration, the z height 320 is the vertical distance from the bottom of the well 305 to the bottom of the object 310. The bottom of the well 305 can be used as a reference point, but if necessary, other areas or parts of the well can be used as reference points. For example, when the objective lens of the microscope is located above the well 305, the z height can be the distance from the top of the well 305 to the top of the object 310. The z height 320 does not have to be exactly the distance from the bottom of the well 305 to the bottom of the object 310, for example, it can be 10%-20% lower or higher and still be used as a sufficiently accurate measure of the z height. Likewise and as noted in more detail below, the z-width can be determined in a pre-scan, such as an estimate of the z-height of the three-dimensional object 310, and used to select a particular scan resolution based on the desired scan speed and / or the desired amount of data. In addition, depending on the position of the objective lens, an x-distance can be used, such as the x-distance from the left side of the aperture 305 to the left side of the object 310, or a y-distance can be used instead of or in addition to the z-height, such as the y-distance from one side of the aperture 305 to one side of the object 310.

[0066] In some embodiments, once the xy coordinates and z height of the three-dimensional object 310 are determined using a pre-scan, the methods and systems described herein can focus on the xy coordinates of the identified three-dimensional object and then scan from the identified z height, for example, scanning can be performed in different z directions in the identified xy plane to provide image slices of the object 310. Other areas in the hole can be ignored or not additionally scanned during the high-resolution scanning step to facilitate faster scanning time and reduce the amount of data. For example and with reference to Figure 4 , scanning in the z direction at different xy planes using a high resolution objective lens, such as a 40X immersion objective lens, can be performed to provide data representative of the object 310. In this simplified illustration, eight xy scan planes (collectively referred to as 410) of different z dimensions are shown as a first xy scan plane starting at a z height 320 determined by the pre-scan, and continuous xy plane scans are performed in the z direction above the z height. For example, the xy plane scans above the z height can be performed at a z-dimensional distance interval of 1 micron to provide multiple image slices of the object 410. The resulting data can be stored and used to construct an image of the object 310 that can be displayed as needed.

[0067] Figure 4A lower resolution pre-scan is shown identifying the xy coordinates of object 310 and the z height 320 of object 310, and the combination of a subsequent higher resolution scan of the identified xy coordinates in the z direction starting from the z height can significantly reduce the scan time and data volume. Figure 2 The amount of data obtained from the scans shown is compared to Figure 4 The scan shown allows the system to focus and scan only the area in the hole 305 where the object 310 is present. The rest of the substrate can be ignored (or not additionally scanned) during the higher resolution scanning step. The reduction in the amount of data can reduce the time required to image the object 310 in the hole 305 and can allow real-time or near real-time imaging using the reduced amount of data.

[0068] In some cases, there is often more than one object in any one well or sample container. These objects can be located at different z-heights and can have different dimensions / shapes overall. Figure 5 , shows a side view in which a well 505 includes a first object 510 and a second object 511 in a matrix 515. The objects 510, 511 are of different sizes and shapes and are present in different areas of the matrix 515. A pre-scan step can be performed at a low magnification to identify the xy coordinates of each object 510, 511 and to identify the z height of each object 510, 511. The data obtained from the pre-scan can then be used to scan each of the objects 510, 511 using their identified respective xy coordinates and z heights. Figure 6 , a z-height 610 is shown for object 510, and a z-height 611 is shown for object 511. Object 510 may be scanned at a high resolution by starting at z-height 610 and scanning over the identified xy coordinates of object 510. Additional scans in the z-direction (over the identified xy coordinates of object 510) may be performed to provide image data of object 510, which may be used to reconstruct a three-dimensional image of object 510. Similarly, object 511 may be scanned at a high resolution by starting at z-height 611 and scanning over the identified xy coordinates of object 511. Additional scans in the z-direction (over the identified xy coordinates of object 511) may be performed to provide image data of object 511, which may be used to reconstruct a three-dimensional image of object 511. A low-magnification pre-scan that identifies the xy coordinates and z height of each object 510, 511, combined with a subsequent higher resolution scan using the identified xy coordinates and z height of each object 510, 511, can allow faster imaging with a reduced amount of data than can be achieved with a high-resolution scan of the entire aperture 505.

[0069] In some cases, the methods described herein can be implemented using a system comprising an excitation source, optics, one or more objective lenses, and a detector. Figure 7A general description of the system is shown. System 700 includes an excitation source (light source) 710 optically coupled to an optional system optical device 720, which can be designed to pass the excitation light, filter it, pulse it, or perform other optical operations on the excitation light. The excitation light from the excitation source 710 is provided to a sample space 730, which can be designed to receive the sample in a container such as a tube, a chamber, a microplate, or other device that can hold the sample. The excitation light is incident on a certain area of ​​the sample on the sample area rack. For example, the excitation light can be used to excite luminophores such as fluorophores present in or on a three-dimensional object. The excited fluorophores can then emit light as fluorescent emission 735. The fluorescent emission 735 can be collected by an objective lens 740 and provided to a detector 750 optically coupled to the objective lens 740. Although not shown, additional optical devices such as gratings, mirrors, rotating disks, filters, dichroic sliders, monochromators, etc. may be present between the objective lens 740 and the detector 750 to select certain wavelengths of light for detection or otherwise directing the light emission to a specific port of the detector 750. The position of the sample in the sample space 730 can be moved, for example, in three dimensions, to change the specific point or area of ​​the sample where the excitation light is incident. Alternatively, the excitation source 710 can be moved to change the specific point or area of ​​the sample where the excitation light is incident. The scanning of the sample can be carried out on the identified xy coordinates and start from the z height to obtain multiple data values ​​on the identification dimension of the object. The detector 750 can be electrically coupled to the processor 760, and the processor 760 can store the fluorescence emission of various x, y and z dimensions and use them (together with the fluorescence emission measured from other scanning) to construct the image of the object in the sample. The image can be displayed on an optional display 770 or can be displayed in other formats, such as using a mobile device, a tablet computer or other electronic device coupled to the processor 760 in a wired or wireless manner. The excitation source 710 can be configured in many different forms, including lamps, lasers, arcs or other devices that can emit light. In some cases as described below, there can be more than one excitation source to provide light of different wavelengths to leave different luminophores. For example, different fluorescent markers have different excitation wavelengths. Detector 750 can be many different types of detectors, including but not limited to photomultiplier tubes, cameras, complementary metal oxide semiconductor (CMOS) cameras, charge coupled device (CCD) cameras, scientific CMOS (sCMOS) cameras, electron multiplying CCD (EMCCD), photodiodes, avalanche photodiodes (APDs), microchannel plates (MCPs), and other detectors that can detect light emission of a specific wavelength from a luminophore. Detector 750 typically detects light emission that is red-shifted compared to the wavelength of excitation source 710, and can be designed to detect a very narrow wavelength range, such as 10nm-50nm, or can detect a wide range of wavelengths, such as 400nm-800nm.Optical filters such as bandpass filters can be used to filter out unwanted emission wavelengths to reduce background signals. Although the excitation source 710 is shown as providing light in a direction approximately ninety degrees to the aperture of the objective lens 940, the light can be provided at approximately the same angle, such as parallel, or in a direction where the light strikes the top or bottom surface of the sample space 730.

[0070] In some instances, although the same system is usually used to perform the pre-scan and scan steps described herein, if necessary, a separate detection system can be used for the pre-scan and scan steps. For example, a low-resolution system can be used for the pre-scan step, and a higher-resolution system can be used for the scan step. The detector of the low-resolution pre-scan system can be the same or different from the detector of the higher-resolution scanning system. For example, there may be a detector with lower sensitivity in the pre-scan system to identify the xy coordinates and z height of the object. There may be a detector with higher sensitivity in the higher-resolution scanning system used in object imaging. In addition, the low-resolution pre-scan system can include a low-power air objective lens, and the higher-resolution scanning system can include a higher-power immersion objective lens. The two systems can share a light source, an optical device, etc., or can include a separate light source, an optical device, etc. However, as described herein, using a single system to perform both pre-scan and scan steps can achieve significant data savings, reduced installation complexity and floor space, and higher scanning speeds.

[0071] In some configurations, the objective used in the pre-scan and scan steps can be part of a confocal laser scanning microscope in order to increase scanning speed and provide high-throughput automated scanning. A diagram of a confocal laser scanning microscope system is shown in Figure 8 810 is shown in FIG. 820 . The system 800 includes at least one laser 810 optically coupled to a mirror 815 or other optical element such as a grating. The mirror 815 reflects light from the laser 810 to a dichroic converter 820 as a beam 811. The beam 811 is incident on an optional confocal spinning disk 825, which typically includes a plurality of pinholes or other holes and another mirror 830, and is provided through an objective lens 835 and provided to a sample present in a sample holder 840. As noted herein, different magnifications and / or objectives may be used for pre-scanning and higher resolution scanning steps as desired. Light emitted from the object being scanned is returned through the objective lens 835 as a beam 841 and is incident on the mirror 830, the optional confocal spinning disk 825, the dichroic converter 820, the mirrors 845, 850, and 855, and may be filtered using a filter 860 before being provided to a detector 865 such as, for example, a camera or reference imager. Figure 7The detector 750 is filtered prior to the other detectors described. In some instances, the laser 810 can include one light emitting diode or multiple light emitting diodes, such as two, three, four, five, six, seven, eight or more light emitting diodes (LEDs), each having its own unique wavelength. If desired, any two sources can have similar wavelengths, but different multiples and / or different pulse durations can be provided. The wavelength of light from the laser 810 is typically selected so that any luminophores (e.g., fluorophores or phosphors) present in the object will be excited and can emit light by way of fluorescence or phosphorescence. In some instances, more than one wavelength of light can be used in parallel with the confocal spinning disk 825 to simultaneously monitor emission at different wavelengths and / or to collect multiple scan points simultaneously to even further increase image processing. In addition, in the presence of the confocal spinning disk 825, photobleaching and phototoxicity can be reduced. Although Figure 8 840. It is not shown in the figure, but the transmission light source can be located above the sample holder 840. If necessary, there can also be more than one detector in the system 800. The detector 865 can be many different types of detectors, including but not limited to photomultiplier tubes, cameras, complementary metal oxide semiconductor (CMOS) cameras, charge coupled device (CCD) cameras, scientific CMOS (sCMOS) cameras, electron multiplication CCD (EMCCD), photodiodes, avalanche photodiodes (APD), microchannel plates (MCP) and other detectors that can detect the light emission of a specific wavelength from a luminophore. In addition, a first detector can exist and be used for pre-scan measurements, and a second detector can be used for higher resolution scanning measurements. Without wishing to be bound by any particular theory, the detector response often limits how fast a scan can be performed. By selecting a lower resolution detector with a faster response time for the pre-scan step, a faster pre-scan can be performed to allow a higher resolution scanning step to be started more quickly. Starting a higher resolution scan earlier can visualize the continuous metabolism of cells, which may not be achieved with a slower scanning speed otherwise. System 800 can be used to obtain optical responses on thin optical sections through an object and then use the obtained optical responses to construct an overall three-dimensional image of the object. The image can be constructed or displayed in real time or near real time using less data and higher scan rates.

[0072] In some cases, the pre-scanning step performed using system 800 can use an air objective lens, while the scanning step can use an immersion (water immersion) objective lens or other immersion objective lens, such as oil immersion. As described herein, the low-power objective lens used during the pre-scanning step can generally be 5X, 10X, 15X or 20X. Using a low-power air objective lens allows xy coordinates and z height (or z coordinate) to be quickly identified during the pre-scanning step. Once these values ​​are identified, the objective lens can be switched or changed to a higher-power objective lens, such as, for example, an immersion objective lens, such as 30X, 40X or higher-power objective lens, can be used during the scanning step. In some cases, system 800 includes three or more different objective lenses to provide the possibility of pre-scanning and / or scanning different objects to be measured at different magnifications. The system 800 or a user can select a specific objective lens to be used for the pre-scanning and scanning steps according to the desired scanning rate, the desired amount of data, the desired resolution and / or the desired image acquisition time. The system 800 can be used with different low power objectives to identify the xy coordinates and z height of each image in each well or container of the sample present on the sample holder 840. These coordinates and values ​​can be stored and used to scan each three-dimensional object at a higher resolution (using a higher power objective) while reducing the overall data volume.

[0073] In some examples, the systems described herein may include two or more separate detectors to allow different detectors to simultaneously detect different emission wavelengths. Fig. 9, shows a system 900 including four light sources 902, 903, 904, and 905. Each of the light sources 902-905 is typically a laser, but if desired, a non-laser source, a chopper, etc. can be used instead of a laser. Light from one or more of the light sources 902-905 is incident on a confocal unit 910 including two rotating disks. In the case of using light sources 904, 905, the light passes through the confocal unit 910 and is incident on a reflector 912. The light from the light sources 902-905 is then incident on a dichroic slide 915 and then on a reflector 920, and then provided to an objective lens 925. The light passes through the objective lens 925 and is incident on a sample in a sample holder 930. Although not shown, there may be a transmitted light source above the sample holder 930. Light from different light sources 902-905 can be incident on the same or different areas of the sample on the sample holder 930. Some of the light is absorbed by the luminophore (e.g., fluorophore or phosphor) in the sample and can be emitted as fluorescent or phosphorescent emission. The light emission returned through the objective lens 925 is incident on the reflector 920, the dichroic slide 915, and the confocal unit 910. The confocal unit 910 can select / direct some of the light to the reflectors 932, 934, respectively. The light emission is then provided to the dichroic slider 935. The light emission can then be provided to the reflector 936 and the reflector 938. For example, the light emission reflected from the reflector 932 can be provided to the dichroic slider 935 and the reflector 938. The light emission reflected from the reflector 934 can be provided to the dichroic slider 935 and the reflector 936. The light emission reflected by the reflector 936 can be provided to the filters 941, 943 and enter the detectors 942, 944. The light emission reflected by the reflector 938 can be provided to the filters 945, 947 and enter the detectors 946, 948. Various filters and other optical elements in the system 900 can be selected to minimize the overlap of emission from the sample so that different luminophore emission signals from the sample can be detected simultaneously quickly and with high accuracy. Detectors 942, 944, 946, and 948 can be the same or different. For example, each of detectors 942, 944, 946, and 948 can independently be a photomultiplier tube, a camera, a complementary / metal oxide semiconductor (CMOS) camera, a charge coupled device (CCD) camera, a scientific CMOS (sCMOS) camera, an electron multiplying CCD (EMCCD), a photodiode, an avalanche photodiode (APD), a microchannel plate (MCP), and other detectors that can detect light emission of a specific wavelength from a luminophore.

[0074] In some cases, the pre-scanning step performed using system 900 can use an air objective lens, while the scanning step can use an immersion (water immersion) objective lens or other immersion objective lens, such as oil immersion. As described herein, the low-power objective lens used during the pre-scanning step can typically be 5X, 10X, 15X or 20X. Using a low-power air objective lens allows the xy coordinates and z height to be quickly identified during the pre-scanning step. Once these values ​​are identified, the objective lens can be switched or changed to a higher-power immersion objective lens, such as a 30X, 40X or higher-power objective lens used during the scanning step. Since many scanned samples may contain living biological cells or living biological organisms, an immersion objective lens can be used to scan cells or organisms without destroying or changing ongoing metabolism and cellular activities. When the three-dimensional object to be imaged using system 900 is a non-biological particle, colloid, nanostructure or nanosystem, or even a dead cell or dead organism, oil immersion can be used during the scanning step, although immersion liquids can also be used for these and other non-biological three-dimensional objects. In some cases, the system 900 includes three or more different objective lenses to provide the possibility of pre-scanning and / or scanning different objects to be measured at different magnifications. The system 900 or the user can select a specific objective lens to be used for the pre-scanning and scanning steps based on the desired scan rate, the desired amount of data, the desired resolution, and / or the desired image acquisition time. As described herein, the system 900 can be used to identify the xy coordinates and z height of each image in each hole or container of the sample present on the sample holder 930. These coordinates and values ​​can be stored and used to scan each three-dimensional object at a higher resolution while reducing the overall data volume.

[0075] In certain embodiments, the methods and systems described herein can be used to provide a three-dimensional image of an object present in a solid or semi-solid matrix, such as, for example, a gel, a hydrogel, a three-dimensional scaffold, or a three-dimensional network. Fig. 10A , three 3D spheres 1010, 1020, and 1030 are shown as being present in the hydrogel matrix 1005. In order to cover all of the spheres 1010, 1020, 1030 in the well, an extensive z-direction scan must be obtained in each field throughout the well. Many fields may not contain any 3D spheres at all, or may contain only portions of the 3D objects. For example, FIG. 10B to FIG. 10F The five fields shown in Fig. 10B sphere 1010 is shown as being imaged in the field of view. Although the z-direction scan captures the entire object (i.e., sphere 1010), many scans in the z-direction above the object are obtained even if there is no object there. This scan increases the scan time and the amount of data. Fig. 10C , the field scan captures the spherical object (ie, sphere 1020), but also scans a large amount of open space above the object (ie, sphere 1020). Fig. 10DThe field of only scans the area of ​​the hydrogel matrix that does not contain any objects at all. Fig.10E In the field of view, only a portion of the object (i.e., sphere 1030) is scanned. Fig.10F In the field of view, the object (ie, sphere 1030) is outside the scan field and is completely missed.

[0076] In some instances, to reduce the amount of data and expedite the generation of a 3D image of an object, a pre-scan can be performed to identify the x, y, and z positions of the object of interest from a monochrome low magnification pre-scan at a low z sampling rate. This position information can then be used to acquire a high magnification / resolution z-stack of the object centered in the rescan experiment. The rescan experiment can be set up using the minimum number of planes required to capture the object of interest. This approach reduces the number of planes, can prevent the acquisition of empty images or partially imaged 3D objects, and reduces the amount of data that needs to be acquired and analyzed. In addition, the identified object of interest will be centered in the rescan field of view. Reference Fig.11A , a pre-scan using a low magnification objective lens can identify the x, y, and z positions of objects 1110, 1120 present in the hydrogel matrix. In this illustration, eight z-plane scans 1131-1138 are obtained and used to identify the x and y positions of each object 1110, 1120. As described herein, the z-plane scans can be used to determine the z height of each of the objects 1110, 1120. These z heights can be used as the initial starting position (in the z direction) of the objects 1110, 1120 during the high-resolution scanning step. Reference Fig. 11B , the z height 1132 of the object 1110 can be used as the starting point for the scan. Scans in the xy plane can be performed in incremental z-direction planes to image the object 1110 at high resolution. For example, an immersion objective can be used to image the object 1110 at high resolution by sequentially scanning the image in the z direction at each xy plane. The pre-scan and scan steps together can reduce the amount of data by 20X, 30X, 40X, or 50X or more to achieve faster image scanning and construction.

[0077] In certain embodiments, the methods and systems described herein can be used to image many different types of cells, tissues, organs, organisms, etc. For example, Fig. 12A is a graphic representation of spherical objects present in a hydrogel. Fig. 12B is a diagram of a hollow object (e.g., a cyst in a hydrogel), and Fig. 12C is an illustration showing a sphere on top of a hydrogel, for example in an aqueous layer including culture medium or other material present on top of the hydrogel. The methods and systems can also be used to monitor cell activity and cell state. For example, cell invasion assays can be performed using the methods and systems described herein ( Fig.12D ), cells seeded in hydrogels ( Fig.12E) or cells seeded on top of the hydrogel ( Fig.12F In the case of imaging of whole tissues, organs or organisms, it is desirable that the matrix surrounding the object to be imaged does not interfere with (or can be distinguished from) the pre-scanning and scanning of the object of interest.

[0078] In some embodiments, the methods described herein may be implemented in the form of a software package or executable code that can be used in combination with an imaging system that includes a processor. Various processors and associated components are described in more detail below. In some instances, the software package may be downloaded from a remote website or otherwise installed from a non-transitory computer-readable medium having executable code, such as instructions, stored thereon. Reference Fig.13, the instructions, when executed by the processor, cause the processor to initiate a pre-scan at step 1302. The pre-scan uses a low-power scan to identify the xy position and z height of each object in the well (or other sample container) at step 1304. A low-power objective lens, such as a 10X air objective lens, can be used for low-power imaging. The processor then stores the recorded, identified xy position and z height of each object at step 1306. The processor can then initiate a high-power scan at step 1312. Once initiated, the high-power scan retrieves the recorded, identified xy position and z height of each object from the pre-scan at step 1314, and then uses this information to scan each object using a high-power objective lens at step 1316. For example, the high-power scan can start at the z height on the xy plane formed by the x,y coordinates using a 40X immersion objective lens. When each pixel is read in this particular xy plane at the z height, the value can be stored at step 1318. The processor can then repeat the high-power scan in the z direction above the z height, for example, 1 micron above the z height in the z direction. The pixel values ​​read in the xy plane of this new z dimension may also be stored. The processor may repeat this process until enough data is obtained from different z stacks to form an image of the entire three-dimensional object. The processor may use these recorded pixel values ​​to construct a three-dimensional image of the scanned object at step 1330, and may display the constructed image at step 1332. If the image needs to be updated so that the image can be viewed visually in real time, or if the image needs to be updated at a later time, the processor may restart the high-power scan at step 1316 and may repeat the process. If only a single image of the object is needed, the processor may end the scan at step 1350. Even if multiple high-power scan / image construction iterations are performed, a single pre-scan may be used to identify the xy position and z height, or if necessary, the pre-scan steps 1302-1306 may be repeated periodically to ensure that cells or other objects have not changed their initial positions identified in the pre-scan. In other configurations, the pre-scan and scan steps may be coupled so that a low-power pre-scan is performed before each high-power scan. In such a configuration, the imaging process may return from decision step 1334 to pre-scan step 1304 (rather than scan step 1316), and the entire process may repeat.

[0079] In other embodiments, the methods described herein may be implemented in conjunction with image correction to account for imaging aberrations and anomalies. For example, a software package containing image correction capabilities may be downloaded from a remote website or otherwise installed from a non-transitory computer-readable medium having executable code, such as instructions, stored thereon. Fig.14, the instructions, when executed by the processor, cause the processor to initiate a pre-scan at step 1402. The pre-scan identifies the xy position and z height of each object in the well (or other sample container) using a low-power scan at step 1404. Low-power imaging can be performed using a low-power objective, such as a 10X air objective. The processor then stores the recorded, identified xy position and z height of each object at step 1406. The processor then corrects the recorded, identified xy position and z height to account for any imaging aberrations or anomalies at step 1408. The processor then stores the corrected, identified x,y position and z height of each object at step 1410. The processor then initiates a high-power scan at step 1412. Once initiated, the high-power scan retrieves the recorded, corrected, identified x,y position and z height of each object from the pre-scan at step 1414, and then uses this information to scan each object using a high-power objective at step 1416. For example, a high-power scan can be initiated at a z height in the xy plane formed by the x,y coordinates using a 40X immersion objective. When each pixel is read in the particular xy plane at the z height, the value can be stored in step 1418. The processor can then repeat the high-power scan in the z direction above the z height, for example, 1 micron above the z height in the z direction. The pixel values ​​read in the xy plane of this new z dimension can also be stored. The processor can repeat this process until enough data is obtained from different z stacks to form an image of the entire three-dimensional object. In step 1420, each pixel can be corrected as it is scanned, or a data set (such as an image) including multiple pixels can be corrected. As discussed in more detail below, the correction can be used to illustrate imaging anomalies. In step 1430, the processor can use the corrected, recorded pixel values ​​(or corrected image sets) to construct a three-dimensional image of the scanned object, and the corrected, constructed image can be displayed in step 1432. If the image needs to be updated so that the image can be viewed visually in real time, or the image needs to be updated at a later time, the processor can restart the high-power scan in step 1416 and the process can be repeated. If only a single image of the object is needed, the processor can end the scan in step 1450. Even if multiple high power scan / image construction iterations are performed, a single pre-scan can be used to identify the xy position and z height, or the pre-scan steps 1402-1410 can be repeated periodically if necessary to ensure that cells or other objects have not changed their initial position identified in the pre-scan. In other configurations, the pre-scan and scan steps can be coupled so that a low power pre-scan is performed before each high power scan. In such a configuration, the imaging process can return from the decision step 1434 to the pre-scan step 1404 (rather than the scan step 1416), and the entire process can be repeated.

[0080] In some embodiments, the various emission signals obtained for the scanned three-dimensional objects can be used to construct a three-dimensional image of each scanned object in the sample, for example, the scan information can be used to register the image. Although the exact method used to construct the image from the optical signal may vary, methods similar to those described in US 9,582,864 can be used for data obtained in all three dimensions. In some instances, it may be necessary to correct the image values ​​to account for any distortion or anomalies. For example, when the optical signal is provided through the matrix and enters the objective lens, geometric distortion is usually caused. When the emitted light passes through the air space between the sample and the objective lens, the use of an air objective lens in the pre-scanning step may cause distortion and chromatic aberration. These distortions / chromatic aberrations can be compensated by automatically correcting the z value of each objective lens. For example, the z correction can correct the offset between the z position of the bottom of the upper plate determined by the infrared autofocus laser and the z position of the bottom of the upper plate determined by the visible wavelength. This value may vary due to incomplete correction of chromatic aberrations between infrared and visible wavelengths. The values ​​of different objectives are also different. Since the measurements are performed in the visible range, this offset can be corrected in order to determine the correct object position relative to a common reference point, the bottom of the upper plate detected using a laser-based autofocus system. If the z-offset of both objectives is corrected, the correct object z-height can be obtained before the higher resolution scanning step begins.

[0081] In certain embodiments, a five-element pattern with a z-stack can be measured and used to correct image anomalies. In one case, a five-element pattern is measured using a z-stack (using the desired visible wavelength), the sharpest plane of the pattern is determined, and this z-height information is used for z correction of a single objective lens relative to the z-height found at the autofocus laser wavelength. The difference in refractive index between the five-element pattern measurement and the sample measurement is taken into account in the calculation. This z correction can also correct for additional variables that cause offsets, such as, for example, autofocus laser alignment. These variables may change and / or be adjusted over time, so this correction can be implemented for each pre-scan measurement to ensure that a higher resolution scan is initiated at the appropriate z-height.

[0082] In some instances, the method and system can adjust one or more images of the sample to correct geometric distortion and / or properly align one or more images. The image correction can be performed for one or both of the pre-scan and scanning steps to obtain a corrected image representation of the actual three-dimensional object. In some instances, the sample is a microtiter plate containing a hole of a three-dimensional biological cell, an organelle, or its components. In one case, an artificial dot pattern that allows image adjustment and registration at various magnifications can be used. For example, an image of one or more dot patterns is obtained, and for each image, the expected distortion-free point position (in the x, y, and z directions) can be determined and compared with their position in the image. In some embodiments, a geometric transformation is performed that moves the center of the imaging point (or otherwise changes the position) to their expected position. These expected positions can then be stored and used, for example, in a higher resolution scanning step. The transformation values ​​can be stored and retrieved before the higher resolution scanning step. Then, for each three-dimensional object sample to be imaged, a set of object images are acquired using the same detector device used to obtain the image of the dot pattern. A stored geometric transformation determined using the dot pattern is applied to the image of the sample for correction / registration, and the corrected image is stored and / or displayed.

[0083] In some embodiments, the precise point pattern used in the image correction process may be different. For example, a similar camera or a similar set of cameras (or other detectors) as used to obtain images of three-dimensional objects is used to obtain one or more images of a set of quasi-periodic grid patterns. The same camera adjustments and the same relative positioning of the field of view as used in obtaining images of the three-dimensional object can be used to obtain images of the grid pattern. When obtaining sample images and grid pattern images, it is assumed that the images obtained using the same camera (or other detector) are prepared under similar (or substantially the same) optical conditions and / or relative calibration of the different cameras, and it is assumed that the relative displacement of the image field remains constant.

[0084] In some embodiments, the known properties of the quasi-periodic grid can be used to reconstruct the true geometry of the acquired image. This step eliminates geometric distortion caused by the optical system. In some embodiments, a method involves using the local uniqueness of the imaged pattern to identify points in the pattern and determine their true positions from a known grid pattern design. In another method, the geometric properties of the quasi-periodic grid can be used to derive the true geometry from known relative positioning constraints (e.g., quantized nearest neighbor positions and quantized angles) for creating the pattern. Based on the knowledge of the true geometry, a transformation is derived and can be applied to eliminate geometric distortion.

[0085] In certain configurations where images from different cameras are obtained, images from different cameras can be correlated using known unique positions of imaging points identified in the pattern. If the unique position of the imaging point is known, the relative position of the image field is derived, or if the field captures a partially overlapping area, the relative position of the image field can be determined. In some embodiments, the geometric transformation derived in the steps described herein can be applied to images of three-dimensional objects obtained using the same detector (e.g., the same camera) and the same optical arrangement as used in obtaining images of quasi-periodic grid patterns. Image correction can be performed only on pre-scan measurements, only on higher resolution scan measurements, or on both. Image correction may be particularly required for pre-scan measurements (which can correct the identified xy coordinates and the identified z height to account for anomalies) to ensure that higher resolution scans are initiated at the correct z height and correct xy plane for each three-dimensional object.

[0086] Without wishing to be bound by any one particular geometric pattern or shape, the use of a quasi-periodic lattice provides a number of benefits over the use of a periodic lattice. For example, by using a quasi-periodic lattice, registration errors in the grid period can be avoided, while the average grid density can be preserved and defined (e.g., quantized) geometric features such as nearest neighbor lengths and angles can be preserved.

[0087] In some embodiments, the pattern used in image correction can be a grid including five-fold symmetry. One method of obtaining such a lattice is to provide a Penrose mosaic and associate the structural elements (points) of the pattern with tile positions or vertices. In one method, the pattern starts with a pentagon that is subdivided into two triangles, which are hierarchically subdivided in the golden section, thereby generating new triangles of the same shape. The points with reduced size are associated with the newly created vertices. In some embodiments, a non-periodic grid (e.g., a grid that is not copied by the conversion of its points) can be used to avoid mismatching of points. For example, a program for adjusting one or more images of a sample to correct geometric distortion and / or properly align one or more images expects to use a unique position that can be determined for a given point. In some instances, it may be desirable that the points of the pattern are uniformly located in the field of interest so that the entire field can be corrected. The level of point size can be used at different scales (e.g., different resolutions / magnifications) at the same time. Certain configurations of quasi-periodic grid patterns can provide some or all of the above-mentioned desired characteristics. A suitable pattern of a quasi-periodic grid is similar to a Penrose mosaic. For example, the golden section triangulation of a pentagon can produce a pattern with desired characteristics. At each step of mesh refinement, the size of the points located at the vertices can be reduced.

[0088] In some embodiments, the pattern may be rotated slightly to avoid parallel alignment with the image border (and avoid any resulting distortion). For example, the image may be rotated from 1 degree-30 degrees (e.g., 9 degrees). In some embodiments, the image may be rotated 1 degree-5 degrees, 3 degrees-10 degrees, 5 degrees-15 degrees, 10 degrees-15 degrees, 10 degrees-20 degrees, 15 degrees-25 degrees, 25 degrees-30 degrees. In some embodiments, the image may be rotated by any suitable degree. The exact rotation size of the image may vary as desired.

[0089] In some instances, an image analysis function can be used to determine a desired focal plane in a pre-scan step, which can be adjusted for the three-dimensional object being measured. For example, the image analysis function can include segmentation of the pre-scan image and subsequent identification and quantification of xy coordinates and z heights. These values ​​can be corrected as described herein before the higher resolution scanning step begins. In some instances, the image analysis function can include determining features such as spatial texture, morphology, biological activity, etc. as needed for biological applications. This can be achieved by providing the user with a selection of predefined image analysis functions or by enabling the user to define or program their own custom image analysis functions.

[0090] In certain configurations, image correction can be performed, typically by acquiring a set of images of the pattern. For each image, the expected undistorted point location can be determined. A geometric transformation can then be calculated to change the point center to the expected location. The transformation can then be stored. For each scanned three-dimensional object, a set of images can be acquired, and for each image a geometric transformation can be applied, and the resulting corrected image can be stored and used to construct a corrected image of each three-dimensional object.

[0091] In other configurations, determining the expected undistorted point locations may be accomplished by geometric reconstruction. For example, for each image in the pattern image set, the process may involve segmenting the image to determine all points, calculating the point centers, and selecting an arbitrary point in the middle of the image and assuming its location is exact. While not necessarily correct in all cases, the nearest neighbor point is typically expected to be at a 36 degree angle and at a scale of 1, The distance It is the golden ratio Based on this geometric relationship, each adjacent point can be assigned an expected position. For each adjacent point identified in the previous step, an expected position can be assigned until there is no longer a point in the image. The point that is in the relative position with its nearest neighbor, exceeds the given tolerance with the expected position can be discarded. In some instances, (for example, arbitrarily) select a common reference point in all pattern images to have the same expected position and apply appropriate displacement to the expected point center. (For example, arbitrarily) select the angle at which the nearest neighbor is expected to be located, and the expected coordinates of the point are rotated accordingly. (For example, arbitrarily) select the baseline length at which the nearest neighbor is expected to be located, and the expected coordinates of the point are scaled accordingly. After these steps, the expected position of all correctly identified points should overlap. The value of actual and expected point position can be stored then. In certain embodiments, an alternative method can be to identify the pattern image position in the expected point position table for generating a pattern. The expected point position can be converted into image coordinates by steps similar to those described herein.

[0092] In other embodiments, the point center can be brought to the expected position to calculate the geometric transformation. In some embodiments, the geometric transformation can be performed as follows: For each table corresponding to the expected and actual positions of the pattern image: (1) calculate the displacement required for each individual point to move to the expected position. This set of displacements, together with the positions to which they are applied, is referred to herein as a displacement field; (2) the displacement field is extended to the complete image by approximating the determined displacement field with a continuous model (e.g., a second-order polynomial). This can be achieved by least squares fitting or other suitable approximation; (3) store the polynomial coefficients for the sample image. Then, in some embodiments, the geometric transformation can be applied to the three-dimensional object image, for example, as follows: For each image in the sample set: (A) calculate the displacement corresponding to each image pixel by the found polynomial, and (B) apply the displacement to the image raster data.

[0093] In view of the benefits of the present disclosure, it will be appreciated by those skilled in the art that other image correction processes and systems may also be used to provide image correction (or data correction) for various images obtained during pre-scan and higher magnification scan measurements. For example, different types of man-made objects or printed patterns may be used for image correction and / or registration, such as square patterns drawn by lines, uneven line patterns, random collection points, sparse samples of randomly positioned beads or quantum dots, etc. The approximate microscope stage position may be determined by using an intrinsic positioning system. Thereafter, one may further correct by identifying certain landmarks of the calibration pattern or object, or alternatively correlating portions of different images from different cameras, filter settings, different objective lenses, and other imaging conditions. Other geometric image registration models and correction processes may be derived and used from such data parameters. Although not required, the five-element pattern may be used as a powerful and essentially simple method to achieve absolute and precise positioning of the entire image field and a single portion of the image.

[0094] In some embodiments, the pre-scan measurements described herein can be used to determine the z-width of each three-dimensional object. The z-width is generally the distance in the z dimension that a three-dimensional object occupies, such as the z-distance from the top to the bottom of the object. The z-width does not need to be exact and can be estimated to select a scan speed, scan resolution, amount of data, etc. that can be used or obtained in a higher magnification scan. Fig.15, a simplified illustration showing three z stack pre-scans z1, z2 and z3 is shown. Although hole 1505 is shown to include a round bottom, a flat bottom or other shapes can also be used to determine the z width of various objects. As described herein, the z dimension distance from the bottom of hole 1505 to z plane z1 can be considered as z height. The z width of object 1510 in matrix 1515 is the z dimension distance between z1 and z3. This distance can be determined or estimated and used to select scanning resolution and / or scanning speed for higher power scanning. For example, many high power scans can use 1 micron or less z distance intervals between different scans. Although this scanning resolution can provide high-resolution images, the total time for scanning of object 1510 may be too slow to capture certain metabolic events or cell changes. In order to capture these events with lower resolution, the system can use z width to select specific z dimension scanning spacing and / or total scanning time to allow observation of the desired physiological or chemical changes experienced by object 1510. For example, instead of using a 1 micron z distance spacing when scanning at a higher power, the z distance spacing can be changed to 2 or 3 microns to provide fewer images but allow monitoring of cell activity by providing an overall faster scan time. In addition, the z width can also be used to select an upper limit on the amount of data. For example, based on the z width and the expected amount of data, the z distance scan spacing can be selected to provide no more than the expected amount of data that will be obtained from the higher power pre-scan. The z width can be determined in the same pre-scan used to determine the z height, or it can be determined in a separate pre-scan.

[0095] In some instances and as described above, the systems and methods described herein may include or use a processor to control some aspects of the system or process. In particular, the processor may receive the emission signal and store it as an image, perform image correction as needed, and then display the corrected image on a display to allow the end user to view the three-dimensional object. The processor may be part of the system or instrument, or may be present in an associated device used with the instrument, such as a computer, a portable computer, a mobile device, etc. For example, the processor may be used to control the position of the sample, detector parameters, image correction, and other operations performed by the system. Such processes may be performed automatically by the processor without user intent, or the user may enter parameters through a user interface. For example, the processor may use signal intensity and fragment peaks and one or more calibration curves to determine the identity and how many specific luminophores are present in each individual cell. The processor may also color-code different markers present in the cell to provide easier visualization of different cell components. In some configurations, the processor may be present in one or more computer systems and / or general hardware circuits, including, for example, a microprocessor and / or suitable software for operating systems, such as to control sample introduction devices, ionization sources, mass analyzers, detectors, etc. In some instances, the detector itself may include its own corresponding processor, operating system, and other features to allow detection of various emission signals. The processor may be integrated into the system or may be present in one or more accessory boards, printed circuit boards, or computers electrically coupled to system components. The processor is typically electrically coupled to one or more storage units to receive data from other components of the system and to allow various system parameters to be adjusted as needed or desired. The processor may be part of a general-purpose computer, such as those based on Unix, Intel PENTIUM-type processors, Motorola PowerPC, Sun UltraSPARC, Hewlett-Packard PA-RISC processors, or any other type of processor. According to various embodiments of the present technology, one or more of any type of computer system may be used. In addition, the system may be connected to a single computer or may be distributed between multiple computers connected via a communication network. It should be understood that other functions may be performed, including network communications, and the technology is not limited to having any specific function or set of functions. In a general-purpose computer system, various aspects may be executed as dedicated software. The computer system may include a processor connected to one or more storage devices, such as a disk drive, memory, or other device for storing data. The memory is typically used to store programs, calibration curves, chemical or physical properties measured by the cell analyzer, and data values ​​during system operation. Components of a computer system may be coupled by an interconnection that may include one or more buses (eg, between components integrated within the same machine) and / or networks (eg, between components residing on separate discrete machines).The interconnection device provides communications (e.g., signals, data, instructions) to be exchanged between system components. The computer system can typically receive and / or issue commands within a processing time, such as a few milliseconds, a few microseconds, or less, to allow the system to be quickly controlled. For example, computer control can be implemented to control pre-scan speed, scan speed, which objective lens is used for imaging, etc. The processor is typically electrically coupled to a power supply, which can be, for example, a DC power supply, an AC power supply, a battery, a fuel cell, or other power supply or power supply combination. The power supply can be shared by other components of the system. The system can also include one or more input devices, such as a keyboard, a mouse, a trackball, a microphone, a touch screen, a manual switch (e.g., an overload switch) and one or more output devices, such as a printing device, a display screen, a speaker. In addition, the system (in addition to or as an alternative to the interconnection device) can contain one or more communication interfaces that connect the computer system to a communication network. The system can also include suitable circuits to convert signals received from various electronic devices present in the system. Such circuitry may reside on a printed circuit board, or may reside on a separate board or device electrically coupled to the printed circuit board via a suitable interface such as a serial ATA interface, an ISA interface, a PCI interface, etc., or via one or more wireless interfaces such as Bluetooth, Wi-Fi, near field communication, or other wireless protocols and / or interfaces.

[0096] In certain embodiments, the storage system used in the system described herein generally includes a computer-readable and writable non-volatile recording medium, in which software code can be stored, which can be used by a program to be executed by a processor, or by information stored on or in a medium to be processed by a program. For example, the medium can be a hard disk, a solid-state drive, or a flash memory. The program or instruction to be executed by the processor can be located locally or remotely, and can be retrieved by the processor through an interconnection mechanism, a communication network, or other means as needed. Typically, in operation, the processor reads data from a non-volatile recording medium into another memory that allows the processor to access information faster than the medium. The memory is typically a volatile random access memory, such as a dynamic random access memory (DRAM) or a static memory (SRAM). It may be located in a storage system or in a memory system. The processor typically processes data within an integrated circuit memory and then copies the data to the medium after the processing is completed. A variety of mechanisms are known for managing data movement between a medium and an integrated circuit memory element, and the technology is not limited thereto. The technology is also not limited to a specific memory system or storage system. In certain embodiments, the system may also include specially programmed dedicated hardware, such as an application-specific integrated circuit (ASIC) or a field programmable gate array (FPGA). The aspects of the technology can be implemented with software, hardware or firmware or any combination thereof. In addition, such methods, behaviors, systems, system elements and components thereof can be implemented as a part of the above-mentioned system or as independent components. Although a specific system is described as a type of system in which various aspects of the technology can be practiced by way of example, it should be understood that various aspects are not limited to being implemented on the described system. Various aspects can be practiced on one or more systems with different architectures or components. The system may include a general-purpose computer system programmable using a high-level computer programming language. These systems may also be implemented using specially programmed dedicated hardware. Various data values ​​corresponding to different xy planes and z heights or z dimension values ​​from pre-scanning may also be stored in a storage system and retrieved and used in the scanning measurement of each three-dimensional object to be scanned.

[0097] In these systems, the processor is typically a commercially available processor, such as the famous Pentium-class processor available from Intel Corporation. Many other processors are also commercially available. Such processors typically execute an operating system, which may be, for example, Windows 95, Windows 98, Windows NT, Windows 2000 (Windows ME), Windows XP, Windows Vista, Windows 7, Windows 8, or Windows 10 operating systems available from Microsoft, MAC OS X available from Apple, such as Snow Leopard, Lion, Mountain Lion or other versions, Solaris operating systems available from Sun Microsystems, or UNIX or Linux operating systems available from various sources. Many other operating systems may be used, and in some embodiments, a set of simple commands or instructions may be used as an operating system. In addition, the processor may be designed as a quantum processor, which is designed to perform one or more functions using one or more qubits.

[0098] In some instances, the processor and the operating system can together define a platform on which applications can be written in a high-level programming language. It should be understood that the technology is not limited to a specific system platform, processor, operating system, or network. In addition, in view of the benefits of the present disclosure, it is obvious to those skilled in the art that the technology is not limited to a specific programming language or computer system. In addition, it should be understood that other suitable programming languages ​​and other suitable systems can also be used. In some instances, hardware or software can be configured to implement cognitive architecture, neural network, or other suitable implementations. If necessary, one or more parts of the computer system can be distributed on one or more computer systems coupled to a communication network. These computer systems can also be general-purpose computer systems. For example, various aspects can be distributed in one or more computer systems that are configured to provide services (e.g., servers) to one or more client computers, or to perform overall tasks as part of a distributed system. Various aspects can be performed on a client-server or multi-layer system, which includes components distributed in one or more server systems that perform various functions according to various embodiments. These components can be executable code, intermediate code (e.g., IL), or interpreted code (e.g., Java), which communicate through a communication network (e.g., Internet) using a communication protocol (e.g., TCP / IP). It should also be understood that the technology is not limited to executing on any particular system or group of systems. In addition, it should be understood that the technology is not limited to any particular distributed architecture, network or communication protocol.

[0099] In some cases, various embodiments can be programmed using object-oriented programming languages, such as, for example, SQL, SmallTalk, Basic, Java, Javascript, PHP, C++, Ada, Python, iOS / Swift, Ruby on Rails, or C# (C-Sharp). Other object-oriented programming languages ​​can also be used. Alternatively, functional, scripting, and / or logic programming languages ​​can be used. Various configurations (e.g., documents created in HTML, XML, or other formats that present aspects of a graphical user interface (GUI) when viewed in a window of a browser program or perform other functions) can be implemented in a non-programming environment. Certain configurations can be implemented as programmatic or non-programmatic elements or any combination thereof. In some cases, the system can include remote interfaces, such as those present on mobile devices, tablet computers, laptops, wearable devices, or other portable devices, which can communicate via wired or wireless interfaces and allow remote operation of the system as needed.

[0100] In some instances, the processor may also include or have access to a database of information about fluorophores, biological cells, and other common information. For example, the database may store the excitation and emission wavelengths of different fluorophores, and the processor may retrieve and use this information based on the user's selection of a specific fluorophore used in the experiment. The instructions stored in the memory may execute a software module or control routine of the system, which may in fact provide a controllable model of the system. The processor may use the information accessed from the database and one or more software modules executed in the processor to determine control parameters or values ​​of different components of the system, such as different detector values, different scan rates, general z boundaries of specific cells, and the like. Using an input interface to receive control instructions and an output interface connected to different system components in the system, the processor may perform active control of the system. For example, the processor may control detectors, optical elements such as confocal units, mirrors, dichroic sliders, etc., objective lenses, sample holder positions, and other components of the system.

[0101] In certain embodiments, the precise three-dimensional object that can be imaged using the methods and systems described herein can vary. The object can be an inanimate object or an animate object or a component of an animate object, for example, an organelle, such as a mitochondria. Whole cells, tissues, organs, and organisms can be imaged as desired.

[0102] In some embodiments, biological cells are imaged using the methods and systems described herein. In some instances, the cells can be prokaryotic cells. For example, a drug discovery method using potential antibiotics can be implemented to determine how much of a particular antibiotic is absorbed by a single bacterial cell and to determine the phenotype of a cell, such as a live cell or a dead cell, to evaluate the efficacy of a particular antibiotic candidate. When the cell is a bacterial cell, the bacterial cell can be a cell from one or more of the following: Acidobacteria, Actinobacteria, Aquificae, Armatimonadetes, Bacteroidetes, Caldiserica, Chlamydiae, Chlorobi, Chloroflexi, Chrysiogenetes, Cyanobacteria, Deferribacteres, Deinococcus-Thermus, Dictyoglomi, phylum Elusimicrobia, Fibrobacteres, Firmicutes, Fusobacteria, Gemmatimonadetes, Lentisphaerae, Nitrospirae, Planctomycetes, Proteobacteria, Spirochaetes, Synergistetes, Tenericutes, Thermodesulfobacteria, Thermotogae, or Verrucomicrobia. Exemplary classes, orders, and / or families of bacterial cells that may be analyzed include, but are not limited to, those from the following: Acidobacteria, Blastocatellia, Holophagae, Rubrobacteria, Thermoleophilia, Coriobacteriia, Acidimicrobiia, Nitriliruptoria, Actinobacteria, Aquificales, Aquificaceae,Hydrogenothermaceae, Desulfurobacteriales, Desulfurobacteriaceae, Thermosulfidibacter, Fimbriimonadia, Armatimonadia, Chthonomonadetes, Rhodothermia, Rhodothermales, Balneolia, Balneolales, Cytophagia, Cytophasales hagales), Sphingobacteria, Sphingobacteriales, Chitinophagia, Chitinophagales, Bacteroidia, Bacteroidales, Flavobacteriia, Flavobacteriales, Caldisericaceae, Chlamydiales, Chlamydiaceae, Candidatus idatus), Clavichlamydiaceae, Parachlamydiales, Criblamydiaceae, Parachlamydiaceae, Simkaniaceae, Waddliaceae, Piscichlamydia candidate, Actinochlamydiaceae candidate, Parilichlamydiaceae candidate, Rhabdochlamydiaceae candidate, Ignavibacteria, Ignavibacteriales eriales), Ignavibacteriaceae, Ignavibacterium, Melioribacter, Chlorobea, Chlorobiales, Chlorobiaceae, Ancalochloris, Chlorobaculum, Chlorobium, Chloroherpeton, Clathrochloris, Pelodictyon,Prosthecochloris, Thermoflexia, Dehalococcoidia, Anaerolineae, Ardenticatenia, Caldilineae, Ktedonobacteria, Thermomicrobia, Chloroflexia, Chrysiogenetes, Chrysiogenales, Chrysiogenaceae, Chroococcales, Chroococcidiopsidales, Gloeobacterales, Nostocales, Oscillatoriales, Pleurocapsales, Spirulinales, Synechococcales, Incertae sedis), Deferribacterale, Deferribacteraceae, Deinococcales, Deinococcaceae, Trueperaceae, Thermales, Thermaceae, Dictyoglomales, Dictyoglomaceae, Elusimicrobia, Endomicrobia, Blastocatellia, Chitinispirillia, Chitinivibrio nia, Fibrobacteria, Bacilli, Bacillales, Lactobacillales, Clostridia, Clostridiales, Halanaerobiales, Natranaerobiales, Thermoanaerobacterales, Erysipelotrichia, Erysipelotrichales, Negativicutes, Selenomonadales,Thermolithobacteria, Fusobacteriia, Fusobacteriales, Leptotrichiaceae, Sebaldella, Sneathia, Streptobacillus, Leptotrichia, Fusobacteriaceae, Cetobacterium, Fusobacterium, Ilyobacter, Propionigenium, Psychrilyobacter, Longimicrobia, Gemmatimonadetes, Oligosphaeria, Lentisphaeria, Nitrospiria, Nitrospira Nitrospirales, Nitrospiraceae, Phycisphaerae, Planctomycetacia, Alphaproteobacteria, Betaproteobacteria, Hydrogenophilalia, Gammaproteobacteria, Acidithiobacillia, Deltaproteobacteria, Epsilonproteobacteria and Oligoflexia, Spirochaetia, Brachyspirales, Brachyspiraceae, Brevinematales, Brevinemataceae, Leptospirales Leptospiraceae, Spirochaetales, Borreliaceae, Spirochaetaceae, Sarpulinaceae, Synergistia, Synergistales, Synergistaceae, Mollicutes, Thermodesulfobacteria,Thermodesulfobacteriales, Thermodesulfobacteriaceae, Thermotogae, Kosmotogales, Kosmotogaceae, Mesoaciditogales, Mesoaciditogaceae, Petrotogales, Petrotogaceae, Thermotogales, Thermotogaceae, Fervidobacteriaceae, Candidatus Epixenosoma, Lentimonas, Methyloacida, Methylacidimicrobium, Methylacidiphilales, Spartobacteria, Opitutae, or Verrucomicrobiae. Individual genera and species within these classes, orders, and families can be selected for analysis using the methods and systems described herein.

[0103] In other embodiments, the cell can be a eukaryotic cell, including "normal" eukaryotic cells present in normal functioning tissues and "abnormal" eukaryotic cells present in a cancerous or abnormal state. In addition, the eukaryotic cell can be derived from a protozoan, a fungus, an animal, a plant, an algae, or other eukaryotic cells. The method and system may be particularly desirable for studying treatments for abnormal cell metabolism, fungal infections, the efficacy of cancer treatments, the efficacy of pesticide treatments, tissue repair states, and other states that can be monitored based on the amount of specific materials present in or transported into the cell.

[0104] Where the cell is a fungal cell, the fungal cell may be from one or more of the following phyla and subphyla: Blastocladiomycota, Chytridiomycota, Glomeromycota, Microsporidia, Neocallimastigomycota, Dikarya (including Deuteromycota), Ascomycota, Pezizomycotina, Saccharomyces, The subdivisions may include, but are not limited to, Saccharomycotina, Taphrinomycotina, Basidiomycota, Agaricomycotina, Pucciniomycotina, Ustilaginomycotina, Entomophthoromycotina, Kickxellomycotina, Mucoromycotina, or Zoopagomycotina. Exemplary classes, orders, and / or families of fungal cells that can be analyzed include, but are not limited to, those from the following: Blastocladiomycetes, Blastocladiales, Blastocladiaceae, Catenariaceae, Coelomomycetaceae, Physodermataceae, Sorochytriaceae, Chytridiomycetes, Chytridiales, Cladochytriales, Rhizophydiales, s), Polychytriales, Spizellomycetales, Rhizophlyctidales, Lobulomycetales, Gromochytriales, Mesochytriales, Synchytriales, Polyphagales, Monoblepharidomycetes, Monoblepharidales, Harpochytriales, Hyaloraphidiomycetes, Hyaloraphidiales,Glomeromycetes, Archaeosporales, Diversisporales, Glomerales, Paraglomerales, Nematophytales, Metchnikovellea, Metchnikovellida, Amphiacanthidae, Metchnikovellidae, Microsporea , Cougourdellidae, Facilisporidae, Heterovesiculidae, Myosporidae, Nadelsporidae, Neonosemoidiidae, Ordosporidae, Pseudonosematidae, Telomyxidae, Toxoglugeidae, Tubulinosematidae, Haplophasea, Chytridiop sida), Chytridiopsidae, Buxtehudiidae, Enterocytozoonidae, Burkeidae, Hesseidae, Glugeida, Glugeidae, Gurleyidae, Encephalitozoonidae, Abelsporidae, Tuzetiidae, Microfilidae, Uncariaceae ikaryonidae), Dihaplophasea, Meiodihaplophasida, Thelohanioidea, Thelohaniidae, Duboscqiidae, Janacekiidae, Pereziidae, Striatosporidae, Cylindrosporidae, Burenelloidea, Burenelloidea, Burenellidae,Amblyosporoidea, Amblyosporidae, Dissociodihaplophasida, Nosematoidea, Nosematidae, Ichthyosporidiidae, Caudosporidae, Pseudopleistophoridae, Mrazekiidae, Culicosporoidea, Culicosporidae ), Culicosporellidae, Golbergiidae, Spragueidae, Ovavesiculoidea, Ovavesiculidae, Tetramicridae, Rudimicrospora, Minisporea, Minisporida, Metchnikovellea, Metchnikovellida, Polaroplasta, P leistophoridea), Pleistophorida, Disporea, Unikaryotia, Diplokaryotia, Neocallimastigomycetes, Neocallimastigales, Neocallimastigaceae, Pezizomycotina, Arthoniomycetes, Coniocybomycetes, Dothideomycetes, Eurotiomycetes, Geoglosomycetes, Laboulbeniomycetes, Lecanoromycetes, Leotiomycetes, Lichinomycetes, Orbiliomycetes, Pezizomycetes, Sordariomycetes, Xylonomycetes, Lahmiales, Itchiclahmadion,Triblidiales, Saccharomycotina, Saccharomycetes, Taphrinomycotina, Archaeorhizomyces, Neolectomycetes, Pneumocystidomycetes, Schizosaccharomycetes, Taphrinomycetes, Arthoniomycetes, Coni ocybomycetes), Dothideomycetes, Eurotiomycetes, Geoglosomycetes, Laboulbeniomycetes, Lecanoromycetes, Leotiomycetes, Lichinomycetes, Orbiliomycetes, Pezizomycetes, Sordariomycetes, Xylonomycetes ), Lahmiales, Medeolariales, Triblidiales, Saccharomycetales, Ascoideaceae, Cephaloascaceae, German Debaryomycetaceae, Dipodascaceae, Endomycetaceae, Lipomycetaceae, Metschnikowiaceae, Phaffiaceae Phaffomycetaceae), Pichiaceae, Saccharomycetaceae, Saccharomycodaceae, Saccharomycopsidaceae, Trichomonascaceae, Archaeorhizomycetes, Neolectomycetes, Pneumocystidomycetes, Schizosaccharomycetes,Taphrinomycetes, Agaricomycotina, Pucciniomycotina, Ustilaginomycotina, Wallemiomycetes, Tremellomycetes, Dacrymycetes, Agaricomycetes, Agaricostilbomycetes, Atractiellomycetes, Classiculomycetes ycetes, Cryptomycocolacomycetes, Cystobasidiomycetes, Microbotryomycetes, Mixiomycetes, Pucciniomycetes, Tritirachiomycetes, Exobasidiomycetes, Ceraceosorales, Doassansiales, Entylomatales, Exobasidiomycetes obasidiales, Georgefischeriales, Microstromatales, Tilletiales, Ustilaginomycetes, Urocystales, Ustilaginales, Malasseziomycetes, Malassezioales, Moniliellomycetes, Moniliellales, Baclomycetes sidiobolomycetes, Neozygitomycetes, Entomophthoromycetes, Asellariales, Dimargaritales, Harpellales, Kickxellales, Mortierellomycetes, Mortierellales, Mucoromycetes, Mucorales, or Endogonales.Individual genera and species within orders and families are available for analysis using the methods and systems described herein.

[0105] Where the cell is a plant cell, the plant cell may be from one or more of the following phyla and subphyla: Nematophytes, Chlorophyta, Palmophyllales, Prasinophyceae, Nephroselmidophyceae, Pseudoscourfieldiales, Pyramimonadophyceae, Mamiellophyceae, Scourfieldiales, Pedinophyceae, Chlorophyceae, and dendrophyceae, Trebouxiophyceae, Ulvophyceae, Chlorophyceae, Streptophyta, Chlorokybophyta, Mesostigmatophyta, Klebsormidiophyta, Charophyta, Chaetosphaeridiales, Coleochaetophyta, Zygnematophyta or Embryophyta. Exemplary classes, orders, families, and genera of plant cells that can be analyzed include, but are not limited to, those from Nematothallus, Cosmochlaina, Nematophytaceae, Nematoplexus, Nematasketum, Prototaxites, Ulvophyceae, Trebouxiophyceae, Chlorophyceae, Chlorodendrophyceae, Mamiellophyceae, phyceae), Nephroselmidophyceae, Palmophyllales, Pedinophyceae, Prasinophyceae, Pseudoscourfieldiales, Pyramimonadophyceae, Scourfieldiales, Palmoclathrus, Palmophyllum, Verdigellas,Prasinococcales, Prasinophyceae, Incertae sedis, Pseudoscourfieldiales, Pyramimonadales, Nephoselmis, Pycnococcaceae, Scourfieldiaceae, Pedinomonas, Resultor, Marsupiomonas, Chlorochtridion tuberculatum, Chlorellales, Prasiolales, Trebouxiales, Bryopsidales, Cladophorales, Dasycladales, Oltmannsiellopsidales, Scotinosphaerales, Trentepohliales, Ulotrichales, Ulvales, Chaetopeltidales, Chaetophorales, Chlamydomonadales, Chlorococcales, Chlorocystidales, Microsporales, Oedogoniales, Phaeophiales lales), Sphaeropleales, Tetrasporales, Chlorokybus, Mesostigmatophyceae, Entransia, Hormidiella, Interfilum, Klebsormidium, Mesostigmatophyceae, Klebsormidiophyceae, Zygnematophyceae, Zygnematales, Desmidiales, Charophyceae, Charales, Chlorokybophyceae, Coleochaetales, Polychaetophora,Chaetosphaeridium, Coleochaetophyceae, Zygnematales, Desmidiales, Bryophytes, Marchantiophyta, Bryophyta, Anthocerotophyta, Horneophytopsida, Tracheophytes, Rhyniophyta, Zosterophyllophyta, Lycopodiophyta, Trimerophytophyta, Pteridophyta, Spermatophytes, Pteridospermatophyta, Pinophyta, Cycadophyta, Ginkgophyta, Gnetophyta, or Magnoliophyta. Individual species within these classes, orders, families, and genera can be selected for analysis using the methods and systems described herein.

[0106] In some examples, one or more structures within a plant organelle can be imaged using the methods and systems described herein. For example, a plant organelle can include, but is not limited to, a plant nucleus, a nuclear membrane, a nuclear envelope, an endoplasmic reticulum, a ribosome, a mitochondria, a vacuole, a chloroplast, a cell membrane, or a cell wall. The plant organelle can be separated from other materials of the cell so that the metabolism and / or function of the isolated plant organelle can be monitored during imaging.

[0107] In the case where the cell is an animal cell, the animal cell can be an embryonic stem cell, an adult stem cell, a tissue-specific stem cell, a mesenchymal stem cell, an induced pluripotent stem cell, an epithelial tissue cell, a connective tissue cell, a muscle tissue cell or a neural tissue cell. The animal cell can be derived from the ectoderm, the endoderm or the mesoderm. Cells derived from the ectoderm include but are not limited to skin cells, anterior pituitary cells, peripheral nervous system cells, neuroendocrine cells, teeth, eye cells, central nervous system cells, ependymal cells and pineal gland cells. Cells derived from the endoderm include but are not limited to respiratory cells, gastric cells, intestinal cells, hepatocytes, gallbladder cells, exocrine pancreatic cells, islet cells, thyroid cells and urothelial cells. Cells derived from the mesoderm include, but are not limited to, osteochondrogenic precursor cells, myofibroblasts, angioblasts, stromal cells, macula densa cells, interstitial cells, telocytes, podocytes, Sertoli cells, Leydig cells, granulocytes, thrombus cells, germ cells, hematopoietic stem cells, lymphoid cells, myeloid cells, endothelial progenitor cells, endothelial colony-forming cells, endothelial stem cells, angioblasts / mesoangioblasts, pericytes, and mural cells.

[0108] In some instances, the animal cell is typically a mammalian cell, such as, for example, a human cell, a dog cell, a horse cell, a cat cell, a cow cell, and other animal cells of one or more of the following mammalian orders: Artiodactyls, Carnivora, Cetacea, Chiroptera, Dermoptera, Edentata, Hyrax, Insectivora, Lagomorpha, Marsupials, Monotremes, Perissodactyls, Pholidota, Pinnipeds, Primates, Proboscidea, Rodentia, Sirenia, and Tubulodonta. In some instances, the mammalian cell may be from the suborder Prosimian or the family Simian. In other instances, the mammalian cell may be from one or more of the following: Lagomorpha, Lemuroidea, Omomyiformes, and Tarsiomorpha. In other instances, the mammalian cell may be from one or more of the stem-nosed monkeys or the order Anthropoides. In some instances, the mammalian cell may be from the family Hominidae or the genus Homo, such as a human cell.

[0109] In some cases, the methods and systems described herein can also be used to image cancerous animal cells to evaluate the efficacy of treatment with a particular agent or material. Although the exact agent may vary depending on the specific type of cancer to be treated, it is expected that the agent causes cancer cell death in some way. Exemplary cancer types whose cells can be imaged include, but are not limited to, acute lymphoblastic leukemia (ALL), acute myeloid leukemia (AML), adrenocortical carcinoma, AIDS-related cancer Kaposi's sarcoma (soft tissue sarcoma), AIDS-related lymphoma (lymphoma), primary CNS lymphoma (lymphoma), anal cancer, appendix cancer, astrocytoma, atypical teratoid / rhabdoid tumor, basal cell carcinoma of the skin, bile duct cancer, bladder cancer, bone cancer, brain tumor, breast cancer, bronchial tumor, Burkitt's lymphoma, carcinoid tumor, unknown primary cancer, cardiac (heart) tumor, atypical Teratoid / rhabdoid tumors, embryonal tumors, germ cell tumors, primary CNS lymphomas, cervical cancer, childhood cancer, bile duct cancer, chordoma, chronic lymphocytic leukemia (CLL), chronic myeloid leukemia (CML), chronic myeloproliferative neoplasms, colorectal cancer, craniopharyngioma, cutaneous T-cell lymphoma, ductal carcinoma, endometrial cancer, uterine cancer, ependymoma, esophageal cancer, olfactory neuroblastoma, Ewing's sarcoma, extracranial germ cell tumors, extragonadal germ cell tumors, eye cancer, intraocular melanoma, retinoblastoma, fallopian tube cancer, fibrous histiocytoma of bone, gallbladder cancer,Gastric cancer, gastrointestinal carcinoid tumors, gastrointestinal stromal tumors, germ cell tumors, extracranial germ cell tumors in children, extragonadal germ cell tumors, ovarian germ cell tumors, testicular cancer, gestational trophoblastic disease, hairy cell leukemia, head and neck cancer, hepatocellular (liver) cancer, histiocytosis, Hodgkin lymphoma, hypopharyngeal bone cancer, head and neck cancer, intraocular melanoma, islet cell tumors, Kaposi's sarcoma, renal cell carcinoma, Langerhans cell histiocytosis, laryngeal cancer, leukemia, lip and oral cancer, liver cancer, lymphoma, male breast cancer, malignant fibrous histiocytoma and osteosarcoma of bone, melanoma, childhood melanoma Pigmented tumors, Merkel cell carcinoma, Mesothelioma, Metastatic cancer, Occult primary metastatic squamous neck cancer (head and neck cancer), Midline cancer with NUT gene alterations, Oral cancer (head and neck cancer), Multiple endocrine neoplasms syndrome, Multiple myeloma / plasma cell neoplasms, Mycosis fungoides (lymphoma), Myelodysplastic syndrome, Myelodysplastic / myeloproliferative neoplasms, Chronic myeloid leukemia (CML), Acute myeloid leukemia (AML), Myeloproliferative neoplasms, Nasal cavity and paranasal sinus cancer (head and neck cancer), Nasopharyngeal cancer (head and neck cancer), Neuroblastoma, Non-Hodgkin lymphoma, Non-small cell Lung cancer, oral cancer, lip and oral cavity cancer and oropharyngeal cancer, osteosarcoma and malignant fibrous histiocytoma of bone, ovarian cancer, pancreatic cancer, pancreatic neuroendocrine tumors (islet cell tumors), papilloma, paraganglioma, paranasal sinus and nasal cavity cancer, parathyroid cancer, penile cancer, pharyngeal cancer (head and neck cancer), pheochromocytoma, pituitary tumor, plasma cell neoplasm / multiple myeloma, pleuropulmonary blastoma, primary central nervous system (CNS) lymphoma, primary peritoneal cancer, prostate cancer, rectal cancer, recurrent cancer, renal cell carcinoma, retinoblastoma, rhabdomyosarcoma, salivary gland cancer, childhood rhabdomyosarcoma, Hemangiomas in children, Ewing's sarcoma (bone cancer), Kaposi's sarcoma (soft tissue sarcoma), osteosarcoma (bone cancer), uterine sarcoma, Sezary syndrome (lymphoma), skin cancer, small cell lung cancer, small intestine cancer, soft tissue sarcoma, squamous cell carcinoma of the skin, occult primary squamous neck cancer, metastatic (head and neck cancer), gastric cancer, T-cell lymphoma, testicular cancer, nasopharyngeal cancer, oropharyngeal cancer, hypopharyngeal cancer, thymoma and thymic cancer, thyroid cancer, transitional cell carcinoma of the renal pelvis and ureter, ureter and renal pelvis cancer, urethral cancer, uterine cancer, uterine sarcoma, vaginal cancer, hemangioma, vulvar cancer, Wilms' tumor and other forms of cancer. ,

[0110] In some cases, the organelles of an animal cell are separated from other components of the animal cell, and then the imaging methods and systems described herein are used to determine the phenotype of the organelles and the material content of the organelles of the animal cell. If desired, the organelle phenotype (or biological response of the organelle) can then be associated with a specific material or component of the organelle. For example, the isolated organelles can include but are not limited to the nucleus, nuclear membrane, microtubules, microfilaments, endoplasmic reticulum, sarcoplasmic reticulum, ribosomes, mitochondria, vacuoles, lysosomes, cell membranes, or other organelles present in animal cells.

[0111] In some instances, the methods and systems described herein can also be used to image viruses, which may be present inside or outside a cell or both. For example, the efficacy of an antiviral agent binding to a viral protein capsid can be assessed using the imaging methods and systems described herein. When analyzing viruses, the virus can be, for example, a double-stranded DNA virus, a single-stranded DNA virus, a double-stranded RNA virus, a positive single-stranded RNA virus, a negative single-stranded RNA virus, a single-stranded RNA retrovirus (retrovirus), or a double-stranded DNA retrovirus. Various specific viruses include, but are not limited to, Papovaviridae, Adenoviridae, Herpesviridae, Herpesvirales, Ascoviridae, Ampullaviridae, Asfarviridae, Baculoviridae, Fuselloviridae, Globuloviridae, Guttaviridae, Hytrosaviridae, Iridoviridae, Lipothrixviridae, Nimaviridae, Poxviridae, Tectiviridae, Corticoviridae, Sulfolobus, lfolobus), Caudovirales, Corticoviridae, Tectiviridaea, Ligamenvirales, Ampullaviridae, Bicaudaviridae, Clavaviridae, Fuselloviridae, Globuloviridae, Guttaviridae, Turriviridae, Ascovirus, Baculovirus, Hytrosaviridae, Iridoviridae, Polydnaviruses, Mimiviridae, Marseillevirus, Megavirus,Mavirus virophage, Sputnik virophage, Nimaviridae, Phycodnaviridae, Pleolipoviruses, Plasmaviridae, Pandoraviridae, Dinodnavirus, Rhizidiovirus, Salterprovirus, Sphaerolipoviridae, Anelloviridae e), Bidnaviridae, Circoviridae, Geminiviridae, Genomoviridae, Inoviridae, Microviridae, Nanoviridae, Parvoviridae, Spiraviridae, Amalgaviridae, Birnaviridae, Chloroviridae hrysoviridae), Cystoviridae, Endornaviridae, Hypoviridae, Megabirnaviridae, Partitiviridae, Picobirnaviridae, Quadriviridae, Reoviridae, Totiviridae, Nidovirales, small RNA Picornavirales, Tymovirales, Mononegavirales, Bornaviridae, Filoviridae, Mymonaviridae, Nyamiviridae, Paramyxoviridae, Pneumoviridae, Rhabdoviridae, Sunviridae,Anphevirus, Arlivirus, Chengtivirus, Crustavirus, Wastrivirus, Bunyavirales, Feraviridae, Fimoviridae, Hantaviridae, Jonviridae, Nairoviridae, Peribunyaviridae, Phasmaviridae, Phenuiviridae, Tospoviridae, Arenaviridae, Ophioviridae, Orthomyxoviridae, Deltavirus, Taastrup virus, Alpharetrovirus, Avianleukosis virus; Rous sarcoma virus virus), Betaretrovirus, Mouse mammary tumor virus, Gammaretrovirus, Murine leukemia virus, Felineleukemia virus, Bovine leukemia virus, Human T-lymphotropic virus, Epsilonretrovirus, Walleye dermal sarcoma virus, Lentivirus, Human immunodeficiency virus 1, Simian and Felineimmunodeficiency viruses, Spumavirus, Simian foamy virus foamyvirus), Orthoretrovirinae, Spumaretrovirinae, Metaviridae, Pseudoviridae,Retroviridae, Hepadnaviridae, or Caulimoviridae. Various species within these classes, orders, families, and genera can be selected for analysis using the methods and systems described herein.

[0112] In certain configurations, the imaging methods and systems can detect one or more markers present on antibodies that specifically bind to sites on individual cells. Antibodies are typically polypeptides comprising multiple amino acids coupled to each other by peptide bonds. Antibodies may include immunoglobulin chains or fragments thereof, which include at least one immunoglobulin variable domain sequence. The term antibody includes, for example, monoclonal antibodies (including full-length antibodies with immunoglobulin Fc regions), polyclonal antibodies, and other polypeptides that can specifically bind to one or more sites on cells. In one embodiment, the antibody molecule includes a full-length antibody or a full-length immunoglobulin chain. In another embodiment, the antibody molecule includes an antigen-binding or functional fragment of a full-length antibody, or a full-length immunoglobulin chain. The amino acids of the antibody may be natural or synthetic, and include both amino and acid functional groups, and may be included in polymers of naturally occurring amino acids. Exemplary amino acids include naturally occurring amino acids, analogs, derivatives, and homologs thereof; amino acid analogs with variant side chains; and all stereoisomers of any of the above. D- or L- optical isomers of amino acids and peptide mimetics may be used. The term antibody also includes complete molecules and functional fragments thereof. The constant region of an antibody may be altered, e.g., mutated, to alter a property of the antibody (e.g., to increase or decrease one or more of Fc receptor binding, antibody glycosylation, the number of cysteine ​​residues, effector cell function, or complement function).

[0113] If desired, conservative amino acid residue replacement can be used to produce antibodies, wherein the amino acid residue is replaced by an amino acid residue with a similar side chain. In view of the benefits of the present disclosure, those skilled in the art will select amino acid residue families with similar side chains. These families include amino acids with basic side chains (e.g., lysine, arginine, histidine), acidic side chains (e.g., aspartic acid, glutamic acid), uncharged polar side chains (e.g., glycine, asparagine, glutamine, serine, threonine, tyrosine, cysteine), non-polar side chains (e.g., alanine, valine, leucine, isoleucine, proline, phenylalanine, methionine, tryptophan), β-branched side chains (e.g., threonine, valine, isoleucine) and aromatic side chains (e.g., tyrosine, phenylalanine, tryptophan, histidine). The exact length of the antibody can vary as desired. In some cases, the synthesized polypeptides can be coupled to each other to form antibodies. The antibody can be linear or branched, it can include modified amino acids and can be interrupted by non-amino acids. The antibody may be modified, for example, by disulfide bond formation, glycosylation, lipidation, acetylation, phosphorylation, or any other manipulation such as conjugation to a label or marker component. The polypeptide may be isolated from a natural source, may be produced by recombinant techniques from a eukaryotic or prokaryotic host, or may be the product of a synthetic procedure.

[0114] Antibodies can be expressed in a host system by inserting a suitable nucleic acid sequence into a host organism. The terms "nucleic acid," "nucleic acid sequence," "nucleotide sequence," or "polynucleotide sequence" and "polynucleotide" are used interchangeably. These terms generally refer to a polymeric form of nucleotides (deoxyribonucleotides or ribonucleotides or their analogs) of any length. Polynucleotides can be single-stranded or double-stranded, and if single-stranded, can be coding strands or non-coding (antisense) strands. Polynucleotides can include modified nucleotides, such as methylated nucleotides and nucleotide analogs. The nucleotide sequence may be interrupted by non-nucleotide components. Polynucleotides can be further modified after polymerization, such as by being conjugated to a labeling component. Nucleic acid can be a recombinant polynucleotide, or a polynucleotide of genomic, cDNA, semisynthetic or synthetic origin that does not exist in nature or is connected to another polynucleotide in a non-natural arrangement.

[0115] The antibody can be separated from the host or other expression or production organism. As used herein, the term "isolated" refers to material removed from its original or natural environment (e.g., if it is naturally occurring, the natural environment). For example, a naturally occurring polynucleotide or polypeptide present in a living animal is not isolated, but the same polynucleotide or polypeptide separated from some or all coexisting materials in the natural system by human intervention is isolated. Such polynucleotides can be part of a vector and / or such polynucleotides or polypeptides can be part of a composition and still be isolated because such vector or composition is not part of the environment in which it is found in nature.

[0116] In one configuration, the antibody molecule binds to an animal cell, such as a mammalian cell. For example, the antibody molecule specifically binds to an epitope on a mammalian cell, such as a linear or conformational epitope (e.g., an epitope described herein). In some embodiments, the antibody molecule binds to one or more extracellular Ig-like domains present on a mammalian cell, such as the first, second, third, or fourth extracellular Ig-like domain of a specific epitope.

[0117] In one embodiment, the antibody molecule is a monospecific antibody molecule and binds a single epitope, e.g., a monospecific antibody molecule having multiple immunoglobulin variable domain sequences, each immunoglobulin variable domain sequence binding to the same epitope.

[0118] In another embodiment, the antibody molecule is a multispecific antibody molecule, for example, it includes multiple immunoglobulin variable domain sequences, wherein the first immunoglobulin variable domain sequence of multiple immunoglobulin variable domain sequences has binding specificity to the first epitope, and the second immunoglobulin variable domain sequence of multiple immunoglobulin variable domain sequences has binding specificity to the second epitope. In one embodiment, the first and second epitopes are on the same antigen, for example, the same protein (or the subunit of a multimeric protein). In other cases, the first and second epitopes overlap. In one embodiment, the first and second epitopes do not overlap. In another embodiment, the first and second epitopes are on different antigens, for example, different proteins (or different subunits of a multimeric protein). In another embodiment, the multispecific antibody molecule includes the third, fourth or fifth immunoglobulin variable domain. In one embodiment, the multispecific antibody molecule is a bispecific antibody molecule, a trispecific antibody molecule or a tetraspecific antibody molecule.

[0119] In some examples, multispecific antibody molecules can be bispecific antibody molecules. Bispecific antibodies have specificity for no more than two antigens or two binding sites. The bispecific antibody molecule is characterized by a first immunoglobulin variable domain sequence with binding specificity to a first epitope and a second immunoglobulin variable domain sequence with binding specificity to a second epitope. In another embodiment, the first and second epitopes are on the same antigen, such as the same protein (or subunit of a multimeric protein). In one embodiment, the first and second epitopes overlap. In another embodiment, the first and second epitopes do not overlap. In one embodiment, the first and second epitopes are on different antigens, such as different proteins (or different subunits of a multimeric protein). In other embodiments, the bispecific antibody molecule includes a heavy chain variable domain sequence and a light chain variable domain sequence with binding specificity to the first epitope, and a heavy chain variable domain sequence and a light chain variable domain sequence with binding specificity to the second epitope. In other cases, the bispecific antibody molecule includes a half antibody with binding specificity to the first epitope and a half antibody with binding specificity to the second epitope. In another embodiment, the bispecific antibody molecule comprises a half antibody or fragment thereof having binding specificity for a first epitope and a half antibody or fragment thereof having binding specificity for a second epitope. In one embodiment, the bispecific antibody molecule comprises a single chain variable fragment (scFv) or fragment thereof having binding specificity for a first epitope and a scFv or fragment thereof having binding specificity for a second epitope.

[0120] In other configurations, antibody molecules include double antibodies and single-chain molecules, as well as antigen-binding fragments of antibodies (e.g., Fab, F(ab')2, and Fv). For example, an antibody molecule may include a heavy (H) chain variable domain sequence (abbreviated herein as VH) and a light (L) chain variable domain sequence (abbreviated herein as VL). In one embodiment, an antibody molecule includes or consists of a heavy chain and a light chain (referred to herein as a half antibody). In another example, an antibody molecule includes two heavy (H) chain variable domain sequences and two light (L) chain variable domain sequences, thereby forming two antigen-binding sites, such as Fab, Fab', F(ab')2, Fc, Fd, Fd', Fv, single-chain antibodies (e.g., scFv), single variable domain antibodies, double antibodies (Dab) (divalent and bispecific) and chimeric (e.g., humanized) antibodies, which can be produced by modifying complete antibodies or synthesized de novo using recombinant DNA technology. Without being limited by any one configuration, these functional antibody fragments generally retain the ability to selectively bind to their respective antigens or receptors. Antibodies and antibody fragments can be from any class of antibodies, including but not limited to IgG, IgA, IgM, IgD and IgE, and from any subclass of antibodies (e.g., IgG1, IgG2, IgG3 and IgG4). The preparation of antibody molecules can be monoclonal or polyclonal. Antibody molecules can also be human, humanized, CDR-transplanted or in vitro produced antibodies. Antibodies can have a heavy chain constant region selected from, for example, IgG1, IgG2, IgG3 or IgG4. Antibodies can also have a light chain selected, for example, κ or λ. The term "immunoglobulin" (Ig) is used interchangeably with the term "antibody" herein.

[0121] Examples of antigen-binding fragments of antibody molecules include: (i) Fab fragments, monovalent fragments consisting of VL, VH, CL and CH1 domains; (ii) F(ab')2 fragments, bivalent fragments comprising two Fab fragments linked by a disulfide bridge at the hinge region; (iii) Fd fragments consisting of VH and CH1 domains; (iv) Fv fragments consisting of the VL and VH domains of a single arm of an antibody, (v) diabody (dAb) fragments consisting of a VH domain; (vi) camel or camelid variable domains, (vii) single-chain Fv (scFv), see, e.g., Bird et al. (1988) Science 242:423-426; and Huston et al. (1988) Proc. Natl. Acad. Sci. USA 85:5879-5883, (viii) single-domain antibodies. These antibody fragments are obtained using conventional techniques known to those with skill in the art, and the fragments are screened for utility in the same manner as are intact antibodies.

[0122] In some examples, antibodies that can be used for the imaging methods and systems described herein may also be single-domain antibodies. Single-domain antibodies may include antibodies whose complementary determining regions are part of single-domain polypeptides. Examples include, but are not limited to, heavy chain antibodies, antibodies naturally free of light chains, single-domain antibodies derived from conventional 4-chain antibodies, engineered antibodies, and single-domain scaffolds other than those derived from antibodies. Single-domain antibodies may be any antibody or any future single-domain antibody present in the art. Single-domain antibodies may be derived from any species, including, but not limited to, mice, humans, primates, camels, llamas, fish, sharks, goats, rabbits, and cattle. Single-domain antibodies are naturally occurring single-domain antibodies, referred to as heavy chain antibodies free of light chains. Such single-domain antibodies are described, for example, in WO94 / 04678. For clarity, the variable domains derived from natural heavy chain antibodies free of light chains are referred to herein as VHH or nanobodies, to distinguish them from the conventional VH of four-chain immunoglobulins. Such VHH molecules may be derived from antibodies produced in species of the Camelidae family, such as camels, llamas, dromedaries, alpacas and guanacos. Other species besides the Camelidae family may produce heavy chain antibodies that are naturally free of light chains, and such VHHs are within the scope of the present invention.

[0123] In certain instances, the VH and VL regions can be subdivided into hypervariable regions, called "complementarity determining regions" (CDRs), interspersed with more conserved regions, called "framework regions" (FRs or FWs). The extent of the framework regions and CDRs has been precisely defined by a variety of methods (see Kabat, EA et al. (1991) Sequences of Proteins of Immunological Interest, Fifth Edition, U.S. Department of Health and Human Services, National Institutes of Health, Publication No. 91-3242; Chothia, C. et al. (1987) J. Mol. Biol. 196:901-917; and the AbM definition used by Oxford Molecular's AbM antibody modeling software. See generally, for example, Protein Sequence and Structure Analysis of Antibody Variable Domains, Antibody Engineering Lab Manual (Editors: Duebel, S. and Kontermann, R., Springer-Verlag, Heidelberg). As used herein, the terms "complementarity determining region" and "CDR" refer to the amino acid sequence within the variable region of an antibody that confers antigen specificity and binding affinity. Typically, each heavy chain variable region has three CDRs (HCDR1, HCDR2, HCDR3), and each light chain variable region has three CDRs (LCDR1, LCDR2, LCDR3). The precise amino acid sequence boundaries of a given CDR can be determined using any of a number of well-known schemes, including Kabat et al. (1991), Sequences of Proteins of Immunological Interest, Fifth Edition, Public Health Service, National Institutes of Health, Bethesda, Maryland ("Kabat" numbering scheme), AL-Lazikani et al., (1997) JMB 273, 927-948 ("Chothia" numbering scheme) described those. As used herein, CDRs defined according to the "Chothia" numbering scheme are sometimes also referred to as "hypervariable loops".

[0124] For example, under Kabat, the CDR amino acid residues in the heavy chain variable domain (VH) are numbered 31-35 (HCDR1), 50-65 (HCDR2), and 95-102 (HCDR3); and the CDR amino acid residues in the light chain variable domain (VL) are numbered 24-34 (LCDR1), 50-56 (LCDR2), and 89-97 (LCDR3). Under Chothia, the CDR amino acid residues in VH are numbered 26-32 (HCDR1), 52-56 (HCDR2), and 95-102 (HCDR3); and the amino acid residues in VL are numbered 26-32 (LCDR1), 50-52 (LCDR2), and 91-96 (LCDR3). By combining the CDR definitions of Kabat and Chothia, CDRs consist of amino acid residues 26-35 (HCDR1), 50-65 (HCDR2) and 95-102 (HCDR3) in human VH and amino acid residues 24-34 (LCDR1), 50-56 (LCDR2) and 89-97 (LCDR3) in human VL. Unless otherwise specified, an antibody molecule may comprise any combination of one or more Kabat CDRs and / or Chothia hypervariable loops. Under all definitions, each VH and VL typically comprises three CDRs and four FRs, arranged in the following order from amino terminus to carboxyl terminus: FR1, CDR1, FR2, CDR2, FR3, CDR3, FR4. An immunoglobulin variable domain sequence refers to an amino acid sequence that can form an immunoglobulin variable domain structure. For example, the sequence may include all or part of the amino acid sequence of a naturally occurring variable domain. For example, the sequence may include or exclude one, two or more N- or C-terminal amino acids, or may contain other changes that are compatible with the formation of a protein structure.

[0125] In some instances, antibodies generally include one or more antigen binding sites or epitopes. The term antigen binding site refers to a part of an antibody molecule that includes a determinant that forms an interface that is combined with a cell site or its epitope. For proteins (or protein mimics), an antigen binding site generally includes one or more loops (at least four amino acids or amino acid mimics) that form an interface that is combined with a cell polypeptide. Typically, the antigen binding site of an antibody molecule includes at least one or two CDRs and / or hypervariable loops, or more typically includes at least three, four, five or six CDRs and / or hypervariable loops.

[0126] In some instances, different antibodies can compete with each other to bind to a specific epitope or site of a cell. The terms "competition" or "cross competition" are used interchangeably herein, referring to the ability of an antibody molecule to interfere with the binding of another antibody molecule. Interference to binding can be direct or indirect (e.g., carried out by the allosteric regulation of an antibody molecule or a target). The degree to which an antibody molecule can interfere with the binding of another antibody molecule to a target and whether it can therefore be called competition can be determined using competitive binding assays, such as FACS assays, ELISA or BIACORE assays. In certain embodiments, competitive binding assays are quantitative competitive assays. In certain embodiments, when the binding of the first antibody molecule to a target in a competitive binding assay (e.g., competitive assays as described herein) is reduced by 10% or more (e.g., 20% or more, 30% or more, 40% or more, 50% or more, 55% or more, 60% or more, 65% or more, 70% or more, 75% or more, 80% or more, 85% or more, 90% or more, 95% or more, 98% or more, 99% or more), the first antibody molecule is claimed to compete with the second antibody molecule for binding to the target. As used herein, the term "epitope" refers to the antigenic portion that specifically interacts with an antibody molecule. Such portions are referred to herein as epitope determinants, typically including elements such as amino acid side chains or sugar side chains or being a part of elements such as amino acid side chains or sugar side chains. Epitope determinants can be defined by methods known in the art or disclosed herein, for example, by crystallography or by hydrogen-deuterium exchange. At least one or some portions that specifically interact with epitope determinants on an antibody molecule are typically located in a CDR. Typically, an epitope has a specific three-dimensional structural feature. An epitope may also include a specific charge feature. Some epitopes are linear epitopes, while other epitopes are conformational epitopes.

[0127] In some examples, the antibodies useful in the imaging methods described herein can be monoclonal antibodies. As used herein, the term "monoclonal antibody" or "monoclonal antibody composition" refers to a preparation of antibody molecules of a single molecular composition. A monoclonal antibody composition exhibits a single binding specificity and affinity for a specific epitope. Monoclonal antibodies can be prepared by hybridoma technology or by methods that do not use hybridoma technology (e.g., recombinant methods).

[0128] In other examples, the antibody molecule can be a polyclonal or monoclonal antibody. In other embodiments, the antibody can be produced recombinantly, for example by phage display or by combinatorial methods. For example, phage display and combinatorial methods for generating antibodies are known in the art (as described, for example, in Ladner et al., U.S. Pat. No. 5,223,409; Kang et al., International Publication No. WO 92 / 18619; Dower et al., International Publication No. WO 91 / 17271; Winter et al., International Publication No. WO 92 / 20791; Markland et al., International Publication No. WO 92 / 15679; Breitling et al., International Publication No. WO 93 / 01288; McCafferty et al., International Publication No. WO 92 / 01047; Garrard et al., International Publication No. WO 92 / 09690; Ladner et al., International Publication No. WO 90 / 02809; Fuchs et al., (1991) Bio / Technology 9:1370-1372; Hay et al. (1992) Hum Antibod Hybridomas 3:81-85; Huse et al. (1989) Science 246:1275-1281; Griffiths et al. (1993) EMBO J 12:725-734; Hawkins et al. (1992) J Mol Biol 226:889-896; Clackson et al. (1991) Nature 352:624-628; Gram et al. (1992) PNAS 89:3576-3580; Garrad et al. (1991) Bio / Technology 9:1373-1377; Hoogenboom et al. (1991) Nucleic Acids Res. and Barbas et al. (1991) Proceedings of the National Academy of Sciences (PNAS) 88:7978-7982, the entire contents of which are incorporated herein by reference).

[0129] In one embodiment, the antibody is a fully human antibody (e.g., an antibody prepared in a mouse that has been genetically engineered to produce antibodies from human immunoglobulin sequences), or a non-human antibody, such as a rodent (mouse or rat), goat, primate (e.g., monkey), camel antibody. Preferably, the non-human antibody is a rodent (mouse or rat antibody). Methods for producing rodent antibodies are known in the art. Transgenic mice carrying human immunoglobulin genes can be used instead of mouse systems to produce human monoclonal antibodies. Splenocytes from these transgenic mice immunized with the antigen of interest are used to generate hybridomas that secrete human monoclonal antibodies (mAbs) with specific affinity for epitopes from human proteins (see, e.g., Wood et al., International Application WO 91 / 00906; Kucherlapati et al., PCT Publication WO 91 / 10741; Lonberg et al., International Application WO 92 / 03918; Kay et al., International Application 92 / 03917; Lonberg, N. et al. 1994 Nature 368:856-859; Green, LL et al. 1994 Nature Genet, 7:13-21; Morrison, SL et al. 1994 Proc. Natl. Acad. Sci. USA 81:6851-6855; Bruggeman et al. 1993 Year Rev. Immunol. The antibody may be one in which the variable region or a portion thereof, such as a CDR, is produced in a non-human organism, such as a rat or mouse. Chimeric, CDR-grafted and humanized antibodies may be used. The antibody is produced in a non-human organism, such as a rat or mouse, and then modified, for example, in the variable framework or constant region to reduce antigenicity in humans.Chimeric antibodies can be produced by recombinant DNA techniques known in the art (see Robinson et al., International Patent Publication No. PCT / US86 / 02269; Akira et al., European Patent Application No. 184,187; Taniguchi, M., European Patent Application No. 171,496; Morrison et al., European Patent Application No. 173,494; Neuberger et al., International Application No. WO 86 / 01533; Cabilly et al., U.S. Patent No. 4,816,567; Cabilly et al., European Patent Application No. 125,023; Better et al. (1988 Science 240:1041-1043); Liu et al. (1987) Proceedings of the National Academy of Sciences (PNAS) 84:3439-3443; Liu et al. 1987, J. Immunol. 》139:3521-3526; Sun et al. (1987) "Proceedings of the National Academy of Sciences (PNAS)" 84:214-218; Nishimura et al., 1987, "Canc. Res." 47:999-1005; Wood et al. (1985) "Nature" 314:446-449; and Shaw et al., 1988, "Journal of the National Cancer Institute (J. Natl Cancer Inst.)" 80:1553-1559).

[0130] In certain instances, humanized or CDR-grafted antibodies will have at least one or two but usually all three acceptor CDRs (of heavy and or light immunoglobulin chains) replaced by donor CDRs. Antibodies may be replaced by at least a portion of non-human CDRs or only a portion of CDRs may be replaced by non-human CDRs. It may only be necessary to replace the number of CDRs required for the humanized antibody to bind to a specific epitope. Preferably, the donor will be a rodent antibody, such as a rat or mouse antibody, and the acceptor will be a human framework or a human consensus framework. Typically, the immunoglobulin providing the CDR is referred to as a "donor," and the immunoglobulin providing the framework is referred to as an "acceptor." In one embodiment, the donor immunoglobulin is a non-human (e.g., rodent) immunoglobulin. The acceptor framework is a naturally occurring (e.g., human) framework or a consensus framework, or a sequence that is about 85% or higher, preferably 90%, 95%, 99% or higher, identical thereto. As used herein, the term "consensus sequence" refers to a sequence formed by the most frequently occurring amino acids (or nucleotides) in a family of related sequences (see, e.g., Winnaker, From Genes to Clones (Verlagsgesellschaft, Weinheim, Germany 1987)). In a protein family, each position in the consensus sequence is occupied by the most frequently occurring amino acid at that position in the family. If two amino acids occur with the same frequency, either one may be included in the consensus sequence. "Consensus framework" refers to the framework region in the consensus immunoglobulin sequence.

[0131] In some examples, antibodies can be humanized by methods known in the art (see, e.g., Morrison, SL, 1985, Science 229: 1202-1207, Oi et al., 1986, BioTechniques 4: 214, and Queen et al., U.S. Pat. No. 5,585,089, U.S. Pat. No. 5,693,761, and U.S. Pat. No. 5,693,762, the entire contents of which are hereby incorporated by reference). Humanized or CDR-grafted antibodies can be produced by CDR-grafting or CDR replacement, in which one, two or all CDRs of an immunoglobulin chain can be replaced. See, e.g., U.S. Pat. No. 5,225,539; Jones et al. 1986 Nature 321:552-525; Verhoeyan et al. 1988 Science 239:1534; Beidler et al. 1988 J. Immunol. 141:4053-4060; Winter U.S. Pat. No. 5,225,539, the entire contents of which are expressly incorporated herein by reference. Winter describes a CDR-grafting method that can be used to prepare humanized antibodies (British Patent Application GB 2188638A, filed March 26, 1987; Winter U.S. Pat. No. 5,225,539), the contents of which are expressly incorporated by reference. Humanized antibodies in which specific amino acids have been replaced, deleted or added can also be used. Criteria for selecting amino acids from donors are described in, e.g., U.S. Pat. No. 5,585,089, e.g., columns 12-16 of U.S. Pat. No. 5,585,089, e.g., columns 12-16 of U.S. Pat. No. 5,585,089, the contents of which are incorporated herein by reference. EP 519596A1, published on December 23, 1992 by Padlan et al., describes other techniques for humanizing antibodies.

[0132] In some instances, the antibody molecule can be a single-chain antibody. Single-chain antibodies (scFV) can be engineered (see, for example, Colcher, D. et al. (1999) "Annual of the New York Academy of Sciences (Ann NY Acad Sci)" 880:263-80; and Reiter, Y. (1996) "Clin Cancer Res" 2:245-52). Single-chain antibodies can be dimerized or polymerized to produce multivalent antibodies with specificity for different epitopes of the same target protein. In other embodiments, the antibody molecule has a heavy chain constant region selected from, for example, IgG1, IgG2, IgG3, IgG4, IgM, IgA1, IgA2, IgD and IgE heavy chain constant regions; in particular, selected from, for example, IgG1, IgG2, IgG3 and IgG4 (e.g., people) heavy chain constant regions. In another embodiment, the antibody molecule has a light chain constant region selected from, for example, κ or λ (e.g., people) light chain constant regions. The constant region can be changed, for example, mutated, to change the properties of the antibody (e.g., to increase or decrease one or more of the following: Fc receptor binding, antibody glycosylation, the number of cysteine ​​residues, effector cell function, and / or complement function). In one embodiment, the antibody has: effector function, and can repair complement. In other embodiments, the antibody does not recruit effector cells or repair complement. In another embodiment, the ability of the antibody to bind to the Fc receptor is reduced or lost. For example, it is an isotype or subtype, fragment or other mutant that does not support binding to the Fc receptor, for example, it has a mutagenized or deleted Fc receptor binding region.

[0133] Methods for changing the constant region of an antibody are known in the art. Functionally altered antibodies can be produced by replacing at least one amino acid residue in the constant portion of an antibody with a different residue, for example antibodies with altered affinity for effector ligands such as FcR on a cell or the C1 component of complement (see, e.g., EP 388,151A1, U.S. Patent No. 5,624,821 and U.S. Patent No. 5,648,260, the entire contents of which are incorporated herein by reference). Similar types of changes can be described, which will reduce or eliminate these functions if applied to immunoglobulins of murine or other species.

[0134] In some instances, the antibody can be derivatized or connected to another functional molecule (e.g., another peptide or protein) or coupled to a label. As used herein, a "derivatized" antibody is a modified antibody. The method of derivatization includes but is not limited to adding a fluorophore, phosphor, light scattering part, luminescent part that is complexed with a metal, radionucleotide, toxin, enzyme or affinity ligand such as biotin. The antibody can include derivatives and other modified forms of the antibody, including immunoadhesin molecules. For example, an antibody molecule can be functionally connected (by chemical coupling, gene fusion, non-covalent binding or other means) to one or more other molecular entities that can mediate the combination of an antibody or an antibody portion with another molecule (e.g., a streptavidin core region or a polyhistidine tag), such as another antibody (e.g., a bispecific antibody or a double antibody), a detectable agent, one or more labels, a cytotoxic agent, a medicament and / or a protein or peptide. A type of derivative antibody molecule is produced by cross-linking two or more antibodies (having the same type or different types, for example, to produce a bispecific antibody). Suitable cross-linking agents include heterobifunctional, those having two distinct reactive groups separated by a suitable spacer (e.g., m-maleimidobenzoyl-N-hydroxysuccinimide ester) or homobifunctional (e.g., disuccinimidyl suberate). Such linkers are available from Pierce Chemical Company III of Rockford.

[0135] In some instances, the antibody may include a detectable label selected based on a specific excitation wavelength provided by the imaging system. Useful detectable agents with which antibody molecules can be derived (or labeled) include fluorescent compounds, various enzymes, prosthetic groups, luminescent materials, bioluminescent materials, fluorescent emitting metal atoms such as europium (Eu), phosphorescent labels and phosphorescent emitting metal atoms, and transition metals, lanthanides, and radioactive materials (described below).

[0136] In certain embodiments, cells or other objects imaged using the methods and systems described herein may include fluorescent markers, for example, on the inside or on the outside surface or in the cell membrane, cell wall or organelle. Exemplary fluorescent markers include, but are not limited to, fluorescein, fluorescein isothiocyanate, rhodamine, 5-dimethylamine-1-naphthalenesulfonyl chloride, phycoerythrin, and the like. Antibodies may also be derivatized with detectable enzymes, such as alkaline phosphatase, horseradish peroxidase, β-galactosidase, acetylcholinesterase, glucose oxidase, and the like. When antibodies are derivatized with detectable enzymes, detection is performed by adding additional reagents that the enzyme uses to produce a detectable reaction product. For example, when the detectable agent horseradish peroxidase is present, the addition of hydrogen peroxide and diaminobenzidine produces a detectable colored reaction product. Antibody molecules may also be derivatized with prosthetic groups (e.g., streptavidin / biotin and avidin / biotin). For example, antibodies may be derivatized with biotin and detected by indirect measurement of avidin or streptavidin binding. Other examples of suitable fluorescent materials include umbelliferone, fluorescein, fluorescein isothiocyanate, rhodamine, dichlorotriazinylamine fluorescein, dansyl chloride or phycoerythrin; examples of luminescent materials include luminol, and examples of bioluminescent materials include luciferase, luciferin and aequorin.

[0137] Labeled antibody molecules can be used, for example, to characterize one or more phenotypes or biological responses of cells using a cell analyzer. Antibodies are often used to detect the presence (or absence) of antigens to assess the abundance and expression pattern of proteins, monitoring protein levels in tissues as part of a clinical testing program, such as to determine the efficacy of a given treatment regimen or otherwise distinguish certain cells in a cell population from one another. The imaging methods and systems described herein can be used to monitor real-time changes in cells by monitoring increases or decreases in signals from labels.

[0138] In certain embodiments, the antibody molecule may be conjugated to another molecular entity, typically a label or a therapeutic (e.g., cytotoxic or cytostatic) agent or moiety. Radioisotopes may be used for diagnostic or therapeutic applications and may also be measured using the imaging methods described herein. Radioisotopes that may be coupled to the antibody include, but are not limited to, alpha, beta, or gamma emitters, or beta and gamma emitters. Such radioisotopes include, but are not limited to, iodine ( 131 I or 125 I), yttrium ( 90 Y), Lutetium ( 177 Lu), Actinium ( 225 Ac), praseodymium, astatine ( 211 At), rhenium ( 186 Re、Bi( 212 Bior 213 Bi), indium ( 111 In), Technetium ( 99m Tc), phosphorus (32 P), rhodium ( 188 Rh), sulfur ( 35 S), carbon ( 14 C) Tritium 3 H), chromium ( 51 Cr), chlorine ( 36 Cl), cobalt ( 57 Co or 58 Co、Iron( 59 Fe、Selenium( 75 Se) or Gallium ( 67 Ga). Radioactive isotopes that can be used as therapeutic agents include, but are not limited to, yttrium ( 90 Y), Lutetium ( 177 Lu), Actinium ( 225 Ac), praseodymium, astatine ( 211 At), rhenium ( 186 Re、Bi( 212 Bior 213 Bi) and rhodium ( 188 Rh). Radioactive isotopes useful as labels, for example for diagnostics, include iodine ( 131 I or 125 I), indium ( 111 In), Technetium ( 99m Tc), phosphorus ( 32 P), carbon ( 14 C) and tritium ( 3 H), or one or more of the therapeutic isotopes listed above. Radioactive materials can be imaged indirectly by monitoring the increase in fluorescence emission of receptor molecules, which can receive energy from the radioactive decay of the radioisotope. For example, imaging systems and methods can be used to detect whether these radioisotopes have been taken up by specific cells, and how much radioisotope is present in each cell.

[0139] In some embodiments, the antibody molecule can also be conjugated to a therapeutic agent. Radioactive isotopes with therapeutic activity have been mentioned. Examples of other therapeutic agents include paclitaxel, cytochalasin B, gramicidin D, ethidium bromide, emetine, mitomycin, etoposide, teniposide, vincristine, vinblastine, colchicine, doxorubicin, daunorubicin, dihydroxy anthracin dione, mitoxantrone, mithramycin, actinomycin D, 1-dehydrotestosterone, glucocorticoids, procaine, tetracaine, lidocaine, propranolol, puromycin, maytansinoids such as maytansinol (see U.S. Patent No. 5,208,020), CC-1065 (see U.S. Patent No. 5,475,092, 5,585,499, 5,846,545) and analogs or homologues thereof. Therapeutic agents include, but are not limited to, antimetabolites (e.g., methotrexate, 6-mercaptopurine, 6-thioguanine, cytarabine, 5-fluorouracil, dacarbazine), alkylating agents (e.g., nitrogen mustard, thiotepa, chlorambucil, CC-1065, melphalan, carmustine (BSNU) and lomustine (CCNU), cyclophosphamide, busulfan, dibromomannitol, streptozotocin, mitomycin C and cis-dichlorodiamine platinum (II) (DDP) cisplatin), anthracyclines (e.g., daunorubicin (formerly daunorubicin) and doxorubicin), antibiotics (e.g., dactinomycin (formerly actinomycin), bleomycin, mithramycin and anthramycin (AMC)) and antimitotic agents (e.g., vincristine, vinblastine, paclitaxel and maytansine alkaloids). If desired, the antibody can be conjugated to both the therapeutic agent and the detectable label. The efficacy of treatment using therapeutic conjugated antibodies can be assessed using the imaging systems and methods described herein.

[0140] In certain embodiments, the antibody molecule is a multispecific (e.g., bispecific or trispecific) antibody molecule. Protocols for producing bispecific or heterodimeric antibody molecules are known in the art; including, but not limited to, for example, the "knob-in-hole" method, as described in, for example, U.S. Patent No. 5,731,168; electrostatic steering Fc pairing, as described in, for example, WO 09 / 089004, WO 06 / 106905, and WO 2010 / 129304; chain exchange engineered domain (SEED) heterodimer formation, as described in, for example, WO 07 / 110205; Fab arm exchange, as described in, for example, WO 08 / 119353, WO 2011 / 131746, and WO 2013 / 060867; diabody conjugates, e.g., using heterobifunctional reagents having amine-reactive groups and thiol-reactive groups to produce bispecific structures by antibody cross-linking, as described, e.g., in U.S. Pat. No. 4,433,059; bispecific antibody determinants produced by recombining half antibodies (heavy-light chain pairs or Fabs) from different antibodies by cycles of reduction and oxidation of disulfide bonds between the two heavy chains, as described, e.g., in U.S. Pat. No. 4,444,878; trifunctional antibodies, e.g., three Fab' fragments cross-linked by thiol-reactive groups, as described, e.g., in U.S. Pat. No. 5,273,743; biosynthetic binding proteins, such as scFv pairs cross-linked by C-terminal tails, preferably by disulfide or amine reactive chemical cross-linking, as described, for example, in U.S. Pat. No. 5,534,254; bifunctional antibodies, such as Fab fragments with different binding specificities that have replaced constant domains by dimerization with leucine zippers (e.g., c-fos and c-jun), as described, for example, in U.S. Pat. No. 5,582,996; bispecific and oligospecific monovalent and oligovalent receptors, such as those that are usually associated with light chains through one antibody VH-CH1 regions of two antibodies (two Fab fragments) connected by a polypeptide spacer between the CH1 region of one antibody and the VH region of another antibody, as described, for example, in U.S. Pat. No. 5,591,828; bispecific DNA-antibody conjugates, such as cross-linking of antibodies or Fab fragments by double-stranded blocks of DNA, as described, for example, in U.S. Pat. No. 5,635,602; bispecific fusion proteins, such as expression constructs containing two scFvs with a hydrophilic helical peptide linker and full constant regions between them, as described, for example, in U.S. Pat. No. 5,637,481; Multivalent and multispecific binding proteins, such as dimers of polypeptides having a first domain with an Ig heavy chain variable region binding region and a second domain with an Ig light chain variable region binding region, often referred to as diabodies (also including higher order structures created for bispecific, trispecific or tetraspecific molecules), as described, for example, in U.S. Pat. No. 5,837,242; miniantibody constructs having linked VL and VH chains further linked to an antibody hinge and CH3 region via a peptide spacer, which can dimerize to form bispecific / multivalent molecules, as described, for example, in U.S. Pat. No. 5,837,242.821; VH and VL domains connected in either direction with a short peptide linker (e.g., 5 or 10 amino acids) or with no linker at all, which can form dimers to form bispecific diabodies; trimers and tetramers, as described, for example, in U.S. Pat. No. 5,844,094; a series of VH domains (or VL domains in family members) further associated with the VL domain by peptide bonds to a cross-linkable group at the C-terminus to form a series of FVs (or scFvs), as described, for example, in U.S. Pat. No. 5,864,019; and single-chain binding polypeptides having both VH and VL domains connected by a peptide linker are combined into multivalent structures by non-covalent or chemical cross-linking to form, for example, homobivalent, heterobivalent, trivalent, and tetravalent structures using scFV or diabody type formats, as described, for example, in U.S. Pat. No. 5,869,620. Additional exemplary multispecific and bispecific molecules and methods of making them are found, e.g., in U.S. Pat. No. 5,910,573, U.S. Pat. No. 5,932,448, U.S. Pat. No. 5,959,083, U.S. Pat. No. 5,989,830, U.S. Pat. No. 6,005,079, U.S. Pat. No. 6,239,259, U.S. Pat. No. 6,294,353, U.S. Pat. No. 6,333,396, U.S. Pat. No. 6,476,198 , U.S. Patent No. 6,511,663, U.S. Patent No. 6,670,453, U.S. Patent No. 6,743,896, U.S. Patent No. 6,809,185, U.S. Patent No. 6,833,441, U.S. Patent No. 7,129,330, U.S. Patent No. 7,183,076, U.S. Patent No. 7,521,056, U.S. Patent No. 7,527,787, U.S. Patent No. 7,534,866, U.S. Patent No. 7,612,181、US2002004587A1、US2002076406A1、US2002103345A1、US2003207346A1、US2003211078A1、US2004219643A1、US2004220388A1、US2004242847A1、US2005003403A1、US2005004352A1、US2005069552A1、US2005079170A1、US2005100543A1、US2005136049A1、US2005136051A1、US2005163782A1、US2005266425A1、US2006083747A1、US2006120960A1、US2006204493A1、US2006263367A1、US2007004909A1、US2007087381A1、US2007128150A1、US2007141049A1、US2007154901A1、US2007274985A1、US2008050370A1、US2008069820A1、US2008152645A1、US2008171855A1、US2008241884A1、US2008254512A1、US2008260738A1、US2009130106A1、US2009148905A1、US2009155275A1、US2009162359A1、US2009162360A1、US2009175851A1 US2009175867A1、US2009232811A1、US2009234105A1、US2009263392A1、US2009274649A1、EP346087A2、WO0006605A2、WO02072635A2、WO04081051A1、WO06020258A2、WO2007044887A2、WO2007095338A2、WO2007137760A2、WO2008119353A1、WO2009021754A2、WO2009068630A1、WO9103493A1、WO9323537A1、WO9409131A1、WO9412625A2、WO9509917A1、WO9637621A2、WO9964460A1。,

[0141] In other embodiments, the antibody molecule (e.g., monospecific, bispecific or multispecific antibody molecule) is covalently linked to, for example, fused to, another partner (e.g., a protein, such as one, two or more cytokines), for example, as a fusion molecule such as a fusion protein. In other embodiments, the fusion molecule includes one or more proteins, such as one, two or more cytokines. In one embodiment, the cytokine is one, two, three or more interleukins (IL) selected from IL-1, IL-2, IL-12, IL-15 or IL-21. In one embodiment, the bispecific antibody molecule has a first binding specificity to a first target, a second binding specificity to a second target, and is optionally connected to an interleukin (e.g., IL-12) domain, such as a full-length IL-12 or a portion thereof. "Fusion protein" and "fusion polypeptide" refer to a polypeptide having at least two covalently linked parts, wherein each part is a polypeptide having different properties. The property can be a biological property, such as in vitro or in vivo activity. The property can also be a simple chemical or physical property, such as binding to a target molecule, catalysis of a reaction, etc. The two parts can be directly linked by a single peptide bond or by a peptide linker, but are in frame with each other. If desired, the fusion protein can include a detectable label that can be detected using a cell analyzer as described herein.

[0142] Although certain fluorophores are described above in conjunction with antibodies, other fluorophores or phosphors can be monitored using the imaging methods and systems described herein. For example, the excitation wavelength used in the imaging process is typically selected based on the specific fluorophore or phosphor present in the imaged cell or object. For example, excitation wavelengths within the ultraviolet, visible, and infrared ranges can be used to excite fluorophores. Illustrative fluorophores include, but are not limited to, dyes such as, for example, acridine dyes (e.g., acridine orange), tetramethylrhodamine, isothiocyanate derivatives (e.g., FITC, TRITC), phenanthridine derivatives (e.g., propidium iodide), 4', 6-diamidino-2-phenylindole (DAPI), and bisbenzimido Hoechst dyes (named 33258, 33342, and 34580). In other cases, Alexa Fluor dyes such as sulfonated rhodamine derivatives and reactive intermediates comprising maleimide, succinimide esters, and hydrazide groups. In some instances, cyanine dyes such as Cy2, Cy3, Cy5, Cy7 and their derivatives, and other molecules with partially saturated indole nitrogen heterocyclic cores, wherein two aromatic units are connected by polyene bridges of different carbon numbers. In other cases, fluorophores can be chelated or otherwise attached to one or more metals such as calcium, and can present the emission wavelength of variation in the presence and absence of a combination with a metallic substance. Some xanthene derivatives Fura red, which are polynuclear imidazoles and benzofuran heterocycles, fluorescent yellow dyes, BCECF, SNARF-1 and other materials can be used as fluorescent probes to monitor the variation of metal concentrations in cells or the variation of cell pH. BODIPY and other species can be used for example to monitor the variation of some organelles such as mitochondria. If desired, quantum dots, nanoparticles, nanostructures or nanosystems that can emit light can also be used to monitor cell activity during cell imaging.

[0143] In certain embodiments, although the methods and systems described herein can be used to provide three-dimensional imaging of cells or other objects in a sample, cells or other objects may be present on a two-dimensional surface. For example, two-dimensional (2D) cell culture generally involves culturing cells on a plate made of plastic or other materials. Cells are adhered to the surface by one or more coatings or other materials. The coating on the flat surface can be intentionally selected to have different heights so that the cultured cells do not have to be flat. The pre-scanning step described herein can be used to identify different heights and allow cells to be imaged more quickly in different z dimensions in 2D cell culture. In some instances, the methods and systems described herein can be used to image cells in 2D cell culture using a material matrix glue layer. In other cases, 2D culture can be performed in a Transwell plate, where a 2D cell layer is present at any height above the bottom of the plate. For example, a Transwell insert can suspend cells above the bottom surface of a plate or well. Pre-scanning methods can be used to determine the xy position and z height of each hole of the insert, and higher resolution scanning can be used to image cells. As long as the matrix is ​​generally transparent for imaging at the specific wavelength used, the methods and systems described herein can be used to image cells grown in various different ways.

[0144] Certain specific examples are described to further illustrate some of the novel and inventive aspects of the present technology.

[0145] Example 1

[0146] This example is intended to illustrate the amount of data that can be obtained from a typical high-magnification scan of a well comprising a three-dimensional object using a laser scanning confocal microscope. Fig.16 , shows an illustration of using high-power scanning to scan each well of a plate. For example, an immersion objective can be used to scan the wells of a 384-well plate with a hydrogel matrix of about 500 microns in height and an object of about 30-50 microns in z width at high power (63X). For 225 fields of view (FOV), approximately 1000 planar scans are required to cover each well. This scan generates approximately 1 terabyte of data per well or 384 terabytes of data per microplate. This amount of data increases processing time and requires considerable storage capacity.

[0147] Example 2

[0148] Compared to Example 1, by using a pre-scan to identify the xy-coordinates and z-height of the object and then performing a higher magnification scan, the amount of data acquired can be reduced. Fig.17, using a laser scanning confocal microscope, a pre-scan can be performed using a 10X air objective and allows the system to reduce the scan to 25 fields of view. By scanning only the objects identified by the pre-scan at a higher magnification scan (e.g., using a 63X immersion objective), the amount of data can be reduced to about 18 gigabytes per well or about 7 terabytes for the entire 384 microplate. The combination of pre-scan and higher magnification scan can reduce the amount of data to 54X less than the amount of data obtained in Example 1.

[0149] Example 3

[0150] This example is intended to illustrate the scanning time of using laser scanning confocal microscope to scan whole 384 microwell plates. For the purpose of this explanation, it is assumed that the object is a spheroid randomly distributed in the gel matrix. The matrix height is 500 microns, and the spheroid z width is 40-60 microns. For scanning the single hole of 384 microwell plates, it is estimated that the total scanning time is about 105 minutes, wherein 481 planes are scanned, and the plane spacing is 1 micron. 81 visual fields can be scanned through the scanning time of 105 minutes.

[0151] Example 4

[0152] Compared to Example 3, the time spent scanning spherical objects in the wells of a 384-well microplate using a laser scanning confocal microscope can be greatly reduced by using a pre-scan to identify the xy coordinates and z height of the object and then performing a higher magnification scan. Prior to pre-scanning using a 10X air objective, four fields of view are used for the pre-scan step, with 24 planes distributed over 350 microns in the z dimension. The results of the pre-scan can then be used to scan the identified image at a higher magnification using a 40X immersion objective. For example, in a higher magnification scan, 81 planes with a height distribution of 80 microns (a scanning pitch of approximately 1 micron in the z direction) can be used. The total scanning time for imaging the object in the well (pre-scan + high magnification scan) is estimated to be less than 6 minutes. Compared to Example 3, the pre-scan and scanning steps allow the imaging of the object in the well to be accelerated by approximately 19X.

[0153] Example 5

[0154] HeLa cell spheroids were grown in a Geltrex matrix and imaged using a combination of pre-scans (10X air objective) and higher magnification scans (40X water immersion objective). Spheroids were randomly distributed throughout the matrix. For the pre-scan, a z-stack spanning 450 μm at 15 μm intervals in the z dimension was used (see Fig.18A For higher magnification scans, z-stacks spanning 80 μm at 1 μm intervals in the z dimension were used (see Fig.18B ). The combination of the pre-scan and scan steps reduced the data volume by approximately 35X and the scan time by approximately 23X.

[0155] Example 6

[0156] MDCK cells were cultured in 3-D Life hydrogel matrices (Cellendes, DE) and imaged using a combination of lower magnification pre-scans and higher magnification scans. The combination of pre-scans and scans reduced the data by 54-fold and the scan time by 36-fold. The height of the gel was 500 μm, and the MDCK cells formed 3D aggregates throughout the gel. The cells were imaged at the higher magnification scans using a 63X water immersion objective and at the lower magnification scans using a 10X air objective. 3D visualization of the pre-scans obtained using a 10x objective (24 planes, 15 μm step size) is shown in Fig.19A middle. Fig.19B Shown are XYZ views of MDCK cysts grown in the presence of RGD peptide (which promotes cyst formation) or a scrambled control peptide (Scrambled Peptide). Images were acquired using a 63x water immersion objective (160 planes, 0.5 μm step size).

[0157] After collection, MDCK cyst morphology was analyzed using the Harmony 4.9 3D analysis tool commercially available from PerkinElmer Instruments, Inc. (Waltham, MA). The lumen of the MDCK cyst was segmented and the volume was calculated. Single cell results were displayed as Fig.19C The box plots in Figure 2. The left box plot represents the results without added peptide, the center box plot represents the results with added scrambled peptide, and the right box plot represents the results with added RGD-peptide.

[0158] Example 7

[0159] The methods and systems described herein can also be used to identify cells within larger structures, such as cells within a living organism. Tumor cells were imaged within a zebrafish larva. RFP-positive tumor cells were segmented in a pre-scan using a 5x air objective and scanned at higher magnification using a 20x water immersion objective. The z-height of the identified tumor cells varied between 150 microns and 550 microns.

[0160] To cover the entire well of a 96-well CellCarrier Ultra plate, 9 fields were acquired with a 30 μm step size using a z-stack of 25 planes. Fig. 20A A 5x pre-scan, maximum intensity projection of the global image and individual images is shown. Fig. 20B Segmented tumor cells are shown on a global image with coordinates for a higher magnification scan. Fig. 20C Shown is a maximum intensity projection of a global image acquired with a 20x water immersion objective (280 planes, 1 micron step size). Fig.20D , 20E20F and 20F each show a 3D view of a higher power scan of one cluster from three identified segmented tumor cell clusters.

[0161] When introducing elements of the examples disclosed herein, the articles "a," "an," "the," and "said" are intended to indicate that there are one or more elements. The terms "comprising," "including," and "having" are intended to be open ended and indicate that there may be other elements in addition to the listed elements. Given the benefit of this disclosure, one of ordinary skill in the art will recognize that various components of an example may be interchanged or replaced with various components in other examples.

[0162] While certain aspects, examples, and embodiments have been described above, those skilled in the art, given the benefit of this disclosure, will recognize that additions, substitutions, modifications, and variations of the illustrative aspects, examples, and embodiments disclosed are possible.

Claims

1. A microscope system configured to image a three-dimensional object in a three-dimensional matrix, the system comprising: a sample holder configured to receive a sample comprising the three-dimensional object in the three-dimensional matrix; a first light source optically coupled to the sample holder and configured to illuminate at least some portion of the three-dimensional object in the three-dimensional matrix received by the sample holder, wherein a wavelength provided by the first light source is selected to excite at least one substance present in the three-dimensional object; at least one objective lens optically coupled to the light source and configured to receive light emission from at least one excited species present in the three-dimensional object in the three-dimensional matrix; a detector optically coupled to said at least one objective lens and configured to receive said light emission from said objective lens; as well as a processor electrically coupled to the detector, wherein the microscope system is configured to pre-scan the three-dimensional matrix using a first magnification objective lens to identify an xy position of the three-dimensional object in the three-dimensional matrix and to identify a z-height of the three-dimensional object in the three-dimensional matrix, and record the identified xy position and z-height as a first image set, and wherein the microscope system is configured to scan the identified three-dimensional object in a z direction using the identified z-height of the identified three-dimensional object using a second magnification objective lens to provide a three-dimensional image of at least a portion of the three-dimensional object, wherein the magnification of the second magnification objective lens is greater than the magnification of the first magnification objective lens, wherein the processor is configured to execute instructions to construct a three-dimensional image of the three-dimensional object in the three-dimensional matrix from the image set obtained from the scanning step, and to execute instructions to perform image correction on the image set before constructing the three-dimensional image of the three-dimensional object, and wherein the image correction is performed using a second image set, the second image set being one or more images of a two-dimensional graphic pattern obtained using the detector in the same fixed position used to obtain the image set, wherein the two-dimensional graphic pattern comprises points in a dot matrix, and wherein the absolute positions of the points are determined, and wherein the images of the image set are to be corrected using the one or more images of the two-dimensional graphic pattern to correct geometric distortions in the images of the image set.

2. The microscope system of claim 1, wherein the system comprises a first objective lens and a second objective lens, and wherein the first objective lens is used as the first magnification objective lens and the second objective lens is used as the second magnification objective lens.

3. The microscope system of claim 2, wherein the first objective is an air objective and the second objective is a water immersion objective.

4. A microscope system according to claim 3, wherein the detector includes at least one camera and the first light source includes at least one laser, and wherein the system is configured to perform laser scanning confocal microscopy using the camera and the laser during the pre-scanning and scanning of the three-dimensional object.

5. The microscope system of claim 1, wherein the points are located at vertices that define one or more geometric shapes, and wherein the lattice of points is non-periodic.

6. The microscope system of claim 1, wherein the system comprises at least two separate light sources configured to provide light of different wavelengths to the sample on the sample holder.

7. The microscope system of claim 6, further comprising a confocal unit optically coupled to each of the at least two separate light sources and located between the at least two separate light sources and the sample holder.

8. The microscope system of claim 7, wherein the system comprises at least two separate detectors configured to receive light of different wavelengths emitted by the sample on the sample holder.

9. The microscope system of claim 8, wherein each of the at least two separate detectors comprises a camera.

10. The microscope system of claim 8, wherein one of the at least two separate detectors is configured to detect light emission during the pre-scanning step, and another of the at least two separate detectors is configured to detect light emission during the scanning step.

11. A microscope system according to claim 8, wherein the system is configured to use the first magnification objective lens of the microscope to identify the xy position of each of the multiple individual three-dimensional objects in the matrix and to identify the z height of each of the identified multiple three-dimensional objects in the matrix, and wherein the system is configured to use the second magnification objective lens to scan each identified three-dimensional object in the z direction using the respective identified z height of each identified three-dimensional object to provide a three-dimensional image of at least a portion of each of the three-dimensional objects present in the matrix.

12. The microscope system of claim 7, wherein the system comprises four independent light sources and four independent cameras, each light source being configured to provide light of a different wavelength to the sample on the sample holder, and each camera being configured to detect light emission from the sample.

13. A microscope system according to claim 1, wherein the system is further configured to identify the z-width of the identified three-dimensional object in the matrix and use the identified z-width to select a scanning time or select an amount of data obtained from the scan of the three-dimensional object in the matrix.

14. A microscope system configured to image a three-dimensional object in a three-dimensional matrix, the system comprising: a sample holder configured to receive a sample comprising the three-dimensional object in the three-dimensional matrix; a first light source optically coupled to the sample holder and configured to illuminate at least some portion of the three-dimensional object in the three-dimensional matrix received by the sample holder, wherein a wavelength provided by the first light source is selected to excite at least one substance present in the three-dimensional object; at least one objective lens optically coupled to the light source and configured to receive light emission from the at least one excited substance present in the three-dimensional object in the three-dimensional matrix, a detector optically coupled to said at least one objective lens and configured to receive said light emission from said objective lens; as well as a processor electrically coupled to the detector, wherein the microscope system is configured to pre-scan the three-dimensional matrix using a first magnification objective lens to identify an xy position of a three-dimensional object in the three-dimensional matrix and to identify a z height of the three-dimensional object in the three-dimensional matrix and record the identified xy position and z height as a first image set, correct imaging anomalies from the pre-scan to provide a corrected first image set, scan the identified three-dimensional object in a z direction using the corrected z height and the corrected xy position from the corrected first image set using a second magnification objective lens to provide a second image set, and correct imaging anomalies from the scan to provide a corrected second image set, wherein the magnification of the second magnification objective lens is greater than the magnification of the first magnification objective lens, wherein the first image set is image corrected using another image set, wherein the other image set is one or more images obtained from a two-dimensional graphic pattern using the detector in the same fixed position used to obtain the image set obtained from the scanning step, wherein the two-dimensional graphic pattern includes points in a dot matrix and wherein the absolute positions of the points are determined, and wherein the images of the first image set are corrected using one or more images of the two-dimensional graphic pattern to correct geometric distortions in the images of the first image set and provide the corrected image set.

15. The microscope system of claim 14, wherein the system comprises a first objective lens and a second objective lens, and wherein the first objective lens serves as the first magnification objective lens and the second objective lens serves as the second magnification objective lens.

16. The microscope system of claim 15, wherein the first objective is an air objective and the second objective is a water immersion objective.

17. A microscope system according to claim 16, wherein the detector includes at least one camera and the first light source includes at least one laser, and wherein the system is configured to perform laser scanning confocal microscopy using the camera and the laser during the pre-scanning and scanning of the three-dimensional object.

18. The microscope system of claim 17, wherein the processor is configured to execute instructions to construct a three-dimensional image of the three-dimensional object in the three-dimensional matrix based on images obtained from the set of images.

19. The microscope system of claim 18, wherein the points are at vertices that define one or more geometric shapes, and wherein the lattice of points is non-periodic.

20. The microscope system of claim 19, wherein the system comprises at least two separate light sources configured to provide light of different wavelengths to the sample on a sample holder.

21. The microscope system of claim 20, further comprising a confocal unit optically coupled to each of the at least two separate light sources and located between the at least two separate light sources and the sample holder.

22. The microscope system of claim 21, wherein the system comprises at least two separate detectors configured to receive light of different wavelengths emitted by the sample on the sample holder.

23. The microscope system of claim 22, wherein each of the at least two separate detectors comprises a camera.

24. A microscope system according to claim 22, wherein one of the at least two separate detectors is configured to detect light emission during the pre-scanning step, and another of the at least two separate detectors is configured to detect light emission during the scanning step.

25. A microscope system according to claim 22, wherein the system is configured to use the first magnification objective lens of the microscope to identify the xy position of each of the multiple individual three-dimensional objects in the matrix and to identify the z height of each of the multiple three-dimensional objects identified in the matrix, and wherein the system is configured to use the second magnification objective lens to scan each identified three-dimensional object in the z direction using the respective identified z height of each identified three-dimensional object to provide a three-dimensional image of at least a portion of each of the three-dimensional objects present in the matrix.

26. The microscope system of claim 21, wherein the system comprises four independent light sources and four independent cameras, each light source being configured to provide light of a different wavelength to the sample on the sample holder, and each camera being configured to detect light emissions from the sample.

27. A microscope system according to claim 14, wherein the system is further configured to identify the z-width of the identified three-dimensional object in the matrix and use the identified z-width to select a scanning time or select an amount of data obtained from a scan of the three-dimensional object in the matrix.

28. A non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed by a processor, cause the processor to: pre-scanning a substrate using a first magnification objective to identify an xy position of a three-dimensional object in the substrate and to identify a z height of the three-dimensional object in the substrate and recording the identified xy position and z height as a first image set, correcting imaging anomalies from the pre-scan to provide a corrected first image set, scanning the identified three-dimensional object in a z direction using a second magnification objective lens using the corrected z height and the corrected xy position from the corrected first image set to provide a second image set, correcting imaging anomalies from the scan to provide a corrected second image set, and constructing an image of the three-dimensional object in the matrix using the corrected second image set, wherein the magnification of the second magnification objective lens is greater than the magnification of the first magnification objective lens, wherein the first image set is image corrected using another image set, wherein the other image set is one or more images obtained from a two-dimensional graphic pattern using a detector in the same fixed position used to obtain the image set obtained from the scanning step, wherein the two-dimensional graphic pattern includes points in a dot matrix and wherein the absolute positions of the points are determined, and wherein the images of the first image set are corrected using the one or more images of the two-dimensional graphic pattern to correct geometric distortions in the images of the first image set and provide the corrected image set.