Microscope system

Abstract

The present disclosure provides a microscope system (500), comprising: a light source (502) for transmitting a single light beam; a first rotating disk (506) having a converging microlens array, which rotates in a first direction and is positioned along an optical axis, and is used to split the single light beam into a plurality of light beams to form a multi-focal pattern; a second rotating disk (508) having a pinhole array, which rotates in the first direction and is positioned along the optical axis, and is used to block out-of-focus light beams among the plurality of light beams of each multi-focal pattern and allow in-focus light beams among the plurality of light beams to pass through the rotating disk (506) having the pinhole array; and a third rotating disk (516) having a diverging microlens array, which rotates in the first direction and is positioned along the optical axis, wherein the first rotating disk (506) having the converging microlens array, the second rotating disk (508) having the pinhole array, and the third rotating disk (516) having the diverging microlens array rotate synchronously relative to each other.

Description

[0001] This application is a divisional application of an invention patent application with an application date of June 6, 2018, application number 201880004664.1 (international application number PCT / US2018 / 036265), and invention name “Multi-focus structured illumination microscopy system and method”. Technical Field

[0002] This document relates to multi-focal structured illumination microscopy, and in particular to multi-focal structured illumination microscopy systems and methods for generating multiple multi-focal fluorescence emissions resulting from a multi-focal pattern of a sample. Background Art

[0003] Classical fluorescence microscopy is limited in resolution by the wavelength of light, known as the "diffraction limit," which limits lateral resolution to about 200 nm and axial resolution to about 500 nm at typical excitation and emission wavelengths when the sample emits fluorescence detected by the microscope. Confocal microscopy is an optical imaging technique used to increase optical resolution beyond the diffraction limit by using point illumination and a spatial pinhole arrangement to eliminate out-of-focus emission light from specimens thicker than the focal plane, thereby delivering images with a resolution 1.41 times that of the diffraction limit by a method that requires tightly closing the pinhole. Unfortunately, closing the pinhole reduces the signal level of light emitted from the sample to a degree that makes this particular super-resolution method impractical. In addition, confocal microscopy must perfectly align the excitation of the illumination beam from the microscope with the pinhole / detector, because a misaligned pinhole results in a reduction and weakening of the light signal being detected, and a reduction in the axial optical section of the sample itself. Therefore, misalignment of a confocal microscope can cause a reduction in the light signal.

[0004] A method for resolution enhancement for confocal microscopy has been discovered that uses an array of detectors, such as pixels in a camera image, where each detector in the array produces a separate confocal image. If the detector array is small enough, each confocal image formed can be equivalent to a similar confocal image formed by a confocal microscope with a tightly closed pinhole, so that when the confocal images are properly aligned, a resolution of 1.41 times that of a diffraction-limited microscope is achieved. In addition, deconvolution provides a further increase in image resolution. However, this detector array arrangement is limited in that only a single excitation point is scanned throughout the two-dimensional plane of the sample, which limits the speed at which the sample can be scanned and the subsequent detection of the sample's fluorescent emission.

[0005] Another type of microscopy, called structured illumination microscopy (SIM), illuminates the sample using a spatially modulated excitation intensity that is translated and rotated to different positions relative to the sample, taking a wide-field image at each translation and rotation. The raw images are appropriately processed to produce a final image that has a lateral resolution that is twice that of a conventional wide-field microscope. Although such a SIM system generates images with a spatial resolution that is twice that of a conventional microscope, temporal resolution is sacrificed when the final image is produced because time is required to acquire each of the multiple raw images. SIM can also be used to filter out out-of-focus blur, called "optical sectioning." However, such optical sectioning is performed in a computational manner and is therefore subject to shot (Poisson) noise. Therefore, SIM is not suitable for thick or highly stained samples when background fluorescence may cause the shot noise contribution to exceed the in-focus signal.

[0006] Therefore, there is a need in the art for a structured illumination microscopy system that produces multi-focal excitation patterns of a specimen for each high-resolution image without sacrificing scanning speed and that is resistant to shot noise that can interfere with SIM images. Summary of the invention

[0007] In one embodiment, the microscopy system may include a light source for transmitting a single light beam and a beam splitter for splitting the single light beam into a plurality of light beams to form a multi-focal pattern. A scanner scans the plurality of light beams forming the multi-focal pattern onto a sample so that the sample generates a plurality of fluorescence emissions generated by each multi-focal pattern. A focusing component then defines an aperture, the aperture being configured to physically block out-of-focus fluorescence emissions of the plurality of fluorescence emissions generated by each multi-focal pattern and allow in-focus fluorescence emissions to pass through the aperture. Additionally, a scaling component scales down the plurality of in-focus fluorescence emissions generated by each multi-focal pattern so that each of the plurality of in-focus fluorescence emissions is scaled down by a predetermined factor to generate a plurality of scaled in-focus fluorescence emissions generated by each multi-focal pattern. A summing component sums each of the plurality of scaled in-focus fluorescence emissions to generate a plurality of summed, scaled in-focus fluorescence emissions, which form a composite image of the plurality of summed, scaled in-focus fluorescence emissions.

[0008] In another embodiment, the microscopy system may include a light source for transmitting a single light beam; a beam splitter for splitting the single light beam into a plurality of light beams to form a multi-focal pattern, wherein each of the plurality of multi-focal patterns defines a plurality of focal points; a scanner for scanning the plurality of light beams forming each of the plurality of multi-focal patterns onto a sample so that the sample generates a plurality of fluorescent emissions generated by each multi-focal pattern, wherein each of the plurality of multi-focal patterns defines a plurality of fluorescent focal points; a detector for collecting the plurality of fluorescent emissions generated by each multi-focal pattern; and a processing system for processing the collected multi-focal fluorescent emissions from the detector, including: a processing system in operable communication with a database; A processor is provided for storing a plurality of collected multifocal fluorescence emissions, wherein the processor removes out-of-focus fluorescence emissions generated by each of the plurality of multifocal patterns to leave only in-focus fluorescence emissions generated by each of the plurality of multifocal patterns, wherein the processor then scales the in-focus fluorescence emissions generated by each of the plurality of multifocal patterns in a local pinch operation, wherein each of the plurality of fluorescence emissions generated by each multifocal pattern in the local pinch operation maintains a same proportional distance as another plurality of fluorescence emissions generated by the multifocal pattern when the plurality of fluorescence emissions are pinched to produce the scaled in-focus fluorescence emissions; wherein the processor sums the plurality of multifocal in-focus fluorescence emissions to produce a composite image.

[0009] In yet another embodiment, a method for multi-focus structured illumination microscopy may comprise:

[0010] Generate a single beam;

[0011] splitting a single light beam into multiple light beams, wherein the focus of each of the multiple light beams forms multiple multi-focal patterns,

[0012] illuminating the sample using the plurality of light beams to form each of a plurality of multi-focal patterns, wherein the illuminated sample produces a plurality of fluorescent emissions produced by each of the plurality of multi-focal patterns;

[0013] performing a pinhole operation wherein out-of-focus fluorescence emissions from the plurality of fluorescence emissions are blocked and only in-focus fluorescence emissions from the plurality of fluorescence emissions are allowed to pass during the pinhole operation;

[0014] scaling each focal point of a plurality of in-focus fluorescent emissions produced by the plurality of multi-focal patterns by a predetermined factor to produce a plurality of scaled in-focus fluorescent emissions produced by the plurality of multi-focal patterns; and

[0015] Each of the multiple scaled in-focus fluorescence emissions is summed to form a composite image.

[0016] In yet another embodiment, the microscopic system may include a light source for transmitting a single light beam; a first microlens array that splits the single light beam into a plurality of light beams to form at least one multi-focal pattern; and a scanner that scans the plurality of light beams forming the at least one multi-focal pattern onto a sample so that the sample generates a plurality of fluorescence emissions using each of the at least one multi-focal pattern. Additionally, the pinhole array blocks out-of-focus fluorescence emissions of each of the at least one multi-focal pattern and allows in-focus fluorescence emissions to pass through the pinhole array. The second microlens array generates a non-inverted image of the plurality of light beams with a one-half magnification, wherein the scanner rescans the non-inverted image of the plurality of light beams. Finally, the camera captures the scanned non-inverted image.

[0017] In a further embodiment, the microscopy system may include a light source for transmitting a single light beam; a rotating disk with a microlens array for splitting the single light beam into multiple light beams to form a multifocal pattern; a rotating disk with a pinhole array for blocking out-of-focus light beams of the multiple light beams of each multifocal pattern and allowing in-focus light beams of the multiple light beams to pass through the rotating disk with the pinhole array, wherein the rotating disk with the microlens array rotates in series with the rotating disk with the pinhole array to scan the multiple light beams on the sample and generate multiple fluorescence emissions; and a camera for capturing images of the multiple fluorescence emissions in each multifocal pattern generated by the sample.

[0018] In another embodiment, the microscopic system may include a light source for transmitting a single light beam, a first rotating disk having a converging microlens array that rotates in a first direction and is positioned along an optical axis, for splitting the single light beam into a plurality of light beams to form a multi-focal pattern, a second rotating disk having a pinhole array that rotates in the first direction and is positioned along the optical axis, for blocking out-of-focus light beams of the plurality of light beams of each multi-focal pattern and allowing in-focus light beams of the plurality of light beams to pass through the rotating disk having the pinhole array, and a third rotating disk having a diverging microlens array that rotates in the first direction and is positioned along the optical axis, wherein the first rotating disk having the converging microlens array, the second rotating disk having the pinhole array, and the third rotating disk having the diverging microlens array rotate synchronously relative to each other.

[0019] Additional objects, advantages and novel features will be set forth in the following description or will become apparent to one skilled in the art upon examination of the following drawings and detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a simplified block diagram illustrating a method for generating a multi-focal pattern in a multi-focal structured illumination (MSIM) system;

[0021] Figure 2 is a simplified block diagram illustrating another method of generating a multi-focal pattern in another embodiment of a multi-focal SIM system;

[0022] Figure 3 It is shown Figure 1 A simplified illustration of various components of one embodiment of a multi-focal SIM system;

[0023] Figure 4 It is shown Figure 2 A simplified illustration of various components of one embodiment of a multi-focal SIM system;

[0024] Figure 5 is a simplified illustration showing a multi-focal pattern having multiple focal points; and

[0025] Figure 6 is a simplified diagram showing a zoom operation that zooms in-focus fluorescence emission caused by a multi-focal pattern emitted by a sample;

[0026] Figure 7 This diagram is used by Figure 4 A flowchart of a method for performing scaling and summing operations on a processing system in a multi-focal SIM system;

[0027] Figure 8 is a flow chart illustrating a method of obtaining a lattice vector when performing out-of-focus fluorescence filtering, scaling and summing operations of a processing system;

[0028] Fig. 9 is a flow chart illustrating a method of obtaining an offset vector when performing out-of-focus fluorescence filtering, scaling and summing operations of a processing system;

[0029] Fig.10 is a flow chart illustrating a method of obtaining a shift vector when performing out-of-focus fluorescence filtering, scaling and summing operations of a processing system;

[0030] Fig.11 is a flow chart illustrating a method of performing pinhole masking when performing out-of-focus fluorescence filtering, scaling and summing operations of a processing system;

[0031] Fig.12 is a flow chart illustrating a method of performing scaling and local pinching when performing out-of-focus fluorescence filtering, scaling and summing operations of a processing system;

[0032] Fig.13 is a flow chart illustrating a method of performing summing when performing out-of-focus fluorescence filtering, scaling, and summing operations of a processing system;

[0033] Fig.14 is a simplified diagram illustrating an illumination system used during testing of one embodiment of a multi-focal SIM system;

[0034] Figures 15A-15DVarious images of samples embedded in fluorescent mounting medium are illustrated;

[0035] Figure 16A-16B Various images of doubly labeled three-dimensional samples embedded in fluorescent mounting medium are illustrated; and

[0036] Figures 17A-17D illustrates various images of a sample using a multifocal illumination system after pinhole focusing, zooming, and three-dimensional deconvolution;

[0037] Fig.18 is a simplified diagram showing an embodiment of a swept-field hardware MSIM system;

[0038] Fig.19 is an image of a super-resolution image of microtubules with a swept-field MSIM system;

[0039] Fig. 20 is a simplified diagram illustrating one embodiment of a spinning disk confocal microscopy system; and

[0040] Fig.21 is a simplified diagram illustrating an embodiment of a spinning disk MSIM system.

[0041] Corresponding reference numerals indicate corresponding elements in the drawings. The headings used in the drawings should not be construed as limiting the scope of the claims. DETAILED DESCRIPTION

[0042] In modern microscopy, structured illumination microscopy (SIM) can be used to examine single cells by using spatially patterned light to excite sample fluorescence that is later detected, and to process one or more images to produce super-resolution images with a resolution twice that of traditional wide-field microscopy images. However, SIM systems sacrifice speed for higher resolution (multiple raw images are taken for each super-resolution image). In addition, optical sectioning in SIM systems is performed computationally, and is therefore prone to shot noise inherent in the fluorescence background. This limits the thickness of the sample that can be inspected, so other microscopic techniques need to be used when inspecting thicker samples. For example, confocal microscopy systems use a pinhole arrangement to physically filter out (reject) out-of-focus light, which allows only light from a specific focus in the emitted light emitted by the sample to be detected by the system, thereby producing high-contrast optical sectioning images of relatively thicker samples than what can be achieved with a SIM system. Confocal microscopes can also provide enhanced resolution relative to traditional wide-field microscopes. However, this enhanced image resolution of confocal microscopy is obtained by shrinking the pinhole arrangement, which results in a prohibitive loss of the corresponding fluorescence emission signal detected from the sample. Modified confocal microscopy has been shown to increase image resolution to the resolution level of SIM systems without sacrificing emission signal intensity; however, the slow scanning speeds achieved by the microscope make it impractical for research purposes.

[0043] Thus, embodiments of a multifocal SIM (MSIM) system as described herein include specific hardware components, attributes, and features that address issues associated with achieving high image resolution at high scan speeds and signal intensities required by conventional microscope imaging systems, and that provide better performance on thick samples than currently commercially available SIM systems. The MSIM system described herein includes various embodiments of hardware components that generate a multifocal excitation pattern for each image taken of a sample to produce high resolution images at high scan rates without significant signal loss relative to conventional confocal microscopy and SIM systems. In addition, the MSIM system uses only an arrangement of hardware components (such as a pinhole mirror and a microlens array) to perform the scaling, pinhole, and summing steps, rather than using a processor and software arrangement to perform the same operations as in structured illumination microscopy. Further details of the multifocal SIM system and method are discussed in more detail below.

[0044] Referring to the accompanying drawings, various embodiments of a multi-focal SIM (MSIM) system are shown in FIG. Figure 1-21 , and are generally indicated as 100, 200, 300, 400, and 500. Figure 1, a general embodiment of an MSIM system designated as 100 is shown. The MSIM system 100 performs an illumination pattern generation operation 102, which splits a single light beam into a plurality of light beams to generate one or more multi-focal patterns, wherein each multi-focal pattern defines an arrangement of focal points of each of the plurality of light beams. For example, Figure 5 A simplified illustration of one example of a multi-focal pattern 170 is shown in FIG. In one embodiment, the multi-focal pattern 170 may include a plurality of focal points 172 arranged in a particular multi-focal pattern 170 defined by a plurality of beams, wherein each multi-focal pattern 170 defines a different arrangement of focal points of the plurality of beams such that for a given density of focal points 172, the multi-focal pattern 170 maximizes the distance between any two closest focal points 172, thereby minimizing crosstalk. As used herein, the term crosstalk refers to the ability of nearby beams to cause excitation and fluorescence from other focal points to cause a focal point to appear as if it originates from the focal point in question. In a scanning operation 104, the plurality of beams in each multi-focal illumination pattern are rasterized onto the illuminated sample 106 such that the sample 106 emits a plurality of fluorescent emissions. In a de-scan operation 108, the fluorescent emissions emitted by the sample 106 are rasterized, which redirects the plurality of fluorescent emissions to remove out-of-focus fluorescent emissions in a focusing operation 110. In the focusing operation 110, out-of-focus fluorescent emissions are blocked and only in-focus fluorescent emissions are allowed to pass through for processing.

[0045] The in-focus fluorescence emissions caused by each multi-focal pattern are then scaled using a scaling operation 112 that locally shrinks each fluorescence emission by a predetermined factor. Figure 6 In one embodiment of the scaling operation 112 shown in FIG. 1 , a local contraction of the fluorescence foci 172 induced in a single multi-focal pattern 170 occurs. For example, the local contraction of the fluorescence foci 172A and 172B in the multi-focal pattern 170 to the scaled fluorescence foci 172A′ and 172B′, respectively, is such that the distance 300 between the geometric centers 175 of the fluorescence foci 172A and 172B and the distance 302 between the fluorescence foci 172A and 172B remain the same regardless of the degree of scaling applied to the fluorescence foci 172 of the multi-focal pattern 170. In other words, the scaling operation 112 locally contracts the fluorescence foci 172 without inversion while keeping the relative distance between each fluorescence foci the same. After the scaling operation 112, the scaled in-focus fluorescence emission of each multi-focal illumination pattern is then rasterized in a rescanning operation 114, which allows the contracted in-focus fluorescence emission induced by each multi-focal pattern to be collected by the detector and summed to produce a composite high-resolution image in a collection and summing operation 116.

[0046] In some embodiments, after the collection and summing operation 116, the composite image may undergo a deconvolution operation 118, which performs a level of deblurring that further enhances the resolution of the composite image. The deconvolution operation 118 may be any conventional deconvolution operation 118, such as the freely available parallel iterative deconvolution plugin of Piotyr Wendykier.

[0047] refer to Figure 2 , illustrates another method for generating a multi-focal pattern according to another embodiment of an MSIM system designated as 200. The MSIM system 200 performs an illumination pattern generation operation 202, in which a single beam is split into multiple beams to generate one or more multi-focal patterns of the multiple beams. In a scanning operation 204, the multiple beams for each multi-focal pattern are rastered onto a sample 206. The sample 206 generates fluorescence emissions in response to the multi-focal pattern, which are detected in a collection operation 208. In the collection operation 208, a detector collects the in-focus fluorescence emissions and transmits the collected data to a processing system that performs a processing operation 210. The processing operation 210 removes the out-of-focus fluorescence emissions in each multi-focal pattern, scales the remaining in-focus fluorescence emissions, and then sums the scaled in-focus fluorescence emissions to generate a composite image composed of the multiple fluorescence emissions created by the multi-focal pattern. In some embodiments, the collected data may then undergo a deconvolution operation 212 similar to the deconvolution operation 118, which performs a deblurring of the composite image to further enhance image resolution.

[0048] refer to Figure 3 , one embodiment of the multi-focal SIM system 100 may include an illumination source 120 (e.g., a laser) to generate a single light beam 122, which is transmitted through a beam splitter 124 (such as a microlens array). The beam splitter 124 splits the single light beam 122 into a plurality of light beams 126, wherein each of the plurality of light beams 126 has a different focus, which together form a multi-focal pattern 128. In some embodiments, a first lens 130 images each multi-focal pattern 128 to pass through a dichroic mirror 132 in the A direction and onto a scanning device 134 such as a galvanometer for performing a scanning operation 104.

[0049] During the scanning operation 104 , the scanning device 134 rasterizes each multi-focal pattern 128 of the plurality of light beams 126 onto the sample 144 through the arrangement of the second lens 136 , the tube lens 138 , and the objective lens 140 and onto the sample 144 .

[0050] In response to the sample 144 being illuminated by the multi-focal pattern 128 of the plurality of light beams 126, the sample 144 emits fluorescent emissions 142 caused by the multi-focal pattern 128 composed of the light beams 126. Then, the plurality of fluorescent emissions 142 of each multi-focal pattern 128 emitted by the illuminated sample 144 are captured by the objective lens 140, and pass through the tube lens 138 and the second lens 136 and arrive at the scanning device 134, which de-scans each of the plurality of fluorescent emissions 142 by rasterizing the fluorescent emissions 142 from the second lens 136, and arrives at the dichroic mirror 132 in a direction B opposite to the direction A in which the plurality of light beams 126 directly pass through the dichroic mirror 132. In the direction B, the plurality of fluorescent emissions 142 are redirected by the dichroic mirror 132 to pass through the third lens 146, where the plurality of fluorescent emissions 142 are then focused onto the pinhole array 148 for the pinhole operation 110.

[0051] During pinhole operation 110, the pinhole array 148 physically blocks and filters out out-of-focus fluorescence emissions 142 and allows only in-focus fluorescence emissions 142A to pass through the pinhole array 148. In one embodiment, the pinhole array 148 may include multiple apertures configured to allow only in-focus fluorescence emissions 142A to pass through the pinhole array 148 while blocking any fluorescence emissions 142 that do not pass through one of the apertures.

[0052] After passing through the pinhole array 148, the plurality of in-focus fluorescence emissions 142A are scaled using the first microlens array 150 arranged in series with the second microlens array 152 using the scaling operation 112 discussed above, which locally shrinks each focal point of the corresponding multi-focal pattern 128 by a predetermined factor (e.g., a factor of 2). In one embodiment, the first microlens array 150 collimates the multi-focal pattern 128 of the in-focus fluorescence emissions 142A, while the second microlens array 152 receives the collimated in-focus fluorescence emissions 142A and modifies the focal length of the collimated in-focus fluorescence emissions 142A such that each focal point is reduced by a predetermined factor to achieve a local shrinkage of the multi-focal pattern 128. In some embodiments, the scaled in-focus fluorescence emissions 142B caused by each multi-focal pattern 128 are then redirected by the first reflector 154 to pass through the fourth lens 158 to focus the scaled in-focus fluorescence emissions 142B onto the second reflector 156. The second mirror 156 redirects the scaled in-focus fluorescence emission 142B to pass through the emission filter 160, which allows only the scaled in-focus fluorescence emission 142B having a specific wavelength range (e.g., 515 nm) to pass through the emission filter 160. Once the scaled in-focus fluorescence emission 142B is filtered, the scanning device 134 rasterizes the fluorescence emission 142B onto the detector 164 through the fifth lens 162, so that the scaled in-focus fluorescence emission 142B of each multi-focal pattern 128 is collected and the in-focus fluorescence emission 142B is summed by the detector 164 to produce a composite image 143. The process of collecting the fluorescence emission 142B caused by the multi-focal pattern 128 is repeated until the entire field of view has been illuminated and all resulting fluorescence emissions 142B are collected by the detector 164.

[0053] In some embodiments, the detector 164 may be a camera whose shutter remains open for exposure as the scaled in-focus fluorescence emission 142B is rastered onto the detector 164 for collection until the entire field of view is illuminated and all collected data is received.

[0054] In some embodiments, composite images 143 are transmitted to processing system 166 , which performs a deconvolution operation 118 , which performs a deblurring function to enhance the resolution of each composite image 143 .

[0055] refer to Figure 4, one embodiment of a multi-focal SIM system 200 for generating a multi-focal pattern 223 may include an illumination source 201 that generates a single beam 203 that is transmitted through a beam expander 205 that produces an expanded single beam 203 that is reflected by a beam steering mirror 207 for a scanning operation 204 that scans the expanded beam 203 onto a beam splitter 209, such as a commercially available digital micromirror device (DMD) or a swept field confocal unit. In some embodiments, the DMD generates and switches the multi-focal pattern 223, where each focus is an illuminated spot on the multi-focal pattern 223. Each illumination spot is created by a single DMD mirror pixel in the ON position, such that a portion of the expanded beam 203 is reflected from the single DMD mirror pixel. The beam splitter 209 performs an illumination pattern generating operation 202 that splits the scanned expanded light beam 203 into a plurality of expanded light beams 203A that together form a sequence of multi-focal patterns 223. The plurality of expanded light beams 203A for each multi-focal pattern 223 pass through the first tube lens 211 to directly pass through the dichroic mirror 213 along the direction C.

[0056] After passing directly through the dichroic mirror 213, the expanded light beam 203A is focused by the objective lens 215 onto the sample plane for sample illumination 206 of the sample 217 and generating a plurality of fluorescent emissions 222 emitted by the sample 217. In a collection operation 208, the fluorescent emissions 222 are focused back through the objective lens 215 and onto the dichroic mirror 213 along a direction D perpendicular to the direction C, such that the fluorescent emissions 222 of each multi-focal pattern 223 are redirected and pass through a second tube lens 219. The second tube lens 219 focuses the fluorescent emissions 222 onto the detector 221, thereby collecting each multi-focal pattern 223 in the form of collected data 227 for transmission to the processing system 225, which filters out out-of-focus light and performs a scaling and summing operation 210. The process of collecting the fluorescent emissions 222 caused by each multi-focal pattern is repeated until the entire field of view has been illuminated. In some embodiments, processing system 225 may include a processor 230 in operable communication with detector 221 that processes collected data 227 stored in a database 232 (such as a computer-readable medium) by executing instructions that will be discussed in more detail below.

[0057] In one embodiment, the processing system 225 filters out the out-of-focus fluorescence emissions 222 and performs a scaling and summing operation 210 on the collected data 227 for each multi-focal pattern 223 using a computerized process for processing the collected data 227. Figure 7, a flow chart illustrates a method of performing out-of-focus fluorescence filtering, scaling and summing operations 210 using a processing system 225. At box 1000, the processing system 225 performs automatic lattice detection of a multi-focal illumination pattern 223. Specifically, the automatic grid detection determines the location of each illumination focus of a particular multi-focal pattern 223 in order to perform pinhole masks and local shrinkage of various fluorescence focuses. For example, the automatic grid detection determines five vectors to fully specify the location of all foci of each fluorescence emission 222 in a particular multi-focal pattern 223 in an image acquisition series. In some embodiments, the five vectors may be a grid vector, a shift vector, and an offset vector. Two grid vectors specify a two-dimensional displacement between any two adjacent illumination spots (e.g., foci). In this way, any grid point that is displaced by a grid vector will fall on another grid point. Due to thermal shift, the exact location of each illumination spot can change over time, so using the following method to accurately extract all vectors for each measurement data set from the collected data 227 provides the best results.

[0058] refer to Figure 8, illustrates a method for accurately extracting all vectors of each measurement data set of the collected data 227 when the processing system 225 performs the out-of-focus filtering, scaling and summing operation 210. The offset vector specifies the absolute position of the illumination spot that is closest to the center of the multi-focal pattern in any one image acquisition series processed by the processing system 225. The shift vector specifies the distance that each illumination spot of a particular multi-focal pattern 223 is moved between consecutive images. Because the multi-focal pattern 223 is rasterized in a two-dimensional plane, a "fast" shift vector is applied at each step of the operation 210, and a "slow" shift vector is applied when the "fast" shift vector has completed the illumination of a row. As used herein, the term "fast" shift vector specifies those grid points in a single row, and the term "slow" shift vector specifies consecutive rows. At box 2000, each raw image representing the fluorescence 142B captured from a particular multi-focal pattern 223 is multiplied by a Hann window (which prevents ringing in the Fourier domain) and Fourier transformed. At box 2002, the resulting Fourier amplitudes are averaged to obtain a high signal-to-noise ratio measurement of the peak position in Fourier space. At box 2004, the averaged resulting Fourier amplitudes are filtered by a bandpass filter. At box 2006, the search for spikes or peaks in the Fourier domain is completed. Because the original image of the Fourier amplitudes used for summing is expected to be a periodic grid of peaks, the size of each periodic grid is determined by determining the local Fourier maximum (or grid element, or noise). At box 2008, the harmonics of the three lowest spatial frequency peaks are determined by computationally searching for the peak intensity at and near multiples of the expected frequency. At box 2010, given the location of the three lowest spatial frequency peaks with harmonics, the peaks located at their vector sum and vector difference are searched, and these vectors constitute the grid vectors. In this method, peaks in a periodic grid predict the location of other peaks in the same grid, where each grid point is located at the vector sum of an integer number of Fourier grid vectors. At box 2012, the processing system 225 solves the resulting system of equations by linear squares to obtain three grid vectors. Any single peak in the Fourier domain will have some position error due to noise or pixelation. Using the least squares method, the information provided by each peak produces a more accurate grid size measurement. At box 2014, the processing system 225 sums the grid vectors to determine the error vector, and then subtracts one-third of the error vector from each Fourier grid vector. At box 2016, any two of the two Fourier grid vectors are selected and transformed to obtain real space grid vectors. It has been found that obtaining two grid vectors for a multi-focal illumination pattern makes it substantially easier to measure the offset vector and the shift vector.

[0059] refer to Fig. 9, shows a method for determining an offset vector when the processing system 225 performs the out-of-focus blur filtering, scaling and summing operation 210. At box 3000, a first raw image is median filtered. At box 3002, a grid of points from the measured grid vector is constructed using a zero offset vector. At box 3004, a collection of smaller images centered at each grid point are extracted from the median filtered raw image using interpolation for sub-pixel centering. At box 3006, the smaller images are averaged to form a "grid average" image. At box 3008, the brightest pixel in the grid average image is determined. At box 3010, an interpolation technique is used to estimate the sub-pixel position of each intensity peak in the grid average image.

[0060] refer to Fig.10 , shows a method for determining the shift vector when the processing system 225 performs the out-of-focus blur filtering, scaling and summing operation 210. At box 4000, a set of expected Fourier peak positions is constructed from the measured Fourier grid vectors. At box 4002, the extracted phase is extracted for each image number of the transformed collected data 227. At box 4004, the processing system 525 performs phase unwrapping on each series of multi-focus patterns 223 to remove the 2π jump. It has been noted that the amplitude of the Fourier transform of each original image is independent of the shift of the original image. Information about the displacement is encoded in the phase of the Fourier transform. At box 4006, the processing system 225 reshapes each phase series in the two-dimensional series into a two-dimensional array having the same size as the scan pattern (e.g., 16 pixels×14 pixels). After reshaping each phase series, at box 4008, a straight line is fitted to the average slope of each phase series in the "fast" and "slow" directions. In each successive raw image detected, the illumination is shifted by X FAST At the end of each row, the lighting is also shifted by X SLOW If the original image is shifted by X, then the phase of the peak in Fourier space at position k is shifted by k*X. At block 4010, the processing system 225 solves the resulting system of equations for X FAST and X SLOW At block 4012, offset vectors are constructed for the first and last frames of the series of multi-focus patterns 223. The offset vector for the last frame is calculated using the same process as that for the first frame, which provides a strong constraint that greatly increases the accuracy of the estimate. At block 4014, based on X FAST and X SLOW Once the predicted offset vector is constructed, the difference between the two vectors is determined as an error vector, which is divided by the number of original images in the scan and obtained from X SLOW Minus.

[0061] refer to Fig.11 , shows a method of a pinhole mask for out-of-focus blur filtering when the processing system 225 performs the out-of-focus blur, scaling and summing operation 210. At box 5000, for each raw image collected during the collection operation 208, a set of expected illumination positions based on the vectors determined above is constructed. At box 5002, smaller sub-images centered at each illumination position are extracted using interpolation techniques, and then at box 5004, each centered sub-image is multiplied by a two-dimensional Gaussian mask by the processing system 525. These process steps simulate the effects of a physical pinhole. At box 5006, the background can optionally be subtracted from each sub-image, and the centered sub-image can be multiplied by a correction factor given by the calibration data. At box 5008, each hot pixel can optionally be median filtered by the processing system 525.

[0062] refer to Fig.12 , shows a method for scaling and performing local shrinkage when the processing system 225 performs the out-of-focus blur filtering, scaling and summing operation 210. At block 6000, each masked sub-image is resampled to reduce the masked sub-image by a factor of approximately two. At block 6002, each scaled masked sub-image is added to the original image being processed.

[0063] refer to Fig.13 , shows a method for summing when the processing system 225 performs the out-of-focus blur filtering, scaling and summing operation 210. At block 7000, the processed raw images are summed to form a composite image. At block 7002, the raw images may optionally be summed to produce a calculated "wide field of view" image.

[0064] The resulting composite image produced by the out-of-focus blur filtering, scaling and summing operation 210 has been shown to have a resolution better than √2 times that of a typical wide-field image produced by conventional wide-field microscopy techniques. Similar to conventional spinning disk or swept-field confocal microscopy, the out-of-focus blur filtering, scaling and summing operation 210 also greatly improves optical sectioning.

[0065] refer to Fig.18, an embodiment of an MSIM system designated as 300 is shown. MSIM system 300 may be a swept field hardware arrangement in which a similar degree of resolution enhancement as discussed above for MSIM systems 100 and 200 is achieved. The primary difference between MSIM system 300 and MSIM systems 100 and 200 is that operations associated with the zoom, pinhole, and summing steps are implemented in MSIM system 300 using hardware rather than software required for MSIM systems 100 and 200. In this embodiment, MSIM system 300 may include a light source 302 that emits a beam 305, which is transmitted through a converging (+) microlens array 304, and the resulting focused beam is relayed through a first scan lens 306, scanned by a galvanometer 308, and then through a second scan lens 310, wherein the first scan lens 306 and the second scan lens 310 are in a 4f configuration with respect to the intermediate image plane.

[0066] As further shown, the galvanometer 308 can be located at the focal point between the first and second scan lenses 306 and 310 and sweep the excitation focal point across the sample plane 316, thereby producing a swept field of excitation covering the imaging field. In addition, a telescopic arrangement of a tube lens 312 and an objective lens 314 is located between the second scan lens 310 and the sample plane 316, which telescopic arrangement demagnifies the intermediate stage and produces an array of excitation focal points that are swept across the sample plane 316 by the galvanometer 308. The resulting fluorescence generated by the excitation focal points swept across the sample plane 316 returns along the same path through the objective lens 314 and the tube lens 312, which is scanned by the galvanometer 308, but is diverted using a dichroic mirror 318, which is located between the converging microlens array 304 and the first scan lens 306. Once diverted, the fluorescence emission passes through the pinhole array 320, thereby greatly reducing out-of-focus fluorescence emission.

[0067] The resulting in-focus fluorescence emission is then relayed using a 4f telescopic pair consisting of a first relay lens 322, which focuses the in-focus fluorescence emission onto a first mirror 324, which diverts the in-focus fluorescence emission to pass through a second relay lens 330, which is then diverted by a second mirror 326 and a third mirror 328 to a second converging (+) microlens array 332. In one embodiment, the second converging (+) microlens array 332 can be located one focal length in front of the focus that would otherwise be formed by the second relay lens 330, thereby producing an erect (non-inverted) image of the fluorescence emission focus with one-half magnification. The erect image is then relayed by another telescopic arrangement of a third scan lens 334 and a fourth scan lens 336 arranged in a 4f configuration, where it can be rescanned by a galvanometer 308 located at the focus between the third scan lens 334 and the fourth scan lens 336. A camera 338 with an emission filter 340 captures the final image. The MSIM system 300 can generate enhanced resolution images such as Fig.19 Figure 2 shows AlexaFluor 488nm labeled microtubules in fixed U2OS cells. The apparent width of each microtubule is approximately 200nm. Fig.19 The image shown in is original without any post-processing deconvolution, which is expected to further increase the resolution of the image.

[0068] Fig. 20 An example of a spinning disk confocal microscopy system designated as 400 is shown. System 400 may include a light source 402 for generating a light beam that passes through a spinning disk 404 with a converging (+) microlens array and then through a spinning disk 406 with a matching pinhole array. The resulting excitation is then imaged onto a sample 412 using an optical arrangement of a tube lens 408 and an objective lens 410. In this arrangement, the rotation of both the spinning disk 404 with a converging (+) microlens array and the spinning disk 406 with a matching pinhole array is performed in tandem so that the created excitation focus covers the field of view of the sample 412. Fluorescent emission originating from the sample 412 then passes back through the objective lens 410, the tube lens 408, and the spinning disk 406 with a matching pinhole array. The fluorescent emission is then diverted using a dichroic mirror 414 so that the fluorescent emission passes through a telescope arrangement 416, which is captured by a camera 420 through an emission filter 418.

[0069] Reference Fig.21 , shows another embodiment of an MSIM system with spinning disk multi-focus structured illumination microscopy, designated as 500, based on Fig. 204. In one embodiment, a conventional spinning disk confocal microscope is converted into a resolution doubling device with one major modification. The MSIM system 500 may include a light source 502 that generates a light beam that is redirected by a dichroic mirror 504, passes through a spinning disk 506 with a converging (+) microlens array and then passes through a spinning disk 508 with a pinhole array before being focused by a tube lens 510 and an objective lens 512 onto a sample 514. Once the sample 514 is illuminated, the fluorescent emission generated by the sample 514 passes back through the objective 512 and the tube lens 510, which focus the fluorescent emission through a spinning disk 516 with a diverging (-) microlens array positioned along the optical axis, which is caused to rotate synchronously with the spinning disk 506 with a converging (+) microlens array and the spinning disk 508 with a pinhole array before the fluorescent emission passes through the telescope arrangement 518 and is then captured by the camera 520 after the fluorescent emission is filtered by the emission filter 522. In one embodiment, the spinning disk 516 with a diverging (-) microlens array should contain the same number of microlenses as the spinning disk 506 with a converging (+) microlens array, and the microlenses should be spaced at the same spatial locations on each of the spinning disks 506 and 516. In some embodiments, if the focal length of the microlenses used for the spinning disk 516 with a diverging (-) microlens array is half the focal length of the spinning disk 506 with a converging (+) microlens array, and the spinning disks 506 and 516 are spaced apart by the difference in focal lengths, then this arrangement will form a Galilean telescope with 1 / 2 magnification. If such a telescope is positioned at an appropriate distance from the upright, reduced image thus created, the rotation of the spinning disks 516 and 506 synchronized with the camera exposure performs the desired pinhole, zoom and summing operations required for resolution doubling. An important advantage of the spinning disk arrangement of the MSIM system 500 is that the number of transmitting optical devices is significantly reduced, with nine replacing seventeen. Another advantage is that if the spinning disk 516 with a diverging (-) microlens array can be easily manufactured, the optical setup for the MSIM systems 300, 400 and 500 may be easier to align than a conventional scanning field microscope, making the MSIM technique easy to convert to a spinning disk.

[0070] test

[0071] To investigate the potential of the Multifocal SIM System 200 for biological imaging, antibody-labeled microtubules were imaged in human osteosarcoma (U2OS) cells embedded in fluoromount. Fig.14), microscope system, sample preparation and data processing of captured images. The use of multi-focus patterns in combination with deconvolution allows us to study a variety of samples at an imaging rate of 1 Hz with a resolution as low as 145 nm laterally and 400 nm axially. Compared to conventional structured illumination microscopy systems, the three-dimensional image samples provided by the Multifocus SIM System 200 are 5-8 times thicker than those of conventional structured illumination microscopy systems. In the current study, microtubules were imaged in live transgenic zebrafish embryos at a depth greater than 45 μm from the coverslip surface. In addition, a four-dimensional SIM dataset of GFP-tagged histones in live nematode embryos was obtained.

[0072] For testing, a periodic grid approximating equilateral triangles was used for our illumination spot locations because, for a given spot density, this particular pattern maximizes the distance between any two nearest neighbor spots, thereby minimizing crosstalk. The multifocal illumination pattern was translated one digital micromirror device (DMD) pixel at a time, which corresponds to a step size of 120 nm in the sample plane. Larger step sizes do not illuminate the sample uniformly given visible streak artifacts, while smaller step sizes increase acquisition time and dose without increasing image quality.

[0073] The multifocal pattern was imaged onto the sample, which was mounted on a commercial inverted microscope, and a scientific-grade complementary metal oxide semiconductor camera (sCMOS) was used to record a raw image for each multifocal pattern position. By varying the spacing between illumination points, acquisition speed can be traded for section quality. It was found that widely spaced focuses had less crosstalk, but additional multifocal illumination patterns were required to evenly illuminate the sample. In contrast, dense focuses had more crosstalk, but correspondingly fewer multifocal patterns were required to evenly illuminate the sample. It has been found that a multifocal pattern with 16 pixel horizontal spacing and 14 pixel vertical spacing between scan points provides good results in the biological samples studied. For a 480 pixel x 480 pixel field of view, the resulting 224 raw exposures taken at 222 Hz correspond to a super-resolution image acquisition rate of approximately 1 Hz.

[0074] To investigate the potential of the Multifocal SIM System 200 for biological imaging, we imaged antibody-labeled microtubules in human osteosarcoma (U2OS) cells embedded in fluorescent mounting medium, as shown in Figure 2. Figures 15A-15DAs shown in the image shown. Compared to the wide field image, the multi-focal illuminated, pinhole, summed and scaled image produced by the SIM system 200 improves image resolution, including image contrast. In addition, parallel iterative deconvolution further improves the resulting composite image, which reveals features that were previously blurred by diffraction. The full width at half maximum (FWHM) intensity of the microtubules in the multi-focal SIM system 200 image is 145nm, which is two times higher than wide field imaging. Similar experiments on 110nm sub-diffraction beads confirmed this result (multi-focal SIM system 200 FWHM 146+ / -15nm vs. wide field FWHM 284+ / -32nm, N=80 beads). Assuming the same 222Hz native frame rate for each microscope system, the total acquisition time for a 48μm×49μm field of view is approximately 1 second, which is 6500 times higher than a conventional image scanning microscope (ISM) system.

[0075] The applicability of the multifocal SIM system 200 for dual labeled three-dimensional samples was also investigated, as shown in the sample shown in FIG16 . Z-stack images of fixed cells embedded in fluorescent mounting medium were used. The volumes obtained for microtubules immunolabeled with Alexa Fluor 488 and mitochondria stained with Mitotracker Red were 3.7 μm thick, with each slice separated by 100 nm. Compared to wide-field images obtained with the same total illumination dose, the three-dimensional multifocal SIM system 200 images provide a significant increase in image contrast, due to the combination of physically removing out-of-focus light (via a digital pinhole) and computationally removing out-of-focus light (via three-dimensional deconvolution).

[0076] The final composite image produced by the multifocal SIM system 200 has an approximately two-fold improvement in resolution over widefield imaging; microtubules and "worm-like" mitochondria are better resolved. For example, better resolution of sub-diffraction gaps at the ends of each mitochondria is achieved, including microtubule pairs separated by more than 200nm. Surprisingly, the multifocal SIM system 200 also improves axial resolution by approximately two-fold over widefield images, as microtubules have an apparent axial FWHM of approximately 400nm. This result was confirmed on 100nm sub-diffraction beads (multifocal SIM system FWHM 402+ / -49nm; widefield 826+ / -83nm, N=80 beads).

[0077] The multifocal SIM system 200 is also applied to three-dimensional imaging of thicker live samples, where the pinhole operation physically filters out out-of-focus light that would otherwise swamp the in-focus optical signal. To demonstrate this capability, live, immobile zebrafish embryos expressing a GFP transgene that labels microtubules were imaged.

[0078] 241 slices were acquired at a 2D imaging rate of 1 Hz, separated by 0.2 μm, using multifocal illumination according to the Multifocal SIM System 200. After pinhole focusing, zooming and 3D deconvolution, a volume with a thickness of 48.2 μm was achieved, as shown in Figures 17A-17D The image of the sample shown is shown. Structural features such as the boundary between two adjacent somites, the alignment of microtubules along the somite boundary, and microtubule-free regions corresponding to the nuclei of developing muscle cells are clearly visible in the stack. An estimate of the z position of subsequent deeper nuclei within the stack suggests that the imaged volume contains 6 to 7 cell layers.

[0079] In one test, the imaging rate of the multi-focal SIM system 200 captured dividing cells in the epidermis without significant motion blur in the image. The resolution enhancement of the multi-focal SIM system 200 was achieved throughout the volume, as the spacing between microtubule pairs at the site of cell division was resolved to better than 200 nm, and the microtubules in the epidermis had a FWHM of 175+ / -33 nm (N=30) laterally and 496+ / -65 nm (N=21) axially.

[0080] Lighting system

[0081] In the illumination system, all optics were mounted on an optical bench (Kinetic Systems, Vibraplane Model #5704-3660-23SPL) to minimize mechanical vibrations. To excite fluorescence, two lasers were used: a 150 mW, 561 nm laser (561, Coherent, Sapphire 561-150 CW CDRH) and a 200 mW, 488 nm laser (488, Coherent, Sapphire 488-200 CDRH). Illumination was controlled using mechanical shutters (Thorlabs, SH05 and SC10) placed after each laser. The beam was combined by a dichroic mirror (DC, Chroma, 525dcxru) and expanded 6.7 times by a beam expander consisting of two achromatic lenses (Edmund, f=30 mm, NT49-352-INK; and Thorlabs, f=200 mm, AC254-200-A-ML0). The expanded beam was directed to a digital micromirror device (DMD, Digital Light Innovations, D4100 DLP 0.55" XGA) with a 24 degree off normal such that in the ON position the micromirrors tilted the output beam perpendicular to the DMD face. The center order of the resulting illumination pattern was reduced by a factor of 1.5 by a beam reducer (Thorlabs, f=75mm, AC254-075-A-ML and f=50mm, AC254-050-A-ML0 aligned in a 4f configuration) such that the DMD face was re-imaged to the back focal plane of the 180mm tube lens inside a microscope (Olympus, IX-81). These components are shown in Fig.14 After entering the left port of the microscope, the beam sequentially passes through (i) a tube lens; (ii) a dichroic mirror (Chroma, zt405 / 488 / 561); and (iii) a 60x objective (Olympus, PlanApo, NA 1.45TIRF for single cells, or UPLSAPO 60XS, NA 1.3 for zebrafish and worm embryos), resulting in a total reduction of 90x between the DMD and the illuminated sample. The illumination at the sample covers a circular area of ​​approximately 50 μm in diameter.

[0082] Microscope system

[0083] Structured illumination microscopy (SIM) imaging was performed on an Olympus IX81 inverted microscope equipped with left and right ports and an automated XY stage (200 μm range, Applied Scientific Instrumentation, PZ-2000) with an attached Z piezo stage. Patterned excitation (e.g., multifocal illumination pattern) created by the DMD was introduced into the microscope via the left port. Fluorescence emitted by the illuminated sample was collected by the objective, reflected by the dichroic mirror (Chroma, zt405 / 488 / 561), passed through the 180 nm tube lens inside the microscope, appropriately filtered to filter out pump light (Semrock, LP02-488RE-25 and NF03-561E-25), and detected by a scientific-grade complementary metal oxide semiconductor (sCMOS) camera (Cooke, pco.edge) mounted on the right port. Correct alignment of the sCMOS along the optical axis is critical to achieving near diffraction-limited performance. To help position the camera correctly, a 60x objective is often used to image with a 10x air objective (Olympus, CPlanFl 10x, 0.3NA), which is more sensitive to errors in axial alignment. A fixed illumination pattern (similar to that used in SIM) is placed on the fluorescent lake sample, and the camera is translated along the optical axis until the apparent size of each illumination spot is minimized.

[0084] Sample preparation

[0085] U2OS cells were cultured in standard culture medium (DMEM-HG (Invitrogen, 11960), sodium pyruvate (Invitrogen, 11360), GlutaMAX (Invitrogen, 35050) and 10% heat-inactivated fetal bovine serum (Invitrogen, 11082)) on ethanol-sterilized, cleaned #1.5 25 mm diameter coverslips (Warner Instruments, 64-0715). To stain samples for microtubules, cells were incubated in Cytoskeletal buffer (CB, 10 mM MOPS, 138 mM KCl, 2 mM MgCl) for 3 s.l. 2 , 2 mM EGTA, 0.01% NaN 3 The cells were fixed in a mixture of 0.5% glutaraldehyde, 0.37% formaldehyde, and 0.3% Triton X-100 in 160 mM sucrose, pH 6.1. After fixation, the cells were washed in CB, quenched with 100 nM glycine, washed in CB, and washed in antibody dilution buffer (AbDil, 150 mM NaCl, 20 nM Tris, 0.1% Triton X-100, 0.1% NaN3 and 2% bovine serum albumin, pH 7.4). Primary monoclonal antibodies (Invitrogen, 32-2500) were incubated with cells diluted to 2 μg / mL in AbDil for one hour at room temperature. After the primary antibody incubation, the cells were eluted in phosphate-buffered saline and then incubated with secondary Alexa Fluor 488 labeled antibodies (Invitrogen, A-11001) at a dilution of 1:200 in AbDil for 1 hour.

[0086] Prior to fixation, samples for two-color experiments were initially stained with Mitotracker Red (Invitrogen, M7512) according to the manufacturer's instructions. After mitochondrial labeling, microtubules were stained using the above procedure. All samples were placed in fluorescent mounting medium G (Electron Microscopy Solutions, 17984-25) on standard 25 mm x 75 mm slides (SPI supplied, #01251-AB) and sealed with nail polish.

[0087] a) Sub-diffractive beads

[0088] Yellow-green or red fluorescent beads (Invitrogen, F8803, 110 nm diameter; Invitrogen F8801, 100 nm diameter) were used for all point spread function (PSF) measurements. The beads were diluted with a 1:1300 slurry concentration (1:200 in distilled water, 1:13 in ethanol) and spread on a clean glass cover slip. After air drying for 5 minutes to evaporate the ethanol, the cover slip was washed twice in distilled water to remove unattached beads. After air drying again, the beads were mounted in fluorescent mounting medium or silicone oil on a slide and sealed with nail polish.

[0089] b) Zebrafish samples

[0090] Zebrafish Tg carrying dclk2-GFP transgene (XlEef1a1:dclk2-GFP) io008 Embryos used in Figure 16A-16B Thick MSIM experiments shown. To construct this, the linear plastids containing the transgene were injected into one-cell zebrafish embryos along with Tol2 mRNA. Fluorescent embryos were raised to adulthood and crossed to select for germline transmission by screening the offspring for GFP expression.

[0091] Collect Tg(XlEdf1a1:dclk2-GFP) through natural spawning io0008Embryos were maintained at 28 degrees Celsius. Embryos 24 hours post fertilization were anesthetized with tricaine (Sigma, E105210, 600 μM final concentration in embryo culture medium (60 mg instant sea salt (Petsmart) per liter of double distilled water) before imaging by the multifocal SIM system 200. Anesthetized embryos were placed on round coverslips, fixed in 1% low melting point agarose (Cambrex, 50080), placed in round coverslip holders (ASI, I-3033-25D), covered with embryo culture medium, and imaged at room temperature.

[0092] Data processing

[0093] After the raw images are acquired using the above-described illumination and microscope systems, each set of collected raw images of the sample is processed into a super-resolution image using a processing system 225 having software written in the Python programming language. The processing steps employed by the processing system 225 are: (i) automatic grid detection to accurately determine the corresponding positions of the illumination points; (ii) digital pinhole masking around each detected illumination spot to filter out-of-focus light, and optical flat field processing using calibration data; (iii) local shrinkage (e.g., scaling) and resampling of the area around each illumination spot to increase the resolution by a factor of √2; (iv) summing the processed raw images to produce a super-resolution composite image; and (v) restoring the full 2x resolution enhancement using conventional deconvolution techniques. These processing steps of focusing, scaling, and summing operations 210 implemented by the processing system 225 of the multifocal SIM system 200 are discussed in more detail above.

[0094] It should be understood from the foregoing that although specific embodiments have been illustrated and described, various modifications may be made thereto without departing from the spirit and scope of the present invention, which will be apparent to those skilled in the art. Such changes and modifications are within the scope and teachings of the present invention, as defined in the appended claims of the present invention.

Claims

1. A microscopy system (500), comprising: a light source (502) for transmitting a single light beam; a first rotating disk (506) having a converging microlens array, rotating in a first direction and positioned along the optical axis, for splitting the single light beam into a plurality of light beams to form a multi-focal pattern; a second rotating disk (508) having a pinhole array and rotating in a first direction and positioned along the optical axis, for blocking out-of-focus light beams among the plurality of light beams of each multi-focal pattern and allowing in-focus light beams among the plurality of light beams to pass through the rotating disk (508) having the pinhole array, a third rotating disk (516) having a diverging microlens array rotating in a first direction and positioned along the optical axis, wherein the first rotating disk (506) having a converging microlens array, the second rotating disk (508) having a pinhole array, and the third rotating disk (516) having a diverging microlens array rotate synchronously relative to each other and generate a scaled in-focus image without requiring software steps of scaling, pinholeing, and summing the multiple light beams; an emission filter positioned along the optical axis for allowing fluorescent emission having a specific wavelength range to pass through the emission filter and be captured by a camera; as well as A telescope arrangement (518) is positioned between the emission filter and the third rotating disk (516) having a diverging microlens array.

2. The microscope system (500) according to claim 1, wherein the focal length of the microlenses of the third rotating disk (516) having the divergent microlens array is half the focal length of the microlenses embedded in the first rotating disk (506) having the converging microlens array.

3. The microscope system (500) according to claim 1, wherein the third rotating disk (516) having a diverging microlens array has the same number of microlenses as the first rotating disk (506) having a converging microlens array.

4. The microscope system (500) according to claim 1, wherein the microlenses of the third rotating disk (516) having a diverging microlens array are spaced apart at the same spatial position as the first rotating disk (506) having a converging microlens array.

5. The microscope system (500) according to claim 3, wherein the spacing between the first rotating disk (506) having the converging microlens array and the rotating disk (516) having the diverging microlens array is determined by the difference in focal lengths of the first rotating disk (506) having the converging microlens array and the rotating disk (516) having the diverging microlens array, thereby arranging a Galilean telescope with a one-half magnification.

Images