Spectral unmixing of fluorescence imaging using radiofrequency multiplexed excitation data

Abstract

The present disclosure includes embodiments of systems, devices, and methods for sorting multiple cells of a sample. Multiple raw images including complex-valued pixels in frequency space can be generated from fluorescence intensity data of multiple channels of fluorescence emission of fluorophores, which fluorescence emission is induced by fluorescence imaging using radio frequency multiplexed excitation in time space. The raw images can be spectrally unmixed before making a sorting decision.

Description

[0001] Cross-reference to related applications and fields

[0002] The present invention claims priority under 35 U.S.C. §119(e) to U.S. Provisional Patent Application No. 62 / 822,789, filed on March 22, 2019, the disclosure of which is incorporated herein by reference. The present invention generally relates to apparatus and methods for determining characteristics of particles flowing through a flow cytometer, such as via sample fluorescence analysis, and more particularly to apparatus and methods for sorting particles, such as in a flow cytometer to sort cells based on cell characteristics. Background Art

[0003] Separating subpopulations and even single cells from heterogeneous populations has multiple applications in modern biology and biomedicine. Some conventional techniques for separating cell subpopulations include fluorescence activated cell sorting (FACS), magnetic activated cell sorting (MACS), laser capture microdissection and DEP array sorting. Although these techniques are often used in cell sorting applications, there are many disadvantages. For example, FACS, which is widely used in all fields of cell biology, lacks subcellular resolution, and therefore only makes sorting decisions based on the mean value of cell parameters. In addition, due to the high waiting time when making sorting decisions based on conventional sorting methods of imaging cells, it is usually not used in high-throughput cell separation applications. Therefore, it is necessary to be used for, for example, in a flow cytometer system to sort cells. Improved methods and systems. Summary of the invention

[0004] The content disclosed herein includes embodiments of systems, devices, and methods for sorting multiple cells in a sample. The method includes: under the control of a hardware processor (or a general processor, such as a virtual processor): irradiating a cell in a sample including multiple cells with a laser beam having multiple different optical frequencies to induce fluorescence emission of multiple fluorophores associated with the cell. The method can include: detecting fluorescence emission at different wavelengths to obtain fluorescence intensity data of multiple channels corresponding to multiple fluorophores in time space. The method can include: generating multiple raw images of multiple pixels including complex values ​​in frequency space from the fluorescence intensity data of multiple channels. The method can include: for each of the fluorescence intensity data of multiple channels, using one or more channel phase adjustments to generate multiple phase-adjusted images of multiple pixels including complex values ​​in frequency space from multiple raw images. The method can include: using an unmixing matrix on the complex values ​​of corresponding pixels of multiple phase-adjusted images to generate multiple unmixed images of multiple pixels including complex values ​​in frequency space from multiple phase-adjusted images. The method can include generating a plurality of phase-corrected images comprising a plurality of pixels in frequency space that are complex or real values ​​based on a property of each of a plurality of fluorophores from a plurality of unmixed images. The method can include determining a sorting decision for the cell based on the plurality of phase-corrected images.

[0005] The content disclosed herein includes an embodiment of a method for sorting multiple cells in a sample. The method includes: under the control of a processor (e.g., a hardware processor, and a virtual processor): irradiating cells in a sample including multiple cells with a laser beam having multiple different optical frequencies to induce fluorescence emission of multiple fluorophores associated with the cells. The method can include: detecting fluorescence emissions at different wavelengths to obtain fluorescence intensity data of multiple channels corresponding to multiple fluorophores in time space. The method can include: generating multiple original images including complex values ​​in frequency space from the fluorescence intensity data of the multiple channels. The method can include: using an unmixing matrix on the corresponding complex values ​​of the multiple original images to generate multiple unmixed images including complex values ​​or real values ​​in frequency space from the multiple original images. The method can include: determining a sorting decision for the cell based on the multiple unmixed images.

[0006] The disclosure herein includes embodiments of a method for sorting a plurality of cells in a sample. The method can include: receiving, under control of a hardware processor, fluorescence intensity data of a plurality of channels of fluorescence emission at different wavelengths from a plurality of fluorophores associated with a cell in a sample including the plurality of cells, the fluorescence intensity data of the plurality of channels corresponding to the plurality of fluorophores in time and space. Inducing the fluorescence emission after irradiating the cell with a laser beam having a plurality of different optical frequencies. The method can include: generating a plurality of raw images including complex values ​​in frequency space from the fluorescence intensity data of the plurality of channels. The method can include: using an unmixing matrix on corresponding complex values ​​in the plurality of raw images to generate a plurality of unmixed images including complex or real values ​​in frequency space from the plurality of raw images. The method can include: determining a sorting decision for the cell based on the plurality of unmixed images. The method can include: irradiating the cell with a laser beam to induce fluorescence emission of the fluorescence intensity data of the plurality of channels in time and space.

[0007] In some embodiments, the plurality of raw images include a plurality of pixels of complex values ​​of the plurality of raw images. In some embodiments, the method can include: for each of the fluorescence intensity data of the plurality of channels, using one or more channel phase adjustments to generate a plurality of phase adjusted images including complex values ​​in frequency space from the plurality of raw images, wherein generating a plurality of unmixed images includes: using an unmixing matrix on corresponding complex values ​​of the plurality of phase adjusted images to generate a plurality of unmixed images from the plurality of phase adjusted images. The plurality of phase adjusted images can include a plurality of pixels of complex values ​​of the plurality of phase adjusted images. In some embodiments, the method can include: based on a property of each of the plurality of fluorophores, generating a plurality of phase corrected images including complex values ​​in frequency space from the plurality of unmixed images, wherein determining a sorting decision includes: determining a sorting decision for the cell based on the plurality of phase corrected images. The plurality of phase corrected images can include a plurality of pixels of complex values ​​of the plurality of phase corrected images.

[0008] In some embodiments, generating multiple raw images includes: using a time-frequency space transform to generate multiple raw images in frequency space from the fluorescence intensity data of multiple channels. The time-frequency space transform can include a Fourier transform. The Fourier transform can include a discrete Fourier transform. The discrete Fourier transform can include a fast Fourier transform. The Fourier transform can include a sliding window Fourier transform. The size of the sliding window of the sliding window Fourier transform can be m1. Generating multiple raw images can include using a time-frequency transform matrix representing the time-frequency space transform to generate multiple raw images.

[0009] In some embodiments, one or more channel phases include a bright field phase, a calibration phase, and a phase offset for each of the multiple channels. One or more channel phase adjustments can correct or account for channel phases, such as bright field phases, calibration phases, and phase offsets. Generating multiple phase adjustment images for each of the multiple channels can include: multiplying the complex values ​​of the original images in the multiple original images corresponding to the channel to generate a phase adjustment image in the multiple phase adjustment images corresponding to the channel. One or more channel phase adjustments can include or only include complex values. One or more channel phase adjustments can include one or more real values. Generating multiple phase adjustment images can include using one or more channel phase adjustment matrices representing one or more channel phase adjustments to generate multiple phase adjustment images. The method can include: determining or receiving the one or more channel phase adjustments.

[0010] In some embodiments, generating a plurality of unmixed images comprises: generating a vector comprising complex values ​​of corresponding pixels of a plurality of phase adjustment images; multiplying the vector by an unmixing matrix to generate an unmixing vector comprising unmixed complex values; and generating a plurality of unmixed images comprising corresponding pixels having unmixed complex values. The vector can be of size 1×n1, the unmixing matrix can be of size n1×n2, and / or the unmixing vector can be of size 1×n2. In some embodiments, generating a plurality of unmixed images comprises: for corresponding pixels in a plurality of phase adjustment images, using an unmixing matrix from a plurality of unmixing matrices on the complex values ​​of the corresponding pixels to generate a plurality of unmixed images from the plurality of phase adjustment images. The plurality of unmixing matrices can comprise m2 unmixing matrices. The unmixing matrix can comprise or comprise only real values ​​or complex values. The method can comprise: determining or receiving the unmixing matrix.

[0011] In some embodiments, generating a plurality of phase-corrected images includes: using a plurality of fluorophore phase corrections for a fluorophore in a plurality of fluorophores to generate a phase-corrected image in a plurality of phase-corrected images from an unmixed image in a plurality of unmixed images corresponding to the fluorophores of a plurality of channels. The plurality of fluorophore phase corrections for the fluorophores can be related to properties of the fluorophores. The properties of the fluorophores can include lifetimes of the fluorophores. The plurality of fluorophore phase corrections for the fluorophores can include complex values. The method can include: determining or receiving a plurality of lifetime corrections. Generating a plurality of phase-corrected images can include: generating a plurality of phase-corrected images using a fluorophore phase correction matrix representing a plurality of fluorophore phase corrections for the fluorophores.

[0012] In some embodiments, the method includes generating a plurality of visual representations of the plurality of phase-corrected images based on real parts of the complex values ​​of the plurality of phase-corrected images. The method can include generating a plurality of visual representations of the plurality of phase-corrected images based on amplitudes of the complex values ​​of the plurality of phase-corrected images. Determining a sorting decision can include determining a sorting decision for the cell based on the plurality of visual representations of the plurality of phase-corrected images. The method can include generating a combined visual representation from the plurality of visual representations.

[0013] In some embodiments, two or more of generating multiple original images, generating multiple phase adjustment images, generating multiple unmixed images, and generating multiple phase correction images are performed using a combination matrix. Generating multiple original images and generating multiple phase adjustment images can include: using a combination matrix to generate multiple phase adjustment images from fluorescence intensity data of multiple channels. Generating multiple unmixed images and generating multiple phase correction images can include: using a combination matrix including an unmixing matrix to generate multiple phase correction images from multiple phase adjustment images. Generating multiple original images, generating multiple phase adjustment images, generating multiple unmixed images, and generating multiple phase correction images can include: using a combination matrix including an unmixing matrix to generate multiple phase correction images from fluorescence intensity data of multiple channels.

[0014] In some embodiments, the method includes: generating corrected fluorescence intensity data for multiple channels from multiple phase-corrected images. Generating corrected fluorescence intensity data for multiple channels can include: using a frequency-time spatial transform to generate corrected fluorescence intensity data for multiple channels from multiple phase-corrected images. The method can include: determining an estimate of a cell feature based on the corrected fluorescence intensity data for multiple channels, wherein determining a sorting decision includes determining a sorting decision for the cell based on the estimate of the cell feature. The cell feature can include cell size, a ratio of the size of the cell in two different dimensions, co-localization of fluorescence emission by two or more of a plurality of fluorophores associated with the cell, a ratio of the size of the cell's cytoplasm and nucleus, a degree of punctateness of the cell's fluorescence emission, a measure of the spatial distribution of the fluorescence emission, a measure of cell position or orientation, a measure of cell eccentricity, a measure of similarity of the cell to a reference cell, a measure of the degree to which the cell is in the focus of a laser beam, or a combination thereof.

[0015] In some embodiments, the laser beam includes a reference laser beam and a plurality of radio frequency frequency shifted laser beams. The number of the plurality of radio frequency frequency shifted laser beams can be m2. The reference laser beam can overlap with the plurality of radio frequency frequency shifted laser beams in space. The plurality of radio frequency frequency shifted laser beams do not overlap with each other in space. One or more of the reference laser beam and the plurality of radio frequency frequency shifted laser beams can induce fluorescence emission of fluorophores in the plurality of fluorophores. Irradiating the cell can include irradiating a plurality of spatial positions of the cell simultaneously with the plurality of radio frequency frequency shifted laser beams. The plurality of spatial positions of the cell do not overlap with each other.

[0016] In some embodiments, the number of multiple original images, the number of multiple phase adjustment images, the number of multiple unmixed images, and the number of multiple phase correction images are the same. The number of multiple original images, the number of multiple phase adjustment images, the number of multiple unmixed images, and / or the number of multiple phase correction images can be n1. Multiple original images can be associated with a first time dimension and a first frequency dimension, multiple phase adjustment images can be associated with a second time dimension and a second frequency dimension, multiple unmixed images can be associated with a third time dimension and a third frequency dimension, and / or multiple phase correction images can be associated with a fourth time dimension and a fourth frequency dimension. Two or more of the first time dimension, the second time dimension, the third time dimension, and the fourth time dimension can have the same size and / or the same number of pixels along the dimension. The first time dimension, the second time dimension, the third time dimension, and the fourth time dimension can have the same size and / or the same number of pixels. The first frequency dimension, the second frequency dimension, the third frequency dimension, and / or the fourth frequency dimension have a size of m2 and / or m2 pixels. In some embodiments, m2 is equal to 1 / 2*m1. In some embodiments, m2 is less than 1 / 2*m1. Two or more of the first frequency dimension, the second frequency dimension, the third frequency dimension, and the fourth frequency dimension can have the same size and / or the same number of pixels. The first frequency dimension, the second frequency dimension, the third frequency dimension, and the fourth frequency dimension can have the same size and / or the same number of pixels. The pixel size and / or the number of pixels along the first frequency dimension, the pixel size and / or the number of pixels along the second frequency dimension, the pixel size and / or the number of pixels along the third frequency dimension, and / or the pixel size and / or the number of pixels along the fourth frequency dimension can be the same as the number of multiple radio frequency frequency-shifted laser beams. The pixels along the first frequency dimension, the pixels along the second frequency dimension, the pixels along the third frequency dimension, and / or the pixels along the fourth frequency dimension can correspond to multiple radio frequency frequency-shifted laser beams. The pixels along the first frequency dimension, the pixels along the second frequency dimension, the pixels along the third frequency dimension, and / or the pixels along the fourth frequency dimension can correspond to multiple spatial locations of cells.

[0017] In some embodiments, the fluorescence emissions of two of the plurality of fluorophores overlap spectrally. The fluorescence emissions of three of the plurality of fluorophores overlap spectrally. The fluorescence emissions of five of the plurality of fluorophores overlap spectrally. The fluorescence emissions of a first fluorophore and a second fluorophore of the plurality of fluorophores overlap spectrally, and the fluorescence emissions of a second fluorophore and a third fluorophore of the plurality of fluorophores overlap spectrally. The fluorescence emissions of the first fluorophore and the third fluorophore overlap spectrally.

[0018] The content disclosed herein includes embodiments of a cell sorter system. In some embodiments, the cell sorter system includes: a light source (e.g., a laser light source) configured to generate a light beam (e.g., a laser beam) having a plurality of different optical frequencies for inducing fluorescence emission of a plurality of fluorophores at different wavelengths, the plurality of fluorophores being associated with cells in a sample including a plurality of cells. The cell sorter system can include: a plurality of photodetectors configured to detect fluorescence emission of a plurality of fluorophores (e.g., each photodetector is configured to detect fluorescence emission of a fluorophore in the plurality of fluorophores). The cell sorter system can include: a non-transitory memory configured to store executable instructions; and a processor communicating with the non-transitory memory, the hardware processor being programmed by the executable instructions to: cause a laser beam to irradiate a cell to induce fluorescence emission of a plurality of fluorophores associated with the cell. The processor can be programmed by the executable instructions to: cause a plurality of photodetectors to detect fluorescence emission at different wavelengths to obtain fluorescence intensity data of a plurality of channels corresponding to a plurality of fluorophores in time and space. The processor can be programmed by executable instructions to generate a plurality of raw images including a plurality of pixels of complex values ​​in frequency space from the fluorescence intensity data of the plurality of channels. The processor can be programmed by executable instructions to generate a plurality of phase-adjusted images including a plurality of pixels of complex values ​​in frequency space from the plurality of raw images using one or more channel phase adjustments for each of the fluorescence intensity data of the plurality of channels. The processor can be programmed by executable instructions to generate a plurality of unmixed images including a plurality of pixels of complex values ​​in frequency space from the plurality of phase-adjusted images using an unmixing matrix on the complex values ​​of corresponding pixels in the plurality of phase-adjusted images. The processor can be programmed by executable instructions to generate a plurality of phase-corrected images including a plurality of pixels of complex values ​​or real values ​​in frequency space from the plurality of unmixed images based on the properties of each of the plurality of fluorophores. The processor can be programmed by executable instructions to determine a sorting decision for the cell based on the plurality of phase-corrected images.

[0019] The content disclosed herein includes embodiments of a cell sorter system. In some embodiments, the cell sorter system includes: a light source (e.g., a laser light source) configured to generate a light beam (e.g., a laser beam) having a plurality of different optical frequencies for inducing fluorescence emission of a plurality of fluorophores at different wavelengths, the plurality of fluorophores being associated with cells in a sample including a plurality of cells; a plurality of photodetectors configured to detect fluorescence emission of a plurality of fluorophores (e.g., each photodetector is configured to detect fluorescence emission of a fluorophore in the plurality of fluorophores); a non-transitory memory configured to store executable instructions; and a processor communicating with the non-transitory memory, the hardware processor being programmed by the executable instructions to: cause the laser beam to illuminate the cell to induce fluorescence emission of a plurality of fluorophores associated with the cell. The processor can be programmed by the executable instructions to: cause the plurality of photodetectors to detect fluorescence emission at different wavelengths to obtain fluorescence intensity data of a plurality of channels corresponding to a plurality of fluorophores in time space. The processor can be programmed by the executable instructions to: generate a plurality of raw images including complex values ​​in frequency space from the fluorescence intensity data of the plurality of channels. The processor can be programmed with executable instructions to generate a plurality of unmixed images including complex or real values ​​in frequency space from the plurality of original images using an unmixing matrix on corresponding complex values ​​in the plurality of original images. The processor can be programmed with executable instructions to determine a sorting decision for the cell based on the plurality of unmixed images.

[0020] The content disclosed herein includes embodiments of a cell sorter system. In some embodiments, the cell sorter system includes: a light source (e.g., a laser light source) configured to generate a light beam (e.g., a laser beam) having a plurality of different optical frequencies for inducing fluorescence emission of a plurality of fluorophores at different wavelengths, the plurality of fluorophores being associated with cells in a sample including a plurality of cells; a plurality of photodetectors configured to detect fluorescence emission of a plurality of fluorophores (e.g., each photodetector is configured to detect fluorescence emission of a fluorophore in the plurality of fluorophores); a non-transitory memory configured to store executable instructions; and a processor communicating with the non-transitory memory, the hardware processor being programmed by the executable instructions to: receive fluorescence intensity data of a plurality of channels of fluorescence emission at different wavelengths from a plurality of fluorophores associated with the cells, the fluorescence intensity data of the plurality of channels corresponding to a plurality of fluorophores in time space, wherein the fluorescence emission is detected by a plurality of photodetectors, and wherein the fluorescence emission is induced after the cell is irradiated by the laser light source. The processor can be programmed by the executable instructions to: generate a plurality of raw images including complex values ​​in frequency space from the fluorescence intensity data of the plurality of channels. The processor can be programmed with executable instructions to generate a plurality of unmixed images including complex or real values ​​in frequency space from the plurality of original images using an unmixing matrix on corresponding complex values ​​in the plurality of original images. The processor can be programmed with executable instructions to determine a sorting decision for the cell based on the plurality of unmixed images.

[0021] In some embodiments, to receive fluorescence intensity data of multiple channels, the processor is programmed by executable instructions to: cause a laser beam to illuminate a cell to induce fluorescence emission of multiple fluorophores associated with the cell; and cause multiple photodetectors to detect fluorescence emission at different wavelengths to obtain fluorescence intensity data of multiple channels corresponding to multiple fluorophores in time space. Multiple original images can include multiple pixels having the complex values ​​of the multiple original images. The processor can be programmed by executable instructions to: use one or more channel phase adjustments for each of the fluorescence intensity data of the multiple channels to generate multiple phase adjustment images including complex values ​​in frequency space from the multiple original images. To generate multiple unmixed images, the processor can be programmed by executable instructions to: use an unmixing matrix on the corresponding complex values ​​of the multiple phase adjustment images to generate multiple unmixed images from the multiple phase adjustment images. Multiple phase adjustment images can include multiple pixels having the complex values ​​of the multiple phase adjustment images. The processor can be programmed by executable instructions to: generate multiple phase correction images including complex values ​​in frequency space from the multiple unmixed images based on the properties of each of the multiple fluorophores. To determine a sorting decision, the processor can be programmed with executable instructions to: determine a sorting decision for the cell based on the plurality of phase-corrected images.The plurality of phase-corrected images can include a plurality of pixels of complex values ​​of the plurality of phase-corrected images.

[0022] In some embodiments, the hardware processor includes a field programmable gate array (FPGA). In some embodiments, to generate multiple raw images, the processor is programmed by executable instructions to generate multiple raw images in frequency space from the fluorescence intensity data of multiple channels using a time-frequency space transform. The time-frequency space transform can include a Fourier transform. The Fourier transform can include a discrete Fourier transform. The discrete Fourier transform can include a fast Fourier transform. The Fourier transform can include a sliding window Fourier transform. The size of the sliding window of the sliding window Fourier transform can be m1.

[0023] In some embodiments, one or more channel phase adjustment corrections or descriptions include a channel phase (e.g., a channel-specific phase) for a bright field phase, a calibration phase, and a phase offset for each of a plurality of channels. To generate a plurality of phase adjustment images, the processor can be programmed by executable instructions to, for each of a plurality of channels: multiply the complex values ​​of the raw images in a plurality of raw images corresponding to the channel to generate a phase adjustment image in a plurality of phase adjustment images corresponding to the channel. One or more channel phase adjustments can include or include only complex values. One or more channel phase adjustments can include real values. The processor can be programmed by executable instructions to receive and / or determine one or more channel phase adjustments.

[0024] In some embodiments, to generate a plurality of unmixed images, the processor is programmed by executable instructions to: generate a vector comprising complex values ​​of corresponding pixels of a plurality of phase adjustment images; multiply the vector by an unmixing matrix to generate an unmixing vector comprising unmixed complex values; and generate a plurality of unmixed images comprising corresponding pixels having unmixed complex values. The size of the vector can be 1×n1, the size of the unmixing matrix can be n1×n2, and / or the size of the unmixing vector can be 1×n2. To generate a plurality of unmixed images, the processor can be programmed by executable instructions to: for corresponding pixels in a plurality of phase adjustment images, use an unmixing matrix from a plurality of unmixing matrices on the complex values ​​of the corresponding pixels to generate a plurality of unmixed images from the plurality of phase adjustment images. The plurality of unmixing matrices can include m2 unmixing matrices. The unmixing matrix can include or include only real values ​​and / or complex values. The processor can be programmed by executable instructions to: receive and / or determine an unmixing matrix.

[0025] In some embodiments, to generate a plurality of phase-corrected images, the processor is programmed with executable instructions to generate a phase-corrected image in a plurality of phase-corrected images from an unmixed image in a plurality of unmixed images corresponding to fluorophores of a plurality of channels using a plurality of fluorophore phase corrections for fluorophores in a plurality of fluorophores. The plurality of fluorophore phase corrections for the fluorophores can be related to properties of the fluorophores. The properties of the fluorophores can include lifetimes of the fluorophores. The plurality of fluorophore phase corrections for the fluorophores can include complex values. The processor can be programmed with executable instructions to receive and / or determine a plurality of fluorophore phase corrections for the fluorophores.

[0026] In some embodiments, the processor is programmed with executable instructions to generate a plurality of visual representations of the plurality of phase-corrected images based on real parts of the complex values ​​of the plurality of phase-corrected images. The processor can be programmed with executable instructions to generate a plurality of visual representations of the plurality of phase-corrected images based on magnitudes of the complex values ​​of the plurality of phase-corrected images. To determine a sorting decision, the processor can be programmed with executable instructions to determine a sorting decision for the cell based on the plurality of visual representations of the plurality of phase-corrected images. The processor can be programmed with executable instructions to generate a combined visual representation from the plurality of visual representations.

[0027] In some embodiments, the processor is programmed by executable instructions to generate corrected fluorescence intensity data for multiple channels from multiple phase-corrected images. To generate corrected fluorescence intensity data for multiple channels, the processor can be programmed by executable instructions to generate corrected fluorescence intensity data for multiple channels from multiple phase-corrected images using a frequency-space spatial transform. The processor can be programmed by executable instructions to determine an estimate of a cell feature based on the corrected fluorescence intensity data for multiple channels, wherein determining a sorting decision includes determining a sorting decision for the cell based on the estimate of the cell feature. The cell feature can include cell size, a ratio of the size of the cell in two different dimensions, co-localization of fluorescence emission by two or more of a plurality of fluorophores associated with the cell, a ratio of the size of the cell's cytoplasm and nucleus, a punctate degree of fluorescence emission from the cell, a measure of spatial distribution of fluorescence emission, a measure of cell position or orientation, a measure of cell eccentricity, a measure of similarity of the cell to a reference cell, a measure of the degree to which the cell is in the focus of a laser beam, or a combination thereof.

[0028] In some embodiments, the laser beam includes a reference laser beam and a plurality of radio frequency frequency shifted laser beams. The number of the plurality of radio frequency frequency shifted laser beams can be m2. The reference laser beam can overlap with the plurality of radio frequency frequency shifted laser beams in space. The plurality of radio frequency frequency shifted laser beams do not overlap with each other in space. One or more of the reference laser beam and the plurality of radio frequency frequency shifted laser beams can induce fluorescence emission of fluorophores in the plurality of fluorophores. Irradiating the cell can include irradiating a plurality of spatial positions of the cell simultaneously with the plurality of radio frequency frequency shifted laser beams. The plurality of spatial positions of the cell do not overlap with each other.

[0029] In some embodiments, the number of multiple original images, the number of multiple phase adjustment images, the number of multiple unmixed images, and the number of multiple phase correction images are the same. The number of multiple original images, the number of multiple phase adjustment images, the number of multiple unmixed images, and / or the number of multiple phase correction images can be n1. Multiple original images can be associated with a first time dimension and a first frequency dimension. Multiple phase adjustment images can be associated with a second time dimension and a second frequency dimension. Multiple unmixed images can be associated with a third time dimension and a third frequency dimension. Multiple phase correction images can be associated with a fourth time dimension and a fourth frequency dimension. Two or more of the first time dimension, the second time dimension, the third time dimension, and the fourth time dimension can have the same size and / or the same number of pixels along the dimension. The first time dimension, the second time dimension, the third time dimension, and the fourth time dimension can have the same size and / or the same number of pixels. The first frequency dimension, the second frequency dimension, the third frequency dimension, and / or the fourth frequency dimension can have a size of m2 and / or m2 pixels. In some embodiments, m2 is equal to 1 / 2*m1. In some embodiments, m2 is less than 1 / 2*m1. One or more of the first frequency dimension, the second frequency dimension, the third frequency dimension, and the fourth frequency dimension can have the same size and / or the same number of pixels. The first frequency dimension, the second frequency dimension, the third frequency dimension, and the fourth frequency dimension can have the same size and / or the same number of pixels. The pixel size and / or the number of pixels along the first frequency dimension, the pixel size and / or the number of pixels along the second frequency dimension, the pixel size and / or the number of pixels along the third frequency dimension, and / or the pixel size and / or the number of pixels along the fourth frequency dimension can be the same as the number of multiple radio frequency frequency-shifted laser beams. The pixels along the first frequency dimension, the pixels along the second frequency dimension, the pixels along the third frequency dimension, and / or the pixels along the fourth frequency dimension can correspond to multiple radio frequency frequency-shifted laser beams. The pixels along the first frequency dimension, the pixels along the second frequency dimension, the pixels along the third frequency dimension, and / or the pixels along the fourth frequency dimension can correspond to multiple spatial locations of cells.

[0030] In some embodiments, the fluorescence emissions of two of the plurality of fluorophores overlap spectrally. The fluorescence emissions of three of the plurality of fluorophores overlap spectrally. The fluorescence emissions of five of the plurality of fluorophores overlap spectrally. The fluorescence emissions of a first fluorophore and a second fluorophore of the plurality of fluorophores overlap spectrally, and the fluorescence emissions of a second fluorophore and a third fluorophore of the plurality of fluorophores overlap spectrally. The fluorescence emissions of the first fluorophore and the third fluorophore overlap spectrally.

[0031] A further understanding of the various aspects of the present invention can be obtained by referring to the following detailed description in conjunction with the associated drawings which are briefly described below. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A system according to an embodiment of the invention is schematically depicted.

[0033] Figure 2 Cross-sectional beam profiles of multiple RF comb beams are schematically depicted.

[0034] Figure 3 Schematically depicts Figure 2 Superposition of the RF comb beam depicted in and the LO beam with a top-hat beam profile.

[0035] Figure 4 Schematically depicts Figure 3 The combined beam shown in irradiates the sample under analysis.

[0036] Figure 5 An example performance level for a hypothetical fluorophore is schematically depicted.

[0037] Figure 6 Schematically depicts the corresponding Figure 5 Absorption curve of the hypothetical fluorophore in .

[0038] Fig. 7A A detection system according to an embodiment of the present teachings is schematically depicted and includes an optical fiber for transmitting fluorescent radiation. Figure 7B Another detection system according to an embodiment of the present teachings is schematically depicted, wherein fluorescent radiation propagates through free space to a plurality of photodetectors. Figure 7C Brightfield and darkfield image generating arms used in some embodiments of the present teachings are schematically depicted. Fig.7D A detection system used in some embodiments of the present teachings is schematically depicted and includes a detection arm for generating a bright field image and a detection arm integrating the capability to detect excitation radiation scattered from a sample as well as fluorescence radiation emitted by the sample.

[0039] Figure 8 It is schematically depicted that a fluorescence signal generated by a photodetector in an embodiment of a system according to the invention can be amplified by an amplifier, and the amplified signal can be analyzed by an analysis module to construct a fluorescence image of a sample under analysis.

[0040] Fig. 9A and 9B A method for analyzing a fluorescence signal obtained by illuminating a sample with a combined beam consisting of a plurality of RF comb beams and a top-hat profile LO beam according to an embodiment of the present invention is depicted.

[0041] Fig.10Selected components of an exemplary hardware implementation of an analysis module according to an embodiment of the invention are schematically depicted.

[0042] Fig.11A and 11B Another method for analyzing a fluorescence signal obtained by illuminating a sample with a combined beam consisting of a plurality of RF comb beams and a top-hat profile LO beam according to an embodiment of the present invention is depicted.

[0043] Fig. 12A and 12B Yet another method for analyzing a fluorescence signal obtained by illuminating a sample with a combined beam consisting of a plurality of RF comb beams and a top-hat profile LO beam according to an embodiment of the present invention is depicted.

[0044] Fig.13A Schematically depicted is the illumination of a sample by a beam of top-hat profile at a single excitation frequency. Fig. 13B is a schematic diagram of a system according to an embodiment of the present teachings that allows for fluorescence lifetime measurement and fluorescence lifetime imaging.

[0045] Fig.14A is a flow chart depicting determining an estimate of at least one characteristic of particles flowing through a flow cytometer system. Fig. 14B A system for determining an estimate of at least one characteristic in particles flowing through a flow cytometer system according to an embodiment is schematically depicted. Fig. 14C is a flow chart depicting a method for gating particles in a flow cytometer system based on the values ​​of one or more particle characteristics, according to an embodiment. Fig.14D is a flow chart depicting an embodiment for making sorting decisions based on co-localization of fluorescent radiation emitted from particles (eg, cells) in two or more different frequency channels.

[0046] Fig.15 Schematically depicts the corresponding Fig.16 A hypothetical fluorescence time-frequency waveform of the two channels and their product used in the method shown in the flow chart.

[0047] Fig.16 is a flow chart depicting cell sorting based on cell size according to an embodiment of the present invention.

[0048] Fig.17A A hypothetical cell is schematically depicted being illuminated by a hypothetical beam comprising a plurality of radio frequency modulated sub-beams. Fig. 17B Schematically depicts the Fig.17A A hypothetical fluorescence waveform obtained in an illuminated cell and a waveform obtained by squaring the fluorescence waveform are shown in FIG.

[0049] Fig.18 is a flow chart depicting a method for sorting cells based on cell aspect ratio.

[0050] Fig.19A is a flow chart depicting a method for estimating the size ratio of the nucleus to the cytoplasm. Fig.19B is a flow chart depicting a method for sorting cells based on the estimated size ratio of nucleus to cytoplasm.

[0051] Fig. 20A is a flow chart depicting a method for estimating cell size from fluorescent radiation emitted from cells. Fig. 20B is a flow chart depicting a method for sorting cells based on estimating the cell size of fluorescent radiation emitted from the cells.

[0052] Fig.21 Schematically shows the Fig.24 The method described in the flow chart employs an optical beam which is modulated at a modulation frequency and which is used to induce fluorescence emission from cells.

[0053] Fig. 22 A sorting system incorporating the present teachings for sorting cells is schematically depicted.

[0054] Fig.23 Schematically depicts the Fig. 22 An exemplary implementation of an analysis / control module employed in a system.

[0055] Fig.24 is a flow chart depicting an exemplary method for determining a feature or particle and using the feature to make a sorting decision, according to an embodiment.

[0056] Fig.25 is a non-limiting exemplary schematic diagram of spectral unmixing.

[0057] Figures 26A-26B is a composite image showing a non-limiting exemplary result of spectral unmixing.

[0058] Fig. 27 is a flow chart illustrating an exemplary method of spectral unmixing.

[0059] Fig.28A It is a schematic diagram of using a square unmixing matrix to implement spectral unmixing. Figure 28B1-28B2 It is a schematic diagram of using a non-square unmixing matrix to implement spectral unmixing and determine the non-square unmixing matrix.

[0060] Fig.29 is a schematic diagram of fluorophore phase correction.

[0061] Fig.30is a block diagram of an illustrative computing system configured to implement an embodiment of a method of spectral unmixing. DETAILED DESCRIPTION

[0062] In the following detailed description, reference is made to the accompanying drawings which form a part of this description. In the accompanying drawings, similar reference numerals generally identify similar parts unless the context indicates otherwise. The illustrative embodiments described in this detailed description, the drawings, and the claims are not meant to be limiting. Other embodiments can be utilized and other changes can be made without departing from the spirit or scope of the subject matter presented herein. It is readily understood that aspects of the present disclosure as generally described herein and shown in the accompanying drawings can be arranged, replaced, combined, separated, and designed in a variety of different configurations, all of which are expressly contemplated and are hereby made a part of the present disclosure.

[0063] The present teachings generally relate to methods and systems for determining one or more features of particles (such as cells) in a flow cytometer and using those features to sort particles in some embodiments. In the embodiments discussed below, the method is implemented using a computer processor. Unless otherwise indicated, the various terms used to describe the present teachings below have their common meanings in the art. For example, the term "fluorophore" as used herein is consistent with its common meaning in the art, i.e., refers to a fluorescent chemical compound that can emit radiation in response to irradiation with excitation radiation.

[0064] The terms "cytometry" and "flow cytometry" are also used in accordance with their customary meanings in the art. In particular, the term "cytometry" can refer to a technique for identifying and / or sorting or otherwise analyzing cells. The term "flow cytometry" can refer to a cell measurement technique in which cells present in a fluid stream can be identified and / or sorted or otherwise analyzed, for example, by labeling with a fluorescent marker and detecting the fluorescent marker via radiation excitation. The terms "approximately" and "substantially" as used herein mean a maximum variation of 10% or 5% relative to the characteristic containing the numerical value.

[0065] The teachings of the present invention for determining the characteristics of particles (such as cells) and for sorting particles can be implemented in a variety of different ways. Fluorescence and / or scattering data for making sorting decisions can be obtained by using various systems. In some embodiments, particles are illuminated by an optical beam having multiple radio frequency frequency shifted sub-beams and fluorescence from the particles is collected and analyzed according to the present teachings to make sorting decisions. Some examples of systems for eliciting fluorescence data from particles are described below, in which the present teachings can be incorporated, followed by a detailed description of methods and systems for sorting particles according to the present teachings.

[0066] As an example, Figure 1 A system 10 for performing cytometry is schematically depicted, in which the present teachings for sorting particles can be incorporated. The system 10 can be operated in three modes of operation. As discussed in more detail below, in one mode of operation, a sample under investigation can be simultaneously irradiated using multiple excitation frequencies, each of which can be obtained, for example, by shifting the center frequency of a laser beam. More specifically, multiple sample positions can be simultaneously irradiated by a laser beam generated by mixing a reference laser beam (also referred to herein as a local oscillator beam) with multiple radio frequency frequency-shifted laser beams so that each sample position is irradiated by one of the reference beam and the radio frequency-shifted beam to excite the fluorophore of interest (if present) at that position. In some embodiments, the reference beam itself can be generated via a radio frequency frequency shift of the laser beam. Therefore, each spatial position of the sample can be "tagged" with a different beat frequency, which corresponds to the difference between the frequency of the reference beam and the frequency of one of the radio frequency frequency-shifted beams. In other words, the fluorescent radiation emitted by the fluorophore will spatially encode the beat frequency. The fluorescent emission can be detected, and its frequency components can be analyzed to construct a fluorescent image of the sample. Detecting fluorescence radiation emitted by fluorophores after excitation with a reference beam and a radio frequency frequency shifted beam may be referred to as fluorescence imaging with radio frequency multiplexed excitation (FIRE).

[0067] In another mode of operation, the sample can be continuously irradiated by laser beams of multiple excitation frequencies within a time interval. In some such embodiments, the excitation frequency can be obtained by applying a time-varying drive signal to an acousto-optic deflector (AOD) that receives the laser beam. In many embodiments, the frequency of the laser beam is in the range of hundreds of terahertz (THz), for example, about 300 THz to about 1000 THz. The drive signal applied to the AOD is usually in the radio frequency range, for example, about 10 MHz to about 250 MHz. The laser beam passing through the AOD generates multiple diffraction beams, each of which corresponds to a different diffraction order. Although the zeroth order diffraction beam does not show a frequency shift relative to the frequency of the input laser beam, the higher order diffraction beams show a frequency shift corresponding to the frequency of the drive signal or its multiple relative to the frequency of the input laser beam. In some embodiments, if a first order diffraction beam is present in a sample in an analysis state, it is used as an excitation beam for exciting the fluorophore of interest, and the frequency of the first order diffraction beam corresponds to the frequency of the input laser beam shifted by the drive signal. As the drive signal varies over time, the frequency and angular shift of the first order diffracted beam also varies, thereby allowing the sample to be illuminated at different locations with different excitation frequencies. Fluorescence emission (if any) from each illumination location can be collected and analyzed to construct a fluorescent image of the sample.

[0068] In yet another mode of operation, the system 10 can be operated to simultaneously illuminate multiple locations of a sample with a single excitation frequency, which can be generated, for example, by shifting the center frequency of the laser beam by one radio frequency. For example, a horizontal range of a sample can be illuminated by a laser beam with a single excitation frequency. The detected fluorescent radiation can be used to analyze the fluorescent content of the sample (e.g., cells / particles).

[0069] Thus, one advantage of system 10 (other advantages will be discussed below) is that the system provides significant flexibility in acquiring fluorescence emission data in different modes without requiring the use of different instrumentation or requiring any mechanical modifications to the system when switching between different operating modes.

[0070] In certain embodiments, system includes one or more light sources. In some cases, light source is a narrowband light source, which includes but is not limited to narrow wavelength LED, laser or is coupled to one or more optical bandpass filters, diffraction gratings, monochromators or combinations to produce a broadband light source of any combination of narrowband illumination light. In some cases, light source is a single wavelength laser, such as a single wavelength diode laser (e.g., 488nm laser). In certain embodiments, this subject system includes a single light source (e.g., laser). In other embodiments, this subject system includes two or more different light sources, such as 3 or more different light sources, such as 4 or more different light sources, and includes 5 or more different light sources. For example, system may include a first light source (e.g., laser) outputting a first wavelength and a second light source outputting a second wavelength. In other embodiments, system includes a first light source outputting a first wavelength, a second light source outputting a second wavelength and a third light source outputting a third wavelength.

[0071] The wavelength of each light source can be 300nm to 1000nm, such as 350nm to 950nm, such as 400nm to 900nm, and includes 450nm to 850nm. In certain embodiments, the wavelength of the light source corresponds to the absorption maximum (as described below) of one or more fluorophores. For example, the wavelength of the light output by the light source can be 280-310nm, 305-325nm, 320-350nm, 340-375nm, 370-425nm, 400-450nm, 440-500nm, 475-550nm, 525-625nm, 625-675nm and 650-750nm One or more. In certain embodiments, the wavelength of the light output by each light source is selected from 348nm, 355nm, 405nm, 407nm, 445nm, 488nm, 640nm and 652nm.

[0072] The system 10 includes a laser radiation source 12 that generates a laser beam 14. As an example, the frequency of the laser beam can be about 1000 THz to about 300 THz, which corresponds to a vacuum wavelength of about 300 nm to about 1000 nm. The beam diameter of the laser beam (e.g., the beam waist when a Gaussian laser beam is employed) can be, for example, about 0.1 mm to about 10 mm. Without loss of generality, in this embodiment, the laser 12 emits radiation of a wavelength of 488 nm with a beam diameter of about 1 mm.

[0073] The frequency of the laser beam can be selected based on the specific application expected by the system. Specifically, as discussed in more detail below, the laser frequency can be suitable for exciting electronic transitions of the fluorophore of interest, such as via absorption of radiation, so that the fluorophore emits lower frequency fluorescence radiation. A variety of laser sources can be used. Some examples of such laser sources include, but are not limited to, the Sapphire 488-SF, Genesis MX-488-1000-STM (Coherent), OBIS 405-LX (Coherent), Stadus 405-250 sold by Vortran Laser Technology, Sacramento, California, and LOC-660-110 of Newport Corporation, Irvine, California, without affecting generality, in this embodiment, it is assumed that the laser beam has a Gaussian intensity distribution in a plane perpendicular to its propagation direction.

[0074] Mirror 16 receives laser radiation beam 14, and guides the laser beam to acousto-optic deflector (AOD) 18 via reflection. In this embodiment, AOD 18 is mounted on an adjustable column holding mount (A), which allows AOD to rotate around an axis perpendicular to the propagation direction of beam 14. A direct digital synthesizer (DDS) 20 operating under the control of controller 21 can apply one or more drive signals to AOD 18. As an example, in some embodiments, these drive signals can span a frequency range of about 50MHz to about 250MHz. For example, the drive signal applied to AOD can be about 55MHz to about 255MHz, such as about 60MHz to about 200MHz, such as about 65MHz to about 175MHz, such as about 70MHz to about 150MHz, and includes about 75MHz to about 125MHz. In some embodiments, the drive signals can be separated from each other by a frequency of about 0.1MHz to about 4MHz. For example, the drive signals can be separated from each other by frequencies of about 0.2 MHz to about 3.9 MHz, such as about 0.3 MHz to about 3.8 MHz, such as about 0.4 MHz to about 3.7 MHz, such as about 0.5 MHz to about 3.6 MHz, and including about 1 MHz to about 3.5 MHz. In this embodiment, the electronic power amplifier 17 amplifies the RF signal generated by the DDS 20 for application to the AOD 18.

[0075] In an operating mode in which a plurality of excitation frequencies are utilized to simultaneously illuminate a sample, an RF comb generator 20 applies a plurality of RF drive signals to the AOD 18 simultaneously. As an example, the number of RF drive signals applied simultaneously can be about 20 to about 200. The interaction of the laser beam with the drive signal results in the generation of a plurality of angularly separated laser beams, each laser beam having a frequency shift corresponding to one of the drive signals relative to the frequency of the laser beam generated by the laser 12. Without being limited by any particular theory, in the AOD, a piezoelectric transducer can generate radio frequency phonons in a crystal (e.g., a quartz crystal), and the scattering of optical phonons of the laser beam by such radio frequency phonons can result in the generation of a frequency-shifted laser beam. One of these frequency-shifted beams 22 is referred to herein as a "local oscillator" (LO) beam, and the remaining frequency-shifted beams 24 are referred to herein as "RF comb beams". The angular spacing of the frequency-shifted beams can be, for example, about 1 milliradian to about 100 milliradians. For example, the angular spacing of the frequency shifted beams may be 2 mrad to about 95 mrad, such as 3 mrad to about 90 mrad, such as 4 mrad to about 85 mrad, such as 5 mrad to about 80 mrad, and including 10 mrad to about 75 mrad.

[0076] The LO and RF comb beams pass through a lens 26, which in this embodiment is a convex lens with a focal length of about 50 mm. After passing through the lens 26, the LO laser beam is intercepted by a mirror 28, which redirects the LO beam to a different direction (in this embodiment, a direction substantially orthogonal to the original propagation direction of the LO beam). The mirror 28 is positioned relative to the RF comb beam so that these beams miss the mirror 28 and propagate to the lens 30 (which has a focal length of 200 mm in this embodiment). In this way, the LO beam and the RF comb beam are guided along different propagation directions. The use of a reflector 28 in the manner disclosed above allows the use of a single AOD to generate the LO beam and the RF comb beam, and to combine them in the manner discussed below to generate an excitation beam for irradiating the sample. Using a single AOD rather than multiple AODs (e.g., two AODs, one for generating the LO beam and the other for generating the RF comb beam) simplifies the design of the system and also allows the system to be used efficiently in a variety of different operating modes, as discussed in more detail below.

[0077] In some embodiments, the beam profile of the LO beam is modified before being recombined with the RF comb beam. For example, the beam profile of the LO beam can be adjusted (increased or decreased) in spatial dimension, beam shape, intensity, spatial distribution of the beam, or any combination thereof. In some embodiments, the spatial dimension of the beam profile of the LO beam is modified. For example, the beam profile can be adjusted to stretch the beam profile in one or more dimensions (such as along an axis orthogonal to the longitudinal axis of the stream). In one example according to these embodiments, the spatial dimension of the beam profile (e.g., in one or more dimensions) can be increased by 1% or more, such as 2% or more, such as 3% or more, such as 5% or more, such as 10% or more, such as 25% or more, such as 50% or more, such as 75% or more, such as 90% or more, such as 1.5 times or more, such as 2 times or more, such as 3 times or more, and including 5 times or more. In another example according to these embodiments, the spatial dimensions of the beam profile (e.g., in one or more dimensions) can be reduced by 1% or more, such as 2% or more, such as 3% or more, such as 5% or more, such as 10% or more, such as 25% or more, such as 50% or more, such as 75% or more, such as 90% or more, such as 1.5 times or more, such as 2 times or more, such as 3 times or more, and including 5 times or more.

[0078] In other embodiments, the beam shape of the LO beam is modified. For example, the beam shape can be modified to elongate the beam profile in one or more dimensions. In some cases, the beam shape of the LO beam is elongated in a plane perpendicular to the propagation direction of the LO beam. In some embodiments, the shape of the LO beam profile is changed from a circular beam profile to an elliptical beam profile, which is elongated on an axis orthogonal to the longitudinal axis of the flow. In other embodiments, the shape of the LO beam profile is changed from a circular beam profile to a rectangular beam profile, which has a long dimension on an axis orthogonal to the longitudinal axis of the flow. In other embodiments, the intensity of the LO beam is modified. For example, the intensity of the LO beam can be increased by 1% or more, such as 2% or more, such as 3% or more, such as 5% or more, such as 10% or more, such as 25% or more, such as 50% or more, such as 75% or more, such as 90% or more, such as 1.5 times or more, such as 2 times or more, such as 3 times or more, and including 5 times or more. In some embodiments, the intensity of the LO beam is modified to match the intensity of the RF comb beam. For example, the intensity of the LO beam may differ from the intensity of the RF comb beam by 10% or less, such as 9% or less, such as 8% or less, such as 7% or less, such as 6% or less, such as 5% or less, such as 4% or less, such as 3% or less, such as 2% or less, such as 1% or less, such as 0.01% or less, and including the intensity of the LO beam differing from the intensity of the RF comb beam by 0.001% or less. In some cases, the intensities of the LO beam and the RF comb beam are the same.

[0079] In other embodiments, the spatial distribution of the beam profile may also be modified. For example, the LO beam may be modified such that the intensity of the LO beam is no longer Gaussian in one or more dimensions. For example, the LO beam may be modified to have a Gaussian distribution along a first axis parallel to the longitudinal axis of the flow, and a non-Gaussian distribution along a second axis orthogonal to the longitudinal axis of the flow.

[0080] Any beam shaping protocol may be employed to modify the beam profile of the LO beam, including but not limited to refractive and diffractive beam shaping protocols.In some embodiments, the LO beam is modified by a top-hat beam shaper.

[0081] In this embodiment, the LO beam propagates to another convex lens 32 (which has a focal length of approximately 200 mm in this embodiment). The combination of lens 26 and lens 32 amplifies and collimates the LO beam so as to properly fill the back aperture of the top-hat beam shaper 34. More specifically, the LO beam 22 passes through lens 32 and is reflected by mirrors 33 and 35 to the top-hat beam shaper 34.

[0082] The top-hat beam shaper 34 shapes the phase front of the Gaussian LO beam so that a top-hat intensity profile can be formed. More specifically, the LO laser beam 22' leaving the top-hat beam shaper is reflected by the beam splitter 44 and focused by the lens 46 (which has a focal length of 100 mm in this embodiment) onto the intermediate image plane 48. The laser beam on the intermediate image plane 48 has a top-hat intensity profile along the horizontal direction in a plane perpendicular to the propagation direction of the beam. Similar to the AOD 18, in this embodiment, the beam splitter 44 is mounted on an adjustable column holding mount (B). In this embodiment, the top-hat beam shaper generates a top-hat beam profile in which the polarization of the radiation is substantially uniform along the top-hat direction of the beam (in this embodiment, along the horizontal direction).

[0083] Various top-hat beam shapers can be employed. As examples, diffractive optical elements or refractive optical elements with aspheric surfaces can be used to produce beams with appropriate spatial phase fronts that, after being focused by a lens, will produce a top-hat profile pattern at the focal plane of the lens. There are a variety of form factors for such top-hat beam shapers, and in various embodiments of the present teachings, various implementations of the method can be used to create the appropriate LO beam shape at the sample. For example, U.S. Patent 6,295,168, entitled "Refractive Optical System for Converting a Laser Beam into a Collimated Flat Top Beam," and U.S. Patent 7,400,457, entitled "Rectangular Flat Top Beam Shaper," both of which are incorporated herein by reference in their entirety, disclose beam shaping systems that can be used as flat top beam shapers in systems according to some embodiments of the present teachings. Examples of commercially available top-hat beam shapers that can be employed include, for example, the DTH-1D-0.46deg-4mm sold by Osela Corporation of Lachine, Canada.

[0084] As discussed in more detail below, stretching the LO beam along the horizontal direction using a beam shaper provides a number of advantages. For example, it can ensure that the combination of the LO beam and the RF comb beam illuminates multiple sample locations with substantially similar illumination intensities so that the intensities of the LO and RF comb beams are matched throughout the sample locations, thereby creating an intensity amplitude modulation of the fluorescence signal with a high modulation depth. In the absence of such intensity matching, the imaging system may have a small field of view and may not utilize all frequencies (pixels) driving the AOD. Since the modulation depth of the fluorescence signal plays an important role in the system's ability to reconstruct a fluorescent image of the sample, a uniform high modulation depth of the beat frequency excited at all pixels is particularly advantageous for the operation of the system. In addition, the amplitude of the electronic signal applied to the AOD to generate the RF comb beam can be adjusted by controlling the output of the direct digital synthesizer (e.g., by employing the controller 21) so as to equalize the RF comb beams so that their intensities are equal to the intensity of the LO beam at all spatial locations where the RF comb beam and the LO beam overlap. This feature provides the advantage that it ensures a high modulation depth of the intensity amplitude modulation of the fluorescence radiation.

[0085] Refer again Figure 1 , RF comb beam 24 is imaged onto intermediate image plane 38 via a combination of lenses 26 and 30. More specifically, RF comb beam 24 passes through lens 26 and misses mirror 28 to reach lens 30, which directs RF comb beam 24 to intermediate image plane 38 via mirrors 40 and 42.

[0086] Figure 2 The distribution of an exemplary number of RF comb beams in the intermediate image plane 38 is schematically depicted (without loss of generality, for illustration purposes, the number of RF comb beams is chosen to be 6 (labeled RF1, RF2, RF3, RF4, RF5, RF6), but other numbers can also be used). Figure 2 As shown in FIG. 1 , in the intermediate image plane 38, the RF comb beams 24 are spatially separated from each other along the horizontal direction. In other embodiments, two or more of the RF comb beams 24 may partially overlap. Thus, the combination of lenses 26 and 30 converts the angularly separated RF comb beams into a set of spatially separated beams spanning the horizontal range.

[0087] Refer again Figure 1 As discussed above, the beam splitter 44 receives the laser beam 22' exiting the top-hat beam shaper 34 and reflects the beam to the lens 46, which focuses the beam on the intermediate image plane 48, where the LO beam exhibits a top-hat beam profile. The beam splitter also receives the RF comb beam 24 from the intermediate image plane 38 and allows the RF comb beam to pass therethrough. The lens 46 focuses the RF comb beam 24 on the intermediate image plane 48 to combine with the LO beam having a top-hat beam profile to generate a combined beam 49.

[0088] As an example, Figure 3 An exemplary profile of the combined beam 49 in a plane perpendicular to its propagation axis is schematically depicted. The intensity profile of the combined beam is generated by superimposing the intensity profile of the top hat LO beam (schematically shown by the squares) with the intensity profile of the RF comb beam 24 (schematically shown by one of the circles, respectively). As discussed in more detail below, this superposition of the LO beam and the RF comb beam provides a plurality of beat frequencies along the horizontal extent, each beat frequency corresponding to a spatial position along the horizontal extent. When illuminating the horizontal extent of the sample, the fluorescent radiation emitted from the sample location encodes the beat frequency associated with the radiation illuminating the location via amplitude modulation.

[0089] Refer again Figure 1 , convex lens 50 (a 200 mm lens in this embodiment) and objective lens 52 (mounted on an adjustable column holding mount C in this embodiment) form a telescope for relaying the image at intermediate plane 48 onto the sample flowing through flow cell 54. In this embodiment, mirror 56 reflects combined beam 49 to lens 50, and dichroic mirror 58 reflects the combined beam toward objective lens 52 after it passes through lens 50.

[0090] like Figure 4 , the combined beam 49 simultaneously illuminates multiple spatial locations 60 of a sample 62 flowing through a flow cell 54. Thus, each location 60 is illuminated by an overlap of one of the RF comb beams with a portion of the top-hat LO laser beam. At these spatial locations, the radiation will excite fluorophores of interest in the sample, if present. More specifically, in this embodiment, the LO beam and the RF comb beam simultaneously excite fluorophores at multiple sample locations 60, for example, by causing their electrons to transition to excited electronic states.

[0091] In some embodiments, the sample can include a flowing fluid in which a plurality of cells are entrained. In some cases, cells can be labeled using one or more fluorescent markers (fluorophores). Some examples of fluorescent markers include, but are not limited to, fluorescent proteins (e.g., GFP, YFP, RFP), antibodies labeled with fluorophores (e.g., fluorescein isothiocyanate) (FITC), phycoerythrin (PE), allophycocyanin (APC), nucleic acid stains (e.g., 4', 6-diamidino-2-phenylindole (DAPI), SYTO16, propidium iodide (PI), cell membrane stains (e.g., FMI-43) and cell function dyes (e.g., Fluo-4, Indo-1). In other cases, endogenous fluorophores present in cells can be used to induce fluorescent radiation from cells. As discussed in more detail below, such exogenous or endogenous fluorophores respond to irradiation radiation to undergo electronic excitation and emit fluorescent radiation (whose frequency is generally lower than the excitation frequency), which is collected and analyzed.

[0092] By way of illustration and without being limited to any particular theory, Figure 5 Hypothetical energy levels are shown, corresponding to the ground electronic state A and two electronically excited electronic states B and C of the fluorophore. The fluorophore can be excited from its ground electronic state (A) to an excited electronic state (B) via absorption of radiant energy. The fluorophore can then relax to a lower excited state B, for example, via a radiationless transition mediated by a vibrational mode of the fluorophore. The fluorophore can also relax from the lower electronic state C to the ground state via an optical transition, thereby emitting fluorescent radiation having a frequency less than the excitation frequency. It should be understood that this hypothetical example is provided for illustrative purposes only and does not indicate the only mechanism by which fluorescent radiation can be emitted.

[0093] In many cases, fluorophores are capable of absorbing electromagnetic radiation over a range of frequencies from ground state excitation to electronic excited states. As an illustration, Figure 6 Shown with Figure 5 Absorption curves for hypothetical fluorophores discussed in conjunction. In one implementation of an embodiment according to the present teachings, the LO frequency can be selected to be the same as the frequency corresponding to the peak absorption of the fluorophore of interest. The RF frequency-shifted beams can have frequencies separated from the peak absorption by their respective beat frequencies. Typically, these frequency separations are small compared to the absorption bandwidth of the fluorophores, thereby avoiding any reduction in the excitation frequency. As an example and for illustration only, dashed lines A and B schematically depict the frequencies of the LO beam and one of the RF comb beams (the drawings are not drawn to scale for ease of description). Both the LO laser beam and one of the depicted RF comb beams simultaneously illuminate a spatial position of the sample, which causes the fluorescence radiation to exhibit amplitude modulation at the beat frequency corresponding to the difference between the LO comb beam frequency and the RF comb beam frequency.

[0094] Again by way of illustration and without being limited to any particular theory, the electric field applied to the fluorophore via simultaneous illumination with one of the LO beam and the RF comb beam can be mathematically defined as:

[0095]

[0096] Where E com represents the electric field of the combined beam,

[0097] E RF represents the magnitude of the electric field associated with one of the RF comb beams,

[0098] E LO represents the magnitude of the electric field associated with the LO beam,

[0099] represents the frequency of the laser beam generated by the laser 12,

[0100] represents the frequency shift associated with the RF comb beam, and

[0101] cow represents the frequency shift associated with the LO beam.

[0102] The intensity of the fluorescent radiation emitted in response to the superposition of the electric fields of the LO beam and the RF comb beam will be at a value corresponding to (ω RF -ω LO ). Thus, the fluorescence radiation emitted from each spatial position in the sample illuminated by the superposition of the LO beam and one of the RF comb beams exhibits modulation at a beat frequency corresponding to the difference between the radio frequency offset associated with the LO beam and the radio frequency offset associated with the RF comb beam illuminating the spatial position.

[0103] Since the fluorescence emission process requires a finite time (typically 1-10 nanoseconds for common organic fluorophores), if the excitation beat frequency is too high, the emitted fluorescence will not exhibit a high modulation depth. Therefore, in many embodiments, the excitation beat frequency is selected to be much smaller than 1 / τ. f , where τ f is the characteristic fluorescence lifetime of the fluorophore. In some cases, the excitation beat frequency can be greater than 1 / τ f 1% or more, such as 2% or more, such as 3% or more, such as 5% or more, such as 10% or more, such as 25% or more, such as 50% or more, such as 75% or more, such as 90% or more, such as 1.5 times or more, such as 2 times or more, such as 3 times or more, and including 5 times or more. For example, the excitation beat frequency can be smaller than 1 / τ f0.01 MHz or more, such as 0.05 MHz or more, such as 0.1 MHz or more, such as 0.5 MHz or more, such as 1 MHz or more, such as 5 MHz or more, such as 10 MHz or more, such as 25 MHz or more, such as 50 MHz or more, such as 100 MHz or more, such as 250 MHz or more, such as 500 MHz or more, and including 750 MHz or more. In some embodiments, the photodetector is configured to detect light (e.g., luminescence such as fluorescence) from the irradiated sample. In some embodiments, the photodetector may include one or more detectors, such as 2 or more detectors, such as 3 or more detectors, such as 4 or more detectors, such as 5 or more detectors, such as 6 or more detectors, such as 7 or more detectors, and including 8 or more detectors. Any light detection protocol may be employed, including but not limited to active pixel sensors (APS), quadrant photodiodes, image sensors, charge coupled devices (CCD), intensified charge coupled devices (ICCD), light emitting diodes, photon counters, bolometers, pyroelectric detectors, photoresistors, photovoltaic cells, photodiodes, photomultiplier tubes, phototransistors, quantum dot photoconductors or photodiodes, and combinations thereof, and other photodetectors. In some embodiments, the photodetector of interest is configured to detect light from 350 nm to 1200 nm, such as 450 nm to 1150 nm, such as 500 nm to 1100 nm, such as 550 nm to 1050 nm, such as 500 nm to 1000 nm, and including 400 nm to 800 nm. In some embodiments, the photodetector is configured to detect light at a maximum emission of the luminescence, such as 395nm, 421nm, 445nm, 448nm, 452nm, 478nm, 480nm, 485nm, 491nm, 496nm, 500nm, 510nm, 515nm, 519nm, 520nm, 563nm, 570nm, 578nm, 602nm, 612nm, 650nm, 661nm, 667nm, 668nm, 678nm, 695nm, 702nm, 711nm, 719nm, 737nm, 785nm, 786nm or 805nm.

[0104] In some embodiments, the fluorescent radiation emitted by the sample can be collected in a variety of different ways, such as along an optical path perpendicular to the direction of propagation of the excitation beam. In other embodiments, the fluorescent radiation is detected in an epitaxial direction. Detecting fluorescent radiation emitted by one or more fluorophores after being excited by the combined beam 49 can be referred to as fluorescence imaging using radio frequency multiplexed excitation (FIRE).

[0105] Refer again Figure 1 , in this embodiment, fluorescent radiation emitted by one or more fluorophores present in the illuminated sample passes through the objective lens 52 and is transmitted through the dichroic mirror 58 to the photodetector 64. More specifically, in this embodiment, the lens 65 focuses the fluorescent radiation transmitted through the dichroic mirror 58 onto the slit aperture 66. The fluorescent radiation transmitted through the slit passes through the fluorescence emission filter 68 to the photodetector 64. The slit aperture 66 (or an optical filter in other embodiments discussed below) disposed in front of the photodetector substantially allows fluorescent radiation emitted from a specific plane of the sample to pass while rejecting fluorescent emission out of the plane. In addition, the fluorescence emission filter 68 (e.g., a bandpass filter) allows the fluorescent radiation to pass to the photodetector 64 while substantially blocking radiation of other frequencies from passing.

[0106] The photodetector 64 has sufficient RF bandwidth to detect and transmit signals from the entire beat frequency range. Some examples of suitable photodetectors include, but are not limited to, photomultiplier tubes, avalanche photodiodes, PIN photodiodes, and hybrid photodetectors, among others. As an example, in some embodiments, photomultiplier tubes sold by Hamamatsu Corporation (e.g., R3896, R10699, H11462) can be employed. In response to detection of received fluorescent radiation, the photodetector generates a signal (e.g., an analog signal in this embodiment).

[0107] As another example and with reference to Fig. 7A , the fluorescence radiation emitted by the sample in response to simultaneous illumination with the LO beam and the spatially separated RF comb beam passes through the objective lens 52 and the dichroic mirror 58 to be coupled via the lens 100 onto a multimode optical fiber 102 extending from a proximal end 102a to a distal end 102b. More specifically, the proximal end 102a of the optical fiber 102 is positioned near the focal plane of the lens 100 to receive the fluorescence radiation. An outcoupling lens 104 coupled to the distal end 102b of the optical fiber collimates the radiation exiting the optical fiber.

[0108] In many cases, the excitation radiation irradiating the sample excites multiple fluorophores (e.g., organic fluorophores) that can have sufficiently wide radiation absorption spectra so that the excitation frequency falls within the absorption spectra of multiple fluorophores in the sample. Each fluorophore will then emit fluorescence radiation of a different frequency. Without loss of generality and for illustrative purposes, in this embodiment, the detection system includes four photomultiplier tubes 106, 108, 110, and 112, each of which receives a portion of the collimated radiation corresponding to the fluorescence radiation emitted by one of the four fluorophores excited by the excitation radiation in the irradiated sample. More specifically, the dichroic mirror 114 reflects the fluorescence radiation emitted by one of the fluorophores at a first frequency to the photomultiplier tube 106, while allowing the fluorescence radiation of other frequencies to pass. Another dichroic mirror 116 reflects the fluorescence radiation emitted by different fluorophores at different second frequencies to the photomultiplier tube 108, while allowing the remaining radiation containing the fluorescence radiation emitted by another fluorophore at a third frequency to reach the third dichroic mirror 118, which reflects the fluorescence radiation to the photomultiplier tube 110. The dichroic mirror 118 allows the remaining radiation, including fluorescent radiation emitted by the fourth fluorophore at the fourth radiation frequency, to pass to the photomultiplier tube 112 .

[0109] A plurality of bandpass filters 120, 122, 124, and 126, each centered on one of the four fluorescence frequencies, are placed in front of the photomultiplier tubes 106, 108, 110, and 112, respectively. The signals detected by each of the photomultiplier tubes are analyzed in a manner discussed below to generate a fluorescence image at the respective fluorescence frequency. In some embodiments, a single photodetector (e.g., a single photomultiplier tube) can be used, rather than using multiple photodetectors, to detect fluorescence radiation (e.g., a fluorescence frequency corresponding to emission from a single fluorophore).

[0110] In some embodiments, as the sample flows through the flow cell, different horizontal rows of the sample are illuminated and the fluorescent radiation associated with each horizontal row is detected by one or more photodetectors, such as photomultipliers 106 , 108 , 110 , and 112 .

[0111] Figure 7B Schematically depicts the above combination Fig. 7A The detection system discussed above is similar to the detection system except that instead of using optical fibers, the fluorescence radiation including the fluorescence emission from multiple fluorophores passing through the dichroic mirror 58 propagates in free space to the photomultiplier tubes 106, 108, and 112. More specifically, the lens 100 focuses the fluorescence radiation onto an aperture 126 disposed between the lenses 100 and 104, wherein the aperture is capable of rejecting out-of-focus radiation. The lens 104 collimates the radiation passing through the aperture, wherein the collimated radiation is combined with the above Fig. 7A The way the discussion is distributed among the photomultipliers.

[0112] In some embodiments, the system 10 can be configured to use the excitation radiation to provide dark field images and bright field images of the sample (in a flow cell devoid of the sample). As an example, Figure 7C An embodiment of a system 10 is schematically depicted, the system comprising two detection arms 200 and 202 for detecting dark field images and bright field images of a sample, respectively.

[0113] More specifically, the detection arm 200 is positioned perpendicular to the propagation of the excitation radiation so as to receive a portion of the excitation radiation scattered by the sample flowing through the circulation pool. The detection arm 200 includes two lenses 204 and 206, which jointly direct at least a portion of the excitation radiation scattered by the sample to a solid angle subtended by the lens 204 to the photomultiplier tube 208. More specifically, the lens 204 collimates the received scattered radiation, and the lens 206 focuses the collimated scattered radiation onto the photomultiplier tube 208. In this embodiment, an appropriate bandpass filter 210 is disposed in front of the photomultiplier tube 208 to allow radiation having a desired frequency to pass through and reach the photomultiplier tube 208, while blocking radiation at unwanted frequencies. The output of the photomultiplier tube 208 can be processed in a manner known in the art, for example, by an analysis module such as discussed below to generate a dark field image.

[0114] The detection arm 202 in turn includes two lenses 212 and 214, wherein the lens 212 collimates the excitation radiation leaving the flow cell in a forward direction (substantially along the propagation direction of the excitation radiation entering the flow cell 54), and the lens 214 focuses the collimated radiation onto a photodetector 216. An appropriate filter 218 (e.g., a bandpass filter) disposed in front of the photodetector allows other radiation frequencies to be substantially blocked while the excitation frequency is transmitted to the photodetector 216. The output of the photodetector 216 can be processed in a manner known in the art to generate a bright field image of the flow cell.

[0115] Thus, detection arm 200 detects excitation radiation scattered by fluid flowing through the flow cell, and detection arm 202 detects excitation radiation transmitted through the flow cell. When no fluid flows through the flow cell, the signal detected by photomultiplier tube 208 is low, and the signal detected by photodetector 216 is high, because little excitation radiation passing through the flow cell is scattered, and therefore a large percentage of the excitation radiation, and in some cases all of the excitation radiation, is transmitted through the flow cell. Conversely, the flow of a fluid sample through the flow cell can cause the signal generated by photomultiplier tube 208 to increase due to the scattering of a portion of the excitation radiation by the sample, and the signal generated by photodetector 216 to decrease due to the level of excitation radiation transmitted through the flow cell.

[0116] As another example and with reference to Fig.7D In one embodiment of a system according to the present teachings, a detection arm 220a, positioned relative to the flow cell 54 in a direction substantially orthogonal to the direction of propagation of the excitation radiation, includes a photodetector for detecting fluorescence radiation emitted by a plurality of fluorophores in the sample and excitation radiation scattered by the sample. More specifically, the detection arm 220 includes lenses 222 and 224 that direct the fluorescence radiation and the scattered excitation radiation to an aperture 226 that rejects unfocused radiation. Lens 228 collimates the radiation passing through the aperture. A dichroic mirror 230 reflects a portion of the radiation at the excitation frequency onto a photomultiplier tube 232 for detecting a dark field image while allowing the fluorescence radiation to pass through. An appropriate filter 232a (e.g., a bandpass filter) disposed in front of the photomultiplier tube 232 allows the radiation at the excitation frequency to pass through and reach the photomultiplier tube 232 while blocking unwanted radiation frequencies. Another dichroic mirror 234 reflects the fluorescence radiation emitted by the fluorophore at a first frequency onto the photomultiplier tube 236 while allowing the fluorescence radiation emitted by other fluorophores at other frequencies to pass through. Another dichroic mirror 238 reflects fluorescent radiation emitted by another fluorophore at a second frequency onto a photomultiplier tube 240 while allowing fluorescent radiation emitted by yet another fluorophore at a third frequency to pass therethrough, where the fluorescent radiation of the yet another fluorophore is detected by a photomultiplier tube 242. Similar to the previous embodiments, a plurality of filters 236a, 240a, and 242a are disposed in front of the photomultiplier tubes 236, 240, and 242, respectively, to allow transmission of radiation of desired frequencies while substantially blocking radiation of unwanted frequencies.

[0117] Continue to refer to Fig.7D This implementation of the system according to the present teachings also includes another detection arm 220b, which is used, for example, to combine Figure 7C The bright field image is generated in the manner discussed above. More specifically, the detection arm 202 includes two lenses 212 and 214 that focus light onto a photodetector 216 for generating a bright field image of the excitation radiation. A filter 218 (e.g., a bandpass filter) is placed in front of the photodetector 216 to allow the excitation radiation to pass through and reach the detector while rejecting unwanted radiation frequencies.

[0118] Refer again Figure 1 as well as Figure 8 In this embodiment, the transimpedance amplifier 70 can be coupled to the photodetector 64 (or in combination with Figures 7A-7DThe output of each of the photodetectors discussed above is used to amplify the signal generated by the photodetector. The data analysis unit 72 (also referred to herein as an analysis module or analyzer) receives the amplified signal and analyzes the signal to generate a fluorescent image of the sample. The data analysis unit 72 can be implemented in hardware, firmware and / or software. As an example, a method for analyzing the detected fluorescence data can be stored in a read-only memory (ROM) unit of the analysis module to be accessed under the control of the processor to analyze the received fluorescence signal.

[0119] As discussed in more detail below, the analysis method determines frequency components in the output of the photodetector that varies with time, and constructs a fluorescence image of the sample based on those frequency components. A variety of methods can be used to determine the frequency content of the output of the photodetector. Some examples of such suitable methods include, but are not limited to, Fourier transforms, lock-in detection, filtering, I / Q demodulation, homodyne detection, and heterodyne detection.

[0120] As an example, Fig. 9A and 9B An exemplary analysis that can be performed by the analysis module 72 to generate a fluorescence image of a sample is shown. At block (1), the analog amplified signal is digitized to generate digitized fluorescence data. At block (2), an appropriate portion (length) of the digitized data is selected for analysis. For example, fluorescence data corresponding to an illuminated row (also referred to herein as a frame) of the sample can be selected for analysis. Alternatively, a portion of a frame of data can be selected for analysis.

[0121] At block (3), a Fourier transform of the selected data is performed. By way of example, in some embodiments, a Fast Fourier Transform (FFT) of the data is performed to determine the frequency components of the data. In some such embodiments, the window of the FFT can correspond to the frequency selected for data acquisition. For example, for a sampling rate of 256 MHz, 256 samples can produce frequency windows that are 1 MHz apart from each other, e.g., from DC to 128 MHz. The FFT analysis provides a frequency corresponding to a beat frequency at which the fluorescent emission emitted exhibits amplitude modulation.

[0122] Continue to refer to Fig. 9A and 9B, in this embodiment, at block (4), a measure of the amplitude of each frequency component present in the FFT data is calculated by obtaining the square root of the sum of the squares of the real and imaginary parts of the frequency component. Since each frequency component corresponds to one of the beat frequencies used to induce fluorescent radiation from a specific position of the sample, the measure of the amplitude of the frequency component can provide a pixel value for the position associated with the frequency component along the horizontal row of the sample. In this way, the pixel values ​​of the image of the horizontal row of the sample can be determined. As the sample flows through the flow cell in the vertical direction, the above blocks can be repeated for the fluorescent data obtained for each horizontal row of the sample. The pixel values ​​can be used to construct a fluorescent image (block 5).

[0123] As described above, the analysis module 72 can be implemented in hardware, firmware, and / or software using techniques known in the art and in accordance with the present teachings. As an example, Fig.10 An exemplary implementation of the analyzer 72 is schematically depicted and includes an analog-to-digital converter 1002 for receiving the amplified fluorescence signal from the amplifier 70 and digitizing the signal to generate digitized fluorescence data. The analysis module also includes a central processing unit (CPU) 1004 for controlling the operation of the analysis module and including performing calculations and logical operations. The analysis module also includes a ROM (read only memory) 1006, a RAM (random access memory) 1008, and a permanent storage 1010. A communication bus 1012 facilitates communication between the various components of the analysis module, including facilitating communication between the CPU 1004 and other components. The memory module can be used to store instructions for analyzing fluorescence data and analysis results. As an example, in some embodiments, instructions for data analysis, such as for performing a combination of Fig. 9A and 9B The instructions of the above blocks discussed above can be stored in ROM 1006. The CPU can employ the instructions stored in ROM 78 to operate on the digitized fluorescence data stored in RAM 1008 to generate a fluorescence image (e.g., a one-dimensional or two-dimensional image) of the sample. The CPU can implement storage of the fluorescence image in permanent storage 1010 (e.g., in a database). Fig.10 As schematically shown in , the analysis module can optionally include a graphics processing unit (GPU) 1004' for performing calculations of pixel intensities and other quantities from received data (eg, fluorescence data).

[0124] In some embodiments, a lock-in detection technique can be used to implement frequency demodulation of the output signal generated by the photodetector. Fig.11A and 11BIn one such embodiment, the amplified fluorescence signal is digitized (block 1) and multiple copies of the digitized fluorescence signal are generated (block 2), where the number of digitized copies (N) corresponds to the number of frequencies associated with the RF comb beam. Each digitized copy of the signal is multiplied by a sine wave and a cosine wave whose frequency corresponds to a beat frequency to generate multiple intermediate signals (block 2), the beat frequency being equal to the difference between the frequency of one of the RF comb beams and the frequency of the LO beam. Each intermediate signal is passed through a low pass filter (block 3) having a bandwidth equal to half the frequency spacing between the RF comb frequencies.

[0125] For each beat frequency corresponding to one of the RF comb frequencies (in other words, for each frequency corresponding to a spatial position of an illuminated sample), the square root of the sum of the squares of the two filtered intermediate signals corresponding to that frequency is obtained as a measure of the amplitude of the image pixel corresponding to the sample position illuminated by the LO beam and the RF comb beam having that frequency (Block 4). In some embodiments, multiple fluorescence data signals corresponding to the same beat frequency (i.e., corresponding to the same sample position) can be processed in the manner discussed above, and the pixel values ​​can be averaged to obtain an average pixel value.

[0126] As the sample flows through the flow cell in a vertical direction, the above blocks can be repeated for each horizontal row of fluorescence data acquired for the sample. The pixel values ​​can be used to construct a fluorescence image (Block 5).

[0127] The above lock detection method can be implemented in software, firmware and / or hardware. As an example, in one embodiment, the above lock detection method can be implemented using a field programmable gate array (FPGA), particularly if more than six frequencies are used. In some embodiments, a multi-frequency lock-in amplifier can be used, such as the HF2L-MF multi-frequency amplifier sold by Zurich Instruments of Zurich, Switzerland.

[0128] As another example, in some embodiments, the frequency demodulation of the detected fluorescence signal can be achieved by using a bandpass filter-based image demodulation technique. Fig. 12A and 12BIn one embodiment of this frequency demodulation method, the fluorescence signal provided by the photodetector 64 and the amplifier 70 is digitized (block 1), and multiple copies of the digitized signal are generated (block 2), where the number of digitized copies (N) corresponds to the number of frequencies associated with the RF comb beam. Each copy of the digitized fluorescence signal is filtered by passing the signal through a bandpass filter centered on the beat frequency associated with one of the RF comb beams (i.e., the beat frequency associated with a particular location of the sample) (block 3). More specifically, each bandpass filter is centered on one of the N beat frequencies and has a bandwidth equal to half the frequency spacing between adjacent beat frequencies.

[0129] An envelope detector at each beat frequency is employed to estimate the amplitude of each pixel corresponding to the frequency for each horizontal line (block 4). In some cases, a plurality of pixel values ​​corresponding to a pixel obtained by processing a plurality of fluorescence signals corresponding to a sample position associated with the pixel are averaged to obtain an average pixel value. The above blocks can be repeated for the fluorescence data obtained for each horizontal row of the sample as the sample flows through the flow cell in a vertical direction. The pixel values ​​can be used to construct a one-dimensional or two-dimensional fluorescence image of the sample (block 5).

[0130] The analysis module can also be configured to receive and process brightfield and darkfield image data. Figure 7C and Figure 8 , the analysis module 72 can also be configured to receive dark field and bright field image data from the photodetectors 208 and 218 to generate dark field and bright field images. Fig.10 Instructions for generating dark field and bright field images, for example, in a manner known in the art, can be stored in persistent memory 82. Processor 76 can use these instructions to process received dark field and bright field image data to generate images. The analysis module can also be configured to generate a composite image by superimposing, for example, a fluorescent image and one or both of the bright field and dark field images.

[0131] Fluorescence images and bright field and dark field images generated by the system (such as, above system 10) according to this teaching can be used in various ways. For example, the fluorescence image can be integrated to produce a value comparable to the data produced by a conventional flow cytometer. It is also possible to analyze the fluorescence image to determine the position of the fluorescent probe that produces the image (for example, it is possible to determine whether the probe is a cell nucleus, cytoplasm, positioned in an organelle, or outside a cell membrane). In addition, in some applications, multiple fluorescence images obtained by detecting different fluorescence bands all from the same cell can be used to determine the co-localization degree of multiple fluorescent probes in the cell. Additionally, multicolor fluorescence, bright field and dark field images can be used to analyze cell morphology, cell signaling, internalization, cell-cell interaction, cell death, cell cycle and spot counting (for example, FISH) etc.

[0132] As described above, the system 10 can be operated in at least three different modes. In one mode discussed above, the LO beam and multiple RF comb beams simultaneously illuminate a portion of the sample (e.g., a position set along a horizontal range), and the fluorescent radiation emitted from the illuminated position is detected and analyzed to construct a fluorescent image of the sample. In another operating mode, a frequency ramp containing a drive signal is applied to the AOD instead of applying multiple RF drive signals to the AOD simultaneously, so that the frequency of the laser beam changes over time from a starting frequency (f1) to a final frequency (f2). For each drive frequency in the frequency ramp, the frequency of the laser beam is shifted by the drive frequency, and the sample is irradiated by the frequency-shifted laser beam to emit fluorescent radiation from the sample. In other words, in this mode, the system is operated to obtain fluorescent radiation from the sample by continuously irradiating the sample at multiple frequencies within a time interval, and these frequencies are offset from the center laser frequency. The frequency shift generated by the AOD is accompanied by an angular deflection so that the beam can be scanned at high speed over the entire sample using the same optical path.

[0133] More specifically, in this operating mode, the RF frequency synthesizer 10 is used to tilt the drive signal applied to the AOD 18 from a starting frequency (f1) to an ending frequency (f2). As an example, the frequency range of the drive signal tilt can be about 50 MHz to about 250 MHz. In some embodiments, the drive signal tilts from about 100 MHz to about 150 MHz. In this embodiment, the drive frequency varies continuously over time, for example to achieve high speed. In other embodiments, the drive frequency can vary in discrete steps from the starting frequency (f1) to the ending frequency (f2).

[0134] The drive frequency is selected so that the frequency shifted beam will miss mirror 28 and propagate along the optical path defined by lens 26, lens 30, mirrors 40 / 42, beam splitter 44, lens 46, mirror 56, lens 50, mirror 58 and objective lens 52 to illuminate a portion of the sample flowing through the sample holder. The ramp rate is preferably fast enough to improve and preferably prevent any blurring in the vertical direction of the fluorescent image based on the emitted fluorescent radiation as the sample flows through the beam. This can be achieved, for example, by matching the slope to the flow rate of the sample. The laser spot size at the sample can be used to estimate the appropriate rate. As an example, for a laser spot of 1 micron, for a sample flow rate of 0.1 meters per second, the scan time on 1 line should be 10 microseconds or less to avoid image blurring.

[0135] Fluorescent radiation emitted from the sample in response to illumination by the excitation radiation is collected and detected in the manner discussed above. Figure 8 , the fluorescence radiation is detected by the photodetector 64. The detected fluorescence is amplified by the amplifier 70, and the amplified signal is analyzed by the analysis module 72 to reconstruct a fluorescence image of the sample. The reconstruction of the image is performed by assigning horizontal pixel positions to specific times from the starting frequency (f1) to the ending frequency (f2) within the scanning period. In contrast to analyzing the amplitude of the frequency components to obtain pixel values ​​in the above-mentioned operating mode, the demodulation method used in this operating mode only uses the time domain values ​​of the detected fluorescence signal to assign values ​​to the pixels of the image. This process can be repeated when the sample flows in the vertical direction so as to obtain a two-dimensional fluorescence image of the sample.

[0136] Fluorescent radiation emitted by the sample, if any, is collected by photodetector 64. Figure 8 , the detected fluorescent radiation is amplified by the amplifier 70. The analysis module 72 receives the amplified signal. In this operating mode, the analysis module analyzes the fluorescent signal to determine the fluorescent content of the sample (e.g., cell / particle). Since there is only one beam to excite the sample in this operating mode, the beat frequency is not generated in response to the excitation sample. Therefore, there is no image information in the frequency domain of the fluorescent signal. On the contrary, the detected fluorescent signal has image information encoded in the time domain. In this operating mode, the image can be digitally reconstructed using the time value of the detected fluorescent signal as the horizontal pixel coordinate and the digitized voltage value of the fluorescent signal as the pixel value (brightness). Each scan of the driving frequency applied to the AOD will produce a horizontal line (row) of the image. When the sample flows through the irradiated area (point), image reconstruction is achieved via continuous scanning.

[0137] In yet another mode of operation, the system 10 can be operated to simultaneously illuminate multiple locations of a sample with a single excitation frequency, which can be generated, for example, by shifting the center frequency of the laser beam by one radio frequency. Figure 1 In this mode of operation, a single driving RF can be applied to the AOD 18 to generate a laser beam whose frequency is shifted relative to the laser beam entering the AOD 18. In addition, the frequency-shifted laser beam exhibits an angular shift relative to the laser beam entering the AOD, so that the RF laser beam is intercepted by the mirror 28 and is reflected by the mirror 28 to the top-hat beam shaper 34 via the lens 32 and the mirrors 33 and 35. The beam leaving the top-hat beam shaper is reflected by the beam splitter 44 and is focused by the lens 46 onto the intermediate image plane 48. In this plane, as shown in FIG. Fig.13A As schematically shown in FIG. 1 , the laser beam 1300 shows a stretching profile along the horizontal direction.

[0138] The horizontally stretched laser beam is reflected by mirror 56 to convex lens 50. After passing through lens 50, the laser beam is reflected by mirror 58 to objective lens 52. As discussed above, convex lens 50 and objective lens 52 form a telescope for relaying the top-hat profile laser beam from intermediate image plane 48 to the sample flowing through flow cell 54.

[0139] The horizontally stretched laser beam illuminates a horizontal extent of the sample to excite the fluorophore of interest (if present in the sample) along the horizontal extent. Thus, in this operating mode, multiple horizontal positions of the sample are illuminated with the same excitation frequency, unlike the first operating mode in which multiple horizontal positions of the sample are illuminated with different excitation frequencies. This operating mode does not enable the user to obtain images of cells or particles passing by. However, in this operating mode, higher optical powers can generally be applied to the sample than in the other two operating modes, which may be useful for obtaining higher signal-to-noise data if an image is not required. This operating mode can be accessed by simply changing the electronic signal that drives the acousto-optic deflector, without making any mechanical changes to the system.

[0140] Thus, system 10 is capable of operating in three different modes of operation to induce fluorescent radiation from a sample.

[0141] In some embodiments, the fluorescence lifetime measurement can be performed at each spatial location of the sample, for example, by comparing the beat frequency phase of each of the RF frequency-shifted beam and the local oscillator beam with the phase of the corresponding RF component in the detected fluorescence signal. As an example, Fig. 13B A system 10' is shown which is a modified version of the system 10 discussed above which allows for such fluorescence lifetime measurements (not depicted in this figure for simplicity). Figure 14). Specifically, a portion of the RF comb beam incident on the beam splitter 44 is reflected by the beam splitter onto a focusing lens 400 (illustratively, in this embodiment, the lens 400 has a focal length of 200 mm, but other focal lengths can also be used). The lens 400 focuses the portion of the RF comb beam onto a photodiode 402 that detects the excitation beam. The output of the photodiode 402 can be received by the analysis module 72 (see Figure 8 ). The analysis module can provide frequency demultiplexing of the excitation beam and determine the phase of each RF component in the excitation beam, for example using one of the demodulation techniques discussed above. For each RF component in the detected fluorescence signal, this can provide a reference phase that can be compared with the phase of the RF component. For example, the real and imaginary parts of the FFT of the excitation signal or the I and Q components of a lock-in demodulation can be used. Alternatively, the output of a detector for detecting a bright field image of the sample / circulation cell can be used to obtain a reference phase that can be compared with the phase of the fluorescence beat frequency.

[0142] More specifically, the analysis module 72 can provide frequency demultiplexing of the detected fluorescence signal, for example, in the manner discussed above. As will be appreciated by those skilled in the art, for each beat frequency in the fluorescence signal, the phase of the radio frequency component can be compared with the corresponding reference phase of the excitation beam to obtain a spatially resolved fluorescence lifetime measurement and a fluorescence lifetime image.

[0143] In certain embodiments, the subject system includes a flow cytometer system that employs the above optical configuration for detecting light emitted by a sample in a flow. In certain embodiments, the subject system is a flow cytometer system that includes one or more components of a flow cytometer described in the following U.S. Patents: 3,960,449; 4,347,935; 4,667,830; 4,704,891; 4,770,992; 5,030,002; 5,040,890; 5,047,321; 5,245,318; 5,317,162; 5,464,581; 5,483,469; 5,602,039; 5,620,842; 9,095,494; and 9,097,640, the entire disclosures of which are incorporated herein by reference.

[0144] As discussed above, in some embodiments, the subject system is configured for imaging particles (e.g., cells) in a sample flowing as a stream, such as in the stream of a flow cytometer. The flow velocity of the particles in the stream can be 0.00001 m / s or more, such as 0.00005 m / s or more, such as 0.0001 m / s or more, such as 0.0005 m / s or more, such as 0.001 m / s or more, such as 0.005 m / s or more, such as 0.01 m / s or more, such as 0.05 m / s or more, such as 0.1 m / s or more, such as 0.5m / s or more, such as 1m / s or more, such as 2m / s or more, such as 3m / s or more, such as 4m / s or more, such as 5m / s or more, such as 6m / s or more, such as 7m / s or more, such as 8m / s or more, such as 9m / s or more, such as 10m / s or more, such as 15m / s or more, and including 25m / s or more. For example, depending on the size of the stream (e.g., the stream nozzle orifice), the flow rate of the stream in the subject system can be 0.001 μL / min or more, such as 0.005 μL / min or more, such as 0.01 μL / min or more, such as 0.05 μL / min or more, such as 0.1 μL / min or more, such as 0.5 μL / min or more, such as 1 μL / min or more, such as 5 μL / min or more, such as 10 μL / min or more, such as 25 μL / min or more, such as 50 μL / min or more, such as 100 μL / min or more, such as 250 μL / min or more, and including 500 μL / min or more.

[0145] In some aspects, methods and systems are disclosed for providing an estimate of one or more characteristics of a particle (e.g., a cell). As an example, Fig.14AA flow chart is shown that describes an exemplary method for determining one or more characteristics of a particle according to an embodiment of the present teachings. As the particle flows through a flow cytometer system, an RF modulated optical beam is used to illuminate the particle so as to induce at least one radiation response from the particle (block 1). As an example, the RF modulated optical beam can include at least two sub-beams whose optical frequencies are shifted from each other by one RF. In some embodiments, the RF frequency shift can be from about 10 MHz to about 250 MHz. For example, the RF frequency shift can be from about 55 MHz to about 225 MHz, such as from about 60 MHz to about 200 MHz, such as from about 65 MHz to about 175 MHz, such as from about 70 MHz to about 150 MHz, and includes from about 75 MHz to about 125 MHz. As an example, in some embodiments, an RF modulated optical beam can be generated by introducing a laser beam into an acousto-optic deflector (AOD) and applying one or more drive signals to the AOD at one or more radio frequencies so as to generate a plurality of angularly separated sub-beams whose optical frequencies are shifted relative to each other by the radio frequencies, for example, in the manner discussed above.

[0146] In some embodiments, the radiation response induced from the particle in response to illumination of the particle by the radio frequency modulated optical beam can be either fluorescent radiation and / or scattered radiation.

[0147] Continue to refer to Fig.14A , a flow chart of a method for detecting a radiation response emitted from a particle (block 2), and capable of generating time waveform data associated with the radiation response (block 3). Various radiation detection modalities and detectors (such as those discussed above) can be used to detect the induced radiation response. In some embodiments, the generated waveform can be fluorescence waveform data and / or scattering waveform data. The waveform data can be processed to obtain an estimate of at least one feature of the particle. In many embodiments, such processing of the waveform data can be performed to obtain an estimate of at least one characteristic of the particle without generating an image of the particle based on the waveform data. In some embodiments, the processing includes analyzing one or more beat frequencies of the modulated time waveform data to obtain an estimate of at least one feature of the particle. In some embodiments, the processing is performed quickly enough so that the waiting time associated with obtaining an estimate of at least one feature of the particle is less than about 100 microseconds.

[0148] In some embodiments, the above methods can be used to obtain an estimate of any of a dimensional size of a particle, the ratio of particle sizes along two different dimensions, the co-localization of fluorescent radiation emitted by two or more markers associated with the particle, or the accuracy of the radiation response (e.g., the accuracy of the fluorescent radiation emitted by the particle), among others.

[0149] The above method can be used to obtain an estimate of one or more characteristics of a variety of different particles. As an example, the particle can be any of a cell, a small organism (e.g., C. elegans), a bead, a microparticle, a nanoparticle, a viral particle, a bacterium, an exosome, or a pharmaceutical product.

[0150] Fig. 14B A system 1400 is schematically depicted according to one embodiment for estimating at least one characteristic of a particle such as a cell. The exemplary system 1400 includes an illumination system 1402 for illuminating one or more particles flowing through a flow cell in a flow cytometer system using a radio frequency modulated optical laser beam. A detector 1404 is capable of detecting a radiation response (e.g., fluorescent radiation and / or scattered radiation) of the particle in response to its illumination and generating one or more signals indicative of the radiation response. An analyzer 1406 is capable of receiving one or more signals from the detector and generating time waveform data and operating on the waveform data to derive an estimate of one or more characteristics of the particle.

[0151] The analyzer 1406 can employ a variety of different methods to analyze the waveform data to obtain an estimate of one or more features of a particle, such as a cell. As an example, in some embodiments, the particle can be stained with at least two fluorescent markers, wherein each marker is configured to emit fluorescent radiation in response to illumination by radio frequency modulated optical radiation. The fluorescent radiation can be detected and digitized to generate a fluorescent waveform corresponding to one of the markers. The analyzer can operate on the fluorescent waveform to obtain a measure of co-localization of the fluorescent radiation emitted from the marker. Specifically, the analyzer can apply a high pass filter or a band pass filter to at least one of the waveforms to generate at least one filtered waveform, then perform point-by-point multiplication of the waveforms to generate a resultant multiplication waveform, integrate the multiplication waveform to obtain an integral value, and compare the integral value to a predefined threshold to obtain a measure of co-localization. As another example, in some embodiments, an estimate of the size of the particle can be obtained along a direction perpendicular or parallel to the direction of particle flow in a flow cytometer system. For example, in some such embodiments, an estimate of the particle size in a direction perpendicular to the direction of particle flow can be obtained by squaring a fluorescence waveform corresponding to fluorescent radiation emitted by a particle in response to illumination by a radio frequency modulated optical beam, applying a bandpass filter to the squared waveform, integrating the filtered waveform, and comparing the integrated value to a predefined threshold. Additionally, in some embodiments, the analyzer can use scattering data to obtain an estimate of the particle size in a direction parallel to the direction of particle flow.

[0152] As discussed below, one or more estimated characteristics of a particle flowing through a flow cytometer system can be used to reach a sorting decision for the particle, i.e., whether to sort the particle. In the context of using characteristic estimates of cells flowing through a flow cytometer to make sorting decisions about the cells, some examples of processing methods that can be used to operate on waveform data to obtain an estimate of at least one characteristic of a particle are discussed below. It should be understood that such processing methods can be used to obtain estimates of characteristics of particles other than cells, and the estimated characteristics may also not be used for sorting purposes.

[0153] In a related aspect, methods are disclosed for automatically gating (e.g., computer-assisted gating) a population of particles (e.g., cells) flowing through a flow cytometer based on one or more characteristics of the particles. As an example, such methods can gate a subset of a plurality of particles flowing through a flow cytometer based on the size of the particles being within a predetermined range. For example, referring to Fig. 14C Flowchart of the invention, the method can include introducing a sample containing a plurality of particles into a flow cytometer (block 1), and obtaining an estimate of at least one particle characteristic for the plurality of particles from one or more flow cytometer measurements (block 2). Obtaining the at least one particle characteristic can include: irradiating the particles with radiation having at least two optical frequencies that are shifted from one another by one radio frequency as the particles flow through the flow cytometer to induce a radiation response from the particles; detecting the radiation response from the particles to generate time waveform data associated with the response, and processing the time waveform data to obtain a value of the at least one particle characteristic by analyzing one or more beat frequencies that modulate the time waveform data. The method can further include identifying, via a computer processor, a gating indicating that one or more particles have a value of a particle characteristic that is within a predefined range. As an example, particles whose dimensional size (e.g., lateral size) is within a predefined range can be gated.

[0154] Systems (such as Fig. 14B) can be used to perform the above gating method. For example, analyzer 3006 can be programmed to determine an estimate of at least one characteristic of a plurality of particles, for example, based on an analysis of one or more beat frequencies in fluorescent radiation emitted by those particles in response to illumination by a radio frequency modulated optical beam, and determine whether the estimate of the particle characteristic is within a predefined range in order to make a gating decision with respect to the particle (e.g., if the determined characteristic is within the predefined range, the particle will be gated). In some embodiments, the teachings of U.S. Patent No. 8,990,047, entitled "Neighborhood Thresholding in Mixed Model Density Gating," modified based on the present teachings, can be used to gate particles flowing through a flow cytometer. U.S. Patent No. 8,990,047 is incorporated herein by reference in its entirety.

[0155] In some aspects, methods and systems for sorting cells based on interrogation of those cells via radio frequency modulated optical radiation are disclosed, the radio frequency modulated optical radiation being, for example, an optical radiation beam comprising two or more optical frequencies separated by one or more radio frequencies. In some embodiments, the optical beam can include sub-beams separated at multiple angles or in space, each of which has a radio frequency shift relative to each other. In some cases, the use of such a beam allows different locations within a particle (e.g., a cell) to be irradiated with different radio frequency shifted optical frequencies. As discussed in more detail below, such a method can provide a sorting decision by using a time-varying signal generated by a cell in response to the irradiation of the optical beam, without the need to calculate a fluorescent image based on one or more detected signals. Although various embodiments of the method according to the present teachings are discussed below in the context of sorting cells (e.g., in a flow cytometer system), the methods described herein can also be used to sort other types of particles, such as small organisms (e.g., Caenorhabditis elegans), beads, microparticles, nanoparticles, viral particles, bacteria, exosomes, or pharmaceutical products. In some embodiments, particles that can be sorted using the present teachings can have a size (eg, largest dimension) ranging from about 50 nanometers to about 1 millimeter, such as from about 100 nanometers to about 1 micrometer.

[0156] In addition, in many embodiments, the sorting methods discussed herein can be used to provide sorting decisions with low latency, for example, to enable sorting of cells or other particles using a sorting device operating at a high particle throughput (e.g., one that can perform more than 1000 sorting operations per second). As an example, the methods described herein can be used to make sorting decisions with a latency equal to or less than about 100 microseconds, such as about 10 microseconds to about 100 microseconds, or about 20 microseconds to about 80 microseconds, or about 30 microseconds to about 70 or 50 microseconds. The term "latency" is used herein to indicate the time interval between irradiating a particle (e.g., a cell) with interrogating radiation and obtaining a particle characteristic and / or a sorting decision regarding the particle.

[0157] As an example, Fig.14D A flowchart of depicts a method according to one embodiment for sorting particles (e.g., cells) based on the degree of co-localization of fluorescent signals corresponding to two or more fluorophores (e.g., exogenous and / or endogenous fluorophores) emitted from the particles. Without loss of generality, the particles are assumed to be cells in the following discussion. At block (1), cells are optically interrogated via irradiation using an optical radiation beam comprising two or more optical frequencies separated from each other by one or more radio frequencies. In some cases, the optical beam can include sub-beams separated at multiple angles or spatially, each of which has a radio frequency shift relative to each other. The use of such a beam allows different locations within a cell to be irradiated with different optical frequencies that are shifted from each other by one or more radio frequencies. The optical frequencies can be selected to excite two or more fluorophores that are expected to be associated with the cell. For example, the fluorophores can be fluorescent dye molecules that are used to identify cells, such as via staining. As an example, in some embodiments, the optical frequency of the radiation beam can be about 300 THZ to about 1000 THZ, and the radio frequency spacing between the optical frequencies can be, for example, about 50 MHz to about 250 MHz.

[0158] The fluorescence radiation emitted from the excited cells can then be collected in two (or more) separate fluorescence channels, each corresponding to the fluorescence radiation emitted by one of the fluorophores (Block 2). For example, this can be accomplished by using the above combined Fig. 7AThe detector arrangement discussed above is implemented. The fluorescence radiation collected in each channel can be digitized (block 3) and represented as a time series of signal values ​​(fluorescence intensity). In this embodiment and other embodiments, the time series of signal values ​​that can encode the beat frequency present in the fluorescence radiation is referred to as a time-frequency waveform. The digitized time-frequency waveforms corresponding to two or more fluorescence channels are synchronized in time (block 4) and normalized (block 5). For example, the normalization can be achieved by dividing each waveform by its maximum value and multiplying the waveform by a scale factor.

[0159] A high pass filter or a band pass filter can be applied to at least one of the waveforms to generate at least one filtered waveform (block 6). In some embodiments, a low pass filter is applied to allow frequencies less than about 1 MHz to pass and substantially block higher frequencies. Some examples of suitable low pass filters include, but are not limited to, a time domain finite impulse response (FIR) filter or a Fourier domain filter.

[0160] The waveforms can then be multiplied point by point to obtain a multiplied waveform, and the multiplied waveform can be integrated to obtain an integrated value (block 7). As an illustration, Fig.15 Array 1500 and array 1501 are shown, wherein array 1500 represents a filtered normalized time series digitized fluorescence signal detected in one fluorescence channel, and array 1501 represents a filtered normalized time series digitized fluorescence signal detected in another fluorescence channel synchronized in time with array 1500 (for simplicity, in this illustrative example, the number of fluorescence channels is selected to be 2 and the number of array elements is selected to be 10, it should be understood that the number of fluorescence channels can be greater than 2 and the number of array elements can be greater than 10). This data is referred to herein as a time-frequency waveform. The result array 1502 is obtained via point-by-point multiplication of the data in arrays 1500 and 1501.

[0161] The temporal fluorescence signal from each channel includes a beat frequency corresponding to the interference of the radio frequency separation optical frequencies of the optical radiation beam. Thus, the multiplication of their respective digitized data will exhibit the sum and difference of those frequencies. If the signals in two or more fluorescence channels originate from substantially similar spatial locations within the excited cell, then the resulting temporal data obtained by multiplying these signals will contain frequency components at or near DC, based on the degree of co-localization of these signals in the different fluorescence channels originating from the excited cell.

[0162] At block (8), the integration result is compared to a predefined threshold to determine whether the interrogated cell exhibits sufficient colocalization of the fluorescent signal to be identified as a cell that meets the sorting criteria. For example, if the integration result equals or exceeds the predefined threshold, the cell will be selected for sorting. Otherwise, the cell will not be selected for sorting.

[0163] The above sorting method based on fluorescence co-localization can be used in a variety of applications, for example, for translocation analysis.

[0164] In another aspect, a method for sorting cells in a flow cytometer system based on cell size estimation is provided, wherein an estimate of cell size is obtained by analyzing fluorescent radiation emitted by the cells. For example, the method uses the duration of a fluorescence pulse emitted from the cell to estimate the cell dimension along the direction of flow, and analyzes the power contained in the low-pass filtered signal obtained by passing the square of the fluorescence signal through a low-pass filter to perform a sorting decision based on an estimate of the cell's lateral dimension (e.g., the cell dimension orthogonal to the direction of cell flow).

[0165] More specifically, refer to Fig.16 In one embodiment, cells are optically interrogated via illumination with an optical radiation beam comprising two or more optical frequencies separated from each other by one or more radio frequencies (Block 1). Similar to the previous embodiments, for example, the optical beam can include a plurality of angularly or spatially separated sub-beams, each of which has a radio frequency shift relative to each other. The optical frequencies can be selected to excite one or more exogenous and / or endogenous fluorophores associated with the cells. The cells pass through the optical radiation beam and emit fluorescence in response to excitation by the beam. Without loss of generality, in this embodiment, it is assumed that the cells flow through the illumination beam in a vertical direction, and the emitted fluorescence is detected in a lateral (horizontal) direction substantially orthogonal to the direction of cell flow.

[0166] The fluorescent signal emitted from the cells is then detected and digitized to generate a time-frequency waveform (block 2). The detected fluorescent radiation can then be analyzed to estimate the cell size, as discussed below. In some embodiments, the duration of the light scattering pulse emitted from the cells can be used to estimate the cell size along the flow direction.

[0167] The duration of the fluorescence pulse emitted from the cell is related to the residence time of the cell within the interrogating optical radiation beam, which in turn is related to the beam size in the direction parallel to the flow direction, the cell size in the flow direction, and the flow rate of the cell. If the beam illuminating the cell has a diameter (H), and the flow rate of the cell is V, and the size of the cell in the flow direction is D (e.g., diameter), then the size D can be approximated by the following relationship:

[0168] D=V*T-2*H Equation (2)

[0169] where T is the detected optical pulse width. Thus, at block (3), the cell size in the direction of flow is estimated based on the duration of the fluorescence or light scattering pulse emitted from the cell, for example using the above relationship.

[0170] Continue to refer to Fig.16 , to estimate the lateral size of a cell, the digitized fluorescence data is squared (block 4), and a bandpass filter is applied to the squared fluorescence data (block 5). The filtered data is then integrated to obtain a measure of the integrated pulse power (block 6). The integrated pulse power can provide a measure of the lateral size of a cell. More specifically, based on the power present at different frequencies that fall within the bandpass filter band (due to the squaring operation), the integrated pulse power can provide a measure of the lateral size of a cell.

[0171] At block (7), an estimate of the cell size in the direction of flow and / or the integrated pulse power associated with the filtered data can be used to make a sorting decision with respect to the cell. For example, in some embodiments, the estimated cell size in the direction of flow can be compared to a first threshold, and the integrated pulse power can be compared to a second threshold to make a sorting decision. As an example, in some cases, if both the estimated cell size and the integrated pulse power in the direction of flow exceed their respective thresholds, a positive sorting decision is made (i.e., the cell is selected). Alternatively, the sorting decision can rely solely on an estimate of the cell size in the direction of flow or solely on the integrated pulse power. As discussed further below, in some cases, a ratio of the estimated vertical and horizontal cell sizes can be used to make a sorting decision.

[0172] As a further illustration of making sorting decisions based on estimates of cell lateral size, Fig.17A A hypothetical cell is schematically depicted being illuminated by a hypothetical beam comprising a plurality of RF modulated sub-beams having RF modulation extending from 15 MHz to 25 MHz and separated from each other by 1 MHz. The dashed line schematically depicts a cross-sectional view of the illuminated cell in a plane orthogonal to the illumination direction. The fluorescent radiation collected and digitized from the cell can be in the form of a time series array 1700 of digitized fluorescence values, such as Fig. 17B1700 is squared to obtain waveform 1701. A bandpass filter is then applied to the squared fluorescence waveform 1701, wherein the filter allows a selected modulation frequency to pass between two frequencies f1 and f2 (where f2>f1), while substantially blocking those modulation frequencies below f1 or above f2. As an example, the bandpass filter can be a FIR bandpass filter, but other suitable filters known in the art can also be used. The filtered data indicates the power present in the fluorescence pulse detected at modulation frequencies between f1 and f2. The filtered data is then integrated to obtain a measure of the total pulse power at frequencies between f1 and f2. The integrated result can then be compared to a threshold to make a sorting decision. For example, based on the fact that the integrated result is less than a predefined threshold, it can be estimated that the lateral size of the cell is less than a certain value.

[0173] In this example, the cell size results in illuminating the cell with optical radiation whose radio frequency modulation is about 18 MHz to about 21 MHz. Thus, the difference between the maximum and minimum modulation frequencies in the fluorescence data square is about 6 MHz. If it is desired to detect larger sized cells that would result in a frequency difference in the fluorescence data square of about 10 MHz to about 15 MHz, a bandpass filter that discriminates between frequencies below 10 MHz and above 15 MHz can be applied to the fluorescence data square. In this example, applying such a bandpass filter to the fluorescence data square would not result in a signal large enough to indicate the presence of a cell of the desired lateral size because the modulation frequency difference in the fluorescence data square is below 10 MHz.

[0174] The above sorting methods based on cell size can have a variety of different applications, for example, separation of circulating tumor cells (CTCs) based on their size.

[0175] In another aspect, a method for sorting cells based on cell aspect ratio is disclosed. Fig.18, using radiation to excite cells passing through an optical interrogation area, and collecting and digitizing the fluorescent radiation emitted by the cells (block 1) to generate a digitized time-frequency waveform. The digitized fluorescence data is then analyzed in the manner discussed above to estimate the vertical (along the flow direction) cell size (block 2) and the horizontal (along the direction orthogonal to the flow direction) cell size (block 3). More specifically, the vertical size of the cell can be estimated using the time width of the fluorescence or light scattering (or light transmission) pulse emitted by the cell (block 2), and the horizontal size of the cell can be estimated based on applying a bandpass filter to the square of the fluorescence data using, for example, the methods discussed above (block 3). At block (4), the ratio of the estimates of the vertical and horizontal sizes of the cell is determined and compared to a predefined threshold to make a sorting decision with respect to the cell. For example, in some cases, if the ratio exceeds a threshold, a positive sorting decision can be made with respect to the cell.

[0176] The above sorting method based on cell aspect ratio can be used for example for cell cycle analysis, DNA analysis, etc.

[0177] In another aspect, a method of determining the ratio of the size of a cell's nucleus relative to the size of its cytoplasm is disclosed. In some embodiments, such a ratio can be used to make sorting decisions. Fig.19A Flow chart, in one such embodiment, cells are labeled (e.g., stained) with two fluorescent markers, one of which resides on the cell membrane (the boundary of the cytoplasm) and the other of which is able to penetrate the cell membrane to enter the cytoplasm and stain the nucleus (e.g., via the cell's internal machinery) (block 1). Some examples of fluorescent markers that bind to the cell membrane include any antibody labeled with a fluorophore that binds to a membrane protein, such as anti-CD45-FITC or anti-EpCAM-PE. Other common surface proteins that can be used to bind fluorescent markers to cell membranes include, for example, CD3, CD4, CD8, etc. Some examples of suitable nuclear fluorescent stains include, but are not limited to, propidium iodide, SYTO16, 7-AAD, and DAPI.

[0178] The cells are then illuminated with radiation to excite both types of fluorescent markers (block 2), and the emitted fluorescence is detected and digitized in two channels corresponding to fluorescence emission from the cell membrane and from its nucleus (block 3). For example, the fluorescence data corresponding to the cell membrane is used to obtain an estimate of the size of the cytoplasm, and the fluorescence data corresponding to the nucleus is used to obtain an estimate of the size of the nucleus (block 4). For example, the width of the fluorescence or light scattering pulse in each channel can be used, for example, in the manner discussed above, to estimate the size of the cell along one dimension (i.e., along the direction of flow). In addition, the lateral size of the cytoplasm or nucleus can be estimated by applying a bandpass filter to the square of the fluorescence data in the respective channel in the manner discussed above. The result of integrating the bandpass filtered data provides a value that is compared to a predefined threshold to sort cells that are greater than or less than the threshold. In addition, the size estimates of the cells in the two dimensions can be combined, for example, by obtaining the square root of the sum of the squares of the vertical size estimate and the horizontal size estimate to make a sorting decision based on an estimate of the total cell size.

[0179] At block (5), a ratio of size estimates for the cytoplasm and the nucleus is obtained, for example, a ratio of the sizes in the vertical or horizontal direction, or a ratio of the combined sizes. Fig.19B In some embodiments, the ratio of the size of a cell's nucleus relative to the size of its cytoplasm (which can be determined, for example, in the manner discussed above and in Fig.19B The ratio can be used to make a sorting decision (represented in blocks (1)-(5) of the flowchart of ). For example, the ratio can be compared to a predefined threshold to make a sorting decision (block 6). As an example, if the ratio exceeds the threshold, a positive sorting decision is made (i.e., the cell is selected).

[0180] As an example, the above sorting method based on nuclear-cytoplasmic size ratio can be used in the classification of circulating tumor cells.

[0181] In another aspect, a method of estimating the cellular granularity (fluorescence punctateness) of fluorescent radiation emitted from a cell is disclosed, which in some embodiments can be used to sort cells. In other words, a sorting decision can be made based on whether the fluorescent radiation emitted from the cell can be characterized as emanating from a diffuse intracellular distribution of emission centers, or can be characterized as emanating from emission centers that are not diffusely distributed within the cell.

[0182] More specifically, refer to Fig. 20AThe flowchart described, in one embodiment, utilizes optical radiation having two or more optical frequencies separated from each other by one or more radio frequencies to illuminate cells having one or more exogenous and / or endogenous fluorescent markers to induce fluorescent radiation from the markers (block 1). Similar to the previous embodiments, in some cases, the optical beam can include a plurality of angularly or spatially separated sub-beams, each of which has a radio frequency shift relative to each other. As an example, refer to Fig.21 In an illustrative exemplary embodiment, the optical radiation beam can include modulation frequencies f1, f2, f3, f4, f5, which can, for example, span from 10 MHz (f1) to 15 MHz (f5) with an interval of 0.5 MHz. As discussed in more detail below, the point-likeness of the emitted fluorescent radiation can be estimated by the extent to which the modulation frequencies exhibited by the emitted fluorescent radiation differ from those modulation frequencies used to modulate the optical beam (e.g., in this embodiment, differ from f1, f2, f3, f4, f5). Other frequencies that can be frequencies between the frequencies used to modulate the optical beam are referred to herein as "out-of-pixel frequencies." In other words, the "leakage" of the fluorescent power to the out-of-pixel frequencies indicates the degree of point-likeness of the emitted fluorescence; the more leakage, the better the point-likeness of the emitted fluorescence.

[0183] Refer again Fig. 20A , fluorescent radiation can be collected and digitized to generate a time-frequency waveform (block 2). At block (3), a Fourier transform, such as a fast Fourier transform (FFT), of the entire waveform is obtained. The generated FFT is then normalized to a fixed value for comparison between cells, such as by dividing the time domain waveform by its maximum value and rescaling the waveform by a constant factor (block 4). The sum of the "out-of-pixel" window of the FFT is determined (i.e., the window corresponding to frequencies between the modulation frequencies present in the illuminating optical radiation) (block 5). This sum indicates the degree of "leakage" of the fluorescent power to out-of-pixel frequencies, and therefore indicates a measure of the degree of point-likeness of the emitted fluorescent radiation.

[0184] Reference Fig. 20B , in some embodiments, can be determined in the manner discussed above and Fig. 20B The measures of the degree of punctateness of the emitted fluorescent radiation represented in blocks (1)-(5) of the flowchart of can be used to sort cells. In particular, at block (6), the sum of the "out-of-pixel" window of the FFT (which indicates the degree of "leakage" of the fluorescent power to the out-of-pixel frequencies) can be compared to a predefined threshold to make a sorting decision. For example, when it is desired to sort cells that exhibit a relatively high degree of fluorescent punctateness, if the sum is greater than the threshold, the cell is sorted (selected). Alternatively, when it is desired to sort cells that exhibit diffuse fluorescence, if the sum is less than the threshold, the cell is sorted.

[0185] Such cell sorting based on analysis of emitted fluorescent radiation can be used in a variety of applications such as spot counting, fluorescence in situ hybridization (FISH), intracellular trafficking and localization, and droplet-based single-cell analysis.

[0186] The above method of making sorting decisions for particles (e.g., cells flowing in a flow cytometer system) can be implemented in a variety of ways. As an example, Fig. 22 An exemplary system 2200 for sorting cells is schematically depicted, which can employ the present teachings for making sorting decisions. The system 2200 can include a container 2201 for storing a suspension having a plurality of cells suspended therein. The cell suspension container 2201 is fluidically coupled to a sample conduit 2203 via an inlet 2202, which can be formed, for example, of a rigid metal such as stainless steel. The sample conduit 2203 is disposed within a receptacle 2204, which includes an upper cylindrical portion 2204a extending via a tapered portion 2204b to a lower cylindrical portion 2204c having a smaller diameter, the lower cylindrical portion comprising a nozzle 2205 at its distal end. The receptacle 2204 is fluidically coupled to a sheath fluid source 2206. An acoustic vibrator 2208 (e.g., a piezoelectric driver) is coupled to the nozzle and is configured to cause vibration of the nozzle when energized by a generator 2210. As an example, the vibration frequency can be approximately 60 kHz, although other vibration frequencies can also be used.

[0187] In use, the suspension stored in the container 2201 is introduced into the conduit 2203 via the inlet 2202. The thin flowing fluid containing cells leaving the container 2203 is entrained by the sheath fluid and carried to the nozzle 2205. The vibrating motion of the nozzle can be configured in a manner known in the art to divide the flow through the nozzle into a plurality of droplets D, each of which contains a single cell particle. At least some cells are associated with one or more endogenous and / or exogenous fluorophores that can emit fluorescent radiation in response to irradiation with excitation radiation. A wide variety of fluorophores can be used. In some cases, the fluorophore can be attached to an antibody that recognizes a target (e.g., a receptor) on the cell. In some cases, the fluorophore can be attached to a chemical entity that has an affinity for a cell membrane or another cell structure (e.g., a nucleus). In addition, in some cases, a combination of different fluorophores can be used to identify cells.

[0188] Continue to refer to Fig. 22As each cell passes through the interrogation region 2212, a laser beam 2214 generated by an illumination system 2216 illuminates the cell to induce fluorescence radiation from one or more fluorophores associated with the cell. As discussed above, the laser beam can include multiple optical frequencies that are shifted relative to each other by one or more radio frequencies. As an example, the optical beam can be in the form of multiple angularly or spatially separated sub-beams that have radio frequency frequency shifts relative to each other. The fluorescent radiation emitted from the cell can be detected by a detection system 2218 that can include one or more photodetectors. The detected fluorescence can be analyzed by an analysis module 2220 in the manner discussed above to make a sorting decision about the cell.

[0189] As an example, the illumination and detection system can be a combination of the above, for example Figure 1 -9 discussed above. However, it should be understood that the practice of the present teachings for sorting particles is not limited to the use of any particular illumination and detection system. Instead, a variety of different systems can be employed as long as they provide the necessary data (e.g., fluorescence and / or scattering data) used in the above methods for making sorting decisions.

[0190] Refer again Fig. 22 The system 2200 also includes a charging ring 2222, which is powered by a charging circuit 2223, which is in turn under the control of the analysis module 2220. The charging ring can apply a positive or negative charge to the cell droplet as the cell droplet passes through the ring. Alternatively, the ring can allow the cell droplet to pass through without applying a charge to it.

[0191] More specifically, the analysis module 2220 can employ the above teachings to make a sorting decision about the cell. Based on the decision, the analysis module can determine whether the cell droplet needs to be charged, and if so, which charge polarity should be applied to the cell. The analysis module can then control the charging circuit 2223 to apply the necessary charge to the cell droplet via the ring 2222. The cell then passes through the gap between a pair of deflection plates 2224 disposed downstream of the ring 2222. The voltage source 2225 applies a voltage to at least one of the plates 2224 to establish an electric field between the plates. The electric field can deflect the paths of the positive cell droplet and the negative cell droplet in different directions so that they can be collected by the tubes 2226b and 2226c of the cell collector 2226, respectively. Cells that are not charged by the ring 2222 are not deflected and are captured by the tube 2226a of the cell collector.

[0192] The analysis / control modules discussed herein, such as analysis / control module 2220, can be implemented in various different ways in hardware, firmware, and / or software using techniques known in the art and in accordance with the present teachings. As an example, Fig.23An exemplary implementation 2300 of the analysis / control module 2220 is schematically depicted, which includes an analog-to-digital converter 2302 for receiving one or more fluorescence signals from the detection system 2218 and digitizing the signal to generate digitized fluorescence data. The analysis / control module 2300 can also include a processor 2304 for processing the fluorescence data in accordance with the present teachings to obtain a sorting decision about the cells in the interrogation state. The analysis / control module 2300 can also include a ROM (read-only memory) 2306, a RAM (random access memory) 2308, and a permanent memory 2310. A communication bus 2312 facilitates communication between the various components of the analysis module. The memory module can be used to store instructions for analyzing one or more fluorescence signals and analysis results. For example, in some embodiments, instructions for analyzing fluorescence data to obtain a sorting decision in accordance with the present teachings can be stored in the ROM 2306. The processor 2304 can use these instructions to operate on the digitized fluorescence data stored in the RAM 2308 to determine the sorting decision. In some embodiments, the processor can be implemented as an ASIC (application specific integrated circuit) configured to execute instructions according to the present teachings to operate on the fluorescence data to obtain a sorting decision. In this embodiment, the analysis / control module 2300 can also include a communication / control module 2314 for sending appropriate signals to the charging circuit based on the sorting decision to apply an appropriate charge to the cell under interrogation.

[0193] As discussed above, methods according to the present teachings operate on temporal fluorescence data to obtain sorting decisions for particles (e.g., cells) without forming a pixel-by-pixel fluorescence image of the cells. This in turn allows cells to be sorted with low latency (e.g., less than about 100 microseconds), which in turn allows cells to be sorted at high rates. For example, sorting methods according to the present teachings can allow cells to be sorted at a rate equal to or greater than 1000 cells per second, such as sorting cells at about 1000 to about 100,000 cells per second.

[0194] As described above, although not limited to any particular illumination or detection technique, in some embodiments, the cell sorting methods according to the present teachings can be effectively used in flow cytometer systems that employ frequency domain multiplexing to excite rows of pixels, each pixel being "tagged" with a unique radio frequency, which is generated, for example, by the beating of two frequency-shifted elementary beams. Using frequency domain multiplexing, it is possible to detect and read the fluorescent (or scattered) radiation from hundreds of pixels in a single image row using, for example, a single photomultiplier tube (PMT) for each fluorescent color or scattering direction. Since the excitation of each pixel in the image row is modulated at a unique beat frequency, the pixel rate is proportional to the total RF bandwidth of the system, which can provide shot noise limited sensitivity at pixel rates greater than 100 MHz. As discussed above, the sorting methods according to the present teachings can use image data encoded in a time-frequency format to perform sorting decisions without actually computing the image.

[0195] Furthermore, the above described system can be used to sort particles (eg, cells) based on scattered radiation emitted from the particles in response to illumination.The scattered radiation can be detected and analyzed to make a sorting decision, for example, in the manner discussed above.

[0196] As other examples, the following U.S. patents provide information related to sorting systems that can be modified according to the present teachings to practice the sorting methods and systems disclosed herein: U.S. Patent No. 3,380,584, entitled “Particle Separator,” U.S. Patent No. 9,034,259, entitled “Flow Cytometer and Flow Cytometry,” and U.S. Patent No. 7,417,734, entitled “System and Process for Sorting Biological Particles,” the entire contents of each of which are incorporated herein by reference.

[0197] As a further illustration, Appendix A of U.S. application 62 / 822,789, filed on March 22, 2019, which is the priority, the disclosure of which is incorporated herein by reference, provides additional information regarding various aspects of the present teachings.

[0198] It will be appreciated by those skilled in the art that various changes can be made without departing from the scope of the present teachings. In particular, the various characteristics, structures or features of the embodiments discussed above can be combined in appropriate ways. For example, a detection system discussed in conjunction with one embodiment can be used in another embodiment.

[0199] Particle characteristics

[0200] As discussed above, in some embodiments, two or more beamlets with radio frequency amplitude modulation illuminate the sample at spatially distinct locations. The interaction between the sample and each beamlet can produce at least one of an optical scattering, an optical transmission, and an optical fluorescence emission signal, each of which is amplitude modulated using the corresponding radio frequency of the beamlet. The collected signal can be expressed as the sum of the contribution of each modulated beamlet:

[0201] S(t)=∑ i P i (t)·(1+A m cos(ω i t+φ i )) Equation (3)

[0202] Where S(t) represents the collected signal, P i (t) represents the time-varying scattered, transmitted or fluorescent emission signal associated with the ith beamlet, A m represents the modulation depth of the beamlet, and ω i and φ i Represent the angular frequency and phase of the radio frequency modulation of the beamlet, respectively. By assigning each beamlet to a different column of the image and each time instant to a different row of the image, an image representation of the particle can be derived. This image representation is connected to the collected signal via Fourier transform:

[0203] Im(x, y)=R·W·F·S(t) Equation (4)

[0204] Where F is the matrix that implements the short-time Fourier transform, W is the matrix that maps the Fourier components to image pixels, and R is the matrix that performs any desired linear image domain post-processing (such as filtering, background subtraction, and vignetting correction). Any linear property of the particle can be represented by matrix multiplication on the image. Therefore, for the matrix M to calculate any desired linear property, the property can be calculated directly from the collected signal via the following formula, i.e., without first calculating the image:

[0205] M·Im(x,y)=M·R·W·F·S(t)=Q·S(t) Equation (5)

[0206] Q=M·R·W·F Equation (6)

[0207] Where Q is a matrix representing the transformation from the current particle representation to the desired linear characteristic. Thus, any linear characteristic can be calculated by first calculating the matrix Q (e.g., offline in a preprocessing step), and then performing a dot product as indicated in equation (5) (e.g., online) to extract the desired characteristic. In many embodiments, for all characteristics, it is desirable to also subtract the contribution of the background signal to the characteristic. The process can be summarized as follows:

[0208] Calculation: Q = M·R·W·F

[0209] Calculation: Q bkg =M·R·W bkg ·F

[0210] When collecting data:

[0211] If particles are detected:

[0212] Calculation: D = Q S (t)

[0213] otherwise:

[0214] Calculation: D bkg (i) = Q bkg ·S(t)

[0215] Calculation: D bkg =mean(D bkg (i))

[0216] Generate: DD bkg

[0217] As discussed above, in some embodiments, one or more computed characteristics of a particle can be employed to make a sorting decision about the particle. As another example, Fig.24 A flow chart describing the sorting method is shown. At block (1), the above Q and Q_ can be calculated based on the desired particle properties (characteristics). bkg matrix. Subsequently, scattered, transmitted or emitted optical signals from the flow cell of the flow cytometer, illuminated, for example, with radio frequency modulated optical radiation, can be collected and digitized (block 2). The signal can be compared to a threshold crossing to determine whether the signal is from a particle, i.e., whether a particle is present (block 3). If it is determined that a particle is present, each digitized sample signal is multiplied by Q (block 4) and the result is then integrated into a single value (block 5). The background signal is subtracted from the integrated value (block 6) to obtain an estimate of the desired property (feature) of the particle. A sorting decision can be made by comparing the estimate of the property to a threshold (block 7). Continue with reference Fig.24 Flow chart of the invention, which can multiply one or more digitized signals obtained in the absence of particles by Q_ bkg (Block 8), and multiple background measurements can be averaged to obtain the background estimate employed in block (6).

[0218] In some embodiments, various moments of the received signal can be used to obtain information about the spatial distribution of scattered, transmitted or emitted radiation. Image moments are a class of linear image properties that represent weighted sums of the distances of pixels from an arbitrary origin (raised to arbitrary powers):

[0219]

[0220] Each image moment encodes different information about the spatial distribution of scattered, transmitted or emitted light. Higher-order moments weight pixels more heavily depending on their distance from the origin, with each moment providing different information about the signal distribution. In various embodiments, all of these features can be directly computed from the measured signal S(t) by pre-computing the appropriate Q.

[0221] The spatial Fourier components can be calculated in the same way, where M is defined as follows:

[0222] M(u,v)=exp(-j2π(ux+vy)) Equation (8)

[0223] The desired components can be selected prior to the experiment, and a separate Q can be pre-calculated for each component. For example, the Fourier components can be used to determine whether a particle is in the focus of the illuminating radiation.

[0224] Some image features involve nonlinear transformations of the data prior to feature extraction. For a subset of such features, they can be expressed as nonlinear combinations of linear features. Computing such features can involve first computing multiple features in parallel as in the previous section, and then combining their results in a nonlinear manner. There are many useful image features that can be expressed by nonlinear combinations of image moments. For example, the centroid of a pixel can be computed by the ratio of the first-order moment to the zero-order moment:

[0225]

[0226] A number of other particle characteristics can be represented in this way. Table 1 below is a non-exhaustive list.

[0227] Table 1

[0228]

[0229] In some embodiments, the central second order moment can be used to distinguish double peaks. Particles with high second order moments in the scattered or transmitted light signal indicate a larger signal distribution.

[0230] Colocalization and similarity

[0231] In some embodiments, the similarity between a particle and a reference can be calculated in the same manner as colocalization. In the case of colocalization, the two waveforms correspond to different detectors observing the same particle; in the case of similarity, the two waveforms correspond to the same detector observing two different particles. An exemplary algorithm for detecting similarity between a particle and a reference is summarized below:

[0232] Let R represent the waveform corresponding to the reference particle, let Filt be a high-pass or band-pass filter that passes only the modulation frequency, and let N be the number of pixels in the image representation of the reference particle. The algorithm contains:

[0233] calculate

[0234] Calculate R′=Filt(R)

[0235] Calculate R2 = ||R′|| 2 ,

[0236] For each incoming waveform S corresponding to the particle under investigation:

[0237] calculate

[0238] Calculate S′=Filt(S)

[0239] Calculate D = S'·R'

[0240] Calculate S2 = ||S′|| 2

[0241] As a measure of the similarity of a particle to a reference particle, returns

[0242] Spectral unmixing

[0243] Disclosed herein are systems, devices, and methods for compensating or spectrally unmixing images generated from FIRE-enabled imaging flow cytometers. Detecting fluorescence radiation emitted by one or more fluorophores using radio frequency multiplexing excitation (FIRE) after being excited by a combined beam 49 can be referred to as fluorescence imaging. Fluorescence radiation, emission, signal, or waveform detected by a detector configured to (e.g., using a bandpass filter) detect fluorescence radiation of a fluorophore or dye can include fluorescence emission of another fluorophore because the fluorescence emission spectra of the two fluorophores may overlap spectrally. Spectral unmixing can include phase-sensitive analysis based on fluorescence lifetime measurement. For example, spectral unmixing can be implemented on a field programmable gate array (FPGA) for cell sorting applications.

[0244] When the fluorophores have different lifetimes, the fluorescence signals of the waveform have different phases. Fluorescence signals with different phases do not add in a direct manner in real space, but rather in complex space (with real and imaginary components). In some embodiments, spectral unmixing can include analyzing the fluorescence signal in complex space, for example, for different phases of the fluorophores. Spectral unmixing can include analyzing the fluorescence signal in complex space to account for various system effects of optics and electronics inherent in the FIRE system (e.g., on a per-channel basis). The detection system of the flow cytometer can include multiple photodetectors to detect the fluorescence signal and generate multiple channels of fluorescence signals, as described with reference to FIG. Figure 7A-7B If spectral unmixing is applied to a FIRE image generated from the fluorescence signal of each channel (e.g., a FIRE image generated from the fluorescence signal detected by a photodetector) on a per-pixel basis, or if spectral unmixing is not performed, the image can include artifacts resulting from differences in phase contributions of different fluorophores when the fluorophores have different fluorescence lifetimes.

[0245] Fig.25 2500. At action 2504, fluorescence signals of multiple channels detected by multiple photodetectors can be received. For example, antibodies associated with (e.g., bound and conjugated to) different fluorophores can be bound to cells. Fluorescence signals of fluorophores can be detected by multiple photodetectors. For example, one photodetector can be configured to detect fluorescence signals for each fluorophore. Fig.25 Two graphs 2506p11, 2506p2 of FIRE waveforms 2506c1, 2506c2 of two channels are shown. The time dimension of waveforms 2506c1, 2506c2 is shown on the x-axis of each graph 2506p11, 2506p2. The intensity of FIRE waveforms 2506c1, 2506c2 is shown on the y-axis of each graph 2506p1, 2506p2. For example, since each fluorophore overlaps the spectrum of multiple channels, each FIRE waveform 2506c1, 2506c2 can contain signals from multiple fluorophores in each channel. Spectral overlap can cause images (e.g., images 2510i1, 2510i2) to have contributions from fluorophores that are different from those required or of interest to the channel. Compensation can be performed on the image so that the resulting images (e.g., images 2522i1, 2522i2) are each (or primarily) the result of a single fluorophore. The detected waveforms 2506c1, 2506c2 are sensitive to the phase of the fluorophore.

[0246] At action 2508, a Fourier transform (FT) (such as a sliding window fast Fourier transform (FFT) or a discrete Fourier transform (DFT)) can generate raw FT images 2510i1, 2510i2 for each channel. Each raw image 2510i1, 2510i2 can be an array (e.g., a two-dimensional array) of complex-valued pixels 2510i1p00, 2510i1p01, 2510i2p00, 2510i2p01 in frequency space (e.g., in Fourier space). The sliding windows 2510w1, 2510w2 can overlap or not overlap as shown. Imaging can be accomplished by performing complex Fourier analysis on the detected waveforms 2506c1, 2506c2. The generated images 2510i1, 2510i2 can have complex values ​​and can be phase sensitive (e.g., sensitive to the phase of the fluorophore). One dimension of the generated images 2510i1, 2510i2 can correspond to different frequency bins of an FFT or DFT. Another dimension of the generated images 2510i1, 2510i2 can correspond to the time dimension of a sliding window FFT or DFT.

[0247] At action 2512, an intrinsic phase adjustment can be performed. In some embodiments, the phase adjustment can minimize (e.g., eliminate) phase contributions from optics and electronics of different channels. In order to perform compensation or unmixing in image space, the phase contribution of the differences in optics and electronics between channels can be first caused by, for example, multiplying the original FT image by the channel phase adjustment. The resulting set of intrinsic phase correction images 2514i1, 2514i2 can be complex values. A set of signal phase corrections (for correcting signal phase) specific to each readout channel of the flow cytometer can be used for intrinsic phase adjustment. Channel phase can incorporate or include bright field phase, calibration phase, and phase offset. Channel phase adjustment can correct or cause one or more channel phases.

[0248] At action 2516, an unmixing operation can be performed for each pixel 2514i1p00, 2514i1p01, 2514i2p00, 2514i2p01 of the image 2514i1, 2514i2 using an unmixing matrix 2516m to generate an unmixed image 2518i1, 2518i2. Compensation or unmixing can be performed for corresponding pixels of the intrinsic phase-corrected image 2514i1, 2514i2 (such as pixel 2514i1p00, 2514i2p00 and pixel 2514i1p01, 2514i2p01). For example, for the two-channel image 2514i1, 2514i2, the size of the unmixing matrix 2516m can be 2×2. The unmixing matrix 2516m can be applied to corresponding pixels in the images 2514i1, 2514i2 (such as pixels 2514i1p00, 2514i2p00 and pixels 2514i1p01, 2514i2p01) to generate values ​​of corresponding pixels of the unmixed images 2518i1, 2518i2 (such as pixels 2518i1p00, 2518i2p00 and pixels 2518i1p01, 2518i2p01). The unmixing operation on the corresponding pixels can be performed sequentially or in parallel. The unmixed images 2518i1, 2518i2 can be considered to be images of the fluorophores (or images generated based on the fluorescence emission of the fluorophores) without any spectral mixing (or with minimal or reduced spectral mixing).

[0249] The unmixing operation can depend on the type of unmixing performed. For example, the unmixing operation can be based on compensation of area values. Compensation or unmixing can be performed based on area or height data of complex image values. The unmixing matrix 2516m can be derived manually or using any automatic compensation or unmixing program. For each carrier frequency (e.g., the frequency of a reference laser beam or a local oscillator beam used to induce fluorescence emission of each fluorophore), the unmixing matrix 2516m can account for differences in signal intensity due to different fluorescence lifetimes of the fluorophores.

[0250] At act 2520, lifetime phase correction can be performed. After performing unmixing on each pixel of the image, lifetime correction can be applied for each fluorophore to generate a lifetime corrected image having pixels with complex values. This adjustment can correct for phase changes due to the lifetime of the fluorescent dye. A set of "lifetime corrections" can be used for each fluorophore. These can be derived from the fluorescence lifetime of each fluorophore, for example using known values ​​or calculations based on collected sample data.

[0251] In some embodiments, the real part of the complex number for each pixel can be used to generate the value of the pixel 2522i1p00, 2522i1p01, 2522i2p00, 2522i2p01 of the final image 2522i1, 2522i2 for each fluorophore. The real part or magnitude of the complex value for each pixel of the lifetime corrected image can be used to generate the final image 2522i1, 2522i2 of the fluorophore in real number space. The imaginary part of the complex value for each pixel of the lifetime corrected image can be discarded, or not calculated. The images of different fluorophores can be combined to generate an image of a cell, for example, by assigning different colors to the images of different fluorophores and combining the different images with colors to generate an image of the cell.

[0252] In some embodiments, the actions of method 2500 include linear operations as a whole, so multiple actions can be merged. The method can be implemented on an FPGA for sorting applications. In some embodiments, some actions of the method can only include linear operations that can be merged.

[0253] In some embodiments, method 2500 can be implemented in the case of cross-excitation mixed with fluorophores excited from conventional (or non-imaging) lasers. In the case of cross-excitation, the total amount of mixing can be measured. Image data from fluorophores excited from conventional lasers can be used to determine the spatial distribution of cross-excited fluorophores. Additionally or alternatively, with prior knowledge of relevant cell surface markers, one of the FIRE channels containing the appropriate spatial distribution can be used as a substitute for one or more cross-excited fluorophores. For example, if a cross-excited fluorophore is attached to a surface marker, it can be assumed that the spatial distribution of the fluorophore is the same as the spatial distribution of another surface marker of the fluorophore excited from one of the FIRE channels.

[0254] Figures 26A-26B is a composite image showing a non-limiting exemplary result of spectral unmixing. Fig.26A is a composite image of FIRE images of CD3+ T cells stained with anti-CD16 / CD56 antibodies conjugated to phycoerythrin (PE), anti-CD3 antibodies conjugated to PE-Texas Red, anti-DRAQ5 antibodies conjugated to PE-Cy5, and anti-CD4 antibodies conjugated to fluorescein isothiocyanate (FITC), without performing spectral unmixing. The cells shown are all CD16 / CD56 negative, but these cells are Fig.26A This is due to the spillover of these cells from the FITC-labeled anti-CD4 antibody into the PE channel. Fig.26B After spectral unmixing Fig.26A Composite image of FIRE images of CD3+ T cells shown in . Fig.26BIt is shown that the spectral unmixing method disclosed herein successfully eliminates any trace of this spillover in the final image of the cells.

[0255] Spectral unmixing method

[0256] Embodiments of a cell sorter system are disclosed herein. In some embodiments, the cell sorter system includes: a light source (e.g., a laser light source) configured to generate a light beam (e.g., a laser beam) having a plurality of different optical frequencies for inducing fluorescence emission of a plurality of fluorophores at different wavelengths, the plurality of fluorophores being associated with cells in a sample including a plurality of cells. The cell sorter system can include: a plurality of photodetectors configured to detect fluorescence emission of a plurality of fluorophores (e.g., each photodetector is configured to detect fluorescence emission of a fluorophore in the plurality of fluorophores). The cell sorter system can include: a non-transitory memory configured to store executable instructions; and a processor communicating with the non-transitory memory, the hardware processor being programmed by executable instructions to: cause a laser beam to irradiate a cell to induce fluorescence emission of a plurality of fluorophores associated with the cell. The computing system can cause a plurality of photodetectors to detect fluorescence emission at different wavelengths to obtain fluorescence intensity data of a plurality of channels corresponding to a plurality of fluorophores in time space. The computing system can generate a plurality of raw images of a plurality of pixels including complex values ​​in frequency space from the fluorescence intensity data of a plurality of channels. The computing system can generate a plurality of phase adjusted images including a plurality of pixels having complex values ​​in frequency space from a plurality of raw images using one or more channel phase adjustments for each of a plurality of channels of fluorescence intensity data. The computing system can generate a plurality of unmixed images including a plurality of pixels having complex values ​​in frequency space from the plurality of phase adjusted images using an unmixing matrix on complex values ​​of corresponding pixels in the plurality of phase adjusted images. The computing system can generate a plurality of phase corrected images including a plurality of pixels having complex values ​​in frequency space from the plurality of unmixed images based on properties of each of a plurality of fluorophores. The computing system can determine a sorting decision for a cell based on the plurality of phase corrected images.

[0257] The content disclosed herein includes embodiments of a cell sorter system. In some embodiments, the cell sorter system includes: a light source (e.g., a laser light source) configured to generate a light beam (e.g., a laser beam) having a plurality of different optical frequencies for inducing fluorescence emission of a plurality of fluorophores at different wavelengths, the plurality of fluorophores being associated with cells in a sample including a plurality of cells; a plurality of photodetectors configured to detect fluorescence emission of a plurality of fluorophores (e.g., each photodetector is configured to detect fluorescence emission of a fluorophore in the plurality of fluorophores); a non-transitory memory configured to store executable instructions; and a processor communicating with the non-transitory memory, the hardware processor being programmed by the executable instructions to: cause the laser beam to illuminate the cell to induce fluorescence emission of a plurality of fluorophores associated with the cell. The computing system is capable of causing a plurality of photodetectors to detect fluorescence emission at different wavelengths to obtain fluorescence intensity data of a plurality of channels corresponding to a plurality of fluorophores in time space. The computing system is capable of generating a plurality of raw images including complex values ​​in frequency space from the fluorescence intensity data of the plurality of channels. The computing system can use an unmixing matrix on corresponding complex values ​​in the plurality of original images to generate a plurality of unmixed images including complex values ​​in frequency space from the plurality of original images. The computing system can determine a sorting decision for the cell based on the plurality of unmixed images.

[0258] Embodiments of a cell sorter system are disclosed herein. In some embodiments, the cell sorter system includes: a light source (e.g., a laser light source) configured to generate a light beam (e.g., a laser beam) having a plurality of different optical frequencies for inducing fluorescence emission of a plurality of fluorophores at different wavelengths, the plurality of fluorophores being associated with cells in a sample including a plurality of cells; a plurality of photodetectors configured to detect fluorescence emission of a plurality of fluorophores (e.g., each photodetector is configured to detect fluorescence emission of a fluorophore in the plurality of fluorophores); a non-transitory memory configured to store executable instructions; and a processor communicating with the non-transitory memory, the hardware processor being programmed by the executable instructions to: receive fluorescence intensity data of a plurality of channels of fluorescence emission at different wavelengths from a plurality of fluorophores associated with the cells, the fluorescence intensity data of the plurality of channels corresponding to a plurality of fluorophores in time space, wherein the fluorescence emission is detected by a plurality of photodetectors, and wherein the fluorescence emission is induced after the cells are irradiated by the laser light source. The computing system is capable of generating a plurality of raw images including complex values ​​in frequency space from the plurality of channels of fluorescence intensity data. The computing system can use an unmixing matrix on corresponding complex values ​​in the plurality of original images to generate a plurality of unmixed images including complex values ​​in frequency space from the plurality of original images. The computing system can determine a sorting decision for the cell based on the plurality of unmixed images.

[0259] Fig. 272700 is a flow chart illustrating an exemplary method 2700 for spectral unmixing. The method 2700 may be implemented in a set of executable program instructions stored on a computer-readable medium (such as one or more disk drives) of a computer system. For example, Fig.30 The computing system 3000 shown in FIG. 2 and described in more detail below can execute a set of executable program instructions to implement the method 2700. When the method 2700 is started, the executable program instructions can be loaded into a memory such as a RAM and executed by one or more processors (e.g., one or more field programmable gate arrays (FPGAs)) of the computing system 3000. Although the method 2700 is relatively simple, the method 2700 can be executed by the computer system 3000. Fig.30 , but the description is illustrative only and is not intended to be limiting. In some embodiments, method 2700 or portions thereof can be performed serially or in parallel by multiple computing systems. Some blocks of method 2700 can be combined and performed in parallel. Method 2700 can be used to determine a sorting decision for cells of a sample, or to determine a sorting decision for cells of a sample.

[0260] The computing system 3000 can be operably coupled to a flow cytometer (e.g., see Fig. 22 The computing system 3000 can be part of a cell sorter system (such as a flow cytometer). The computing system 3000 can control a light source (e.g., a laser light source). The light source can be configured to generate light beams (e.g., laser beams, such as combined beam 49) having a plurality of different optical frequencies. The different optical frequencies can be used to induce fluorescence emission (e.g., at different wavelengths) of a plurality of fluorophores associated with cells of the sample. The computing system 3000 can receive signals from a plurality of photodetectors (e.g., reference beams 49). Figure 7A-7B Each photodetector can be configured to detect fluorescence emission of a fluorophore from a plurality of fluorophores.

[0261] After method 2700 starts at block 2704, method 2700 proceeds to block 2708, where the computing system can receive fluorescence intensity data for multiple channels of fluorescence emission (e.g., at different wavelengths) from multiple fluorophores associated with the cell. For example, each of the multiple fluorophores can be associated (e.g., conjugated) with a cellular component binding reagent (e.g., an antibody) that binds to the cell. The multiple channels of fluorescence intensity data (e.g., with reference to Fig.25The waveforms 2506c1, 2506c2) described can correspond to multiple fluorophores in time space. For example, the fluorescence data for each channel can include intensity values ​​detected by a photodetector over a period of time. Fluorescence emission can be detected by multiple photodetectors. For example, referring to Figure 7A-7B The photodetectors 106, 108, 110, 112 described below can detect and generate channels of fluorescence intensity data. After the cells are illuminated by the laser source, fluorescence emission can be induced.

[0262] In some embodiments, to receive fluorescence intensity data of multiple channels, the computing system can cause a laser beam to illuminate a cell to induce fluorescence emission of multiple fluorophores associated with the cell. The computing system can cause multiple photodetectors to detect fluorescence emission at different wavelengths to obtain fluorescence intensity data of multiple channels corresponding to multiple fluorophores in time and space.

[0263] From block 2708, method 2700 proceeds to block 2712, where the computing system can generate a plurality of pixels (e.g., reference pixels) comprising complex values ​​in frequency space (e.g., Fourier space) from the fluorescence intensity data of the plurality of channels. Fig.25 The multiple original images (e.g., referring to the pixels 2510i1p00, 2510i1p01, 2510i2p00, 2510i2p01) described above Fig.25 The computing system can generate a plurality of original images using a time-frequency transformation matrix representing a time-frequency spatial transformation. The plurality of original images can include a plurality of pixels of complex values ​​of the plurality of original images.

[0264] In some embodiments, the laser beam includes a reference laser beam (e.g., an LO beam) and a plurality of RF frequency-shifted laser beams (e.g., an RF comb beam 24). The number of the plurality of RF frequency-shifted laser beams can be m2. The reference laser beam can overlap with the plurality of RF frequency-shifted laser beams in space. In some embodiments, the plurality of RF frequency-shifted laser beams do not overlap with each other in space. The reference laser beam and one or more of the plurality of RF frequency-shifted laser beams can induce fluorescence emission from fluorophores in the plurality of fluorophores. Irradiating the cell can include simultaneously irradiating a plurality of spatial locations (or reference locations) of the cell with the plurality of RF frequency-shifted laser beams. Figure 4 In some embodiments, the plurality of spatial locations of the cells do not overlap with each other.

[0265] In some embodiments, the fluorescence emissions of two of the plurality of fluorophores overlap spectrally. The fluorescence emissions of three of the plurality of fluorophores overlap spectrally. The fluorescence emissions of five of the plurality of fluorophores overlap spectrally. The fluorescence emissions of a first fluorophore and a second fluorophore of the plurality of fluorophores overlap spectrally, and the fluorescence emissions of a second fluorophore and a third fluorophore of the plurality of fluorophores overlap spectrally. The fluorescence emissions of the first fluorophore and the third fluorophore overlap spectrally.

[0266] In some embodiments, to generate multiple raw images, the computing system can use a time-frequency space transform to generate multiple raw images in frequency space from the fluorescence intensity data of multiple channels. The time-frequency space transform can include a Fourier transform. The Fourier transform can include a discrete Fourier transform. The discrete Fourier transform can include a fast Fourier transform. The Fourier transform can include a sliding window Fourier transform. The size of the sliding window of the sliding window Fourier transform can be m1. For example, the size of the sliding window can be, about, at least, or at most m1 data points (e.g., m1 data points of fluorescence intensity data of one channel, such as m1 consecutive data points), such as 5, 10, 20, 50, 100, 2000, 5000, 10000, 2000, 5000, 10000, 2000, 50000, 100000, 200000, 500000, 1000000, 2000000, 5000000, 10000000 data points, or a number or range between any two of these values. As another example, the size of the sliding window can be, about, at least, or at most m1 seconds, such as 1×10 -1 , 2×10 -1 , 5×10 -1 , 1×10 -2 , 2×10 -2 , 5×10 -2 , 1×10 -3 , 2×10 -3 , 5×10 -3 , 1×10 -4 , 2×10 -4 , 5×10 -4 , 1×10 -5 , 2×10 -5 , 5×10 -5 , 1×10 -6 , 2×10 -6 , 5×10 -6 , 1×10 -7 , 2×10 -7 , 5×10 -7 , 1×10 -8 , 2×10-8 , 5×10 -8 , 1×10 -9 , 2×10 -9 , 5×10 -9 In some embodiments, the sliding windows of the sliding window Fourier transform can be 5, 10, 20, 50, 100, 200, 500, 1000, 2000, 5000, 10000 or more. In some embodiments, none of the sliding windows overlap. In some embodiments, two or more sliding windows overlap. For example, the two sliding windows can overlap, overlap about, overlap at least, or overlap at most 5, 10, 20, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 2000, 5000, 10000, 20000, 50000, 100000, 200000, 500000, 1000000 data points, or a number or range between any two of these values. As another example, the two sliding windows can overlap, overlap about, overlap at least, or overlap at most 1×10 -1 , 2×10 -1 , 5×10 -1 , 1×10 -2 , 2×10 -2 , 5×10 -2 , 1×10 -3 , 2×10 -3 , 5×10 -3 , 1×10 -4 , 2×10 -4 , 5×10 -4 , 1×10 -5 , 2×10 -5 , 5×10 -5 , 1×10 -6 , 2×10 -6 , 5×10 -6 , 1×10 -7 , 2×10 -7 , 5×10 -7 , 1×10 -8 , 2×10 -8 , 5×10 -8 , 1×10 -9 , 2×10 -9 , 5×10 -9seconds, or a number or range of seconds between any two of these values. For example, the two sliding windows can overlap, overlap about, overlap at least, or overlap by at most 1%, 2%, 3%, 4%, 5%, 6%, 7%, 8%, 9%, 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, 90%, 95%, 99%, 99.9% of the width of one of the sliding windows.

[0267] After generating the plurality of raw images at block 2712, the method 2700 proceeds to block 2716, where the computing system can generate a plurality of pixels (e.g., referenced to FIG. 2700 ) comprising complex values ​​in frequency space from the plurality of raw images using one or more channel phase adjustments for each of the plurality of channels of fluorescence intensity data. Fig.25 The plurality of phase-adjusted images of pixels 2514i1p00, 2514i1p01, 2514i2p00, 2514i2p01) described above (e.g., referring to Fig.25 The computing system can generate a plurality of phase adjustment images using one or more channel phase adjustment matrices representing one or more channel phase adjustments.

[0268] In some embodiments, the computing system can generate a plurality of phase-adjusted images comprising complex values ​​in frequency space from the plurality of raw images using one or more channel phase adjustments for each of the plurality of channels of fluorescence intensity data. The plurality of phase-adjusted images can include a plurality of pixels of the plurality of complex values ​​of the phase-adjusted images.

[0269] In some embodiments, one or more channel phase adjustment corrections include a channel phase (e.g., a channel-specific phase) of a bright field phase, a calibration phase, and a phase offset for each of a plurality of channels. To generate a plurality of phase adjustment images, the computing system can, for each of a plurality of channels: multiply a complex value of an original image in a plurality of original images corresponding to the channel by one or more channel phase adjustments to generate a phase adjustment image in a plurality of phase adjustment images corresponding to the channel. To generate a plurality of phase adjustment images, the computing system can, for each of a plurality of original images: multiply a complex value of an original image by one or more channel phase adjustments for a channel corresponding to the original image to generate a phase adjustment image in a plurality of phase adjustment images.

[0270] One or more channel phase adjustments can include or only include complex values. One or more channel phase adjustments can include one or more real values. The channel phase matrix of a channel can include one or more channel phase adjustments for the channel. The size of the channel phase adjustment matrix can be n1×n1. In some embodiments, the channel phase adjustment matrix can be a diagonal matrix. For example, only the diagonal elements of the channel phase adjustment matrix are non-zero. The diagonal elements of the channel phase adjustment matrix can be complex values, or can all be complex values. In order to generate multiple phase adjustment images, the computing system can, for each channel in the multiple channels: multiply the complex values ​​of each row of the original image in the multiple original images corresponding to the channel by the channel phase adjustment matrix for the channel to generate the corresponding row of the phase adjustment image in the multiple phase adjustment images corresponding to the channel. In order to generate multiple phase adjustment images, the computing system can, for each original image in the multiple original images: multiply the complex values ​​of each row of the original image by the channel matrix for the channel to generate the corresponding row of the phase adjustment image in the multiple phase adjustment images. A channel phase adjustment matrix used to correct one or more channel phases for each channel can be unique to the channel.The computing system can determine or receive one or more channel phase adjustments.

[0271] Appendix A of U.S. Application No. 62 / 822,789 filed on March 22, 2019, which is the priority, shows a non-limiting exemplary method for performing phase calibration (e.g., for generating a phase-adjusted image), and the disclosure of which is incorporated herein by reference. Appendix B of U.S. Application No. 62 / 822,789 filed on March 22, 2019, which is the priority, shows a non-limiting exemplary method for measuring phase at many frequencies simultaneously, and the disclosure of which is incorporated herein by reference.

[0272] From block 2716, method 2700 proceeds to block 2720, where the computing system can adjust corresponding pixels of the plurality of phase images (e.g., with reference to Fig.25 The unmixing matrix (e.g., referring to Fig.25 The unmixing matrix 2516m described above is used to generate a plurality of pixels including complex values ​​in the frequency space from the plurality of phase-adjusted images (e.g., referring to Fig.25 The multiple unmixed images (eg, referring to the pixels 2518i1p00, 2518i2p00 and pixels 2518i1p01, 2518i2p01) described above Fig.25The unmixed images 2518i1, 2518i2 are described. Each unmixed image can be considered an image of a fluorophore (or an image generated from the fluorescence emission of a fluorophore) without any spectral mixing (or with minimal or reduced spectral mixing) of the fluorescence emission from other fluorophores. To generate the multiple unmixed images, the computing system can use an unmixing matrix on the corresponding complex values ​​of the multiple phase adjustment images to generate the multiple unmixed images from the multiple phase adjustment images.

[0273] In some embodiments, to generate the plurality of unmixed images, the computing system can generate a plurality of phase-adjusted images including corresponding pixels (e.g., referring to Fig.25 The computing system is able to compare this vector with the unmixing matrix (e.g., reference Fig.25 The unmixing matrix 2516m described above is multiplied to generate a matrix including unmixed complex values ​​(e.g., reference Fig.25 The computing system can generate a plurality of unmixed images including corresponding pixels having unmixed complex values. The size of the vector can be 1×n1, the size of the unmixing matrix can be n1×n2, and / or the size of the unmixing vector can be 1×n2. For example, n1 and / or n2 can be 1, 2, 3, 4, 5, 6, 2, 3, 4, 5, 6, 7, 8, 9, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 200, 500, 1000, or a number or range between any two of these values. In order to generate multiple unmixed images, the computing system can use an unmixing matrix in a plurality of unmixing matrices on the complex values ​​of the corresponding pixels in the plurality of phase adjustment images to generate multiple unmixed images from the plurality of phase adjustment images. The plurality of unmixing matrices can include m2 unmixing matrices, such as 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 200, 500, 1000 unmixing matrices, or a number or range of unmixing matrices between any two of these values. The unmixing matrix can include or only include real values ​​and / or complex values.

[0274] In some embodiments, (1) the number of multiple photodetectors, the number of multiple original images, and / or the number of multiple phase adjustment images (e.g., n1), and (2) the number of multiple fluorophores, the number of multiple unmixed images, and / or the number of phase correction images (e.g., n2) can be the same. In some embodiments, (1) the number of multiple photodetectors, the number of multiple original images, and / or the number of multiple phase adjustment images (e.g., n1), and (2) the number of multiple fluorophores, the number of multiple unmixed images, and / or the number of phase correction images (e.g., n2) can be different. For example, the number of multiple phase adjustment images (e.g., n1) can be less than the number of multiple unmixed images (e.g., n2). As another example, the number of multiple phase adjustment images (e.g., n1) can be greater than the number of multiple unmixed images (e.g., n2). For example, each of the multiple phase adjustment images can be generated from fluorescence intensity data detected by different photodetectors among the multiple photodetectors or fluorescence intensity data received from different photodetectors among the multiple photodetectors (e.g., after transformation to frequency space and / or phase adjustment). The number of multiple original images, the number of multiple phase-adjusted images, and the number of multiple photodetectors can be the same (e.g., n1). Fluorescence intensity data of multiple fluorophores can be detected by or using multiple photodetectors. Each of the multiple phase-corrected images (and / or each of the multiple unmixed images) can correspond to one of the multiple fluorophores. The number of multiple unmixed images, the number of multiple phase-corrected images, and the number of multiple fluorophores can be the same (e.g., n2). More photodetectors can be used to detect the fluorescence emission of fewer fluorophores. For example, 10 photodetectors can be used to generate 10 channels of fluorescence intensity data of the fluorescence emission of eight fluorophores. Each of the multiple unmixed images can correspond to one of the multiple fluorophores. Each of the multiple unmixed images can be an image of one of the multiple fluorophores.

[0275] Square unmixing matrix Compensation can be performed where the number of channels is equal to the number of fluorophores (eg, for regional data). The unmixing matrix can be calculated once and used for all events and all pixels in each event. Fig.28A Figure 1 is a schematic diagram of spectral unmixing using a square unmixing matrix. The unmixing matrix is ​​also called the spillover matrix. Fig.28A It is shown that if the number of channels (shown as three, collected by detectors D1, D2, and D3) and the number of fluorophores (shown as three, labeled FL1, FL2, and FL3) are the same, then the square unmixing matrix (or the inverse of the square mixing matrix) M -1 can be used to determine the unmixed value F from the detected value D. Fig.28AIn the example shown in , the fluorescence emission detected by detector D1 includes 100% of the fluorescence emission of fluorophore FL1 (e.g., the fluorescence emission of FL1 detected by D1 is normalized to 100%), 15% of the fluorescence emission of fluorophore FL2 (e.g., relative to the fluorescence emission of FL1 detected by D1 normalized to 100%), and 1% of the fluorescence emission of fluorophore FL3 (e.g., relative to the fluorescence emission of FL1 detected by D1 normalized to 100%). The fluorescence detected by detector D2 includes 100% of the fluorescence emission of fluorophore FL2, 12% of the fluorescence emission of fluorophore FL1, and 0% of the fluorescence emission of fluorophore FL3. The fluorescence detected by detector D3 includes 100% of fluorophore FL3, 1% of fluorophore FL1, and 20% of fluorophore FL2. The relationship between the fluorescence emission from the three fluorophores and the fluorescence emission detected by the three detectors can be expressed as D=M*F. M can be used. -1 To perform spectral unmixing. For example, it is possible to use M -1 , by determining M -1 *D, to determine the fluorescence emission of the three fluorophores F from the fluorescence emission D.

[0276] Non-square unmixing matrix . When we use more (or fewer) channels than fluorophores, we can perform spectral unmixing. Figure 28B1-28B2 Schematic diagram of using a non-square unmixing matrix to implement spectral unmixing and determine the non-square unmixing matrix. The number of photodetectors and the number of fluorophores can be different, and the non-square unmixing matrix can be used for spectral unmixing of fluorescence intensity data detected by the photodetectors. Figure 28B1 In the example shown in FIG. 27 , 12 photodetectors detect the fluorescence emission of three fluorophores to generate 12 channels of fluorescence intensity data. Each channel of fluorescence intensity data can include a plurality of pixels having a plurality of pixels after the time-frequency spatial transformation at block 2712 and after performing the reference Fig. 27 The value in the frequency space of the image (eg, the original image or the phase-adjusted image) after the phase adjustment at block 2716 described. For example, each of D1, ..., D12 in FIG. 28B can be a reference Fig. 27 For example, D1 and D2 can be reference pixels of each of the multiple original images or phase-adjusted images described. Fig.25 Pixels 2514i1p00, 2514i2p00 of the phase adjustment image of the two channels described. As another example, D1 and D2 can be reference Fig.25The pixels 2510i1p00, 2510i2p00, D1, ..., D12 of the raw image of the two channels described can correspond to 12 pixels from 12 channels corresponding to one frequency window or beat frequency. As another example, Figure 28B1 D1 in can include multiple pixels corresponding to different frequency bins (eg, of the beat frequency). Similarly, each of D2, ..., D12 can include multiple pixels corresponding to different frequency bins. The size of each of D1, ..., D12 can be m2×1. Figure 28B2 It is shown that if the number of channels and the number of fluorophores are different, the unmixing matrix can be determined using ordinary least squares (OLS) and / or weighted least squares (WLS) for spectral unmixing.

[0277] The computing system can receive and / or determine an unmixing matrix or multiple unmixing matrices (e.g., m2). In some embodiments, the unmixing matrix (or multiple unmixing matrices, one for each frequency bin or pixel, or one for two or more frequency bins or pixels) can be determined using ordinary least squares (OLS), weighted least squares (WLS), Poisson regression, or a combination thereof. Other methods can be used to determine the unmixing matrix.

[0278] Ordinary Least Squares In some embodiments, the unmixing matrix can be determined once using ordinary least squares (e.g., for area data) and used for spectral unmixing (e.g., as Figure 28B2 As shown to determine ). The unmixing matrix can be computed once using singular results for all events (and all pixels in each event). Determining the unmixing matrix using ordinary least squares and / or performing spectral unmixing using such an unmixing matrix can be computed quickly (e.g., based on the number of calculations required).

[0279] Weighted Least Squares . In some embodiments, the unmixing matrix can be determined using a weighted least squares method. The unmixing matrix determined based on the weighted least squares method can account for or correct Poisson noise (e.g., the variance of the pixel value can increase with the value). It may be necessary to determine the unmixing matrix (or multiple unmixing matrices for different frequency windows or pixels) via matrix inversion for each multiple corresponding pixels. The unmixing matrix can be determined via matrix inversion for each event (e.g., the value of each multiple corresponding pixel of different channels, or the values ​​of all corresponding pixels of different frequency windows of different channels of a corresponding sliding window from a sliding window Fourier transform). For example, the unmixing matrix (or Figure 28B2The matrix W in can be obtained by referring to Fig.25 The corresponding pixels 2514i1p01, 2514i2p01 of the phase-adjusted images in the two channels are subjected to matrix inversion to determine another unmixing matrix. Using the weighted least squares method to determine the unmixing matrix and / or using such an unmixing matrix to perform spectral unmixing can be more computationally demanding than using the ordinary least squares method to determine the unmixing matrix and / or using such an unmixing matrix to perform spectral unmixing.

[0280] Due to the use of a weighting matrix W, the computational overhead of WLS may be higher (e.g., much higher) than OLS. The unmixing matrix may need to be recalculated for each event. Calculating the mixing matrix requires matrix inversion, which is computationally expensive. The weights that go into the weight matrix can be based on an estimate of the variance of each pixel value (or each metric). The weight matrix can be different for each pixel (or each multiple corresponding pixels). The variance calculation can depend on the statistical information inherent in the FIRE measurement. For each pixel (or each multiple corresponding pixels) in each event, WLS can require a different and / or new unmixing matrix calculation.

[0281] Poisson regression . In some embodiments, the unmixing matrix (or multiple unmixing matrices for different frequency windows or pixels) can be determined using Poisson regression and / or a generalized linear model based on a Poisson noise model. For example, the unmixing matrix can be determined using maximum likelihood estimation (MLE), using optimization techniques such as convex optimization techniques. Using Poisson regression to determine the unmixing matrix can include event-by-event convex optimization solutions. The unmixing matrix determined using Poisson regression and / or using this unmixing matrix to perform spectral unmixing can correctly account for or correct noise (e.g., per-channel noise). In some embodiments, the unmixing matrix determined using Poisson regression can outperform the unmixing matrix determined using ordinary least squares or weighted least squares. Using Poisson regression to determine the unmixing matrix and / or using this unmixing matrix to perform spectral unmixing can be more computationally intensive than using weighted least squares to determine the unmixing matrix and / or using this unmixing matrix to perform spectral unmixing.

[0282] Reference Fig. 27After generating the plurality of unmixed images at block 2720, method 2700 proceeds to block 2724, where the computing system can generate a plurality of phase-corrected images from the plurality of unmixed images based on a characteristic of each of the plurality of fluorophores. The computing system can generate the plurality of phase-corrected images using a plurality of fluorophore phase correction matrices representing a plurality of fluorophore phase corrections for the fluorophores. Each phase-corrected image can include a plurality of pixels of complex values ​​in frequency space. The plurality of phase-corrected images can include a plurality of pixels of complex values ​​for the plurality of phase-corrected images. The computing system can generate a plurality of phase-corrected images including complex values ​​in frequency space from the plurality of unmixed images based on a property of each of the plurality of fluorophores. The computing system can determine or receive a plurality of fluorophore phase corrections for the fluorophores.

[0283] In some embodiments, each phase-corrected image can include a plurality of pixels of real value in frequency space. The plurality of phase-corrected images can include a plurality of pixels of real value of the plurality of phase-corrected images. The computing system can generate a plurality of phase-corrected images including real value in frequency space from the plurality of unmixed images based on a property of each of the plurality of fluorophores.

[0284] In some embodiments, to generate a plurality of phase-corrected images, the computing system can use a plurality of fluorophore phase corrections (e.g., time) for fluorophores in the plurality of fluorophores corresponding to the channels to generate a phase-corrected image in the plurality of phase-corrected images from an unmixed image in the plurality of unmixed images corresponding to the channels in the plurality of channels. In some embodiments, to generate a plurality of phase-corrected images, the computing system can use one or more fluorophore phase corrections (e.g., a set of lifetime corrections) for fluorophores in the plurality of fluorophores to generate a phase-corrected image in the plurality of phase-corrected images from an unmixed image in the plurality of unmixed images corresponding to the fluorophores in the plurality of fluorophores. The plurality of fluorophore phase corrections for the fluorophores can be related to properties of the fluorophores. The properties of the fluorophores can include lifetimes of the fluorophores. The plurality of fluorophore phase corrections for the fluorophores can include complex values.

[0285] In order to generate multiple phase-corrected images, the computing system can, for each unmixed image in the multiple unmixed images: multiply the complex values ​​of each row of the unmixed image by the fluorophore phase correction matrix for the fluorophore to generate a corresponding row of the phase-corrected image in the multiple phase-corrected images corresponding to the fluorophore. The fluorophore phase correction matrix for the fluorophore can include one or more fluorophore phase corrections for the fluorophore. The fluorophore phase correction matrix for the fluorophore can be unique to the fluorophore. The fluorophore phase correction matrix can include or only include one or more complex values. The fluorophore phase correction matrix can include one or more real values. The size of the fluorophore phase correction matrix can be n2×n2. In some embodiments, the fluorophore phase correction matrix can be a diagonal matrix. For example, only the diagonal elements of the fluorophore phase correction matrix are non-zero. The diagonal elements of the fluorophore phase correction matrix can be complex values, or can all be complex values. The fluorophore phase correction matrix can be unique for each fluorophore.

[0286] In some embodiments, each channel phase adjustment matrix (or channel phase adjustment) includes elements with complex values. Each channel phase adjustment matrix can include elements with only complex values. Each channel phase adjustment matrix can include one or more elements with real values ​​(e.g., the imaginary part of the complex value is zero). In some embodiments, each unmixing matrix includes elements with real values. Each unmixing matrix can include elements with only real values. In some embodiments, each fluorophore phase correction matrix includes elements with complex values. Each channel phase adjustment matrix can include elements with only complex values. Each channel phase adjustment matrix can include one or more elements with real values ​​(e.g., the imaginary part of the complex value is zero).

[0287] In some embodiments, each original image includes pixels having complex values. Each original image can include pixels having only complex values. Each original image can include one or more pixels having real values. In some embodiments, each phase-adjusted image includes pixels having complex values. Each phase-adjusted image can include pixels having only complex values. Each phase-adjusted image can include one or more pixels having real values. In some embodiments, each unmixed image includes pixels having complex values. Each unmixed image can include pixels having only complex values. Each unmixed image can include one or more pixels having real values.

[0288] In some embodiments, each phase-corrected image includes pixels having complex values. Each phase-corrected image can include pixels having only complex values. Each phase-corrected image can include one or more pixels having real values. Each phase-corrected image can include pixels having only real values. For example, a fluorophore phase correction matrix for a fluorophore can rotate the complex values ​​of the pixels of the unmixed image of the fluorophore to the axis of the real part so that the imaginary part of each pixel of the fluorophore phase-corrected image is zero. As another example, a fluorophore phase correction matrix for a fluorophore can rotate the complex values ​​of the pixels of the unmixed image of the fluorophore to the axis of the real part so that the imaginary part of each pixel of the fluorophore phase-corrected image is close to zero and is discarded (or not calculated at all).

[0289] Fig.29 Fluorophore phase correction is schematically shown. A pixel of the unmixed image of a fluorophore can be a complex value including a signal phase S and a noise phase N. The complex value of the pixel can be phase corrected. Based on the lifetime of the fluorophore, a fluorophore phase for the fluorophore (e.g., a fluorophore phase corresponding to the beat frequency of the pixel) can be determined and used for fluorophore phase correction. For example, the fluorophore phase can be (or be based on) It can be used to determine a fluorophore phase correction for the fluorophores.The fluorophore phase correction (eg, in the form of a fluorophore phase correction matrix) can be used to correct the fluorophore phases of the pixels of the unmixed image. Fig.29 The real part of the complex value of the pixel after phase correction is shown to be closer to the signal of the complex value of the pixel in the unmixed image than the amplitude of the complex value of the pixel. In some embodiments, a higher signal-to-noise ratio can be achieved by shifting (e.g., rotating using a fluorophore phase correction) the complex value of each pixel of the unmixed image corresponding to a fluorophore based on the known phase of the fluorophore and taking the real part of the complex value of the pixel after fluorophore phase correction.

[0290] Reference Fig. 27 In some embodiments, the number of the plurality of original images, the number of the plurality of phase adjustment images, the number of the plurality of unmixed images, and the number of the plurality of phase correction images are the same. The number of the plurality of original images, the number of the plurality of phase adjustment images, the number of the plurality of unmixed images, and / or the number of the plurality of phase correction images can be n1. The plurality of original images can be associated with a first time dimension and a first frequency dimension. The plurality of phase adjustment images can be associated with a second time dimension and a second frequency dimension. The plurality of unmixed images can be associated with a third time dimension and a third frequency dimension. The plurality of phase correction images can be associated with a fourth time dimension and a fourth frequency dimension.

[0291] Two or more of the first time dimension, the second time dimension, the third time dimension, and the fourth time dimension can have the same size and / or the same number of pixels along the dimension. The first time dimension, the second time dimension, the third time dimension, and the fourth time dimension can have the same size and / or the same number of pixels. The first frequency dimension, the second frequency dimension, the third frequency dimension, and / or the fourth frequency dimension can have a size of m2 and / or m2 pixels (or Fourier transform window), such as 5, 10, 20, 50, 100, 2000, 5000, 10000, 2000, 5000, 10000, 20000, 50000, 100000, 200000, 500000, 1000000, 200000, 500000, 1000000 data points, or a number or range of data points between any two of these values. One or more of the first frequency dimension, the second frequency dimension, the third frequency dimension, and the fourth frequency dimension can have the same size and / or the same number of pixels. The first frequency dimension, the second frequency dimension, the third frequency dimension, and the fourth frequency dimension can have the same size and / or the same number of pixels. In different implementations, m2 and m1 can be different. For example, m2 can be equal to 1 / 2*m1. As another example, m2 is less than 1 / 2*m1. m2 and 1 / 2*m1 can differ, differ approximately, differ by at least, or differ by at most 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 50000, 100000, 200000, 500000, 1000000, 2000000, 5000000, 10000000, or a number or range between any two of these values.

[0292] The pixel size and / or number of pixels along the first frequency dimension, the pixel size and / or number of pixels along the second frequency dimension, the pixel size and / or number of pixels along the third frequency dimension, and / or the pixel size and / or number of pixels along the fourth frequency dimension can be the same as the number of multiple RF frequency-shifted laser beams. The pixels along the first frequency dimension, the pixels along the second frequency dimension, the pixels along the third frequency dimension, and / or the pixels along the fourth frequency dimension can correspond to multiple RF frequency-shifted laser beams. Windows of Fourier transforms (e.g., discrete Fourier transforms) that do not correspond to RF frequency-shifted laser beams or the frequencies of the RF frequency-shifted laser beams can be discarded (or not calculated). The pixels along the first frequency dimension, the pixels along the second frequency dimension, the pixels along the third frequency dimension, and / or the pixels along the fourth frequency dimension can correspond to multiple spatial locations of cells.

[0293] In some embodiments, the computing system can generate a plurality of visual representations of the plurality of phase-corrected images based on real parts of the complex values ​​of the plurality of phase-corrected images. The computing system can generate a plurality of visual representations of the plurality of phase-corrected images based on magnitudes of the complex values ​​of the plurality of phase-corrected images. The computing system can generate a combined visual representation (e.g., Fig.26B To generate a combined visual representation, the computing system can assign a color to each of the plurality of visual representations (or corresponding fluorophores); generate a plurality of color visual representations in the plurality of visual representations based on the colors assigned to the visual representations, and combine the plurality of color visual representations to generate the combined visual representation.

[0294] From block 2724, method 2700 proceeds to block 2728, where the computing system determines a sorting decision for the cell based on the plurality of phase-corrected images. To determine the sorting decision, the computing system can determine a sorting decision for the cell based on the plurality of phase-corrected images. To determine the sorting decision, the computing system can determine a sorting decision for the cell based on a plurality of visual representations of the plurality of phase-corrected images. To determine the sorting decision, the hardware processor is programmed with executable instructions to determine a sorting decision for the cell based on a number of pixels of one or more of the plurality of phase-corrected images above a sorting decision threshold. The sorting decision threshold can be, about, more than, or less than 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 200, 300, 400, 500, 600, 700, 800, 900, 1000, 2000, 3000, 4000, 5000, 6000, 7000, 8000, 9000, 10000, or a number or range between any two of these values. To determine the sorting decision, the hardware processor is programmed with executable instructions to determine a sorting decision for the cell based on a percentage of pixels in one or more of the plurality of phase-corrected images that are above the sorting decision threshold. The sorting decision threshold can be, be approximately, be more than or less than 1%, 2%, 3%, 4%, 5%, 6%, 7%, 8%, 9%, 10%, 11%, 12%, 13%, 14%, 15%, 16%, 17%, 18%, 19%, 20%, 21%, 22%, 23%, 24%, 25%, 26%, 27%, 28%, 29%, 30%, 31%, 32%, 33%, 34%, 35%, 36%, 37%, 38%, 39%, 40%, 41%, 42%, 43%, 44%, 45%, 46%, 47%, 48%, 49%, 50%, 51% %, 52%, 53%, 54%, 55%, 56%, 57%, 58%, 59%, 60%, 61%, 62%, 63%, 64%, 65%, 66%, 67%, 68%, 69%, 70%, 71%, 72%, 73%, 74%, 75%, 76%, 77%, 78%, 79%, 80%, 81%, 82%, 83%, 84%, 85%, 86%, 87%, 88%, 89%, 90%, 91%, 92%, 93%, 94%, 95%, 96%, 97%, 98%, 99%, 99.9%, or a number or range between any two of these values. Method 2700 ends at block 2732.

[0295] Two or more of blocks 2712, 2716, 2720, 2724, and 2728 can be performed together. In some embodiments, the computing system can perform two or more of the following: generating multiple original images, generating multiple phase adjustment images, generating multiple unmixed images, and generating multiple phase-corrected images is performed using a combination matrix including or representing two or more of a time-frequency transform matrix, a channel phase adjustment matrix, an unmixing matrix, and a fluorophore phase correction matrix. To generate multiple original images and generate multiple phase adjustment images, the computing system can use a combination matrix including or representing a time-frequency transform matrix and a channel phase adjustment matrix to generate multiple phase adjustment images from fluorescence intensity data of multiple channels. To generate multiple unmixed images and generate multiple phase-corrected images, the computing system can use a combination matrix including or representing an unmixing matrix and a fluorophore phase correction matrix to generate multiple phase-corrected images from multiple phase adjustment images. For example, corresponding pixel rows of a plurality of phase-adjusted images (e.g., pixels 2514i1p00, 2514i1p01, ... of phase-adjusted image 2514i1 and pixels 2514i2p00, 2514i2p01, ... of phase-adjusted image 2514i2) can be combined into a row vector. The row vector can be multiplied by a combined matrix including or representing an unmixing matrix and a fluorophore phase correction matrix to generate a row vector including corresponding pixel rows of a plurality of phase-corrected images (e.g., pixels 2522i1p00, 2522i1p01, ... of phase-corrected image 2522i1 and pixels 2522i2p00, 2522i2p01, ... of phase-corrected image 2522i2). In order to generate multiple original images, generate multiple phase-adjusted images, generate multiple unmixed images, and generate multiple phase-corrected images, the computing system is able to use a combined matrix including or representing a time-frequency transformation matrix, a channel phase adjustment matrix, an unmixing matrix, and a fluorophore phase correction matrix to generate multiple phase-corrected images from the fluorescence intensities of multiple channels.

[0296] In some embodiments, the computing system can generate multiple channels of corrected fluorescence intensity data from multiple phase-corrected images. To generate the multiple channels of corrected fluorescence intensity data, the computing system can use a frequency-space to time-space transform to generate multiple channels of corrected fluorescence intensity data from multiple phase-corrected images. The computing system can determine an estimate of a cell characteristic based on the corrected fluorescence intensity data for the multiple channels. Determining a sorting decision can include determining an estimate of a cell characteristic based on an estimate of a cell characteristic as described herein (such as with reference to FIG. Fig.14A , 14C, 14D, 16, 18, 19A, 19B, 20A, 20B, and 28, or portions thereof) to determine the sorting decision of the cell. The cell characteristics can include cell size, a ratio of the sizes of the cell in two different dimensions, co-localization of fluorescence emission by two or more of a plurality of fluorophores associated with the cell, a ratio of the sizes of the cytoplasm and the nucleus of the cell, the degree of punctateness of the fluorescence emission of the cell, a measure of the spatial distribution of the fluorescence emission, a measure of cell location or orientation, a measure of cell eccentricity, a measure of the similarity of the cell to a reference cell, a measure of the degree to which the cell is in the focus of a laser beam, or a combination thereof.

[0297] Execution Environment

[0298] exist Fig.30 , which depicts the overall architecture of an example computing device 3000 configured to implement the metabolite, annotation, and gene integration system disclosed herein. Fig.30 The overall architecture of the computing device 3000 depicted in FIG. 3 includes an arrangement of computer hardware and software components. The computing device 3000 may include: Fig.30 3060, all of which can communicate with each other through a communication bus. The network interface 3020 can provide connectivity to one or more networks or computing systems. The processing unit 3010 can therefore receive information and instructions from other computing systems or services via a network. The processing unit 3010 can also communicate to the memory 3070 and communicate from it, and also provide output information to the optional display 3050 via the input / output device interface 3040. The input / output device interface 3040 can also accept input from an optional input device 3060, such as a keyboard, a mouse, a digital pen, a microphone, a touch screen, a gesture recognition system, a voice recognition system, a game board, an accelerometer, a gyroscope or other input devices.

[0299] Memory 3070 may contain computer program instructions (grouped into modules or components in some embodiments) that are executed by processing unit 3010 to implement one or more embodiments. Memory 3070 generally includes RAM, ROM, and / or other persistent, auxiliary, or non-transitory computer-readable media. Memory 3070 may store an operating system 3072 that provides computer program instructions for use by processing unit 3010 in general management and operation of computing device 3000. Memory 3070 may also contain computer program instructions and other information for implementing various aspects of the present disclosure.

[0300] For example, in one embodiment, memory 3070 includes memory for performing spectral unmixing (such as referring to Fig.25 The spectral unmixing method 2500 described and referenced Fig. 27 2700) of the spectral unmixing method 2700 described herein. In addition, the memory 3070 may include or communicate with a data storage area 3090 and / or one or more other data storage areas that store fluorescence emissions or signals, raw images, phase adjusted images, unmixed images, phase corrected images, and / or sorting decisions.

[0301] the term

[0302] As used herein, the term "determined" or "determining" encompasses a wide variety of actions. For example, "determining" may include mathematically calculating, computing, processing, deriving, investigating, searching (e.g., searching in a table, a database, or another data structure), ascertaining, etc. Moreover, "determining" may also include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory), etc. Moreover, "determining" may also include resolving, selecting, choosing, establishing, etc.

[0303] As used herein, the term "provided" or "providing" encompasses a wide variety of actions. For example, "providing" may include storing a value in a location on a storage device for subsequent retrieval, sending a value directly to a recipient via at least one wired or wireless communication medium, sending or storing a reference to a value, etc. "Providing" may also include encoding, decoding, encrypting, decrypting, verifying, authenticating, etc. via a hardware element.

[0304] As used herein, the term "selectively" or "selective" may encompass a wide variety of actions. For example, a "selective" process may include determining an option from a plurality of options. A "selective" process may include one or more of: a dynamically determined input, a preconfigured input, or a user-initiated input for making a determination. In some implementations, an n-input switch may be included to provide selectivity functionality, where n is the number of inputs for making a selection.

[0305] As used herein, the term "message" encompasses various formats for communicating (e.g., sending or receiving) information. A message may include a machine-readable collection of information (such as an XML document, a fixed-field message, a comma-delimited message, etc.). In some implementations, a message may include a signal for transmitting one or more representations of the information. Although described in the singular, it should be understood that a message may be composed, sent, stored, received, etc., in multiple parts.

[0306] As used herein, "user interface" (also referred to as an interactive user interface, graphical user interface, or UI) may refer to a web-based interface that contains data fields, buttons, or other interactive controls for receiving input signals or for providing electronic information or for providing information to a user in response to any received input signals. TM 、FLASH TM , JAVA TM , .NET TM 、WINDOWS OS TM , macOS TM , web services or rich site summary (RSS) technology to implement the UI in whole or in part. In some implementations, the UI can be included in a standalone client (e.g., thick client, fat client) configured to communicate (e.g., send or receive data) according to one or more aspects described.

[0307] As used herein, a "data store" may be implemented in a hard drive, solid-state memory, and / or any other type of non-transitory computer-readable storage medium accessible by or to a device (such as an access device, server, or other computing device described). As is known in the art, the data store may also or alternatively be distributed or partitioned across multiple local and / or remote storage devices without departing from the scope of the present disclosure. In other embodiments, the data store may be included in or implemented in a data storage network service.

[0308] Those skilled in the art will appreciate that any of a variety of different technical means or techniques may be used to represent information, messages, and signals. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0309] Those skilled in the art will also appreciate that the various illustrative logic blocks, modules, circuits and algorithmic steps described in conjunction with the disclosed embodiments herein can be implemented as electronic hardware, computer software or a combination of the two. In order to clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits and steps have been generally described above in terms of their functions. Whether this functionality is implemented as hardware or software depends on specific applications and the design constraints imposed on the entire system. The technician can implement the described functions in various ways for each specific application, but this implementation decision should not be interpreted as causing departure from the scope of the present invention.

[0310] The technology described herein can be implemented in hardware, software, firmware or any combination thereof. Such technology can be implemented in any of various devices (such as specially programmed event processing computers, wireless communication devices or integrated circuit devices). Any features described as modules or components can be implemented together in an integrated logic device, or can also be implemented as discrete but interoperable logic devices respectively. If implemented in software, the technology can be at least partially implemented by a computer-readable data storage medium, which includes a program code, and the program code contains instructions for executing one or more of the above methods when executed. The computer-readable data storage medium can form a part of a computer program product, which can include packaging materials. The computer-readable medium may include a memory or a data storage medium, such as a random access memory (RAM) (such as a synchronous dynamic random access memory (SDRAM)), a read-only memory (ROM), a non-volatile random access memory (NVRAM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic or optical data storage medium, etc. The computer-readable medium can be a non-temporary storage medium. Additionally or alternatively, the technology may be at least partially implemented by a computer-readable communication medium that carries or communicates program code in the form of instructions or data structures (such as a propagating signal or wave) and can be accessed, read and / or executed by a computing device.

[0311] The program code can be executed by a specially programmed sorting strategy processor, which can include one or more processors, such as one or more digital signal processors (DSPs), configurable microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Such a graphics processor can be specifically configured to perform any of the techniques described in this disclosure. A combination of computing devices, such as a DSP with a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration in at least a portion of the data connection, can implement one or more of the described features. In some aspects, the functionality described herein can be provided in a dedicated software module or hardware module configured for encoding and decoding, or incorporated into a dedicated sorting control card.

[0312] Those skilled in the art will understand that the terms used generally herein, and particularly in the appended claims (e.g., the bodies of the appended claims), are generally intended to be "open" terms (e.g., the term "including" should be interpreted as "including but not limited to," the term "having" should be interpreted as "having at least," the term "comprising" should be interpreted as "including but not limited to," etc.). Those skilled in the art will also understand that if a certain number of an introduced claim clause is intended, that intent will be explicitly recited in the claim, and in the absence of such a recitation, no such intent is present. For example, to aid understanding, the appended claims may contain usage of the introductory phrases "at least one" and "one or more" to introduce claim recitations. However, the use of such phrases should not be construed to imply that a claim recitation introduced by the indefinite article "a" or "an" limits any particular claim containing such introduced claim recitation to embodiments containing only one such recitation, even when the same claim includes the introductory phrases "one or more" or "at least one" and an indefinite article such as "a" or "an" (e.g., "a" and / or "an" should be construed to mean "at least one" or "one or more"); this is also true for the use of the definite article used to introduce a claim recitation. In addition, even if a specific number of introduced claim recitations is explicitly recited, one skilled in the art will recognize that such recitation should be construed to mean at least the recited number (e.g., the unmodified recitation of "two recitations" without other modifiers means at least two recitations, or means two or more recitations). Furthermore, where a convention similar to “at least one of A, B, and C, etc.” is used, such construction is generally intended to be in the sense that a person skilled in the art would understand the convention (e.g., “a system having at least one of A, B, and C” would include but is not limited to systems having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, and C, etc.). Where a convention similar to “at least one of A, B, or C, etc.” is used, such construction is generally intended to be in the sense that a person skilled in the art would understand the convention (e.g., “a system having at least one of A, B, or C” would include but is not limited to systems having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, and C, etc.). A person skilled in the art would also understand that, in practice, any disjunctive word and / or phrase presenting two or more alternative terms, whether in the specification, claims, or drawings, should be understood to contemplate the possibility of including one of the terms, either of the terms, or both of the terms. For example, the phrase “A or B” would be understood to include the possibility of “A” or “B” or “A and B”.

[0313] In addition, where features or aspects of the disclosure are described in terms of Markush groups, those skilled in the art will recognize that the disclosure is also thereby described in terms of any individual member or subgroup of members of the Markush group.

[0314] As will be understood by those skilled in the art, for any and all purposes, such as in terms of providing written descriptions, all ranges disclosed herein also encompass any and all possible sub-ranges and combinations of sub-ranges thereof. Any listed range can be easily identified as fully described, and the same range can be decomposed into at least half, one-third, one-quarter, one-fifth, one-tenth, etc. As a non-limiting example, each range discussed herein can be easily decomposed into a lower third, a middle third, and an upper third, etc. As will be understood by those skilled in the art, all languages ​​(such as "at most", "at least", "greater than", "less than", etc.) include the narrated numbers, and refer to the ranges that can be subsequently decomposed into the above-mentioned sub-ranges. Finally, as will be understood by those skilled in the art, the range includes each individual component. Therefore, for example, a group with 1-3 articles refers to a group with 1, 2 or 3 articles. Similarly, a group with 1-5 articles refers to a group with 1, 2, 3, 4 or 5 articles, and the like.

[0315] The method disclosed herein includes one or more steps or actions for implementing the described method. Without departing from the scope of the claims, the method steps and / or actions can be interchangeable with each other. In other words, unless a specific order of steps or actions is indicated, the order and / or use of specific steps and / or actions can be modified without departing from the scope of the claims.

[0316] Although various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for illustrative purposes and are not intended to be limiting, with the true scope and spirit being indicated by the appended claims.

Claims

1. A method for sorting a plurality of cells of a sample, the method comprising: Under the control of the hardware processor: irradiating cells in a sample including a plurality of cells with a laser beam having a plurality of different optical frequencies to induce fluorescence emission from a plurality of fluorophores associated with the cells; detecting fluorescence emission at different wavelengths to obtain fluorescence intensity data of multiple channels corresponding to multiple fluorophores in time and space; Generating a plurality of raw images including a plurality of pixels of complex values ​​in a frequency space from the fluorescence intensity data of the plurality of channels, wherein generating the plurality of raw images comprises: using a time-frequency space transform to generate the plurality of raw images in a frequency space from the fluorescence intensity data of the plurality of channels; generating, for each of the plurality of channels of fluorescence intensity data, a plurality of phase-adjusted images including a plurality of pixels of complex values ​​in frequency space from the plurality of original images using one or more channel phase adjustments; Using an unmixing matrix on the complex values ​​of corresponding pixels of the plurality of phase adjusted images to generate a plurality of unmixed images including a plurality of pixels having complex values ​​in frequency space from the plurality of phase adjusted images; generating a plurality of phase-corrected images comprising a plurality of pixels having complex or real values ​​in frequency space from the plurality of unmixed images based on a property of each of the plurality of fluorophores; and A sorting decision is determined for the cell based on the plurality of phase-corrected images.

2. A method for sorting a plurality of cells of a sample, the method comprising: Under the control of the hardware processor: irradiating cells in a sample including a plurality of cells with a laser beam having a plurality of different optical frequencies to induce fluorescence emission from a plurality of fluorophores associated with the cells; detecting fluorescence emission at different wavelengths to obtain fluorescence intensity data of multiple channels corresponding to multiple fluorophores in time and space; A plurality of complex-valued images in a frequency space are generated from the fluorescence intensity data of the plurality of channels. original images, wherein generating the plurality of original images comprises: using a time-frequency space transform to generate a plurality of original images in a frequency space from the fluorescence intensity data of the plurality of channels; generating a plurality of unmixed images including complex values ​​or real values ​​in frequency space from the plurality of original images using an unmixing matrix on corresponding complex values ​​of the plurality of original images; and A sorting decision is determined for the cell based on the plurality of unmixed images.

3. The method according to claim 2, comprising: The cells are illuminated with a laser beam to induce fluorescence emission for the fluorescence intensity data of the plurality of channels in space and time.

4. The method according to claim 2 or 3, wherein: The plurality of original images include a plurality of pixels of complex values ​​of the plurality of original images.

5. The method according to claim 2, comprising: generating a plurality of phase-adjusted images including complex values ​​in frequency space from a plurality of original images using one or more channel phase adjustments for each of the plurality of channels of fluorescence intensity data, The generating of the plurality of unmixed images comprises: using an unmixing matrix on corresponding complex values ​​of the plurality of phase-adjusted images to generate the plurality of unmixed images from the plurality of phase-adjusted images.

6. The method according to claim 5, wherein: The plurality of phase adjustment images include a plurality of pixels of complex values ​​of the plurality of phase adjustment images.

7. The method according to claim 2, comprising: generating a plurality of phase-corrected images comprising complex values ​​in frequency space from the plurality of unmixed images based on a property of each of the plurality of fluorophores, Wherein, determining the sorting decision comprises: determining a sorting decision for the cell based on the plurality of phase-corrected images.

8. The method according to claim 7, wherein: The plurality of phase-corrected images include a plurality of complex-valued pixels of the plurality of phase-corrected images.

9. The method according to claim 1, wherein: The time-frequency space transformation comprises a Fourier transformation.

10. The method according to claim 9, wherein: The Fourier transform includes a discrete Fourier transform.

11. The method according to claim 10, wherein: The discrete Fourier transform includes a fast Fourier transform.

12. The method according to claim 9, wherein: The Fourier transform comprises a sliding window Fourier transform.

13. The method according to claim 12, wherein: The size of the sliding window of the sliding window Fourier transform is m1.

14. The method according to claim 1 or 2, wherein: Generating the plurality of original images includes generating the plurality of original images using a time-frequency space transformation matrix representing the time-frequency space transformation.

15. The method according to claim 1, wherein: The one or more channel phase adjustments correspond to, or correct, one or more channel phases including a brightfield phase, a calibration phase, and a phase offset for each of the plurality of channels.

16. The method according to claim 1, wherein: Generating a plurality of phase adjustment images for each of the plurality of channels comprises: Complex values ​​of original images in the plurality of original images corresponding to the channels are multiplied to generate a phase-adjusted image in the plurality of phase-adjusted images corresponding to the channels.

17. The method according to claim 1, wherein: Generating a plurality of phase adjustment images includes generating the plurality of phase adjustment images using one or more channel phase adjustment matrices representing the one or more channel phase adjustments.

18. The method according to claim 1, wherein: The one or more channel phase adjustments comprise complex values.

19. The method of claim 1, comprising determining or receiving the one or more channel phase adjustments.

20. The method according to claim 1 or 2, wherein: Generating multiple unmixed images includes: generating a vector comprising complex values ​​of corresponding pixels of a plurality of phase adjusted images; multiplying the vector with an unmixing matrix to generate an unmixing vector comprising unmixed complex values; and A plurality of unmixed images are generated, the unmixed images including corresponding pixels having the unmixed complex values.

21. The method according to claim 20, wherein: The size of the vector is 1×n1, wherein the size of the unmixing matrix is ​​n1×n2, and / or the size of the unmixing vector is 1×n2.

22. The method according to claim 13, wherein: Generating multiple unmixed images includes: For corresponding pixels of the plurality of phase adjustment images, an unmixing matrix from a plurality of unmixing matrices is used on complex values ​​of the corresponding pixels to generate a plurality of unmixed images from the plurality of phase adjustment images.

23. The method according to claim 22, wherein: The plurality of unmixing matrices include m2 unmixing matrices.

24. The method according to claim 1 or 2, wherein: The unmixing matrix includes real values.

25. The method according to claim 24, wherein: The unmixing matrix includes only real values.

26. The method according to claim 1 or 2, wherein: The unmixing matrix includes real values.

27. The method of claim 1 or 2, comprising determining the unmixing matrix.

28. The method of claim 1, wherein: Generating multiple phase-corrected images includes: A phase-corrected image in a plurality of phase-corrected images is generated from an unmixed image in a plurality of unmixed images corresponding to the fluorophores in the plurality of fluorophores using a plurality of fluorophore phase corrections for the fluorophores in the plurality of fluorophores.

29. The method according to claim 28, wherein: The multi-fluorophore phase correction used for a fluorophore is related to the properties of the fluorophore.

30. The method of claim 29, wherein: The properties of the fluorophore include the lifetime of the fluorophore.

31. The method of claim 1, wherein: The plurality of fluorophore phase corrections comprises complex values.

32. The method of claim 1, comprising determining a plurality of fluorophore phase corrections for the fluorophores.

33. The method of claim 1, wherein: Generating a plurality of phase-corrected images includes generating the plurality of phase-corrected images using a fluorophore phase correction matrix for the fluorophores, the fluorophore phase correction matrix for the fluorophores representing a plurality of fluorophore phase corrections for the fluorophores.

34. The method of claim 1, wherein: Determining the sorting decision includes determining a sorting decision for the cell based on a number of pixels in one or more of the plurality of phase-corrected images that are above a sorting decision threshold.

35. The method of claim 1, wherein: Determining the sorting decision includes determining a sorting decision for the cell based on a percentage of pixels in one or more of the plurality of phase-corrected images that are above a sorting decision threshold.

36. The method of claim 1, comprising generating a plurality of visual representations of a plurality of phase-corrected images based on real parts of the complex values ​​of the plurality of phase-corrected images.

37. The method of claim 1, comprising generating a plurality of visual representations of a plurality of phase-corrected images based on the magnitudes of the complex values ​​of the plurality of phase-corrected images.

38. The method according to claim 36 or 37, wherein: Determining the sorting decision includes determining a sorting decision for a cell based on a plurality of visual representations of the plurality of phase-corrected images.

39. A method according to claim 36 or 37, comprising generating a combined visual representation from a plurality of visual representations.

40. The method of claim 1, wherein: Generating a plurality of original images and generating a plurality of phase-adjusted images includes: generating a plurality of phase-adjusted images from the fluorescence intensity data of the plurality of channels using a combination matrix.

41. The method of claim 1, wherein: Generating a plurality of unmixed images and generating a plurality of phase-corrected images includes generating a plurality of phase-corrected images from a plurality of phase-adjusted images using a combining matrix including an unmixing matrix.

42. The method of claim 1, wherein: Generating a plurality of original images, generating a plurality of phase-adjusted images, generating a plurality of unmixed images, and generating a plurality of phase-corrected images includes generating a plurality of phase-corrected images from the fluorescence intensity data of the plurality of channels using a combination matrix including an unmixing matrix.

43. The method of claim 1, comprising generating corrected fluorescence intensity data for a plurality of channels from the plurality of phase-corrected images.

44. The method of claim 43, wherein: Generating the plurality of channels of corrected fluorescence intensity data includes generating the plurality of channels of corrected fluorescence intensity data from the plurality of phase-corrected images using a frequency-time spatial transform.

45. The method of claim 43 or 44, comprising determining an estimate of a cell characteristic based on the corrected fluorescence intensity data of the plurality of channels, wherein: Determining a sorting decision includes determining a sorting decision for the cell based on an estimate of a characteristic of the cell.

46. ​​The method of claim 45, wherein: The cell characteristics include cell size, a ratio of the sizes of the cell in two different dimensions, co-localization of fluorescence emission of two or more fluorophores among a plurality of fluorophores associated with the cell, a ratio of the sizes of the cytoplasm and the nucleus of the cell, the degree of punctateness of the fluorescence emission of the cell, a measure of the spatial distribution of the fluorescence emission, a measure of the cell's position or orientation, a measure of the eccentricity of the cell, a measure of the similarity of the cell to a reference cell, a measure of the extent to which the cell is in the focus of a laser beam, or a combination thereof.

47. The method of claim 13, wherein: The laser beams include a reference laser beam and a plurality of radio frequency-shifted laser beams.

48. The method of claim 47, wherein: The number of the plurality of RF frequency-shifted laser beams is m2.

49. The method of claim 47, wherein: The reference laser beam spatially overlaps with the plurality of RF frequency-shifted laser beams.

50. The method of claim 47, wherein: The plurality of radio frequency-shifted laser beams do not spatially overlap with each other.

51. The method of claim 47, wherein: The reference laser beam and one or more of the plurality of radio frequency-shifted laser beams are capable of inducing fluorescence emission from a fluorophore among the plurality of fluorophores.

52. The method of claim 47, wherein: Irradiating the cell includes utilizing multiple radio frequency frequency-shifted laser beams to simultaneously irradiate multiple spatial locations of the cell.

53. The method of claim 52, wherein: The multiple spatial locations of the cells do not overlap with each other.

54. The method of claim 1, wherein: The number of the plurality of original images, the number of the plurality of phase adjustment images, the number of the plurality of unmixed images, and the number of the plurality of phase correction images are the same.

55. The method according to claim 1, in, The number of the plurality of original images and / or the number of the plurality of phase-adjusted images is n1, and / or The number of the multiple unmixed images and / or the number of the multiple phase-corrected images is n2.

56. The method according to claim 47, in, The plurality of original images are associated with a first time dimension and a first frequency dimension, wherein the plurality of phase-adjusted images are associated with a second time dimension and a second frequency dimension, wherein the plurality of unmixed images are associated with a third time dimension and a third frequency dimension, and / or Therein, the plurality of phase-corrected images are associated with a fourth time dimension and a fourth frequency dimension.

57. The method of claim 56, wherein: Two or more of the first time dimension, the second time dimension, the third time dimension, and the fourth time dimension have the same size and / or the same number of pixels along the dimension.

58. The method of claim 57, wherein: The first time dimension, the second time dimension, the third time dimension and the fourth time dimension have the same size and / or the same number of pixels.

59. The method of claim 56, wherein: The first frequency dimension, the second frequency dimension, the third frequency dimension and / or the fourth frequency dimension have a size of m2 and / or m2 pixels.

60. The method of claim 59, wherein m2 is equal to 1 / 2*m1.

61. The method of claim 59, wherein m2 is less than 1 / 2*m1.

62. The method according to any one of claims 56 to 59, wherein: Two or more of the first frequency dimension, the second frequency dimension, the third frequency dimension, and the fourth frequency dimension have the same size and / or the same number of pixels.

63. The method of claim 62, wherein: The first frequency dimension, the second frequency dimension, the third frequency dimension and the fourth frequency dimension have the same size and / or the same number of pixels.

64. The method according to any one of claims 56 to 59, wherein: the pixel size and / or the number of pixels along a first frequency dimension, the pixel size and / or the number of pixels along the second frequency dimension, the pixel size and / or number of pixels along a third frequency dimension, and / or The pixel size and / or the number of pixels along the fourth frequency dimension are the same as the number of the plurality of RF frequency-shifted laser beams.

65. The method of claim 64, wherein: Pixels along the first frequency dimension, pixels along the second frequency dimension, pixels along the third frequency dimension, and / or pixels along the fourth frequency dimension correspond to the plurality of radio frequency shifted laser beams.

66. The method of claim 64, wherein: Pixels along the first frequency dimension, pixels along the second frequency dimension, pixels along the third frequency dimension, and / or pixels along the fourth frequency dimension correspond to a plurality of spatial locations of the cell.

67. The method of claim 1 or 2, wherein: The fluorescence emissions of two of the plurality of fluorophores spectrally overlap.

68. A cell sorter system comprising: a laser light source configured to generate a laser beam having a plurality of different optical frequencies for inducing fluorescence emission at different wavelengths from a plurality of fluorophores associated with cells in a sample comprising a plurality of cells; a plurality of photodetectors configured to detect fluorescence emissions from the plurality of fluorophores; a non-transitory memory configured to store executable instructions; and A processor in communication with the non-transitory memory, the hardware processor being programmed with executable instructions to: allowing the laser beam to illuminate the cell to induce fluorescence emission from a plurality of fluorophores associated with the cell; enabling the plurality of photodetectors to detect fluorescence emission at different wavelengths to obtain fluorescence intensity data of a plurality of channels corresponding to a plurality of fluorophores in time and space; generating a plurality of raw images including a plurality of pixels of complex values ​​in a frequency space from the fluorescence intensity data of the plurality of channels, wherein, to generate the plurality of raw images, the processor is programmed by executable instructions to: generate the plurality of raw images in a frequency space from the fluorescence intensity data of the plurality of channels using a time-frequency space transform; generating, for each of the plurality of channels of fluorescence intensity data, a plurality of phase-adjusted images including a plurality of pixels of complex values ​​in frequency space from the plurality of original images using one or more channel phase adjustments; Using an unmixing matrix on complex values ​​of corresponding pixels in the plurality of phase adjusted images to generate a plurality of unmixed images comprising a plurality of pixels having complex values ​​in frequency space from the plurality of phase adjusted images; generating a plurality of phase-corrected images comprising a plurality of pixels having complex or real values ​​in frequency space from the plurality of unmixed images based on a property of each of the plurality of fluorophores; and A sorting decision is determined for the cell based on the plurality of phase-corrected images.

69. A cell sorter system comprising: a laser light source configured to generate a laser beam having a plurality of different optical frequencies for inducing fluorescence emission at different wavelengths from a plurality of fluorophores associated with cells in a sample comprising a plurality of cells; a plurality of photodetectors configured to detect fluorescence emissions from the plurality of fluorophores; a non-transitory memory configured to store executable instructions; and A processor in communication with the non-transitory memory, the hardware processor being programmed with executable instructions to: receiving fluorescence intensity data for multiple channels of fluorescence emission at different wavelengths from a plurality of fluorophores associated with the cell, the plurality of channels of fluorescence intensity data corresponding to the plurality of fluorophores in time and space, wherein the fluorescence emission is detected by a plurality of photodetectors, and wherein the fluorescence emission is induced after the cell is illuminated by a laser light source; generating a plurality of raw images including complex values ​​in a frequency space from the fluorescence intensity data of the plurality of channels, wherein, to generate the plurality of raw images, the processor is programmed by executable instructions to: generate the plurality of raw images in a frequency space from the fluorescence intensity data of the plurality of channels using a time-frequency space transform; Using the unmixing matrix on corresponding complex values ​​in the plurality of original images to generate a plurality of unmixed images including complex values ​​or real values ​​in frequency space from the plurality of original images; and A sorting decision is determined for the cell based on the plurality of unmixed images.

70. The cell sorter system according to claim 69, wherein: In order to receive the fluorescence intensity data of the plurality of channels, the processor is programmed with executable instructions to: allowing the laser beam to illuminate the cell to induce fluorescence emission from a plurality of fluorophores associated with the cell; as well as The multiple photodetectors are configured to detect fluorescence emission at different wavelengths to obtain fluorescence intensity data of multiple channels corresponding to multiple fluorophores in time and space.

71. The cell sorter system according to claim 69 or 70, wherein: The plurality of original images include a plurality of pixels having the complex values.

72. The cell sorter system according to claim 69, wherein the processor is programmed with executable instructions to: generate a plurality of phase adjusted images including complex values ​​in frequency space from a plurality of raw images using one or more channel phase adjustments for each of the plurality of channels of fluorescence intensity data, and To generate the plurality of unmixed images, the processor is programmed by executable instructions to generate the plurality of unmixed images from the plurality of phase-adjusted images using an unmixing matrix on corresponding complex values ​​of the plurality of phase-adjusted images.

73. The cell sorter system according to claim 72, wherein: The plurality of phase adjustment images include a plurality of pixels having the complex values.

74. The cell sorter system according to claim 69, wherein the processor is programmed with executable instructions to: generate a plurality of phase-corrected images comprising complex values ​​in frequency space from the plurality of unmixed images based on a property of each of the plurality of fluorophores, and Wherein, to determine the sorting decision, the processor is programmed with executable instructions to: determine a sorting decision for the cell based on the plurality of phase-corrected images.

75. The cell sorter system according to claim 74, wherein: The plurality of phase-corrected images include a plurality of pixels of complex values ​​or real values ​​of the plurality of phase-corrected images.

76. The cell sorter system according to claim 68, wherein: The hardware processor includes a field programmable gate array (FPGA).

77. The cell sorter system according to claim 68, wherein: The time-frequency space transformation comprises a Fourier transformation.

78. The cell sorter system according to claim 77, wherein: The Fourier transform includes a discrete Fourier transform.

79. The cell sorter system according to claim 78, wherein: The discrete Fourier transform includes a fast Fourier transform.

80. The cell sorter system according to claim 77, wherein: The Fourier transform comprises a sliding window Fourier transform.

81. The cell sorter system according to claim 80, wherein: The size of the sliding window of the sliding window Fourier transform is m1.

82. The cell sorter system according to claim 68, wherein: The one or more channel phase adjustments correspond to, or correct, one or more channel phases including a brightfield phase, a calibration phase, and a phase offset for each of the plurality of channels.

83. The cell sorter system according to claim 68, wherein: To generate multiple phase adjustment images, the processor is programmed by executable instructions to: for each channel in the multiple channels: multiply the complex values ​​of the original images in the multiple original images corresponding to the channel to generate a phase adjustment image in the multiple phase adjustment images corresponding to the channel.

84. The cell sorter system according to claim 68, wherein: The one or more channel phase adjustments comprise complex values.

85. The cell sorter system according to claim 68, wherein: The processor is programmed with executable instructions to receive the one or more channel phase adjustments.

86. The cell sorter system according to claim 68 or 69, wherein: To generate the plurality of unmixed images, the processor is programmed with executable instructions to: generating a vector comprising complex values ​​of corresponding pixels of a plurality of phase adjusted images; multiplying the vector with an unmixing matrix to generate an unmixing vector comprising unmixed complex values; and A plurality of unmixed images are generated, the unmixed images including corresponding pixels having the unmixed complex values.

87. The cell sorter system according to claim 86, wherein: The size of the vector is 1×n1, wherein the size of the unmixing matrix is ​​n1×n2, and / or the size of the unmixing vector is 1×n2.

88. The cell sorter system according to claim 81, wherein: To generate multiple unmixed images, the processor is programmed by executable instructions to: for corresponding pixels of the multiple phase-adjusted images, use an unmixing matrix from a plurality of unmixing matrices on complex values ​​of the corresponding pixels to generate multiple unmixed images from the multiple phase-adjusted images.

89. The cell sorter system according to claim 88, wherein: The plurality of unmixing matrices include m2 unmixing matrices.

90. The cell sorter system according to claim 68 or 69, wherein: The unmixing matrix includes real values.

91. The cell sorter system according to claim 90, wherein: The unmixing matrix includes only real values.

92. The cell sorter system according to claim 68 or 69, wherein: The unmixing matrix includes real values.

93. The cell sorter system according to claim 68 or 69, wherein: The processor is programmed with executable instructions to receive the unmixing matrix.

94. The cell sorter system according to claim 68, wherein: To generate a plurality of phase-corrected images, the processor is programmed with executable instructions to use a plurality of fluorophore phase corrections for fluorophores in the plurality of fluorophores to generate a phase-corrected image in the plurality of phase-corrected images from an unmixed image in the plurality of unmixed images corresponding to the fluorophores in the plurality of fluorophores.

95. The cell sorter system according to claim 94, wherein: The multi-fluorophore phase correction for a fluorophore is related to the properties of the fluorophore.

96. A cell sorter system according to claim 95, wherein the property of the fluorophore includes the lifetime of the fluorophore.

97. The cell sorter system according to claim 68, wherein: The multiple fluorophore phase corrections for the fluorophores include complex values.

98. The cell sorter system according to claim 68, wherein: The processor is programmed with executable instructions to receive a plurality of fluorophore phase corrections for fluorophores.

99. The cell sorter system according to claim 68, wherein: To determine a sorting decision, the hardware processor is programmed with executable instructions to: determine a sorting decision for the cell based on a number of pixels in one or more of the plurality of phase-corrected images that are above a sorting decision threshold.

100. The cell sorter system according to claim 68, wherein: To determine a sorting decision, the hardware processor is programmed with executable instructions to: determine a sorting decision for the cell based on a percentage of pixels in one or more of the plurality of phase-corrected images that are above a sorting decision threshold.

101. The cell sorter system according to claim 68, wherein: The processor is programmed with executable instructions to generate a plurality of visual representations of the plurality of phase-corrected images based on real parts of the complex values ​​of the plurality of phase-corrected images.

102. The cell sorter system according to claim 68, wherein: The processor is programmed with executable instructions to generate a plurality of visual representations of the plurality of phase-corrected images based on magnitudes of the complex values ​​of the plurality of phase-corrected images.

103. The cell sorter system according to claim 101 or 102, wherein: To determine a sorting decision, the processor is programmed with executable instructions to: determine a sorting decision for the cell based on the plurality of visual representations of the plurality of phase-corrected images.

104. The cell sorter system of claim 101 or 102, wherein the processor is programmed with executable instructions to generate a combined visual representation from the plurality of visual representations.

105. The cell sorter system according to claim 68, wherein: The processor is programmed with executable instructions to generate corrected fluorescence intensity data for a plurality of channels from a plurality of phase-corrected images.

106. The cell sorter system according to claim 105, wherein: To generate corrected fluorescence intensity data for the plurality of channels, the processor is programmed with executable instructions to generate corrected fluorescence intensity data for the plurality of channels from the plurality of phase-corrected images using a frequency-time spatial transform.

107. The cell sorter system according to claim 105 or 106, wherein: The processor is programmed with executable instructions to determine an estimate of a characteristic of the cell based on the corrected fluorescence intensity data for the plurality of channels, wherein determining a sorting decision comprises determining a sorting decision for the cell based on the estimate of the characteristic of the cell.

108. The cell sorter system according to claim 107, wherein: The cell characteristics include cell size, a ratio of the sizes of the cell in two different dimensions, co-localization of fluorescence emission of two or more fluorophores among a plurality of fluorophores associated with the cell, a ratio of the sizes of the cytoplasm and the nucleus of the cell, the degree of punctateness of the fluorescence emission of the cell, a measure of the spatial distribution of the fluorescence emission, a measure of the cell's position or orientation, a measure of the eccentricity of the cell, a measure of the similarity of the cell to a reference cell, a measure of the extent to which the cell is in the focus of a laser beam, or a combination thereof.

109. The cell sorter system according to claim 81, wherein: The laser beams include a reference laser beam and a plurality of radio frequency-shifted laser beams.

110. The cell sorter system according to claim 109, wherein: The number of the plurality of RF frequency-shifted laser beams is m2.

111. The cell sorter system according to claim 109, wherein: The reference laser beam spatially overlaps with the plurality of RF frequency-shifted laser beams.

112. The cell sorter system according to claim 109, wherein: The plurality of radio frequency-shifted laser beams do not spatially overlap with each other.

113. The cell sorter system according to claim 109, wherein: The reference laser beam and one or more of the plurality of radio frequency-shifted laser beams are capable of inducing fluorescence emission from a fluorophore among the plurality of fluorophores.

114. The cell sorter system according to claim 109, wherein: Irradiating the cell includes utilizing multiple radio frequency frequency-shifted laser beams to simultaneously irradiate multiple spatial locations of the cell.

115. The cell sorter system according to claim 114, wherein: The multiple spatial locations of the cells do not overlap with each other.

116. The cell sorter system according to claim 68, wherein: The number of the plurality of original images, the number of the plurality of phase adjustment images, the number of the plurality of unmixed images, and the number of the plurality of phase correction images are the same.

117. The cell sorter system according to claim 68, in, The number of the plurality of original images and / or the number of the plurality of phase-adjusted images is n1, and / or The number of the multiple unmixed images and / or the number of the multiple phase-corrected images is n2.

118. The cell sorter system according to claim 109, in, The plurality of original images are associated with a first time dimension and a first frequency dimension, wherein the plurality of phase-adjusted images are associated with a second time dimension and a second frequency dimension, wherein the plurality of unmixed images are associated with a third time dimension and a third frequency dimension, and / or Therein, the plurality of phase-corrected images are associated with a fourth time dimension and a fourth frequency dimension.

119. The cell sorter system according to claim 118, wherein: Two or more of the first time dimension, the second time dimension, the third time dimension, and the fourth time dimension have the same size and / or the same number of pixels along the dimension.

120. The cell sorter system according to claim 119, wherein: The first time dimension, the second time dimension, the third time dimension and the fourth time dimension have the same size and / or the same number of pixels.

121. The cell sorter system according to claim 118, wherein: The first frequency dimension, the second frequency dimension, the third frequency dimension and / or the fourth frequency dimension have a size of m2 and / or m2 pixels.

122. A cell sorter system according to claim 121, wherein m2 is equal to 1 / 2*m1.

123. A cell sorter system according to claim 121, wherein m2 is less than 1 / 2*m1.

124. A cell sorter system according to any one of claims 118 to 121, wherein Two or more of the first frequency dimension, the second frequency dimension, the third frequency dimension, and the fourth frequency dimension have the same size and / or the same number of pixels.

125. The cell sorter system of claim 124, wherein: The first frequency dimension, the second frequency dimension, the third frequency dimension and the fourth frequency dimension have the same size and / or the same number of pixels.

126. A cell sorter system according to any one of claims 118 to 121, wherein the pixel size and / or the number of pixels along a first frequency dimension, the pixel size and / or the number of pixels along the second frequency dimension, the pixel size and / or number of pixels along a third frequency dimension, and / or The pixel size and / or the number of pixels along the fourth frequency dimension are the same as the number of the plurality of RF frequency-shifted laser beams.

127. The cell sorter system of claim 126, wherein: The pixels along the first frequency dimension, the pixels along the second frequency dimension, the pixels along the third frequency dimension, and / or the pixels along the fourth frequency dimension correspond to a plurality of radio frequency shifted laser beams.

128. The cell sorter system of claim 126, wherein: Pixels along the first frequency dimension, pixels along the second frequency dimension, pixels along the third frequency dimension, and / or pixels along the fourth frequency dimension correspond to a plurality of spatial locations of the cell.

129. The cell sorter system according to claim 68 or 69, wherein: The fluorescence emissions of two of the plurality of fluorophores spectrally overlap.

Images