Classification workflow for flexible image-based particle sorting

Abstract

The image-based classification workflow uses unsupervised clustering to help users identify subpopulations of interest for sorting. Labeled cell images are used to fine-tune a supervised classification network for specific experiments. The workflow allows users to select populations for sorting in the same way for various applications. The supervised classification network is very fast, allowing it to make real-time sorting decisions as cells travel through the device. The workflow is more automated and has fewer user steps, thus improving ease of use. The workflow uses machine learning to avoid human error and biases from manual gating. The workflow does not require users to be image processing experts, thus increasing ease of use.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Provisional Patent Application Serial No. 63 / 116,090, filed November 19, 2020, entitled “CLASSIFICATION WORKFLOW FOR FLEXIBLE IMAGE BASED PARTICLE SORTING”, entitled “CLASSIFICATION WORKFLOW FOR FLEXIBLE IMAGE BASED PARTICLE SORTING”, the entire contents of which are incorporated herein by reference for all purposes. Technical Field

[0003] This invention relates to cell sorting. More specifically, this application relates to image-based cell sorting. Background Technology

[0004] Traditional fluorescence-activated cell sorting relies on labeling cells with fluorescent markers and provides very limited cell morphology information. However, some applications require cell morphology information for accurate cell sorting, while others are not suitable for fluorescent markers. Furthermore, traditional fluorescence-activated cell sorting uses manual gating to establish fluorescent marker-based sorting criteria. However, manual gating is time-consuming and prone to bias.

[0005] Many biological applications can benefit from cell sorting, but this is not possible with current flow cytometry-based cell sorters because they do not perform imaging to identify cells of interest.

[0006] One existing approach uses conventional image processing and feature extraction to make sorting decisions from cell images, while another uses deep learning networks for real-time sorting and classification. A drawback of conventional image analysis methods is that they require the user to know how to perform quantitative image analysis, a skill lacking for most cell sorting users. Published deep learning methods are limited because they require offline recognition of cell images for training the convolutional network. This is a labor-intensive, manual process, and training a deep learning network for a specific application can take hours to days, thus failing to provide a flexible solution for diverse applications. Summary of the Invention

[0007] The image-based classification workflow uses unsupervised clustering to help users identify subpopulations of interest for sorting. Labeled cell images are used to fine-tune a supervised classification network for specific experiments. The workflow allows users to select populations for sorting in the same way for various applications. The supervised classification network is very fast, allowing it to make real-time sorting decisions as cells travel through the device. The workflow is more automated and has fewer user steps, improving ease of use. The workflow uses machine learning to avoid human error and biases from manual gating. The workflow does not require users to be image processing experts, thus increasing ease of use.

[0008] In one aspect, a method includes using a pre-trained feature encoder from cell images; performing unsupervised clustering to identify populations, wherein the unsupervised clustering receives output from the pre-trained feature encoder; implementing a classifier to fine-tune supervised classification; and using the classifier to perform real-time classification of cells during active sorting. The feature encoder detects and measures feature values ​​from the cell images. The feature encoder is implemented using a neural network. The feature encoder is scalable to accommodate 1 to 12 image channels. Performing unsupervised clustering includes classifying cells in the cell images into clusters. The method also includes manually or automatically determining the cell populations to be sorted based on the results of the unsupervised clustering. After unsupervised clustering, a user labels clusters based on viewed clusters and representativeness information, wherein clusters are labeled as "sorted" or "not sorted." The classifier uses the classifier results from the unsupervised clustering to fine-tune a convolutional neural network. The classifier is configured to be retrained for each experiment.

[0009] In another aspect, an apparatus includes a non-transitory memory for storing an application and a processor coupled to the memory, the application being used to: pre-train a feature encoder using cell images; perform unsupervised clustering to identify populations, wherein the unsupervised clustering receives output from the pre-trained feature encoder; implement a classifier to fine-tune supervised classification; and perform real-time classification of cells using the classifier during active sorting, the processor being configured to process the application. The feature encoder detects and measures feature values ​​from the cell images. The feature encoder is implemented using a neural network. The feature encoder is scalable to accommodate 1 to 12 image channels. Performing unsupervised clustering includes classifying cells in the cell images into clusters. The apparatus is also configured to automatically determine cell populations to be sorted based on the results of the unsupervised clustering. A user labels clusters after unsupervised clustering based on viewed clusters and representativeness information, where clusters are labeled as "sorted" or "not sorted". The classifier uses the classifier results from the unsupervised clustering to fine-tune a convolutional neural network. The classifier is configured to be retrained for each experiment.

[0010] In another aspect, a system includes a first device and a second device configured to acquire cell images, the second device being configured to: pre-train a feature encoder using the cell images; perform unsupervised clustering to identify populations, wherein the unsupervised clustering receives output from the pre-trained feature encoder; implement a classifier to fine-tune supervised classification; and perform real-time cell classification using the classifier during active sorting. The feature encoder detects and measures feature values ​​from the cell images. The feature encoder is implemented using a neural network. The feature encoder is scalable to accommodate 1 to 12 image channels. Performing unsupervised clustering includes classifying cells in the cell images into clusters. The system also includes manually or automatically determining the cell populations to be sorted based on the results of the unsupervised clustering. A user labels clusters after unsupervised clustering based on viewed clusters and representativeness information, wherein clusters are labeled as "sorted" or "not sorted." The classifier uses the classifier results from the unsupervised clustering to fine-tune a convolutional neural network. The classifier is configured to be retrained for each experiment. Attached Figure Description

[0011] Figure 1 The illustration shows an application example of an image activated cell sorter (IACS) according to some embodiments.

[0012] Figure 2 The diagram illustrates a classification workflow according to some embodiments.

[0013] Figure 3 The diagram illustrates a flowchart of a feature encoder and unsupervised clustering according to some embodiments.

[0014] Figure 4 The illustration shows a diagram of how labeled images are used to fine-tune supervised classification according to some embodiments.

[0015] Figure 5 The diagram illustrates real-time image classification using supervised classification according to some embodiments.

[0016] Figure 6 The diagram illustrates the overall configuration of a biosample analyzer according to some embodiments.

[0017] Figure 7 A block diagram of an exemplary computing device configured to implement a classification workflow according to some embodiments is shown. Detailed Implementation

[0018] Image-based classification workflows identify subpopulations in samples, allowing users to select the subpopulations to be purified and then fine-tune a supervised classification system that will make real-time sorting decisions.

[0019] The classification workflow is a solution that allows for flexible, image-based cell classification systems adapted to a variety of biological applications. Using unsupervised clustering to label cell images to fine-tune an efficient supervised classification network for real-time sorting decisions is a novel idea that improves the ease of use of image-based cell sorters.

[0020] The sorting workflow addresses an unmet need for flow cytometer-based cell sorting customers: it allows for cell sorting applications that require imaging that cannot be performed on traditional fluorescence-activated cell sorting, and it allows sorting and classification to be optimized for a variety of applications.

[0021] Optimized supervised classification networks enable real-time image classification, making image-based cell sorting possible. This allows applications typically performed using microscopy or high-content imaging (neither of which has sorting capabilities) to be used on cell sorters.

[0022] There are many potential applications for image-based cell sorting. The workflow uses unsupervised clustering to identify subpopulations present in a sample, and the user can then select one or more clusters containing cells of interest. A supervised classification system is then fine-tuned to identify the specific cells of interest that the user wants to sort.

[0023] The classification workflow involves performing unsupervised clustering using a pre-trained feature encoder that identifies common features in cell images. The resulting clusters represent subpopulations present in the samples. The user then selects clusters containing cells of interest and labels the cells within these clusters. The labeled cell images are then used for supervised fine-tuning of an optimized classifier, which is used for real-time cell sorting.

[0024] The classification workflow can be used with Image Activated Cell Sorting Instrument (IACS).

[0025] Figure 1 The illustrations show application examples of IACS according to some embodiments. IACS can be used for fluorescence localization (e.g., co-expression of markers), immunoflow cytometry (e.g., counting FISH spots), extracellular vesicles (e.g., exosomes), and cell activation reactions (e.g., FL texture (speckle, smooth, dotted)). Immunoflow cytometry applications are not possible using conventional cell counters because individual spots may not be detectable / distinguished. However, it is possible to use IACS for applications that distinguish and detect individual spots. In another example, for fluorescence localization, IACS can detect specific locations of different signals (e.g., red and green), while conventional cell counters can only provide the intensity of each signal but not the location of the signal within the cell.

[0026] Figure 2The diagram illustrates a classification workflow according to some embodiments. Image-based or IACS classification workflows include steps such as pre-sorting settings comprising individual elements / steps and active sorting. The pre-sorting settings allow for the identification of cells to be sorted and subsequent fine-tuning of the supervised classification network. The active sorting portion uses an optimized and fine-tuned supervised classification network to make real-time sorting decisions as cells flow through the device.

[0027] In step 200, a feature encoder is pre-trained. A specific number of cells from the experiment pass through the system. The feature encoder is able to indicate to the user what is in the sample. As the cells pass through the feature encoder, the feature encoder detects / measures feature values ​​from the image, and the output of the feature encoder is fed into unsupervised clustering. The feature encoder is scalable to accommodate 1-12 image channels (e.g., bright field and multiple fluorescence channels). Any number or range of image channels can be used. An example of a feature encoder is described in U.S. Patent Application Serial No. 17 / 222,131, filed April 5, 2021, entitled “A FRAMEWORK FOR IMAGE BASED UNSUPERVISED CELL CLUSTERING AND SORTING,” the entire contents of which are incorporated herein by reference for all purposes.

[0028] In step 202, unsupervised clustering is used to identify populations (e.g., grouping similar cells together). Unsupervised clustering uses the output of a feature encoder to group / classify cells into clusters. Cell grouping can be performed in any manner (e.g., based on detected features, size, and / or any other property). The user can view the results of unsupervised clustering to determine the cell populations to be sorted. The user can view a representative set of images of a single event and / or the feature values ​​of the extracted clusters (e.g., fluorescence intensity or other feature values). The user can label clusters based on the clusters viewed and the representativeness information (e.g., after unsupervised clustering). For example, the user can label clusters as "sorted" or "not sorted". Multiple "sorted" clusters are possible. In some embodiments, the labeling process is automated using machine learning, neural networks, and artificial intelligence.

[0029] In step 204, the classifier is implemented to fine-tune supervised classification. The classifier can use classifier results / labels from unsupervised clustering to fine-tune shallow or convolutional neural networks, for example. The classifier can be retrained for each experiment.

[0030] In step 206, real-time classification is performed during active sorting. A trained classifier is used for real-time classification based on sorting. Unlike conventional cell sorters that use a single-channel detector to measure signal intensity, the classification workflow processes the entire image (e.g., 50 pixels × 50 pixels). In some embodiments, fewer or additional steps are implemented. In some embodiments, the order of the steps is modified.

[0031] Figure 3 The diagram illustrates a flowchart of a feature encoder and unsupervised clustering according to some embodiments. In step 300, cell images from the sorted samples are acquired. A fraction of the samples to be sorted is transported to an IACS to capture cell images. The cell images are input to a neural network-based feature encoder. In step 302, the feature encoder (e.g., a neural network) is implemented. In some embodiments, the feature encoder is pre-trained (e.g., offline) to detect general features in bright-field and fluorescence images through a large dataset of cell images. In step 304, unsupervised clustering (e.g., based on hierarchical density) is performed. The resulting cell clusters represent different subpopulations of cells present in the sample. In step 306, the user examines the clusters (e.g., representative images, measured image features) to determine which clusters contain cells of interest. The advantage of this approach is that it allows users to identify and select populations to be sorted (and not sorted) regardless of the application type and specific fluorescence staining panel used in the experiment. The clusters are analogous to a ground truth set that can be used to train supervised classification. Labeled images from the clusters are used to train (or retrain) the supervised classification network. Once a supervised classification network is trained and optimized for accuracy and speed, it can be used for live sorting (e.g., cells via a system). In live sorting, images of cells are captured, and supervised classification determines whether the cells are to be sorted for real-time classification to physically sort the cells, which are then separated by a purifier.

[0032] Figure 4 The diagram illustrates the use of labeled images to fine-tune supervised classification according to some embodiments. Labeled images 400 are used to fine-tune a supervised classification neural network 402 for a specific experiment. In some embodiments, the supervised classification neural network 402 is the same as or a portion of a trained classifier 204. A high-speed classification network is used to make sorting decisions fast enough to achieve real-time performance with the desired accuracy.

[0033] Figure 5The diagram illustrates real-time image classification using supervised classification according to some embodiments. A trained classifier 204 is used to make real-time sorting decisions for active sorting 206. The remaining sorted samples travel on the instrument (with cells arriving one at a time). Multiple image channels can be acquired for each cell. The trained classifier 204 classifies each event into one of the sorting groups or does not sort it based on the image. The sorting decision is made in the time it takes for a cell to move from the imaging region to the sorting-driven region (e.g., within less than 500 microseconds).

[0034] Experiments can consist of one image per cell or multiple images per cell (e.g., four image channels used per cell). The system can scale based on the implementation scheme (e.g., based on the number of active image channels used). Different numbers of image channels may use different architectures, so instances are trained on all possible numbers of image channels, and the appropriate optimized architecture is selected based on the active image channels for each experiment. The workflow can accept any type of input from any application, so users do not need to write different rules for different experiments.

[0035] Figure 6 The diagram illustrates the overall configuration of a biosample analyzer according to some embodiments.

[0036] Figure 6 An example configuration of the biosample analyzer disclosed herein is shown. Figure 6 The illustrated biosample analyzer 6100 includes: a light irradiation unit 6101 that illuminates a biological sample S flowing in a flow channel C; a detection unit 6102 that detects the light generated by irradiating the biological sample S; and an information processing unit 6103 that processes information about the light detected by the detection unit. For example, the biosample analyzer 6100 is a flow cytometer or an imaging cytometer. The biosample analyzer 6100 may include a sorting unit 6104 for separating specific biological particles P from the biological sample. For example, the biosample analyzer 6100 including the sorting unit is a cell sorter.

[0037] (Biological sample)

[0038] The biological sample S can be a liquid sample containing biological particles. For example, the biological particles can be cellular or non-cellular biological particles. Cells can be living cells, and more specific examples include blood cells such as red blood cells and white blood cells, and reproductive cells such as sperm and fertilized eggs. Furthermore, cells can be cells collected directly from a sample such as whole blood, or can be cultured cells obtained after culturing. Non-cellular biological particles, for example, are extracellular vesicles, or particularly exosomes and microvesicles. Biological particles can be labeled with one or more labeling substances, such as dyes (especially fluorescent dyes) and antibodies labeled with fluorescent dyes. Note that particles other than biological particles can be analyzed by the biosample analyzer of this disclosure, and beads, etc., can be analyzed for calibration, etc.

[0039] (Flow channel)

[0040] The flow channel C is designed to form a flow of biological sample S. Specifically, the flow channel C can be designed to form a flow in which biological particles contained in the biological sample are substantially arranged in a row. The flow channel structure including the flow channel C can be designed to form laminar flow. Specifically, the flow channel structure is designed to form a laminar flow in which the flow of the biological sample (sample flow) is surrounded by the flow of sheath fluid. The design of the flow channel structure can be suitably chosen by those skilled in the art, or a known design can be employed. The flow channel C can be formed in a flow channel structure such as a microchip (a chip with micron-sized flow channels) or a flow cell. The width of the flow channel C is 1 mm or less, or specifically, it can be not less than 10 μm and not greater than 1 mm. The flow channel C and the flow channel structure including the flow channel C can be made of materials such as plastic or glass.

[0041] The biosample analyzer of this disclosure is designed so that biological samples flow in a flow channel C, or specifically, so that biological particles in the biological sample are irradiated by light from the light irradiation unit 6101. The biosample analyzer of this disclosure can be designed such that the irradiation point on the biological sample is located within the flow channel structure forming the flow channel C, or it can be designed such that the irradiation point is located outside the flow channel structure. An example of the former could be a configuration in which light is emitted onto the flow channel C in a microchip or flow cell. In the latter case, biological particles leaving the flow channel structure (particularly its nozzle portion) can be irradiated with light, for example, using an air-jet flow cytometer.

[0042] (Light Illumination Unit)

[0043] The light illumination unit 6101 includes a light source unit that emits light and a light-guiding optical system that guides the light to an illumination point. The light source unit includes one or more light sources. The type of light source is, for example, a laser light source or an LED. The wavelength of the light emitted from each light source can be any wavelength of ultraviolet, visible, and infrared light. The light-guiding optical system includes, for example, optical components such as beam splitters, mirrors, or optical fibers. The light-guiding optical system may also include a lens group for focusing the light, and, for example, an objective lens. There may be one or more illumination points where the biological sample and the light intersect. The light illumination unit 6101 can be designed to collect light emitted from one or different light sources to an illumination point.

[0044] (Detection unit)

[0045] The detection unit 6102 includes at least one photodetector for detecting light generated by emitting light onto the biological particle. For example, the light to be detected can be fluorescence or scattered light (such as one or more of the following: forward-scattered light, back-scattered light, and side-scattered light). For example, each photodetector includes one or more light-receiving elements and has an array of light-receiving elements. Each photodetector may include one or more photomultiplier tubes (PMTs) and / or photodiodes, such as APDs and MPPCs, as light-receiving elements. For example, the photodetector includes a PMT array, wherein multiple PMTs are arranged in a one-dimensional orientation. The detection unit 6102 may also include an image sensor, such as a CCD or CMOS. Using the image sensor, the detection unit 6102 can acquire images of the biological particle (e.g., bright-field images, dark-field images, or fluorescence images).

[0046] The detection unit 6102 includes a detection optical system that directs light of a predetermined detection wavelength to a corresponding photodetector. The detection optical system includes a beam-splitting unit such as a prism or diffraction grating, or a wavelength-separating unit such as a dichroic mirror or optical filter. The detection optical system is, for example, designed to disperse light generated by illumination onto biological particles and detect the dispersed light using a photodetector that is larger than the number of fluorescent dyes labeling the biological particles. Flow cytometers that include such detection optical systems are called spectroscopic flow cytometers. Furthermore, the detection optical system is, for example, designed to separate light corresponding to the fluorescence wavelength band of a specific fluorescent dye from the light generated by illumination of biological particles and to detect the separated light using a corresponding photodetector.

[0047] The detection unit 6102 may further include a signal processing unit that converts the electrical signal obtained by the photodetector into a digital signal. The signal processing unit may include an A / D converter as a device for performing the conversion. The digital signal obtained by the conversion performed by the signal processing unit can be transmitted to the information processing unit 6103. The digital signal can be processed by the information processing unit 6103 as light-related data (hereinafter also referred to as "optical data"). For example, the optical data may be optical data including fluorescence data. More specifically, the optical data may be data on light intensity, and the light intensity may be light intensity data including fluorescence (the light intensity data may include characteristic quantities such as area, height, and width).

[0048] (Information Processing Unit)

[0049] The information processing unit 6103 includes, for example, a processing unit that performs processing of various types of data (e.g., optical data) and a storage unit that stores various types of data. When the processing unit acquires optical data corresponding to a fluorescent dye from the detection unit 6102, the processing unit can perform fluorescence leakage correction (compensation process) on the light intensity data. In the case of a spectroscopic flow cytometer, the processing unit also performs a fluorescence separation process on the optical data and acquires the light intensity data corresponding to the fluorescent dye. The fluorescence separation process can be performed using, for example, the unmixing method disclosed in JP 2011-232259A. If the detection unit 6102 includes an image sensor, the processing unit can acquire morphological information about the biological particles based on the image acquired by the image sensor. The storage unit can be designed to store the acquired optical data. The storage unit can also be designed to further store spectral reference data to be used in the unmixing process.

[0050] In the case where the biosample analyzer 6100 includes a sorting unit 6104, described later, the information processing unit 6103 can determine whether to sort biological particles based on optical data and / or morphological information. The information processing unit 6103 then controls the sorting unit 6104 based on the determined result, and the biological particles can be sorted by the sorting unit 6104.

[0051] The information processing unit 6103 can be designed to output various types of data (e.g., optical data and images). For example, the information processing unit 6103 can output various types of data generated based on optical data (e.g., two-dimensional graphs or spectrograms). The information processing unit 6103 can also be designed to accept various types of data input and to accept user-gated graph processes. The information processing unit 6103 may include output units (e.g., a display) or input units (e.g., a keyboard) for performing outputs or inputs.

[0052] The information processing unit 6103 can be designed as a general-purpose computer and can be designed as an information processing device including, for example, a CPU, RAM, and ROM. The information processing unit 6103 can be included in a housing that includes a light irradiation unit 6101 and a detection unit 6102, or it can be located outside the housing. Furthermore, various processes or functions to be performed by the information processing unit 6103 can be implemented via a server computer or the cloud connected to a network.

[0053] (Sorting Unit)

[0054] The sorting unit 6104 performs sorting of biological particles based on the determined result executed by the information processing unit 6103. The sorting method may involve generating droplets containing biological particles through vibration, applying a charge to the droplets to be sorted, and controlling the direction of droplet travel via electrodes. The sorting method may also involve sorting by controlling the direction of travel of biological particles in a flow channel structure. The flow channel structure may, for example, have a control mechanism based on pressure (injection or aspiration) or charge. An example of a flow channel structure may be a chip with a flow channel structure (e.g., the chip disclosed in JP 2020-76736A), wherein the flow channel C branches downstream into a recovery flow channel and a waste flow channel, and specific biological particles are collected in the recovery flow channel.

[0055] Figure 7 A block diagram of an exemplary computing device configured to implement a classification workflow according to some embodiments is shown. The computing device 700 is capable of acquiring, storing, calculating, processing, communicating, and / or displaying information such as images and videos. The computing device 700 is capable of implementing any aspect of the classification workflow. Generally, suitable hardware architectures for implementing the computing device 700 include a network interface 702, memory 704, a processor 706, I / O devices 708, a bus 710, and a storage device 712. The choice of processor is not critical, as long as a suitable processor with sufficient speed is selected. Memory 704 can be any conventional computer memory known in the art. Storage device 712 can include hard disk drives, CD-ROMs, CDRWs, DVDs, DVDRWs, high-definition discs / drives, ultra-HD drives, flash memory cards, or any other storage device. The computing device 700 can include one or more network interfaces 702. Examples of network interfaces include network interface cards (NICs) connected to Ethernet or other types of LANs. I / O devices 708 can include one or more of the following: keyboard, mouse, monitor, screen, printer, modem, touchscreen, button interface, and other devices. The classification workflow application 730 used to implement the classification workflow may be stored in storage device 712 and memory 704 and processed as an application would normally be processed. Figure 7More or fewer components as shown can be included in the computing device 700. In some embodiments, this includes categorized workflow hardware 720. Although Figure 7 The computing device 700 includes an application 730 and hardware 720 for classifying workflows, which can be implemented on the computing device in hardware, firmware, software, or any combination thereof. For example, in some embodiments, the classification workflow application 730 is programmed in memory and executed using a processor. In another example, in some embodiments, the classification workflow hardware 720 is programmable hardware logic including gates specifically designed to implement the classification workflow.

[0056] In some embodiments, the categorized workflow application 730 includes multiple applications and / or modules. In some embodiments, a module also includes one or more sub-modules. In some embodiments, fewer or additional modules can be included.

[0057] Examples of suitable computing devices include personal computers, laptop computers, computer workstations, servers, mainframe computers, handheld computers, personal digital assistants, cellular / mobile phones, smart appliances, game consoles, digital cameras, digital camcorders, camera phones, smartphones, portable music players, tablets, mobile devices, video players, video disc burners / players (e.g., DVD burners / players, high-definition disc burners / players, ultra-high-definition disc burners / players), televisions, home entertainment systems, augmented reality devices, virtual reality devices, smart jewelry (e.g., smartwatches), vehicles (e.g., autonomous vehicles), or any other suitable computing device.

[0058] To utilize the classification workflow described herein, a device such as a microscope with a camera is used to acquire content, and the device is capable of processing the acquired content. The classification workflow can be implemented with user assistance or automatically without user intervention.

[0059] In operation, a classification workflow is used for IACS. This workflow can also be used with spectroscopic cell sorters or other conventional cell sorters. The advantage of this classification workflow, which combines unsupervised clustering and supervised classification, is that it reduces bias and human error that can occur when using current methods of manually sequentially gating to identify subpopulations of interest. IACS performs high-speed imaging of cells, enabling new applications that are not possible with traditional flow cytometry. IACS combines the high throughput of flow cytometry with the high content information of microscopy. IACS sorts cell subpopulations to isolate / purify these populations for downstream experiments. Cell sorting occurs in less than 500 microseconds.

[0060] Some examples of flexible image-based particle sorting classification workflows

[0061] 1. A method comprising:

[0062] Use a cell image pre-trained feature encoder;

[0063] Perform unsupervised clustering to identify groups, wherein the unsupervised clustering receives the output from a pre-trained feature encoder;

[0064] Implement a classifier for fine-tuning supervised classification; and

[0065] The classifier is used to perform real-time cell classification during active sorting.

[0066] 2. The method as described in Clause 1, wherein the feature encoder detects and measures feature values ​​from the cell image.

[0067] 3. The method as described in Clause 1, wherein the feature encoder is implemented using a neural network.

[0068] 4. The method as described in Clause 1, wherein the feature encoder is scalable to accommodate 1 to 12 image channels.

[0069] 5. The method as described in Clause 1, wherein performing the unsupervised clustering includes classifying the cells of the cell image into clusters.

[0070] 6. The method as described in Clause 1 further includes manually or automatically determining the cell population to be sorted based on the results of the unsupervised clustering.

[0071] 7. The method as described in Clause 5, wherein the user labels the clusters based on viewing the clusters and representativeness information after the unsupervised clustering, wherein the clusters are labeled as “sorted” or “not sorted”.

[0072] 8. The method as described in Clause 1, wherein the classifier uses classifier results from the unsupervised clustering to fine-tune the convolutional neural network.

[0073] 9. The method as described in Clause 1, wherein the classifier is configured to be retrained for each experiment.

[0074] 10. An apparatus comprising:

[0075] Non-transitory memory for storing applications, which are used for:

[0076] Use a cell image pre-trained feature encoder;

[0077] Perform unsupervised clustering to identify groups, wherein the unsupervised clustering receives the output from a pre-trained feature encoder;

[0078] Implement a classifier for fine-tuning supervised classification; and

[0079] The classifier is used during active sorting to perform real-time cell classification; and

[0080] A processor coupled to the memory, the processor being configured to process the application.

[0081] 11. The apparatus of Clause 10, wherein the feature encoder detects and measures feature values ​​from a cell image.

[0082] 12. The apparatus as described in Clause 10, wherein the feature encoder is implemented using a neural network.

[0083] 13. The apparatus as described in Clause 10, wherein the feature encoder is scalable to accommodate 1 to 12 image channels.

[0084] 14. The apparatus as described in Clause 10, wherein performing the unsupervised clustering includes classifying the cells of the cell image into clusters.

[0085] 15. The apparatus as described in Clause 10 further automatically determines the cell population to be sorted based on the results of the unsupervised clustering.

[0086] 16. The apparatus as described in Clause 15, wherein a user labels the clusters based on viewing the clusters and representativeness information after the unsupervised clustering, wherein the clusters are labeled as “sorted” or “not sorted”.

[0087] 17. The apparatus of claim 10, wherein the classifier uses classifier results from the unsupervised clustering to fine-tune the convolutional neural network.

[0088] 18. The apparatus as described in Clause 10, wherein the classifier is configured to be retrained for each experiment.

[0089] 19. A system comprising:

[0090] The first device, configured to acquire cell images; and

[0091] The second device is configured to:

[0092] Use a cell image pre-trained feature encoder;

[0093] Perform unsupervised clustering to identify groups, wherein the unsupervised clustering receives the output from a pre-trained feature encoder;

[0094] Implement a classifier for fine-tuning supervised classification; and

[0095] The classifier is used to perform real-time cell classification during active sorting;

[0096] 20. The system as described in Clause 19, wherein the feature encoder detects and measures feature values ​​from a cell image.

[0097] 21. The system as described in Clause 19, wherein the feature encoder is implemented using a neural network.

[0098] 22. The system as described in Clause 19, wherein the feature encoder is scalable to accommodate 1 to 12 image channels.

[0099] 23. The system as described in Clause 19, wherein performing the unsupervised clustering includes classifying the cells of the cell image into clusters.

[0100] 24. The system as described in Clause 19 further includes manually or automatically determining the cell population to be sorted based on the results of the unsupervised clustering.

[0101] 25. The system as described in Clause 23, wherein a user labels the clusters based on viewing the clusters and representativeness information after the unsupervised clustering, wherein the clusters are labeled as “sorted” or “not sorted”.

[0102] 26. The system as described in Clause 19, wherein the classifier uses classifier results from the unsupervised clustering to fine-tune the convolutional neural network.

[0103] 27. The system as described in Clause 19, wherein the classifier is configured to be retrained for each experiment.

[0104] The present invention has been described with reference to specific embodiments, which incorporate details that aid in understanding the construction and operating principles of the invention. Such reference to specific embodiments and their details herein is not intended to limit the scope of the appended claims. It will be apparent to those skilled in the art that various other modifications may be made to the selected embodiments for illustration without departing from the spirit and scope of the invention as defined in the claims.

Claims

1. A method comprising: Use a cell image pre-trained feature encoder; Performing unsupervised clustering to identify a population, wherein the unsupervised clustering receives output from a pre-trained feature encoder, wherein performing the unsupervised clustering includes classifying cells in the cell image into clusters, wherein a user labels the clusters after the unsupervised clustering based on viewing the clusters and representativeness information, wherein the clusters are labeled as "sorted" or "unsorted"; Implement a classifier to fine-tune supervised classification using the labeled cell images; as well as During active sorting, the classifier is used to perform real-time cell classification by processing an entire image of each cell as the cells travel through the device.

2. The method of claim 1, wherein the feature encoder detects and measures feature values ​​from the cell image.

3. The method of claim 1, wherein the feature encoder is implemented using a neural network.

4. The method of claim 1, wherein the feature encoder is scalable to accommodate 1 to 12 image channels.

5. The method of claim 1, further comprising manually or automatically determining the cell population to be sorted based on the results of the unsupervised clustering.

6. The method of claim 1, wherein the classifier uses classifier results from the unsupervised clustering to fine-tune the convolutional neural network.

7. The method of claim 1, wherein the classifier is configured to be retrained for each experiment.

8. An apparatus comprising: Non-transitory memory for storing applications, which are used for: Use a cell image pre-trained feature encoder; Performing unsupervised clustering to identify a population, wherein the unsupervised clustering receives output from a pre-trained feature encoder, wherein performing the unsupervised clustering includes classifying cells in the cell image into clusters, wherein a user labels the clusters after the unsupervised clustering based on viewing the clusters and representativeness information, wherein the clusters are labeled as "sorted" or "unsorted"; Implement a classifier to fine-tune supervised classification using the labeled cell images; as well as During active sorting, the classifier is used to perform real-time cell classification by processing an entire image of each cell as the cells travel through the device. as well as A processor coupled to the memory, the processor being configured to process the application.

9. The apparatus of claim 8, wherein the feature encoder detects and measures feature values ​​from the cell image.

10. The apparatus of claim 8, wherein the feature encoder is implemented using a neural network.

11. The apparatus of claim 8, wherein the feature encoder is scalable to accommodate 1 to 12 image channels.

12. The apparatus of claim 8 further comprises automatically determining the cell population to be sorted based on the results of the unsupervised clustering.

13. The apparatus of claim 8, wherein the classifier uses classifier results from the unsupervised clustering to fine-tune the convolutional neural network.

14. The apparatus of claim 8, wherein the classifier is configured to be retrained for each experiment.

15. A system comprising: The first device is configured to acquire cell images; as well as The second device is configured to: Use a cell image pre-trained feature encoder; Performing unsupervised clustering to identify a population, wherein the unsupervised clustering receives output from a pre-trained feature encoder, wherein performing the unsupervised clustering includes classifying cells in the cell image into clusters, wherein a user labels the clusters after the unsupervised clustering based on viewing the clusters and representativeness information, wherein the clusters are labeled as "sorted" or "unsorted"; Implement a classifier to fine-tune supervised classification using the labeled cell images; as well as During active sorting, the classifier is used to perform real-time cell classification by processing an entire image of each cell as the cells travel through the device.

16. The system of claim 15, wherein the feature encoder detects and measures feature values ​​from the cell image.

17. The system of claim 15, wherein the feature encoder is implemented using a neural network.

18. The system of claim 15, wherein the feature encoder is scalable to accommodate 1 to 12 image channels.

19. The system of claim 15, further comprising manually or automatically determining the cell population to be sorted based on the results of the unsupervised clustering.

20. The system of claim 15, wherein the classifier uses classifier results from the unsupervised clustering to fine-tune the convolutional neural network.

21. The system of claim 15, wherein the classifier is configured to be retrained for each experiment.

Images