Method for classifying cells

Abstract

The present disclosure provides example embodiments for automatically or semi-automatically classifying cells in microscopic images of biological samples. These embodiments include methods of selecting training sets for the development of a classifier model. The disclosed selection embodiments can allow for retraining of a classifier model using training examples that have undergone the same or similar culture conditions as a target sample. These selection embodiments can reduce the human effort required to specify the training examples. The disclosed embodiments also include classifying individual cells based on metrics determined for the cells using phase contrast imaging and out-of-focus brightfield imaging. These metrics can include size, shape, texture, and intensity-based metrics. These metrics are determined based on segmentation of the underlying imagery. In some embodiments, the segmentation is based on phase contrast imaging and / or out-of-focus brightfield imaging of the biological sample.

Description

Technical Field

[0001] This application is an international application claiming priority to U.S. Application No. 17 / 099,983, filed November 17, 2020, which is incorporated herein by reference. Also incorporated by reference are U.S. Application No. 16 / 265,910, filed February 1, 2019, and U.S. Application No. 16 / 950,368, filed November 17, 2020. Background Technology

[0002] Current methods for segmenting cells in biological specimens require fluorescently labeled proteins; for example, marker-controlled segmentation algorithms set thresholds for nuclear localization proteins (such as histones). Alternative label-free techniques exist, such as layer-by-layer imaging, transverse shearing interferometry, and digital holography, but these require complex image acquisition settings and sophisticated image formation algorithms, and are time-consuming. Another label-free technique involves deep learning algorithms (e.g., convolutional neural networks), which require extensive training on large image datasets and are slow to process. Other methods use bright-field images under defocus conditions, which requires specialized hardware such as pinhole apertures and does not allow for cell-by-cell segmentation.

[0003] Cell classification in microscopic images (e.g., cell classification whose location and extent in the image have been determined by segmentation) facilitates a variety of applications, including assessing the effects of various experimental conditions by quantifying the impact of these conditions based on increases or decreases in the number of cells present in the sample, and / or the proportion of cells corresponding to various conditions (e.g., the ratio of differentiated to undifferentiated). Cell classification can be performed manually; however, such manual classification can be time-consuming and labor-intensive, and may lead to inaccurate cell classification. Automated methods are also available; however, these methods may require fluorescently labeled proteins, which could disrupt the natural biology of the cells, or may require a large set of training examples to train the automated algorithm. Summary of the Invention

[0004] In one aspect, an exemplary method for cell classification is disclosed. The method includes: (i) obtaining an image set of a plurality of biological samples, wherein the image set includes at least one image of each of the plurality of biological samples; (ii) obtaining an indication of a first cell group in the plurality of biological samples and an indication of a second cell group in the plurality of biological samples, wherein the first cell group is associated with a first condition and the second cell group is associated with a second condition; (iii) determining a first plurality of metric groups based on the image set, the indication of the first cell group, and the indication of the second cell group, wherein the first plurality of metric groups includes: a metric group for each cell in the first cell group and a metric group for each cell in the second cell group; (iv) generating a model based on the first plurality of metric groups using a supervised learning algorithm to distinguish cells in the first cell group and cells in the second cell group, thereby generating a trained model; (v) determining a second plurality of metric groups based on the image set, wherein the second plurality of metric groups includes: a metric group for each cell present in a target sample; and (vi) classifying cells in the target sample, wherein classifying cells includes applying the trained model to the metric groups of cells.

[0005] In another aspect, an exemplary method for cell classification is provided. The method includes: (i) acquiring three or more images of a target sample, wherein the target sample comprises one or more cells centered on a focal plane for the target sample, wherein the three or more images include: a phase-contrast image, a first bright-field image, and a second bright-field image, wherein the first bright-field image represents an image of the target sample focused above the focal plane at a first defocus amount, and wherein the second bright-field image represents an image of the target sample focused below the focal plane at a second defocus amount; (ii) determining cell images of the target sample based on the first bright-field image and the second bright-field image; (iii) determining a target segmentation map of the target sample based on the cell images and the phase-contrast image; (iv) determining a metric group for each cell present in the target sample based on two or more images of the target sample and the target segmentation map; and (v) classifying the cells in the target sample, wherein classifying the cells includes applying the cell metric group to a trained classifier.

[0006] In another aspect, an exemplary method for cell classification is provided. The method includes: (i) obtaining two or more images of a target sample, wherein the target sample comprises one or more cells centered on a focal plane for the target sample, wherein the two or more images include: a phase-contrast image and one or more bright-field images, wherein the one or more bright-field images include at least one bright-field image representing an image of the target sample not focused at the focal plane; (ii) determining a metric group for each cell present in the target sample based on the two or more images; and (iii) classifying the cells in the target sample by applying a trained model to the metric group of the cells.

[0007] In another aspect, a non-transitory computer-readable medium is provided, configured to store at least computer-readable instructions that, when executed by one or more processors of a computing device, cause the computing device to perform controller operations to perform any of the methods described above.

[0008] In another aspect, a system for analyzing biological specimens is provided, comprising: (i) an optical microscope; (ii) a controller, wherein the controller includes one or more processors; and (iii) a non-transitory computer-readable medium configured to store at least computer-readable instructions that, when executed by the controller, cause the controller to perform controller operations to perform any of the methods described above.

[0009] The features, functions, and advantages already discussed can be implemented independently in various examples or combined in other examples, and further details can be found in the following description and figures. Attached Figure Description

[0010] The patent or application documents contain at least one color drawing. A copy of the color drawing of this patent or patent application publication will be provided by the Patent Office upon request and payment of the necessary fees.

[0011] Figure 1 This is a functional block diagram of a system based on an exemplary implementation;

[0012] Figure 2 A block diagram depicts a computing device and a computer network according to an exemplary implementation;

[0013] Figure 3 A flowchart of a method according to an exemplary implementation is shown;

[0014] Figure 4 Images of biological specimens according to an exemplary implementation are shown;

[0015] Figure 5 An image of another biological specimen according to an exemplary implementation is shown;

[0016] Figure 6A Experimental results are shown in a cell-by-cell segmentation mask of cell image response 24 hours after apoptosis in HT1080 fibrosarcoma following camptothecin (CPT, cytotoxic) treatment, generated according to an exemplary implementation.

[0017] Figure 6B It shows according to Figure 6AThe implementation is based on cell subpopulations classified by red (Nuclight Red, a cell health indicator, "NucRed") and green fluorescence (Caspase 3 / 7, an apoptosis indicator);

[0018] Figure 6C It shows according to Figure 6A After CPT treatment, the red population decreased (indicating loss of live cells), red and green fluorescence increased (indicating early apoptosis), and green fluorescence increased after 24 hours (indicating late apoptosis).

[0019] Figure 6D It shows according to Figure 6A The implementation of the concentration response time process of the early apoptotic population (showing the percentage of total cells with red and green fluorescence);

[0020] Figure 6E Experimental results are shown in a cell-by-cell segmentation mask of cell image response 24 hours after the time course of apoptosis in HT1080 fibrosarcoma following cyclohexylamide (CHX, which inhibits cell growth) treatment, generated according to an exemplary implementation.

[0021] Figure 6F It shows according to Figure 6E The implementation is based on cell subpopulations classified by red (Nuclight Red, a cell health indicator, "NucRed") and green fluorescence (Caspase 3 / 7, an apoptosis indicator);

[0022] Figure 6G It shows according to Figure 6E Following CHX treatment, apoptosis was absent, but cell count decreased.

[0023] Figure 6H It shows according to Figure 6E The implementation of the concentration response time process of the early apoptotic population (showing the percentage of total cells with red and green fluorescence);

[0024] Figure 7A The per-cell segmentation mask applied to a phase-contrast image for label-free cell counting of adherent cells is illustrated using a per-cell segmentation analysis generated according to an exemplary implementation. Both label-free per-cell analysis and erythrocyte nuclear counting analysis were used to analyze A549 cells labeled with NucLight Red reagent at different densities to verify label-free counts over time;

[0025] Figure 7B This shows the situation where there is no contrasting image in the background. Figure 7A Cell-by-cell segmentation mask;

[0026] Figure 7C It shows according to Figure 7A The implementation of phase counting and NucRed counting data at different densities;

[0027] Figure 7D It shows according to Figure 7A The implementation of the data showed the correlation of the count data over 48 hours and demonstrated that the R² value was 1 when the slope was 1.

[0028] Figure 8 A flowchart of a method according to an exemplary implementation is shown;

[0029] Figure 9 A flowchart of a method according to an exemplary implementation is shown;

[0030] Figure 10 A flowchart of a method according to an exemplary implementation is shown;

[0031] Figure 11 Exemplary microscopic images and associated exemplary segmentation diagrams are shown;

[0032] Figure 12A An exemplary annotated micrograph is shown;

[0033] Figure 12B An exemplary annotated micrograph is shown;

[0034] Figure 13 An exemplary schematic diagram of the holes in a porous sample plate is shown;

[0035] Figure 14A and Figure 14B This demonstrates the experimental prediction accuracy of the method described in this paper;

[0036] Figure 15A and Figure 15B This demonstrates the experimental prediction accuracy of the method described in this paper; and

[0037] Figure 16A , Figure 16B and Figure 16C The experimental prediction accuracy of the method described in this paper is demonstrated compared to label-based classification.

[0038] The accompanying drawings are for illustrative purposes, but it should be understood that the invention is not limited to the arrangements and means shown in the drawings. Detailed Implementation

[0039] I. Overview

[0040] Microscopic imaging of biological samples facilitates a variety of analyses of the sample contents and their responses to various experimental conditions. Such analyses may include counting cells after cell sorting to determine the effect of applied conditions. For example, a sample may include groups of differentiated and undifferentiated cells, and analysis of the sample may include determining the proportion of differentiated cells, for example, to determine the effectiveness of applied conditions in causing undifferentiated cells to differentiate. To perform this analysis, each cell within the sample needs to be located and then sorted. This sorting process can be performed manually. However, manual sorting is very expensive, time-consuming, and can lead to inaccurate sorting.

[0041] The embodiments described herein demonstrate various methods for automatically classifying cells based on phase-contrast images, bright-field images, composites of phase-contrast and / or bright-field images, or other microscopic images of cells. Some of these embodiments involve training a model to classify cells using a specific set of cells from one or more biological samples. This trained model can then be applied to additional cells to classify those additional cells. To classify a specific cell, a set of cell metrics is determined based on one or more images representing the cell. Such metrics may include metrics related to cell size and / or shape. Such metrics may additionally or alternatively relate to cell texture or intensity, as represented in one or more phase-contrast images, bright-field images, fluorescence images, or composite images. For example, one or more metrics may relate to cell texture (e.g., variability and / or variable structure of brightness or intensity on cell regions) in a fluorescence image or some other type of image (e.g., phase-contrast, bright-field). The determined set of cell metrics can then be applied to a trained model to classify cells.

[0042] Cell groups used to train a model can be identified in a variety of ways. In some examples, cells can be manually indicated by the user. This could include the user manually indicating all the wells of a multi-well sample plate. Additionally or alternatively, the user could manually indicate individual cells in one or more biological samples. In another example, the user could specify time points to indicate cell groups; for example, setting a first time point before which all cells in the sample belong to a first group (e.g., undifferentiated group), and setting a second time point after which all cells in the sample belong to a second group (e.g., differentiated group). In some examples, cells can be indicated automatically or semi-automatically. This could include identifying cell groups based on the fluorescence images of the cells (e.g., cells with fluorescence signals above a threshold could be assigned to a first group, while cells with fluorescence signals below a threshold could be assigned to a second group). In yet another example, unsupervised or semi-supervised learning algorithms could cluster or otherwise group cells into groups that could then be used to train a classifier.

[0043] II. Exemplary Architecture

[0044] Figure 1 This is a block diagram illustrating an operating environment 100, which includes or relates to, for example, an optical microscope 105 and a biological specimen 110 having one or more cells. The following description... Figures 3 to 5 , Figure 8 , Figure 9 and Figure 10 Methods 300, 800, 900, and 1000 illustrate embodiments of methods that can be implemented in the operating environment 100.

[0045] Figure 2 This is a block diagram illustrating an example of a computing device 200 according to an exemplary implementation, configured to interact directly or indirectly with an operating environment 100. The computing device 200 can be used to perform... Figures 3 to 5 , Figure 8 , Figure 9 and Figure 10 The functions of the methods shown and described below. Specifically, computing device 200 may be configured to perform one or more functions, including, for example, an image generation function based in part on images obtained by optical microscope 105. Computing device 200 has a processor 202 and also has a communication interface 204, a data memory 206, an output interface 208, and a display 210, each connected to a communication bus 212. Computing device 200 may also include hardware to enable communication within computing device 200 and between computing device 200 and other devices (e.g., not shown). For example, the hardware may include a transmitter, a receiver, and an antenna.

[0046] Communication interface 204 may be a wireless interface and / or one or more wired interfaces, allowing short-range and long-range communication to one or more networks 214 or one or more remote computing devices 216 (e.g., tablet 216a, personal computer 216b, laptop 216c, and mobile computing device 216d). Such a wireless interface may provide communication under one or more wireless communication protocols, such as Bluetooth, WiFi (e.g., IEEE 802.11), LTE, cellular communication, Near Field Communication (NFC), and / or other wireless communication protocols. Such a wired interface may include an Ethernet interface, a Universal Serial Bus (USB) interface, or a similar interface for communication via wires, twisted pairs, coaxial cables, optical links, fiber optic links, or other physical connections to a wired network. Therefore, communication interface 204 may be configured to receive input data from one or more devices and may also be configured to send output data to other devices.

[0047] The communication interface 204 may also include user input devices, such as a keyboard, keypad, touch screen, touchpad, computer mouse, trackball and / or other similar devices.

[0048] Data storage 206 may include or take the form of one or more computer-readable storage media that can be read or accessed by processor 202. The computer-readable storage medium may include volatile and / or non-volatile storage components, such as optical, magnetic, organic, or other memory or disk storage devices, which may be integrated wholly or partially with processor 202. Data storage 206 is considered a non-transitory computer-readable medium. In some examples, data storage 206 may be implemented using a single physical device (e.g., a single optical, magnetic, organic, or other memory or disk storage unit), while in other examples, data storage 206 may be implemented using two or more physical devices.

[0049] Data storage 206 is therefore a non-transitory computer-readable storage medium, and executable instructions 218 are stored thereon. Instructions 218 include computer-executable code. When instructions 218 are executed by processor 202, processor 202 performs functions. Such functions include, but are not limited to, receiving bright-field images from optical microscope 100 and generating phase-contrast images, confluence masks, cell images, seed masks, cell-by-cell segmentation masks, and fluorescence images.

[0050] Processor 202 may be a general-purpose processor or a special-purpose processor (e.g., a digital signal processor, an application-specific integrated circuit, etc.). Processor 202 may receive input from communication interface 204 and process the input to generate output stored in data memory 206 and output to display 210. Processor 202 may be configured to execute executable instructions 218 (e.g., computer-readable program instructions) stored in data memory 206 and executable to provide the functionality of computing device 200 described herein.

[0051] Output interface 208 outputs information to display 210, or to other components. Therefore, output interface 208 can be similar to communication interface 204, and can be a wireless interface (e.g., a transmitter) or a wired interface. For example, output interface 208 can send commands to one or more controllable devices.

[0052] Figure 2The computing device 200 shown may also represent a local computing device 200a in the operating environment 100, for example, communicating with the optical microscope 105. This local computing device 200a can perform one or more steps of methods 300, 800, 900, and 1000 described below, and can receive input from a user and / or send image data and user input to the computing device 200 to perform all or some steps of methods 300, 800, 900, and / or 1000. Furthermore, in an optional exemplary embodiment, The platform can be used to perform one or more of methods 300, 800, 900, and 1000, and includes the combined functionality of computing device 200 and optical microscope 105.

[0053] Figure 3 A flowchart is shown of an exemplary method 300 for performing cell-by-cell segmentation of one or more cells of a biological specimen 110 according to an exemplary implementation. Figure 8 , Figure 9 and Figure 10 Flowcharts are shown for exemplary methods 800, 900, and 1000, respectively, for performing cell-by-cell classification of one or more cells in a biological specimen 110 according to exemplary embodiments. For example, Figure 3 , Figure 8 , Figure 9 and Figure 10 Methods 300, 800, 900, and 1000 shown demonstrate how they can be used with... Figure 2 An example of a method used with computing device 200. Furthermore, the device or system can be used or configured to perform... Figure 3 , Figure 8 , Figure 9 and / or Figure 10 The logical functions are illustrated. In some cases, components of a device and / or system can be configured to perform functions, such that the components are configured and constructed with hardware and / or software to achieve this performance. Components of a device and / or system can be arranged such that, when operated in a particular manner, they are suited, capable of, or adapted to perform functions. Methods 300, 800, 900, and 1000 may include one or more operations, functions, or actions as illustrated in one or more of the boxes in these figures (e.g., boxes 305 to 330). Although these boxes of each method are shown in sequential order in each figure, some of these boxes may also be performed in parallel and / or in an order different from that described herein. Furthermore, multiple boxes may be combined into fewer boxes, split into additional boxes, and / or deleted based on the target embodiment.

[0054] It should be understood that, for the processes and methods disclosed herein, as well as other processes and methods, the flowchart illustrates the function and operation of one possible implementation of this example. In this respect, each box may represent a module, segment, or portion of program code, which includes one or more instructions executable by a processor to implement a specific logical function or step in the process. The program code may be stored on any type of computer-readable medium or data storage, such as storage devices including disks or hard disk drives. Furthermore, the program code may be encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of art. Computer-readable media may include non-transitory computer-readable media or storage, such as computer-readable media that store data for short periods, such as register memory, processor cache, and random access memory (RAM). Computer-readable media may also include non-transitory media, such as secondary or permanent long-term storage devices, such as read-only memory (ROM), optical disks or magnetic disks, and optical disc read-only memory (CD-ROM). Computer-readable media may also be any other volatile or non-volatile storage system. Computer-readable media can be considered, for example, tangible computer-readable storage media.

[0055] also, Figure 3 , Figure 8 , Figure 9 , Figure 10 Each box in the present disclosure, as well as each box in other processes and methods disclosed herein, may represent a circuit that is wired to perform a specific logical function of the process. Alternative implementations are included within the scope of the examples of this disclosure, wherein, depending on the function involved, the functions may not be performed in the order shown or discussed, including substantially simultaneously or in reverse order, as will reasonably be understood by those skilled in the art.

[0056] III. Exemplary Methods

[0057] As used in this article, a "bright-field image" refers to an image obtained by microscopy based on a biological sample illuminated from below (so that light waves pass through the transparent parts of the biological sample). Thus, varying levels of brightness are captured in a bright-field image.

[0058] As used herein, a “phase-contrast image” refers to an image obtained directly or indirectly via a microscope based on a biological sample illuminated from below, capturing the phase shift of light passing through the biological sample due to differences in refractive index across different parts of the sample. For example, as light waves pass through a biological specimen, the amplitude (i.e., brightness) and phase of the light waves change in a manner dependent on the characteristics of the biological specimen. As a result, a phase-contrast image has brightness intensity values ​​associated with pixels, the variations of which render denser areas with higher refractive indices darker in the resulting image, while sparser areas with lower refractive indices are rendered brighter. Phase-contrast images can be generated using various techniques, including generation from a Z-stack of bright-field images.

[0059] As used in this article, “Z-stack” or “Z-scan” for brightfield images refers to a digital image processing method that combines multiple images taken at different focal lengths to provide a composite image with greater depth of field (i.e., thickness of the focal plane) compared to any single source brightfield image.

[0060] As used in this article, the "focal plane" refers to a plane perpendicular to the lens axis of an optical microscope, on which biological specimens can be observed at the optimal focal point.

[0061] As used in this article, “defocus” refers to the distance above or below the focal plane that allows a biological specimen to be observed in a defocused state.

[0062] As used in this article, a “merging mask” refers to a binary image in which pixels are identified as belonging to one or more cells in a biological specimen, such that pixels corresponding to one or more cells are assigned a value of 1, and the remaining pixels corresponding to the background are assigned a value of 0, and vice versa.

[0063] As used in this article, a “cell image” refers to an image generated based on at least two bright-field images obtained in different planes to enhance the contrast of cells relative to the background.

[0064] As used in this article, a “seed mask” refers to a binary pixelated image generated based on a set pixel intensity threshold.

[0065] As used herein, a “cell-by-cell segmentation mask” refers to an image with binary pixelation (i.e., the processor assigns a value of 0 or 1 to each pixel) such that the cells of biosample 110 are each displayed as distinct regions of interest. A cell-by-cell segmentation mask can advantageously allow for label-free counting of the cells displayed therein, allow for determination of the entire area of ​​a single adherent cell, allow for analysis based on cell texture metrics and cell shape descriptors, and / or allow for detection of the boundaries of individual cells, including for adherent cells that tend to form sheets, where each cell can contact many other neighboring cells in biosample 110.

[0066] As used in this paper, "region growing iteration" refers to a single step in an iterative image segmentation method in which the region of interest ("ROI") is iteratively expanded by acquiring one or more initially identified single or multiple sets of pixels (i.e., "seeds") and adding neighboring pixels to those sets. The processor uses a similarity metric to determine which pixels to add to the growing region, and a stopping criterion is defined for the processor to determine when region growing is complete.

[0067] As used herein, a “trained model” refers to a model (e.g., an artificial neural network, a Bayesian predictor, a decision tree) used for prediction and / or classification whose parameters (e.g., weights, filter bank coefficients), structure (e.g., the number of hidden layers and / or units, the interconnection pattern of such units), or other configured properties have been trained on a set of training data (e.g., through reinforcement learning, through gradient descent, through analysis to determine the maximum likelihood of model parameters) to generate outputs predicting the class classification of cells (e.g., live cells / dead cells, differentiated cells / undifferentiated cells).

[0068] Now for reference Figures 3 to 5 ,use Figures 1 to 2 The computing device illustrates method 300. Method 300 includes, at block 305, processor 202 generating at least one phase-contrast image 400 of a biological specimen 110, the biological specimen comprising one or more cells centered on a focal plane of the biological specimen 110. Then, at block 310, processor 202 generates a confluence mask 410 in the form of a binary image based on the at least one phase-contrast image 400. Next, at block 315, processor 202 receives a first bright-field image 415 of one or more cells in the biological specimen 110 at a defocused amount above the focal plane, and receives a second bright-field image 420 of one or more cells in the biological specimen 110 at a defocused amount below the focal plane. Then, at block 320, processor 202 generates a cell image 425 of the one or more cells in the biological specimen based on the first bright-field image 415 and the second bright-field image 420. At block 325, processor 202 generates a seed mask 430 based on the cell image 425 and the at least one phase-contrast image 400. In box 330, processor 202 generates an image of one or more cells in a biological specimen based on seed mask 430 and confluence mask 410, the image showing cell-by-cell segmentation mask 435.

[0069] like Figure 3As shown, in block 305, the processor 202 generating at least one phase-contrast image 400 of the biological specimen 110 (including one or more cells of the biological specimen 110 centered on the focal plane) includes: the processor 202 receiving a Z-scan of a bright-field image, and then generating at least one phase-contrast image 400 based on the Z-scan of the bright-field image. In various embodiments, the biological specimen 110 may be dispersed in multiple wells in a well plate representing an experimental group.

[0070] In an alternative embodiment, method 100 includes processor 202 receiving at least one fluorescence image and then calculating the fluorescence intensity of one or more cells in a biosample 110 within a cell-by-cell segmentation mask 435. In this embodiment, the fluorescence intensity corresponds to the level of a protein of interest, such as an antibody labeling a cell surface marker (e.g., CD20) or an annexin-V reagent that induces fluorescence corresponding to cell death. Furthermore, determining the fluorescence intensity within individual cell boundaries can enhance subpopulation identification and allow for the calculation of subpopulation-specific measures (e.g., the average area and eccentricity of all dead cells, as defined by the presence of annexin-V).

[0071] In another embodiment, at block 310, the processor 202 generates a confluence mask 410 in binary image form based on at least one contrasting image 400, including applying one or more local texture filters or brightness filters to enable the identification of pixels belonging to one or more cells in the biological specimen 110. Example filters may include, but are not limited to, local range filters, local entropy filters, local standard deviation filters, local brightness filters, and Gabor wavelet filters. Figure 4 and Figure 5 Example confluence mask 410 is shown.

[0072] In yet another alternative embodiment, the optical microscope 105 determines the focal plane of the biological specimen 110. Furthermore, in various embodiments, the defocusing amount can range from 20 μm to 60 μm. The optimal defocusing amount is determined based on the optical characteristics of the objective lens used, including its magnification and working distance.

[0073] exist Figure 5In another embodiment shown, at block 320, the processor 202 generates a cell image 425 based on a first bright-field image 415 and a second bright-field image 420 by: the processor 202 enhancing the first bright-field image 415 and the second bright-field image 420 based on a third bright-field image 405 centered on the focal plane, utilizing at least one of a plurality of pixel-level mathematical operations or feature detection. An example of pixel-level mathematical operations includes addition, subtraction, multiplication, division, or any combination of these operations. The processor 202 then calculates transformation parameters to align the first bright-field image 415 and the second bright-field image 420 with at least one contrasting image 400. Next, the processor 202 combines the brightness level of each pixel in the aligned second bright-field image 420 with the brightness level of the corresponding pixel in the aligned first bright-field image 415 to form the cell image 425. The combination of brightness levels of each pixel can be achieved by any of the mathematical operations described above. The technical effect of generating the cell image 425 is to remove bright-field artifacts (e.g., shadows) and enhance image contrast to increase cell detection of the seed mask 430.

[0074] In yet another alternative embodiment, at block 320, the processor 202 generates a cell image 425 of one or more cells in the biological specimen 110 based on the first bright-field image 415 and the second bright-field image 420, including: the processor 202 receiving one or more user-defined parameters that determine one or more threshold levels and one or more filter sizes. The processor 202 then applies one or more smoothing filters to the cell image 425 based on the one or more user-defined parameters. The technical effect of the smoothing filters is to further increase the accuracy of cell detection in the seed mask 430 and increase the likelihood that each cell will be assigned a seed. The smoothing filter parameters are selected to accommodate different adherent cell morphologies, such as flat vs. round, protruding cells, clustered cells, etc.

[0075] In another alternative embodiment, at block 325, the processor 202 generates a seed mask 430 based on the cell image 425 and at least one contrasting image 400, comprising: the processor 202 modifying the cell image 425 such that each pixel at or above a threshold pixel intensity is identified as a cell seed pixel, thereby producing a seed mask 430 with binary pixelation. The technical effect of binary pixelation of the seed mask is that it allows comparison with the corresponding binary pixelation of the confluence mask. The binary pixelation of the seed mask is also used as the starting point for the region growing iterations discussed below. For example, in yet another alternative embodiment, the seed mask 430 may have multiple seeds, each seed corresponding to a single cell in the biological specimen 110. In this embodiment, method 300 further includes, before processor 202 generates an image showing one or more cells in a biological specimen illustrating cell-by-cell segmentation mask 435, processor 202 compares seed mask 430 and confluence mask 410, and eliminates one or more regions from seed mask 430 that are not arranged in confluence mask 410, and eliminates one or more regions from confluence mask 410 that do not contain one of a plurality of seeds from seed mask 430. The technical effect of these eliminated regions is to exclude small, bright objects (e.g., cell debris) that generate seeds, and to increase the recognition of seeds used in the region growth iterations described below.

[0076] In another alternative embodiment, at block 330, the processor 202 generates an image of one or more cells in a biological specimen 110 showing a cell-by-cell segmentation mask 435 based on the seed mask 430 and the confluence mask 410, comprising: the processor 202 performing a region growth iteration for each of the active seed sets. The processor 202 then repeats the region growth iteration for each seed in the active seed set until the growth region of a given seed reaches one or more boundaries of the confluence mask 410 or overlaps with the growth region of another seed. The processor 202 selects an active seed set for each iteration based on attributes of corresponding pixel values ​​in the cell image. Furthermore, the technical effect of using at least one phase-contrast image 400 and bright-field images 415, 420, 405 is that the seeds correspond to bright spots in the cell image 425 and high-texture regions in the phase-contrast image 400 (i.e., the overlap between the confluence mask 410 and the seed mask 430 will be described in more detail below). Another technical effect produced by using the confluence mask 410, at least one phase-contrast image, and bright-field images 415, 420, 405 is the increased accuracy in identifying individual cell locations and cell boundaries in the cell-by-cell segmentation mask 435. As an example, this advantageously allows for the quantification of features such as cell surface protein expression.

[0077] In yet another alternative embodiment, method 300 may include processor 202 applying one or more filters in response to user input to remove objects based on one or more cell texture metrics and cell shape descriptors. Processor 202 then modifies an image of a biological specimen displaying a cell-by-cell segmentation mask in response to the application of the one or more filters. Example cell texture metrics and cell shape descriptors include, but are not limited to, cell size, perimeter, eccentricity, fluorescence intensity, aspect ratio, solidity, Feret's diameter, phase contrast entropy, and phase contrast standard deviation.

[0078] In another alternative embodiment, method 300 may include processor 202 determining a cell count of the biosample 110 based on an image of one or more cells in the biosample 110 illustrating a cell-by-cell segmentation mask 435. As a result of the defined cell boundaries shown in the cell-by-cell segmentation mask 435, the aforementioned cell counting is advantageously permitted, for example... Figure 4 As shown. In one alternative embodiment, one or more cells in the biological specimen 110 are one or more of adherent and non-adherent cells. In another embodiment, adherent cells may include: one or more different cancer cell lines, including human lung cancer cells, fibroblast cells, breast cancer cells, and ovarian cancer cells; or human microvascular cell lines, including human umbilical vein cells. In an alternative embodiment, processor 202 performs region growth iterations such that different smoothing filters are applied to non-adherent cells (including human immune cells, such as PMBC and Jurkat cells) instead of adherent cells to improve the approximation of cell boundaries.

[0079] As an example, a non-transitory computer-readable medium stores program instructions thereon that, when executed by processor 202, perform a set of actions including generating at least one phase-contrast image 400 of the biological specimen 110, comprising one or more cells, based on at least one bright-field image 405 centered on the focal plane of the biological specimen 110. Then, processor 202 generates a confluence mask 410 in the form of a binary image based on the at least one phase-contrast image 400. Next, processor 202 receives a first bright-field image 415 of one or more cells in the biological specimen 110 at a defocused amount above the focal plane and a second bright-field image 420 of one or more cells in the biological specimen 110 at a defocused amount below the focal plane. Then, processor 202 generates a cell image 425 of one or more cells based on the first bright-field image 415 and the second bright-field image 420. Processor 202 also generates a seed mask 430 based on the cell image 425 and the at least one phase-contrast image 400. Furthermore, the processor 202 generates images of one or more cells in the biological specimen 100 based on the seed mask 430 and the confluence mask 410, the images showing the cell-by-cell segmentation mask 435.

[0080] In an alternative embodiment, the non-transitory computer-readable medium further includes enabling the processor 202 to receive at least one fluorescence image and enabling the processor 202 to calculate the fluorescence intensity of one or more cells in a biological specimen within a cell-by-cell segmentation mask.

[0081] In another alternative embodiment, the non-transitory computer-readable medium further includes enabling processor 202 to generate a seed mask 430 based on cell image 425 and at least one contrasting image 400. The non-transitory computer-readable medium also includes enabling processor 202 to modify cell image 410 such that each pixel at or above a threshold pixel intensity is identified as a cell seed pixel, thereby causing seed mask 430 to have binary pixelation.

[0082] In yet another alternative embodiment, the seed mask 430 has a plurality of seeds, each seed corresponding to a single cell. The non-transitory computer-readable medium also includes, before the processor 202 generates an image showing one or more cells in the biological specimen 110 of the cell-by-cell segmentation mask 435, causing the processor 202 to compare the seed mask 430 and the confluence mask 410, and to eliminate one or more regions from the seed mask 430 that are not arranged in the confluence mask 410, and to eliminate one or more regions from the confluence mask 410 that do not contain one of the plurality of seeds of the seed mask 430.

[0083] In another alternative embodiment, the program instructions that cause processor 202 to generate an image showing one or more cells in a biological specimen 110 illustrating a cell-by-cell segmentation mask 435 based on seed mask 430 and confluence mask 410 include: processor 202 performing a region growth iteration for each of the active seed sets. The non-transitory computer-readable medium then further includes causing processor 202 to repeat the region growth iteration for each seed in the active seed set until the growth region of a given seed reaches one or more boundaries of confluence mask 410 or overlaps with the growth region of another seed.

[0084] The non-transitory computer-readable medium also includes enabling processor 202 to apply one or more filters in response to user input to remove objects based on one or more cell texture metrics and cell shape descriptors. Furthermore, processor 202 modifies an image of biological specimen 110 showing a cell-by-cell segmentation mask 435 in response to the application of one or more filters.

[0085] Now for reference Figure 8 ,use Figures 1 to 2The computing device illustrates an exemplary method 800 for cell classification. Method 800 includes, at block 805, a processor (e.g., processor 202) obtaining an image set of multiple biological samples, wherein the image set includes at least one image of each of the multiple biological samples. Then, at block 810, the processor obtains an indication of a first cell group in the multiple biological samples and an indication of a second cell group in the multiple biological samples, wherein the first cell group is associated with a first condition and the second cell group is associated with a second condition. Next, at block 815, the processor determines a first plurality of measurement groups based on the image set, the indication of the first cell group, and the indication of the second cell group, wherein the first plurality of measurement groups includes a measurement group for each cell in the first cell group and a measurement group for each cell in the second cell group. At block 820, the processor generates a model based on the first plurality of measurement groups using a supervised learning algorithm to distinguish cells in the first cell group and cells in the second cell group, thereby generating a trained model. At block 825, the processor determines a second plurality of measurement groups based on the image set, wherein the second plurality of measurement groups includes a measurement group for each cell present in the target sample. Then, in box 830, the processor classifies the cells in the target sample, wherein classifying the cells includes applying a trained model to a set of cell metrics. Method 800 may include additional steps or features.

[0086] Now for reference Figure 9 ,use Figures 1 to 2 The computing device illustrates another exemplary method 900 for cell classification. Method 900 includes, at block 905, a processor (e.g., processor 202) acquiring three or more images of a target sample, wherein the target sample comprises one or more cells centered on a focal plane of the target sample, wherein the three or more images include a phase-contrast image, a first bright-field image, and a second bright-field image, wherein the first bright-field image represents an image of the target sample focused at a first defocus amount above the focal plane, and wherein the second bright-field image represents an image of the target sample focused at a second defocus amount below the focal plane. Then, at block 910, the processor determines cell images of the target sample based on the first bright-field image and the second bright-field image. Next, at block 915, the processor determines a target segmentation map of the target sample based on the cell images and the phase-contrast image. At block 920, the processor determines a metric group for each cell present in the target sample based on two or more images of the target sample and the target segmentation map. Then, at block 925, the processor classifies the cells in the target sample, wherein classifying the cells includes applying the cell metric group to a trained classifier. Method 900 may include additional steps or features.

[0087] Now for reference Figure 10 ,use Figures 1 to 2The computing device illustrates yet another exemplary method 1000 for cell classification. Method 1000 includes, in block 1005, a processor (e.g., processor 202) acquiring two or more images of a target sample, wherein the target sample comprises one or more cells centered on a focal plane of the target sample, wherein the two or more images include a phase-contrast image and one or more bright-field images, wherein the one or more bright-field images include at least one bright-field image representing an image of the target sample not focused on the focal plane. Then, in block 1010, the processor determines a metric group for each cell present in the target sample based on the two or more images. Next, in block 1015, the processor classifies the cells in the target sample by applying a trained model to the metric group of cells. Method 1000 may include additional steps or features.

[0088] As described above, the non-transitory computer-readable medium stores program instructions thereon, which, when executed by the processor 202, can perform any of the functions of the aforementioned method.

[0089] IV. Experimental Results

[0090] The example implementation allows tracking cell health across subpopulations over time. For example... Figure 6A Experimental results are shown in the cell-by-cell segmentation mask of the phase-contrast image response 24 hours after the time process of apoptosis in HT1080 fibrosarcoma following camptothecin (CPT, cytotoxic) treatment, generated according to an exemplary implementation. NucLight Red (nuclear active labeling) and interference-free Multiple readings of the Caspase 3 / 7 green reagent (apoptosis indicator) are used to determine cell health. Figure 6B It shows according to Figure 6A Implementation, use Cell-by-cell analysis software tool, classifying cell subpopulations based on red and green fluorescence. Figure 6C It shows according to Figure 6A According to the implementation plan, after CPT treatment, the red population decreased (indicating loss of live cells), red and green fluorescence increased (indicating early apoptosis), and green fluorescence increased after 24 hours (indicating late apoptosis). Figure 6D It shows according to Figure 6A The concentration-response time process of the early apoptotic population was observed (showing the percentage of total cells exhibiting red and green fluorescence). The values ​​shown are the mean ± SEM values ​​from 3 wells.

[0091] In another example, Figure 6EExperimental results are shown using a cell-by-cell segmentation mask generated according to an exemplary implementation, representing the cell image response 24 hours after the time course of apoptosis in HT1080 fibrosarcoma following cyclohexylamide (CHX, which inhibits cell growth) treatment. NucLight Red (nuclear active labeling) and interference-free Multiple readings of the Caspase 3 / 7 green reagent (apoptosis indicator) are used to determine cell health. Figure 6F It shows according to Figure 6E The implementation plan uses Cell-by-cell analysis software tool, classifying cell subpopulations based on red and green fluorescence. Figure 6G It shows according to Figure 6E The implementation scheme showed that apoptosis was absent after CHX treatment, but cell count was reduced (data not shown). Figure 6H It shows according to Figure 6E The implementation scheme, concentration response time process of the early apoptotic population (percentage of total cells showing red and green fluorescence). The values ​​shown are the mean ± SEM of 3 wells.

[0092] Figure 7A The use of, according to an exemplary implementation scheme, via Software-generated cell-by-cell segmentation analysis was performed using a cell-by-cell segmentation mask applied to a phase-contrast image for label-free cell counting of adherent cells. Both label-free cell-by-cell analysis and erythrocyte nuclear counting analysis were used to analyze A549 cells labeled with NucLight Red reagent at different densities to validate label-free counts over time. Figure 7B This shows the situation where there is no contrasting image in the background. Figure 7A Cell-by-cell segmentation mask. Figure 7C It shows according to Figure 7A The implementation scheme, the time process of phase count and red count data with different densities. Figure 7D It shows according to Figure 7A The implementation scheme was described, the correlation of count data over 48 hours was analyzed, and an R² value of 1 was shown when the slope was 1. This was replicated across a range of cell types. The values ​​shown are the mean ± SEM values ​​from 4 wells.

[0093] V. Example of Cell Classification

[0094] Cell classification algorithms based on sample images containing cells facilitate a variety of applications. This can include quantifying the properties of cells and / or cell samples, quantifying the cell sample's response to applied experimental conditions (e.g., the toxicity or efficacy of a hypothetical drug or treatment), or assessing other information about the sample. Cell classification facilitates such applications by allowing the determination of the number of cells of each class in the sample. Such classification can include two-class classifications or classifications into two or more classes. In some examples of classification, cells can be classified as live or dead cells, stem cells or mature cells, undifferentiated or differentiated cells, wild-type or mutant cells, epithelial or mesenchymal cells, normal cells or cells morphologically altered by an applied compound (e.g., cells altered by the application of a therapeutic compound targeting the cytoskeleton), or between two or more additional or alternative classifications. Cells can also be assigned multiple categories, selected from a corresponding set of multiple distinct enumerated categories. For example, cells can be classified as live cells (from the possible categories of "live cells" and "dead cells") and differentiated cells (from the possible categories of "differentiated cells" and "undifferentiated cells").

[0095] The embodiments described herein classify cells by determining a set of metrics for a specific cell. This set of metrics is determined from one or more microscopic images of the cell. Particularly useful in determining such metrics are one or more defocused bright-field images of the cell, or synthetic images determined therefrom and / or combined with phase-contrast images of the cell. For example, one or more metrics of the cell can be determined from each of the phase-contrast image of the cell and the cell image of the cell (as determined above). Determining this set of metrics typically involves segmenting the image to determine which part of the image corresponds to the cell. As described elsewhere herein, segmentation itself is determined based on one or more images. Furthermore, segmentation can be used to determine one or more metrics (e.g., cell size, one or more metrics related to cell shape, etc.). This set of metrics is then applied to a model to classify the cells.

[0096] Figure 11 An exemplary cell-by-cell segmentation mask (bright lines) is depicted applied to a phase-contrast image 1100 of a biological sample comprising a plurality of cells, including an example cell 1110. The cell-by-cell segmentation mask depicts a portion of the phase-contrast image 1100 corresponding to cell 1110; this is indicated by dark lines 1150 indicating the portion of the cell-by-cell segmentation mask corresponding to example cell 1110. The portion of the phase-contrast image 1100 within the dark lines 1150 can be used to determine one or more metrics of example cell 1110 (e.g., texture-related metrics, intensity-related metrics), and so can the portion 1150 of the cell-by-cell segmentation mask depicting example cell 1110 (e.g., size-related metrics, shape-related metrics).

[0097] One or more of the methods described above can be used to segment one or more microscopic images of a biological sample to locate cells within the sample. Additionally or alternatively, one or more microscopic images of the sample can be applied to a convolutional neural network that has been trained to generate such segmentation maps. This may include applying phase-contrast images of the sample and cell images.

[0098] Segmentation maps can be used to determine cell size metrics. This can include using segmentation maps to determine the cell's area, the number of pixels in the image occupied by the cell, the percentage of pixels and / or area in the image occupied by the cell, the cell's perimeter, the cell's maximum Feret diameter, or some other metric related to cell size.

[0099] Segmentation maps can also be used to determine one or more shape descriptor measures of a cell. Such shape descriptor measures may be the roundness of the cell, the roundness of the cell's convex hull, or the proportion of the cell's convex hull occupied by the cell, the cell's aspect ratio (i.e., the ratio of the cell's maximum length to its orthogonal axis), the cell's geographic centroid, the cell's intensity-weighted centroid or the difference between the two centroids, or some other measure related to cell shape.

[0100] Additional metrics may include those relating to cell texture and / or intensity, as shown in one or more microscopic images of the cell. Such microscopic images of the cell may include phase-contrast images, bright-field images, fluorescence images, or other images of the cell. Images may include composite images. As described above, such composite images may include cell images generated from two or more bright-field images focused on their respective different planes relative to the cellular contents of the biological sample. Another exemplary composite image is a composite of a phase-contrast image and one or more bright-field images (e.g., a composite of a phase-contrast image and a cell image). Determining such texture- or intensity-based metrics may include determining the metric based on a segmentation map, or based on image pixels corresponding to a specific cell.

[0101] Texture metrics can be determined from the variations and / or texture of the pixel group representing a cell. This can include calculating one or more metrics on a neighborhood basis; for example, for a given pixel, a texture value can be determined based on the pixel group surrounding the given pixel within a specified distance. Such neighborhood texture values ​​can then be averaged across the pixels of the cell to produce a total texture value for that cell. Such texture values ​​can include range values, variance or standard deviation, entropy, contrast values, uniformity values, and / or some texture-based measurements, where the range value is the difference between the maximum and minimum intensity values ​​within the pixel group, the contrast value is a measurement of local variations present in the pixel group, and the uniformity value is a measurement of uniformity within the pixel group.

[0102] Intensity-based metrics may include: the average brightness of cells in an image, the standard deviation of cell brightness in an image, the minimum brightness of cells in an image, the maximum brightness of cells in an image, the brightness of a specified percentile of cell pixels in an image, a measure of the kurtosis or skewness of the distribution of brightness values ​​on cells in an image, or some other metric based on the brightness or variation of cells in one or more images.

[0103] Once a set of metrics for a specific cell is determined, that set of metrics can be used to classify that cell. This may include applying the set of metrics to a trained model. Such a model may include one or more of principal component analysis, independent component analysis, support vector machines, artificial neural networks, lookup tables, regression trees, ensembles of regression trees, decision trees, ensembles of decision trees, k-nearest neighbor analysis, Bayesian inference, and logistic regression.

[0104] The model's output can be a simple indication of the cell's definitive category, with a set of metrics applied to the model. Alternatively, the model can output one or more values ​​indicating the cell's category. These values ​​can then be compared to a threshold to classify the cell. For example, if the model's output value is greater than the threshold, the cell can be classified as a "live cell," while if the model's output value is less than the threshold, the cell can be classified as a "dead cell." The value of this threshold can be determined by an algorithm, for example, as part of a process of training the model based on training data. Additionally or alternatively, the threshold can be set by the user. For example, the user can adjust the threshold based on visual feedback indicating the classification of cells in one or more microscopic images. The user can adjust the threshold after an initial threshold is generated via an algorithmic process.

[0105] Figure 12A and Figure 12B An example of a basic real-time iterative process or other iterative process is shown, in which the user adjusts a threshold and receives visual feedback on the effect of adjusting the cell classification in a biological sample. Figure 12A Elements of an example user interface during a first time period are depicted. The example user interface includes a first annotated image 1200a of the biological sample (e.g., an annotated phase-contrast image). The first annotated image 1200a is annotated to indicate cells in the sample and to indicate cell classification based on a first threshold value. Figure 12A As shown, the first cell category is represented in red, and the second cell category is represented in blue.

[0106] The threshold can then be updated to a second value via user input. This input may include the user pressing a real or virtual button to increase or decrease the threshold value, the user operating a keyboard or other device to input the threshold value, the user moving a slider or dial to adjust the threshold value, or the user engaging in other user input actions to adjust the threshold to the second value. The second value of the threshold is then applied to reclassify the cells in the sample. This reclassification is then visually presented to the user in the form of an updated second annotated image 1200b of the biological sample, as shown below. Figure 12B As shown, the second annotated image 1200b is annotated to indicate the cells in the sample and to indicate the cell classification according to a second value updated based on a threshold. The classification of some cells changes as the threshold is adjusted, and therefore the second annotated image 1200b reflects this change. This update process can be performed multiple times. For example, as a result of user-adjusted thresholds, the update process can be performed at a rate of once every 20 milliseconds or at some other rate to approximate real-time updates of cell classification.

[0107] Supervised training methods and a suitable training dataset can be used to train a model for cell classification. The training dataset includes a set of metrics determined for each cell in two or more sets of training cells. Each set of training cells corresponds to a specific category or set of categories that the model can be trained to distinguish. The set of metrics in the training dataset can be determined as described above by identifying the set of metrics for a specific training cell within a specific set based on one or more microscopic images of that specific training cell.

[0108] In some examples, training cells can be placed in the wells of the same multi-well sample plate containing target cells to be classified based on the training cells. This has the advantage of training the model on training cells that have already been exposed to the same or similar environment or other conditions as the target cells, without requiring manual annotation of a large number of individual cells. Alternatively, training cells can be placed in the wells of a first multi-well sample plate, and target cells can be placed in the wells of a second, different multi-cell sample plate. Such first and second multi-well sample plates can be cultured in the same incubator or exposed to the same or similar environmental conditions.

[0109] The various images and / or metrics used to train the model may be the same as or different from the various images and / or metrics applied to the trained model to classify unknown cells. For example, fluorescent markers may be present in biological samples containing training cells but may not be present in samples containing unknown target cells to be classified. This can improve model training while avoiding the complexity or confounding of adding fluorescent markers to the target samples. Additionally or alternatively, fluorescent markers may be used to assign training cells to appropriate groups prior to training the model.

[0110] Training cells in two (or more) training cell groups can be identified in several ways. In some examples, training cell groups can be manually identified by the user. This can include the user manually indicating individual cells in each of two or more groups. This indication can be performed using a user interface depicting an image of cells within a biological sample, whether the image has been segmented or not. Additionally or alternatively, the user can manually indicate the corresponding training category for all wells in a multi-well plate. Any cells detected in wells indicated in this manner will be assigned to the appropriate category for training the model. The user can indicate such wells based on knowledge of the well's condition. For example, a particular well may contain a substance that induces cell death, and the user could then indicate that such a well contains cells belonging to the "dead cell" category for training the model. The advantage of indicating training cell groups in this well-by-well manner is that it requires relatively little user time and effort (e.g., compared to the user indicating individual cells for training).

[0111] Figure 13 Elements of an exemplary user interface 1300 are depicted, which a user can use to indicate that one or more wells of a multi-well sample plate correspond to one of two or more categories, and then train a model to distinguish said categories. The user interface 1300 depicts the relative positions of the wells of the multi-well sample plate, where each well is represented by a corresponding square. Additional information about each well can be provided. This additional information may include information about the contents of the well, conditions applied to the well, an image of the contents of the well, or some other information. The user can then indicate groups of wells corresponding to the respective categories. As shown, the user has indicated a first group of wells 1310a as corresponding to a first category (e.g., the "live cells" category) and a second group of wells 1310b as corresponding to a second category (e.g., the "dead cells" category).

[0112] It should be noted that the indication of cell groups (e.g., by indicating a single cell, by indicating all wells of a multi-well sample plate, by consistently indicating cells in an automated or semi-automated manner) may include indicating cells at one or more designated time points. For example, indicating a first cell group may include indicating wells at a first time point (e.g., indicating a live cell group when all or most of the cells in the wells are live cells), and indicating a second cell group may include indicating the same wells at a second time point (e.g., indicating a dead cell group when all or most of the cells in the wells are dead cells).

[0113] Before using the obtained training data to train the model, the indicated cell groups or the thus determined metric groups may be filtered or otherwise modified. This is done to reduce the time or number of iterations required to fit the data, thereby producing a more accurate model without overfitting the training data, or improving the trained model and / or the training process. Such filtering or other preprocessing steps may include comprehensive balancing of the training cell groups, subsampling of the training cell groups, and / or normalizing the values ​​of the determined metrics (e.g., normalizing each determined metric so that the aggregate metric values ​​of all cells in the training data occupy a standard range and / or conform to a specified distribution).

[0114] Additionally or alternatively, training cell groups can be identified automatically or semi-automatically using algorithms or other methods. This can include identifying training cell groups using the presence or absence of fluorescent markers. This can include obtaining fluorescence images of biological samples containing fluorescent markers and, based on the fluorescence images, identifying first and second cell groups in the sample, respectively, according to whether the average fluorescence intensity of the cells is greater than or less than a threshold level.

[0115] In another example, an unsupervised training process can be used to classify cells in training images. This can include identifying two or more cell clusters in the training images. The user can then manually classify a limited number of cells into individual categories selected from a set of two or more categories. These manually classified cells can be cells that have already been clustered through the unsupervised training process, or they can be new cells. The manual classification can then be used to assign cell clusters to the appropriate categories within the set of two or more categories. The manual classification can be cell-by-cell, whole-well based, or some other method of manual cell classification.

[0116] Various advantageous arrangements have been described for purposes of illustration and description, but are not intended to be exhaustive or limited to the forms disclosed. Many modifications and variations will be apparent to those skilled in the art. Furthermore, different advantageous examples may describe different advantages compared to other advantageous examples. The selection and description of one or more examples are intended to best explain the principles and practical applications of the examples, and to enable those skilled in the art to understand the various examples of this disclosure and the various modifications suitable for the particular intended use.

[0117] VI. Experimental Classification Results

[0118] Cell classification is improved when one or more metrics are used, determined from cell images of cells (i.e., composite images determined from two or more off-focus bright-field images). Figure 14A and Figure 14BThe accuracy of cell classification as live or dead cells is shown in multiple samples treated with camptothecin (a cytotoxic compound that can cause cell death, "CMP") or the experimental control compound ("VEH"). Figure 13 A illustrates a classification based on a set of metrics determined by a cell-by-cell segmentation mask of the sample (e.g., area, perimeter) and a phase-contrast image of the sample (e.g., average phase-contrast brightness). Figure 13 B illustrates a classification based on the aforementioned metrics and additional metrics determined from cell images of the sample (e.g., average brightness of the cell images). Figure 14A and Figure 14B The overall accuracy for representing all cells increased from 0.82 to 0.94, with the F1 statistic increasing from 0.89 to 0.96 (using live cells as the "positive" category).

[0119] Figure 15A and Figure 15B This illustrates how the improved accuracy of cell sorting (live or dead cells) affects the determined cell mortality rate in some samples over time. Figure 15A The image shows a sample for which the determined cell mortality rate over time was determined based on a cell-by-cell segmentation mask (e.g., area, perimeter) and a phase-contrast image of the sample (e.g., average phase-contrast brightness). The red trajectory represents the rate determined by the trained model, while the blue trajectory represents the true rate. Figure 15B The sample used the above-described metrics, along with additional metrics determined from cell images of the sample (e.g., average brightness of the cell images), to determine the determined cell mortality rate over time from a trained model.

[0120] The classification method described in this article facilitates cell classification with accuracy approaching that of fluorescently labeled methods. This allows for accurate classification without the costs, complexity, or experimental confounding effects that can be associated with the use of fluorescent labeling. In the experiment, A549 cells were treated for 72 hours with escalating concentrations of the cytotoxic compound camptothecin (0.1–10 μM) in the presence of annexin V reagent. Cells were classified as dead or live based on the fluorescent annexin reaction (live cells = low fluorescence, dead cells = high fluorescence). The classification results based on annexin V are shown below. Figure 16A As shown. The metric-based method described in this paper is used to train a model using label-free features of dead cells (10 μM, 72 h) and live cells (excipient, 0–72 h). This model is then applied to classify cells as live or dead to obtain the percentage of dead cells corresponding to the annexin V response. The results of this label-free classification are shown below. Figure 16B As shown. Figure 16CThe superposition of 72-hour mortality percentage concentration response curves calculated using annexin V or a label-free method is shown, demonstrating that the predicted responses within this concentration range are comparable, and the EC50 values ​​are similar (annexin V EC50). 50 =6.6E-07; Unmarked EC50 = 5.3E-07M -1 ).

Claims

1. A method for classifying cells, the method comprising: A set of images of multiple biological samples is obtained, wherein each biological sample includes one or more first cells centered on a first focal plane, wherein for each biological sample, the set of images includes a first phase-contrast image and a first bright-field image not focused at the first focal plane; Obtain an indication of a first cell group in the plurality of biological samples and an indication of a second cell group in the plurality of biological samples, wherein the first cell group is associated with a first condition and the second cell group is associated with a second condition; Based on the image group, the indication of the first cell group, and the indication of the second cell group, a first plurality of measurement groups are determined, wherein the first plurality of measurement groups include: a measurement group for each cell of the first cell group and a measurement group for each cell of the second cell group; Based on the first plurality of metric groups, a supervised learning algorithm is used to generate a model to distinguish cells in the first cell group and cells in the second cell group, thereby generating a trained model; Based on the image set, a second plurality of measurement sets are determined, wherein the second plurality of measurement sets include a measurement set for each cell present in the target sample; and Classifying cells in the target sample, wherein classifying the cells includes: applying the trained model to the metric set of the cells. The target sample includes one or more second cells centered on the second focal plane, and the images in the image group depicting the target sample include a second phase-contrast image and a second bright-field image not focused at the second focal plane.

2. The method of claim 1, wherein applying the trained model to the set of metrics of the cell comprises: Generating model output values ​​based on the metric set of the cells, wherein classifying the cells further includes comparing the model output values ​​with a threshold.

3. The method according to claim 2, further comprising: Displaying an annotated image of the target sample, wherein the annotated image of the target sample includes indications of the cells and indications of the cell classification; Receive user input indicating that the threshold should be updated; The cells are reclassified by comparing the model output value with the updated threshold. as well as The updated annotated image of the target sample is displayed, wherein the updated annotated image of the target sample includes indications of the cells and indications of the reclassification of the cells.

4. The method of claim 1, wherein determining the set of metrics for the cell comprises determining at least one of the following: size metric, shape descriptor metric, texture metric, or intensity-based metric.

5. The method of claim 1, wherein the trained model comprises at least one of principal component analysis, independent component analysis, support vector machine, artificial neural network, lookup table, regression tree, regression tree ensemble, decision tree, decision tree ensemble, k-nearest neighbor analysis, Bayesian inference, and logistic regression.

6. The method of claim 1, wherein the one or more bright-field images include a first bright-field image and a second bright-field image, wherein the first bright-field image represents an image of the target sample focused above the focal plane at a first defocus amount, wherein the second bright-field image represents an image of the target sample focused below the focal plane at a second defocus amount, and wherein the method further comprises: The cell image of the target sample is determined based on the first bright-field image and the second bright-field image, wherein the set of metrics for determining the cells includes determining at least one metric based on the cell image.

7. The method of claim 1, wherein the fluorescent label is present in the cells of the first cell group and the cells of the second cell group, and wherein the fluorescent label is not present in the target sample.

8. The method according to claim 1, wherein the first cell group and the second cell group are both placed in the wells of the first porous sample plate, and wherein the target sample is placed in the wells of the second porous sample plate.

9. The method according to claim 1, wherein the first cell group, the second cell group and the target sample are all placed in the pores of the porous sample plate.

10. The method of claim 9, further comprising: The indication displays the relative positions of the pores in the porous sample plate, wherein the first cell group is present in the first pore group of the porous sample plate, and wherein the second cell group is present in the second pore group of the porous sample plate, wherein obtaining the indication of the first cell group and the indication of the second cell group includes: after displaying the indication of the relative positions of the pores in the porous sample plate, receiving user input indicating the relative positions of the first pore group and the second pore group within the porous sample plate.

11. The method according to claim 1, further comprising: Before generating the trained model, the first plurality of metric groups are preprocessed by at least one of the following operations: normalizing at least one metric in the first plurality of metric groups, comprehensively balancing the first plurality of metric groups between the metric groups of each cell in the first cell group and the metric groups of each cell in the second cell group, and subsampling the first plurality of metric groups.

12. The method of claim 1, wherein the first cell group and the second cell group comprise fluorescent labels, and wherein the image group of the plurality of biological samples comprises: At least one fluorescence image depicting the first cell group and at least one fluorescence image depicting the second cell group, wherein obtaining an indication of the first cell group within the plurality of biological samples includes: identifying the first cell group using at least one fluorescence image depicting the first cell group, and wherein Indications for obtaining the second cell group in the plurality of biological samples include: identifying the second cell group using at least one fluorescent image depicting the second cell group.

13. The method of claim 1, wherein classifying the cells in the target comprises: The cells are classified as at least one of the following: live cells or dead cells; stem cells or mature cells; epithelial cells or mesenchymal cells; and undifferentiated cells or differentiated cells.

14. A method for classifying cells, the method comprising: A set of images of multiple biological samples is obtained, wherein each biological sample includes one or more first cells centered on a first focal plane, wherein for each biological sample, the set of images includes a first phase-contrast image and a first bright-field image not focused at the first focal plane; Based on the image set, a supervised learning algorithm is used to generate a model to distinguish cells in the first cell group and cells in the second cell group, thereby generating a trained model. Obtain three or more images of a target sample, wherein the target sample includes one or more second cells centered on a focal plane for the target sample, wherein the three or more images include a phase-contrast image, a first bright-field image, and a second bright-field image, wherein the first bright-field image represents an image of the target sample focused above the focal plane at a first defocus amount, and wherein the second bright-field image represents an image of the target sample focused below the focal plane at a second defocus amount; The cell image of the target sample is determined based on the first bright-field image and the second bright-field image; The target segmentation map of the target sample is determined based on the cell image and the phase-contrast image; Based on three or more images of the target sample and the target segmentation map, determine the metric group for each cell present in the target sample; as well as Classifying cells in the target sample, wherein classifying the cells includes applying the metric set of the cells to the trained model.

15. The method of claim 14, wherein determining the set of metrics for the cell comprises determining at least one of the following: size metric, shape descriptor metric, texture metric, or intensity-based metric.

16. The method of claim 14, wherein determining the set of metrics for the cell comprises: At least one metric from the metric group of the cell is determined based on the phase-contrast image.

17. The method of claim 14, wherein determining the target segmentation map of the target sample based on the first bright-field image and the second bright-field image comprises: At least the first bright-field image and the second bright-field image, as well as the phase-contrast image, are applied to the convolutional neural network.

18. A method for classifying cells, the method comprising: A set of images of multiple biological samples is obtained, wherein each biological sample includes one or more first cells centered on a first focal plane, wherein for each biological sample, the set of images includes a first phase-contrast image and a first bright-field image not focused at the first focal plane; Based on the image set, a supervised learning algorithm is used to generate a model to distinguish cells in the first cell group and cells in the second cell group, thereby generating a trained model. Obtain two or more images of a target sample, wherein the target sample comprises one or more cells centered on a focal plane for the target sample, wherein the two or more images comprise: a phase-contrast image and one or more bright-field images, wherein the one or more bright-field images comprise at least one bright-field image representing an image of the target sample not focused at the focal plane; Based on the two or more images, determine the metric group for each cell present in the target sample; and The cells in the target sample are classified by applying the trained model to the set of metrics of the cells.

19. The method of claim 18, wherein the two or more images of the target sample comprise a first brightfield image and a second brightfield image, wherein the first brightfield image represents an image of the target sample focused above the focal plane at a first defocus amount, wherein the second brightfield image represents an image of the target sample focused below the focal plane at a second defocus amount, and wherein the method further comprises: The cell image of the target sample is determined based on the first bright-field image and the second bright-field image, wherein the set of metrics for determining the cells includes determining at least one metric based on the cell image.

20. A non-transitory computer-readable medium configured to store at least computer-readable instructions that, when executed by one or more processors of a computing device, cause the computing device to perform controller operations to perform the method according to any one of claims 1 to 19.

21. A system for analyzing biological specimens, the system comprising: Optical microscope; A controller, wherein the controller includes one or more processors; as well as A non-transitory computer-readable medium configured to store at least computer-readable instructions that, when executed by the controller, cause the controller to perform controller operations to perform the method according to any one of claims 1 to 19.

Images