Method for monitoring living cells

Abstract

A method for monitoring one or more live cells includes obtaining a non-fluorescent image of a sample comprising one or more live cells further comprising a fluorescent protein-based nuclear translocation reporter (FTR), obtaining a fluorescent image of the FTR in the live cells in the sample, identifying nuclear pixels of the non-fluorescent image corresponding to the nuclei of the live cells via a computational model, identifying a first pixel of the fluorescent image corresponding to the nuclei and a second pixel of the fluorescent image not corresponding to the nuclei based on the nuclear pixels, and calculating an indicator representing a first amount of FTR located within the nuclei of the live cells and a second amount of FTR not located within the nuclei of the live cells based on a first intensity of the first pixel and a second intensity of the second pixel.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is an international application claiming priority to U.S. patent application Ser. No. 17 / 740,313, filed May 9, 2022, the contents of which are incorporated herein by reference. [Background technology]

[0002] background

[0002] Numerous signaling pathways promote cell survival and proliferation, and their dysregulation leads to, for example, the development, progression, and recurrence of cancer. Standard methods for assessing intracellular signaling pathways are end-point assays that require cell lysis and often involve time-consuming sample preparation and / or analytical workflows. Summary of the Invention [Means for solving the problem]

[0003] overview

[0003] A first example includes a method for monitoring one or more living cells, the method including: (a) acquiring a non-fluorescent image of a sample including one or more living cells, the one or more living cells including a fluorescent protein-based nuclear translocation reporter; (b) acquiring a fluorescent image of the fluorescent protein-based nuclear translocation reporter in the one or more living cells in the sample; (c) identifying nuclear pixels in the non-fluorescent image corresponding to the nuclei of the one or more living cells via a computational model; (d) identifying a first pixel in the fluorescent image corresponding to the nuclei and a second pixel in the fluorescent image that does not correspond to the nuclei based on the nuclear pixels; and (e) calculating an index representing a first amount of the fluorescent protein-based nuclear translocation reporter located in the nuclei of the one or more living cells and a second amount of the fluorescent protein-based nuclear translocation reporter not located in the nuclei of the one or more living cells based on a first intensity of the first pixel and a second intensity of the second pixel.

[0004]

[0004] A second example, if executed by a computing device, causes the computing device to: (a) obtain a non-fluorescent image of a sample comprising one or more live cells that include a fluorescent protein-based nuclear translocation reporter in the one or more live cells; (b) obtain a fluorescent image of the fluorescent protein-based nuclear translocation reporter in the one or more live cells in the sample; (c) identify nuclear pixels of the non-fluorescent image corresponding to the nuclei of the one or more live cells via a computational model; (d) identify a first pixel of the fluorescent image corresponding to the nucleus and a second pixel of the fluorescent image not corresponding to the nucleus based on the nuclear pixels; and (e) calculate an indicator representing a first amount of the fluorescent protein-based nuclear translocation reporter located within the nuclei of the one or more live cells and a second amount of the fluorescent protein-based nuclear translocation reporter not located within the nuclei of the one or more live cells based on a first intensity of the first pixel and a second intensity of the second pixel. The non-transitory computer-readable medium stores instructions for causing the above functions to be executed.

[0005]

[0005] A third example includes an optical microscope, a fluorescence microscope, one or more processors, and a non-transitory computer-readable medium that stores instructions for causing the system, if executed by the one or more processors, to: (a) obtain a non-fluorescent image of a sample comprising one or more live cells that include a fluorescent protein-based nuclear translocation reporter in the one or more live cells; (b) obtain a fluorescent image of the fluorescent protein-based nuclear translocation reporter in the one or more live cells in the sample; (c) identify nuclear pixels of the non-fluorescent image corresponding to the nuclei of the one or more live cells via a computational model; (d) identify a first pixel of the fluorescent image corresponding to the nucleus and a second pixel of the fluorescent image not corresponding to the nucleus based on the nuclear pixels; and (e) calculate an indicator representing a first amount of the fluorescent protein-based nuclear translocation reporter located within the nuclei of the one or more live cells and a second amount of the fluorescent protein-based nuclear translocation reporter not located within the nuclei of the one or more live cells based on a first intensity of the first pixel and a second intensity of the second pixel.

[0006]

[0006] The fourth example includes a method for training a computational model. The method includes generating a first label for a first pixel of a fluorescence image of a sample, the first label indicating whether the first pixel represents a nucleus in the sample; based on the first label, generating a second label for a second pixel of a first non-fluorescence image of the sample, the second label indicating whether the second pixel represents a nucleus in the sample; and training a computational model using the second label and the first non-fluorescence image to identify pixels in the second non-fluorescence image that represent nuclei.

[0007]

[0007] The fifth example includes a non-transitory computer-readable medium that stores instructions which, when executed by a computing device, cause the computing device to perform functions including generating a first label for a first pixel of a fluorescence image of a sample, the first label indicating whether the first pixel represents a nucleus in the sample; based on the first label, generating a second label for a second pixel of a first non-fluorescence image of the sample, the second label indicating whether the second pixel represents a nucleus in the sample; and training a computational model using the second label and the first non-fluorescence image to identify pixels in the second non-fluorescence image that represent nuclei.

[0008]

[0008] The sixth example includes a system that includes one or more processors and a non-transitory computer-readable medium that stores instructions which, when executed by the one or more processors, cause the system to perform functions including generating a first label for a first pixel of a fluorescence image of a sample, the first label indicating whether the first pixel represents a nucleus in the sample; based on the first label, generating a second label for a second pixel of a first non-fluorescence image of the sample, the second label indicating whether the second pixel represents a nucleus in the sample; and training a computational model using the second label and the first non-fluorescence image to identify pixels in the second non-fluorescence image that represent nuclei.

[0009] As used herein, when the terms "substantially", "nearly", or "about" are used, it is not necessary to reach the recited characteristic, parameter, or value exactly, but rather means that deviations or variations, including for example, tolerances, measurement errors, limits of measurement accuracy, and other factors known to those skilled in the art, may occur to the extent that they do not prevent the effect intended by the subject characteristic. In some of the examples disclosed herein, "substantially", "nearly", or "about" means within ±0 to 5% of the recited value.

[0010]

[0010] These and other aspects, advantages, and alternatives will become apparent to those skilled in the art by reading the following detailed description, with appropriate reference to the accompanying drawings. Further, it should be understood that the summary and other descriptions presented herein, as well as the drawings, are intended for illustrative purposes only and thus various modifications are possible.

Brief Description of the Drawings

[0011] Brief Description of the Drawings

Figure 1

[0011] It is a block diagram of an operating environment according to an example.

Figure 2

[0012] It is a block diagram of a computing device according to an example.

Figure 3

[0013] It is a fluorescence image according to an example.

Figure 4

[0014] It is a binary map according to an example.

Figure 5

[0015] It is a binary map according to an example.

Figure 6

[0016] It is a non-fluorescence image according to an example.

Figure 7

[0017] It is a block diagram of a method for training a computational model according to an example.

Figure 8

[0018] It is a schematic representation of a non-fluorescence image according to an example.

Figure 9

[0019] It is a schematic representation of a fluorescence image according to an example.

Figure 10

[0020] A block diagram of a method for monitoring cells according to one example.

Figure 11

[0021] A phase image, a predicted nucleus map, and a predicted nucleus map are shown according to one example.

Figure 12

[0022] A scatter plot of predicted nucleus area versus target nucleus area according to one example.

Figure 13

[0023] A scatter plot of predicted nucleus area versus target nucleus area according to one example.

Figure 14

[0024] A scatter plot of the ratio of predicted nuclei to the ratio of target nuclei according to one example.

Figure 15

[0025] A scatter plot of the ratio of predicted nuclei to the ratio of target nuclei according to one example.

Figure 16

[0026] A scatter plot of the ratio of predicted nuclei to the ratio of target nuclei according to one example.

Figure 17

[0027] A scatter plot of the ratio of predicted nuclei to the ratio of target nuclei according to one example.

DETAILED DESCRIPTION OF THE INVENTION

[0012] Detailed Description

[0028] As described above, there is a need for an improved technique for evaluating signal transduction pathways in live cells. The present disclosure includes a method for monitoring one or more live cells. The method includes obtaining a non-fluorescent image (e.g., bright-field image, dark-field image, or phase-contrast image) of a sample containing one or more live cells. The one or more live cells include a fluorescent protein-based nuclear translocation reporter (FTR) (e.g., the cell membrane or cell wall of the one or more live cells surrounds the FTR). The fluorescent protein-based nuclear translocation reporter may be any such reporter that shuttles between the nucleus and / or outside the nucleus in response to a stimulus of interest, as described in more detail below. The method also includes obtaining a fluorescent image of the fluorescent protein-based nuclear translocation reporter in one or more live cells in the sample. This generally includes illuminating the one or more live cells and selectively detecting the fluorescence emitted by the protein-based nuclear translocation reporter. In one embodiment, the fluorescent image is obtained using the same field of view as the non-fluorescent image. The method also includes identifying nuclear pixels of the non-fluorescent image corresponding to the nucleus of the one or more live cells via a computational model. Any suitable computational model may be used, including but not limited to a vision transformer (ViT), a convolutional neural network, or another artificial neural network. The method also includes identifying a first pixel of the fluorescent image corresponding to the nucleus and a second pixel of the fluorescent image not corresponding to the nucleus based on the nuclear pixels. The method also includes calculating an indicator representing a first amount of the fluorescent protein-based nuclear translocation reporter located within the nucleus of the one or more live cells and a second amount of the fluorescent protein-based nuclear translocation reporter not located within the nucleus of the one or more live cells based on a first intensity of the first pixel and a second intensity of the second pixel. Thus, the region of the fluorescent image associated with the nucleus can be identified without using a separate fluorescent marker for nuclear labeling. This simplifies image analysis significantly and frees up fluorescent channels for the analysis of other aspects of the cell that are considered suitable for the intended applications (e.g., tracking the location and dynamics of proteins, organelles, and other cellular components).

[0013]

[0029] A computational model is generally trained to recognize nuclei in a non-fluorescent image before being used to classify pixels of an unlabeled non-fluorescent image. Thus, a method of training a computational model includes generating a first label for a first pixel of a fluorescent image of a sample, the first label indicating whether the first pixel represents a nucleus in the sample. The first label may take the form of a binary map and / or can be generated, for example, via thresholding. The method also includes generating a second label for a second pixel of a first non-fluorescent image of the sample based on the first label, the second label indicating whether the second pixel represents a nucleus in the sample. In one embodiment, this may be performed by applying a binary map to the first non-fluorescent image. The method also includes training a computational model using the second label and the first non-fluorescent image to identify pixels of the second non-fluorescent image that represent nuclei.

[0014]

[0030] FIG. 1 is a block diagram showing an exemplary operating environment 100 of the present disclosure, including a system 10 and a sample 110. The system 10 includes a computing device 200a and an optical assembly 103 including an optical microscope 105 and a fluorescence microscope 107. Also shown are a computing device 200 and a network 214, which are described in more detail below.

[0015]

[0031] FIG. 2 is a block diagram showing an exemplary computing device 200 configured to communicate directly or indirectly with the operating environment 100. In particular, the computing device 200 may be configured to perform one or more functions including an image generation function based in part on images acquired by the optical microscope 105 and / or the fluorescence microscope 107. The computing device 200 includes a processor 202 along with a communication interface 204, a data storage 206, an output interface 208, and a display 210, each of which is connected to a communication bus 212. The computing device 200 may also include hardware that enables communication within the computing device 200 and between the computing device 200 and other devices (not shown, for example). The hardware may include, for example, a transmitter, a receiver, and an antenna.

[0016]

[0032] The communication interface 204 may be a wireless interface and / or one or more wired interfaces that enable both short-range and long-range communication to one or more networks 214 or one or more remote computing devices 216 (e.g., such as tablet 216a, personal computer 216b, laptop computer 216c, and mobile computing device 216d). Such a wireless interface can operate under one or more wireless communication protocols such as Bluetooth, WiFi (e.g., Institute of Electrical and Electronics Engineers (IEEE) 802.11 protocol), LTE (Long-Term Evolution), cellular communication, Near Field Communication (NFC), and / or other wireless communication protocols. Such a wired interface may include an Ethernet interface, a Universal Serial Bus (USB) interface, or a similar interface that communicates via a wire, twisted pair wire, coaxial cable, optical link, fiber optic link, or other physical connection to a wired network. Thus, the communication interface 204 may be configured to receive input data from one or more devices and may also be configured to transmit output data to other devices.

[0017]

[0033] The communication interface 204 may also include user input devices such as, for example, a keyboard, keypad, touch screen, touch pad, computer mouse, trackball, and / or other similar devices.

[0018]

[0034] Data storage 206 may include or take the form of one or more computer-readable storage media that are readable or accessible by processor 202. The computer-readable storage media may include volatile and / or non-volatile storage elements such as optical, magnetic, organic, or other memory or disk storage, etc., and all or part of these can be integrated with processor 202. Data storage 206 is considered to be a non-transitory computer-readable medium. In some examples, data storage 206 can be implemented using a single physical device (e.g., one optical, magnetic, organic, or other memory or disk storage device), but in other examples, data storage 206 can be implemented using two or more physical devices.

[0019]

[0035] Data storage 206 thus includes executable instructions 218 that it stores. Instructions 218 include computer-executable code. If instructions 218 are executed by processor 202, processor 202 is caused to execute any of the functions described herein. Data storage 206 also includes a computational model 400 that it stores.

[0020]

[0036] Processor 202 may be a general-purpose processor or a dedicated processor (e.g., a digital signal processor, an application-specific integrated circuit, etc.). Processor 202 can receive inputs from communication interface 204, process these inputs, and generate output content that is stored in data storage 206 and output to display 210. Processor 202 may be configured to execute executable instructions 218 (e.g., computer-readable program instructions) stored in data storage 206 to provide the functions of computing device 200 described herein.

[0021]

[0037] The output interface 208 also provides information to the display 210 or other elements. Thus, the output interface 208 may be similar to the communication interface 204 and may be a wireless interface (e.g., a transmitter) or a wired interface. The output interface 208 can, for example, send commands to one or more controllable devices.

[0022]

[0038] The computing device 200 shown in FIG. 2 may represent a local computing device 200a within the operating environment 100 that is in communication with, for example, an optical microscope 105 and / or a fluorescence microscope 107. The local computing device 200a may perform one or more steps of the plurality of methods described below, may receive input from a user, and / or may send image data and user input to the computing device 200 to perform all or part of the steps of the plurality of methods. Also, in one optional exemplary embodiment, the plurality of methods can be performed using the Incucyte® platform, which includes a combination of the functions of the computing device 200, the optical microscope 105, and the fluorescence microscope 107.

[0023] Training of the computational model

[0039] The method of the present disclosure includes identifying nuclear pixels of a non-fluorescent image corresponding to the nuclei of one or more live cells via a computational model. The computational model is trained to identify nuclei without using nuclear labels.

[0024]

[0040] FIG. 3 shows exemplary functionality related to using a fluorescence image for training a computational model 400 to identify pixels of a non-fluorescent image corresponding to nuclei. The computational model 400 can be stored, for example, in the data storage 206 of the computing device 200.

[0025]

[0041] The computing device 200 can generate a first label 402 for a first pixel 404 of the fluorescence image 406 of the sample 408. The first label 402 indicates whether the first pixel 404 represents a nucleus in the sample 408. The fluorescence image 406 is generally acquired after or simultaneously with irradiating the sample 408 to cause the fluorescent nuclear marker in the sample 408 to emit light to indicate the positions of the nuclei in the sample 408. During the acquisition of the fluorescence image 406, light that does not correspond to the emission wavelength range of the fluorescent marker is generally filtered out, so that only the light fluoresced by the fluorescent marker (e.g., nucleus) is shown in the fluorescence image 406.

[0026]

[0042] FIG. 3 shows a fluorescence image 406 of a sample 408 including a plurality of (e.g., living) cells and corresponding nuclei represented by a first pixel 404 surrounded by a first label 402. The fluorescence image 406 in FIG. 3 presents an example of a plurality of fluorescence images 406 of a plurality of samples 408. The fluorescence image 406 also includes some first pixels 404 disposed outside the first label 402. In this example, the user manually checks a plurality of fluorescence images 406 and manually generates the first label 402 using a user interface (e.g., by drawing a rectangle around some of the first pixels 404 through a click-and-drag operation). Thus, the first label 402 can be generated in the form of metadata indicating the pixel positions corresponding to the nuclei of the cells. It can also generally be inferred that the first pixels 404 not marked by the first label 402 do not correspond to the nuclei of the cells. For example, such pixels may correspond to positions outside the cells or within the cytoplasm.

[0027]

[0043] FIG. 4 shows further exemplary functions related to the training of the computing model 400. More specifically, FIG. 4 shows the result of a more automated process of labeling the first pixels 404 of the fluorescence image 406 in the form of a binary map 410. The binary map 410 is a compressed form of the fluorescence image 406 shown in FIG. 3, as will be described later. The binary map 410 in FIG. 4 presents an example of a plurality of binary maps 410 corresponding to a plurality of fluorescence images 406.

[0028]

[0044] As shown, the computing device 200 generates a first label 402 for a first pixel 404 of the fluorescence image 406 of the sample 408. The first label 402 indicates whether the first pixel 404 represents a nucleus in the sample 408. That is, the first label 402 corresponds to the first pixel 404 that represents a nucleus in the sample 408, while the other first pixels 404 do not represent nuclei in the sample 408. In actual operation, the first label 402 has the same shape and position in both the fluorescence image 406 and the corresponding binary map 410, but FIGS. 3 and 4 present suitable examples.

[0029]

[0045] To generate the first label 402 of the binary map 410, the computing device 200 may perform thresholding on the intensity of the first pixel 404 of the fluorescence image 406. Regardless of the color model used, there are multiple ways to convert the pixel values defined by the color model to a grayscale that represents only the pixel intensity. Thus, performing thresholding involves classifying the first pixel 404 as a nuclear pixel (e.g., indicated by the first label 402 in the binary map 410) or a non-nuclear pixel based on whether the intensity of the first pixel 404 exceeds a threshold intensity. Nuclear pixels generally become brighter than non-nuclear pixels due to the fluorescent nuclear marker used to identify the nuclear region of the cell. (In FIGS. 3 and 4, for illustration purposes, the nuclear pixels are actually darker than the non-nuclear pixels.) For example, the pixel intensity can be defined on a scale of 0 to 1, and the threshold can be 0.8. Thus, the first pixel 404 with an intensity greater than 0.8 is considered a nuclear pixel, and the first pixel 404 with an intensity of 0.8 or less is considered a non-nuclear pixel. The computing device 200 applies the first label 402 (e.g., only the first label 402) to the position corresponding to the nuclear pixel. Other intensity scales and threshold intensities are also possible.

[0030]

[0046] Further, the threshold value may be redefined or adjusted based on manual inspection. For example, a person can check the result of the threshold processing and determine that the first pixel 404 can be more accurately classified into nuclear pixels and non-nuclear pixels by using different threshold intensities for the threshold processing. In one embodiment, the computing device 200 requests an input indicating a second threshold intensity via a user interface. This request may take the form of, for example, a displayed user prompt. Next, the computing device 200 receives an input indicating the (e.g., new) second threshold intensity and reclassifies the first pixel 404 into nuclear pixels and non-nuclear pixels based on whether the intensity of the first pixel 404 exceeds the second threshold intensity. The user generally decides to end the process if it is determined that the updated threshold intensity is optimized to accurately classify nuclear pixels and non-nuclear pixels.

[0031]

[0047] An exemplary binary map 410 is shown in FIG. 5, and an exemplary non-fluorescent image 412 of the sample 408 is shown in FIG. 6. The non-fluorescent image 412 in FIG. 6 presents an example of a number of non-fluorescent images 412 corresponding to a number of samples 408.

[0032]

[0048] In this context, the computing device 200 generates a second label 414 for a second pixel 416 of the non-fluorescent image 412 of the sample 408 based on the first label 402 (e.g., of the binary map 410). The second label 414 indicates whether the second pixel 416 represents a nucleus in the sample 408. In actual operation, the second label 414 actually has the same shape and position as the first label 402, but FIG. 6 presents a preferred example. Accordingly, the generation of the second label 414 may include applying the position and shape of the first label 402 in the binary map 410 to the non-fluorescent image 412.

[0033]

[0049] Next, the computing device 200 trains the calculation model 400 using the second label 414 and the non-fluorescent image 412 to identify the pixels of the non-fluorescent image that represent nuclei.

[0034]

[0050] That is, the calculation model 400 evaluates the second label 414 and the non-fluorescent image 412 (e.g., the second pixel 416) to identify common attributes of the second pixel 416 labeled as corresponding to the nucleus. Next, the computing device 200 uses these attributes to more accurately classify the unlabeled pixels of the non-fluorescent image as nuclei or non-nuclei. The higher the pixel intensity, the higher the likelihood that the calculation model 400 will label the unlabeled pixel as a nucleus. More generally, the computing device 200 adjusts various weighting coefficients corresponding to the algorithm of the calculation model 400 based on the evaluation of the second label 414 and the non-fluorescent image 412, so that the calculation model 400 can more accurately classify whether the unlabeled pixel is a nucleus or a non-nucleus.

[0035]

[0051] FIG. 7 is a block diagram of a method 300 for training the calculation model 400. As shown in FIG. 7, the method 300 includes one or more operations, functions, or actions as shown in blocks 302, 304, and 306. Although the blocks are shown to be performed sequentially, these blocks may be executed in parallel and / or in an order different from the order described herein. Also, various blocks may be combined into fewer blocks, divided and / or excluded into additional blocks, based on the desired implementation.

[0036]

[0052] In block 302, the method 300 includes generating a first label 402 for a first pixel 404 of the fluorescent image 406 of the sample 408, where the first label 402 indicates whether the first pixel 404 represents a nucleus in the sample 408. Block 302 is as described above with reference to FIGS. 3 and 4.

[0037]

[0053] In block 304, the method 300 includes generating a second label 414 for a second pixel 416 of the non-fluorescent image 412 of the sample 408 based on the first label 402, where the second label 414 indicates whether the second pixel 416 represents a nucleus in the sample 408. Block 304 is as described above with reference to FIGS. 5 and 6.

[0038]

[0054] In block 306, method 300 includes training computational model 400 using second label 414 and non-fluorescent image 412 to identify pixels of the non-fluorescent image representing the nucleus. Block 306 is as described above with reference to FIG. 6.

[0039] Use of the trained computational model

[0055] Referring to FIG. 8, optical microscope 105 acquires non-fluorescent image 412 of sample 408 including live cells 502A and live cells 502B. Any suitable nucleated live cells may be used. In one embodiment, one or more cells are adherent cells. In another embodiment, one or more cells may be mammalian cells. In another embodiment, one or more mammalian cells may be adherent mammalian cells. All of the one or more cells may be of the same cell type or may include different cell types. This is referred to as step (a) in the following claims. Cell 502A includes nucleus 504A and cytoplasm 506A. Cell 502B includes nucleus 504B and cytoplasm 506B. Sample 408 also includes background region 508 including any portion of sample 408 that is not part of cell 502A or cell 502B.

[0040]

[0056] The cells 502A (e.g., nucleus 504A and cytoplasm 506A) and cells 502B (e.g., nucleus 504B and cytoplasm 506B) of sample 408 contain protein-based nuclear translocation reporters at various concentrations that can be used for monitoring various cellular processes. As used herein, the term "fluorescent protein-based nuclear translocation reporter" is a fusion protein that includes a fluorescent protein and a protein that shuttles between the nucleus and / or outside the nucleus in response to a stimulus of interest. Any suitable fluorescent protein including, but not limited to, green fluorescent protein, red fluorescent protein, yellow fluorescent protein, blue fluorescent protein, orange fluorescent protein, near-infrared fluorescent protein, and any derivatives thereof may be used as being suitable for the intended use. For example, derivatives of green fluorescent protein (GFP) include, but are not limited to, EGFP, Emerald, Superfolder GFP, Azami Green, mWasabi, TagGFP, TurboGFP, AcGFP, ZsGreen, and T-Sapphire. In an exemplary embodiment, the fluorescent protein-based nuclear import reporter (FTR) may include a protein kinase translocation reporter that includes any reporter that moves in and out of the nucleus in response to protein kinase and phosphatase activities within the cell. In one embodiment, the protein kinase translocation reporter may include a human FoxO1 protein fused to a fluorescent protein including, but not limited to, TagGFP2. In this embodiment, the reporter may sometimes be referred to as "KTR", and since FoxO1 is phosphorylated by the active Akt kinase, the protein kinase to be monitored is Akt. If Akt is active, phosphorylation of KTR enhances nuclear export and localizes KTR to the cytoplasm. If Akt is inactive, dephosphorylation of KTR weakens nuclear export more than nuclear import and localizes KTR to the nucleus. Akt activity can thus be quantified using the ratio of nuclear to cytoplasmic (or nuclear to total cell) fluorescence.

[0041]

[0057] Non-limiting examples of other FTRs include phosphatase translocation reporters (responsive to phosphatase activity within the cell), protease translocation reporters (responsive to protease activity within the cell), and analyte-responsive translocation reporters, but are not limited thereto, and the analyte may be any analyte including hydrogen ions, potassium ions, calcium ions, and the like. Since the fluorescence of the FTR is usually dominated by other acquired light, the FTR is typically indistinguishable in non-fluorescent images 412.

[0042]

[0058] The FTR is expressed by the cell and may be encoded by any vector capable of expressing the FTR within the cell used in the methods of the present disclosure. Any suitable method for introducing the expression vector into the cell may be used. In one embodiment, the expression vector may be introduced by transiently transfecting the cell. In other embodiments, the cell may be stably transfected (e.g., via viral infection using a lentivirus, etc.) such that the FTR can be stably expressed within the cell.

[0043]

[0059] The computing device 200 uses the computational model 400 to identify the nuclear pixels of the non-fluorescent image 412 corresponding to the nucleus 504A or nucleus 504B. This is referred to hereinafter as step (c). As described above, the computational model 400 is trained to recognize nuclear pixels in unlabeled non-fluorescent images.

[0044]

[0060] Referring to FIG. 9, the fluorescence microscope 107 acquires a fluorescence image 406 of the FTR in the sample 408 (e.g., in cells 502A and 502B). This is referred to as step (b) below. In fluorescence imaging, since the fluorescence emitted from the FTR in the sample 408 is selectively acquired, the luminance levels of the regions of the sample 408 are different due to changes in the concentration of the FTR in the sample 408. As shown in FIG. 9 with various gray levels, the nuclei 504A and the cytoplasm 506B have a high concentration of FTR, the background region 508 has an FTR level of almost zero, and the cytoplasm 506A and the nucleus 504B have a low level of FTR. In other examples, there is a correlation between a high FTR intensity and a brighter gray shade, but in FIG. 9, for easier explanation, a low FTR intensity is mapped to a brighter gray shade.

[0045]

[0061] The computing device 200 uses the nuclear pixels of the non-fluorescence image 412 (e.g., the pixels corresponding to the nuclei 504A and 504B) to identify a first pixel of the fluorescence image 406 corresponding to the nucleus 504A or 504B, and a second pixel of the fluorescence image 406 not corresponding to the nucleus 504A or 504B. This is referred to as step (d) below. Since the fluorescence image 406 and the non-fluorescence image 412 have the same field of view, a binary map of the nuclear pixels of the non-fluorescence image 412 can be applied to the fluorescence image 406 to identify the first pixel of the fluorescence image 406 corresponding to the nucleus 504A or 504B.

[0046]

[0062] Next, the computing device 200 measures an index representing a first intensity of the first pixel (e.g., the pixel corresponding to the nucleus 504A and / or 504B in the fluorescence image 406), a second intensity of the second pixel (the background region 508, the cytoplasm 506A, and / or the cytoplasm 506B in the fluorescence image 406), a first amount of FTR located within the nucleus 504A and / or 504B, and a second amount of FTR not located within the nucleus 504A or 504B. This is referred to as step (e) below.

[0047]

[0063] Generally, an indicator takes the form of a ratio, but an indicator can also take the form of a difference. Other examples are possible. When the indicator is described as representing a first amount of FTR located within the nucleus and a second amount of FTR not located within the nucleus, it may mean that the ratio of the first amount of FTR located within the nucleus and the second amount of FTR not located within the nucleus can be derived from the indicator even if it is not directly represented by the indicator.

[0048]

[0064] Sample 408 can be fully characterized as a combination of background region 508, cell 502A, and cell 502B. In other examples, sample 408 may contain even more cells. Thus, sample 408 in this example can be mathematically expressed as follows.

[0049]

[0065] Sample 408 = Background 508 + Nucleus 504A + Cytoplasm 506A + Nucleus 504B + Cytoplasm 506B

[0050]

[0066] Thus, indicator R can take the following form.

[0051]

[0067] R = G(F(504A, 504B), F(508, 506A, 506B))

[0052]

[0068] Here, F is a function that returns the sum, average value (e.g., statistical average), or median of its arguments, and G is a function that returns the ratio or difference of its arguments. Generally, multiple instances of function F take the same form (e.g., sum, average value, or statistical average) in a given example of indicator R.

[0053]

[0069] Thus, in this example, indicator R may be a value obtained by dividing, or subtracting from, the sum, average value, or median of the pixel intensities of nucleus 504A and nucleus 504B by the sum, average value, or median of the pixel intensities of background 508, cytoplasm 506A, and cytoplasm 506B. Based on these principles, indicator R can take many other forms.

[0054]

[0070] R=G(F(504A+504B),F(508+506A+506B+504A+504B))

[0055]

[0071] R=G(F(506A+506B),F(508+506A+506B+504A+504B))

[0056]

[0072] In some examples, the computing device 200 segments the background from the cells in the non-fluorescent image 412 of the sample 408 to exclude second pixels that do not belong to cells from the calculation of the second intensity of the second pixels. Thus, the index R may also take the form:

[0057]

[0073] R=G(F(504A,504B),F(506A,506B))

[0058]

[0074] R=G(F(504A+504B),F(506A+506B+504A+504B))

[0059]

[0075] R=G(F(506A+506B),F(506A+506B+504A+504B))

[0060]

[0076] In some examples, a single cell may be of interest. Thus, the computing device 200 may identify (e.g., via a clustering algorithm) a nuclear pixel in the non-fluorescent image 412 that corresponds to the nucleus 504A, and identify (e.g., via a clustering algorithm) a first pixel in the fluorescent image 406 that corresponds to the nucleus 504A and a second pixel that is within the cytoplasm 506A. Thus, in some examples, the index R may relate to only a single cell.

[0061]

[0077] R=G(F(504A),F(508,506A))

[0062]

[0078] R=G(F(504A),F(508+506A+504A))

[0063]

[0079] R=G(F(506A),F(508+506A+504A))

[0064]

[0080] R=G(F(504A),F(506A))

[0065]

[0081] R=G(F(504A),F(506A+504A))

[0066]

[0082] R=G(F(506A),F(506A+504A))

[0067]

[0083] As will be appreciated by those skilled in the art, the methods of the present disclosure can be used to assess the effect of a test compound on an activity to which FTR is responsive. Thus, in one embodiment, sample 408 is contacted with a test compound, and computing device 200 performs steps (a)-(e) multiple times (e.g., over a period of time) to determine the effect of the test compound on a first amount of FTR located within nucleus 504A and / or nucleus 504B and a second amount of FTR not located within nucleus 504A and / or nucleus 504B. Any suitable test compound may be used, including, but not limited to, small molecules, proteins, peptides, nucleic acids, lipids, carbohydrates, etc. The effect of the test compound on the localization of FTR provides a measure of the effect of the test compound on the activity to which FTR is responsive.

[0068]

[0084] In some instances, indicator R provides a measure of kinase, phosphatase, or protease activity in cells 502A and / or cells 502B.

[0069]

[0085] In some examples, index R provides a measure of the analyte concentration within cell 502A and / or cell 502B.

[0070]

[0086] Figure 10 is a block diagram of a method 600 for monitoring cells 502A and / or cells 502B. As shown in Figure 10, method 600 includes one or more operations, functions, or procedures as shown in blocks 602, 604, 606, 608, and 610. Although the blocks are shown to be performed sequentially, these blocks may be executed in parallel and / or in an order different from the order described herein. Also, various blocks may be combined into fewer blocks, divided and / or excluded into additional blocks, based on the desired implementation.

[0071]

[0087] In block 602, method 600 includes obtaining a non-fluorescent image 412 of a sample 408 that includes one or more live cells 502A, 502B. The one or more live cells 502A, 502B may include a fluorescent protein-based nuclear translocation reporter. Block 602 is as described above with reference to Figure 8.

[0072]

[0088] In block 604, method 600 includes obtaining a fluorescent image 406 of a fluorescent protein-based nuclear translocation reporter in one or more live cells 502A, 502B in sample 408. Block 604 is as described above with reference to Figure 9.

[0073]

[0089] In block 606, method 600 includes identifying nuclear pixels of non-fluorescent image 412 corresponding to nuclei 504A, 504B of one or more live cells 502A, 502B via computational model 400. Block 606 is as described above with reference to Figure 8.

[0074]

[0090] In block 608, method 600 includes identifying a first pixel of fluorescent image 406 corresponding to nuclei 504A, 504B and a second pixel of fluorescent image 406 not corresponding to nuclei 504A, 504B based on the nuclear pixels. Block 608 is as described above with reference to Figure 9.

[0075]

[0091] In block 610, method 600 includes calculating an indicator R representing a first amount of a fluorescent protein-based nuclear translocation reporter located within the nuclei 504A, 504B of one or more live cells 502A, 502B and a second amount of the fluorescent protein-based nuclear translocation reporter not located within the nuclei 504A, 504B of one or more live cells 502A, 502B, based on a first intensity of a first pixel and a second intensity of a second pixel. Block 610 is as described above with reference to FIG. 9.

[0076] Example

[0092] A U-Net-based CNN was trained to detect the nuclei of A549 cells and SK-MES-1 cells. The input to the model was a phase contrast microscopy image, and the output was a binary nuclear map (see FIG. 11). The binary nuclear map was obtained by pixel-wise thresholding of the fluorescence image of the nuclear marker. The model was trained for 30 epochs at a learning rate of 10-4 using the Adam optimizer for each of the two cell lines to maximize the Dice coefficient between the predicted fluorescence map and the expected fluorescence map. The nuclear segmentation results were evaluated by comparing the total area of the marked nuclei in the predicted image and the target image, which showed a strong correlation (R2 = 89% for A549 and 97% for SK-MES-1, see FIGS. 12 and 13).

[0077]

[0093] Both nuclear segmentation models may slightly overestimate the nuclear area for two main reasons. First, the predicted nuclear size is on average slightly larger than the expected nuclear size. This problem can be solved by fine-tuning the threshold selection, model training, and post-processing. Second, the efficiency of the nuclear marker is less than 100%, meaning that not all nuclei are necessarily marked in the expected nuclear map. The CNN learns to predict nuclei based on the appearance of nuclei in phase contrast images, which generally means that more nuclei are predicted than are present in the target. This problem is particularly prominent in the case of the A549 dataset used in this case study and can be mitigated by manual verification of the model predictions by cell biologists.

[0078]

[0094] Next, the output from each model was used together with the fluorescence image to calculate how much fluorescence was present inside the nucleus compared to the total fluorescence. The output was converted to binary values (pixels belonging to the nucleus are 1, pixels outside the nucleus are 0) to obtain the predicted nuclear map. The fluorescence image was multiplied by this map pixel by pixel to obtain the fluorescence inside the nucleus. The pixel intensities inside the predicted nucleus were summed and divided by the sum of the fluorescence intensities of all pixels to obtain the predicted ratio readout value. The same procedure was also performed on the target nuclear map to obtain the target ratio for comparison.

[0079]

[0095] For both cell types, there was a strong correlation between the ratio obtained in the present invention and the fluorescence-based ratio being compared (R2 = 97% for both A549 and SK-MES-1, see Figures 14 and 15), which means that the KTR ratio can be reliably quantified. It should be noted that the predicted KTR ratio was underestimated in both cell types, which is a direct result of the overestimation of the nuclear area shown above and can be solved in two different ways. First, the size of the nucleus can be sorted out as described above. Second, a multiplication factor can be calculated using another calibration set and used to adjust the predicted KTR ratio.

[0080]

[0096] The KTR ratio was also calculated for each cell. In addition to nuclear segmentation, an instance segmentation model was trained to segment individual cells. The instance segmentation model is based on a central mask and was trained to segment individual cells for a diverse dataset of multiple cell types. Then, the segmentation of each individual cell was used to determine the cell area and ratio, and the KTR ratio of each individual cell was calculated as (fluorescence inside the nucleus) / (total fluorescence inside the cell). For this part, cells with a nuclear area below a certain threshold were excluded, but this was often a problem as the nuclei of some cells were not marked or were much smaller than expected targets.

[0081]

[0097] Although it has more noise than the KTR ratio of the whole image, it has a positive correlation with the KTR ratio of a single cell (R2 = 55% for A549 and R2 = 78% for SK-MES-1, see Figures 16 and 17). The noise level can be improved by fine-tuning both the cell and nuclear segmentation models, but this result shows that insights into the heterogeneity of the KTR ratio can be obtained using a single fluorescent KTR marker without the need for a nuclear or membrane marker to facilitate segmentation.

[0082]

[0098] Although different advantageous configurations have been described for purposes of illustration and explanation, these are not intended to be exhaustive or to limit to examples of the disclosed form. Many changes and modifications will be apparent to those skilled in the art. Further, a plurality of different advantageous examples can exhibit different advantages compared to other advantageous examples. The one or more examples selected have been selected and described to best explain the principles of the examples, the practical application, and to enable those skilled in the art to understand the present disclosure through various examples with various modifications made in accordance with the particular use intended.

Claims

**Claim 1** A method for monitoring one or more living cells, comprising: (a) obtaining a non-fluorescent image of a sample comprising one or more living cells, wherein the one or more living cells comprise a fluorescent protein-based nuclear translocation reporter; (b) obtaining a fluorescent image of the fluorescent protein-based nuclear translocation reporter in the one or more living cells in the sample; (c) identifying, via a computational model, nuclear pixels of the non-fluorescent image corresponding to the nuclei of the one or more living cells; (d) identifying, based on the nuclear pixels, first pixels of the fluorescent image corresponding to the nuclei and second pixels of the fluorescent image not corresponding to the nuclei; (e) calculating an index representing a first amount of the fluorescent protein-based nuclear translocation reporter located within the nuclei of the one or more living cells and a second amount of the fluorescent protein-based nuclear translocation reporter not located within the nuclei of the one or more living cells, based on a first intensity of the first pixels and a second intensity of the second pixels. A method comprising the above steps. **Claim 2** The method according to claim 1, wherein the index is a ratio of the first amount to the second amount. **Claim 3** The method according to any one of claims 1 to 2, wherein the computational model is trained using a non-fluorescent image having pixels labeled as corresponding to nuclei or not corresponding to nuclei. **Claim 4** The method according to claim 3, wherein the label of the non-fluorescent image is generated by applying a threshold algorithm to the fluorescent image. **Claim 5** The method according to any one of claims 1 to 4, wherein identifying the nuclear pixels comprises generating a binary map indicating whether each pixel of the non-fluorescent image represents a nucleus. **Claim ⑥** The method according to any one of claims 1 to 5, wherein the computational model is trained using a fluorescent image having nuclei tagged with a fluorescent nuclear marker. **Claim 7** The method according to any one of claims 1 to 6, wherein obtaining the non-fluorescent image comprises obtaining a bright-field image, a dark-field image, or a phase-contrast image. **Claim 8** The method according to any one of claims 1 to 7, wherein the computational model is a Vision Transformer (ViT), a pre-trained model, a convolutional model, or an artificial neural network. **Claim 9** The method according to any one of claims 1 to 8, wherein calculating the index comprises calculating a sum of the first intensities.

10. wherein the sum is a first sum, and calculating the index further comprises calculating a second sum of the second intensities, and comparing the first sum with the second sum, The method according to claim 9.

11. The method according to claim 10, wherein comparing the first sum with the second sum comprises calculating a ratio of the first sum to the second sum.

12. The method according to claim 10, wherein comparing the first sum with the second sum comprises calculating a difference between the first sum and the second sum.

13. The method according to any one of claims 1 to 12, wherein calculating the index comprises calculating an average value of the first intensities.

14. wherein the average value is a first average value, and calculating the index further comprises calculating a second average value of the second intensities, and comparing the first average value with the second average value, The method according to claim 13.

15. The method according to claim 14, wherein comparing the first average value with the second average value comprises calculating a ratio of the first average value to the second average value.

16. The method according to claim 14, wherein comparing the first average value with the second average value comprises calculating a difference between the first average value and the second average value.

17. wherein the second pixel corresponds to the cytoplasm of the one or more living cells, and calculating comprises calculating an index representing a first amount of a fluorescent protein-based nuclear translocation reporter located in the nucleus of the one or more living cells and a second amount of the fluorescent protein-based nuclear translocation reporter located in the cytoplasm, based on the first intensity of the first pixel and the second intensity of the second pixel. The method according to any one of claims 1 to 16.

18. The method according to any one of claims 1 to 16, wherein calculating comprises calculating an index representing the first amount of the fluorescent protein-based nuclear translocation reporter located in the nucleus and the second amount of the fluorescent protein-based nuclear translocation reporter located in the one or more living cells, based on the first intensity of the first pixel and the second intensity of the second pixel.

19. The second pixel corresponds to the cytoplasm of the one or more live cells, and the calculating includes calculating an index representing a second amount of the fluorescent protein-based nuclear translocation reporter located within the cytoplasm and a third amount of the fluorescent protein-based nuclear translocation reporter located within the one or more live cells based on the first intensity of the first pixel and the second intensity of the second pixel. The method according to any one of claims 1 to 16.

20. The method according to any one of claims 1 to 19, wherein the fluorescent protein-based nuclear translocation reporter is selected from the group including a protein kinase translocation reporter, a phosphatase translocation reporter, a protease translocation reporter, and an analyte-responsive translocation reporter.

21. The method according to any one of claims 1 to 20, further including segmenting a background from cells in the non-fluorescent image of the sample and excluding the second pixels not belonging to the cells from the calculation of the second intensity of the second pixels.

22. The method according to any one of claims 1 to 21, wherein the one or more live cells are adherent mammalian cells.

23. The method according to any one of claims 1 to 22, wherein the method is performed to monitor a signal transduction pathway within the one or more live cells.

24. The method further includes contacting the sample with a test compound and performing steps (a) to (e) multiple times to determine the effect of the test compound on the first amount of the fluorescent protein-based nuclear translocation reporter located within the nucleus of the one or more live cells and the second amount of the fluorescent protein-based nuclear translocation reporter not located within the nucleus of the one or more live cells. The method according to any one of claims 1 to 23.

25. The method according to any one of claims 1 to 24, wherein the index provides a measure of kinase, phosphatase, or protease activity in the one or more live cells.

26. The method according to any one of claims 1 to 25, wherein the index provides a measure of analyte concentration in the one or more live cells.

27. Identifying the nuclear pixels includes identifying the nuclear pixels of the non-fluorescent image corresponding to a single nucleus of a single cell of the one or more live cells. Identifying the first pixel and the second pixel includes identifying the first pixel corresponding to the single nucleus and the second pixel within the cytoplasm of the single cell, Calculating the index includes calculating an index representing the first amount of the fluorescent protein-based nuclear translocation reporter located in the nucleus and the second amount of the fluorescent protein-based nuclear translocation reporter located in the cytoplasm of the single cell. The method according to any one of claims 1 to 26.

28. Identifying the nuclear pixel includes identifying the nuclear pixel of the non-fluorescent image corresponding to the single nucleus of the single cell among the one or more living cells, Identifying the first pixel and the second pixel includes identifying the first pixel corresponding to the nucleus and the second pixel within the cytoplasm of the single cell, Calculating the index includes calculating an index representing the first amount of the fluorescent protein-based nuclear translocation reporter located in the single nucleus and the third amount of the fluorescent protein-based nuclear translocation reporter located in the single cell. The method according to any one of claims 1 to 26.

29. Identifying the nuclear pixel includes identifying the nuclear pixel of the non-fluorescent image corresponding to the single nucleus of the single cell among the one or more living cells, Identifying the first pixel and the second pixel includes identifying the first pixel corresponding to the nucleus and the second pixel within the cytoplasm of the single cell, Calculating the index includes calculating an index representing the second amount of the fluorescent protein-based nuclear translocation reporter located in the cytoplasm of the single cell and the third amount of the fluorescent protein-based nuclear translocation reporter located in the single cell. The method according to any one of claims 1 to 26.

30. A non-transitory computer-readable medium storing instructions that, when executed by a computing device, cause the computing device to execute the method according to any one of claims 1 to 29.

31. A system for monitoring one or more living cells, An optical microscope, A fluorescence microscope, One or more processors, A non-transitory computer-readable medium that, if executed by the one or more processors, stores instructions for causing the system to execute the method according to any one of claims 1 to 29 A system comprising the same. **Claim 32** A system for monitoring one or more living cells, comprising: An optical microscope; A fluorescence microscope; One or more processors; A non-transitory computer-readable medium that, if executed by the one or more processors, causes the system to (a) Obtain a non-fluorescent image of a sample containing one or more living cells, the one or more living cells containing a fluorescence protein-based nuclear translocation reporter, via the optical microscope; (b) Obtain a fluorescent image of the fluorescence protein-based nuclear translocation reporter in the one or more living cells in the sample via the fluorescence microscope; (c) Identify nuclear pixels of the non-fluorescent image corresponding to the nuclei of the one or more living cells via a computational model; (d) Identify a first pixel of the fluorescent image corresponding to the nucleus and a second pixel of the fluorescent image not corresponding to the nucleus based on the nuclear pixels; (e) Calculate an index representing a first amount of the fluorescence protein-based nuclear translocation reporter located within the nucleus of the one or more living cells and a second amount of the fluorescence protein-based nuclear translocation reporter not located within the nucleus of the one or more living cells based on a first intensity of the first pixel and a second intensity of the second pixel A non-transitory computer-readable medium storing instructions for causing the system to perform functions including the above A system comprising the same. **Claim 33** A method for training a computational model for identifying pixels of a non-fluorescent image representing a nucleus, the method comprising: Generating a first label for a first pixel of a fluorescent image of a sample, the first label indicating whether the first pixel represents a nucleus in the sample; Generating a second label for a second pixel of a first non-fluorescent image of the sample based on the first label, the second label indicating whether the second pixel represents a nucleus in the sample; Training a computational model using the second label and the first non-fluorescent image to identify pixels of a second non-fluorescent image representing a nucleus A method comprising the above. **Claim 34** The method according to claim 33, wherein generating the first label includes performing threshold processing on the intensity of the first pixel.

35. The method according to claim 34, wherein performing the threshold processing includes classifying the first pixel into a nuclear pixel and a non-nuclear pixel based on whether the intensity of the first pixel exceeds a threshold intensity.

36. Receiving, via a user interface, an input indicating a second threshold intensity; and reclassifying the first pixel into the nuclear pixel and the non-nuclear pixel based on whether the intensity of the first pixel exceeds the second threshold. The method according to claim 35, further comprising.

37. The method according to any one of claims 33 to 36, wherein generating the first label includes generating a binary map indicating whether each pixel of the first pixel represents a nucleus of the sample.

38. The method according to claim 37, wherein generating the second label includes applying the binary map to the first non-fluorescent image.

39. The method according to any one of claims 33 to 38, wherein training the computational model includes minimizing an error between a pixel of the second non-fluorescent image identified by the computational model and a pixel identified by the second label.

40. A non-transitory computer-readable medium storing instructions that, when executed by a computing device, cause the computing device to perform the method according to any one of claims 33 to 39.

41. A system for training a computational model to identify pixels of a non-fluorescent image representing a nucleus, the system comprising: one or more processors; and a non-transitory computer-readable medium storing instructions that, when executed by the one or more processors, cause the system to perform the method according to any one of claims 33 to 39.

42. The system according to claim 41, further comprising an optical microscope and a fluorescence microscope. The system according to claim 41, further comprising.

Images