Analysis system, analysis device, analysis program, and analysis method

Abstract

In the present invention, a cell image acquisition unit acquires a first image of a cell in which a first component is labeled and a second image of the cell in which a second component differing from the first component is labeled. A feature amount calculation unit extracts, from the first image and the second image, feature amounts of individual components that constitute the cell. A correlation calculation unit uses the feature amounts to calculate a correlation between the first component and the second component.

Description

Analysis system, analysis device, analysis program, and analysis method

[0001] The present application claims priority to Japanese Patent Application No. 2022-111995, filed on July 12, 2022, the contents of which are incorporated herein by reference.

[0002] In biological science, medicine, and the like, it is known that the health, disease, and other states of living organisms are correlated with, for example, the states of cells and intracellular organelles. Therefore, analyzing these correlations is one way to solve various problems in biological science, medicine, and the like. Furthermore, analyzing the signaling pathways between cells and within cells can be useful, for example, in research into industrial biosensors and pharmaceuticals aimed at disease prevention.

[0003] Various analytical techniques for cells, tissue fragments, etc., such as techniques using image processing, are known (see, for example, Patent Document 1). Such conventional techniques, for example, involve image processing and analysis of images of cells acquired from a living organism, etc., by acquiring image data of the cells at predetermined time intervals and comparing the acquired image data with image data relating to the cell morphology acquired at different times to calculate correlations and differences in features of the cell morphology. This makes it possible to determine the activity level of the acquired cells, which can be useful in elucidating biological phenomena such as canceration and pathological transformation of cells.

[0004] US Patent Application Publication No. 2014 / 0099014

[0005] However, with the above-mentioned conventional techniques, there are cases where the information obtained is insufficient, or where analysis cannot be performed sufficiently, etc. Thus, with the above-mentioned conventional techniques, there are cases where it is difficult to analyze cells.

[0006] The present invention has been made in consideration of the above circumstances, and one of its objects is to provide an analysis system, an analysis device, an analysis program, and an analysis method that are capable of analyzing cells.

[0007] (1) The present invention has been made to solve the above-mentioned problems, and one aspect of the present invention is an analysis system including: an acquisition unit that acquires a first image in which a first component is labeled on a cell; and a second image in which a second component different from the first component is labeled on the cell; a feature extraction unit that extracts features for each component that makes up the cell from each of the first image and the second image; and a correlation calculation unit that calculates the correlation between the first component and the second component using the features.

[0008] (2) Another aspect of the present invention is an analysis device including a correlation calculation unit that calculates a correlation between a first component and a second component using feature amounts for each component constituting a cell, the feature amounts being extracted from a first image in which a first component is labeled for a cell and a second image in which a second component different from the first component is labeled for the cell.

[0009] (3) Another aspect of the present invention is an analysis program for causing a computer of an analysis device to execute a procedure for calculating a correlation between a first component and a second component using feature amounts for each component constituting a cell, the feature amounts being extracted from a first image in which a first component is labeled for a cell and a second image in which a second component different from the first component is labeled for the cell.

[0010] (4) Also, one aspect of the present invention is an analysis method in an analysis system, the analysis method including: an acquisition step in which an acquisition unit acquires a first image in which a first component is labeled for a cell and a second image in which a second component different from the first component is labeled for the cell; a feature extraction step in which a feature extraction unit extracts features for each component that makes up the cell from each of the first image and the second image; and a correlation calculation step in which a correlation calculation unit calculates the correlation between the first component and the second component using the features.

[0011] According to the present invention, cells can be analyzed.

[0012] FIG. 1 is a schematic diagram showing an example of the configuration of an analysis system 1 according to an embodiment of the present invention. FIG. 2 is an explanatory diagram for explaining an overview of an analysis technique according to the embodiment. FIG. 3 is a diagram showing an example of a captured image according to the embodiment. FIG. 4 is a schematic block diagram showing an example of the configuration of the analysis system 1 according to the embodiment. FIG. 5 is a schematic diagram showing an example of feature amount information according to the embodiment. FIG. 6 is a flow diagram showing an example of an operation of the analysis system 1 according to the embodiment. FIG. 7 is another schematic diagram for explaining a network according to the embodiment. FIG. 8 is another schematic diagram for explaining a network according to the embodiment. FIG. 9 is another schematic diagram for explaining a network according to the embodiment. FIG. 10 is a schematic diagram showing an example of a network according to the embodiment. FIG. 11 is a schematic diagram showing an example of a network according to Modification 1 according to the embodiment. FIG. 12 is a schematic diagram showing an example of feature amount information according to Modification 1. FIG. 13 is a flow diagram showing an example of an operation of the analysis system 1 according to Modification 1. FIG. 14 is a schematic diagram showing an overview of Modification 2 according to the embodiment. FIG. 15 is a schematic diagram showing an example of feature amount information according to Modification 2. FIG. 16 is a schematic diagram showing an example of a network according to Modification 2. FIG. 17 is a schematic diagram showing an example of target biological substance detection in Modification 3 according to the embodiment. FIG. 18 is a schematic diagram showing another example of target biological substance detection in Modification 3 according to the embodiment.

[0013] (Embodiment) Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0014] <Analysis System> Fig. 1 is a schematic diagram showing an example of the configuration of an analysis system 1 according to an embodiment of the present invention. The analysis system 1 performs analysis processing on images acquired by imaging cells or the like. In the following description, images acquired by imaging cells or the like will also be simply referred to as cell images. The analysis system 1 includes an analysis device 10, a microscope device 20, and a display unit 30.

[0015] The analysis device 10 is a computer such as a server, which analyzes cell images captured by the microscope device 20 and displays the analysis results on a display unit 30. The microscope device 20 is a biological microscope and includes a motorized stage 21 and an imaging unit 22. The motorized stage 21 can arbitrarily move the position of an object to be imaged in a predetermined direction (e.g., a certain direction in a horizontal two-dimensional plane). The imaging unit 22 includes an imaging element such as a CCD (Charge-Coupled Device), a CMOS (Complementary MOS), or a PMT (Photomultiplier Tube), and images the object to be imaged on the motorized stage 21. Note that the microscope device 20 does not necessarily have to include the motorized stage 21, and the stage may not move in a predetermined direction.

[0016] More specifically, the microscope device 20 has the functions of, for example, a differential interference contrast microscope (DIC), a phase contrast microscope, a fluorescence microscope, a confocal microscope, a super-resolution microscope, etc. The microscope device 20 captures an image of a culture vessel placed on a motorized stage 21. This culture vessel is, for example, a well plate WP. The culture vessel may also be, for example, a slide chamber, a petri dish, etc. The microscope device 20 irradiates light onto cells cultured in multiple wells W of the well plate WP, and captures the transmitted light that has passed through the cells as a cell image. This allows for the acquisition of images such as a transmission DIC image of the cells, a phase contrast image, a dark-field image, and a bright-field image. Note that the well plate WP may have only one well W.

[0017] The microscope device 20 captures, as a cell image, the fluorescence emitted by the fluorescent stain. For example, the microscope device 20 captures, as a cell image, the fluorescence emitted from a chromogenic substance itself incorporated into a biological material (an example of a component) or the fluorescence emitted by a substance having a chromophore binding to the biological material. The microscope device 20 may capture, as a cell image, the fluorescence emitted from the biological material by irradiating the cell with excitation light that excites the fluorescent substance. In other words, since the biological material is labeled with fluorescence, the analysis system 1 can identify the biological material by capturing the fluorescence. This allows the analysis system 1 to acquire fluorescence images, confocal images, and super-resolution images. Note that the imaged object may be a color other than fluorescent, as long as it can capture an image that allows the biological material to be identified.

[0018] The method for acquiring cell images is not limited to optical microscopy. For example, electron microscopy may also be used. Furthermore, images containing cells, as described below, may be obtained by using images acquired by different methods to acquire correlations. That is, the type of image containing cells may be selected appropriately. In this embodiment, cells include, for example, primary cultured cells, established cultured cells, cells in tissue slices, etc. To observe cells, a sample may be a cell aggregate, a tissue sample, an organ, or an individual (such as an animal) to obtain an image containing cells. The state of the cells is not particularly limited and may be live or fixed, either "in vivo" or "in vitro." Furthermore, information on the live state and information on the fixed state may be combined.

[0019] The state of the cell may be selected appropriately depending on the purpose. For example, fixed or unfixed cells may be selected depending on the type of intracellular component to be distinguished (e.g., protein, organelle). Furthermore, when acquiring the dynamic behavior of a fixed cell, multiple fixed cells under different conditions are created and the dynamic behavior is acquired. The type of intracellular component to be distinguished is not limited to the nucleus. Organelles are cellular organelles and include various organelles such as mitochondria and cell membranes. One type of organelle is the cytoskeleton, which is primarily composed of three types of protein fibers: actin filaments, intermediate filaments, and microtubules.

[0020] Furthermore, in order to observe cells, the cells may be pre-treated before observation. Of course, in order to observe cells, the cells may be observed without being pre-treated. When observing cells, the cells are stained by immunostaining and then observed. For example, a staining solution is selected based on the type of intracellular structure to be distinguished. Furthermore, any staining method can be used. For example, various special stains mainly used for tissue staining, hybridization using the binding of base sequences, etc.

[0021] Alternatively, cells may be treated with a luminescent protein (e.g., a luminescent protein expressed from an introduced gene (e.g., a luciferase gene)) and observed. For example, the luminescent protein used may be selected based on the type of intracellular nuclear structure to be distinguished. Also, observation may be performed using a fluorescent dye (e.g., DAPI, Hoechst, etc.). Furthermore, pretreatments for analyzing correlation acquisition, such as the means for observing cells and / or the method for staining cells, may be appropriately selected depending on the purpose. For example, when obtaining dynamic cell behavior, dynamic information on cells may be obtained using an optimal method, and when obtaining intracellular signal transduction, information on intracellular signal transduction may be obtained using an optimal method. These pretreatments may be different depending on the purpose. Furthermore, the number of types of pretreatments selected depending on the purpose may be reduced. For example, even if the optimal methods for obtaining dynamic cell behavior and the method for obtaining intracellular signal transduction are different, obtaining each piece of information using different methods would be cumbersome. Therefore, if it is sufficient to obtain each piece of information, a common method may be used instead of the optimal method.

[0022] The well plate WP has a plurality of wells W. In this example, the well plate WP has 96 wells W (12 x 8). Cells are cultured in the wells W under specific experimental conditions. The specific experimental conditions include temperature, humidity, culture period, time elapsed since application of a stimulus, the type and strength of the stimulus applied, the presence or absence of a stimulus, and the induction of biological characteristics. Examples of stimuli include physical stimuli such as electricity, sound waves, magnetism, and light, and chemical stimuli such as the administration of a substance or drug. Furthermore, biological characteristics are characteristics that indicate the stage of cell differentiation, morphology, cell number, etc., and also include, for example, the cell cycle.

[0023] <Multiple Fluorescence Staining> FIG. 2 is an explanatory diagram for explaining an overview of the analysis method according to this embodiment. The analysis system 1 acquires cell images by repeating the steps of fluorescent labeling (e.g., immunostaining), imaging, and fluorescent bleaching (decolorization) (also referred to as "multiple fluorescence staining"). This single step is also referred to as a staining imaging step. Different biological substances are fluorescently labeled in each staining imaging step. For example, in the case of immunostaining, different antibodies are used for fluorescent staining. Here, the analysis system 1 acquires cell images of different colors in the same well W at each staining imaging step. In other words, the analysis system 1 acquires cell images in which different biological substances are colored or luminescent at each staining imaging step in the same cell. This allows the analysis system 1 to identify different biological substances at each staining imaging step in the same cell. In other words, the analysis system 1 can identify more proteins in the same cell than the number (e.g., one) that can be identified in a single imaging step.

[0024] In one staining and imaging process, cells stained with dyes of different colors (wavelengths) are imaged, for example, through four or five-fold staining. The analysis system 1 identifies the cytoplasm and intracellular components by, for example, separating the images for each color and performing masking or the like. The analysis system 1 changes at least some of the staining from the previous four or five-fold staining process and performs the next staining and imaging process. At least some of the staining is a stain that colors or luminesces biological materials other than the biological material identified in the previous process. This allows the analysis system 1 to identify different biological materials in the same cell in each staining and imaging process.

[0025] In the first fluorescent labeling, staining shown in schematic diagram F1 is performed in well W. An antibody AB1 labeled with luminescent dye F11 is contained in the staining solution, and the antibody AB1 binds to an antigen AG1 of the cells to be observed. The antigen AG1 is an antigen of protein A. After staining, the microscope device 20 captures the luminescence from the luminescent dye F11 to capture an image in which protein A can be identified. The analysis system 1 acquires an image G11 in which the cytoplasm can be identified, an image G12 in which organelles can be identified, an image G13 in which protein A can be identified, and an image G14 in which the nucleus can be identified.

[0026] In the first fluorescent bleaching, the fluorescent bleaching shown in the schematic diagram F2 is performed in the same well W as in the first bleaching. The bleaching solution contains a substance that bleaches the luminescent dye F11, such as hydrogen peroxide, and bleaches the staining of protein A. Note that bleaching is not limited to bleaching the luminescent dye; for example, the antibody AB1 may be separated from the antigen AG1 and removed.

[0027] In the second fluorescent labeling, staining shown in schematic diagram F3 is performed in well W. The staining solution contains antibody AB2 labeled with luminescent dye F21, and antibody AB2 binds to antigen AG2 of the cells being observed. Antigen AG2 is an antigen of protein B. After staining, the microscope device 20 captures the luminescence from the luminescent dye F21 to capture an image in which protein B can be identified. The analysis system 1 acquires an image G21 in which the cytoplasm can be identified, an image G22 in which organelles can be identified, an image G23 in which protein A can be identified, and an image G24 in which the nucleus can be identified.

[0028] Here, image G11 is image G21, image G12 is image G22, and image G13 is image G32, all of which have the same feature amounts. In other words, since the same cell is being observed, the cytoplasm, organelles, and nucleus are images with the same feature amounts. Meanwhile, the antibody for protein A is bleached and therefore does not appear in the image, and image G23 of the newly stained protein B is obtained. Note that in the second fluorescent labeling, the cytoplasm, organelles, and nucleus do not need to be stained. Furthermore, multiple proteins may be stained and imaged in a single staining and imaging process.

[0029] FIG. 3 shows an example of an image captured according to this embodiment. This figure shows nine images acquired by the analysis system 1 arranged in three rows and three columns. Each image is an image of one type of protein, and in this example, the analysis system 1 has acquired a total of nine types of images. These images are of the same cell, and the microscope device 20 captured the same position on the well W under the same imaging conditions (magnification, etc.). Because each single cell is located at the same position, the analysis system 1 can identify nine types of intracellular proteins in each single cell. This allows the analysis system 1 to use the same coordinate system for the positions of biological substances such as proteins across different images and to identify the cells that each biological substance constitutes across images. The analysis system 1 may also perform alignment (enlargement / reduction, rotation, and translation) of multiple images acquired through different staining imaging processes based on the positions of the nucleus and cytoplasm (cell wall) within the cell.

[0030] <Configuration Example of Analysis System> Fig. 4 is a schematic block diagram showing an example of the configuration of an analysis system 1 according to this embodiment. The analysis system 1 includes an analysis device 10, a microscope device 20, and a display unit 30 (display device). The analysis device 10 includes a calculation unit 100, a storage unit 200, and a result output unit 300. Note that the images processed by the analysis device 10 are not limited to images captured by the microscope device 20, and may be, for example, images stored in advance in the storage unit 200 included in the analysis device 10 or images stored in advance in an external storage device (not shown).

[0031] The calculation unit 100 functions by the processor executing a program stored in the storage unit 200. Some or all of these functional units of the calculation unit 110 may be configured with hardware such as an LSI (Large Scale Integration) or an ASIC (Application Specific Integrated Circuit). The calculation unit 100 includes a cell image acquisition unit 101, a feature calculation unit 102, a noise component removal unit 103, and a correlation extraction unit 104.

[0032] The cell image acquisition unit 101 acquires cell images captured by the imaging unit 22. Here, the cell image acquisition unit 101 acquires multiple cell images captured in a series of staining imaging steps in units of multiplexed fluorescence staining. The cell image acquisition unit 101 assigns to the cell images observation identification information that identifies the multiplexed fluorescence staining (series of staining imaging steps) and staining identification information that specifies the staining imaging step, as well as observed cell identification information that identifies the cell to be observed (well W) and experimental condition identification information that identifies the experimental conditions. Note that these identification information may be assigned by the imaging unit 22.

[0033] The feature calculation unit 102 calculates multiple types of feature amounts for the cell images provided by the cell image acquisition unit 101. These feature amounts include the area, average brightness, and variance of the cell images. In other words, the feature amounts represent characteristics derived from information acquired from the captured cell images. Here, the feature calculation unit 102 extracts specific color (wavelength) components from the images to separate them into cell images of each color component. Note that the image (or staining identification information), the color components in the images, and the relationship between the color components and the biological materials may be input in advance by the user and stored in the type storage unit 201. For the cell images of each color component, the feature calculation unit 102 calculates feature amounts for each biological material corresponding to the color component (see FIG. 2 ). Detailed examples of feature amounts will be described later.

[0034] In addition, when there are multiple images of the same cell that differ in time (unit of multi-fluorescence staining) or time series corresponding to the elapsed time of culturing a specific cell, or that differ due to changes in the cell state, such as differentiation, the feature calculation unit 102 may obtain positional information indicating a different change in luminance from the calculated change in luminance distribution over a predetermined time, or from the change in the calculated luminance distribution associated with a change in the cell state, such as differentiation, and use the change in luminance as a feature. In this case, multiple images that differ in changes in the cell state, such as differentiation, may be used, rather than being limited to changes over time. Alternatively, positional information indicating a different change in luminance may be used as a feature. For example, the feature may be the behavior of the cell within a predetermined time, or the behavior associated with a change in the cell state, such as differentiation, or the change in the shape of the cell within a predetermined time, or the change associated with a change in the cell state, such as differentiation. Furthermore, if no change within a predetermined time or a change associated with a change in the cell state, such as differentiation, is observed in the captured cell images, the absence of change may also be used as a feature.

[0035] The noise component removal unit 103 removes noise components (noise) from the features calculated by the feature calculation unit 102. The noise component removal unit 103 identifies each single cell or at least a part of its components (cytoplasm, nucleus, cell membrane). The noise component removal unit 103 generates feature information by associating each single cell or its components with each piece of material identification field information (type of protein identified by staining) attached to the cell image and the features from which noise has been removed (see FIG. 5 ).

[0036] The correlation extraction unit 104 extracts a specific correlation from among the multiple correlations between feature quantities calculated by the feature quantity calculation unit 102, based on the likelihood of the feature quantities after the noise component removal unit 103 has removed the noise components. Here, the likelihood is a numerical value that represents the likelihood of inferring a predetermined condition from the result when the result is calculated according to the predetermined condition. The likelihood is also a cost function that represents the likelihood of a parameter to be estimated when the data follows a probabilistic model. The noise removal by the noise component removal unit 103 and the extraction of the correlation by the processing of the correlation extraction unit 104 may be performed simultaneously, for example, using the graphical lasso method.

[0037] The result output unit 300 outputs the calculation results by the calculation unit 100 to the display unit 30. Note that the result output unit 300 may output the calculation results by the calculation unit 100 to an output device other than the display unit 30, a storage device, or the like. The display unit 30 is a display device such as a display, and displays the calculation results output by the result output unit 300. The display unit 30 may be a part of the analysis device 20.

[0038] 5 is a schematic diagram showing an example of feature information according to this embodiment. The feature information in this diagram is such that, for each staining identification information, substance identification information, and cell composition information, feature information is associated with observation target identification information that identifies the cell (well W) being observed and cell identification information that identifies a single cell. That is, each piece of feature information represents a feature of a biological substance (protein in this diagram) indicated by the substance identification information for each composition within a single cell in the observation target cell of each well.

[0039] In this figure, the feature amount for the cells in one well W1 is the value X c , p ,e The feature amount for the cells in another well W2 is expressed as a value Y c , p ,e The feature amount is, for example, a brightness value such as average brightness or total brightness, and these brightness values ​​are associated with the abundance of molecules. Variable c is a variable (cell identification information) that identifies a single cell, and its value is, for example, 1 for single cell (Cell) 1A, 2 for 1B, 3 for 1C, etc. Variable p is a variable (substance identification information) that identifies a biological substance, and its value is, for example, 1 for protein 1, 2 for protein 2, and 3 for protein 3. Proteins 1 and 2 are identified by the staining and imaging process of staining identification 1. Protein 3 is identified by the staining and imaging process of staining identification 2. Variable e is a variable (cell configuration information) that identifies the cell's configuration, and its value is, for example, 1 for nucleus (Nucl), 2 for cytoplasm (Cyto), and 3 for actin domain (AD). Actin domains are tangled, mesh-like structures formed by actin filaments. The cell may be made up of structures formed by the polymerization of biological substances other than actin, such as other types of organelles. The region where each biological substance exists is also called a domain.

[0040] In this figure, for example, for a single cell 1A on well W1, the total brightness value in the nucleus of the cell for protein 1 is X 1,1,1 , the total brightness value in the cytoplasm is X 1,1,2 , the total brightness value of the actin domain is X 1,1,3 In FIG. 5, for convenience, feature information for one feature is shown, but there may be two or more feature values, and X c , p ,e is X 1 c , p ,e , X 2 c , p ,e , .... For example, in a single cell identified by the variable c, the value of the feature value f of the structure identified by e of the biological material identified by the variable p is f c , p ,e The variable f is a variable that identifies the feature, and the feature is, for example, an average brightness value of X 1 c , p ,e , the sum of the brightness values ​​is X 2 c , p ,e , the maximum brightness value is X 3 c , p ,e , the minimum luminance value is X 4 c , p ,e , the area of ​​the configuration is X 5 c , p ,e , the circularity of the structure is X 6 c , p ,e , the standard deviation of the brightness values ​​is X 7 c , p ,e Furthermore, when the analysis system 1 acquires images of different cells (different wells W) at different elapsed times, for example, and calculates the feature amount, the feature amount is calculated by using a variable w that identifies the cell (well W) to be observed, as follows: X w ,f c , p ,e It is expressed as: X w ,f c , p ,e is the value X of the feature quantity of the observation target cell (well W) identified by the variable w. f c , p ,e For example, in FIG. W1 ,1 c , p ,e is X c , p ,e corresponds to X W2 ,1 c , p ,e Is Y c , p ,e The variable w corresponds to the observed cell identification information.

[0041] By multi-fluorescence staining, feature quantity X f c , p ,e In this case, one identical cell (the same well W) is measured. In this way, in the analysis system 1, the same cell is observed by multiple fluorescence staining, so that even between cell images with different colors or staining imaging processes, each single cell can be identified by position information, etc., and feature amounts can be associated with each single cell (cell identification information: variable c).

[0042] <Example of Operation of Analysis System> FIG. 6 is a flow diagram showing an example of the operation of the analysis system 1 according to this embodiment. Note that the operation shown here is merely an example, and operation steps may be omitted or added. In the example operation shown in this figure, the steps of fluorescent labeling by immunostaining or the like, imaging, and fluorescent bleaching (decolorization) are repeated using multiple fluorescence staining. The calculation unit 100 of the analysis system 1 uses cell images of captured cells to extract multiple types of feature amounts from the cell images and calculates whether the extracted feature amounts or their changes are correlated. If the calculation results show that the feature amounts or their changes are correlated, the calculation unit 100 determines that there is a correlation. Note that a correlation between feature amounts may also be referred to as a correlation. The following description will be given with reference to the configuration of the analysis system 1 shown in FIG. 4.

[0043] (Step S10) Fluorescent labeling by immunostaining is performed on the cells in each well W of the well plate WP. In this step S10, at least some of the cells are fluorescently labeled differently each time it is repeated. Different fluorescent labels are used to label different biological substances that fluoresce, and are performed to enable the different biological substances to be distinguished and observed.

[0044] (Step 101) The cell image acquisition unit 101 acquires a cell image of a fluorescently labeled cell. This cell image includes images of multiple types of biological tissues of different sizes, such as genes, proteins, and organelles. When there are multiple wells W to be observed, the cell image acquisition unit 101 acquires a cell image from each well W to be observed. The cell image acquisition unit 101 acquires an image using the imaging unit 22, and extracts an area corresponding to a cell from the image. For example, the cell image acquisition unit 101 extracts an area corresponding to a single cell from the cell image. This enables the analysis system 1 to distinguish, from the captured image, areas corresponding to each cell from other areas.

[0045] (Step S102) The feature calculation unit 102 extracts images of single cells included in the cell images acquired in step S101. The feature calculation unit 102 extracts images of single cells by performing image processing using a known method on the cell images. In this example, the feature calculation unit 102 extracts cell images by image region extraction, pattern matching, or the like.

[0046] (Step S103) Next, the feature calculation unit 102 determines the cell type for the cell image extracted in step S102. Note that the processing of step S103 may be omitted, and the feature calculation unit 102 may not need to determine the cell type.

[0047] (Step S104) Based on the determination result in step S30, the feature calculation unit 102 determines the cellular components contained in the cell image extracted in step S20. Here, the cellular components include organelles such as the nucleus, lysosomes, Golgi apparatus, and mitochondria, as well as proteins, second messengers, mRNA, metabolic products, nuclear structures, and genes.

[0048] The feature calculation unit 102 assigns cell identification information to each single cell. The feature calculation unit 102 associates the cell identification information with identification information of the cell (well W) to be observed, staining identification information, experimental condition identification information, a cell image of the single cell, cell position information indicating the position of the single cell, shape information indicating the shape of the single cell, and component information indicating the components within the single cell, and stores the associated information in the cell information storage unit 203. The component information includes component identification information that identifies the components within the cell, the type of component, and the area where the component is located (also referred to as the "component area").

[0049] Below, the components within a cell will be explained using the terms "constituent region" and "target biological material." A "target biological material" is a material contained in a constituent region and is a material for which feature values ​​are to be calculated. A "target biological material" is, for example, a material that constitutes at least a part of a constituent region, or a material that is distributed within a constituent region. "Target biological material" is a protein, mitochondria, etc. When an aggregate of "target biological materials" forms a region, this aggregate is a structure within the cell and can become a "constituent region."

[0050] (Step S105) The feature calculation unit 102 calculates the feature of each target biological substance in each constituent region of each single cell determined in step S104. These feature include pixel brightness values, the area of ​​a region in the image, the variance of pixel brightness, and the shape of a region in the image. When pixel brightness values ​​are used as feature values, brightness values ​​for each wavelength may also be used as feature values. There are multiple types of feature values ​​depending on the constituent elements of the cell.

[0051] As an example, the feature values ​​of an image of a cell nucleus include the total luminance value within the nucleus, the area of ​​the nucleus, and the shape of the nucleus. The feature values ​​of an image of the cytoplasm include the total luminance value within the cytoplasm, the area of ​​the cytoplasm, and the shape of the cytoplasm. The feature values ​​of an image of a region where each organelle, such as actin identified by fluorescent labeling, is present include material identification information indicating the type of protein that forms the organelle, the total luminance value within the region, the area of ​​the region, and the shape of the region. The feature values ​​of an image of an entire cell include the total luminance value within the cell, the area of ​​the cell, and the shape of the cell. The feature values ​​of an image of a mitochondrion include the fragmentation rate.

[0052] The feature calculation unit 102 performs a process of calculating feature amounts for each image identified and separated by a fluorescent label. This allows the feature calculation unit 102 to calculate feature amounts for each target biological substance of an organelle. If the target biological substance is not fluorescently labeled, an image of the target biological substance under observation may contain other substances. The feature amounts of the target biological substance under observation may not be calculated correctly due to the influence of other substances. Since the feature calculation unit 102 identifies the target biological substance under observation using multiple fluorescent staining, it can calculate feature amounts for each target biological substance with higher accuracy than when the target biological substance is not fluorescently labeled. The feature calculation unit 102 may calculate feature amounts by normalizing them to values ​​between 0 (zero) and 1, for example.

[0053] The feature calculation unit 102 may also calculate the feature based on information about experimental conditions for the cells associated with the cell image. For example, in the case of a cell image captured when an antibody is reacted with the cells, the feature calculation unit 102 may calculate a feature specific to the antibody reaction. In the case of a cell image captured when the cells are stained or when a fluorescent protein is added to the cells, the feature calculation unit 102 may calculate a feature specific to the stained cells or the fluorescent protein. In these cases, the storage unit 200 may include an experimental condition storage unit 202. The experimental condition storage unit 202 stores, for each cell image, information about experimental conditions for the cells associated with the cell image. The feature calculation unit 102 supplies the feature calculated in step S105 to the noise component removal unit 103.

[0054] (Step S106) The noise component removal unit 103 removes noise components from the feature amounts calculated in step S105. Specifically, the noise component removal unit 103 acquires information indicating the normal or abnormal range of the feature amount. This information indicating the normal or abnormal range of the feature amount is determined in advance based on the characteristics of the cells captured in the cell image. For example, among the feature amounts of the image of the cell nucleus, the normal range for the total intranuclear luminance value is determined based on the characteristics of the image of the cell nucleus.

[0055] If the calculated feature value is not within this normal range, the noise component removal unit 103 removes the feature value as a noise component. Here, when removing a feature value as a noise component, the noise component removal unit 103 removes it for each cell. Specifically, multiple feature values ​​may be calculated for a certain cell. For example, for a certain cell, the total intracellular luminance value, total nuclear luminance value, nuclear area, and nuclear shape may each be calculated as feature values. In this case, when removing the total intracellular luminance value for a certain cell as a noise component, the noise component removal unit 103 also removes the total nuclear luminance value, nuclear area, and nuclear shape of the cell. In other words, if at least one feature value among the multiple feature values ​​calculated for a certain cell is not within the normal range, the noise component removal unit 103 also removes the other feature values ​​of the cell.

[0056] That is, the noise component removal unit 103 removes noise components from the feature amounts supplied to the correlation extraction unit 104 for each cell imaged in the cell image, based on information indicating the characteristics of the cells imaged in the cell image. By configuring in this manner, the noise component removal unit 103 can remove feature amounts with relatively low reliability on a cell-by-cell basis when such feature amounts have relatively low reliability. In other words, the noise component removal unit 103 can improve the reliability of feature amounts.

[0057] The noise component removal unit 103 performs noise component removal processing for each image identified and separated by a fluorescent label. In other words, the noise component removal unit 103 performs processing for each channel of the fluorescently stained image. This allows the noise component removal unit 103 to determine a normal range or abnormal range for each target biological material and to identify feature values ​​in the abnormal range as noise components. If the target biological material is not fluorescently labeled, the image of the target biological material under observation may contain other materials. The normal range or abnormal range of the feature value of the target biological material under observation may not be appropriately determined due to the influence of other materials, or a feature value that should be in the normal range may be identified as being in the abnormal range. Because the noise component removal unit 103 identifies the target biological material under observation using multiple fluorescent staining, it can remove noise components more accurately for each target biological material than when the material is not fluorescently labeled.

[0058] If the calculated feature amount falls within this normal range, the noise component removal unit 103 supplies the feature amount to the correlation extraction unit 104. Note that the noise component removal unit 103 is not an essential component, and can be omitted depending on the state of the cell image and the state of calculation of the feature amount.

[0059] (Step S107) The cell image acquisition unit 101 determines whether or not the processes of steps S101 to S106 have been performed on the cells in all wells W in the well plate WP. If the processes have been performed on the cells in all wells W (YES), the process of step S11 is performed. On the other hand, if the processes have not been performed on the cells in at least one well W (NO), the process of step S101 is performed.

[0060] (Step S11) The cell image acquisition unit 101 determines whether all staining and imaging steps have been completed, i.e., whether multiplex fluorescence staining has been completed. If all staining and imaging steps have been completed (YES), the process of step S131 is performed. On the other hand, if at least one staining and imaging step has not been completed (NO), the process of step S12 is performed.

[0061] (Step S12) At least one of the fluorescent labels used in the most recent step S10 is bleached. Then, in step S10, a target biological material different from the target biological material whose fluorescent label has been reduced or removed by bleaching in step S12 is stained, and a new staining imaging process is performed.

[0062] (Step S131) ​​The correlation extraction unit 104 calculates the correlation between the feature amounts of the target biological substances from the feature amounts of the target biological substances in each constituent region. Here, the correlation extraction unit 104 calculates the correlation between the feature amounts calculated in step S105 for each well W in all staining imaging steps of the multiplexed fluorescence staining. This allows the analysis system 1 to calculate the correlation between the feature amounts of each target biological substance. The correlation extraction unit 104 stores the correlation calculated for each well W in the cell information storage unit 203. Note that, when the graphical lasso method is used, for example, the processing of step S106 may be performed simultaneously with step S131.

[0063] (Step S132) The correlation extraction unit 104 determines whether the process of calculating correlations in step S131 has been performed on the cells in all wells W in the well plate WP. If the process has been performed on the cells in all wells W (YES), the process of step S133 is performed. On the other hand, if the process has not been performed on the cells in at least one well W (NO), the process of step S131 is performed.

[0064] (Step S133) The result output unit 300 generates a network representing the correlations from the correlations calculated for each well W in step S132. For example, the result output unit 300 generates a covariation network. The result output unit 300 may extract a specific correlation from multiple correlations based on the likelihood of the feature amounts, and generate a covariation network based on the extracted specific correlation. (Step S134) The display unit 30 displays the network generated in step S133 on the display.

[0065] <Covariation Network> An example of a specific correlation in this embodiment will be described in detail. Hereinafter, in the covariation network, target biological substances such as proteins are referred to as "nodes." In other words, a "node" is a target biological substance that can be identified by fluorescent labeling using staining. The region (constituent region) of a component within a cell where a node exists is referred to as a "location." Cellular organelles, such as the cell nucleus, lysosomes, Golgi apparatus, and mitochondria, i.e., intracellular components such as organelles, can be either "nodes" or "locations." A network of intracellular structures is represented by connecting multiple nodes with edges. Note that a "location" may also be an area represented by coordinates (pixels) of a captured image.

[0066] FIG. 7 is a schematic diagram illustrating a network according to this embodiment. In the example shown in FIG. 7, at location 50, the correlation between the feature of node P1 and the feature of node P2 is shown by connecting them by an edge 61. In this embodiment, the correlation between the feature of node P1 and the feature of node P2 is shown by connecting them by an edge 61, thereby displaying the entire intracellular network. This allows the network to be compared with pathways published by KEGG (Kyoto Encyclopedia of Genes and Genomes) and the like.

[0067] Figure 7 shows an example of the relationship between edges and the feature values ​​of the nodes connected by the edges. As mentioned above, feature values ​​include the area, shape, average brightness, and variance of the cell image. By being able to confirm what feature values ​​are associated with the nodes connected by the edges, biological implications can be obtained.

[0068] 8A and 8B are schematic diagrams illustrating another example of a network according to the present embodiment. Fig. 8A shows that the shape of node P1 and the shape of node P2 are connected by an edge 62 indicated by a solid line.

[0069] Figure 8(2) shows that the shape of node P1 and the brightness of node P2 are connected by a dashed edge 63. Since the shape is related to protein aggregation and the brightness is related to protein concentration, it is suggested that the concentration of node P2 may cause aggregation of node P1.

[0070] FIG. 8(3) shows that the luminance of node P1 and the luminance of node P2 are connected by an edge 64 indicated by a dashed line.

[0071] 8 illustrates, as an example, the cases where the shape of node P1 and the shape of node P2 are linked, the case where the shape of node P1 and the luminance of node P2 are linked, and the case where the luminance of node P1 and the luminance of node P2 are linked, but this is not a limitation. For example, the shape of node P1 and the variance of node P2 may be linked, the luminance of node P1 and the shape of node P2 may be linked, or the luminance of node P1 and the variance of node P2 may be linked. Furthermore, for example, the variance of node P1 and the shape of node P2 may be linked, or the variance of node P1 and the luminance of node P2 may be linked, or the variance of node P1 and the variance of node P2 may be linked. In other words, for all combinations of the feature amounts of node P1 and node P2, i.e., for nine types of combinations, the correlation between the feature amounts is indicated by being linked by edges.

[0072] Furthermore, the magnitude of the correlation value of multiple nodes may be displayed by the length of the edge connecting the multiple nodes, or may be represented by the thickness of the edge. Here, the correlation value of multiple nodes can be calculated by any method. For example, when the correlation value of multiple nodes is represented by the thickness of the edge connecting the multiple nodes, the edge may be made thicker as the correlation value increases, and the edge may be made thinner as the correlation value decreases.

[0073] FIG. 9 is another schematic diagram for explaining a network according to this embodiment. FIG. 9 shows an example of edge types. Edge types include "promotion" and "inhibition." When the feature value of one node changes from a steady state, and the feature value of the other node changes from the steady state, the nodes are connected by an edge indicating "promotion." Hereinafter, one node connected by an edge indicating promotion is referred to as the promotion source, and the other node is referred to as the promotion destination.

[0074] Furthermore, when the feature value of one node changes from a steady state and the feature value of the other node returns to the steady state, the nodes are connected by an edge indicating "inhibition." Hereinafter, one node connected by an edge indicating inhibition is referred to as the "inhibitor," and the other node is referred to as the "inhibitor." In this way, edges may represent causal relationships (edges with directions) instead of correlations. Note that when the feature value of one of multiple nodes increases, the feature value of the other node also increases, the nodes may be connected by an edge indicating "promotion." Furthermore, when the feature value of one of multiple nodes increases, the feature value of the other node decreases, the nodes may be connected by an edge indicating "inhibition."

[0075] FIG. 9(1) shows that the shape of node P1 and the shape of node P2 are connected by an edge 62A, which is a solid line with an arrow pointing from node P2 to node P1. This indicates that when the feature value of node P2 changes from its steady state, the feature value of node P1 also changes from its steady state. FIG. 9(2) shows that the shape of node P1 and the shape of node P2 are connected by an edge 62B, which is a solid line with a rhombic shape on the node P1 side. This indicates that when the feature value of node P2 changes from its steady state, the feature value of node P1 returns to its steady state. While FIG. 9 shows promotion and inhibition for combinations of the shape of node P1 and the shape of node P2, this example is not limiting. For example, for all combinations of the feature value of node P1 and the feature value of node P2, an edge indicating promotion or inhibition can be displayed based on the correlation between the features.

[0076] The result output unit 300 may switch the display of correlations between feature amounts and location information. Specifically, the result output unit 300 switches the display resolution of correlations between feature amounts and location information in accordance with an operation signal input by the user. The processing performed by the result output unit 300 will be described in detail below.

[0077] For example, the result output unit 300 switches edges connecting multiple nodes in accordance with the operation signal, thereby making it possible to compare the network with the KEGG.

[0078] <Edge Integration> When a single correlation represents multiple correlations between feature quantities, the result output unit 300 switches the display from the single correlation to multiple correlations between feature quantities. That is, the result output unit 300 integrates multiple edges connecting feature quantities between two nodes. Here, the correlated feature quantities of the two nodes connected by the edge to be integrated may be the same or different. Furthermore, when there are multiple edges indicating promotion connecting two nodes, the result output unit 300 may integrate the multiple edges indicating promotion. In this case, the result output unit 300 may integrate edges whose promotion source and promotion destination are the same, i.e., whose edge directions are the same.

[0079] Furthermore, when there are multiple edges indicating inhibition connecting two nodes, the result output unit 300 may integrate the multiple edges. In this case, the result output unit 300 may integrate edges where the inhibition source and inhibition destination are the same, that is, edges with the same edge direction. Furthermore, when there are multiple edges connecting two nodes and the multiple edges include an edge with a positive correlation value and an edge with a negative correlation value, the result output unit 300 may integrate the edges with the positive correlation value and the edges with the negative correlation value.

[0080] <Edge Separation> Furthermore, when multiple correlations between feature quantities are displayed, the result output unit 300 switches the display of the multiple correlations between the feature quantities to a single correlation. In other words, when an edge connecting two nodes is formed by integrating multiple edges, the result output unit 300 separates the edge for each combination of feature quantities. Here, the feature quantities having correlation between the two nodes connected by the separated edge may be the same or different. Furthermore, when an edge connecting two nodes is formed by integrating multiple edges, the result output unit 300 may separate the edge into an edge indicating promotion and an edge indicating inhibition. In this case, the result output unit 300 may separate the edges into edges with the same direction. Furthermore, when an edge present at a specific location is formed by integrating multiple edges, the result output unit 300 may separate the edge. Furthermore, when an edge connecting specific nodes is formed by integrating multiple edges, the result output unit 300 may separate the edge.

[0081] <Deleting and Adding Edges> The result output unit 300 deletes correlations between feature quantities. That is, the result output unit 300 deletes edges connecting two nodes. For example, the result output unit 300 deletes edges where the correlation value of feature quantities between two nodes is less than a predetermined threshold, and edges that are lower than a predetermined order from the top. The result output unit 300 also deletes edges where intracellular structures indicated by two nodes are the same, edges connected to nodes not related to a preset node, and edges that show correlations in preset feature quantities. The result output unit 300 may also change the correlation coefficient. The result output unit 300 may also display the correlations between the deleted feature quantities.

[0082] A specific example of processing performed by the result output unit 300 will be described below. FIG. 10 is another schematic diagram for explaining the network according to this embodiment. The left diagram of FIG. 10 shows that location 51 and location 52 exist within location 50, that node P1 exists in location 51, and that nodes P1 and P2 exist in location 52. Furthermore, node P1 present in location 51 is connected to node P1 present in location 52 by edge 61, and nodes P1 and P2 present in location 52 are connected by edges 62, 63, and 64. The right diagram of FIG. 10 shows node P1 and node P2 present in location 52 in the left diagram of FIG. 10 connected by edge 65.

[0083] For example, a case will be described in which, with the network shown on the left in FIG. 10 displayed on the display unit 30, an operation to check thick edges, specifically, an operation to integrate edges 62, 63, and 64, is performed on the analysis device 10. When an operation signal representing the integration operation is supplied to the result output unit 300, the result output unit 300 acquires correlations between feature quantities of the network displayed on the display unit 30. The result output unit 300 then switches the granularity of the correlations between the feature quantities in response to the operation signal. Specifically, the result output unit 300 integrates multiple correlations between specific feature quantities into a single correlation. For example, the result output unit 300 connects nodes P1 and P2 present at location 52 with edge 65, which integrates edges 62, 63, and 64, to switch to the network shown on the right in FIG. 10. This reduces the number of networks connecting P1 and P2.

[0084] Here, the result output unit 300 may perform statistical processing using the correlation values ​​between the feature amounts of the nodes connected by edge 62, the correlation values ​​between the feature amounts of the nodes connected by edge 63, and the correlation values ​​between the feature amounts of the nodes connected by edge 64, and may display the statistical values ​​obtained by the statistical processing near the edge 65, or may change the thickness of the edge 65 depending on the statistical values. For example, the result output unit 300 may calculate the sum or average of the correlation values ​​between the feature amounts of the nodes connected by edge 62, the correlation values ​​between the feature amounts of the nodes connected by edge 63, and the correlation values ​​between the feature amounts of the nodes connected by edge 64. At this time, the result output unit 300 may perform statistical calculations using the absolute values ​​of the correlation values.

[0085] Furthermore, the result output unit 300 may select the maximum value from the correlation values ​​between the feature amounts of the nodes connected by edge 62, the correlation values ​​between the feature amounts of the nodes connected by edge 63, and the correlation values ​​between the feature amounts of the nodes connected by edge 64, and may display the selected maximum value near edge 65, or may change the thickness of edge 65 depending on the selected maximum value. At this time, the result output unit 300 may perform statistical calculations using the absolute values ​​of the correlation values.

[0086] Furthermore, the result output unit 300 may change the thickness of the edge 65 in accordance with the number of edges between node P1 and node P2 relative to the total number of combinations of the number of feature amounts of node P1 and the number of feature amounts of node P2. Hereinafter, the number of edges showing a high correlation between node P1 and node P2 relative to the total number of combinations of the number of feature amounts of node P1 and the number of feature amounts of node P2 will be referred to as a "high correlation rate."

[0087] FIG. 11 is another schematic diagram illustrating a network according to this embodiment. FIG. 11 shows an example of calculating a high correlation rate between node P1 and node P2. In FIG. 11, m1, m2, and m3 shown in node P1 indicate the features of node P1, and n1, n2, n3, and n4 shown in node P2 indicate the features of node P2. In the upper diagram of FIG. 11, feature m1 and feature n2 are connected with a high correlation rate, feature m3 and feature n2 are connected with a high correlation rate, and feature m3 and feature n3 are connected with a high correlation rate. In other words, the number of edges showing a high correlation connecting the features of node P1 and the features of node P2 is three.

[0088] Here, the total number of edge combinations between node P1 and node P2 is calculated by multiplying the number of feature quantities of node P1 by the number of feature quantities of node P2. In other words, the total number of edge combinations between node P1 and node P2 is 3 × 4 = 12. Therefore, the high correlation rate is 3 / 12 = 0.25. The lower diagram of FIG. 11 shows that when the correlation value is expressed as a number rate, integrating the edges in the upper diagram of FIG. 11 results in a correlation value of 0.25. While the edge connecting the feature quantities of node P1 and node P2 exhibits a high correlation, it may also be designated as an edge regardless of the level of correlation. The result output unit 300 may recognize as an edge those features m1, m2, and m3 of node P1 and features n1, n2, and n3 of node P2 that are found to be correlated. 11, if there are edges between m1 and n2, between m3 and n2, and between m3 and n3, the result output unit 300 calculates that the number of edges is 3 / 12=0.25 out of a total of 3×4=12. In this case, the result output unit 300 can calculate the ratio of correlated edges to correlable edges as the number rate.

[0089] The result output unit 300 outputs information indicating that the granularity of the correlation between specific feature quantities has been switched to the result output unit 300. The result output unit 300 displays the network shown on the right side of Fig. 10 on the display unit 30 based on the information indicating that the granularity of the correlation between specific feature quantities has been switched, which is supplied from the result output unit 300. The edge 65 is displayed with a thickness according to the correlation value between the nodes connected by edge 62, the correlation value between the nodes connected by edge 63, and the correlation value between the nodes connected by edge 64, so that the user can check the correlation value between node P1 and node P2 from the thickness of the edge.

[0090] Returning to FIG. 10 , the following description will continue. For example, a case will be described in which an operation to check the feature quantities, specifically an operation to separate edge 65, is performed on the analysis device 10 while the network shown on the right side of FIG. 10 is displayed on the display unit 30. When the result output unit 300 detects an operation signal representing the separation operation, it acquires the correlations between the feature quantities of the network displayed on the display unit 30. Then, the result output unit 300 switches the granularity of the correlations between the feature quantities in response to the operation signal. Specifically, the result output unit 300 separates the correlations between specific feature quantities into multiple correlations. For example, the result output unit 300 separates edge 65 into edges 62, 63, and 64, and switches to the correlations between the feature quantities of the network shown on the left side of FIG. 10 .

[0091] The result output unit 300 outputs information indicating that the number of edges representing the correlation between specific feature quantities has been switched to the result output unit 300. The result output unit 300 displays the network on the display unit 30 based on the information indicating that the granularity of the correlation between specific feature quantities has been switched, which is supplied from the result output unit 300. Since the edge 65 is displayed by the edge 62, the edge 63, and the edge 64, the user can confirm the feature quantities having a correlation between the node P1 and the node P2.

[0092] <Network Example> FIG. 12 is a schematic diagram illustrating an example of a network according to this embodiment. FIG. 12 is a schematic diagram illustrating a network generated for cells at specific time points. In this diagram, each node represented by a circle containing a character string represents a target biological substance, such as a protein or mitochondria. In this diagram, each node represented by a circle containing a character string represents a target biological substance, such as a protein or mitochondria. Lines (edges) connecting nodes represent correlations. Circles at both ends of edges indicate protein localization information for each node (also referred to as "subnodes"). Subnodes are identified by localization information and are represented, for example, by different colors. For example, subnodes represent the nucleus, cytoplasm, and actin domain, and are represented by different colors (e.g., blue, red, and yellow). This allows the analysis system 1 to provide a user with information that makes it easy to recognize correlations between target biological substances represented by nodes and those localized at locations represented by each subnode.

[0093] For example, FIG. 12 shows a covariation network representing a group of proteins that move in synchronization with a specific stimulus. A node with many lines (edges) (e.g., pAkt (Ser473)) can be seen to be an important molecule that controls the transmission of a specific stimulus. Furthermore, communities consisting of groups of molecules with strong connections can be extracted from the covariation network using a graph clustering method or the like. For example, communities related to synthesis or regulation, such as glycogen synthesis or gluconeogenesis regulation, can be extracted. In each community network, if a specific node has many subnodes of a specific type, it can be seen that the location indicated by the subnode accumulates enzymes and functions in the synthesis or regulation indicated by the community. The analysis system 1 can extract nodes with many lines (edges) and highlight the nodes as nodes representing important molecules. The analysis system 1 can also extract communities based on the correlation of the covariation network. Furthermore, the analysis system 1 can highlight nodes or subnodes as noteworthy nodes based on the number of subnodes of a specific type in each node in the entire covariation network or in each community. Highlighting includes display modes such as changing the color of a node from the color of other nodes, surrounding the node, adding a mark such as an arrow, or lighting up the node.

[0094] Summary of the Embodiment As described above, in the present embodiment, the analysis system 1 acquires a first image in which a first target biological substance (an example of a first constituent element) is labeled on a cell, and a second image in which a second target biological substance (an example of a second constituent element) different from the first target biological substance is labeled on the cell. The first and second images are captured using different staining imaging processes (see FIG. 3). The feature calculation unit 102 extracts features for each target biological substance constituting the cell from each of the first and second images (see FIG. 5). The correlation extraction unit 104 uses these features to calculate the correlation between the first target biological substance and the second target biological substance. This allows the analysis system 1 to identify more target biological substances using the same cell and extract more features from the same cell. For example, the analysis system can extract a number of features calculated by multiplying the number of types of features by the number of labels (6 or more, e.g., 30). Therefore, the analysis system 1 can calculate correlations using more features, thereby enabling more accurate cell analysis. Furthermore, since the analysis system 1 uses the same cells, the correlation between the cells themselves does not decrease compared to when other cells are used. In other words, when different cells are used, the distribution of target biological substances within the cells is different, and the accuracy of the correlation may not necessarily be good. In contrast, the analysis system 1 according to this embodiment uses the same cells, so it is possible to image and calculate the correlation while maintaining the distribution of target biological substances, and the accuracy of the correlation can be improved compared to when other cells are used.

[0095] Furthermore, because the analysis system 1 can extract many feature quantities from the same cell, it may be possible to sufficiently calculate correlations, for example, without culturing and observing similar cells. Furthermore, because the analysis system 1 observes the same cell, it can obtain feature quantities with high correlations by comparing them with other cells. Furthermore, because the analysis system 1 can distinguish target biological substances by labeling, it can extract feature quantities for each target biological substance with higher accuracy than when no distinction is made.

[0096] Furthermore, in this embodiment, the feature calculation unit 102 in the analysis system 1 identifies at least some single cells and extracts feature values ​​of the first target biological substance and the second target biological substance for each single cell. For example, in the example of FIG. 5 , the analysis system 1 extracts feature values ​​of protein 1 and protein 3 for each single cell 1A-1E. The correlation extraction unit 104 calculates correlations using feature values ​​of the first target biological substance and the second target biological substance for each single cell. This allows the analysis system 1 to acquire, for the same cell, feature value sets of target biological substances identifiable by their labels, equal to the number of single cells. For example, the analysis system 1 can extract feature values ​​equal to the number of feature types multiplied by the number of labels (6 or more, e.g., 30), and then multiplied by the number of single cells. Therefore, the analysis system 1 can calculate correlations using a larger number of feature values, thereby enabling more accurate cell analysis. For example, the analysis system 1 can sufficiently calculate correlations without acquiring time-varying changes in each feature value.

[0097] In this embodiment, the cell image acquisition unit 101 acquires a first image and a second image of the same cell at a specific time point, each image captured with a different label. The same specific time point includes the time interval between capturing the first and second images. For example, the same specific time point includes the time within the time when the process of removing the label from the first target biological substance and the process of labeling the second target biological substance are performed. The feature calculation unit 102 extracts feature values ​​of the first target biological substance and the second target biological substance for at least some of the single cells. The correlation extraction unit 104 calculates the correlation at the specific time point using the feature values ​​of the first target biological substance and the feature values ​​of the second target biological substance for each single cell. In this way, the analysis system 1 extracts feature values ​​of the first target biological substance and the feature values ​​of the second target biological substance for each single cell, thereby enabling the correlation to be calculated at each time point. Since the analysis system 1 can acquire feature values ​​for at least the number of single cells, it can sufficiently calculate the correlation without, for example, acquiring the time change of each feature value, thereby generating a network at a specific time point.

[0098] However, the analysis system 1 may calculate correlations or generate networks using feature quantities at multiple time points. For example, the analysis system 1 may set multiple time intervals and calculate correlations for each time interval using feature quantities at time points included in each time interval. The multiple time intervals may be set by the user or may be automatically set by the analysis system 1. When automatically setting the time intervals, for example, the analysis system 1 may automatically set the time intervals so that the number of feature quantity samples is equal to or greater than a predetermined number, or so that the number of feature quantity samples at each time interval is approximately the same. Furthermore, for example, the analysis system 1 may automatically set the time intervals so that the change in correlation (e.g., difference, rate of change) using feature quantities at time points included in the time interval falls within a predetermined range. Note that the automatic setting may also include proposing the automatic setting, and may include a process in which the user approves the proposal.

[0099] (Modification 1) The analysis system 1 may calculate correlations for each of a plurality of specific points in time, and generate and display a network.

[0100] <Network Example> FIG. 13 is a schematic diagram illustrating an example of a network according to Modification Example 1 of this embodiment. This diagram is a schematic diagram illustrating a case where multiplexed fluorescent staining is performed at specific time intervals. In this manner, the analysis system 1 may generate a network by calculating the correlation of feature amounts at each time point and display the network along the time axis. This diagram illustrates a case where multiplexed fluorescent staining is performed on cells at elapsed times t = 5 minutes, 15 minutes, and 30 minutes, and a network is generated based on the feature amounts at each time point. The analysis system 1 displays the networks at each time point in a comparable manner. For example, the analysis system 1 may display the network along the time axis of elapsed time t, or may display a video whose playback time elapses along the time axis of elapsed time t. The analysis system 1 may also calculate the change in correlation (e.g., difference between specific time points, rate of change) using the feature amounts at each elapsed time t, and highlight networks or nodes whose change in correlation is greater than or equal to a predetermined value. The predetermined value may be determined in advance or based on a statistical value or ranking of the change in total correlation.

[0101] <Feature Amount Information> Fig. 14 is a schematic diagram showing an example of feature amount information according to Modification Example 1. This diagram shows feature amount information for one well W. When multiple wells W are used to observe multiple cells, the feature amount information in Fig. 14 is provided for each well W. The feature amount information in this diagram is associated with staining identification information, substance identification information, and cell composition information for a time point indicating a specific time point and cell identification information identifying a single cell. In other words, each piece of feature amount information represents a feature amount of a target biological substance (protein in this diagram) indicated by substance identification information for each specific time point (t1, t2, ...) for each composition within a single cell in the observation target cell. In this diagram, the feature amount for a cell is expressed as a value X c , p ,e ，t where t represents a specific time point. The feature amount at the specific time point t1 is expressed as X c , p ,e ,ｔ１ and the value X in FIG. c , p ,e is equivalent to

[0102] <Example of Operation of Analysis System> Figure 15 is a flow diagram showing an example of the operation of the analysis system 1 according to Modification 1. Note that the operation shown here is just an example, and operation steps may be omitted or added. Comparing Figure 15 with Figure 6, steps S21, S22, S23, S231, S233, and S234 are different. The processing of each of the other steps is the same as the processing of the steps with the same reference numerals in Figure 6, so a description of the processing of the other steps will be omitted.

[0103] (Step S21) The cell image acquisition unit 101 acquires time information representing multiple specific time points (in the example of FIG. 13, t=5 minutes, 15 minutes, and 30 minutes), and determines whether each of these specific time points has been reached. If a specific time has been reached (YES), the process of step S22 is performed. If a specific time has not been reached (NO), the cell image acquisition unit 101 repeats the process of step S21.

[0104] (Step S22) The cell image acquisition unit 101 determines whether the multi-fluorescence staining has been completed for all specific time points. The cell image acquisition unit 101 may, for example, determine whether a preset end time has been reached, or may determine the completion of the multi-fluorescence staining based on a user operation. If the multi-fluorescence staining has been completed for all time points (YES), the process of step 231 is performed. At this point, the analysis system 1 acquires and stores the feature information shown in FIG. 14 . On the other hand, if the multi-fluorescence staining has not been completed for at least one time point (NO), the process of step S10 is performed on the cell at that specific time point, thereby starting a second multi-fluorescence staining. Note that the cell re-stained in step S10 is the same cell as the cell used in the previous multi-fluorescence staining. However, the cell does not have to be the same cell; it may be a cell cut from a target object, such as a cell fragment, at a location different from the location previously stained with the multi-fluorescence staining.

[0105] (Step S231) The correlation extraction unit 104 calculates correlations between feature quantities of target biological substances for each specific time point based on the feature quantities of the target biological substances in each constituent region (the feature quantity information in FIG. 15 ). Note that, when the graphical lasso method is used, the processing of step S106 may be performed simultaneously with step S231. (Step S23) The correlation extraction unit 104 determines whether correlation calculations have been completed for all specific time points. If correlation calculations have been completed for all specific time points (YES), the processing of step S233 is performed. If correlation calculations have not been completed for at least one specific time point (NO), the processing of step S231 is performed for the specific time points for which correlation calculations have not been completed. (Step S233) The result output unit 300 generates a network representing the correlations for each specific time point from the correlations calculated for each well W in step S132. (Step S134) The display unit 30 displays the network generated in step S133 on the display for each specific time point.

[0106] As described above, in the analysis system 1 according to this modification, the cell image acquisition unit 101 acquires time information representing multiple specific time points. The feature calculation unit 102 identifies at least some single cells from each of the first and second images for each identical cell, and extracts feature amounts of the first target biological substance and the second target biological substance for each of the single cells. The correlation extraction unit 104 calculates a correlation for each of the multiple specific time points. For example, in the example of FIG. 12 , the feature calculation unit 102 extracts feature amounts at each of the specific time points at t = 5 minutes, 15 minutes, and 30 minutes, and the correlation extraction unit 104 calculates a correlation for each specific time point.

[0107] This allows the analysis system 1 to calculate information such as correlation at each specific time point and changes in correlation between specific time points. The analysis system 1 identifies multiple types of target biological substances in the same cell and calculates feature quantities for all of the target biological substances for each single cell (see FIG. 5). At each specific time point, the analysis system 1 can extract a number of feature quantities calculated by multiplying the number of feature types by the number of labels (6 or more, for example, 30), and then multiplying that number by the number of single cells, thereby generating a network for each specific time point.

[0108] Furthermore, the result output unit 300 outputs the correlation between the first target biological substance and the second target biological substance for each specific time point. For example, the result output unit 300 generates a network for each specific time point, and the display unit 30 displays each network on a display so that it can be compared between specific time points. This allows the analysis system 1 to provide the user with information such as the correlation at each specific time point and changes in the correlation between specific time points.

[0109] (Modification 2) The analysis system 1 may calculate correlations for each of a plurality of cell cycles, and generate and display a network.

[0110] FIG. 16 is a schematic diagram illustrating an overview of Modification 2 according to this embodiment. The analysis system 1 uses multiplexed fluorescent staining to detect the dynamics of target biological materials in multiple (e.g., 30) stains, thereby identifying the cell cycle. The cell cycle refers to the series of events that occur during the process in which one cell produces two daughter cells, and the cycle itself. The cell cycle is divided into interphase and M phase. Interphase is divided into G1, S, and G2 phases. M phase consists of mitosis and cytokinesis. During mitosis, sister chromatids separate at both poles of the cell, and during subsequent cytokinesis, the cytoplasm splits to produce two cells. Cells that temporarily or reversibly cease division are considered to have entered a stationary phase called G0 phase.

[0111] An example of the feature calculation unit 102 of the analysis system 1 identifying the cell cycle will be described. However, the feature calculation unit 102 may also identify the cell cycle using other methods (such as other staining or other image analysis), such as using DNA content. For example, when PCNA (proliferating cell nuclear antigen) staining is performed, the feature calculation unit 102 identifies cells that exhibit intranuclear punctate structures as cells in the S phase (late G1 to S phase). An intranuclear punctate structure is a structure in which many punctate points are present in the nucleus of a cell in an image, and this is, for example, a cell in which nuclear pore complexes are unevenly distributed.

[0112] The feature calculation unit 102 stains cyclin B with, for example, an antibody against cyclin B, and identifies cells with high nuclear brightness as cells in the early G2 phase (G2 phase to M phase). Cells with high nuclear brightness are, for example, cells with brightness values ​​higher than a threshold, cells with a region with a brightness value higher than the threshold having a larger area than a threshold, cells with a ratio of the region with a brightness value higher than the threshold being equal to or greater than a reference value, or cells with a circularity of the region with a brightness value higher than the threshold being equal to or greater than a threshold. The feature calculation unit 102 determines whether a cell is in the M phase based on the brightness value and circularity of a cell stained with, for example, DAPI (4',6-diamidino-2-phenylindole). Circularity is a numerical value that represents the complexity of a figure drawn in an image or the like. The maximum circularity value is 1, and the more complex the figure, the smaller the numerical value becomes. The feature calculation unit 102 classifies cells in which the circularity of a region having a brightness value higher than a threshold is equal to or greater than a threshold as M-phase cells. The feature calculation unit 102 stores the identified cell cycle in the storage unit 200 as part of feature information.

[0113] <Feature Information> FIG. 17 is a schematic diagram showing an example of feature information according to Modification Example 2. This diagram shows feature information for one well W. Note that when multiple wells W are used to observe multiple cells, the feature information shown in FIG. 17 is provided for each well W. The feature information in this diagram associates feature information for each staining identification information, material identification information, and cell configuration information with a time point indicating a specific time point, a cell cycle, and cell identification information identifying a single cell. The feature information in FIG. 17 is obtained by adding the cell cycle information for each cell to the feature information in FIG. 14 . For example, at a specific time point t1, cells 1A and 1C are in the S phase, cells 1B and 1E are in the G2 phase, and cell 1D is in the M phase. Cell 1E is in the G2 phase at the specific time point t1, but is in the M phase at the specific time point t2.

[0114] The correlation extraction unit 104 calculates the correlation between the feature amounts of the target biological substances in each component region for each specific time point and cell cycle from the feature amounts of the target biological substances (the feature amount information in FIG. 17 ). However, the correlation extraction unit 104 does not have to calculate the correlation for each specific time point, and may calculate the correlation for each cell cycle using the feature amounts for multiple specific time points.

[0115] <Network Example> FIG. 18 is a schematic diagram showing an example of a network according to Modification Example 2. FIG. 17 is a schematic diagram showing a network generated for each cell cycle. In the network shown in this figure, target biological substances are arranged at equal intervals around a circle (or at the vertices of a regular polygon). Each equally spaced node represents a target biological substance. Lines connecting the nodes represent correlations. The points where the lines indicating correlations connect at each node are edges, and the edges represent types of feature quantities. According to the network shown in this figure, it can be seen that the covariation networks for the G1 and S phases are similar, but the G2 phase is completely different. By using this network, it is possible to estimate a consensus on cellular states, such as cellular responses to drugs and drug efficacy, from differences in the structure of the covariation network, enabling drug screening based on the network structure.

[0116] As described above, in the analysis system 1 according to this modification, the feature calculation unit 102 identifies the cell cycle of each single cell from each of the first image in which the first target biological substance is labeled and the second image in which the second target biological substance is labeled. The feature calculation unit 102 extracts features for the first target biological substance and features for the second target biological substance for each cell cycle. The correlation extraction unit 104 calculates the correlation for each cell cycle. This allows the analysis system 1 to analyze cells in each cell cycle.

[0117] Alternatively, the result output unit 300 may calculate correlations of feature quantities at specific time points and for each cell cycle to generate networks, and the display unit 30 may display each network along the time axis for each cell cycle. Alternatively, the result output unit 300 may calculate changes in correlation for each specific time point for each cell cycle and generate display information according to the magnitude of the changes. For example, the result output unit 300 may highlight networks or nodes for which the change in correlation is equal to or greater than a predetermined value for each cell cycle.

[0118] (Other Modifications, etc.) The analysis system 1 may calculate correlations for each pathological condition, and generate and display a network. For example, cells are collected from an affected area for each pathological condition, and the collected cells are used as the subject of observation (test) by the analysis system 1. The analysis system 1 extracts feature amounts from the cells to be observed for each pathological condition, calculates correlations between the feature amounts, and generates a network.

[0119] Specifically, the cell image acquisition unit 101 acquires a first image in which a first target biological substance is labeled and a second image in which a second target biological substance is labeled for each of a cell to be examined in a first pathological state and a cell to be examined in a second pathological state different from the first pathological state. The feature calculation unit 102 extracts features for each target biological substance constituting the cell from the first image and the second image for each of the first pathological state and the second pathological state. The correlation extraction unit 104 calculates the correlation between the first target biological substance and the second target biological substance for each of the first pathological state and the second pathological state.

[0120] The analysis system 1 may display the network along the progression (time axis) of the pathology. The analysis system 1 may display a network for each pathological state for each cell cycle. The result output unit 300 may also calculate a change in correlation according to a change in the pathological state and generate display information according to the magnitude of the change. For example, the result output unit 300 may highlight networks or nodes whose change in correlation is equal to or greater than a predetermined value.

[0121] The analysis system 1 may associate the calculated correlation with analysis data obtained by another observation for the cells being observed. The analysis data may be, for example, sequence data from a genetic analysis. The analysis system 1 associates the correlation with the analysis data via a single cell, a component of a single cell, a target biological material, or a specific time point, or a combination thereof. The analysis system 1 may calculate and output the correlation between the correlation and the analysis data.

[0122] Specifically, the result output unit 300 acquires analytical data obtained by analyzing related elements (e.g., gene sequences) different from the target biological material. The result output unit 300 associates and outputs the correlation between the analytical data of the related elements and the target biological material. When associating the analytical data with the correlation via a specific time point, the cell image acquisition unit 101 acquires analytical data for each specific time point. The result output unit 300 associates and outputs the correlation between the analytical data of the related elements and the target biological material for each specific time point. Note that the analytical data may be data obtained by analysis using fluorescent staining that is the same as or similar to the multiplex fluorescent staining. This allows the analysis system 1 to associate the correlation with other analytical data and provide information.

[0123] In the above embodiment (including modified examples, the same applies throughout this specification), the correlation may include information based on the arrangement of single cells. The arrangement may be a value based on the relative positions of the single cells, or a value based on the absolute position on the cell or on the image. These values ​​are, for example, the distance between the single cells, or the number of cells (minimum value) present between the single cells. The distance between the single cells may be the distance between the centers of the single cells, or the minimum distance among the distances between the boundaries of the single cells. Furthermore, these values ​​related to the arrangement may be the distance on each axis of space.

[0124] In the above embodiment, the feature quantities of the first and second target biological substances include the amounts of the labeled target biological substances. As an example, a luminance value corresponds to this feature quantity. The labels of the first and second target biological substances are normalized by the brightness per unit of the first and second target biological substances. The analysis system 1 calculates the amounts of the first and second target biological substances based on brightness information representing the brightness of the first and second images. Here, normalization may be performed using the staining substance used for fluorescent staining, or may be performed based on the luminance values ​​of each fluorescent stain in the captured image. Specifically, for each staining substance, a luminance value (also referred to as a "unit luminance value") per unit or a predetermined number of target biological substances is measured. The feature quantity calculation unit 102 determines the feature quantity by dividing the measured luminance value by the unit luminance. The unit brightness value may be calculated in advance or may be calculated during observation using multiple fluorescence staining (for example, after step S11 or before step S131 in FIG. 6, or after step S22 or before step S232 in FIG. 15). This allows the analysis system 1 to standardize the amount of target biological material between each staining imaging process even if different fluorescent stains are used, thereby making it possible to compare the amounts of target biological materials.

[0125] The specific time point may be a time interval. For example, the same specific time point includes a time point within a specific period in the cell cycle (e.g., G1 (Gap 1) phase). Within a specific period in the cell cycle (within the G1 period), cells are unlikely to divide, and therefore can be considered to be the same cell. The specific period is within the cell cycle, and may be, for example, the G0 (Gap 0) phase, the S (synthesis) phase, the G2 (Gap 2) phase, or an interphase. Furthermore, for example, the same specific time point may include a time point within the time between the step of removing the label from the first target biological material and the step of labeling the second target biological material. In other words, the period of imaging using a single multifluorescence staining may be considered to be a specific time point.

[0126] <Analysis> The following describes the processing performed by the analysis system 1 when calculating feature amounts. Cells under observation are separated on an image, and cell regions are calculated. Here, an example for quantitatively analyzing various phenomena is shown below.

[0127] <Protein Localization Analysis> The feature calculation unit 102 detects a region of interest based on user operations and settings, and extracts feature amounts of biological substances that travel between the nucleus and cytoplasm of cells separated by image analysis within the region of interest. These biological substances are fluorescently stained with antibodies, fluorescent proteins, or the like. The feature calculation unit 102 extracts feature amounts of, for example, proteins, which are substances that travel between the nucleus and cytoplasm of cells. For example, when a protein localized inside the nucleus moves to the cytoplasm that surrounds the outside of the nucleus, the feature calculation unit 102 extracts the following values ​​as feature amounts from images before and after the protein movement. Furthermore, even when a protein localized in the cytoplasm moves inside the nucleus, the feature calculation unit 102 extracts feature amounts from images before and after the protein movement. Note that the feature amounts shown below are merely examples, and other feature amounts may be extracted.

[0128] For example, the feature amounts are total luminance value of the nucleus / total luminance value of the cytoplasm, total luminance value of the nucleus / total luminance value of the cells, average luminance value of the nucleus / average luminance value of the cytoplasm, average luminance value of the nucleus / average luminance value of the cells, variance of luminance values ​​within the nucleus, average luminance values ​​within the nucleus, variance of luminance values ​​within the cytoplasm, average luminance values ​​within the cells, average luminance values ​​within the cytoplasm, median luminance value of the nucleus / median luminance value of ... cells, median luminance values ​​within the nucleus, and median luminance value of the cytoplasm.

[0129] The feature calculation unit 102 may also extract feature amounts of a substance traveling between the nuclear membrane and the nucleus of a cell, or feature amounts of a substance traveling between the nuclear membrane and the cytoplasm of a cell. The feature calculation unit 102 extracts, for example, feature amounts of a protein such as Nup98 as a substance traveling between the above-mentioned cellular structures. For example, when a protein localized in the nuclear membrane (nucleus) of a cell moves to the nucleus (nuclear membrane) of the cell, or when a protein localized in the nuclear membrane (cytoplasm) of a cell moves to the cytoplasm (nuclear membrane) of the cell, the feature calculation unit 102 extracts the following values ​​as feature amounts from images before and after the protein transfer. Note that the feature amounts shown below are merely examples, and other feature amounts may also be extracted.

[0130] For example, the feature quantities are total luminance value of the nuclear membrane / total luminance value of the nuclei, average luminance value of the nuclear membrane / average luminance value of the nuclei, variance of luminance values ​​within the nucleus, number of bright spots within the cytoplasm, average luminance value of bright spots within the cell, total luminance value of bright spots within the cell (total value per cell), area of ​​each bright spot, average area of ​​bright spots per cell, median luminance value of the nuclear membrane / median luminance value of the nuclei, total luminance value of the nuclear membrane / total luminance value of the cytoplasm, average luminance value of the nuclear membrane / average luminance value of the cytoplasm, median luminance value of the nuclear membrane / median luminance value of the cytoplasm, nuclear membrane Total brightness value of the nuclear membrane / total brightness value of the cells, mean brightness value of the nuclear membrane / mean brightness value of the cells, median brightness value of the nuclear membrane / median brightness value of the cells, total brightness value of the nuclear membrane, mean brightness value of the nuclear membrane, nuclear membrane brightness variance, nuclear membrane median, variance of brightness values ​​within the cytoplasm, mean brightness value within the cell, mean brightness value within the cytoplasm, median brightness value of the nucleus / median brightness value of the cytoplasm, median brightness value of the nuclear membrane / median brightness value of the cells, median brightness value within the nucleus, median brightness value of the cytoplasm.

[0131] <Analysis I of Protein Aggregate Formation> When a substance uniformly distributed in a predetermined region of a cell forms an aggregate (spot), the feature calculation unit 102 extracts feature amounts of the substance from images before and after the aggregation. For example, the feature calculation unit 102 extracts feature amounts from proteins such as GSK3β and p-GSK3β as substances uniformly distributed in a predetermined region of a cell. For example, the feature calculation unit 102 extracts the following values ​​as feature amounts from images before and after the protein aggregates. Note that the feature amounts shown below are merely examples, and other feature amounts may be extracted.

[0132] For example, the feature values ​​are total luminance value of the nucleus / total luminance value of the cell, total luminance value of the nucleus / total luminance value of the cytoplasm, average luminance value of the nucleus / average luminance value of the cell, average luminance value of the nucleus / average luminance value of the cytoplasm, variance of luminance values ​​within the cell, number of spots, number of spots within the nucleus / number of spots outside the nucleus.

[0133] <Analysis of Protein Aggregate Formation II> When substances uniformly distributed in a predetermined region of a cell are sub-aggregated to form a specific aggregate (domain), the feature calculation unit 102 extracts feature amounts of the substances that form the specific aggregate (domain). The feature calculation unit 102 extracts feature amounts of, for example, proteins that are substances that form the specific aggregate (domain). Examples of proteins include actin, SNX-9, p-Akt (S473), WASH1, and EEA1. The feature calculation unit 102 extracts the following values ​​as feature amounts from images before and after the protein forms the specific aggregate (domain). Note that the feature amounts shown below are merely examples, and other feature amounts may be extracted.

[0134] For example, the features include the variance of the cytoplasmic brightness values, the area of ​​the domains, the number of domains within the cell (total and average values ​​per cell), the total brightness value of the domains / the total brightness value of the cytoplasm, and the average brightness value of the domains / the average brightness value of the cells.

[0135] <Analysis of Protein Co-Localization> When substances uniformly distributed in a predetermined region of a cell are sub-assembled to form a specific aggregate (domain), the feature calculation unit 102 extracts features of other substances that aggregate in the same location as the specific aggregate (domain). Furthermore, when no specific aggregate is formed, the feature calculation unit 102 analyzes whether multiple substances exist in the same location by extracting the features of the substances. For example, the feature calculation unit 102 extracts features of proteins, which are substances that aggregate in the same location as the specific aggregate (domain).

[0136] For example, when actin, a type of protein, forms a domain (actin domain), the feature calculation unit 102 extracts feature amounts from the luminance values ​​of the protein distributed in a certain range around the domain. The feature calculation unit 102 extracts the following values ​​as feature amounts of the luminance values ​​of the protein. Note that the feature amounts shown below are merely examples, and other feature amounts may also be extracted.

[0137] For example, the features may be the variance of the brightness values ​​of proteins in a certain range around the actin domain, the total brightness value of proteins on the actin domain / the total brightness value of proteins in the entire cell, the average brightness value of proteins on the actin domain / the average brightness value of proteins in the entire cell, the total brightness value of proteins in a certain range around the actin domain / the total brightness value of proteins in the entire cell, the average brightness value of proteins on the actin domain / the average brightness value of proteins in a certain range around the domain, or the total brightness value of proteins on the actin domain / the total brightness value of proteins in a certain range around the domain.

[0138] Furthermore, for example, when actin and another protein form a domain, the feature calculation unit 102 extracts the following values ​​as feature amounts based on the formed domain. Note that the feature amounts shown below are merely examples, and other feature amounts may also be extracted.

[0139] For example, the feature amount is the area of ​​the overlapping region of the actin domain and the protein domain / the area of ​​the total region of the actin domain and the protein domain, and the variance of the brightness values ​​of the protein in a certain range around the actin domain.

[0140] <Modification 3: Target Biological Substance Detection> In this modification, the analysis system 1 displays, for example, the display shown in Fig. 19 or Fig. 20. Details of the display will be described later.

[0141] In the covariation network of FIG. 18 , edges based on correlation values ​​are displayed for each combination of nodes. For example, edges with correlation values ​​equal to or greater than a threshold are displayed. When there are a large number of edges, it may be desirable for the analysis system 1 to suggest which nodes (target biological substances) should be focused on. The analysis system 1 according to this modified example displays target biological substances (also referred to as "target generating substances of interest"). Target generating substances of interest are target biological substances that have a significant cellular response or medicinal effect, such as drugs. Displaying target generating substances of interest includes displaying the names and nodes of the target generating substances in a manner that is identifiable, displaying the target generating substances in a manner that is identifiable by humans, etc.

[0142] There may also be a need for users to understand the relationships between covariation networks (drugs or cell cycles), for example, whether notable nodes are similar or different. The analysis system 1 may display a target product substance for each of a plurality of covariation networks, thereby presenting the user with information for determining whether target products are similar or different, for example, for drugs or cell cycles. For example, the analysis system 1 may display target products for each drug, for each cell cycle, or for each drug and cell cycle.

[0143] Specifically, the correlation extraction unit 104 of the analysis system 1 calculates the correlation anomaly of each target product substance for each covariation network. The correlation anomaly represents the degree of change in the correlation value (the breakdown of the correlation) of each target product substance when comparing the covariation network to be calculated (also referred to as the "target network") with a reference covariation network (also referred to as the "reference network"). The reference network is a covariation network generated from the feature quantities of each target biological substance in a cell when a drug solvent (e.g., water) is applied to the cell. However, the reference network may be a fixed covariation network or a covariation network generated from the feature quantities of each target biological substance in a specific cell. Furthermore, the reference network may be a covariation network that is relative to the target network, such as a covariation network before the use of a drug or a change in the cell cycle. A target product substance with a high correlation anomaly is a substance with a large change in correlation value, and is presumed to have a high cellular response or efficacy of the drug or other drug used. The result output unit 300 generates data for displaying target generating substances with high correlation anomalies in an identifiable manner as target generating substances of interest. The generated data is displayed on the display unit 30.

[0144] The correlation anomaly calculation can be performed using, for example, the method described in "Tsuyoshi Ide, 'Anomaly Detection from Latent Graph Structure,' Technical Report of the 1st Workshop on Latent Dynamic, 2010," but other methods may also be used. The correlation extraction unit 104 reads feature quantities (e.g., feature quantities calculated by the feature calculation unit 102 for each drug and cell cycle) in the reference network and the target network, and creates a dataset. The correlation extraction unit 104 also reads ρ set by the user. ρ is a coefficient of a normalization term (penalty term) and serves as a threshold for the correlation coefficient (an example of a correlation value). For example, if the correlation coefficient is below the threshold, that is, if the correlation between the target substances is weak, the correlation is ignored. The correlation extraction unit 104 calculates a sample covariance matrix based on the feature quantities for each network. The correlation extraction unit 104 calculates a precision matrix for each network based on the sample covariance matrix using the graphical lasso method. The correlation extraction unit 104 calculates the correlation anomaly for each target generating substance based on the precision matrices of the reference network and the target network. The correlation anomaly is, for example, the Kullback-Leibler pseudo-distance (referred to as "KL distance"). The correlation extraction unit 104 swaps the data of the reference network and the target network, and determines the larger value of the KL distance when swapped compared to when not swapped as the correlation anomaly.

[0145] FIG. 19 is a schematic diagram illustrating an example of target biological substance detection in Modification Example 3 according to this embodiment. This diagram shows an example of a cluster map displayed as a result of tree-type (hierarchical) cluster analysis performed by the analysis system 1. In this diagram, the vertical axis represents the target biological substance (covariance network nodes), and the horizontal axis represents the cells (covariance networks) in each cell cycle in which each drug was administered. The drugs used were 2 μM (mol / L) bleomycin, 1 μM and 3 μM cytarabine, and 2 mM aspirin, respectively, for cells in the S, G1, G2, or M phases of the cell cycle. The analysis system 1 analyzed these cells and calculated the correlation anomaly.

[0146] In this map, cells of each cell cycle to which each drug was administered are colored according to the degree of correlation anomaly for each target biological substance, and the entire map serves as a heat map of correlation anomalies. In this map, the user can identify target biological substances colored with colors indicating high degrees of correlation anomaly for each cell cycle to which each drug was administered. The order of the vertical and horizontal axes of the map has been rearranged in order of increasing similarity by the correlation extraction unit 104 performing a tree-type cluster analysis based on the degrees of correlation anomaly.

[0147] Figure 19 shows the results when ρ = 0.75. In this figure, the horizontal axis was rearranged in the following order: S-phase cells treated with 2 μM bleomycin (Belomycin 2 μM S), S-phase cells treated with 1 μM cytarabine (Cytarabine 1 μM S), S-phase cells treated with 3 μM cytarabine (Cytarabine 3 μM S), G1-phase cells treated with 2 μM bleomycin (Belomycin 2 μM G1), G1-phase cells treated with 1 μM cytarabine (Cytarabine 1 μM G1), and G1-phase cells treated with 3 μM cytarabine (Cytarabine 3 μM G1). The next order was G2 or M phase cells treated with 1 μM cytarabine (Cytarabine 1 μM S), G2 or M phase cells treated with 2 μM bleomycin (Bleomycin 2 μM S), G2 or M phase cells treated with 3 μM cytarabine (Cytarabine 3 μM S), S phase cells treated with 2 mM aspirin (Aspirin 2 mM S), G1 phase cells treated with 2 mM aspirin (Aspirin 2 mM G1), and G2 or M phase cells treated with 2 mM aspirin (Aspirin 2 mM S).

[0148] From the order of these horizontal axes, it can be seen that when bleomycin and cytarabine are used, the similarity of the correlation abnormality of the target biological substances is more in the cell cycle than in the drug, and the cellular response and efficacy are similar. On the other hand, aspirin has a similar effect on all cell cycles and only a small change in color, it can be seen that it has little effect on the state of normal cells.

[0149] Regarding target biological substances, in S-phase cells treated with bleomycin, SDH (succinate dehydrogenase) is colored with a color indicating a high degree of correlation abnormality, allowing the user to identify it as a target biological substance. Furthermore, in G1-phase cells treated with bleomycin, γH2AX is colored with the color indicating the highest degree of correlation abnormality, and IDH3 and Cyclin B are also colored with colors indicating high degrees of correlation abnormality, allowing the user to identify them as target biological substances. Similarly, in G2-phase or M1-phase cells treated with cytarabine, ERK is colored with a color indicating a high degree of correlation abnormality, allowing the user to identify them as target biological substances. This display allows the user to recognize target biological substances and estimate which cell cycle cells have a high cellular response to which drug, or which cell cycle cells have a high efficacy for which drug. Furthermore, this display allows the user to determine which target biological substances are likely to be responsible for the high cellular response or efficacy. The user can then refer to the covariation network of the relevant drug and cell cycle and focus on the node of the target biological substance to further identify target biological substances that are affecting the target biological substance.

[0150] FIG. 20 is a schematic diagram illustrating another example of target biological substance detection in Modification 3 of this embodiment. The vertical and horizontal axes and coloring are the same as those in FIG. 19 . However, FIG. 19 is a diagram where ρ = 0.6 is set, and the arrangement of drugs and cell cycles on the vertical axis and target biological substances on the horizontal axis is different. In this figure, the horizontal axis is sorted by drug (reagent) order (classified by drug). That is, when weak correlations are included ( FIG. 20 : ρ = 0.6), there is a similarity in cellular response and efficacy for each drug. However, when narrowing down to strong correlations ( FIG. 19 : ρ = 0.75), similarities in cellular response and efficacy for each cell cycle become apparent when using bleomycin and cytarabine. In this way, analysis system 1 allows the user to set the coefficient ρ of the normalization term (penalty term) and sorts the data in order of the highest similarity between drugs and cell cycles. This allows the user to grasp the similarity in cellular response and efficacy for each drug and cell cycle while adjusting the strength of correlation.

[0151] For the target biological substance, γH2AX is colored with a color indicating a high correlation abnormality in cells throughout the cell cycle treated with bleomycin and in S-phase cells treated with cytarabine, allowing the user to identify the target biological substance. Furthermore, since γH2AX is colored with a color indicating a high correlation abnormality in cells treated with bleomycin compared to S-phase cells treated with cytarabine, the user can predict that bleomycin will have a higher cellular response and efficacy to γH2AX than cytarabine. In this way, the analysis system 1 displays information (color) indicating the level of correlation abnormality by arranging each drug and cell cycle (covariation network). This allows the user to compare the level of cellular response and efficacy between drugs and cell cycles. Furthermore, the analysis system 1 displays information (color) indicating the level of correlation abnormality by arranging each target biological substance (node) in addition to the drug and cell cycle. This allows the user to compare the level of cellular response and efficacy between drugs and cell cycles for each target biological substance or target biological substance.

[0152] In S-phase cells treated with 3 μM cytarabine, ERK is colored with a color indicating a high correlation abnormality throughout the entire cell cycle, allowing the user to identify it as a biological substance of interest. Furthermore, in S-phase cells treated with 3 μM cytarabine, ERK is colored with a color indicating a high correlation abnormality compared to S-phase cells treated with 1 μM cytarabine. This allows the user to understand that increasing the cytarabine concentration can increase the correlation abnormality of ERK, for example, potentially enhancing the cellular response of ERK. In this way, the analysis system 1 displays information (color) indicating the level of correlation abnormality by arranging the drug, cell cycle, and drug concentration. This allows the user to compare the cellular response and efficacy for each drug concentration.

[0153] In Modification 3, when the value of ρ is set large and those with strong correlation coefficients are retained ( FIG. 19 : ρ = 0.75), and when the value of ρ is set small and those with weak correlation coefficients are also retained ( FIG. 20 : ρ = 0.6), the target biological substance is different, and the difference in the order of the horizontal axis indicates that the similarities between drugs and cell cycles are also different. Thus, in analysis system 1 according to this modification, the user can set the coefficient ρ of the normalization term (penalty term), making it possible to identify target biological substances according to correlation.

[0154] As described above, in the analysis system 1 according to the third modification, the correlation extraction unit 104 (an example of a correlation calculation unit) calculates, for each component, a correlation anomaly that represents a change between a correlation calculated using the feature amount of a first component of a first single cell and the feature amount of the second component, and a correlation calculated using the feature amount of the first component of a second single cell and the feature amount of the second component. The result output unit 300 outputs information that enables identification of the correlation anomaly or the component according to the correlation anomaly, and the display unit 30 displays the information. This allows the user to identify, for example, components with a high correlation anomaly and recognize components with a high cellular response or effect.

[0155] The result output unit 300 outputs information that allows identification of correlation abnormalities for multiple single cells or cell cycles, and the display unit 30 displays the information. This allows the user to recognize differences in correlation abnormalities for each single cell or cell cycle, and for example, to recognize components that have a high response or effect when a first single cell changes to a second single cell for each single cell or cell cycle. When a drug is used on a single cell, the result output unit 300 outputs information that allows identification of correlation abnormalities for multiple drugs or drug concentrations or amounts, and the display unit 30 displays the information. This allows the user to recognize differences in correlation abnormalities for each drug or drug concentration or amount, and for example, to recognize components that have a high drug response or pharmacological effect for each drug or drug concentration or amount.

[0156] The result output unit 300 sorts the multiple single cells, cell cycles, drugs used on the single cells, or drug concentrations or amounts according to the degree of correlation anomaly among the multiple components. This makes it easier for the user to recognize, for example, single cells, cell cycles, drugs, or drug concentrations or amounts that have similar or different correlations. The user can recognize, for example, components with high responsiveness or effectiveness for each single cell, cell cycle, drug, or drug concentration or amount. The result output unit 300 may sort, for example, the covariation network of FIG. 18 according to the degree of correlation anomaly among the multiple components.

[0157] The result output unit 300 may highlight components according to the degree of correlation anomaly. The result output unit 300 may highlight components with a high degree of correlation anomaly, for example, may highlight nodes representing components with a high degree of correlation anomaly in the covariation network of Fig. 18. This allows the user to identify components of interest (target biological substances), and by checking the correlation between the component and other components, for example, it becomes easier to recognize components that show a high correlation that contributes to a reaction or effect.

[0158] The result output unit 300 may also sort the results by single cells, cell cycles, drugs used on single cells, and drug concentrations or amounts, or by a combination thereof, and display the correlation anomaly level for each component. Sorting includes sorting by group or by order. For example, in the maps of Figures 19 and 20, the result output unit 300 may sort the results by cell cycle order (e.g., G1 phase, S phase, G2 phase, M phase) for each drug. This allows the user to easily recognize cell cycles with high cellular responses or efficacy for each drug, and target products (substances of interest) that contribute to each drug. The result output unit 300 may also sort the results by cell cycle order. This allows the user to easily recognize drugs with high cellular responses or efficacy for each cell cycle, and target products (substances of interest) that contribute to each drug.

[0159] Note that a portion of the analysis device 10 in the above-described embodiment may be implemented by a computer. In this case, the control function may be realized by recording a program for implementing the control function on a computer-readable recording medium and loading and executing the program recorded on the recording medium into a computer system. Note that the term "computer system" as used herein refers to a computer system built into the analysis device 10, including hardware such as an OS and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into a computer system. Furthermore, the term "computer-readable recording medium" may also include media that dynamically store programs for a short period of time, such as communication lines used when transmitting programs via networks such as the Internet or telephone lines, or media that store programs for a fixed period of time, such as volatile memory within a computer system that serves as a server or client in such cases. Furthermore, the program may be a program that implements a portion of the above-described functions, or may be a program that can implement the above-described functions in combination with a program already stored in the computer system. Furthermore, part or all of the analysis device 10 in the above-described embodiment may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each functional block of the analysis device 10 may be individually implemented as a processor, or part or all of the functional blocks may be integrated into a processor. Furthermore, the integrated circuit implementation method is not limited to LSI, and may be implemented using a dedicated circuit or a general-purpose processor. Furthermore, if an integrated circuit implementation technology that can replace LSI emerges due to advances in semiconductor technology, an integrated circuit based on that technology may be used.

[0160] One embodiment of the present invention has been described in detail above with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes and the like are possible within the scope that does not deviate from the gist of the present invention.

[0161] REFERENCE SIGNS LIST 1 analysis system, 10 analysis device, 20 microscope device, 30 display unit, 100 calculation unit, 200 storage unit, 300 result output unit, 101 cell image acquisition unit, 102 feature amount calculation unit, 103 noise component removal unit, 104 correlation extraction unit, 201 type storage unit, 202 experimental condition storage unit, 203 cell information storage unit

Claims

1. an acquisition unit that acquires a first image in which a first component is labeled on a cell, and a second image in which a second component different from the first component is labeled on the cell after a step of removing the label of the first component and a step of labeling a second component are performed on the cell; a feature extraction unit that extracts features of each component constituting a cell from each of the first image and the second image of the same cell; a correlation calculation unit that calculates a correlation between the first component and the second component by using the feature amount; An analysis system comprising:

2. The acquisition unit acquires time information representing a plurality of the specific points in time, The same specific time point includes a time point within the time when the step of removing the label of the first component and the step of labeling the second component are performed, the feature extraction unit identifies at least a portion of single cells from each of the first image and the second image for each of the specific time points and for each of the same cells, and extracts a feature amount of the first component and a feature amount of the second component for each of the specific time points and for each of the single cells; The correlation calculation unit calculates the correlation for each specific time point. The analysis system according to claim 1 .

3. The acquisition unit assigns a time point identification that identifies the same specific time point and a cell identification that identifies the cell to the first image and the second image, and assigns different label identifications to the first image and the second image, The feature extraction unit extracts the feature for each of the time point identification, the cell identification, and the marker identification, The correlation calculation unit calculates the correlation for each time point identification. The analysis system according to claim 2 .

4. the correlation calculation unit calculates the correlation at the specific time point using the feature amount of the first component and the feature amount of the second component for each of the single cells, and calculates an amount of change in correlation between different specific time points; An output unit that outputs the correlation between the first component and the second component based on the amount of change in the correlation. The analysis system according to claim 2 .

5. an output unit that outputs the correlation between the first component and the second component, the correlation being calculated using the feature amount; Further equipped with the correlation calculation unit calculates, for each component, a correlation anomaly degree representing a degree of change between a correlation calculated using the feature amount of the first component of the first single cell and the feature amount of the second component, and a correlation calculated using the feature amount of the first component of the second single cell and the feature amount of the second component; The output unit outputs information based on the correlation anomaly degree. The analysis system according to claim 1 .

6. The output unit outputs information capable of identifying a component according to the correlation anomaly degree. The analysis system according to claim 5 .

7. The output unit outputs information capable of identifying the correlation abnormality degree for each of a plurality of single cells or for each cell cycle. The analysis system according to claim 5 .

8. The correlation calculation unit performs classification based on the correlation anomaly degree, and the output unit rearranges information about the drug used on the single cell according to the classification, and outputs information that allows a correlation anomaly degree to be identified for each piece of information about the drug used on the single cell. The analysis system according to claim 5 .

9. The cells are collected from an affected area for each pathological condition, The acquisition unit acquires the first image and the second image for each of a cell to be examined in a first pathological state and a cell to be examined in a second pathological state different from the first pathological state, the feature extraction unit extracts, for each of the first pathological condition and the second pathological condition, a feature for each component constituting a cell from the first image and the second image; The correlation calculation unit calculates a correlation between the first component and the second component for each of the first pathological condition and the second pathological condition. The analysis system according to claim 1 .

10. The acquisition unit acquires analysis data obtained by analyzing a gene sequence related to the cell for each specific time period, The method further includes an output unit that outputs the gene sequence analysis data and the correlation between the components for each specific time point in association with each other. The analysis system according to claim 2 .

11. the feature amounts of the first and second components include amounts of labeled components; The marker of the first component and the marker of the second component are normalized in terms of brightness per one of the first component and the second component for each specific time point, and the feature amount extraction unit calculates the amount of the component based on brightness information representing the brightness of the first image and the second image for each specific time point. The analysis system according to claim 2 .

12. the cells include a plurality of single cells; The correlation includes information based on the location of the single cell. The analysis system according to claim 1 .

13. a correlation calculation unit that calculates a correlation between the first component and the second component of the same cell using feature amounts for each component constituting a cell, which are extracted from a first image in which a first component is labeled for a cell, and a second image in which a second component different from the first component is labeled for the cell after a step of removing the label of the first component and a step of labeling a second component are performed for the cell; An analysis device comprising:

14. The computer of the analysis device a step of calculating a correlation between the first component and the second component of the same cell using feature amounts for each component constituting a cell, the feature amounts being extracted from a first image in which a first component is labeled for a cell, and a second image in which a second component different from the first component is labeled for the cell after a step of removing the label of the first component and a step of labeling a second component are performed for the cell; An analysis program for executing the above.

15. An analysis method in an analysis system, comprising: an acquisition process in which an acquisition unit acquires a first image in which a first component is labeled for a cell, and a second image in which a second component different from the first component is labeled for the cell after a step of removing the label of the first component and a step of labeling a second component are performed; a feature extraction step in which a feature extraction unit extracts a feature for each component constituting a cell from each of the first image and the second image of the same cell; and a correlation calculation step in which a correlation calculation unit calculates a correlation between the first component and the second component using the feature; The analysis method has the following features: