Observation device, observation method, and program

Abstract

Provided is an observation device for analyzing a cell image, the observation device comprising: an image acquisition unit that, at a pixel resolution lower than the pixel resolution of a first image used for analysis, acquires a second image in a region wider than a region used for the analysis; an index calculation unit that calculates a value of an index pertaining to a cell for each partial region included in the second image and corresponding to the region used for analysis; and a region selection unit that selects a region to be used for analysis from a plurality of the partial regions on the basis of the values of the indexes calculated by the index calculation unit.

Description

Observation device, observation method, and program

[0001] This application claims priority to Japanese Patent Application No. 2023-132533, filed on August 16, 2023, the contents of which are incorporated herein by reference.

[0002] In biological science, medicine, and the like, it is known that the health, disease, and other states of living organisms are correlated with, for example, the states of cells and intracellular organelles. Therefore, analyzing these correlations is one way of solving various problems in biological science, medicine, and the like. Furthermore, analyzing the transmission pathways of information transmitted between or within cells can be useful, for example, in research into industrial biosensors and pharmaceuticals aimed at disease prevention. Various analytical techniques for cells, tissue fragments, and the like are known, such as techniques using image processing (see, for example, Patent Document 1).

[0003] U.S. Patent No. 9,280,698

[0004] One aspect of the present invention is an observation device for analyzing cell images, comprising: an image acquisition unit that acquires a second image of an area larger than the area used for the analysis at a pixel resolution lower than the pixel resolution of a first image used for the analysis; an index calculation unit that calculates a value of a cell-related index for each partial area included in the second image that corresponds to the area used for the analysis; and an area selection unit that selects an area to be used for the analysis from a plurality of the partial areas based on the index values ​​calculated by the index calculation unit.

[0005] One aspect of the present invention is an observation method for analyzing a cell image, comprising: acquiring a second image of an area larger than the area used for the analysis at a pixel resolution lower than the pixel resolution of a first image used for the analysis; calculating a value of a cell-related index for each partial area included in the second image that corresponds to the area used for the analysis; and selecting an area to be used for the analysis from the plurality of partial areas based on the index value calculated by the calculation.

[0006] One aspect of the present invention is a program for causing a computer that analyzes cell images to acquire a second image of an area larger than the area used for the analysis at a pixel resolution lower than the pixel resolution of a first image used for the analysis, calculate a value of a cell-related index for each partial area included in the second image that corresponds to the area used for the analysis, and select an area to be used for the analysis from a plurality of the partial areas based on the index value calculated by the calculation.

[0007] 1 is a diagram showing an example of the configuration of an observation device according to a first embodiment. FIG. 2 is a block diagram showing an example of the functional configuration of an observation device according to the first embodiment. FIG. 3 is a diagram showing an example of the configuration of a microscope device according to the first embodiment. FIG. 4 is a diagram showing an example of the type of indices according to the first embodiment. FIG. 5 is a diagram showing an example of a combination of a first indices and a second indices according to the first embodiment. FIG. 6 is a diagram showing an example of the flow of a first preparation process according to the first embodiment. FIG. 7 is a diagram showing an example of the flow of an observation area selection process according to the first embodiment. FIG. 8 is a diagram showing an example of a preliminary image according to the first embodiment. FIG. 9 is a diagram showing an example of the flow of a second preparation process according to the first embodiment. FIG. 10 is a diagram showing an example of the flow of a main imaging and analysis process according to the first embodiment. FIG. 11 is a block diagram showing an example of the functional configuration of an observation device according to a second embodiment. FIG. 12 is a diagram showing an example of the flow of an observation area selection process according to the second embodiment. FIG. 13 is a block diagram showing an example of the functional configuration of an observation device according to a third embodiment. FIG. 14 is a diagram showing an example of the flow of a first preparation process according to the third embodiment. FIG. 15 is a diagram showing an example of a preliminary image in which a foreign substance is imaged according to the third embodiment. FIG. 16 is a diagram showing an example of the flow of an observation area selection process according to the third embodiment.

[0008] The present invention will be described below through embodiments of the invention. The following embodiments do not limit the scope of the invention. Not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0009] (First embodiment) The first embodiment will be described in detail below with reference to the drawings. In the following description, an XYZ Cartesian coordinate system is set, and the positional relationship of each part will be described with reference to this XYZ Cartesian coordinate system. The plane perpendicular to the optical axis of the objective lens is defined as the XY plane. The direction parallel to the optical axis of the objective lens is defined as the Z-axis direction. FIG. 1 is a diagram showing an example of the configuration of an observation device 1 according to this embodiment. The observation device 1 is also referred to as an observation system.

[0010] The observation device 1 performs image processing on captured images of cells, etc. The observation device 1 includes a microscope device 20, an analysis device 10, a display unit 30, and an operation unit 40.

[0011] The microscope device 20 is a microscope for observing a magnified image of a sample placed on an electric stage 21. The sample is specifically a biological sample, bead, or the like to be observed. A biological sample is, for example, a thick fluorescently stained cell. In the following description, an image obtained by capturing an image of a cell or the like is also simply referred to as a cell image.

[0012] The well plate WP is held in a dedicated holder and placed on the motorized stage 21. The well plate WP has one or more wells W. In this embodiment, the well plate WP has 96 wells W (8 x 12), as shown in FIG. 1 . The number of wells W in the well plate WP is not limited to this, and may be 6 wells W (2 x 3), 12 wells W (3 x 4), 24 wells W (4 x 6), 384 wells W (16 x 24), 1536 wells W (32 x 48), or 6144 wells W (64 x 96). Cells are cultured in the wells W under specific experimental conditions. Specific experimental conditions include temperature, humidity, culture period, time elapsed since application of a stimulus, the type, strength, concentration, and amount of the stimulus applied, the presence or absence of the stimulus, and the induction of biological characteristics. Examples of stimuli include physical stimuli such as electricity, sound waves, magnetism, and light, as well as chemical stimuli such as the administration of a substance or drug. Biological characteristics are characteristics that indicate the stage of cell differentiation, morphology, cell number, behavior of molecules within the cell, morphology and behavior of organelles, each shape, behavior of intranuclear structures, behavior of DNA molecules, behavior and modification of RNA molecules, behavior and modification of protein molecules, etc. In the following description, these biological characteristics are also referred to as components that make up a cell. In this example, the microscope device 20 observes cells seeded in each of multiple wells W of a well plate WP. Note that the well plate WP is not limited to a well plate WP, and dishes (e.g., 35 mm dishes) may also be used. In this case, multiple dishes are held in a dedicated holder and placed on the motorized stage 21.

[0013] The analysis device 10 analyzes an image captured by the microscope device 20. The analysis device 10 is a computer device. In this example, the analysis device 10 determines an observation area suitable for analysis based on a preliminary image captured by the microscope device 20. The preliminary image is an image used to determine an observation area suitable for analysis. The preliminary image is, for example, an image acquired at a magnification lower than the magnification at which the main image is acquired. The main image is an image used for image analysis.

[0014] The preliminary image, although it depends on the type of well plate WP, is captured at a magnification lower than that of the main image, and captures a roughly full view of each well W. Also, depending on the type of well plate WP, the preliminary image does not capture a roughly full view of each well W, but only the vicinity of the center of each well W. The analysis device 10 analyzes the main image captured in the observation area.

[0015] The display unit 30 displays images analyzed by the analysis device 10. The images displayed on the display unit 30 include images generated based on the analysis results of the analysis performed by the analysis device 10. The display unit 30 includes, for example, a liquid crystal display.

[0016] The operation unit 40 is operated by a user. When operated by the user, the operation unit 40 outputs an operation signal. This operation signal is supplied to the analysis device 10. The analysis device 10 acquires various information supplied by the user based on the operation signal supplied from the operation unit 40. The operation unit 40 includes, for example, a keyboard 40a and a mouse 40b.

[0017] Next, an example of the functional configuration of the observation device 1 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the functional configuration of the observation device 1. The microscope device 20 is a biological microscope, and includes, in addition to the motorized stage 21 described above, a motorized stage driving unit 25, an imaging unit 22, an objective lens 23, and an objective lens driving unit 24.

[0018] The motorized stage 21 can arbitrarily move the position of the object in a predetermined direction (for example, a certain direction in a horizontal two-dimensional plane). The motorized stage 21 is driven by a motorized stage driver 25. The motorized stage driver 25 has a motor that moves the position of the motorized stage 21 in the X-axis direction or the Y-axis direction, which are directions perpendicular to the optical axis of the objective lens 23. By driving the motor, the motorized stage driver 25 can move the motorized stage 21 in the X-axis direction or the Y-axis direction.

[0019] The imaging unit 22 includes an imaging element such as a CCD (Charge-Coupled Device) or a CMOS (Complementary MOS), and captures an image of an object on the motorized stage 21. An image of the cell placed on the motorized stage 21 is formed on the imaging surface of the imaging unit 22 via an objective lens 23 and an imaging lens (not shown).

[0020] The objective lens 23 receives light that has passed through the cells. Note that the light received by the objective lens 23 is not limited to light that has passed through the cells. The objective lens 23 may also receive light reflected by the cells. Note that the objective lens 23 may be, for example, an immersion objective lens such as an oil immersion objective lens or a water immersion objective lens, or a dry objective lens that is not compatible with immersion. The objective lens driver 24 has a motor that moves the position of the objective lens 23 in the Z-axis direction, which is the optical axis direction. The objective lens 23 can be moved in the Z-axis direction by moving a revolver (not shown) that holds the objective lens 23 up and down.

[0021] More specifically, the microscope device 20 has functions such as a differential interference contrast microscope (DIC), a phase contrast microscope, a fluorescence microscope, a confocal microscope, a super-resolution microscope, a two-photon excitation fluorescence microscope, a light sheet microscope, a light field microscope, a quantitative phase contrast microscope, or a holographic microscope.

[0022] The microscope device 20 images cells in a container placed on a motorized stage 21. This container is, for example, a well plate WP, a dish, or a slide chamber. The microscope device 20 captures images of the cells by irradiating light onto the cells in multiple wells W of the well plate WP and detecting the transmitted light that has passed through the cells. This allows the microscope device 20 to acquire images such as transmission DIC images, phase contrast images, dark-field images, and bright-field images of the cells. Furthermore, the microscope device 20 captures images of the cells by irradiating excitation light onto cells to which a fluorescent substance has been attached and detecting the fluorescence emitted from the cells.

[0023] Here, a detailed configuration of the microscope apparatus 20 will be described with reference to FIG. 3 . FIG. 3 is a diagram showing an example of the configuration of the microscope apparatus 20 according to this embodiment. In addition to the above-described motorized stage 21, imaging unit 22, objective lens 23, objective lens driver 24 (not shown in FIG. 2 ), and motorized stage driver 25 (not shown in FIG. 2 ), the microscope apparatus 20 includes a fluorescence excitation light source 26 and a transmitted illumination system 27. The objective lens 23 includes a high-magnification objective lens 231 and a low-magnification objective lens 232. The high-magnification objective lens 231 is, for example, a 10x objective lens or a 20x objective lens. The low-magnification objective lens 232 is, for example, a 2x objective lens or a 4x objective lens. Note that the combination of the high-magnification objective lens 231 and the low-magnification objective lens 232 is not limited to these combinations and is determined based on the relative magnification of one objective lens with respect to the other.

[0024] Here, the high-magnification objective lens 231, the fluorescence excitation light source 26, the transmitted illumination system 27, the dichroic mirror 28, and the image capturing unit 22 are included in the first optical system. The first optical system is an optical system for acquiring a main image at high magnification. The main image is acquired by the high-magnification objective lens 231. The low-magnification objective lens 232, the fluorescence excitation light source 26, the transmitted illumination system 27, the dichroic mirror 28, and the image capturing unit 22 are included in the second optical system. The second optical system is an optical system for acquiring a pre-image at low magnification. The low magnification is a magnification lower than the magnification used to acquire the main image. In other words, the main image is acquired at high magnification and therefore corresponds to a micro image or a high-magnification image, while the pre-image is acquired at low magnification and therefore corresponds to a macro image or a low-magnification image.

[0025] In this embodiment, as an example, the first optical system and the second optical system are provided as one detection optical system equipped with a plurality of objective lenses in the microscope device 20. That is, the microscope device 20 includes a detection optical system (first optical system and second optical system) equipped with a plurality of objective lenses, namely, a high-magnification objective lens 231 and a low-magnification objective lens 232.

[0026] The first optical system will now be described. The first optical system acquires the main image by epi-fluorescence observation or transmitted light brightfield observation. First, epi-fluorescence observation will be described. The fluorescence excitation light source 26 irradiates the cell C1 in the well W of the well plate WP with excitation light L1. The excitation light L1 irradiated from the fluorescence excitation light source 26 is incident on the dichroic mirror 28. The excitation light L1 incident on the dichroic mirror 28 is reflected by the dichroic mirror 28 in the direction of the well plate WP. The excitation light L1 reflected by the dichroic mirror 28 passes through the high-magnification objective lens 231 and is irradiated on the cell C1.

[0027] The cell irradiated with the excitation light L1 emits fluorescence L2. The fluorescence L2 passes through the high-magnification objective lens 231 and enters the dichroic mirror 28. The fluorescence L2 that enters the dichroic mirror 28 is transmitted by the dichroic mirror 28 toward the image capturing unit 22. The fluorescence L2 that has passed through the dichroic mirror 28 is formed on the image capturing surface of the image capturing unit 22 by an image capturing lens (not shown). The image capturing unit 22 captures the image of the formed cell. The image capturing unit 22 outputs the image of the cell C1 to the analysis device 10.

[0028] Next, transmitted light bright-field observation will be described. The transmitted light illumination system 27 irradiates the cell C1 with illumination light L3. The wavelength of the illumination light L3 is, for example, 720 nm, but is not limited to this. The illumination light L3 irradiated onto the cell C1 passes through the high-magnification objective lens 231 and enters the dichroic mirror 28. The illumination light L3 that enters the dichroic mirror 28 is transmitted by the dichroic mirror 28 toward the image capturing unit 22. The illumination light L3 that passes through the dichroic mirror 28 is focused on the image capturing surface of the image capturing unit 22 by an imaging lens (not shown). The image capturing unit 22 captures the focused image of the cell C1. The image capturing unit 22 outputs the captured image of the cell to the analysis device 10. When transmitting transmitted light bright-field observation, the dichroic mirror 28 may be removed from the light path.

[0029] Next, the second optical system will be described. The second optical system acquires a preliminary image by transmitted light bright-field observation or epi-fluorescence observation. The method by which the second optical system acquires a preliminary image is the same as the method by which the first optical system acquires a main image, except that a low-magnification objective lens 232 is used instead of the high-magnification objective lens 231, and therefore a description thereof will be omitted.

[0030] In this embodiment, an example will be described in which the detection optical system provided in the microscope device 20 includes multiple objective lenses, but this is not limiting. The first optical system and the second optical system may be provided as separate optical systems in the observation device 1. In other words, the microscope device 20 may include a first optical system for acquiring a main image and a second optical system for acquiring a preliminary image.

[0031] In this embodiment, a fluorescent fusion protein is expressed, or live cells are stained with a chemical reagent or the like to acquire cell images. In yet another embodiment, cells are fixed and stained, and cell images are acquired. Fixed cells cease to metabolize. Therefore, when observing intracellular changes over time using fixed cells after applying a stimulus to the cells, multiple containers seeded with cells must be prepared. For example, there may be a case where a stimulus is applied to cells, and it is desired to observe changes in the cells after a first hour and after a second hour different from the first hour. In this case, after applying a stimulus to the cells and the first hour has elapsed, the cells are fixed and stained, and cell images are acquired.

[0032] Meanwhile, a different container is prepared from the cells used for observation at the first time, and after a second time has elapsed after applying a stimulus to the cells, the cells are fixed and stained, and cell images are acquired. This allows for estimating intracellular changes over time by observing the changes in the cells at the first time and the second time. Furthermore, the number of cells used to observe intracellular changes between the first and second times is not limited to one. Therefore, multiple cell images are acquired at each of the first and second times. For example, if the number of cells to be observed for intracellular changes is 1,000, 2,000 cells are imaged at the first and second times. Therefore, to obtain detailed intracellular changes in response to a stimulus, multiple cell images are required for each imaging timing from the stimulus, resulting in the acquisition of a large number of cell images.

[0033] Furthermore, the microscope device 20 may capture, as the above-described cell image, luminescence or fluorescence from a chromogenic substance itself incorporated into the biological material, or luminescence or fluorescence generated when a substance having a chromophore binds to the biological material. Note that the method for acquiring cell images is not limited to an optical microscope. For example, the method for acquiring cell images may also be an electron microscope. In other words, the type of cell image may be selected as appropriate.

[0034] In this embodiment, the cells are, for example, primary cultured cells, established cultured cells, tissue slice cells, etc. To observe cells, the sample to be observed may be a cell aggregate (cell mass, spheroid, or organoid), a tissue sample, an organ, or an individual (such as an animal), and images containing the cells may be obtained. The state of the cells is not particularly limited, and they may be in a living state or a fixed state. Of course, information on the living state and information on the fixed state may be combined.

[0035] Cells may also be treated with a chemiluminescent or fluorescent protein (e.g., a chemiluminescent or fluorescent protein expressed from an introduced gene, such as green fluorescent protein (GFP)) and observed. Alternatively, cells may be observed using immunostaining or staining with a chemical reagent. Observation using a combination of these methods is also possible. For example, it is possible to select the luminescent protein to be used depending on the type of intracellular structure to be identified (e.g., organelles such as the Golgi apparatus).

[0036] Furthermore, the means for observing these cells, the pretreatment for analysis such as the method for staining the cells, etc. may be selected appropriately depending on the purpose. For example, when obtaining dynamic cell behavior, dynamic information on the cells may be obtained using an optimal method, and when obtaining intracellular signaling, information on intracellular signaling may be obtained using an optimal method. The pretreatment selected may differ depending on the purpose.

[0037] Returning to FIG. 2 , we will continue to explain the functional configuration of the observation device 1. The analysis device 10 is an analysis device that analyzes cell images. For example, it performs image analysis of the cell's response to a stimulus. In this embodiment, the analysis device 10 is a computer device that determines an observation area in which to capture an actual image to be used for analysis based on a preliminary image, captures an image of cells as the actual image in the determined observation area, and analyzes the actual image. The analysis device 10 includes a calculation unit 100, a memory unit 200, a result output unit 300, an operation detection unit 400, and a control signal output unit 500.

[0038] The calculation unit 100 functions when the processor executes a program stored in the storage unit 200. Some or all of these functional units of the calculation unit 100 may be configured with hardware such as an LSI (Large Scale Integration) or an ASIC (Application Specific Integrated Circuit). The calculation unit 100 includes, as its functional units, a cell image acquisition unit 101, an image determination unit 102, an observation area selection unit 103, an analysis unit 107, an autofocus unit 108, an abnormality detection unit 110, an illumination condition determination unit 111, and an analysis type input unit 115.

[0039] The cell image acquisition unit 101 acquires cell images from another device that captures captured images. In this example, the cell image acquisition unit 101 acquires cell images captured by the imaging unit 22 included in the microscope device 20. The cell image acquisition unit 101 supplies the acquired cell images to the index calculation region change unit 105, the index calculation unit 104, the region selection unit 106, and the analysis unit 107.

[0040] Here, the cell images acquired by the cell image acquisition unit 101 from the microscope device 20 include a main image, a preliminary image, a pre-ROI image (described later), and an overhead image. The images have pixels corresponding to the number of effective pixels of the imaging unit 22. The pixel resolution of the preliminary image is lower than that of the main image. Furthermore, the area captured as the preliminary image is wider than the area captured as the main image.

[0041] Note that pixel resolution is, for example, the size of the range in which an object is imaged per pixel (corresponding to one pixel of an image sensor) that constitutes an image. Pixel resolution can be expressed by the formula "(pixel resolution) = (size of the range in which an object is imaged) / (number of effective pixels of an image sensor)." Pixel resolution is also referred to as resolution or resolution. Furthermore, the images acquired by the cell image acquisition unit 101 include multiple images of the culture state of cells captured in chronological order, and multiple images of cells cultured under various experimental conditions.

[0042] The image determination unit 102 performs various determinations on the image based on image recognition. In this embodiment, the image determination unit 102 determines cells based on the nuclei of cells in the image or the outlines of cells in the image. The image determination unit 102 also determines the type of well plate WP based on the shape, size, number, etc. of wells in the image. The images determined by the image determination unit 102 include overhead images, pre-images, pre-ROI images, and main images. The image recognition used by the image determination unit 102 for determination is, for example, image recognition based on artificial intelligence (AI). Note that pattern matching (template matching) or the like may be used for image recognition.

[0043] The observation region selection unit 103 determines an observation region for each well W. There is one observation region for each well W. This observation region is a so-called region of interest (ROI). The observation region selection unit 103 includes an index calculation unit 104, an index calculation region modification unit 105, and a region selection unit 106.

[0044] The index calculation unit 104 calculates the value of a cell-related index for each partial region provisionally set in the preliminary image by the index calculation region change unit 105. Here, the partial region corresponds to the region used for analysis. In other words, the partial region included in the preliminary image captured at low magnification corresponds to the actual image captured at high magnification. The index whose value is calculated by the index calculation unit 104 is an index based on a combination of a first index and a second index. In other words, the index whose value is calculated by the index calculation unit 104 includes the first index and the second index. The type of index calculated by the index calculation unit 104 is stored in advance in the storage unit 200 as index information A1. Specific examples of the type of index will be described later. The index calculation unit 104 supplies the calculated index value to the region selection unit 106.

[0045] The index calculation area modification unit 105 modifies a partial area provisionally set in the preliminary image so that the index calculation unit 104 can calculate an index value. The index calculation area modification unit 105 sequentially modifies the partial area in the preliminary image each time the index calculation unit 104 calculates an index value. As an example, the index calculation area modification unit 105 modifies the position of the partial area in the preliminary image without changing the size of the partial area. Sequentially modifying the partial area in the preliminary image is also referred to as scanning the preliminary image. The index calculation area modification unit 105 supplies information about the modified partial area to the index calculation unit 104.

[0046] The region selection unit 106 determines a region to be used for analysis from a plurality of partial regions based on the value of the index calculated by the index calculation unit 104. That is, the region selection unit 106 determines an observation region to be imaged at high magnification as a main image for each well W based on the value of the index calculated by the index calculation unit 104. As an example, the region selection unit 106 selects an observation region based on a combination of the value of a first index and the value of a second index. Specific examples of combinations of the first index and the second index will be described later.

[0047] The analysis unit 107 acquires the main image from the cell image acquisition unit 101. The analysis unit 107 analyzes the cells in the main image acquired from the cell image acquisition unit 101. The analysis is, for example, a cell assay such as measuring the increase or decrease in the amount of protein expression in the cells, evaluating cytotoxicity, evaluating changes in cell morphology, or measuring the proportion of apoptotic cells.

[0048] The analysis unit 107 supplies the analysis results to the result output unit 300. The result output unit 300 acquires the analysis results from the analysis unit 107. The result output unit 300 displays the analysis results acquired from the analysis unit 107 on the display unit 30.

[0049] The operation detection unit 400 detects an operation signal output from the operation unit 40. The operation detection unit 400 supplies the detected operation signal to the calculation unit 100.

[0050] The analysis type input unit 115 inputs the type of analysis. Here, the user inputs the type of analysis by operating the operation unit 40. The operation unit 40 supplies an operation signal corresponding to the input type of analysis to the analysis device 10. The analysis type input unit 115 acquires analysis type information based on the operation signal supplied from the operation unit 40. The analysis type information is information indicating the type of analysis. Note that the analysis type input unit 115 may acquire the analysis type information from an external device provided separately from the analysis device 10.

[0051] The index calculation unit 104 determines an index for calculating an index value according to the analysis type information acquired by the analysis type input unit 115. The index calculation unit 104 determines the index based on index information A1. The index information A1 is information in which the type of analysis is associated with the type of index. For example, the index information A1 is two-dimensional tabular data consisting of rows and columns in which the type of index is stored for each type of analysis. The index information A1 is stored in advance in the storage unit 200. Note that the index information A1 may be created by the user operating the operation unit 40.

[0052] The storage unit 200 stores various types of information, such as index information A1, and is configured using a storage device such as a magnetic hard disk drive or a semiconductor storage device.

[0053] Here, the types of the first index and the second index for which the index calculation unit 104 calculates values ​​will be described. FIG. 4 is a diagram showing an example of the types of indexes according to this embodiment. The first index and the second index are selected from the following A) to H). A) cell coverage area ratio, B) cell density distribution, C) cell count, D) cell fluorescence intensity, E) cell confluency, F) cell circularity, G) area per cell, H) cell multinucleation rate.

[0054] There are two methods for determining the observation area of ​​each well W using an index: a) selecting a partial area that is estimated to be approximately equal to the entire cell population in the pre-image (well W) (representative value); b) selecting a partial area from the entire cell population in the pre-image (well W) where the index indicates either the maximum value, minimum value, average value, or a predetermined percentage value (for example, 75% value) (the value you want to identify from the range from the maximum value to the minimum value). In the case of a), it is necessary to calculate the absolute value of the difference between the "index value of each partial area" and the "index value of the entire pre-image." In the case of b), the "index value of each partial area" itself is used.

[0055] The cell coverage area ratio is the "ratio of the area of ​​cells contained in (occupying) a predetermined region to the area of ​​the entire predetermined region." To calculate the cell coverage area ratio, the area is calculated, for example, based on the number of pixels in the image.

[0056] The cell density distribution is a standard deviation based on the distribution of the number of cells contained in a predetermined region. That is, the cell density distribution is calculated by dividing the predetermined region into smaller regions and based on the distribution of the number of cells contained in each of the smaller regions. Note that the cell density distribution may be calculated based on the k-nearest neighbor method or the like instead of the standard deviation.

[0057] The cell count is the number of cells contained in a predetermined area.

[0058] The fluorescence intensity of a cell is the average fluorescence intensity of each cell contained in a predetermined region, i.e., the fluorescence intensity of a cell is the sum of the fluorescence intensity of each cell contained in the predetermined region divided by the number of cells contained in the predetermined region.

[0059] The cell density is the number of cells within a predetermined region, among the distances between the cells and their nearest neighbors, where the distance is equal to or less than a predetermined distance. The distance between cells is the distance between the centers of gravity of the cells. The center of gravity of a cell is the center of gravity when the outline of the cell is represented as a figure. The cell density may be the average value of the distances between the cells within a predetermined region and their nearest neighbors.

[0060] The circularity of a cell is, for each cell contained in a predetermined region, the number of cells whose value calculated by, for example, formula (1) is equal to or less than a predetermined value.

[0061]

[0062] That is, the value calculated by formula (1) is the value obtained by dividing the area of ​​the cell by the square of the perimeter of the cell and multiplying the result by 4π. Note that the closer the value calculated by formula (1) is to 1, the more circular the cell is. Generally, dividing cells and dead cells tend to be circular, so the circularity of a cell is an index that can easily detect the number of these cells.

[0063] The area per cell is the average area of ​​each cell contained in a given region, i.e., the area per cell is the total area of ​​the cells contained in the given region divided by the number of cells contained in the given region.

[0064] The multinuclear rate of cells is the average number of nuclei of each cell contained in a given region, i.e., the total number of nuclei of cells contained in a given region divided by the number of cells contained in the given region.

[0065] The definitions of the indices described above are merely examples, and the values ​​of the indices may be evaluated using other definitions. The indices shown in Fig. 4 are merely examples, and indices other than those shown in Fig. 4 may be used as the first or second indices.

[0066] Next, a combination of the first index and the second index will be described. FIG. 5 is a diagram showing an example of a combination of the first index and the second index according to this embodiment. As described below, each of the first index and the second index is evaluated as a statistical quantity of each partial region scanned in the preliminary image, and the region selection unit 106 uses this to determine the observation region from the preliminary image. As described above, the preliminary image captures a roughly complete view of the well W. The statistical quantity may be, for example, a representative value, an average value, a maximum value, a minimum value, or a predetermined percentage value (e.g., a 75% value).

[0067] The first combination is a combination of a representative value of the cell coverage area ratio and a minimum value of the cell density distribution. To identify the representative value of the cell coverage area ratio, the absolute value (referred to as absolute value X11) of the difference between the "cell coverage area ratio of the relevant partial region" and the "cell coverage area ratio of the entire preliminary image" is calculated for each partial region. Furthermore, to identify the minimum value of the cell density distribution, a standard deviation (referred to as standard deviation X12) based on the number distribution of cells contained in the partial region is calculated. The value of the index based on the first combination is calculated, for example, as the product of the absolute value X11 and the standard deviation X12 for each partial region.

[0068] When the first combination is used, the region selection unit 106 selects, as the observation region, a partial region having the smallest index value based on the first combination. That is, a partial region having a cell coverage area ratio estimated to be substantially equal to that of all cells in the preliminary image and having the smallest cell density distribution is selected.

[0069] The index based on the first combination is suitable for selecting a region with a representative number of cells and that is not affected by cell density, and is therefore effective in reducing the variability in analysis that depends on the selection of the observation region.

[0070] The second combination is a combination of a representative value of the circularity of cells and a representative value of the area per cell. To identify the representative value of the circularity of cells, the absolute value of the difference between the "circularity of cells in the partial region" and the "circularity of cells in the entire preliminary image" (referred to as the absolute value X21) is calculated for each partial region. To identify the representative value of the area per cell, the absolute value of the difference between the "area per cell in the partial region" and the "area per cell in the entire preliminary image" (referred to as the absolute value X22) is calculated for each partial region. The value of the index based on the second combination is calculated, for example, as the product of the absolute value X21 and the absolute value X22 for each partial region.

[0071] When the second combination is used, the region selection unit 106 selects, as the observation region, a partial region in which the value of the index based on the second combination is minimum. That is, a partial region in which the circularity of the cells is estimated to be approximately equal to that of all the cells in the preliminary image and the area per cell is estimated to be approximately equal to that of all the cells in the preliminary image is selected.

[0072] The index based on the second combination can extract cells with representative morphologies, making it suitable for analysis using parameters other than morphology (such as fluorescence intensity and number). Furthermore, the index based on the second combination can perform analysis while avoiding cells with abnormal morphology, making it effective in suppressing analysis variations caused by variations in cell morphology. In other words, it reduces the effort required for outlier removal after analysis.

[0073] The third combination is a combination of the maximum fluorescence intensity of the cells and the maximum cell count. To identify the maximum fluorescence intensity of the cells, the fluorescence intensity of the cells is calculated for each partial region. To identify the maximum cell count, the cell count is calculated for each partial region. The index value based on the third combination is calculated, for example, as the product of the fluorescence intensity value of the cell and the cell count value for each partial region.

[0074] When the third combination is used, the region selection unit 106 selects the partial region having the maximum value of the index based on the third combination as the observation region. The index based on the third combination is suitable for image analysis because the brightness value of the cells is high. Furthermore, the index based on the third combination is suitable for obtaining as many cells as possible.

[0075] The fourth combination is a combination of a representative value of cell density and a representative value of cell count. To identify the representative value of cell density, the absolute value of the difference between the "density of the partial region" and the "overall cell density of the preliminary image" (referred to as absolute value X41) is calculated for each partial region. To identify the representative value of cell count, the absolute value of the difference between the "cell count of the partial region" and the "overall cell count of the preliminary image" (referred to as absolute value X42) is calculated for each partial region. The value of the index based on the fourth combination is calculated, for example, as the product of the absolute value X41 and the absolute value X42 for each partial region.

[0076] When the fourth combination is used, the region selection unit 106 selects, as the observation region, a partial region for which the value of the index based on the fourth combination is the smallest. In other words, a partial region in which the cell density and the cell number are estimated to be approximately equal to those of all the cells in the preliminary image is selected.

[0077] The cells cultured in the well W include densely packed cells and isolated cells, and the degree of crowding varies from subregion to subregion. The index based on the fourth combination is suitable for suppressing the variation in analysis due to the variation in the degree of crowding.

[0078] The fifth combination is a combination of the minimum value of the cell multinucleation rate and the representative value of the area per cell. To identify the minimum value of the cell multinucleation rate, the cell multinucleation rate is calculated for each partial region. To identify the representative value of the area per cell, the absolute value (referred to as absolute value X52) of the difference between the "area per cell in the partial region" and the "area per cell in the entire preliminary image" is calculated for each partial region. The value of the index based on the fifth combination is calculated, for example, as the product of the value of the cell multinucleation rate and the absolute value X52 for each partial region.

[0079] When the fifth combination is used, the region selection unit 106 selects, as the observation region, a partial region having a minimum value of the index based on the fifth combination. The index based on the fifth combination is suitable for performing analysis while avoiding multinucleated cells.

[0080] The sixth combination is a combination of the minimum value of the fluorescence intensity of the cells and the maximum value of the cell count. To identify the minimum value of the fluorescence intensity of the cells, the fluorescence intensity of the cells is calculated for each partial region. To identify the maximum value of the cell count, the cell count is calculated for each partial region. The value of the index based on the sixth combination is calculated, for example, by dividing the value of the cell count by the value of the fluorescence intensity of the cells for each partial region.

[0081] When the sixth combination is used, the region selection unit 106 selects as the observation region a partial region in which the value of the index based on the sixth combination is the largest. The index based on the sixth combination is suitable for increasing the number of cells as much as possible in a region in which no fluorescence is generated from the fluorescent substance that stains dead cells. Furthermore, the index based on the sixth combination is suitable for analyzing only live cells, excluding cells that died when a certain drug is administered, for example, by staining dead cells and excluding them from the analysis.

[0082] The seventh combination is a combination of the 75% value of the area per cell and the maximum cell count. To determine the 75% value of the area per cell, the area per cell is calculated for each partial region. Based on the area per cell value of each partial region, the 75% value is calculated using a known method for calculating the 75% value. Furthermore, to determine the maximum cell count, the cell count is calculated for each partial region. To calculate the index value based on the seventh combination, for example, first, the percentage value of the "area per cell" for each partial region is calculated, and then the difference between the calculated value (X%) for that partial region and 75% is divided by 100% (referred to as relative value X71). Next, the difference between the cell count for each partial region and the "maximum cell count" is divided by the "maximum cell count" (referred to as relative value X72). The index value based on the seventh combination is calculated as the product of relative value X71 and relative value X72.

[0083] When the seventh combination is used, the region selection unit 106 selects the partial region having the smallest value of the index based on the seventh combination as the observation region. The index based on the seventh combination is suitable for acquiring cells with large areas while minimizing outliers and acquiring as many cells as possible.

[0084] The eighth combination is the maximum value of the circularity of the cell. In the eighth combination, a single index is used instead of combining the first index and the second index. To identify the maximum value of the circularity of the cell, the circularity of the cell is calculated for each partial region. The value of the index based on the eighth combination is the circularity of the cell itself.

[0085] When the eighth combination is used, the region selection unit 106 selects the partial region with the largest index value based on the eighth combination as the observation region. The index based on the eighth combination is suitable for extracting cells in the mitotic phase. Note that cells with a high circularity tend to be dead cells.

[0086] For a combination of indices in which the product of the statistics of the first index and the statistics of the second index is evaluated, a weighted product may be evaluated instead of the product. For example, in the first combination as described above, the value of the index based on the first combination is calculated as the product of the absolute value X11 and the standard deviation X12. The value of the index based on the first combination may be calculated as the sum of the weighted absolute value X11 (aX11) and the weighted standard deviation X12 (bX12). The same applies to the indices based on the second, third, fourth, fifth, and seventh combinations. The value of the index based on the sixth combination may be calculated as the difference between the weighted value of the number of cells and the weighted value of the fluorescence intensity of the cells.

[0087] In this embodiment, an example in which the index is a combination of the first index and the second index will be described, but the present invention is not limited to this. A single index may be used as the index. For example, the index may be any one of the eight indexes shown in FIG. 4 . Alternatively, the index may be a combination of three or more indexes.

[0088] 6 to 10, a process flow in which the observation device 1 captures cell images and analyzes the captured cell images will be described. The process performed by the observation device 1 includes a first preparation process, a second preparation process, and a main imaging and analysis process, in this order.

[0089] 6 is a diagram showing an example of the flow of the first preparation process according to this embodiment. The first preparation process is a process for determining an observation area using a preliminary image.

[0090] Step S10: The analysis type input unit 115 inputs the type of analysis. The user inputs the type of analysis by operating the operation unit 40. The analysis type input unit 115 acquires analysis type information based on the operation signal supplied from the operation unit 40. Thereafter, the analysis device 10 executes the process of step S20.

[0091] Step S20: The microscope device 20 captures an overhead image of the entire well plate WP or most of the well plate WP. The microscope device 20 captures the overhead image by transmitted light bright-field observation using the second optical system. The cell image acquisition unit 101 acquires the overhead image. The cell image acquisition unit 101 supplies the overhead image to the image evaluation unit 102. The analysis device 10 then executes the process of step S30.

[0092] Step S30: The image determination unit 102 determines the type (container type) of the well plate WP. The image determination unit 102 determines the type (container type) of the well plate WP based on, for example, the shape, size, and number of wells W captured in the overhead image, based on image recognition. Here, the image determination unit 102 uses the magnification at which the overhead image was captured to determine the size of the wells W. Thereafter, the analysis device 10 executes the process of step S40.

[0093] Step S40: The image evaluation unit 102 aligns the well plate WP using the overhead image and corrects (acquires) information on the center position of each well of the well plate WP. Thereafter, the analysis device 10 executes the process of step S50.

[0094] Step S50: The microscope device 20 captures a pre-image for each well W by transmitted light bright-field observation using the second optical system. The image determination unit 102 determines whether or not cells are present in the pre-image. The image determination unit 102 determines the portion of the pre-image that corresponds to a cell based on image recognition. Thereafter, the analysis device 10 executes the process of step S60.

[0095] Step S60: The observation region selection unit 103 performs an observation region selection process, which is a process for determining an observation region for each of a plurality of wells W in which cells exist. Details of the observation region selection process will be described later with reference to FIG. 7.

[0096] Next, the observation area selection process will be described with reference to Fig. 7. Fig. 7 is a diagram showing an example of the flow of the observation area selection process according to this embodiment. As described above, the observation area selection process shown in Fig. 7 is executed by the observation area selection unit 103 as the process of step S60.

[0097] Step S100: The index calculation region changing unit 105 provisionally sets a partial region in the preliminary image for each well W. Thereafter, the observation region selecting unit 103 executes the process of step S110.

[0098] Here, the pre-image will be described with reference to Fig. 8. Fig. 8 is a diagram showing an example of a pre-image according to this embodiment. Fig. 8 shows a pre-image LP as an example of a pre-image acquired by the cell image acquisition unit 101. As described above, in this example, the pre-image LP is an image in which a roughly full view of the well W is captured.

[0099] The well W includes a seeding area CA, where cells are seeded. Due to various conditions, cells cannot be uniformly seeded in this seeding area CA. Because the seeding area CA contains cells after culture, the direction in which cells grow during the culture period is not constant. For example, cells may be cultured in a specific direction, resulting in uneven cell distribution. Therefore, the seeding area CA includes areas with a small number of cells or areas where cells overlap and are not suitable for analysis. The partial area CPA1 is an example of a partial area set by the index calculation area modification unit 105. Returning to FIG. 7 , we will continue with the description of the observation area selection process.

[0100] Step S110: The index calculation unit 104 calculates the value of an index for the partial region provisionally set by the index calculation region change unit 105. Here, the index calculation unit 104 reads out index information A1 from the storage unit 200, and determines an index to calculate a value based on the read out index information A1 and in accordance with the analysis type information acquired by the analysis type input unit 115. The index information A1 is, for example, the first to eighth combinations described above.

[0101] In the present embodiment, the case where the index for calculating a value is automatically determined has been described as an example, but this is not limiting. For example, an input unit for an index for calculating a value may be provided so that a user can operate the operation unit 40 to input the type of index for calculating a value in the process of step S10 in Fig. 6 (inputting the type of analysis) or after the process of step S50 in Fig. 6 ends and before the process of step S60 starts.

[0102] For example, when the index determined according to the analysis type information is the first combination, the index calculation unit 104 calculates the product of the absolute value X11 and the standard deviation X12. As described above, the absolute value X11 is the absolute value of the difference between the cell coverage area rate of each partial region and the cell coverage area rate of the entire preliminary image. The standard deviation X12 is the standard deviation of the distribution of the number of cells contained in the microregions obtained by further dividing the partial region. The index calculation unit 104 calculates the absolute value X11 and the standard deviation X12 based on the result of image recognition of the partial region by the image determination unit 102. Note that the absolute value X11 and the standard deviation X12 may be calculated by the image determination unit 102, and the calculated values ​​may be supplied to the index calculation unit 104. Thereafter, the observation region selection unit 103 performs the process of step S120.

[0103] Step S120: The index calculation area change unit 105 determines whether the entire selection target area in the preliminary image has been scanned by the partial area. The selection target area is the area in the preliminary image that is scanned by the partial area. The selection target area may be the entire preliminary image, or the remaining part of the entire preliminary image excluding the peripheral parts.

[0104] If the index calculation area modification unit 105 determines that the entire target area in the preliminary image has been scanned by the partial areas (step S120; YES), the observation area selection unit 103 executes the process of step S 140. On the other hand, if the index calculation area modification unit 105 determines that the entire selection target area in the preliminary image has not been scanned by the partial areas (step S120; NO), the observation area selection unit 103 executes the process of step S130.

[0105] Step S130: The index calculation region modification unit 105 modifies the partial region. The index calculation region modification unit 105 modifies the partial region, for example, by translating (shifting) the partial region by a predetermined position in the X direction in the preliminary image. If the translation causes the partial region to protrude from the selection target region, the index calculation region modification unit 105 translates the partial region by a predetermined position in the Y direction in the preliminary image. Thereafter, the index calculation region modification unit 105 again translates the partial region by a predetermined position in the X direction in the preliminary image. In other words, the index calculation region modification unit 105 scans in the X direction while changing the position in the Y direction. The index calculation region modification unit 105 may also scan in the Y direction while changing the position in the X direction. The partial regions may be selected in any order.

[0106] When the index calculation region modification unit 105 modifies a partial region, the modified partial region may have an overlapping portion with an already selected partial region. The index calculation region modification unit 105 may modify the partial region so that the modified partial region does not overlap with an already selected partial region.

[0107] In the example shown in FIG. 8, the index calculation area change unit 105 changes the partial area from the partial area CPA1 to the partial area CPA2.

[0108] The description of the observation area selection process will be continued by returning to Fig. 7. After that, the observation area selection unit 103 executes the process of step S110 again.

[0109] Step S140: The region selection unit 106 selects (determines) an observation region to be used for analysis from the plurality of partial regions, based on the index value of each observation region calculated by the index calculation unit 104. In this embodiment, the region selection unit 106 selects one observation region to be used for analysis from the plurality of partial regions, based on the index value calculated by the index calculation unit 104. In this case, for example, when the first combination is selected from the index information A1, the region selection unit 106 selects the partial region having the smallest index value based on the first combination as the observation region.

[0110] The area selection unit 106 selects the partial area with the highest evaluation based on the index values ​​calculated by the index calculation unit 104. For example, if the type of index is the first combination, the area selection unit 106 selects the partial area with the smallest index value based on the first combination as the observation area. The area selection unit 106 outputs the observation area of ​​each well W to the result output unit 300. This completes the observation area selection process for the observation area selection unit 103.

[0111] 6, the description of the first preparation process will be continued. After that, the result output unit 300 executes the process of step S70. Step S70: The result output unit 300 causes the display unit 30 to display the result (the selection result of the observation area) supplied from the area selection unit 106.

[0112] The display unit 30 displays multiple pre-images in a row, corresponding to the addresses of the wells W in the well plate WP, and displays the observation area of ​​each well W superimposed on the pre-images. Alternatively, multiple wells W may be displayed as a top view of the well plate WP, and the observation area of ​​each well W may be superimposed on the wells W. Since the observation area is set only for wells W in which cells are present, no observation area is displayed for pre-images in which no cells are present or for wells W in which no cells are present. With this, the observation device 1 completes the first preparation process.

[0113] Next, the second preparation process will be described with reference to Fig. 9. Fig. 9 is a diagram showing an example of the flow of the second preparation process according to this embodiment. The second preparation process is executed when, after the first preparation process described above is completed, the user checks the results displayed on the display unit 30 in the process of step S70 (Fig. 6), determines whether to proceed with the second preparation process, and operates the operation unit 40 to press a "next button" (not shown).

[0114] Step S210: A laser beam from a laser light source (not shown in Figure 3) is irradiated onto the bottom surface of the well plate, the intensity of the reflected light is detected by a detector (not shown), and the material of the well plate WP is determined based on the intensity. After that, the analyzer 10 executes the process of step S220.

[0115] Step S220: The autofocus unit 108 performs autofocus processing. Through the autofocus processing, the autofocus unit 108 determines the observation position in the Z-axis direction for each observation region of each well W. The autofocus unit 108 selects an autofocus algorithm depending on the container material.

[0116] For example, when the container material is glass, the autofocus unit 108 selects an autofocus algorithm based on image contrast (image contrast AF). Alternatively, a method may be selected in which autofocus is performed on the bottom surface (interface) of the well plate WP and then offset by a predetermined amount in the optical axis direction. Furthermore, when the container material is plastic, for example, the autofocus unit 108 selects an autofocus algorithm based on AI. The autofocus method to be applied is selected depending on the material of the well plate WP.

[0117] Here, the control signal output unit 500 sets the focal position of the objective lens 23. The control signal output unit 500 sets the focal position of the objective lens 23 to the observation position in the Z axis direction determined by the autofocus unit 108. The control signal output unit 500 supplies the objective lens driving unit 24 with a control signal that moves the objective lens 23 so that the focal position of the objective lens 23 coincides with the observation position in the Z direction. The objective lens driving unit 24 drives the objective lens 23 using the control signal. Thereafter, the observation device 1 executes the processing of step S230.

[0118] Step S230: The microscope device 20 captures bright-field images and fluorescent images as pre-ROI images at the observation positions in the Z-axis direction for each observation region in each well W. The microscope device 20 captures bright-field images by transmitted light bright-field observation using the first optical system (high magnification), and fluorescent images by epi-fluorescence observation using the first optical system (high magnification). Note that if a fluorescent image is not captured as the main image (described later), there is no need to capture a fluorescent image as the pre-ROI image.

[0119] The cell image acquisition unit 101 acquires a bright-field image and a fluorescent image. After that, the analysis device 10 executes the process of step S240.

[0120] Step S240: The abnormality detection unit 110 determines whether there is an abnormality in cell morphology or condensation in the bright-field image. The abnormality detection unit 110 also determines whether there is an abnormality in the signal in the fluorescent image.

[0121] The abnormality detection unit 110 outputs the presence or absence of abnormalities in cell morphology in the bright-field image, the presence or absence of condensation, and the presence or absence of signal abnormalities in the fluorescence image to the result output unit 300. The illumination condition determination unit 111 determines illumination (excitation) conditions (such as the intensity of excitation light) based on the fluorescence image, and outputs them to the result output unit 300. Thereafter, the analysis device 10 executes the process of step S250.

[0122] Step S250: The result output unit 300 causes the display unit 30 to display the results (detection results) supplied from the anomaly detection unit 110 and the illumination condition determination unit 111. With this, the observation device 1 ends the second preparation process.

[0123] If the area selection unit 106 selects multiple partial areas as candidate observation areas for each well W based on the index values ​​calculated by the index calculation unit 104, it executes the processes from step S220 to step S240 for the multiple partial areas and displays the results on the display unit 30. In this case, the user determines the observation area for each well W by selecting one from the candidate observation areas for each well W.

[0124] Next, the main imaging analysis process will be described with reference to Fig. 10. Fig. 10 is a diagram showing an example of the flow of the main imaging analysis process according to this embodiment. The main imaging analysis process is executed when, after the second preparation process described above is completed, the user checks the results displayed on the display unit 30 in step S250, determines whether or not to proceed with the main imaging analysis process, and operates the operation unit 40 to press a "next button" (not shown).

[0125] Step S310: The microscope device 20 captures a bright-field image and a fluorescent image as the main image at the observation position in the Z-axis direction for each observation region for each observation region in each well W. The microscope device 20 captures a bright-field image by transmitted light bright-field observation using the first optical system (high magnification), and captures a fluorescent image by epi-fluorescence observation using the first optical system (high magnification) under the illumination conditions determined by the illumination condition determination unit 111.

[0126] The cell image acquisition unit 101 acquires a bright field image and a fluorescent image, and supplies the acquired images to the analysis unit 107. Thereafter, the analysis device 10 executes the process of step S320.

[0127] Step S320: The analysis unit 107 analyzes the actual image acquired from the cell image acquisition unit 101. The analysis unit 107 supplies the analysis results to the result output unit 300. Thereafter, the analysis device 10 executes the process of step S330.

[0128] Step S330: The result output unit 300 displays the analysis results supplied from the analysis unit 107 on the display unit 30. With this, the observation device 1 ends this imaging and analysis process.

[0129] In this embodiment, the observation area selection process for each well W determines the type of indicator depending on the type of analysis, but it is also possible to use only one set of indicators (a combination of two types of indicators, for example, the first combination) that have been stored in advance in the memory unit 200.

[0130] Although the observation region of each well W is automatically determined by the observation region selector 103 in the above example, this is not limiting. For example, in the process of step S10 of FIG. 6 (inputting the type of analysis), an observation region determination method input unit may be provided so that the user can operate the operation unit 40 to select either of the following processes c) and d). c) Automatic determination by the observation region selector 103; d) Predetermined determination of a predetermined region (e.g., the center region of the well W). If process d) is input into the observation region determination method input unit, the process shown in FIG. 7 is not performed. Instead, the observation region selector 103 determines a predetermined region for each of the multiple wells W containing cells, and then executes the process of step S70 shown in FIG. 6.

[0131] As described above, the observation device 1 according to this embodiment is an observation device that analyzes cell images and includes an image acquisition unit (in this embodiment, the microscope device 20), an index calculation unit 104, and a region selection unit 106. The image acquisition unit (in this embodiment, the microscope device 20) acquires a second image (in this embodiment, the preliminary image) of a region larger than the region used for analysis, at a pixel resolution lower than the pixel resolution of a first image (in this embodiment, the main image) used for analysis. The index calculation unit 104 calculates the value of a cell-related index for each partial region included in the second image (in this embodiment, the preliminary image) that corresponds to the region used for analysis. The region selection unit 106 selects a region to be used for analysis (in this embodiment, the observation region) from the multiple partial regions based on the index values ​​calculated by the index calculation unit 104.

[0132] With this configuration, the observation device 1 according to this embodiment can select an area to be used for analysis (an observation area in this embodiment) from among the partial areas included in the second image (a preliminary image in this embodiment) based on the index values ​​calculated for each of the partial areas, thereby preventing the selection of the area to be used for analysis (an observation area in this embodiment) from being arbitrary. Therefore, the observation device 1 according to this embodiment can reduce variation in the analysis results due to the selection of the area to be used for analysis.

[0133] In HCA (High Content Analysis), images are captured for each well, and quantitative analysis is performed. The analysis results may vary depending on how the observation area within the well is selected. For example, because cell behavior varies even within a single well, the selection of the observation area within the well can cause variations in the analysis results. If an area with few cells is selected as the observation area, AF may not be applied, or statistical analysis may not be possible.

[0134] Conventionally, each well was observed and the observation area used for imaging and analysis was determined. Observing each well individually to determine the observation area was extremely time-consuming. Furthermore, conventionally, the observation area was determined to be the center of the well, multiple wells with identical conditions were prepared, and statistically outlier wells were excluded from analysis. This increased reagent costs and the number of experiments required.

[0135] On the other hand, the observation device 1 according to this embodiment can select the region of the observation target that is most suitable for observation. The region that is most suitable for observation is, for example, a representative location when various cell behaviors in one well are taken into consideration.

[0136] Furthermore, in the observation device 1 according to this embodiment, the index includes a first index and a second index (in this embodiment, the index is a combination of the first index and the second index). With this configuration, in the observation device 1 according to this embodiment, the region to be used for analysis can be selected based on an index including the first index and the second index, so a more suitable region can be selected compared to when the region to be used for analysis is selected based on the value of a single index.

[0137] Furthermore, in the observation device 1 according to this embodiment, the first index and the second index are selected from the following A) to H): A) cell coverage area ratio, B) cell density distribution, C) cell count, D) cell fluorescence intensity, E) cell confluency, F) cell circularity, G) area per cell, H) cell multinucleation rate With this configuration, in the observation device 1 according to this embodiment, the region to be used for analysis can be selected based on the first index and the second index selected from the above A) to H), and therefore the region can be selected according to the purpose of the analysis.

[0138] The observation device 1 according to this embodiment includes another input unit (in this embodiment, an analysis type input unit 115). The input unit (in this embodiment, the analysis type input unit 115) inputs the type of analysis. The analysis type input unit 115 determines the index for calculating the index value according to the type of analysis.

[0139] Furthermore, in the observation device 1 according to this embodiment, the image acquisition unit (in this embodiment, the microscope device 20) includes a detection optical system (in this embodiment, a first optical system and a second optical system) having a plurality of objective lenses (in this embodiment, a high-magnification objective lens 231 and a low-magnification objective lens 232). The first image (in this embodiment, the main image) is acquired by a first objective lens (in this embodiment, the high-magnification objective lens 231) among the plurality of objective lenses. The second image (in this embodiment, the pre-image) is acquired by a second objective lens (in this embodiment, the low-magnification objective lens 232) among the plurality of objective lenses, the second objective lens having a lower magnification than the first objective lens (in this embodiment, the high-magnification objective lens 231).

[0140] With this configuration, the observation device 1 according to this embodiment has a simpler configuration than when the optical system for acquiring the first image (in this embodiment, the main image) and the optical system for acquiring the second image (in this embodiment, the pre-image) are provided separately in the image acquisition unit (in this embodiment, the microscope device 20).

[0141] Furthermore, in the observation device 1 according to this embodiment, the image acquisition unit (in this embodiment, the microscope device 20) may include a first optical system for acquiring a first image (in this embodiment, the main image) and a second optical system for acquiring a second image (in this embodiment, the pre-image).

[0142] With this configuration, the observation device 1 according to this embodiment makes it easier to achieve the desired imaging conditions (magnification) compared to when an optical system for acquiring a first image (in this embodiment, the main image) and an optical system for acquiring a second image (in this embodiment, the pre-image) are provided as an integrated optical system.

[0143] Second Embodiment A second embodiment of the present invention will be described in detail below with reference to the drawings. In the first embodiment, a case where an index is determined depending on the type of analysis is described. In this embodiment, a case where an index is determined depending on the type of cell is described. The observation device 1 according to this embodiment is referred to as observation device 1a, and the analysis device is referred to as analysis device 10a. Note that the same components as those in the first embodiment described above are denoted by the same reference numerals, and descriptions of the same components and operations may be omitted.

[0144] Fig. 11 is a block diagram showing an example of the functional configuration of an observation device 1a according to this embodiment. The observation device 1a includes an analysis device 10a, a microscope device 20, a display unit 30, and an operation unit 40. Comparing the observation device 1a according to this embodiment (Fig. 11) with the observation device 1 according to the first embodiment (Fig. 2), the analysis device 10a is different. Here, the functions of the other components (the microscope device 20, the display unit 30, and the operation unit 40) are the same as those of the first embodiment.

[0145] The analysis device 10a includes a calculation unit 100a, a storage unit 200, a result output unit 300, an operation detection unit 400, and a control signal output unit 500. Comparing the calculation unit 100a ( FIG. 11 ) according to this embodiment with the calculation unit 100 ( FIG. 2 ) according to the first embodiment, the calculation unit 100a differs from the calculation unit 100 according to the first embodiment in the observation area selection unit 103a and the cell type input unit 112a. The functions of the other components (cell image acquisition unit 101, image evaluation unit 102, analysis unit 107, autofocus unit 108, abnormality detection unit 110, and illumination condition determination unit 111) are the same as those of the first embodiment.

[0146] The cell type input unit 112a inputs the cell type. Here, the user inputs the cell type by operating the operation unit 40. The operation unit 40 supplies an operation signal corresponding to the input cell type to the analysis device 10. The cell type input unit 112a acquires cell type information based on the operation signal supplied from the operation unit 40. The cell type information is information indicating the type of cell. Note that the cell type input unit 112a may acquire the cell type information from an external device provided separately from the analysis device 10a.

[0147] The observation area selection unit 103a includes an index calculation unit 104a, an index calculation area modification unit 105, and an area selection unit 106. The functions of the index calculation area modification unit 105 and the area selection unit 106 are the same as those in the first embodiment.

[0148] The index calculation unit 104a determines an index for calculating an index value according to the cell type information acquired by the cell type input unit 112a. The index calculation unit 104a determines the index based on index information A1a. The index information A1a is information in which the cell type and the index type are associated with each other. For example, the index information A1a is two-dimensional tabular data consisting of rows and columns in which the index type is stored for each cell type. The index information A1a is stored in advance in the storage unit 200. Note that the index information A1a may be created by a user operating the operation unit 40.

[0149] [Imaging process] Next, the imaging process by the observation device 1a will be described with reference to Fig. 12. The imaging process by the observation device 1a is different from the imaging process by the observation device 1 according to the first embodiment in the observation area selection process included in the first preparation process.

[0150] In the imaging process by the observation device 1a, in the process of step S10 in Fig. 6, instead of the analysis type input unit 115 inputting the type of analysis, the cell type input unit 112a inputs the type of cell. The user inputs the type of cell by operating the operation unit 40. The cell type input unit 112a acquires cell type information based on an operation signal supplied from the operation unit 40. Note that in addition to the process of the analysis type input unit 115 inputting the type of analysis, the cell type input unit 112a may also input the type of cell.

[0151] Fig. 12 is a diagram showing an example of the flow of observation area selection processing according to this embodiment. The observation area selection processing shown in Fig. 12 is executed by the observation area selection unit 103a as the processing of step S60 in Fig. 6. Note that the processing of steps S400, S420, S430, and S440 is similar to the processing of steps S100, S120, S130, and S140 in Fig. 7, and therefore description thereof will be omitted.

[0152] Step S410: The index calculation unit 104a calculates an index value for the partial region provisionally set by the index calculation region changing unit 105. The index calculation unit 104a determines an index for calculating the index value according to the cell type information acquired by the cell type input unit 112a. Here, the index calculation unit 104a reads out index information A1a from the storage unit 200, and determines an index associated with the cell type information based on the read-out index information A1a. Thereafter, the analysis device 10a executes the process of step S420.

[0153] As described above, the observation device 1a according to this embodiment includes another input unit (in this embodiment, the cell type input unit 112a). The other input unit (in this embodiment, the cell type input unit 112a) inputs the type of cell. The cell type input unit 112a determines the index for calculating the index value according to the type of cell.

[0154] With this configuration, the observation device 1a according to this embodiment can select an area to be used for analysis from among multiple partial areas based on an index corresponding to the cell type, thereby enabling the selection of an area more suitable for analysis compared to when the area is selected based on a predetermined index.

[0155] (Third Embodiment) A second embodiment of the present invention will be described in detail below with reference to the drawings. In this embodiment, when partial regions in which foreign matter is captured exist in a preliminary image, a predetermined value is added to or subtracted from the index value for each of the partial regions. The observation device according to this embodiment is referred to as observation device 1b, and the analysis device is referred to as analysis device 10b. Note that the same components as those in the first embodiment described above are designated by the same reference numerals, and descriptions of the same components and operations may be omitted.

[0156] Fig. 13 is a block diagram showing an example of the functional configuration of an observation device 1b according to this embodiment. The observation device 1b includes an analysis device 10b, a microscope device 20, a display unit 30, and an operation unit 40. Comparing the observation device 1b according to this embodiment (Fig. 13) with the observation device 1 according to the first embodiment (Fig. 2), the analysis device 10b is different. Here, the functions of the other components (the microscope device 20, the display unit 30, and the operation unit 40) are the same as those of the first embodiment.

[0157] The analysis device 10b includes a calculation unit 100b, a storage unit 200, a result output unit 300, an operation detection unit 400, and a control signal output unit 500. Comparing the calculation unit 100b ( FIG. 13 ) according to this embodiment with the calculation unit 100 ( FIG. 2 ) according to the first embodiment, the calculation unit 100b differs from the calculation unit 100 according to the first embodiment in the observation area selection unit 103b and the foreign matter detection unit 113b. The functions of the other components (cell image acquisition unit 101, image evaluation unit 102, analysis unit 107, autofocus unit 108, abnormality detection unit 110, and illumination condition determination unit 111) are the same as those of the first embodiment.

[0158] The foreign matter detection unit 113b detects a foreign matter when it is captured in the preliminary image. The foreign matter may be, for example, a scratch on the surface of the well W, or dust or dirt mixed into the well W. The partial area in the preliminary image where the foreign matter is captured is an area that should be excluded from the observation area. The brightness of the foreign matter in the captured image is generally higher than the brightness of cells. Therefore, for example, the foreign matter may be mistakenly recognized as a cell and analyzed.

[0159] The observation region selection unit 103b includes an index calculation unit 104b, an index calculation region modification unit 105, and a region selection unit 106. The functions of the index calculation region modification unit 105 and the region selection unit 106 are the same as those in the first embodiment.

[0160] When the region selection unit 106 selects partial regions with small index values ​​as regions to be used for analysis, and there are partial regions in which foreign matter is imaged, the index calculation unit 104b adds a predetermined value to the index value of each of the partial regions. On the other hand, when the region selection unit 106 selects partial regions with large index values ​​as regions to be used for analysis, and there are partial regions in which foreign matter is imaged, the index calculation unit 104b subtracts a predetermined value from the index value of each of the partial regions. Here, it is preferable that the predetermined value is sufficiently large so that partial regions in which foreign matter is imaged are not selected as observation regions.

[0161] [Imaging Process] Next, the imaging process by the observation device 1b will be described with reference to Fig. 14 to Fig. 16. The imaging process by the observation device 1b differs from the imaging process by the observation device 1 according to the first embodiment in the first preparation process.

[0162] 14 is a diagram showing an example of the flow of the first preparation process according to this embodiment. Note that the processes from step S510 to step S550 and step S580 are similar to the processes from step S10 to step S50 and step S70 in FIG. 6, and therefore descriptions thereof will be omitted.

[0163] Step S560: The foreign object detection unit 113b determines whether a foreign object is captured in the preliminary image. The foreign object detection unit 113b detects a foreign object based on, for example, the results of image recognition for a partial region performed by the image determination unit 102. If the foreign object detection unit 113b detects a foreign object, it supplies foreign object position information to the index calculation unit 104b. The foreign object position information is information that indicates the position of the foreign object in the preliminary image. Thereafter, the analysis device 10b executes the process of step S570.

[0164] Here, referring to Fig. 15, a case where a foreign substance is captured in a preliminary image will be described. Fig. 15 is a diagram showing an example of a preliminary image in which a foreign substance is captured according to this embodiment. Fig. 15 shows a preliminary image LPb as an example of a preliminary image acquired by the cell image acquisition unit 101. In the preliminary image LPb, a flaw SC1 is captured in the partial area CPA2. The flaw SC1 is an example of a foreign substance. Returning to Fig. 14, the description of the first preparation process will be continued.

[0165] Step S570: The observation region selection unit 103b performs an observation region selection process. The observation region selection unit 103b performs the process shown in Fig. 16 as the observation region selection process, which differs from the observation region selection process (Fig. 7) performed by the observation region selection unit 103 according to the first embodiment.

[0166] 16 is a diagram showing an example of the flow of observation area selection processing according to this embodiment. Note that the processes in step S600 and steps S640 to S660 are similar to the processes in step S100 and steps S120 to S140 in FIG. 7, and therefore descriptions thereof will be omitted.

[0167] Step S610: The index calculation unit 104b determines whether or not a foreign object has been captured in the current partial region based on the foreign object position information supplied from the foreign object detection unit 113b.

[0168] If the index calculation unit 104b determines that a foreign object is imaged in the current partial region (step S610; YES), it executes the process of step S620. On the other hand, if the index calculation unit 104b determines that a foreign object is not imaged in the current partial region (step S610; NO), it executes the process of step S630.

[0169] Step S620: The index calculation unit 104b adds or subtracts a predetermined value to the index value of the partial region. When the region selection unit 106 selects the partial region with the smallest index value as the region to be used for analysis, the index calculation unit 104b adds a predetermined value to the index value of the partial region. On the other hand, when the region selection unit 106 selects the partial region with the largest index value as the region to be used for analysis, the index calculation unit 104b subtracts a predetermined value from the index value of the partial region.

[0170] Here, when the region selection unit 106 selects the partial region with the smallest index value as the region to be used for analysis, if a predetermined value has been added to the index value of the partial region in which the image of a foreign substance has been captured, the index value of that partial region will be a large value. Therefore, that partial region will not be selected as the region to be used for analysis. Similarly, when the region selection unit 106 selects the partial region with the largest index value as the region to be used for analysis, if a predetermined value has been subtracted from the index value of the partial region in which the image of a foreign substance has been captured, the index value of that partial region will be a small value. Therefore, that partial region will not be selected as the region to be used for analysis. Thereafter, the analysis device 10b executes the process of step S630.

[0171] In the observation device 1b according to this embodiment, even if a foreign object is captured in the preliminary image, the index value can be calculated appropriately, and therefore the area to be used for analysis can be selected by excluding the area in which the foreign object is captured.

[0172] In the above-described embodiments, an example has been described in which one pre-image captures one well W and an observation area is determined for each well W, but this is not limiting. A pre-image including multiple wells may be used to determine the observation area for each well. In this case, the microscope device 20 may acquire multiple images and combine them to generate a single image, thereby acquiring a single pre-image. For example, images of multiple adjacent wells in a well plate WP captured at low magnification by the second optical system may be combined to form a single pre-image.

[0173] Alternatively, multiple (e.g., four) wells of the well plate WP may be imaged at a low magnification by the second optical system at one time to obtain a preliminary image. This low magnification is even lower than the magnification used to image a single well W.

[0174] Therefore, the image acquisition unit (in each embodiment, the microscope device 20) may acquire a second image (in each embodiment, a pre-image) by acquiring multiple images and concatenating them to generate a single image.

[0175] Note that a portion of the analysis devices 10, 10a, and 10b in the above-described embodiments, such as the calculation units 100, 100a, and 100b, may be implemented by a computer. In this case, a program for implementing this control function may be recorded on a computer-readable recording medium, and the program may be loaded into a computer system and executed. Note that the term "computer system" refers to a computer system built into the analysis devices 10, 10a, and 10b, including hardware such as an OS and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into computer systems. Furthermore, the term "computer-readable recording medium" may also include media that dynamically store programs for short periods of time, such as communication lines used when transmitting programs via networks such as the Internet or telephone lines, or media that store programs for a fixed period of time, such as volatile memory within the computer system that serves as the server or client in such cases. Furthermore, the program may be designed to implement a portion of the aforementioned functions, or may be capable of implementing the aforementioned functions in combination with a program already stored in the computer system. Furthermore, some or all of the analysis devices 10, 10a, and 10b in the above-described embodiments may be realized as integrated circuits such as LSIs (Large Scale Integration). Each functional block of the analysis devices 10, 10a, and 10b may be individually implemented as a processor, or some or all of them may be integrated into a processor. Furthermore, the integrated circuit implementation method is not limited to LSIs, and may be implemented using dedicated circuits or general-purpose processors. Furthermore, if an integrated circuit implementation technology that can replace LSIs emerges due to advances in semiconductor technology, an integrated circuit based on that technology may be used.

[0176] One embodiment of the present invention has been described in detail above with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes and the like are possible within the scope that does not deviate from the gist of the present invention.

[0177] 1, 1a, 1b... observation device, 20... microscope device, 104, 104a, 104b... index calculation unit, 106... area selection unit

Claims

1. An observation device for analyzing cell images, comprising: an image acquisition unit that acquires a second image of an area larger than the area used for the analysis at a pixel resolution lower than the pixel resolution of a first image used for the analysis; an index calculation unit that calculates a value of a cell-related index for each partial area included in the second image that corresponds to the area used for the analysis; and an area selection unit that selects the area to be used for the analysis from the multiple partial areas based on the index value calculated by the index calculation unit.

2. The observation device according to claim 1, wherein the index includes a first index and a second index.

3. The observation device according to claim 2, wherein the first index and the second index are selected from the following A) to H). A) cell coverage area ratio, B) cell density distribution, C) cell count, D) cell fluorescence amount, E) cell confluency, F) cell circularity, G) area per cell, H) cell multinucleation rate.

4. An observation device as described in any one of claims 1 to 3, further comprising a first input unit for inputting the type of analysis, wherein the index calculation unit determines the index for calculating the value of the index depending on the type of analysis.

5. An observation device as described in any one of claims 1 to 4, further comprising a second input unit for inputting the type of cell, wherein the index calculation unit determines the index for calculating the value of the index depending on the type of cell.

6. An observation device as described in any one of claims 1 to 5, further comprising a third input unit for inputting a method for determining a region to be used in the analysis, the method including a method for determining a region to be used in the analysis based on the index calculation unit and the region selection unit, and a method for determining a predetermined region within the second image as the region to be used in the analysis.

7. An observation device as described in any one of claims 1 to 6, wherein the second image is an image of cells placed in each of a plurality of storage compartments of a container having the plurality of storage compartments, the image being captured for each of the storage compartments.

8. An observation device as described in claim 7, further comprising an image determination unit that determines the presence or absence of a cell in the second image, and the index calculation unit calculates the value of the index for the second image in which the cell is present.

9. An observation device as claimed in any one of claims 1 to 8, wherein the image acquisition unit includes a detection optical system having a plurality of objective lenses, the first image is acquired by a first objective lens among the plurality of objective lenses, and the second image is acquired by a second objective lens among the plurality of objective lenses having a lower magnification than the first objective lens.

10. An observation device as described in any one of claims 1 to 9, wherein the image acquisition section comprises a first optical system for acquiring the first image and a second optical system for acquiring the second image.

11. The observation device according to any one of claims 1 to 10, wherein the image acquisition unit acquires the second image by acquiring a plurality of images and combining them to generate a single image.

12. An observation device as described in any one of claims 1 to 11, wherein when the region selection unit selects the partial region with a small index value as the region to be used in the analysis, and when there is a partial region in which a foreign object is imaged, the index calculation unit adds a predetermined value to the index value of each of the partial regions, and when the region selection unit selects the partial region with a large index value as the region to be used in the analysis, and when there is a partial region in which a foreign object is imaged, the index calculation unit subtracts a predetermined value from the index value of each of the partial regions.

13. An observation device as described in any one of claims 1 to 12, wherein the area selection unit selects a plurality of areas as candidates for areas to be used in the analysis from a plurality of the partial areas based on the index calculated by the index calculation unit.

14. An observation method for analyzing a cell image, comprising: acquiring a second image of an area larger than an area used for the analysis at a pixel resolution lower than the pixel resolution of a first image used for the analysis; calculating a value of a cell-related index for each partial area included in the second image that corresponds to the area used for the analysis; and selecting an area to be used for the analysis from the multiple partial areas based on the index value calculated by the calculation.

15. A program for causing a computer that analyzes cell images to perform the following steps: obtain a second image of an area larger than the area used in the analysis at a pixel resolution lower than the pixel resolution of a first image used in the analysis; calculate a cell-related index value for each partial area included in the second image that corresponds to the area used in the analysis; and select an area to be used in the analysis from the multiple partial areas based on the index value calculated by the calculation.

Images