Cell tracking method, image processing apparatus, and program

Abstract

To provide a cell tracking method of tracking a cell on the basis of a plurality of cell images which are captured in time series, in which the accuracy of analysis concerning a moving state of cells or the like is improved.SOLUTION: A tracking method includes: an extraction process of extracting a tracking region corresponding to a cell for each of a plurality of cell images; a position calculation process of calculating a change in the position of the tracking region extracted in the extraction process on the basis of the plurality of cell images, and tracking a cell on the basis of the calculated positional change of the tracking region; a determination process of determining whether or not a cell to be tracked is in a cell division state on the basis of a cell image; and a stop process of stopping cell tracking when the cell is in a cell division state in the determination process.SELECTED DRAWING: None

Description

Technical Field

[0001] The present invention relates to a cell tracking method , Image Processing Apparatus, and Program thereof.

Background Art

[0002] In the image analysis of an image of cells captured, a cell tracking technique for measuring the position of the captured cells at each time is known (Patent Document 1). For example, in the cell tracking technique, when tracking cells that undergo division, conventionally, tracking may continue on one of the cells after cell division, and tracking may be newly started on the other cell. In such a case, since the center of gravity moves instantaneously during cell division, it can become noise in, for example, the analysis of the movement state of cells by tracking. It is required to improve the accuracy of the analysis of the movement state of cells and the like.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

[0004] One aspect of the present invention is a cell tracking method for tracking cells based on a plurality of cell images captured in time series, and for each of the plurality of cell images, Based on the luminance information of the cell image, it is the tracking target of the cell an extraction step of extracting a tracking region, and Based on the plurality of cell images, calculate the time change of the position of the tracking region extracted in the extraction step, and have an analysis step of analyzing the movement state of the cell based on the time change. When analyzing the movement state of the cell, the analysis step excludes the time when the cell is not moving in the calculation process of the time change. a cell tracking method. One aspect of the present invention is an image processing apparatus that performs cell tracking processing based on a plurality of cell images captured in time series. For each of the plurality of cell images, a region extraction unit that extracts a tracking region that is the tracking target of the cell based on the luminance information of the cell image, and based on the plurality of cell images, calculates the time change of the position of the tracking region extracted by the region extraction unit, and a movement state calculation unit that analyzes the movement state of the cell based on the time change. The movement state calculation unit is an image processing apparatus that excludes the time when the cell is not moving in the calculation process of the time change when analyzing the movement state of the cell. One aspect of the present invention is a program for causing a computer that executes cell tracking based on a plurality of cell images captured in time series to execute, for each of the plurality of cell images, an extraction step of extracting a tracking region that is the tracking target of the cell based on the luminance information of the cell image, and a calculation step of calculating the time change of the position of the tracking region extracted in the extraction step based on the plurality of cell images, and an analysis step of analyzing the movement state of the cell based on the time change. The analysis step is a program that excludes the time when the cell is not moving in the calculation process of the time change when analyzing the movement state of the cell.

Brief Description of the Drawings

[0005]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Embodiments for Carrying Out the Invention

[0006] (First Embodiment) Hereinafter, the first embodiment will be described in detail with reference to the drawings. FIG. 1 is a diagram showing an example of the configuration of an image processing apparatus 1 according to this embodiment. The image processing apparatus 1 analyzes an image P, which is a plurality of images in which cells C are detected at a plurality of times (for example, time T1 to time T7 described later), by tracking the cells C. Tracking of the cells C includes, for example, calculating the trajectories of the cells C in the image P based on specific conditions. The cells C are, for example, adherent cells (for example, mesenchymal stem cells, nerve cells, epithelial cells, etc.). Note that the tracking process in this embodiment by the image processing apparatus 1 may be executed for each time for one cell C, or may be executed in parallel for each time for a plurality of cells C (for example, the first cell, the second cell, the third cell, etc.).

[0007] The automatic culture observation apparatus 200 in this embodiment includes a microscope 2 and a culture chamber (culture apparatus) 20. The microscope 2 is an optical microscope and is, for example, a phase contrast microscope. The microscope 2 performs phase contrast observation in dark contrast and detects one or more cells. The microscope 2 captures the detected cells as an image P. The culture chamber 20 includes a chamber whose internal temperature and humidity are controlled by a control unit or the like in order to culture the cells C stored in a container. Also, in this embodiment, the microscope 2 and the culture chamber 20 may be arranged separately, or the microscope 2 may be arranged inside the culture chamber 20. Note that in this embodiment, the microscope 2 may have a device configuration separated from the culture chamber 20. The image P is, for example, a moving image composed of a plurality of frames. Hereinafter, the i-th frame of the image P may be referred to as an image Pi or the like. Note that the image P may be a time-lapse image captured at a plurality of shooting times.

[0008] The image processing apparatus 1 includes an image acquisition unit 10, a control unit 11, an output unit 12, and a storage unit 13. The image processing apparatus 1 is, for example, a computer. In this embodiment, as an example, the case where the image processing apparatus 1 is provided independently of the microscope 2 will be described, but it is not limited thereto. The image processing apparatus 1 may be provided integrally with the microscope 2.

[0009] The image acquisition unit 10 receives and acquires the image P output from the microscope 2. The control unit 11 includes a tracking area extraction unit (area extraction unit) 110, a position calculation unit 111, a state determination unit 112, a stop control unit 113, and a movement state calculation unit 114.

[0010] The control unit 11 is realized by a CPU (Central Processing Unit), and the tracking area extraction unit 110, the position calculation unit 111, the state determination unit 112, the stop control unit 113, and the movement state calculation unit 114 are each modules realized by the CPU reading a program from a ROM (Read Only Memory) and executing processing.

[0011] The tracking area extraction unit 110 extracts a cell area R (e.g., area R1, area R2, etc.) from the image P in which the cells are imaged using luminance information. The area R includes at least one of a tracking area TR in which the luminance state indicated by the luminance information is the first state X and a tracking stop area SR in which the luminance state indicated by the luminance information is the second state Y. Here, the luminance information is, as an example, a luminance value based on the phase difference of the observation light of the cells in a phase contrast microscope, and the luminance state indicated by the luminance information is, as an example, the state indicated by this luminance value (e.g., a state where the value is large, a state where the value is small, etc.). The luminance information may be a value indicating luminance other than the luminance value, and may be, for example, contrast or the phase difference before being converted into luminance (e.g., brightness). For example, a predetermined area (e.g., area R) of the image P is defined by one pixel or a plurality of pixels among all the pixels constituting the image P. Also, for example, the luminance information of a predetermined area (e.g., area R) in the image P may be each luminance value or the average luminance value of the plurality of pixels.

[0012] The first state X is, as an example, a state in which the luminance value of a specific part (e.g., a tracking area, etc.) in the image P is compared (e.g., calculated) with a predetermined value and is smaller than the predetermined value. In the present embodiment, an example of the first state X in which the luminance value is smaller than the predetermined value is an example in the case of dark contrast. The first state X includes, for example, a state in which the luminance value of the specific part with respect to the predetermined value is relatively or absolutely low. As the predetermined value, for example, the luminance value of the background BG (e.g., the background BG1 described later, etc.) of the first frame of the image P or a threshold value related to a preset luminance value is used. The luminance value of the background BG may be the average value of the luminance values obtained from the entire background BG, or may be the luminance value obtained from a specific part of the background BG. In the present embodiment, as an example of the case where the average value of the luminance values obtained from the entire background BG of the first frame of the image P is commonly used for all the frames included in the image P, an explanation will be given.

[0013] Also, as described above, the first state X includes, as an example, a state in which the contrast is smaller than a predetermined value in dark contrast. The contrast is, as an example, the difference in luminance value, the difference in light and dark, or the shade based on the phase difference of the observation light in the image. In the present embodiment, the first state X is, as an example, a state in which the luminance value based on the phase difference is smaller than the luminance value of the background BG of the image P. Further, the tracking area TR includes an area in which the cell C that is the object of tracking is imaged in the image P.

[0014] The tracking area extraction unit 110 extracts the area R based on the luminance indicated by the luminance information. When the tracking area extraction unit 110 extracts, for example, the tracking area TR as the area R, first, it extracts pixels in the image P whose luminance values are smaller than the luminance value of the background BG. The tracking area extraction unit 110 detects a boundary (edge) based on the extracted pixels. The tracking area extraction unit 110 extracts the tracking area TR by determining a continuous curve indicating the boundary of the tracking area TR based on the detected boundary (edge).

[0015] The region R in this embodiment is a region corresponding to one cell C. As described above, the cell C is, as an example, an adherent cell, and the cell C becomes a floating cell during division. The tracking region TR where the luminance state indicated by the luminance information is the first state X is, as an example, a region corresponding to an adherent cell. On the other hand, the tracking stop region SR where the luminance state indicated by the luminance information is the second state Y is, as an example, a region corresponding to a floating cell.

[0016] The position calculation unit 111 calculates the time change TC of the position of the tracking region TR extracted by the tracking region extraction unit 110 based on a plurality of frames of the image P. The position calculation unit 111 calculates the position of the tracking region TR based on the representative point G of the tracking region TR. Here, the representative point G is one point that represents the position of the tracking region TR in the image P among the points included in the tracking region TR. The representative point G is, as an example, the centroid G1 of the tracking region TR including cells. Note that a point includes one pixel of the image P.

[0017] The state determination unit 112 determines whether the luminance state indicated by the luminance information of the tracking region TR extracted by the tracking region extraction unit 110 is the second state Y different from the first state X, that is, whether the luminance state of the tracking region TR that is the object of tracking has changed from the first state X to the second state Y or not. Here, the second state Y is, as an example, a state where the luminance value of a specific part (e.g., a tracking region, etc.) in the image P is larger than a predetermined value. In this embodiment, an example of the second state Y where the luminance value is larger than a predetermined value is an example in the case of dark contrast. The second state Y includes, for example, a state where the luminance value of the specific part with respect to the predetermined value is relatively or absolutely high. Also, the second state Y is, as an example, a state including a state where the contrast is larger than a predetermined value in dark contrast. In this embodiment, the second state Y is, as an example, a state where the luminance value based on the phase difference is larger than the luminance value of the background BG.

[0018] The stop control unit 113 stops the calculation of the time change TC by the position calculation unit 111 based on the determination result of the state determination unit 112. In this case, for example, the stop control unit 113 stops the calculation of the time change TC by the position calculation unit 111 based on the determination result indicating that the luminance state indicated by the luminance information of the tracking area TR has become the second state Y. Here, the area determined by the state determination unit 112 that the luminance state indicated by the luminance information is the second state Y is referred to as the tracking stop area SR.

[0019] The movement state calculation unit 114 calculates the movement state M of the tracking area TR based on the calculation result of the position calculation unit 111 and the determination result of the state determination unit 112. Here, the movement state M includes the degree of movement of the tracking area TR. As an example, the degree of movement includes the movement distance which is the distance that the representative point G of the tracking area TR has moved, the movement speed which is the speed of the representative point G, the movement acceleration which is the acceleration of the representative point G, the average movement speed, and the average movement acceleration, etc. In this way, the state determination unit 112 determines the luminance state (such as the first state, the second state, etc.) using the magnitude relationship of the luminance values in the tracking area TR, and the movement state calculation unit 114 calculates the movement state M of the tracking area TR based on the luminance state determined by the state determination unit 112.

[0020] The output unit 12 outputs the movement state M calculated by the movement state calculation unit 114 to the presentation unit 3 including it in the analysis result A. The storage unit 13 stores various information used by the control unit 11 for processing. Also, the storage unit 13 stores the above-mentioned calculation results, determination results, and / or the movement state M for each cell. In this case, the storage unit 13 is a database that stores and manages the tracking information of the stored cells (for example, the calculation result of the position calculation unit 111, the determination result of the state determination unit 112, and / or the movement state M of the tracking area TR, etc.). The presentation unit 3 presents the analysis result A output from the image processing device 1. The presentation unit 3 is, for example, a display.

[0021] Here, referring to FIGS. 2 to 4, the cell tracking process of the image processing device 1 will be described. FIG. 2 is a diagram showing an example of an image P of image analysis according to the present embodiment. FIG. 3 is an example of image analysis according to the present embodiment. In order to make the image shown in FIG. 2 easier to understand, it is a diagram schematically depicting the image of FIG. 2 corresponding to each figure in FIG. 2 (e.g., FIG. 2(A), etc.). Images P1 to P7 shown in FIGS. 2 and 3 are frames of the image P at times T1 to T7, respectively.

[0022] The tracking area extraction unit 110 extracts a tracking area TR1 as a tracking area TR from the image P1. The luminance value of the tracking area TR1 in the image P1 is smaller than the luminance value of the background BG1 (hereinafter also referred to as the background BG) in the image P1 used as a predetermined value (threshold value), and the luminance of the tracking area TR1 is in the first state X. The position calculation unit 111 calculates a locus TC1 based on a plurality of frames of the image P as the time change TC of the centroid G1 of the tracking area TR1. In FIG. 3(A), the locus TC1 is calculated based on a plurality of frames of the image P before the time T1 including the image P1. In this way, the position calculation unit 111 calculates the tracking area TR1 specified by the tracking area extraction unit 110 using the image P including a plurality of frames (e.g., the image P1, images before the time T1, etc.).

[0023] In the image P2 of FIG. 3(B), the tracking area TR1 in the image P1 changes to the tracking area TR2, and accordingly, the centroid G1 moves to the centroid G2. Also, a locus TC2 is calculated by the position calculation unit 111 as the time change TC.

[0024] In the image P3 of FIG. 3(C), the tracking area TR2 in the image P2 is changing to the tracking area TR3. The luminance value of the tracking area TR3 in the image P3 is larger than the luminance value of the background BG. In this case, the state determination unit 112 determines that the luminance state indicated by the luminance information of the tracking area TR3 is the second state Y. Then, the stop control unit 113 stops the calculation of the time change TC by the position calculation unit 111 based on the determination result that the luminance state indicated by the luminance information of the tracking area TR3 is the second state Y.

[0025] The tracking regions TR4 to TR7 in the images P4 to P7 from FIG. 3(D) to FIG. 3(G) are each a change of the tracking region TR3 in the image P3. The luminance values of the tracking regions TR4 to TR5 are greater than the luminance value of the background BG. The tracking regions TR4 to TR5 are imaged as regions brighter than the background BG in dark contrast. Also, the luminance values of the tracking regions TR6 to TR7 are smaller than the luminance value of the background BG. The tracking regions TR6 to TR7 are imaged as regions darker than the background BG in dark contrast.

[0026] FIG. 4 is a diagram showing an example of the tracking process of the cell C according to the present embodiment. The cells C1 and C2 correspond to the tracking regions TR1 in the image P1 and the tracking region TR2 in the image P2 in FIG. 3, for example, and tracking is performed from the first frame to the second frame, which is the first frame when the tracking of the cell C is started. The cell C3 corresponds to the tracking region TR3 in the image P3 in FIG. 3, for example, and is a cell floating immediately before cell division. Here, floating cells appear during the division of adherent cells. The floating cells are imaged as regions brighter than the background BG in the image P. In the cell tracking process according to the present embodiment, tracking is stopped for cells determined to be floating.

[0027] The cells C41 and C42 correspond to the tracking region TR6 in the image P6 in FIG. 3, for example, and are cells divided from the cell C3. For the cells C41 and C42, the tracking process of the present embodiment is newly started independently. Tracking is performed from the first frame to the second frame for the cells C41 and C51, which are the first frame and the second frame after the new tracking process is started. On the other hand, tracking is performed from the first frame to the second frame for the cells C42 and C52, which are the first frame and the second frame after the new tracking process is started. In the cell tracking process according to the present embodiment, cell movement and cell division are distinguished in order to improve the accuracy of analyzing the movement of one cell.

[0028] Here, for comparison, refer to FIG. 18 and explain the conventional cell tracking process. FIG. 18 is a diagram showing an example of the conventional cell tracking process. Cells C11, C12, C13, C141, C151, C142, and C152 are the same cells as cells C1, C2, C3, C41, C51, C42, and C52 in FIG. 4, respectively.

[0029] As shown in FIG. 18, in the conventional cell tracking process, cells C11, C12, C13, C141, and C151 are being tracked from the 1st frame to the 5th frame. In the conventional cell tracking process, for cell C13, which is a cell floating immediately before division, continuous tracking is performed without being distinguished from cells C11 and C12, which are in a cell movement state.

[0030] Furthermore, for cells C141 and C151, which are cells after division, continuous tracking is performed from cells C11, C12, and C13. Also, for cells C142 and C152, which are cells after division, new tracking processes are started.

[0031] As described above, in the conventional cell tracking process, cell movement and cell division are not distinguished, and tracking continues even though the cell is dividing, or the tracking that should continue is interrupted. The tracking results vary depending on the case, and the analysis conditions cannot be unified.

[0032] Next, refer to FIGS. 5 and 6 and explain the image processing of the image processing apparatus 1. FIG. 5 is a diagram showing an example of the image processing according to the present embodiment. Step S10: The image acquisition unit 10 acquires the image P output from the microscope 2. The image acquisition unit 10 supplies the acquired image P to the control unit 11. Assume that the image P consists of n frames from the first to the nth frame obtained by continuous shooting of a moving image or still images (e.g., time-lapse shooting) of a sample (e.g., cells).

[0033] Step S20: The control unit 11 executes a movement state calculation process. For example, the movement state calculation process is a process of calculating the movement state M of the cell C imaged in the image P. The movement state calculation process is executed for each frame of the image P (e.g., the first frame, the second frame, etc.). The control unit 11 performs the movement state calculation process on one frame in one execution of the movement state calculation process in step S20. In the first execution of the movement state calculation process in step S20, the control unit 11 targets the first frame of the image P.

[0034] In the following description, the frame that is the target of the current movement state calculation process is referred to as the i-th frame. The frame that was the target of the movement state calculation process immediately before the current movement state calculation process is referred to as the (i - 1)-th frame, etc.

[0035] Here, referring to FIG. 6, the movement state calculation process will be described. FIG. 6 is a diagram showing an example of the movement state calculation process according to the present embodiment. Steps S110 to S160 in FIG. 6 are executed as step S20 in FIG. 5.

[0036] Step S110: The tracking area extraction unit 110 extracts, as the tracking area TR, an area R in the image P where the luminance value is smaller than the luminance value of the background BG of the image P. For example, the tracking area extraction unit 110 extracts a tracking area TR in the image P where the luminance state indicated by the luminance information is the first state X.

[0037] Further, the tracking area extraction unit 110 extracts, from the image P, an area where the luminance value is greater than the luminance value of the background BG of the image P (second state Y) as the tracking stop area SR. Here, the tracking area extraction unit 110 extracts one tracking area TR and one tracking stop area SR. Note that the tracking area extraction unit 110 may extract a plurality of tracking areas TR and a plurality of tracking stop areas SR based on the luminance values described above. Note that when neither an area R where the luminance value is greater than the luminance value of the background BG nor an area R where the luminance value is less than the luminance value of the background BG exists in the image P, the tracking area extraction unit 110 does not have to extract any area.

[0038] Since the image P is a phase difference image captured in dark contrast, the tracking area TR is an area having a lower luminance than the background BG, and the tracking stop area SR is an area having a higher luminance than the background BG.

[0039] Step S120: The position calculation unit 111 extracts the centroid G1 of the tracking area TR as the representative point G of the tracking area TR extracted by the tracking area extraction unit 110. When there are a plurality of tracking areas TR, the position calculation unit 111 extracts the centroid G1 for each of the plurality of tracking areas TR. The position calculation unit 111 stores the centroid position information GI1, which is information indicating the position of the extracted centroid G1, in the storage unit 13.

[0040] Note that the position calculation unit 111 may extract, as the representative point G, a point selected from the tracking area TR based on a predetermined criterion instead of the centroid G1. The predetermined criterion is, for example, that the luminance value in the tracking area TR is greater than or less than a predetermined value. Further, the position calculation unit 111 may extract an arbitrary point inside the tracking area TR as the representative point G based on a predetermined position or an input instruction from the user.

[0041] Step S130: The position calculation unit 111 calculates the time change TC of the centroid G1 of the tracking region TR extracted in step S120 based on a plurality of frames of the image P. As an example of the time change TC of the centroid G1, the position calculation unit 111 calculates the trajectory TC1 of the centroid G1. When there are a plurality of tracking regions TR, the position calculation unit 111 calculates the time change TC of the centroid G1 for each of the plurality of tracking regions TR based on a plurality of frames of the image P.

[0042] Here, the centroid G1 extracted in the current i-th frame is defined as the centroid G1i, and the centroid G1 extracted in the (i - 1)-th frame is defined as the centroid G1i-1. The position calculation unit 111 calculates the trajectory TC1 of the centroid G1 by connecting the centroid G1i and the centroid G1i-1 with a straight line or a curve. That is, the position calculation unit 111 calculates the trajectory TC1 by calculating the centroid G1 of the tracking region TR extracted by the tracking region extraction unit 110 for each of the plurality of frames of the image P.

[0043] When a plurality of tracking regions TR are extracted in step S110, the position calculation unit 111 associates the plurality of tracking regions TRi extracted in the current i-th frame with the plurality of tracking regions TRi-1 extracted in the (i - 1)-th frame. Hereinafter, this association process is referred to as the region association process. For example, the position calculation unit 111 associates the k-th tracking region TR, which is the tracking region TRi-1,k among the plurality of tracking regions TRi-1 extracted in the (i - 1)-th frame, with the j-th tracking region TR, which is the tracking region TRi,j among the plurality of tracking regions TRi extracted in the i-th frame, by the region association process.

[0044] Here, an image processing technique for associating a plurality of regions within a frame between different frames is used for the region association process. In the region association process, as an example, the tracking regions TR that are closest to each other between different frames (for example, regions with a small distance between the tracking regions TR) are associated with each other. The position calculation unit 111 selects, from among the tracking regions TRi extracted in the i-th frame, the region corresponding to the tracking region TRi-1,k extracted in the (i-1)-th frame. The position calculation unit 111 executes this selection process for all of the tracking regions TRi-1 extracted in the (i-1)-th frame.

[0045] If the number of the plurality of tracking regions TRi extracted in the i-th frame is greater than the number of the plurality of tracking regions TRi-1 extracted in the (i-1)-th frame, there exists a tracking region TRi among the plurality of tracking regions TRi that is not associated with any of the plurality of tracking regions TRi-1 extracted in the (i-1)-th frame. This tracking region TRi that is not associated with any of the plurality of tracking regions TRi-1 is designated as the tracking region TRi,u. The position calculation unit 111 sets the starting point of the trajectory TC1 of the tracking region TRi,u to the center of gravity G1 of the tracking region TRi,u.

[0046] For calculating the distance between the tracking regions TR, for example, the position of the tracking region TR is used. In the region association process, as an example, the center of gravity G1 calculated in step S120 is used as the position of the tracking region TR.

[0047] Before starting the process of step S130 here, the position calculation unit 111 reads the center of gravity position information GI1 from the storage unit 13, and based on the read center of gravity position information GI1, performs the process of calculating the above-described trajectory TC1. Further, the position calculation unit 111 stores the calculated trajectory TC1 in the storage unit 13.

[0048] Step S140: The state determination unit 112 determines that the luminance value of the tracking region TR extracted by the tracking region extraction unit 110 is greater than the luminance value of the background BG of the image P. When there are a plurality of tracking regions TR, the state determination unit 112 determines for each of the plurality of tracking regions TR that the luminance value of the tracking region TR extracted by the tracking region extraction unit 110 is greater than the luminance value of the background BG of the image P.

[0049] Here, the tracking region TR extracted in the current i-th frame is defined as the tracking region TRi. The tracking stop region SR extracted in the current i-th frame is defined as the tracking stop region SRi. The tracking region TR extracted in the (i - 1)-th frame is defined as the tracking region TRi-1. The tracking stop region SR extracted in the (i - 1)-th frame is defined as the tracking stop region SRi-1.

[0050] The state determination unit 112 associates the tracking stop region SRi extracted in the i-th frame with the tracking region TRi-1 or the tracking stop region SRi-1 extracted in the (i - 1)-th frame by the above-described region association process. Here, the state determination unit 112 associates the tracking stop region SRi with the tracking region TRi-1 or the tracking stop region SRi-1 based on the calculated trajectory TC1.

[0051] For example, when the position of the centroid G1i of the tracking stop region SRi and the position of the centroid G1i-1 of the tracking region TRi-1 or the tracking stop region SRi-1 are included in the same trajectory TC1, the state determination unit 112 associates the tracking stop region SRi with the tracking region TRi-1 or the tracking stop region SRi-1. The associated regions correspond to the same cell. Note that for the centroid G1 of the tracking stop region SR, the state determination unit 112 causes the position calculation unit 111 to calculate it in step S140.

[0052] When the state determination unit 112 associates the tracking region TRi-1 in the (i - 1)-th frame with the tracking stop region SRi in the i-th frame, it determines that the tracking region TRi-1 has changed to the tracking stop region SRi. That is, the state determination unit 112 determines that the luminance value of the tracking region TRi-1 in the (i - 1)-th frame has changed to a state where it is greater than the luminance value of the background BG of the image P in the i-th frame.

[0053] Step S150: The stop control unit 113 stops the position calculation unit 111 from calculating the time change TC based on the determination result of the state determination unit 112, and ends the tracking of the cell C being tracked. That is, the position calculation unit 111 stops calculating the time change TC for the tracking region TR where the luminance state indicated by the luminance information is greater than the luminance value of the background BG of the image P.

[0054] Note that the stop control unit 113 may stop the position calculation unit 111 from calculating the time change TC based on the combination of the determination result of the state determination unit 112 and any one or more of the following auxiliary conditions. The first auxiliary condition is, for example, that the area of the tracking region TR is equal to or less than a predetermined value. The second auxiliary condition is, for example, that the distance in the image P between the tracking region TRi in the i-th frame and the tracking region TRi-1 in the (i - 1)-th frame is equal to or greater than a predetermined value. The third auxiliary condition is, for example, that the number of frames used to calculate the trajectory TC1 is equal to or less than a predetermined value. In the third auxiliary condition, the predetermined value for the number of frames is, for example, 2 frames.

[0055] Step S160: The movement state calculation unit 114 calculates the movement state M of the tracking region TR based on the calculation result of the position calculation unit 111. Here, since the calculation result of the position calculation unit 111 is the result until the calculation of the time change TC is stopped by the stop control unit 113 based on the determination result of the state determination unit 112, it is based on the determination result of the state determination unit 112. The movement state calculation unit 114 calculates, as the movement state M, for example, the movement distance of the center of gravity G1 and the movement speed of the center of gravity G1. The movement state calculation unit 114 supplies the calculated movement state M to the output unit 12. Thus, the control unit 11 ends the movement state calculation process of the tracking region TR including the cell C.

[0056] Next, returning to FIG. 5, the description of the image processing of the image processing apparatus 1 will be continued. Step S30: The control unit 11 determines whether the end condition is satisfied. Here, the end condition is a condition for ending the repetition of the movement state calculation process in step S20. As an example, the end condition is that the movement state calculation process is executed for a predetermined number of frames of the image P. The predetermined number is, for example, the number of all frames (n frames) constituting the image P. The predetermined number may be a number less than the number of all frames constituting the image P.

[0057] When the control unit 11 determines that the end condition is satisfied (step S30; YES), it executes the process of step S40. On the other hand, when the control unit 11 determines that the end condition is not satisfied (step S30; NO), it changes the frame to be the target of the movement state calculation process among the frames of the image P from the current frame to the next frame. For example, if the control unit 11 has used the i-th frame as the target of the process in the immediately preceding movement state calculation process, it changes the target of the next process to the (i + 1)-th frame. Then, the control unit 11 returns to step S20 and executes the movement state calculation process again.

[0058] Step S40: The output unit 12 outputs an analysis result A including at least one of the moving distance, which is the distance the representative point G of the tracking region TR has moved, the moving speed, which is the speed of the representative point G, the moving acceleration, which is the acceleration of the representative point G, the average moving speed, and the average moving acceleration, to the presentation unit 3. Here, the output unit 12 includes the movement state M calculated by the movement state calculation unit 114 in the analysis result A and outputs it to the presentation unit 3. Thus, the image processing apparatus 1 ends the image processing (movement state calculation process).

[0059] As described above, when the end condition is not satisfied in step S30, the control unit 11 returns to step S20 and executes the movement state calculation process again, so it executes the process of extracting the tracking region TR in step S110 and the process of calculating the time change TC in step S130 again.

[0060] That is, after the stop control unit 113 causes the position calculation unit 111 to stop calculating the time change TC, the tracking area extraction unit 110 extracts the tracking area TR in the first state X again from a plurality of frames of the image P. After the stop control unit 113 causes the position calculation unit 111 to stop calculating the time change TC, the position calculation unit 111 calculates the time change TC of the position of the tracking area TR re-extracted by the tracking area extraction unit 110 based on a plurality of images P.

[0061] In the movement state calculation process of FIG. 6, the position calculation unit 111 stops calculating the time change TC for the tracking area TR in a state where the luminance state indicated by the luminance information is greater than the luminance value of the background BG of the image P. Therefore, the center of gravity G1 of the tracking area TR in a state where the luminance state indicated by the luminance information is greater than the luminance value of the background BG of the image P is excluded from the time change TC calculated by the position calculation unit 111. That is, the position calculation unit 111 excludes the center of gravity G1 when the luminance state indicated by the luminance information of the tracking area TR is in the second state Y and calculates the time change TC. In the image processing apparatus 1 according to the present embodiment, in the analysis of the movement state M of the cell C, since the time when the cell C is not moving can be excluded by the movement state calculation process, the accuracy of the analysis of the movement state M of the cell C can be improved as compared with the case where the time when the cell C is not moving is not excluded. Here, the state where the cell C is not moving includes the state where the cell C is floating immediately before cell division.

[0062] Here, referring to FIGS. 7 to 9, the analysis result A when the movement speeds of a plurality of cells C imaged in the image P are analyzed using the image processing of the above-described image processing apparatus 1 will be described. In FIGS. 7 to 9, as an example of the analysis result A, using an index related to the migration ability of the cell C, the influence of the culture environment such as a coating agent and a nutrient factor contained in the medium on the migration ability is evaluated.

[0063] FIG. 7 is a diagram showing an example of a histogram of the moving speed of cell C according to the present embodiment. In FIG. 7, as analysis result A, the distributions of a plurality of cell Cs with respect to the moving speed are shown as histogram H1a, histogram H1b, and histogram H1c. In this histogram, the number Z of a plurality of cell Cs is shown for each moving speed.

[0064] FIG. 7(A) shows histogram H1a of the moving speed of a plurality of cell Cs under reference culture conditions. FIG. 7(B) shows histogram H1b of the moving speed of a plurality of cell Cs cultured under the first culture conditions changed from the reference culture conditions. FIG. 7(C) shows histogram H1c of the moving speed of a plurality of cell Cs cultured under the second culture conditions changed from the reference culture conditions.

[0065] FIG. 8 is a diagram showing an example of the evaluation result of the ratio of cell Cs with a high moving speed according to the present embodiment. In FIG. 8, as analysis result A, for each of histogram H1a, histogram H1b, and histogram H1c in FIG. 7, the ratio of cell Cs with a moving speed greater than a predetermined value (e.g., a threshold value serving as an evaluation index) is shown as ratio RAa, ratio RAb, and ratio RAc.

[0066] FIG. 9 is a diagram showing an example of the evaluation result of the variation in the moving speed of cell C according to the present embodiment. In FIG. 9, as analysis result A, for each of histogram H1a, histogram H1b, and histogram H1c in FIG. 7, the variation DSa, variation DSb, and variation DSc of the moving speed are shown.

[0067] In the present embodiment, although an example in which the microscope 2 captures an image P, which is a phase contrast image, in dark contrast has been described, the present invention is not limited to this. The microscope 2 may capture the image P in bright contrast. When the image P is captured in bright contrast, for example, the first state X is a state in which the luminance value is greater than a predetermined value, and the second state Y is a state in which the luminance value is less than a predetermined value.

[0068] In this embodiment, an example has been described in which one type of predetermined value (e.g., the luminance value of the background BG) is used to determine the first state X and the second state Y, but the present invention is not limited to this. Two types of predetermined values may be used to determine the first state X and the second state Y. When two types of predetermined values are used, in an example of dark contrast, the first state X is, for example, a state where the luminance value is smaller than the first predetermined value, and the second state Y is, for example, a state where the luminance value is larger than the second predetermined value. Here, the first predetermined value is smaller than the second predetermined value. The first predetermined value is, for example, the luminance value of the background BG. The second predetermined value is, for example, a luminance value preset by a user or the like or a luminance value preset based on prior experimental results or the like. Note that when two types of predetermined values are used, if the luminance value of the region R is an intermediate value between the first predetermined value and the second predetermined value, the region R is, for example, not determined to be in either the first state X or the second state Y, or is determined to be in the first state X.

[0069] In this embodiment, an example has been described in which, when the first state X or the second state Y is determined, the average value of the luminance values obtained from the entire background BG of the first frame of the image P is commonly used for all the frames included in the image P, but the present invention is not limited to this. As described above, as the predetermined value commonly used for all the frames included in the image P, the luminance value obtained from a specific part of the background BG may be used. Also, as the predetermined value commonly used for all the frames included in the image P, the luminance value obtained from the background BG of a frame other than the first frame of the image P (e.g., the i-th frame other than the first frame or the last frame) may be used. As the predetermined value commonly used for all the frames included in the image P, the average value of the luminance values obtained from the background BG of one or more frames included in the image P may be commonly used for all the frames included in the image P. Also, the predetermined value commonly used for all frames included in the image P may be a value preset by a user or the like. The preset value may be set without depending on the image P, or may be set based on prior experimental results or the like.

[0070] Also, the predetermined value may be different for each frame included in the image P. For example, as the predetermined value different for each frame included in the image P, the luminance value obtained from the background BG of each frame of the image P may be used for each frame. Also, for example, as the predetermined value different for each frame included in the image P, a plurality of preset values may be used for each frame.

[0071] As described above, the image processing apparatus 1 according to the present embodiment includes a region extraction unit (in this example, the tracking region extraction unit 110), a position calculation unit 111, a state determination unit 112, and a movement state calculation unit 114. The region extraction unit (in this example, the tracking region extraction unit 110) extracts a tracking region TR in which the state of the luminance (in this example, the luminance value) indicated by the luminance information is in the first state X (in this example, the state where the luminance value is smaller than the luminance value of the background BG of the image P in dark contrast) from a plurality of images (in this example, a plurality of frames of the image P) in which cells C are respectively detected at a plurality of times. The position calculation unit 111 calculates the time change TC of the position of the tracking region TR extracted by the region extraction unit (in this example, the tracking region extraction unit 110) based on a plurality of images (in this example, a plurality of frames of the image P). The state determination unit 112 determines that the state of the luminance (in this example, the luminance value) indicated by the luminance information of the tracking region TR extracted by the region extraction unit (in this example, the tracking region extraction unit 110) is a second state Y (in this example, the state where the luminance value is larger than the luminance value of the background BG of the image P in dark contrast) different from the first state X. The movement state calculation unit 114 calculates the movement state M (in this example, the movement speed) of the tracking area TR based on the calculation result of the position calculation unit 111 and the determination result of the state determination unit 112. Here, the calculation result of the position calculation unit 111 includes, as an example, the result of calculating the time change TC of the center of gravity G1 of the tracking area TR. Also, the determination result of the state determination unit 112 includes, as an example, the result of determining that the luminance value of the tracking area TR is in a state where it is greater than the luminance value of the background BG.

[0072] With this configuration, the image processing apparatus 1 according to the present embodiment can improve the accuracy of cell tracking by performing tracking of a single cell using the determination result of whether or not a single cell is floating immediately before cell division. Therefore, the accuracy of analyzing the movement state M of the cell can be improved as compared with the case where it is not based on the determination result of whether or not the cell is floating immediately before cell division.

[0073] Also, in the image processing apparatus 1 according to the present embodiment, the position calculation unit 111 calculates the time change TC by calculating the position (in this example, the center of gravity G1) of the tracking area TR extracted by the area extraction unit (in this example, the tracking area extraction unit 110) for each of a plurality of images (in this example, a plurality of frames of the image P). With this configuration, in the image processing apparatus 1 according to the present embodiment, since the analysis of the movement state M of the cell can be executed based on the time change TC, the accuracy of analyzing the movement state M of the cell can be improved as compared with the case where it is not based on the time change TC.

[0074] Also, in the image processing apparatus 1 according to the present embodiment, the position calculation unit 111 excludes the position (in this example, the center of gravity G1) of the tracking area TR when the state of the luminance (in this example, the luminance value) indicated by the luminance information of the tracking area TR is in the second state Y (in this example, the state where the luminance value is greater than the luminance value of the background BG of the image P in dark contrast) and calculates the time change TC. With this configuration, in the image processing apparatus 1 according to the present embodiment, since cells floating immediately before cell division can be excluded from the analysis target of the movement state M and tracked, the accuracy of the analysis of the movement state M of cells can be improved compared to the case where cells floating immediately before cell division are not excluded.

[0075] When cells divide, the movement of the center of gravity of the cells occurs instantaneously. Therefore, if tracking is performed on cells during cell division in the analysis of migration ability by tracking processing, the average speed may be overestimated, which can become noise in the analysis result of the migration ability analysis. Also, as described above, in the conventional cell tracking processing, the analysis conditions cannot be unified, and there may be variations in the numerical values of the results of the migration ability analysis. In the image processing apparatus 1 according to the present embodiment, by distinguishing the movement distance of cells from division in the tracking processing, noise with respect to the analysis result of the migration ability analysis can be removed, so that the accuracy of the analysis of the movement state M of cells can be improved.

[0076] Also, in the image processing apparatus 1 according to the present embodiment, the movement state M includes the mobility of the tracking region TR. With this configuration, in the image processing apparatus 1 according to the present embodiment, since the mobility of the tracking region TR can be calculated, the accuracy of the analysis of the mobility of cells can be improved compared to the case where it is not based on the determination result of whether the cells are floating immediately before cell division.

[0077] Also, in the image processing apparatus 1 according to the present embodiment, the movement state M includes the mobility (in this example, the movement speed) of the tracking region TR. With this configuration, in the image processing apparatus 1 according to the present embodiment, since the accuracy of tracking a plurality of cells captured in the image P can be improved, the accuracy of the analysis of the movement state M of the plurality of cells can be improved compared to the case where it is not based on the determination result of whether the cells are floating immediately before cell division.

[0078] Also, for example, in the image processing apparatus 1 according to the present embodiment, when the observation method (e.g., phase difference observation method, differential interference observation method) is the phase difference observation method and the observation condition is dark contrast, the first state X is a state where the luminance (in this example, the luminance value) indicated by the luminance information is smaller than a predetermined value (in this example, the luminance value of the background BG), and the second state Y is a state where the luminance (in this example, the luminance value) indicated by the luminance information is larger than a predetermined value (in this example, the luminance value of the background BG). For example, when the observation method is the phase difference observation method and the observation condition is bright contrast, the first state X is a state where the luminance (in this example, the luminance value) indicated by the luminance information is larger than a predetermined value (in this example, the luminance value of the background BG), and the second state Y is a state where the luminance (in this example, the luminance value) indicated by the luminance information is smaller than a predetermined value (in this example, the luminance value of the background BG). With this configuration, in the image processing apparatus 1 according to the present embodiment, since the tracking of cells can be performed based on the magnitude relationship between the luminance (in this example, the luminance value) indicated by the luminance information and a predetermined value (in this example, the luminance value of the background BG), the accuracy of analyzing the movement state M of the cells can be improved as compared with the case where the tracking of cells is not performed based on the magnitude relationship between the luminance (in this example, the luminance value) indicated by the luminance information and a predetermined value (in this example, the luminance value of the background BG).

[0079] Also, in the image processing apparatus 1 according to the present embodiment, the predetermined value includes the luminance (in this example, the luminance value) indicated by the luminance information of the background BG of the image P. With this configuration, in the image processing apparatus 1 according to the present embodiment, there is no need to set a predetermined value in the execution of the above-described image processing. When the above-described image processing is executed using a plurality of image processing apparatuses 1, since it is not necessary to set a predetermined value for each of the plurality of image processing apparatuses 1, the versatility is improved in cases such as sharing or integrating the analysis results A among the plurality of image processing apparatuses 1.

[0080] Also, in the image processing apparatus 1 according to the present embodiment, the position calculation unit 111 calculates the position of the tracking region TR based on the representative point G of the tracking region TR. With this configuration, in the image processing apparatus 1 according to the present embodiment, since cell tracking can be executed based on the representative point G of the tracking region TR, the accuracy of analyzing the movement state M of the cells can be improved compared to the case where it is not based on the representative point G. When cell tracking is performed without relying on the representative point G of the tracking region TR, for example, it is conceivable to select an appropriate point in the tracking region TR for each frame and calculate a trajectory based on the selected point. In this case, the calculated trajectory may be more finely bent than the actual cell trajectory.

[0081] Also, in the image processing apparatus 1 according to the present embodiment, the representative point G is the center of gravity G1 of the tracking region TR. With this configuration, in the image processing apparatus 1 according to the present embodiment, since the tracking process can be executed based on the center of gravity of the cells, the movement state M of the cells can be analyzed based on the center of gravity of the cells.

[0082] Further, the image processing apparatus 1 according to the present embodiment includes a region extraction unit (in this example, a tracking region extraction unit 110), a position calculation unit 111, a state determination unit 112, and a stop control unit 113. The stop control unit 113 stops the calculation of the time change TC by the position calculation unit 111 based on the determination result of the state determination unit 112. With this configuration, in the image processing apparatus 1 according to the present embodiment, based on the determination result of whether the state of the luminance (in this example, the luminance value) indicated by the luminance information of the tracking region TR is the second state Y (in this example, the state where the luminance value is greater than the luminance value of the background BG of the image P in dark contrast), the tracking of the dividing cells can be stopped. Therefore, the accuracy of analyzing the movement state M of the cells can be improved compared to the case where the tracking of the dividing cells is not stopped.

[0083] Also, in the image processing apparatus 1 according to the present embodiment, the region extraction unit (in this example, the tracking region extraction unit 110) extracts the tracking region TR in the first state X (in this example, a state where the luminance value is smaller than the luminance value of the background BG of the image P in dark contrast) again from a plurality of images (in this example, a plurality of frames of the image P) after the stop control unit 113 causes the position calculation unit 111 to stop calculating the time change TC. The position calculation unit 111 calculates the time change TC of the position of the tracking region TR extracted again by the region extraction unit (in this example, the tracking region extraction unit 110) based on a plurality of images (in this example, a plurality of frames of the image P) after the stop control unit 113 causes the position calculation unit 111 to stop calculating the time change TC. With this configuration, in the image processing apparatus 1 according to the present embodiment, tracking processing can be restarted for the cells for which tracking has been stopped, so the accuracy of analyzing the movement state M of the cells can be increased compared to the case where tracking processing is not restarted. Further, the image processing apparatus 1 according to the present embodiment using the above-described image processing can start tracking new cells after division by extracting the tracking region TR in the first state X again after cell division.

[0084] (Second Embodiment) Hereinafter, the second embodiment will be described in detail with reference to the drawings. In the above first embodiment, the case where the image processing apparatus calculates the movement state of the tracking region in the first state was described. In the present embodiment, the case where the image processing apparatus measures the time when the luminance state indicated by the luminance information of the tracking region is in the second state will be described. The image processing apparatus according to the present embodiment is referred to as the image processing apparatus 1a.

[0085] FIG. 10 is a diagram showing an example of the configuration of the image processing apparatus 1a according to the present embodiment. Comparing the image processing apparatus 1a (FIG. 10) according to the present embodiment with the image processing apparatus 1 (FIG. 1) according to the first embodiment, the control unit 11a is different. Here, the functions of the other components (image acquisition unit 10, output unit 12, and storage unit 13) are the same as those in the first embodiment. The description of the same functions as those in the first embodiment is omitted, and in the second embodiment, the description will focus on the parts different from the first embodiment.

[0086] The control unit 11a includes a tracking area extraction unit 110, a position calculation unit 111, a state determination unit 112, a stop control unit 113, a movement state calculation unit 114, and a splitting time measurement unit (time measurement unit) 115a. The functions of the tracking area extraction unit 110, the position calculation unit 111, the state determination unit 112, the stop control unit 113, and the movement state calculation unit 114 are the same as those in the first embodiment.

[0087] The splitting time measurement unit 115a measures the floating time TD based on the determination result of the state determination unit 112. Here, the floating time TD includes the length of time during which the luminance state indicated by the luminance information of the tracking area TR is in the second state Y. As an example, the splitting time measurement unit 115a measures the floating time TD based on the number of frames of the image P determined to be in the second state Y. Here, as an example, the splitting time measurement unit 115a measures the floating time TD by converting the number of frames of the image P including the tracking area TR with a high luminance value based on the frame rate interval.

[0088] Next, referring to FIG. 11, the movement state calculation process involving the measurement of the floating time TD will be described. FIG. 11 is a diagram showing an example of the movement state calculation process according to the present embodiment. The movement state calculation process in FIG. 11 is executed in step S20 of the image processing in FIG. 5. Note that since the processes in step S210, step S220, step S230, step S240, step S260, and step S270 are the same as the processes in step S110, step S120, step S130, step S140, step S150, and step S160 in FIG. 6, the description thereof will be omitted.

[0089] Step S250: The floating time TD is measured by the division time measurement unit 115a based on the determination result of the state determination unit 112. Here, when it is determined that the luminance value of the tracking region TR extracted by the tracking region extraction unit 110 is greater than the luminance value of the background BG of the image P, the floating time TD of the tracking region TR is measured using a plurality of frames.

[0090] As described in step S140 of FIG. 6, when the state determination unit 112 associates the tracking region TRi-1 in the (i-1)th frame with the tracking stop region SRi in the ith frame, it is determined that the tracking region TRi-1 has changed to the tracking stop region SRi. The division time measurement unit 115a starts the above-described region association process (in this case, also referred to as the division cell association process) for the tracking stop region SR based on the determination result of the state determination unit 112.

[0091] The division time measurement unit 115a associates the tracking stop region SRi in the ith frame with the tracking stop region SRi+1 in the (i+1)th frame by the region association process. The division time measurement unit 115a ends the region association process when there is no tracking stop region SRi+2 in the (i+2)th frame associated with the tracking stop region SRi+1.

[0092] The division time measurement unit 115a measures, as the floating time TD, the number of frames in which the tracking stop regions SRi associated with each other in the region association process are extracted.

[0093] Referring now to FIG. 12, an example of measuring the floating time TD will be described. FIG. 12 is a diagram showing an example of an image used for measuring the floating time TD according to the present embodiment. FIG. 13 is a diagram schematically depicting the image shown in FIG. 12 for easier understanding. Similar to the first embodiment, the image P of the present embodiment is a plurality of images (e.g., image Pi-1, image Pi) in which cells C are detected at a plurality of times. Each of the images Pi-1, Pi, Pi+1, and Pi+2 shown in FIGS. 12 and 13 is the (i-1)th frame, the ith frame, the (i+1)th frame, and the (i+2)th frame in the image P. The tracking region extraction unit 110 extracts a tracking region TRi-1, a tracking stop region SRi, a tracking stop region SRi+1, and a tracking region TRi+2 from each of the images Pi-1, Pi, Pi+1, and Pi+2.

[0094] The state determination unit 112 associates the tracking region TRi-1 with the tracking stop region SRi and determines that the tracking region TRi-1 has changed to the tracking stop region SRi. The division time measurement unit 115a starts the region association process based on the determination result of the state determination unit 112. The division time measurement unit 115a associates the tracking stop region SRi with the tracking stop region SRi+1. Since the tracking stop region SR is not extracted from the image Pi+2, the division time measurement unit 115a ends the region association process.

[0095] The division time measurement unit 115a measures the number of frames in which the tracking stop region SRi and the tracking stop region SRi+1 associated in the region association process are extracted. The state determination unit 112 measures 2, which is the number of the images Pi and Pi+1, as the floating time TD. Note that the division time measurement unit 115a may convert the measured floating time TD from the number of frames (e.g., 2, which is the number of the images Pi and Pi+1) to time based on the frame interval in the imaging of the image P.

[0096] Note that in step S40 shown in FIG. 5, the output unit 12 outputs the analysis result Aa based on the floating time TD to the presentation unit 3. Further, the output unit 12 may output to the presentation unit 3 an analysis result including the movement state M of the tracking region TR and the floating time TD. Here, the floating time TD is considered to correspond to the length of the M phase (mitosis phase) in the cell cycle. The division time measurement unit 115a estimates and measures the length of the M phase of the cell cycle by tracking floating cells that appear during the division of adherent cells. The control unit 11a can indirectly measure the period of the M-phase checkpoint by measuring the length of the M phase, and evaluate the influence on the cells.

[0097] Here, referring to FIG. 14, a specific example of the analysis result Aa of the floating time will be described. FIG. 14 is a diagram showing an example of a histogram of the floating time TD according to the present embodiment. In FIG. 14, as the analysis result Aa, the distributions of a plurality of cells C with respect to the floating time TD are shown as a histogram H2a and a histogram H2b. In these histograms, the number Z of a plurality of cells C is shown for each floating time TD.

[0098] FIG. 14(A) shows a histogram H2a of the floating time TD of a plurality of cells C under reference culture conditions. FIG. 14(B) shows a histogram H2b of the floating time TD of a plurality of cells C cultured under the first culture conditions changed from the reference culture conditions. The peak PKb of the histogram H2b is at a position where the floating time TD is longer than the peak PKa of the histogram H2a, and in the first culture conditions, the floating time TD tends to be longer than in the reference culture conditions.

[0099] For example, in the first culture conditions, the period during which the cells C are in a floating state is longer than in the reference culture conditions, and there is a possibility that the M-phase checkpoint is not operating normally compared to the reference culture conditions. Therefore, the image processing apparatus 1a according to the present embodiment can evaluate the influence of canceration and the like based on the above analysis result Aa.

[0100] As described above, the image processing apparatus 1a according to the present embodiment includes a time measurement unit (in this example, the division time measurement unit 115a). The time measurement unit (in this example, the division time measurement unit 115a) measures the time (floating time TD) when the luminance state indicated by the luminance information of the tracking region TR is the second state Y based on the determination result of the state determination unit 112. With this configuration, in the image processing apparatus 1a according to the present embodiment, since the length of the M phase in the cell cycle can be estimated and measured using the floating time TD, the state of the cell can be analyzed based on the length of the M phase.

[0101] (Third Embodiment) Hereinafter, the third embodiment will be described in detail with reference to the drawings. In the present embodiment, a case will be described in which the image processing apparatus associates the region before division with the plurality of regions after division when the region in the image is divided into a plurality of regions. The image processing apparatus according to the present embodiment is referred to as an image processing apparatus 1b.

[0102] FIG. 15 is a diagram showing an example of the configuration of the image processing apparatus 1b according to the present embodiment. Comparing the image processing apparatus 1b (FIG. 15) according to the present embodiment with the image processing apparatus 1 (FIG. 1) according to the first embodiment, the control unit 11b is different. Here, the functions of the other components (image acquisition unit 10, output unit 12, and storage unit 13) are the same as those in the first embodiment. The description of the same functions as those in the first embodiment will be omitted, and in the third embodiment, the description will focus on the parts different from the first embodiment.

[0103] The control unit 11b includes a tracking region extraction unit 110, a position calculation unit 111, a state determination unit 112, a stop control unit 113, a movement state calculation unit 114, and a labeling unit 116b. The functions of the tracking region extraction unit 110, the position calculation unit 111, the state determination unit 112, the stop control unit 113, and the movement state calculation unit 114 are the same as those in the first embodiment.

[0104] The labeling unit 116b assigns a label (e.g., group number, identifier, etc.) L that identifies the associated regions R to the associated regions R between the different frames described above. Here, the regions R to which the labeling unit 116b assigns the label L include the tracking region TR and the tracking stop region SR.

[0105] Here, referring to FIG. 16, the movement state calculation process involving the labeling process, which is the process of assigning the label L to the region R, will be described. FIG. 16 is a diagram showing an example of the movement state calculation process according to the present embodiment. The movement state calculation process in FIG. 16 is executed in step S20 of the image processing in FIG. 5. Note that since the processes in step S310, step S320, step S330, step S360, and step S370 are the same as the processes in step S110, step S120, step S130, step S150, and step S160 in FIG. 6, the description thereof will be omitted.

[0106] Step S340: The labeling unit 116b performs a labeling process of assigning the label L to the associated region R. The labeling unit 116b associates the region Ri-1 extracted in the (i - 1)-th frame with the region Ri extracted in the i-th frame by the above-described region association process.

[0107] Here, referring to FIG. 17, the details of the region association process will be described. FIG. 17 is a diagram showing an example of the region association process according to the present embodiment. In FIG. 17, the image Pi-1 which is the (i - 1)-th frame, the image Pi which is the i-th frame, the image Pi+1 which is the (i + 1)-th frame, and the image Pi+2 which is the (i + 2)-th frame are shown.

[0108] The labeling unit 116b associates the region R1i+2 extracted in the (i + 2)-th frame with the region R1i+1 extracted in the (i + 1)-th frame by the region association process. Similarly, the labeling unit 116b associates the region R2i+1 extracted in the (i + 2)-th frame with the region R2i+1 extracted in the (i + 1)-th frame by the region association process.

[0109] Furthermore, the labeling unit 116b associates the region R1i+1 extracted in the (i + 1)-th frame with the region Ri extracted in the i-th frame by the region association process. Similarly, the labeling unit 116b associates the region R2i+1 extracted in the (i + 1)-th frame with the region Ri extracted in the i-th frame by the region association process. Here, although the region Ri and the region R1i+1 are already associated, the labeling unit 116b further associates the region R2i+1 with the region Ri. That is, the labeling unit 116b may associate a plurality of regions from the region Ri+1 extracted in the (i + 1)-th frame with one region Ri extracted in the i-th frame.

[0110] The labeling unit 116b associates the region Ri extracted in the i-th frame with the region Ri-1 extracted in the (i - 1)-th frame by the region association process.

[0111] The labeling unit 116b assigns the same label L to the associated regions Ri-1, Ri, R1i+1, R2i+1, R1i+2, and R2i+2. That is, the labeling unit 116b assigns the same label (e.g., the first label L1) L to all of the associated regions R to form one group. Here, assigning the label L to the region R means associating the region R with the label L.

[0112] Note that the labeling unit 116b may further assign the same label (e.g., the second label L21) L to the region R1i+1 and the region R1i+2 to form one group, and assign the same label (e.g., the second label L22) L to the region R2i+1 and the region R2i+2 to form one group. By assigning the second label (e.g., the second label L21 and the second label L22), the generation number can be identified each time the cell C divides. Thus, the second label can be used as a label for identifying the generation of the cell C.

[0113] Note that when the region R is the tracking region TR, the labeling unit 116b may use the result of the region association process executed by the position calculation unit 111 in step S330 in the labeling process.

[0114] Step S350: The state determination unit 112 determines in combination with the result of the labeling process in step S340 that the luminance value of the tracking region TR extracted by the tracking region extraction unit 110 is greater than the luminance value of the background BG of the image P. The state determination unit 112 determines, for example, based on the label L that the tracking region TRi is divided into the region R1i+1 and the region R2i+1. When the tracking region TRi is divided into two regions and the luminance value is greater than the luminance value of the background BG of the image P, the state determination unit 112 sets the determined tracking region TRi as the tracking stop region SRi.

[0115] The labeling unit 116b may include information indicating the assigned label L in the analysis result Ab. That is, the analysis result Ab includes information on the pedigree analysis of cells. In step S40 of FIG. 5, the output unit 12 outputs the analysis result Ab including the label L to the presentation unit 3.

[0116] In the image processing apparatus 1b according to the present embodiment, in cell tracking, since it is possible to determine that a cell has divided based on the label L, the accuracy of analyzing the movement state M of the cell can be improved as compared with the case where the label L is not used. Further, in the image processing apparatus 1b according to the present embodiment, based on the label L, it can be determined that the tracking region TRi extracted in the i-th frame is divided into the region R1i+1 and the region R2i+1 extracted in the (i + 1)-th frame, so that pedigree analysis of cells can be performed.

[0117] Note that, in each of the above-described embodiments, an example in the case where the microscope 2 is a phase contrast microscope has been described, but the present invention is not limited thereto. The microscope 2 may be a differential interference contrast microscope. In a differential interference contrast microscope, an optical path difference (e.g., a difference in optical path length) of light (observation light) passing through the cell C is converted into contrast to obtain a luminance value of an image. Therefore, when the microscope 2 is a differential interference contrast microscope, the luminance indicated by the luminance information of the tracking region TR is a luminance value based on the optical path difference of the observation light. In this case, as an example, the first state X is a state in which the optical path difference is smaller than a predetermined value and the luminance value is smaller than a predetermined value, and the second state Y is a state in which the optical path difference is larger than a predetermined value and the luminance value is larger than a predetermined value.

[0118] Therefore, the luminance indicated by the luminance information of the tracking region TR is a luminance value based on the phase difference or the optical path difference. Since the contrast of the tracking region TR is the phase difference or the optical path difference, in the image processing apparatus 1 of each of the above-described embodiments, as the contrast, the phase difference or the optical path difference can be used to determine the state of the tracking region TR, so that a phase difference image or an image captured by a differential interference contrast microscope can be used as an object of analysis.

[0119] Note that, a part of the image processing apparatuses 1, 1a, and 1b in the above-described embodiments, for example, the tracking area extraction unit 110, the position calculation unit 111, the state determination unit 112, the stop control unit 113, the movement state calculation unit 114, the splitting time measurement unit 115a, and the labeling unit 116b may be realized by a computer such as a server or a client. In that case, a program for realizing this control function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed to be realized. Here, the “computer system” refers to a computer system built in the image processing apparatuses 1, 1a, and 1b and includes hardware such as an OS and peripheral devices. Further, the “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, etc., and a storage device such as a hard disk built in a computer system. Furthermore, the “computer-readable recording medium” also includes something that holds a program dynamically for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line, and something that holds a program for a certain time like a volatile memory inside a computer system serving as a server or a client in that case. Also, the above program may be for realizing a part of the above-described functions, and may further be something that can be realized in combination with a program already recorded in a computer system for the above-described functions.

[0120] Note that, for example, in this embodiment, a program is provided for causing a computer to execute: an area extraction step of extracting a tracking area TR from an image in which cells are imaged, where the luminance state indicated by the luminance information is a first state X; a position calculation step of calculating a temporal change in the position of the tracking area TR extracted in the area extraction step based on a plurality of the images; a state determination step of determining that the luminance state indicated by the luminance information of the tracking area TR extracted in the area extraction step is a second state Y different from the first state X; and a movement state calculation step of calculating a movement state of the tracking area TR based on the calculation result of the position calculation step and the determination result of the state determination step.

[0121] Further, for example, in this embodiment, a program is provided for causing a computer to execute: an area extraction step of extracting a tracking area TR from an image in which cells are imaged, where the luminance state indicated by the luminance information is a first state X; a position calculation step of calculating a temporal change in the position of the tracking area TR extracted in the area extraction step based on a plurality of the images; a state determination step of determining that the luminance state indicated by the luminance information of the tracking area TR extracted in the area extraction step is a second state Y different from the first state X; and a stop control step of stopping the calculation of the temporal change based on the determination result of the state determination step.

[0122] Also, part or all of the image processing apparatuses 1, 1a, and 1b in the above-described embodiment may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each functional block of the image processing apparatuses 1, 1a, and 1b may be individually made into a processor, or part or all of them may be integrated and made into a processor. Further, the method of integrating into a circuit is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. Also, when a circuit integration technology that replaces LSI appears due to the progress of semiconductor technology, an integrated circuit using such technology may be used.

[0123] The above has described in detail an embodiment of the present invention with reference to the drawings. However, the specific configuration is not limited to the above, and various design changes and the like can be made without departing from the gist of the present invention.

Explanation of Signs

[0124] 1, 1a, 1b... image processing apparatus; 1, 2... microscope; 110... tracking area extraction unit; 111... position calculation unit 111; 112... state determination unit; 113... stop control unit 113; 114... movement state calculation unit; 115a... division time measurement unit; 116b... labeling unit 116b; X... first state; Y... second state; R... area; TR... tracking area; SR... tracking stop area

Claims

1. A cell tracking method for performing cell tracking based on a plurality of cell images captured in a time series, comprising: For each of the plurality of cell images, an extraction step of extracting a tracking region that is a tracking target of the cell based on the luminance information of the cell image; An analysis step of calculating a time change in the position of the tracking region extracted in the extraction step based on the plurality of cell images, and analyzing the movement state of the cell based on the time change, When analyzing the movement state of the cell, the analysis step excludes the time when the cell is not moving in the calculation process of the time change. A cell tracking method.

2. Excluding the time when the cell is not moving means excluding the position of the tracking region where the luminance state indicated by the luminance information of the tracking region is in a predetermined state from the calculation target of the time change. The cell tracking method according to claim 1.

3. The state where the cell is not moving is the state of the cell immediately before cell division and during cell division of the cell. In the analysis step, in order to exclude the time when the cell is not moving, the position of the tracking region where the luminance state indicated by the luminance information of the tracking region is in a predetermined state is excluded from the calculation target of the time change. The cell tracking method according to claim 1.

4. By determining whether the luminance state indicated by the luminance information of the tracking region is a first state corresponding to a moving state or a second state corresponding to a cell division state based on the cell image, it is further determined whether the cell that is the tracking target is in a moving state or in a cell division state where the tracking region is excluded from the calculation target of the time change. A determination step is provided. The analysis step is as follows: When it is determined in the determination step that the luminance state indicated by the luminance information of the tracking region is the first state, the moving state of the cell is analyzed based on the calculated time change, and When it is determined in the determination step that the luminance state indicated by the luminance information of the tracking region has changed from the first state to the second state, the position of the tracking region in the case of the second state is not used for calculating the time change, and the moving state of the cell is analyzed based on the calculated time change, and When it is determined in the determination step that the luminance state indicated by the luminance information of the tracking region has changed from the second state to the first state, the tracking of the cell is restarted by restarting the calculation of the time change, and the moving state of the cell is analyzed based on the time change calculated after the tracking of the cell is restarted The cell tracking method according to claim 1.

5. When it is determined in the determination step that the luminance state indicated by the luminance information of the tracking region has changed from the second state to the first state, in the extraction step, a new tracking region is extracted for a plurality of cells after cell division in order to start tracking of the cell from the plurality of cell images, For the new tracking region, new tracking is performed for each of the plurality of cells after cell division, and the moving state of the cell is analyzed based on the time change The cell tracking method according to claim 4.

6. Further, in the analysis step, as an analysis result of the plurality of tracked cells, the migration ability of the cell is analyzed by comparing the data distribution situation of the time change of the cell under the reference culture conditions The cell tracking method according to claim 5.

7. Further having a storage step of storing, for each cell, any one of the centroid position of the tracking region extracted in the extraction step, the calculation result of the time change, the determination result of the determination step, and the analysis result of the analysis step as the tracking information of the cell, The cell tracking method according to claim 4.

8. Furthermore, in the analysis step, when it is determined in the determination step that the cell is in the cell division state, the division time required for the cell to divide is measured based on the time change during the period of the cell division state. The cell tracking method according to claim 4.

9. Furthermore, in the analysis step, as an analysis result of a plurality of cells determined to be in the cell division state in the determination step, the cell cycle of the cell is analyzed by comparing the data distribution status of the division time of the cell under the reference culture conditions. The cell tracking method according to claim 8.

10. It further has a label - attaching step of attaching a label for identifying the tracking region extracted by the extraction step, In the label - attaching step, between the frames of the cell images at different imaging times before and after cell division of the cells to be tracked, the same label is attached to the tracking region by region - correspondence processing. The cell tracking method according to claim 1.

11. An image processing apparatus for performing cell tracking processing based on a plurality of cell images captured in a time series, For each of the plurality of cell images, a region extraction unit that extracts a tracking region that is the object of tracking of the cell based on the luminance information of the cell image, A movement state calculation unit that calculates the time change of the position of the tracking region extracted by the region extraction unit based on the plurality of cell images and analyzes the movement state of the cell based on the time change, Comprising, When analyzing the movement state of the cell, the movement state calculation unit excludes the time when the cell is in a non - moving state in the calculation process of the time change. Image processing apparatus.

12. Excluding the time when the cell is not moving means excluding the position of the tracking region where the luminance state indicated by the luminance information of the tracking region is in a predetermined state from the calculation target of the time change. The image processing apparatus according to claim 11.

13. The state where the cell is not moving is the state of the cell immediately before cell division and during cell division of the cell. The movement state calculation unit excludes the position of the tracking region where the luminance state indicated by the luminance information of the tracking region is in a predetermined state from the calculation target of the time change in order to exclude the time when the cell is not moving. The image processing apparatus according to claim 11.

14. Based on the cell image, by determining whether the luminance state indicated by the luminance information of the tracking region is a first state corresponding to a moving state or a second state corresponding to a cell division state, it is further provided with a state determination unit for determining whether the cell that is the tracking target is in a moving state or in a cell division state where the tracking region is excluded from the calculation target of the time change. The movement state calculation unit When the state determination unit determines that the luminance state indicated by the luminance information of the tracking region is the first state, the movement state of the cell is analyzed based on the calculated time change, and When the state determination unit determines that the luminance state indicated by the luminance information of the tracking region has changed from the first state to the second state, the position of the tracking region in the case of the second state is not used in the calculation of the time change, and the movement state of the cell is analyzed based on the calculated time change, and When the state determination unit determines that the luminance state indicated by the luminance information of the tracking region has changed from the second state to the first state, the tracking of the cell is restarted by restarting the calculation of the time change, and the movement state of the cell is analyzed based on the time change calculated after the tracking of the cell is restarted. The image processing apparatus according to claim 11.

15. When it is determined by the state determination unit that the luminance state indicated by the luminance information of the tracking area has changed from the second state to the first state, the area extraction unit extracts a new tracking area for a plurality of cells after cell division in order to start tracking of the cells from the plurality of cell images, The movement state calculation unit performs new tracking on each of the plurality of cells after cell division for the new tracking area, and analyzes the movement state of the cells based on the time change The image processing apparatus according to claim 14.

16. Further comprising a label assigning unit that assigns a label for identifying the tracking area extracted by the area extraction unit, The label assigning unit assigns the same label to the tracking area by area correspondence processing between frames at different imaging times before and after cell division of the cells that are the objects of tracking of the cells The image processing apparatus according to claim 11.

17. A program for causing a computer to perform tracking of cells based on a plurality of cell images captured in time series, For each of the plurality of cell images, an extraction step of extracting a tracking area that is the object of tracking of the cells based on the luminance information of the cell images, An analysis step of calculating a time change in the position of the tracking area extracted in the extraction step based on the plurality of cell images, and analyzing the movement state of the cells based on the time change, A program for causing the above to be executed, In the analysis step, when analyzing the movement state of the cells, the time when the cells are in a non-moving state is excluded in the calculation process of the time change, Program.

Images