A method for detecting cell division status based on cell division lineage relationships

Abstract

The present invention provides a cell division detection method based on the cell division lineage relationship, step S1: according to the set time step, take the cell image in the time difference incubator; step S2: perform target detection on the cell image in each time step, and extract the characteristics of each cell; step S3: traverse all cell pairs in two consecutive time steps, splice the characteristics of two cells in each cell pair into a vector, input it into a multi-layer perceptron, and obtain the probability that the two cells in the cell pair are the same cell and the probability that one cell is generated by the division of another cell; step S5: repeat step S4 until all cell images in all time steps are detected, and judge the division state according to the obtained probability, and construct a cell division tree. The cell division tree structure is constructed through the relationship between cells, so as to effectively restore the complete cell proliferation process.

Description

Technical Field

[0001] The present invention relates to the technical field of cell division recognition, and in particular to a method for detecting cell division status based on cell division lineage relationship. Background Art

[0002] Cell division is crucial for the growth, development and proliferation of organisms. Cell division is the process of living cells multiplying their number, from one cell to multiple cells. The cell before division is called the mother cell, and the new cell formed after division is called the daughter cell. Due to the influence of the cell environment or external factors, cells may undergo abnormal division or reverse division. Among them, the characteristic of abnormal division is that multiple daughter cells are formed when a cell divides, and the characteristic of reverse division is that two cells merge into the same daughter cell. These two special division situations may lead to genomic instability and cell dysfunction. By analyzing the state of cell division, it is helpful to predict and reveal the basic mechanism of organism development and the pathological mechanism of disease. The time difference incubator can provide cells with a stable in vitro culture environment, and can periodically and continuously obtain images of the entire process of cell division. Combined with the images, time and other data of the cell division process periodically recorded by the electron microscope in the incubator, researchers can determine whether the cell division is normal. However, this often requires high labor costs and is highly dependent on the experience of experts. With the development of deep learning, the use of computer vision models can effectively capture image features, thereby helping researchers to judge the state of cell division more quickly and accurately. At present, there are still the following problems in using machine learning to classify cell division states:

[0003] (1) Cell division is a dynamic process. A cell in the current image may be derived from the division of another cell in the previous time. This contains strong temporal correlation information. To determine the division of a cell, it is necessary to comprehensively consider the cell state before and after the division. If the cell information at different time sequences is not correlated, the accuracy of the division state determination may be reduced.

[0004] (2) The normal cell division process determines the kinship between cells: the relationship between two or more daughter cells produced by the division of a mother cell can be regarded as a mother-child relationship, and the relationship between daughter cells produced by the division of the same mother cell can be regarded as a brother relationship. However, abnormal cell division does not conform to the above kinship. How to accurately track the kinship between cells during the division process and construct a complete cell lineage map is of great significance for discovering abnormal states of the division process. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a method for detecting cell division based on the relationship between cell division lineages, comprising the following steps:

[0006] Step S1: taking images of cells in the time-lapse incubator according to the set time step;

[0007] Step S2: performing target detection on the cell image within each time step and extracting the features of each cell;

[0008] Step S3: traverse all cell pairs within two consecutive time steps, concatenate the features of the two cells in each cell pair into a vector, and input it into the multi-layer perceptron to obtain the probability that the two cells in the cell pair are the same cell and the probability that one cell is generated by the splitting of another cell;

[0009] Step S4: Repeat step S3 until all cell images in all time steps are detected, and determine the division state according to the obtained probability to construct a cell division tree.

[0010] Preferably, the cell image obtained in step S1 is preprocessed by image edge detection, image cropping and image standardization.

[0011] Preferably, the method of performing target detection on the cell image in each time step in step S2 includes:

[0012] Step S21: extracting a feature map of the cell image;

[0013] Step S22: Divide the feature map into The grid is then input into the target detection network, and the confidence score of each grid containing the target is output;

[0014] Step S23: Based on the confidence score, a target bounding box is obtained, and the coordinates of the target bounding box are converted into coordinates relative to the feature map.

[0015] Preferably, the expression for coordinate transformation in step S213 is:

[0016] ;

[0017] ;

[0018] ;

[0019] ;

[0020] In the formula, and Represents the coordinates of the center point of the target bounding box, and Represents width and height information, , , and Respectively represent the offset of the predicted bounding box relative to the grid unit in the x-axis coordinate, y-axis coordinate, width and height, and is the coordinate of the upper left corner of the grid cell, and are the length and width of the grid cell, is the activation function.

[0021] Preferably, after the target bounding box is obtained through step S23, duplicate detections are removed by non-maximum suppression: first, low-confidence bounding boxes are filtered out according to the confidence score of each grid, and then the remaining bounding boxes are non-maximum suppressed to retain the bounding box with the highest confidence; for two bounding boxes A and B, if their IOU exceeds a set threshold, the bounding box with lower confidence is removed, and finally the target bounding box is output.

[0022] Preferably, the loss function of the target detection network during training is focaler-IoU multiplied by -1, where the expression of focaler-IoU is:

[0023] ;

[0024] In the formula, Indicates the set lower threshold. It represents the set upper threshold, and IoU represents the intersection over union ratio.

[0025] Preferably, the expression for extracting each cell feature in step S2 is:

[0026] ;

[0027] In the formula, m and n represent the number of rows and columns of the grid occupied by the cell, respectively; Represents the grid feature of the cell occupying the jth grid in the i-th row of the grid; is the position vector of the cell, i.e., the coordinates of the center point and the upper left corner point of the cell detection frame; represents the pixel area occupied by the cell target box on the i-th row and j-th column; and is the width and length of the grid unit; Represents a concatenation operation.

[0028] Preferably, the training process of the multilayer perceptron in step S3 includes: collecting the true value probability that the cell x at time t and the cell y at time t+1 are the same cell , the true probability that cell y is generated by cell x division is ; If x and y are the same cell, then , ; If y is obtained by splitting x, then , ; If x and y have no relationship, then , , through the loss function To conduct restraint training:

[0029] ;

[0030] In the formula, represents the probability that the two cells predicted by the network are the same cell, It represents the probability predicted by the network that cell y is generated by the division of cell x.

[0031] Preferably, the expression for constructing the cell splitting tree in step S5 is:

[0032]

[0033] represents the relationship between cell x at time t and cell y at time t+1, represents the probability that two cells are the same cell, represents the probability that cell y is generated by the division of cell x, Indicates the set threshold.

[0034] Preferably, in step S5, a tree structure is used to construct a cell division tree, each node stores the ID value of the cell and the shooting time t of the corresponding image when it first appears; the cell in the first cell image is used as the root node; when there is a division relationship between cells, a new node is added to the new cell and a new ID value is assigned.

[0035] The beneficial effects of the present invention include at least: the present invention proposes a new method to identify the kinship relationship between cells in continuous time images, and infers whether the division state of the cells at that time is normal, abnormal division or reverse division through this relationship; based on the above kinship relationship, the present invention proposes a method for constructing a pedigree relationship tree of the cell division process, based on which, the position of the cells at a later time can be effectively determined at a previous time, and at the same time, the ancestor cells of the new cells generated by the division at a later time can be determined at the previous time, thereby effectively restoring the complete cell proliferation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A schematic diagram of a method flow of an embodiment of the present invention;

[0037] Figure 2 A schematic diagram of cell relationship prediction according to an embodiment of the present invention;

[0038] Figure 3 This is a schematic diagram of determining the cell division state through cell relationships according to an embodiment of the present invention;

[0039] Figure 4 This is a schematic diagram of a cell division tree structure according to an embodiment of the present invention;

[0040] Figure 5 This is a schematic diagram of target detection results according to an embodiment of the present invention;

[0041] Figure 6 This is a schematic diagram of feature extraction performed by the cell feature extraction module of Example 3; DETAILED DESCRIPTION

[0042] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the protection scope of the present invention.

[0043] Example 1

[0044] like Figure 1 As shown, an embodiment of the present invention provides a method for detecting cell division based on cell division lineage relationship, comprising the following steps:

[0045] Step S1: Take images of cells in the time-lapse incubator according to the set time step.

[0046] Specifically, collect cell images: use a time-lapse incubator to record the cell proliferation process, ensure that the incubator has appropriate temperature and humidity control, configure the microscope's magnification, light source, filter and other parameters to ensure that the cells can be clearly photographed. Take a cell sample every 15 minutes with a time step of 15 minutes.

[0047] Then, the cell image dataset was constructed and preprocessed. Specifically, the embryo images collected according to the time step were labeled using the image labeling software LabelImg. For each cell in the image, a rectangular bounding box was drawn using the annotation tool to surround it. This bounding box should fit the edge of the cell as closely as possible. After labeling, the coordinates, width, and height information of the cell center point were obtained.

[0048] Cell ID tag: Each cell is given a unique ID value for analyzing cell status. Numbers are used as ID values. The ID value of the cell in the first image, i.e. the starting moment, starts at 1. If division occurs at subsequent moments, the ID value increases.

[0049] Cell division record: When marking cell IDs, an additional document is used to record the cell division relationship for training the cell division prediction module. For example, if a cell with ID 1 divides into cells with IDs 3 and 4 in the next time series image, the division relationships that need to be recorded are 1-3 and 1-4.

[0050] Dataset correction: After all images are labeled, some images are extracted for manual inspection to correct incorrect labels and update the annotation files to ensure accurate labeling.

[0051] Dataset division: After the dataset is labeled, it is divided into training set, validation set and test set according to 6:2:2.

[0052] The purpose of cell image preprocessing is to reduce noise interference, highlight cell features, and standardize input. In the images collected by the optical microscope, the cells are concentrated inside the circular culture medium. In order to filter the background noise information outside the culture dish, the following operations need to be performed on the image.

[0053] Image edge detection: Through the Canny edge detection algorithm, morphological operations such as corrosion and expansion, the contour search method detects the cell area, highlights the edge features of the cell image, and makes the image clearer in visualization and analysis. The edge-processed cell image and the original image are superimposed as input.

[0054] Image cropping: The circular culture medium position is determined based on the detected cell edge information, and then cropped using a rectangular frame of a preset size. The center point of the cropped area corresponds to the center point of the detected cell area to ensure that the cell is centered in the cropped image.

[0055] Image normalization: By cropping and centering the cell images, we can ensure that cells in all input images have similar sizes and positions, thereby improving the generalization ability and accuracy of the model.

[0056] Step S2: Perform target detection on the cell image in each time step and extract the features of each cell.

[0057] In the embodiment of the present invention, when performing target detection, including but not limited to using R-CNN, faster R-CNN, yolo and SSD detection models, feature extraction uses a traditional feature extraction network, including but not limited to using RNN, CNN and Transformer.

[0058] Step S3: Traverse all cell pairs within two consecutive time steps, concatenate the features of the two cells in each cell pair into a vector, and input it into the multi-layer perceptron to obtain the probability that the two cells in the cell pair are the same cell and the probability that one cell is generated by the splitting of another cell.

[0059] Specifically, Figure 2 As shown, a is used to represent the characteristics of cells in the image at time t, and b is used to represent the characteristics of cells in the image at time t+1. There are two cells in the image at time t, and their cell characteristics are represented by a1 and a2 respectively; similarly, there are three cells at time t+1, and their cell characteristics are represented by b1, b2, and b3 respectively. Generally speaking, there are three types of relationships between a cell in the image at time t+1 and a cell in the image at time t.

[0060] Relationship 1: A cell in the image at time t+1 and a cell in the image at time t are the same cell; Relationship 2: A cell in the image at time t+1 is generated by the division of a cell in the image at time t; Relationship 3: There is no direct relationship between a cell in the image at time t+1 and a cell in the image at time t.

[0061] Traverse all combinations of cell pairs in the previous and next images, send the features of each pair of cells into the cell relationship prediction network, and determine which category the kinship between all cells in the previous and next images belongs to.

[0062] Specifically, all possible cell pairs between the cells on the image at time t+1 and the cells on the image at time t are constructed by traversing; let cell y on the image at time t+1 and cell x on the image at time t constitute a pair of cells, and their features are and . Concatenate them into a vector and input it into the multi-layer perceptron to predict two values , The matching relationship between them can be expressed by the following formula:

[0063] ;

[0064] ;

[0065] in, express The vectors are transformed to The network weights, express The vectors are transformed to The network weights. represents the probability that x and y are the same cell. Greater than threshold and When , it is determined that x and y are the same cell; represents the probability that y is generated by splitting x. Greater than threshold and When , it is determined that y is generated by splitting x; if Less than threshold and Less than threshold Then it is determined that x and y have no relationship. The values ​​are between 0 and 1. is a priori value.

[0066] In summary, the relationship between cells x and y It can be expressed as:

[0067]

[0068] In this embodiment, a multilayer perceptron is used as the network, and its loss function is calculated by the following method. The probability that x and y are the true values ​​of the same cell is , the true probability that y is generated by splitting x is ,in Indicates the true value. In the labeled data set, if x and y are the same cell, then , ; If y is obtained by splitting x, then , ; If x and y have no relationship, then , By measuring the true probability , and predicted probability , The difference between them is used as the relationship prediction loss To constrain the network, that is:

[0069] .

[0070] Step S4: Repeat step S3 until all cell images in all time steps are detected, and determine the division state according to the obtained probability to construct a cell division tree.

[0071] Specifically, after obtaining the kinship of all cells in the images at time t and time t+1, the family tree corresponding to the cells in the pair of images can be constructed. If a new cell is generated, a daughter cell node is added and an arrow is used to connect the ancestor cell and the daughter cell. If it is the same cell, no new node is added, such as Figure 3 By comparing the patterns in this way, it is possible to determine whether the cells undergo normal division, abnormal division or reverse division during this period, such as Figure 3 shown.

[0072] Record the maximum number of divisions of each cell. If the maximum number of divisions is 0, it means that the cell has not divided; if the maximum number of divisions is 2, it means that the cell is dividing normally; if the maximum number of divisions is greater than 2, it means that the cell is dividing abnormally. In addition, if the total number of cells at time t+1 is less than that at time t, it means that the cell is reversely dividing.

[0073] By traversing and predicting the cell relationship at two consecutive moments, the cell relationship family tree corresponding to the complete image sequence can be obtained. The embodiment of the present invention uses a tree structure to establish a cell relationship family tree. The tree structure is a nonlinear data structure used to simulate a data set with tree structure properties.

[0074] Each node stores the ID value of the cell and the shooting time t of the corresponding image when the cell first appears, such as Figure 4 As shown. Prediction starts from the first picture in the data set, and the cell in the first picture is used as the root node. When the cell prediction network predicts the cell division relationship, a new node is added to the new cell and a new ID value is assigned. Normally divided cells satisfy the properties of the binary tree structure, that is, one ancestor cell node corresponds to two daughter cell nodes. When abnormal cell division occurs, the properties of the binary tree structure are not satisfied, because abnormal division has one ancestor node corresponding to multiple daughter cell nodes. When reverse cell division occurs, the cell genealogy does not satisfy the characteristics of the tree structure, that is, one daughter cell may have two ancestor nodes, forming a ring structure.

[0075] Example 2

[0076] This embodiment improves the traditional target detection network based on the first embodiment. The target detection network in this embodiment mainly includes: feature extraction and bounding box prediction.

[0077] Feature extraction: Use a convolutional neural network (CNN) to extract features from each of the n embryo images. The CNN model includes multiple convolutional layers, pooling layers, and activation functions, etc., which are used to extract useful feature information from the image. Output the feature maps corresponding to the n images.

[0078] Bounding box prediction: Based on feature extraction, the input image is divided into The network is responsible for predicting the confidence score of each grid containing the target, converting the predicted bounding box coordinates from the offset relative to the grid cell , , , Convert to coordinates relative to the original image. Get the center point coordinates of the target bounding box , And width and height information , , as shown below:

[0079] ;

[0080] ;

[0081] ;

[0082] ;

[0083] in , is the upper left corner of the grid cell, , are the length and width of the grid cell, is the activation function.

[0084] In order to further remove duplicate detections, this embodiment provides the following method for removing duplicate bounding boxes.

[0085] Using the non-maximum suppression method, we first filter out low-confidence bounding boxes based on the confidence score, and then perform non-maximum suppression on the remaining bounding boxes to retain the bounding box with the highest confidence. For two bounding boxes A and B, if their IOU exceeds the set threshold (e.g. 0.5), the bounding box with lower confidence is removed. The output is the bounding box representing the cell location.

[0086] In the object detection network of this embodiment, the Focaler-IoU between the metric and the true target box is multiplied by -1 during training as the regression loss of the bounding box position. The Focaler-IoU is defined as follows:

[0087] ;

[0088] In the formula, Indicates the set lower threshold. represents the upper threshold, and IoU represents the intersection over union ratio. When IoU is less than a lower threshold d, the loss is 0; when IoU is greater than an upper threshold u, the loss is 1, and when IoU is between d and u, the loss is a function that increases linearly according to the IoU value. The target detection loss is used as a constraint for neural network learning. The final detection result is as follows: Figure 5 shown.

[0089] Example 3

[0090] This embodiment provides a cell feature extraction module, which improves the traditional feature extraction network. Specifically, Figure 6 shown.

[0091] After obtaining the cell position, it is necessary to extract features for each individual cell. In the target detection module, the image is divided into Each grid corresponds to a grid feature , represents the jth grid in the i-th row. In order to avoid repeated feature extraction and feature calculation, it is possible to consider obtaining cell features from grid features. A cell may cover multiple grid units, some of which are not completely covered by the target. Therefore, it is possible to consider weighted averaging according to the proportion of grid units covered by the target box through weighted pooling operation. Assuming that the cell target occupies a total of m rows and n columns of the grid, then the feature corresponding to cell t is It can be obtained by the following formula:

[0092] ;

[0093] in , are the width and length of the grid cell, represents the pixel area occupied by the cell target box on the i-th row and j-th column, It is a fusion function, which means merging two vectors into one vector. is the cell position vector, i.e. the coordinates of the center point and the upper left corner of the cell detection frame. By combining the cell position information, position encoding is achieved to better judge the relationship between cells. The output is the feature vector of each cell in the image.

[0094] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. Only the preferred embodiments of the present invention are expressed. The description is more specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. As long as there is no contradiction in the combination of these technical features, they should be considered as the scope recorded in this specification.

[0095] It should be noted that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these modifications and improvements all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for detecting cell division based on cell division lineage relationships, characterized in that: The following steps are involved: Step S1: taking images of cells in the time-lapse incubator according to the set time step; Step S2: performing target detection on the cell image within each time step and extracting features of each cell; Step S3: traverse all cell pairs within two consecutive time steps, concatenate the features of the two cells in each cell pair into a vector, and input it into the multi-layer perceptron to obtain the probability that the two cells in the cell pair are the same cell and the probability that one cell is generated by the splitting of another cell; Step S4: Repeat step S3 until all cell images at all time steps are detected, and determine the division state according to the obtained probability to construct a cell division tree; The method for performing target detection on the cell image in each time step in step S2 includes: Step S21: extracting a feature map of the cell image; Step S22: Divide the feature map into The grid is then input into the target detection network, and the confidence score of each grid containing the target is output; Step S23: obtaining a target bounding box based on the confidence score, and converting the coordinates of the target bounding box into coordinates relative to the feature map; After the target bounding box is obtained through step S23, duplicate detections are removed by non-maximum suppression: first, low-confidence bounding boxes are filtered out according to the confidence score of each grid, and then non-maximum suppression is performed on the remaining bounding boxes to retain the bounding box with the highest confidence; for two bounding boxes A and B, if their IOU exceeds the set threshold, the bounding box with lower confidence is removed, and finally the target bounding box is output.

2. A cell division detection method based on cell division lineage relationship according to claim 1, characterized in that: The cell image obtained in step S1 is preprocessed by image edge detection, image cropping and image normalization.

3. A cell division detection method based on cell division lineage relationship according to claim 1, characterized in that: The expression for coordinate transformation in step S23 is: ； ； ； ； In the formula, and Represents the coordinates of the center point of the target bounding box, and Represents width and height information, , , and Respectively represent the offset of the predicted bounding box relative to the grid unit in the x-axis coordinate, y-axis coordinate, width and height, and is the coordinate of the upper left corner of the grid cell, and are the length and width of the grid cell, is the activation function.

4. A method for detecting cell division based on cell division lineage relationship according to claim 1, characterized in that: The loss function of the target detection network during training is focaler-IoU multiplied by -1, where the expression of focaler-IoU is: ； In the formula, Indicates the set lower threshold. It represents the set upper threshold, and IoU represents the intersection over union ratio.

5. The method for detecting cell division based on cell division lineage relationship according to claim 1, characterized in that: The expression for extracting each cell feature in step S2 is: ； In the formula, m and n represent the number of rows and columns of the grid occupied by the cell, respectively; Represents the grid feature of the cell occupying the jth grid in the i-th row of the grid; is the position vector of the cell, i.e., the coordinates of the center point and the upper left corner point of the cell detection frame; represents the pixel area occupied by the cell target box on the i-th row and j-th column; and is the width and length of the grid unit; Represents a concatenation operation.

6. A method for detecting cell division based on cell division lineage relationship according to claim 1, characterized in that: The training process of the multilayer perceptron in step S3 includes: collecting the true value probability that the cell x at time t and the cell y at time t+1 are the same cell , the true probability that cell y is generated by cell x division is ; If x and y are the same cell, then , ; If y is obtained by splitting x, then , ; If x and y have no relationship, then , , through the loss function To conduct restraint training: ； In the formula, represents the probability that the two cells predicted by the network are the same cell, It represents the probability predicted by the network that cell y is generated by the division of cell x.

7. A cell division detection method based on cell division lineage relationship according to claim 1, characterized in that: The expression for constructing the cell division tree in step S5 is: represents the relationship between cell x at time t and cell y at time t+1, represents the probability that two cells are the same cell, represents the probability that cell y is generated by the division of cell x, Indicates the set threshold.

8. The method for detecting cell division based on cell division lineage relationship according to claim 1, characterized in that: In step S5, a tree structure is used to construct a cell division tree, where each node stores the ID value of the cell and the shooting time t of the corresponding image when it first appears; the cell in the first cell image is used as the root node; when there is a division relationship between cells, a new node is added to the new cell and a new ID value is assigned.

Images