Cell classification and analysis device and method based on mitochondrial subtype of super-resolution image

Abstract

The invention discloses a super-resolution image mitochondrial subtype-based cell classification and analysis device and method, and the method comprises the steps: 1, calculating the characteristics of each mitochondrial according to all mitochondrial segmentation images in cells, carrying out the dimension reduction processing, carrying out the clustering of the mitochondrial characteristics after the dimension reduction, and obtaining the type of each mitochondrial; and step 2, calculating the proportion of each type of mitochondria in the cells according to the cells and the types of the mitochondria obtained in the step 1, determining the characteristics of the cells, performing dimension reduction processing, and clustering the characteristics of the cells after dimension reduction to obtain the types of the cells. According to the method, the form and position parameters of the single mitochondria can be efficiently represented, and favorable conditions are provided for quantifying the cell state.

Description

Technical Field

[0001] The present invention relates to the technical field of super-resolution imaging, and in particular to a cell classification and analysis device and method based on super-resolution image mitochondrial subtypes. Background Art

[0002] Mitochondria are important organelles in cells that are responsible for producing the energy needed by cells. In the past, people generally believed that mitochondria only had a rod-shaped structure. But with the advancement of science and technology, we gradually realized that mitochondria have multiple forms, and the differences in these forms may be related to their functions and metabolic states. Different mitochondrial forms and their different locations in cells may ultimately determine different behaviors or metabolic states of cells.

[0003] Under a fluorescence microscope, mitochondria can be generally divided into three categories: dots, rods, and networks. On the one hand, there are no strictly defined standards between these categories, and there is a lack of efficient quantification tools. On the other hand, classification based solely on morphology lacks comprehensiveness. The intensity of mitochondrial fluorescence, the location of distribution, and specific morphological characteristics may hide key information about the energy utilization process of organisms. Therefore, how to comprehensively construct and model the characteristics of mitochondria is intuitively important for biological research.

[0004] On the other hand, cells are the smallest unit of life, and different states of cells affect all aspects of the organism. The classification of cells plays an important role in clinical applications, drug screening and other fields.

[0005] At present, the common cell classification methods mainly include methods based on artificial features and methods based on neural networks. However, these methods have certain defects, some of which do not take into account the impact of mitochondria on cell function, and some do not have good interpretability. Summary of the invention

[0006] The purpose of the present invention is to provide a cell classification and analysis device and method based on super-resolution image mitochondrial subtypes, which can efficiently characterize the morphology and position parameters of single mitochondria and provide favorable conditions for quantifying cell status.

[0007] To achieve the above object, the present invention provides a method for cell classification and analysis of mitochondrial subtypes based on super-resolution images, which comprises:

[0008] Step 1: Based on the segmented images of all mitochondria in the cell, the features of each mitochondria are calculated and dimension reduction is performed, and the mitochondrial features after dimension reduction are clustered to obtain the categories of each mitochondria;

[0009] Step 2: According to the cell and the categories of each mitochondria obtained in step 1, the proportion of each type of mitochondria in the cell is calculated, the characteristics of the cell are determined and dimensionality reduction is performed, and the cell characteristics after dimensionality reduction are clustered to obtain the cell category.

[0010] Further, step 1 includes obtaining the characteristics of mitochondria using a connected domain method; the characteristics of mitochondria include:

[0011] a. Mitochondrial length l: It is defined as the centerline length of the skeletonized mitochondrial segmentation image x, as shown in the following formula (1);

[0012] l=SUM(skeleton(x)) (1)

[0013] In the formula, the pixel value of the mitochondria part in the mitochondrial segmentation image x is 1, the pixel value of the background part is 0, skeleton is the skeletonization function, and SUM is the summation function;

[0014] b. Mitochondrial volume V: It is defined as the number of pixels in the mitochondrial segmentation image x, as shown in the following formula (2);

[0015] V=SUM(x) (2)

[0016] c. Mitochondrial cross-sectional area S: It is defined as the ratio of mitochondrial volume V to length l, as shown in the following formula (3);

[0017] S=V / l (3)

[0018] d. The Euler characteristic number of mitochondria En: It is defined as the Euler characteristic number of the connected area where the mitochondrial segmentation image x is located, as shown in the following formula (4);

[0019] En=χ(x) (4)

[0020] In the formula, χ is the Euler characteristic;

[0021] e. Solidity of mitochondria: It is defined as the ratio of the volume V of the mitochondria to the convex set volume Covex(x), as shown in the following formula (5);

[0022] Solidity=V / Covex(x) (5)

[0023] f. The length of the major axis and minor axis of the mitochondria: It is defined as the length moment of the major axis and minor axis of the circumscribed ellipse / ellipsoid surrounding the mitochondrial segmentation image x, as shown in the following formula (6);

[0024] moment=Eig(x) (6)

[0025] Where, Eig is the eigenvalue of the mitochondrial segmentation image x;

[0026] g. Average fluorescence brightness of mitochondria It is defined as the average value of the fluorescence brightness I(x) of the connected area where the mitochondrial segmentation image x is located;

[0027] h. Standard deviation of mitochondrial fluorescence brightness std(I(x)): It is defined as the standard deviation of the fluorescence brightness I(x) of the connected area where the mitochondrial segmentation image x is located;

[0028] i. Mitochondrial circumference p;

[0029] j. Circularity of mitochondria: It is defined as the following formula (7):

[0030]

[0031] k. Number of terminal nodes of mitochondria;

[0032] l. The number of crossover nodes of mitochondria.

[0033] Furthermore, the method for obtaining the number of end nodes and cross nodes of mitochondria includes:

[0034] The binary images corresponding to all mitochondria in the cell are skeletonized to obtain the mitochondrial skeleton. The degree of the node of each pixel in the mitochondrial skeleton is calculated. The degree value of the end node is 1, and the degree value of the cross node is 2.

[0035] Furthermore, the "characteristics of cells" in step 2 include the following characteristics:

[0036] a. Polar distribution of mitochondria: It is defined as the divergence between the angular or radial distribution of each mitochondria and the uniform distribution in the polar coordinate system / spherical coordinate system with the geometric center of the cell as the coordinate origin; wherein, the method for obtaining the "divergence between the angular / radial distribution of each mitochondria and the uniform distribution" includes: first, in the binary image corresponding to all mitochondria in the cell that are not point-shaped, according to the angle θ and the Euclidean distance r between each mitochondrial pixel P(x,y) on the binary image and the coordinate origin, the angular distribution of the corresponding mitochondria is obtained according to the angle θ, and the radial distribution of the corresponding mitochondria is obtained according to the Euclidean distance r; then, according to the angular / radial distribution of the mitochondria, within the range of [0,2π], the θ and r of all mitochondrial pixels P(x,y) are statistically histogrammed, thereby obtaining the divergence between the histogram distribution and the uniform distribution, and then obtaining the polar distribution of the mitochondria;

[0037] b. The number of mitochondria in the cell;

[0038] c. The sum of the volumes of all mitochondria in the cell.

[0039] Furthermore, the method for obtaining the “binary image” includes:

[0040] First, the imaging grayscale distribution of the received low signal-to-noise ratio image is adjusted to a uniform and easy-to-segment high signal-to-noise ratio image. Then, the features of the high signal-to-noise ratio image are extracted in the order of low-level features, intermediate features, and gradually transitioned to high-level features to obtain a feature map. Then, the features in the feature map are matched and weighted to obtain each pixel in the high signal-to-noise ratio image. A binary map is output according to the probability that each pixel is a mitochondrial area.

[0041] The present invention also provides a cell classification and analysis device based on super-resolution image mitochondrial subtypes, which comprises:

[0042] The mitochondrial classification module is used to segment the images of all mitochondria in the cell, calculate the features of each mitochondria and perform dimensionality reduction processing, cluster the mitochondrial features after dimensionality reduction, and obtain the categories of each mitochondria;

[0043] The cell classification module is used to calculate the proportion of each type of mitochondria in the cell in which it is located based on the categories of each mitochondria obtained by the cell and mitochondria category acquisition module, determine the characteristics of the cells and perform dimensionality reduction processing, cluster the cell characteristics after dimensionality reduction, and obtain the cell category.

[0044] Furthermore, the mitochondrial classification module uses the connected domain method to obtain the following mitochondrial features:

[0045] a. Mitochondrial length l: It is defined as the centerline length of the skeletonized mitochondrial segmentation image x, as shown in the following formula (1);

[0046] l=SUM(skeleton(x)) (1)

[0047] In the formula, the pixel value of the mitochondria part in the mitochondrial segmentation image x is 1, the pixel value of the background part is 0, skeleton is the skeletonization function, and SUM is the summation function;

[0048] b. Mitochondrial volume V: It is defined as the number of pixels in the mitochondrial segmentation image x, as shown in the following formula (2);

[0049] V=SUM(x) (2)

[0050] c. Mitochondrial cross-sectional area S: It is defined as the ratio of mitochondrial volume V to length l, as shown in the following formula (3);

[0051] S=V / l (3)

[0052] d. The Euler characteristic number of mitochondria En: It is defined as the Euler characteristic number of the connected area where the mitochondrial segmentation image x is located, as shown in the following formula (4);

[0053] En=χ(x) (4)

[0054] In the formula, χ is the Euler characteristic;

[0055] e. Solidity of mitochondria: It is defined as the ratio of the volume V of the mitochondria to the convex set volume Covex(x), as shown in the following formula (5);

[0056] Solidity=V / Covex(x) (5)

[0057] f. The length of the major axis and minor axis of the mitochondria: It is defined as the length moment of the major axis and minor axis of the circumscribed ellipse / ellipsoid surrounding the mitochondrial segmentation image x, as shown in the following formula (6);

[0058] moment=Eig(x) (6)

[0059] Where, Eig is the eigenvalue of the mitochondrial segmentation image x;

[0060] g. Average fluorescence brightness of mitochondria It is defined as the average value of the fluorescence brightness I(x) of the connected area where the mitochondrial segmentation image x is located;

[0061] h. Standard deviation of mitochondrial fluorescence brightness std(I(x)): It is defined as the standard deviation of the fluorescence brightness I(x) of the connected area where the mitochondrial segmentation image x is located;

[0062] i. Mitochondrial circumference p;

[0063] j. Circularity of mitochondria: It is defined as the following formula (7):

[0064]

[0065] k. Number of terminal nodes of mitochondria;

[0066] l. The number of crossover nodes of mitochondria.

[0067] Furthermore, the method for obtaining the number of end nodes and cross nodes of mitochondria includes:

[0068] The binary images corresponding to all mitochondria in the cell are skeletonized to obtain the mitochondrial skeleton. The degree of the node of each pixel in the mitochondrial skeleton is calculated. The degree value of the end node is 1, and the degree value of the cross node is 2.

[0069] Furthermore, the “characteristics of cells” in the cell classification module include the following characteristics:

[0070] a. Polar distribution of mitochondria: It is defined as the divergence between the angular or radial distribution of each mitochondria and the uniform distribution in the polar coordinate system / spherical coordinate system with the geometric center of the cell as the coordinate origin; wherein, the method for obtaining the "divergence between the angular / radial distribution of each mitochondria and the uniform distribution" includes: first, in the binary image corresponding to all mitochondria in the cell that are not point-shaped, according to the angle θ and the Euclidean distance r between each mitochondrial pixel P(x,y) on the binary image and the coordinate origin, the angular distribution of the corresponding mitochondria is obtained according to the angle θ, and the radial distribution of the corresponding mitochondria is obtained according to the Euclidean distance r; then, according to the angular / radial distribution of the mitochondria, within the range of [0,2π], the θ and r of all mitochondrial pixels P(x,y) are statistically histogrammed, thereby obtaining the divergence between the histogram distribution and the uniform distribution, and then obtaining the polar distribution of the mitochondria;

[0071] b. The number of mitochondria in the cell;

[0072] c. The sum of the volumes of all mitochondria in the cell.

[0073] Furthermore, the method for obtaining the “binary image” includes:

[0074] First, the imaging grayscale distribution of the received low signal-to-noise ratio image is adjusted to a uniform and easy-to-segment high signal-to-noise ratio image. Then, the features of the high signal-to-noise ratio image are extracted in the order of low-level features, intermediate features, and gradually transitioned to high-level features to obtain a feature map. Then, the features in the feature map are matched and weighted to obtain each pixel in the high signal-to-noise ratio image. A binary map is output according to the probability that each pixel is a mitochondrial area.

[0075] Since the feature content defined in the present invention has clear mathematical formulas, physical and morphological meanings, it has very good interpretability, which allows cells of different scales, modalities, cell types and batches to be analyzed in a unified manner, which provides favorable support for the accumulation of cell-related data, as well as its comprehensiveness and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 It is a flowchart of a method for cell classification and analysis of mitochondrial subtypes based on super-resolution images according to an embodiment of the present invention.

[0077] Figure 2 It is a schematic diagram of the result obtained by clustering mitochondrial features using T-SNE in an embodiment of the present invention.

[0078] Figure 3 yes Figure 2Schematic diagram of the actual morphological characteristics of different mitochondrial categories.

[0079] Figure 4 It is a schematic diagram of the distribution of different types of cells and LDA division according to the three mitochondrial features extracted in an embodiment of the present invention. DETAILED DESCRIPTION

[0080] In the drawings, the same or similar reference numerals are used to represent the same or similar elements or elements with the same or similar functions. The embodiments of the present invention are described in detail below in conjunction with the drawings.

[0081] In the description of the present invention, the terms "center", "longitudinal", "lateral", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the scope of protection of the present invention.

[0082] like Figure 1 As shown, the cell classification and analysis method based on super-resolution image mitochondrial subtypes provided by the embodiment of the present invention includes:

[0083] Step 1: Segment the image based on all the mitochondria in the cell ( Figure 1 The mitochondrial segmentation results in the image are obtained by calculating the characteristics of each mitochondria and performing dimensionality reduction processing. The mitochondrial characteristics after dimensionality reduction are clustered to obtain the categories of each mitochondria.

[0084] In one embodiment, step 1 uses a connected domain method to obtain the following mitochondrial features:

[0085] a. Mitochondrial length l: It is defined as the centerline length of the skeletonized mitochondrial segmentation image x, as shown in the following formula (1);

[0086] l=SUM(skeleton(x)) (1)

[0087] In the formula, the pixel value of the mitochondrial part in x is 1, the pixel value of the background part is 0, skeleton is the skeletonization function, and SUM is the summation function.

[0088] b. Mitochondrial volume V: It is defined as the number of pixels in the mitochondrial segmentation image x, as shown in the following formula (2);

[0089] V=SUM(x) (2)

[0090] c. Mitochondrial cross-sectional area S: It is defined as the ratio of mitochondrial volume V to length l, as shown in the following formula (3);

[0091] S=V / l (3)

[0092] d. The Euler characteristic number of mitochondria En: It is defined as the Euler characteristic number of the connected area where the mitochondrial segmentation image x is located, as shown in the following formula (4);

[0093] En=χ(x) (4)

[0094] Where χ is the Euler characteristic, which is used to describe an invariant of a topological object.

[0095] e. Solidity of mitochondria: It is defined as the ratio of the volume V of the mitochondria to the convex volume Convex(x), as shown in the following formula (5);

[0096] Solidity=V / Covex(x) (5)

[0097] In the formula, the specific value of Covex(x) is usually obtained by a global threshold segmentation method, or by a neural network-based method, or by other existing methods.

[0098] f. The length of the major axis and minor axis of the mitochondria moment: It is defined as the length of the major axis and minor axis of the circumscribed ellipse / ellipsoid surrounding the mitochondrial segmentation image x, as shown in the following formula (6);

[0099] moment=Eig(x) (6)

[0100] Where Eig is the eigenvalue of the mitochondrial segmentation image x.

[0101] g. Average fluorescence brightness of mitochondria It is defined as the average value of the fluorescence brightness I(x) of the connected area where the mitochondrial segmentation image x is located;

[0102] h. Standard deviation of mitochondrial fluorescence brightness std(I(x)): It is defined as the standard deviation of the fluorescence brightness I(x) of the connected area where the mitochondrial segmentation image x is located;

[0103] i. Mitochondrial circumference p;

[0104] j. Circularity of mitochondria: It is defined as the following formula (7):

[0105]

[0106] k. Number of terminal nodes of mitochondria;

[0107] l. The number of crossover nodes of mitochondria.

[0108] It should be noted that, in addition to being obtained by the above-mentioned embodiments, the characteristics of mitochondria can also be obtained by morphological, dynamic characteristics, etc. For example, if mitochondria are not represented by pixels, but described by certain functions, or imaged by other modalities, the characteristics of mitochondria can also be obtained by other existing methods.

[0109] In the above embodiment, the method for obtaining the number of end nodes and cross nodes of mitochondria includes:

[0110] The binary images corresponding to all mitochondria in the cell are skeletonized to obtain the mitochondrial skeleton. Finally, the degree of the node of each pixel in the mitochondrial skeleton is calculated. The degree value of the end node is 1, and the degree value of the cross node is 2.

[0111] In one embodiment, the “mitochondrial segmentation image” can be obtained by a super-resolution image mitochondrial segmentation and tracking device, which includes a noise reduction network, a segmentation network, and a mitochondrial tracking module.

[0112] Among them, the denoising network is used to receive low signal-to-noise ratio images. The low signal-to-noise ratio image can be a super-resolution mitochondrial image obtained by confocal microscopy, structured light super-resolution microscopy or stimulated emission depletion microscopy. It is known that various factors such as different exposure times, fluorescence efficiency of dyes, camera parameters, etc. may cause the mitochondria and background photographed by the machine to have different grayscale values. In this embodiment, the denoising network adjusts the imaging grayscale distribution of the low signal-to-noise ratio image to a uniform and easy-to-segment high signal-to-noise ratio image, and outputs it. The imaging grayscale distribution here is adjusted to be uniform and easy to segment, which can be understood as the change of pixel values ​​in the image within the range of [0, 1]. Therefore, the denoising network can eliminate the problem of inconsistent grayscale values ​​in the image caused by the above factors to a certain extent.

[0113] The denoising network is trained using a deep learning network, which can use but is not limited to the ResUNet network architecture. During training, the denoising network uses low signal-to-noise ratio images signal(x,y) obtained by imaging from different machines as input. Therefore, it is necessary to shoot time series data of fixed samples of low signal-to-noise ratio, generally more than 10 frames. In order to obtain different low signal-to-noise ratio images, it is possible to change, for example, the exposure time, dye concentration and / or shooting mode. During training, the denoising network uses the high signal-to-noise ratio image target(x,y) as the prediction result and output. Specifically, a single-frame low signal-to-noise ratio image is used as input, and the maximum value of multiple frames of low signal-to-noise ratio images is superimposed to obtain Figure 2The maximum projection high SNR image shown is used as the training target. This method of stacking multiple low SNR images can significantly reduce the noise level and improve the SNR. Finally, the L1 loss function l(x,y) = |signal(x,y)-target(x,y)|, MSE loss SSIM loss and other functions are used as constraints.

[0114] The segmentation network is used to extract features from the high signal-to-noise ratio image in the order of low-level features, intermediate features, and gradually transition to high-level features to obtain a feature map. Among them, low-level features include but are not limited to edges and grayscale values. Intermediate features include shape, contour and texture. High-level features mainly refer to the law of distribution, the way of arrangement, etc. The segmentation network is also used to match and weight the features in the feature map to obtain each pixel in the high signal-to-noise ratio image, the probability that each pixel is a mitochondrial region, and then set an artificial confidence threshold (for example, 0.5) to distinguish the mitochondrial region of the image, and finally output a binary map. Among them, the segmentation network is trained using a deep learning network, and the deep learning network can use but is not limited to the ResUNet network architecture. During training, the segmentation network takes the high signal-to-noise ratio image Max-T output by the denoising network as input. In the data set, the result can be annotated by, for example, manual annotation, that is, marking the pixels of each mitochondria in the high signal-to-noise ratio image Max-T as a training target. The segmentation network outputs a binary map as the prediction result. Finally, functions such as Dice loss or cross entropy loss are used as constraints.

[0115] In one embodiment, in terms of mitochondrial tracking, this embodiment tracks mitochondria based on graph theory methods. The method provided in this embodiment breaks through the limitation of traditional tracking of single mitochondria, and transfers the tracking object from the connected domain to the nodes and edges of graph theory. This concept transforms the traditional object matching problem into a graph matching problem, which greatly reduces the hyperparameters and running time of the tracking algorithm. Mitochondrial tracking is divided into two stages: preprocessing and tracking. The mitochondrial tracking module is used to use the binary map corresponding to the mitochondria as the first binary map, and the binary map corresponding to the mitochondria of shapes other than point-shaped, that is, non-point-shaped mitochondria, as the second binary map.

[0116] The mitochondrial tracking module includes a first mitochondrial matching unit, a topology construction unit, and a second mitochondrial matching unit. The first mitochondrial matching unit is used to match the first binary images of two adjacent frames and output a matching pair of point-shaped mitochondria between the first binary images of the two frames. For example, if the first binary image of the first frame has m point-shaped mitochondria and the first binary image of the second frame has n mitochondria, the matching pair is expressed as an m×n matrix. The element with the i-th row and j-th column of the matrix being 1 indicates that the i-th mitochondria in the first binary image of the first frame matches the j-th mitochondria in the first binary image of the second frame.

[0117] The topology construction unit is used to convert the second binary image into a topology map. The topology construction unit includes a skeletonization subunit, a node category determination subunit, a node distance determination subunit and a topology map construction subunit. The skeletonization subunit is used to skeletonize the second binary image to obtain the mitochondrial skeleton. The node category determination subunit is used to calculate the degree of the node of each pixel in the mitochondrial skeleton to determine the category of the node, and the category of the node is divided into endpoints and intersections. The degree of the node of each mitochondrial skeleton pixel is obtained by using the hitmiss transformation (hit-miss transformation). In the case where the image to be tracked is two-dimensional, the node with a degree of 2 is marked as an ordinary node, and the node with a degree of 1 is marked as an endpoint, otherwise it is marked as an intersection. In the case where the image to be tracked is three-dimensional, appropriate deformation can be made on the basis of the above two-dimensional situation to obtain the degree of the node of each mitochondrial skeleton pixel. The node distance determination subunit is used to calculate the distance between any two nodes. Among them, the distance can be either Euclidean distance, Minkowski distance, geodesic distance, or other distances. The topology map construction subunit is used to construct a topology map according to the node categories and the distance between any two nodes, combined with the endpoints and intersections.

[0118] The second mitochondrial matching unit is used to first determine each connected subgraph in the topological graph as a mitochondrion, and then use the VF2 algorithm to calculate the isomorphic mapping, directly match the isomorphic connected subgraphs between the two adjacent frame topological graphs, and finally match the non-isomorphic connected subgraphs between the two adjacent frame topological graphs, and output the matching pairs between the nodes of the two frame topological graphs. Isomorphism means that the nodes between the two topological graphs can correspond one to one. The second mitochondrial matching unit includes a node traversal subunit, an isomorphism judgment subunit and a non-isomorphic connected subgraph matching subunit. The node traversal subunit is used to traverse each node in the topological graph, obtain a connected subgraph, and determine the connected subgraph as a mitochondrion. Among them, the method of traversing each node in the topological graph can be, for example, taking one of the nodes, and continuously performing a depth-first search until returning to the original node, so that a connected subgraph can be obtained. Then perform the above operation on the nodes that have not been traversed just now until all nodes are traversed. The isomorphism judgment subunit is used to determine whether the connected subgraphs of the two adjacent frame topological graphs are isomorphic. If so, the mapping relationship between each node in the two connected subgraphs is calculated and output. The non-isomorphic connected subgraph matching subunit is used to calculate the mapping relationship between the nodes of the two connected subgraphs determined as non-isomorphic by the isomorphism judgment subunit, and then perform matching, and finally output the matching pairs between the nodes of the two topological graphs. Preferably, the non-isomorphic connected subgraph matching subunit adopts distance measurement methods such as minimizing graph edit distance, maximum flow minimum cut method, and Hungarian algorithm based on Euclidean distance to calculate the mapping relationship between the nodes of the two connected subgraphs determined as non-isomorphic by the isomorphism judgment subunit. Figure 1The first patent can be used to obtain the segmentation, and the second patent can be used to obtain the segmentation. How to obtain the segmentation is not the main content of this patent.

[0119] The above embodiment provides a method for obtaining a binary image by a neural network method, and in fact, those skilled in the art can also obtain a binary image by a direct binarization method, for example: by manually selecting or selecting a suitable division threshold according to the characteristics of the image histogram distribution, and then setting the part greater than the threshold as the foreground, and the other part as the background. The method for selecting the threshold here includes but is not limited to OTSU (Otsu method), Yen method and trigonometric method, etc. In terms of scope, the above method belongs to the global threshold method. In addition, the selection of the threshold value may not be performed globally, for example, the above method can be applied in a given small area to select, and finally the binarization results of different positions are pieced together.

[0120] After the above mitochondrial clustering is completed, each mitochondria can obtain the category to which it belongs. Assuming that mitochondria with similar morphology have similar functions, the proportion of mitochondria of different categories corresponds to different cell states. By counting the number of each mitochondrial category in each cell, the proportion of each category can be obtained.

[0121] Step 2, first according to the cell ( Figure 1 The cell boundary segmentation result in step 1 and the categories of each mitochondria obtained in step 1 are used to calculate the proportion of each type of mitochondria in the cell where it is located, determine the characteristics of the cell and perform dimensionality reduction processing, cluster the cell characteristics after dimensionality reduction, and obtain the cell category.

[0122] Among them, the method of "calculating the proportion of each type of mitochondria in its cell" can determine the proportion of each type of mitochondria in its cell based on the statistical number of each type of mitochondria in each cell.

[0123] In one embodiment, the "characteristics of the cells" in step 2 include the following characteristics:

[0124] a. Polar distribution of mitochondria: It is defined as the divergence between the angular or radial distribution of each mitochondria and the uniform distribution in the polar coordinate system / spherical coordinate system with the geometric center of the cell as the coordinate origin; wherein, the method for obtaining the "divergence between the angular / radial distribution of each mitochondria and the uniform distribution" includes: first, in the binary image corresponding to all mitochondria in the cell that are not point-shaped, according to the angle θ and the Euclidean distance r between each mitochondrial pixel P(x,y) on the binary image and the coordinate origin, the angular distribution of the corresponding mitochondria is obtained according to the angle θ, and the radial distribution of the corresponding mitochondria is obtained according to the Euclidean distance r; then, according to the angular / radial distribution of the mitochondria, within the range of [0,2π], the θ and r of all mitochondrial pixels P(x,y) are statistically histogrammed, thereby obtaining the divergence between the histogram distribution and the uniform distribution using divergence calculation methods such as JT divergence or KL divergence or Wesselstein distance, and then obtaining the polar distribution of the mitochondria;

[0125] b. The number of mitochondria in the cell n;

[0126] c. The sum of the volumes of all mitochondria in the cell ∑V i , V i is the volume of the i-th mitochondria in the cell.

[0127] The embodiment of the present invention further provides a cell classification and analysis device based on super-resolution image mitochondrial subtypes, which includes a mitochondrial classification module and a cell classification module, wherein:

[0128] The mitochondrial classification module is used to segment the images of all mitochondria in the cell, calculate the features of each mitochondria and perform dimensionality reduction processing, cluster the mitochondrial features after dimensionality reduction, and obtain the categories of each mitochondria.

[0129] The cell classification module is used to calculate the proportion of each type of mitochondria in the cell in which it is located based on the categories of each mitochondria obtained by the cell and mitochondria category acquisition module, determine the characteristics of the cells and perform dimensionality reduction processing, cluster the cell characteristics after dimensionality reduction, and obtain the cell category.

[0130] In one embodiment, the mitochondrial classification module uses the connected domain method provided in the above embodiment to obtain the characteristics of the mitochondria: the length l of the mitochondria, the volume V of the mitochondria, the cross-sectional area S of the mitochondria, the Euler characteristic number En of the mitochondria, the solidity of the mitochondria, the length moment of the major axis and the minor axis of the mitochondria, and the average fluorescence brightness of the mitochondria. The standard deviation of mitochondrial fluorescence brightness std(I(x)), the circumference of mitochondria, the circularity of mitochondria, the number of mitochondrial end nodes, and the number of mitochondrial cross nodes.

[0131] In one embodiment, the “characteristics of cells” in the cell classification module include: the polarity distribution of mitochondria, the number of mitochondria in the cell, and the sum of the volumes of all mitochondria in the cell.

[0132] In the above embodiments, commonly used dimensionality reduction methods include PCA (full name in English is "Principal Component Analysis", full name in Chinese is "Principal Component Analysis"), tSNE (full name in English is "t-Distributed Stochastic Neighbor Embedding", full name in Chinese is "t-distributed random neighbor embedding"), Umap (full name in English is "Uniform Manifold Approximation and Projection", full name in Chinese is "Uniform Manifold Approximation and Projection"), etc. For example, PCA remaps the original feature space to a new space with fewer dimensions through orthogonal transformation, while retaining most of the dimensional features of the variance. Figure 2 As shown, Figure 2 It is a projection diagram of the feature distribution in the low-dimensional feature space after using T-SNE to uniformly reduce the dimensionality of all mitochondria in multiple cells at different stages.

[0133] In the above embodiments, the clustering method may use K-Means, mixed Gaussian, UMAP and other clustering methods. For example, when using K-Means to cluster mitochondrial features, first, randomly select k category centers as the original cluster centers, then calculate the distance to each cluster center for each sample in the data set, and divide it into the category of the cluster center closest to it, and finally recalculate the category center of each category, and repeat the previous two steps until the location of the cluster center no longer changes. Figure 3 and Figure 4 As shown, Figure 3 It is through Figure 2 After clustering the mitochondria in the sample, the morphological characteristics of the same category of mitochondria from different cells. Figure 4 The clustering method LDA linear discriminant analysis is used to illustrate that after the dimensionality reduction of the cell features, cells in different stages can be distinguished by a straight line in the low-dimensional space (two-dimensional space). This shows that the stage of the cell can be distinguished based on the morphological characteristics of the mitochondria in the cell. After testing in actual living cell samples, compared with other methods, the present invention takes into account the location distribution characteristics of mitochondria in cells and can better capture the specific changes in mitochondrial morphology in different cells.

[0134] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Those skilled in the art should understand that the technical solutions described in the above embodiments may be modified, or some of the technical features thereof may be replaced by equivalents; these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for cell classification and analysis of mitochondrial subtypes based on super-resolution images, characterized in that: include: Step 1: Based on the segmented images of all mitochondria in the cell, the features of each mitochondria are calculated and dimension reduction is performed, and the mitochondrial features after dimension reduction are clustered to obtain the categories of each mitochondria; Step 2: According to the cell and the categories of each mitochondria obtained in step 1, the proportion of each type of mitochondria in the cell is calculated, the characteristics of the cell are determined and dimensionality reduction is performed, and the cell characteristics after dimensionality reduction are clustered to obtain the cell category.

2. The method for cell classification and analysis of mitochondrial subtypes based on super-resolution images according to claim 1, characterized in that: Step 1 includes obtaining the characteristics of mitochondria using a connected domain method; the characteristics of mitochondria include: a. Mitochondrial length l: It is defined as the centerline length of the skeletonized mitochondrial segmentation image x, as shown in the following formula (1); l=SUM(skeleton(x)) (1) In the formula, the pixel value of the mitochondria part in the mitochondrial segmentation image x is 1, the pixel value of the background part is 0, skeleton is the skeletonization function, and SUM is the summation function; b. Mitochondrial volume V: It is defined as the number of pixels in the mitochondrial segmentation image x, as shown in the following formula (2); V=SUM(x) (2) c. Mitochondrial cross-sectional area S: It is defined as the ratio of mitochondrial volume V to length l, as shown in the following formula (3); S=V / l (3) d. The Euler characteristic number of mitochondria En: It is defined as the Euler characteristic number of the connected area where the mitochondrial segmentation image x is located, as shown in the following formula (4); En=χ(x) (4) In the formula, χ is the Euler characteristic; e. Solidity of mitochondria: It is defined as the ratio of the volume V of the mitochondria to the convex set volume Covex(x), as shown in the following formula (5); Solidity=V / Covex(x) (5) f. The length of the major axis and minor axis of the mitochondria: It is defined as the length moment of the major axis and minor axis of the circumscribed ellipse / ellipsoid surrounding the mitochondrial segmentation image x, as shown in the following formula (6); moment=Eig(x) (6) Where, Eig is the eigenvalue of the mitochondrial segmentation image x; g. Average fluorescence brightness of mitochondria It is defined as the average value of the fluorescence brightness I(x) of the connected area where the mitochondrial segmentation image x is located; h. Standard deviation of mitochondrial fluorescence brightness std(I(x)): It is defined as the standard deviation of the fluorescence brightness I(x) of the connected area where the mitochondrial segmentation image x is located; i. Mitochondrial circumference p; j. Circularity of mitochondria: It is defined as the following formula (7): k. Number of terminal nodes of mitochondria; l. The number of crossover nodes of mitochondria.

3. The method for cell classification and analysis of mitochondrial subtypes based on super-resolution images according to claim 2, characterized in that: The method for obtaining the number of end nodes and cross nodes of mitochondria includes: The binary images corresponding to all mitochondria in the cell are skeletonized to obtain the mitochondrial skeleton. The degree of the node of each pixel in the mitochondrial skeleton is calculated. The degree value of the end node is 1, and the degree value of the cross node is 2.

4. The method for cell classification and analysis of mitochondrial subtypes based on super-resolution images according to claim 2, characterized in that: The "characteristics of cells" in step 2 include the following characteristics: a. Polar distribution of mitochondria: It is defined as the divergence between the angular or radial distribution of each mitochondria and the uniform distribution in the polar coordinate system or spherical coordinate system with the geometric center of the cell as the coordinate origin; wherein, the method for obtaining "the divergence between the angular / radial distribution of each mitochondria and the uniform distribution" includes: first, in the binary image corresponding to all mitochondria in the cell that are not point-shaped, according to the angle θ and the Euclidean distance r between each mitochondrial pixel P(x,y) on the binary image and the coordinate origin, the angular distribution of the corresponding mitochondria is obtained according to the angle θ, and the radial distribution of the corresponding mitochondria is obtained according to the Euclidean distance r; then, according to the angular / radial distribution of the mitochondria, within the range of [0,2π], the θ and r of all mitochondrial pixels P(x,y) are statistically histogrammed, thereby obtaining the divergence between the histogram distribution and the uniform distribution, and then obtaining the polar distribution of the mitochondria; b. The number of mitochondria in the cell; c. The sum of the volumes of all mitochondria in the cell.

5. The method for cell classification and analysis of mitochondrial subtypes based on super-resolution images according to claim 3 or 4, characterized in that: The methods for obtaining "binary image" include: First, the imaging grayscale distribution of the received low signal-to-noise ratio image is adjusted to a uniform and easy-to-segment high signal-to-noise ratio image. Then, the features of the high signal-to-noise ratio image are extracted in the order of low-level features, intermediate features, and gradually transitioned to high-level features to obtain a feature map. Then, the features in the feature map are matched and weighted to obtain each pixel in the high signal-to-noise ratio image. A binary map is output according to the probability that each pixel is a mitochondrial area.

6. A cell classification and analysis device based on super-resolution image mitochondrial subtypes, characterized in that: include: The mitochondrial classification module is used to segment the images of all mitochondria in the cell, calculate the features of each mitochondria and perform dimensionality reduction processing, cluster the mitochondrial features after dimensionality reduction, and obtain the categories of each mitochondria; The cell classification module is used to calculate the proportion of each type of mitochondria in the cell in which it is located based on the categories of each mitochondria obtained by the cell and mitochondria category acquisition module, determine the characteristics of the cells and perform dimensionality reduction processing, cluster the cell characteristics after dimensionality reduction, and obtain the cell category.

7. The cell classification and analysis device based on super-resolution image mitochondrial subtypes according to claim 6, characterized in that: The mitochondrial classification module uses the connected domain method to obtain the following mitochondrial features: a. Mitochondrial length l: It is defined as the centerline length of the skeletonized mitochondrial segmentation image x, as shown in the following formula (1); l=SUM(skeleton(x)) (1) In the formula, the pixel value of the mitochondria part in the mitochondrial segmentation image x is 1, the pixel value of the background part is 0, skeleton is the skeletonization function, and SUM is the summation function; b. Mitochondrial volume V: It is defined as the number of pixels in the mitochondrial segmentation image x, as shown in the following formula (2); V=SUM(x) (2) c. Mitochondrial cross-sectional area S: It is defined as the ratio of mitochondrial volume V to length l, as shown in the following formula (3); S=V / l (3) d. The Euler characteristic number of mitochondria En: It is defined as the Euler characteristic number of the connected area where the mitochondrial segmentation image x is located, as shown in the following formula (4); En=χ(x) (4) In the formula, χ is the Euler characteristic; e. Solidity of mitochondria: It is defined as the ratio of the volume V of the mitochondria to the convex set volume Covex(x), as shown in the following formula (5); Solidity=V / Covex(x) (5) f. The length of the major axis and minor axis of the mitochondria: It is defined as the length moment of the major axis and minor axis of the circumscribed ellipse / ellipsoid surrounding the mitochondrial segmentation image x, as shown in the following formula (6); moment=Eig(x) (6) Where, Eig is the eigenvalue of the mitochondrial segmentation image x; g. Average fluorescence brightness of mitochondria It is defined as the average value of the fluorescence brightness I(x) of the connected area where the mitochondrial segmentation image x is located; h. Standard deviation of mitochondrial fluorescence brightness std(I(x)): It is defined as the standard deviation of the fluorescence brightness I(x) of the connected area where the mitochondrial segmentation image x is located; i. Mitochondrial circumference p; j. Circularity of mitochondria: It is defined as the following formula (7): k. Number of terminal nodes of mitochondria; l. The number of crossover nodes of mitochondria.

8. The cell classification and analysis device based on super-resolution image mitochondrial subtypes according to claim 7, characterized in that: The method for obtaining the number of end nodes and cross nodes of mitochondria includes: The binary images corresponding to all mitochondria in the cell are skeletonized to obtain the mitochondrial skeleton. The degree of the node of each pixel in the mitochondrial skeleton is calculated. The degree value of the end node is 1, and the degree value of the cross node is 2.

9. The cell classification and analysis device based on super-resolution image mitochondrial subtypes according to claim 7, characterized in that: The "Cell Features" in the Cell Classification Module include the following features: a. Polar distribution of mitochondria: It is defined as the divergence between the angular or radial distribution of each mitochondria and the uniform distribution in the polar coordinate system or spherical coordinate system with the geometric center of the cell as the coordinate origin; wherein, the method for obtaining "the divergence between the angular / radial distribution of each mitochondria and the uniform distribution" includes: first, in the binary image corresponding to all mitochondria in the cell that are not point-shaped, according to the angle θ and the Euclidean distance r between each mitochondrial pixel P(x,y) on the binary image and the coordinate origin, the angular distribution of the corresponding mitochondria is obtained according to the angle θ, and the radial distribution of the corresponding mitochondria is obtained according to the Euclidean distance r; then, according to the angular / radial distribution of the mitochondria, within the range of [0,2π], the θ and r of all mitochondrial pixels P(x,y) are statistically histogrammed, thereby obtaining the divergence between the histogram distribution and the uniform distribution, and then obtaining the polar distribution of the mitochondria; b. The number of mitochondria in the cell; c. The sum of the volumes of all mitochondria in the cell.

10. The cell classification and analysis device based on super-resolution image mitochondrial subtypes according to claim 8 or 9, characterized in that: The methods for obtaining "binary image" include: First, the imaging grayscale distribution of the received low signal-to-noise ratio image is adjusted to a uniform and easy-to-segment high signal-to-noise ratio image. Then, the features of the high signal-to-noise ratio image are extracted in the order of low-level features, intermediate features, and gradually transitioned to high-level features to obtain a feature map. Then, the features in the feature map are matched and weighted to obtain each pixel in the high signal-to-noise ratio image. A binary map is output according to the probability that each pixel is a mitochondrial area.

Images