A fast and high-precision cell recognition and segmentation method

Abstract

This invention discloses a fast and high-precision cell identification and segmentation method, which includes performing affine and magnification transformations on cell images and corresponding masks, calculating the center mask, constructing a multi-coder-autoencoder model, training the network model, using the network model for cell identification and segmentation, and using the watershed algorithm to segment the cell image to obtain individual cell images and masks. This invention overcomes the shortcomings of traditional semantic segmentation-based algorithms in handling cell adhesion, while also incorporating the advantages of instance segmentation algorithms for cell segmentation. At the same time, it avoids the space and computational complexity brought about by using instance segmentation algorithms, and has the beneficial effects of simple model, high compatibility, fast speed and strong scalability.

Description

Technical Field

[0001] This invention relates to the field of cell recognition and segmentation technology, specifically to a fast and high-precision cell recognition and segmentation method. Background Technology

[0002] Currently, traditional cell image processing techniques mainly employ image segmentation algorithms to segment cell instances. Among these, the most commonly used algorithms are those based on threshold segmentation, edge detection, and region growing. While these algorithms are simple and easy to implement, they often result in inaccurate segmentation when faced with complex cell morphology, high background noise, or high segmentation accuracy requirements. With the development of deep learning, especially the rise of semantic segmentation and instance segmentation techniques, more and more researchers are beginning to use deep learning algorithms for cell image processing.

[0003] Threshold-based semantic segmentation algorithms primarily rely on a threshold calculated using the "maximum internal variance" method. Taking fluorescence images as an example, data below this threshold is considered background, while data above is considered foreground. The biggest drawback of this method is its inability to handle complex image scenes, particularly bright-field images, where it struggles to effectively distinguish between impurities and cellular regions. Even with adaptive windowing, this problem cannot be completely solved; in fact, it may mistake background noise for foreground. Furthermore, for organoid images with incomplete fluorescence imaging, the fluorescence signal within the organoid structure is weak, typically requiring a "hole-filling" algorithm to achieve better segmentation results. However, this algorithm still suffers from inaccurate identification of cells or organoid images with severely missing internal signals. Since thresholding only distinguishes between foreground and background and cannot segment individual cell instances, subsequent processing such as the watershed algorithm is necessary.

[0004] Semantic segmentation algorithms based on deep learning can avoid the problem of inaccurate identification by applying deep neural networks or convolutional neural networks to cell segmentation to some extent. However, like thresholding algorithms, this method still relies on the watershed algorithm to segment individual cells. For cell images with severe adhesion, it is still unable to achieve accurate identification of individual cells.

[0005] Instance segmentation algorithms based on deep learning can be categorized into two main types. One type uses recurrent convolutional neural networks to segment instances, but due to the increased complexity of the model, especially for dense cell images, the spatial and temporal complexity of the network is higher than that of simple semantic segmentation algorithms. Algorithms such as MASK R-CNN mainly predict different instances / objects into a multi-layer mask and predict the classification of the object corresponding to each layer of the mask. However, this method is also difficult to apply to thousands of cell images.

[0006] Traditional semantic segmentation algorithms can identify the foreground and background of an image, but they cannot accurately segment cells, thus requiring further processing using algorithms such as flooding. Instance segmentation networks can directly identify the location and contour of each cell, but require more data and computational resources to train and run the model, and have a long response time. To address these issues, this invention proposes a novel fast and high-precision cell instance segmentation algorithm. By comprehensively utilizing multiple advanced techniques such as feature extraction, dimensionality reduction, clustering, model training, and fine-tuning, it achieves high-precision instance segmentation of cell images. Compared to traditional algorithms and instance segmentation networks, this invention has higher segmentation accuracy and faster response time, and does not require a large amount of training data and computational resources. Therefore, this invention has broad application prospects and can be used in fields such as biological research, medical diagnosis, and drug development. Summary of the Invention

[0007] The purpose of this invention is to provide a rapid and high-precision method for cell identification and segmentation to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a rapid and high-precision cell identification and segmentation method, comprising the following steps:

[0009] Step 1: Given any set of cell images and the mask corresponding to each cell image, perform affine and magnification transformations on the image set and the mask respectively to obtain new images. and the corresponding new mask;

[0010] Step 2: Calculate the pixel center of each cell in the new mask sequentially to obtain the center mask;

[0011] Step 3: Construct a multi-coder-autoencoder model, where the new image group is the input to the encoder model and the output of the encoder model is the latent space;

[0012] Step 4: Use the PCA algorithm to reduce the dimensionality of the latent space to obtain three-dimensional hidden variables;

[0013] Step 5: Cluster the hidden variables using the self-clustering algorithm K-Means, turning the hidden variables into several clusters;

[0014] Step 6: Utilize the new image from Step 1 The cell mask recognition network is trained with the new mask and the first network model is obtained. The cell mask recognition network is trained with the new mask group in step 1 and the mask before the change and the second network model is obtained.

[0015] Step 7: Use the clustering methods from Step 5 to train the first network model and the second network model respectively, to obtain the optimized first and second optimized models;

[0016] Step 8: Calculate the mask of the cell image to be segmented using the first optimization model. The mask for the cell image to be segmented is calculated using the second optimization model. Then mask Seed point, Using the foreground as a reference, precise segmentation of cells is achieved to obtain individual cell masks.

[0017] Preferably, in step 3, the multi-encoder-autoencoder model also sets different encoders for different spatial variations. The encoder takes the new image group as input and outputs a latent vector.

[0018] Preferably, the multi-encoder-autoencoder model also includes a decoder, which takes the latent vector as input and outputs the latent space.

[0019] Preferably, step 4 utilizes the hidden space. The specific steps of the algorithm for dimensionality reduction are as follows:

[0020] Step 1: Calculate the average value of each dimension in the latent space;

[0021] Step 2: Subtract the average value of the corresponding dimension from the data in each dimension of the latent space to obtain the centered data matrix;

[0022] Step 3: Calculate the covariance matrix of the centered data matrix;

[0023] Step 4: Calculate the eigenvalues ​​and eigenvectors of the covariance matrix;

[0024] Step 5: Sort the eigenvalues ​​in descending order, and select the eigenvectors corresponding to the three largest eigenvalues ​​to form the projection matrix;

[0025] Step 6: Multiply the centered data matrix by the projection matrix to obtain the dimensionality-reduced 3D hidden variables.

[0026] Preferably, the number of clusters in step 5 is 3. Step 7: Duplicate the first network model three times, and label them as follows: Duplicate the second network model three times, and denote them as follows: .

[0027] Preferred, Training separately and and completed training. and , Training separately and And completed the training. and , Training separately and And completed the training. and .

[0028] Preferably, both the first network model and the second network model in step 6 are based on... As the backbone network, the loss functions of both the first and second network models are the same.

[0029] Preferably, the basis for determining the completion of model training in steps 6 and 7 is that the loss function no longer changes significantly.

[0030] Preferably, the size of the new image group obtained in step 1 is .

[0031] Compared with the prior art, the beneficial effects of the present invention are:

[0032] First, the model is simple and requires no special processing of the data; all existing open-source datasets can be used directly.

[0033] Secondly, it has high compatibility. Because the entire task is handled in a divide-and-conquer manner, some strategies are introduced in the algorithm for determining the cell center to reduce the training difficulty of the entire task. Due to the determination of the cell center, even extremely adherent cells can be segmented well.

[0034] Third, it is fast. Its time complexity is twice that of semantic segmentation. However, because it can be computed in parallel, it can be reduced to the time of ordinary semantic segmentation.

[0035] Fourth, it has strong scalability. Since the segmentation seed point and foreground / background recognition are separated, different sub-modules can be scalable or replaced with better-performing sub-algorithms.

[0036] This invention overcomes the shortcomings of traditional semantic segmentation-based algorithms in handling cell adhesion, while also incorporating the advantages of instance segmentation algorithms for cell segmentation, and avoids the space and computational complexity associated with using instance segmentation algorithms. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the higher-precision front and back view recognition algorithm based on latent feature classification of the present invention.

[0038] Figure 2 This is a schematic diagram illustrating the instance segmentation principle implemented by the dual UNET semantic segmentation network in this invention.

[0039] Figure 3 This is a schematic diagram of the multi-encoder-automatic encoder model of the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] Reference Figures 1-3 A rapid and high-precision cell identification and segmentation method includes the following steps:

[0042] Step 1: Given any set of cell images and the corresponding mask For image groups and mask By performing affine and magnification transformations respectively, a new set of images with a size of 256×256 is obtained. and the corresponding new mask ;

[0043] Step 2: Calculate the images sequentially. Corresponding mask The center mask is obtained by tracing the pixel center of each cell. ;

[0044] Step 3: Construct a multi-coder-autoencoder model, new image group The input to the multi-encoder-autoencoder model is the latent space, and the output is the latent space. ;

[0045] Step 4: Hidden Space Dimensionality reduction is performed using the PCA algorithm to obtain the three-dimensional hidden variables. ;

[0046] Step 5: Hidden variables Clustering is performed using the self-clustering algorithm K-Means to hide variables. It becomes several clusters , , and record as ;

[0047] Step 6: Use the image from Step 1 and mask Train the cell mask recognition network and obtain the first trained network model, using the images from step 1. and mask Train the cell mask recognition network and obtain the trained second network model;

[0048] Step 7: Utilize the clustering from Step 5 The first network model and the second network model are trained separately to obtain the optimized first and second optimized models.

[0049] Step 8: Calculate the mask of the cell image to be segmented using the first optimization model. The mask for the cell image to be segmented is calculated using the second optimization model. Then mask Seed point, Using the foreground as a reference, precise segmentation of cells is achieved to obtain individual cell masks.

[0050] Specifically, the multi-encoder-autoencoder and PCA algorithm are used to extract features and reduce dimensions of the image, and then the K-Means self-clustering algorithm is used for clustering to achieve a refined model for different types of images, thereby improving the accuracy and efficiency of single cell recognition. This invention uses cell feature-based clustering to achieve a refined model for different types of images.

[0051] image and center mask and images and mask Together, they were used to train a semantic segmentation network, achieving instance segmentation of cell images. This method can better distinguish cells from the background and outperforms current algorithms in single-cell recognition accuracy.

[0052] By utilizing the watershed algorithm to achieve accurate cell segmentation, and combining it with the multi-encoder-autoencoder and PCA algorithm, this invention has significant advantages in cell image processing and segmentation.

[0053] refer to Figure 3 In step 3, the multi-encoder-autoencoder model also handles different spatial variations. Different encoders were set. encoder Input is an image The output is a hidden vector. The multi-encoder-auto encoder also includes a decoder. decoder The input is a latent vector The output is the implicit space. .

[0054] Specifically, encoder Constructed using a hybrid of convolutional and fully connected layers, for the same image (i=1,2,...,n), all encoders All outputs are sent to the same implicit space, and then the decoder is used. For all encoders in the implicit space The encoder processes the output to reconstruct the original image. With decoder The purpose of manipulating images is to constrain the model to learn latent features associated with the input image, thus enabling the decoder to... The reconstructed image is compared with the original image to determine whether the feature is a hidden feature.

[0055] The multi-encoder-autoencoder model first uses its encoder part to map the input image into a latent space, where each dimension of the latent space can be regarded as a latent feature. The multi-encoder-autoencoder model then uses its decoder part to generate a new image from the latent space.

[0056] Hidden features are not extracted directly from the original data, such as the roundness of the cell nucleus or the average pixel value of the cell membrane. Hidden features are automatically learned during the training process by the multi-encoder-autoencoder model. Hidden features provide a deeper and more abstract representation, enabling the first and second network models to capture patterns that are not easily recognized in the original data.

[0057] encoder The specific structure is as follows:

[0058] First convolutional layer: kernel size is... Step size is 1, padding is 1, number of output channels is 32, and output size is [missing information]. tensor;

[0059] First pooling layer: using Max pooling, output size is The image;

[0060] Second convolutional layer: kernel size is... Step size is 1, padding is 1, number of output channels is 64, and output size is [missing information]. tensor;

[0061] Second pooling layer: using Max pooling, output size is tensor;

[0062] Third convolutional layer: kernel size is... Step size is 1, padding is 1, number of output channels is 128, and output size is [missing information]. tensor;

[0063] Third pooling layer: using Max pooling, output size is The image;

[0064] Flatten: to flatten the dimensions of The image is flattened as ( )-dimensional vector;

[0065] First fully connected layer: Input A 1024-dimensional vector will be output.

[0066] The second fully connected layer takes a 1024-dimensional input and outputs a d-dimensional hidden vector (e.g., d=128).

[0067] decoder The specific structure is as follows:

[0068] First fully connected layer: The input is a d-dimensional hidden vector, and the output is a 1024-dimensional vector;

[0069] Second fully connected layer: Input is 1024-dimensional, output is... dimensional vector;

[0070] Reshaping: dimensional vector reshaping to tensor;

[0071] First anti-pooling layer: for tensor The nearest neighbor upsampling output size is ;

[0072] First deconvolutional layer: kernel size is... Step size is 1, padding is 1, number of input channels is 128, number of output channels is 64, output size is [missing information]. tensor;

[0073] Second anti-pooling layer: for tensor The nearest neighbor upsampling output size is ;

[0074] Second deconvolutional layer: kernel size is... Step size is 1, padding is 1, number of input channels is 64, number of output channels is 32, and output size is [missing information]. ;

[0075] Third anti-pooling layer: for The output size of the tensor after 2x2 nearest neighbor upsampling is... ;

[0076] Third deconvolutional layer: kernel size is... Step size is 1, padding is 1, number of input channels is 32, number of output channels is 1, and output size is [missing information]. ;

[0077] The training of multi-encoder-autoencoder models relies on variational inference. The calculation formula is used to evaluate the optimization level of the model, that is, to determine whether the model meets the requirements. The calculation formula is as follows:

[0078]

[0079] in, This represents the joint probability distribution of the generative model. This indicates that the encoder operates on a given input image. After spatial transformation The conditional probability distribution of the subsequently generated latent vector z. Let z be a random variable derived from the variational distribution q(z). Indicates in Distribution The average value, through continuous searching The maximum value enables the model to better encode and reconstruct the input image.

[0080] refer to Figure 1 and Figure 2 Step 4 will implicit space use The specific steps of the algorithm for dimensionality reduction are as follows:

[0081] First, the hidden space needs to be calculated. The mean of each dimension, using express:

[0082]

[0083] in, Representing implicit space The number of features in each dimension .

[0084] Next, the implicit space Subtract the average value of the corresponding dimension from the data in each dimension. To obtain a centralized data matrix :

[0085]

[0086] Recalculate the centralized data matrix Covariance matrix:

[0087]

[0088] Next, we calculate the eigenvalues ​​and eigenvectors of the covariance matrix C. The eigenvectors represent the directions of the principal components, and the eigenvalues ​​represent the variance along the directions of the principal components.

[0089]

[0090] Sort the eigenvalues ​​in descending order, and select the eigenvectors corresponding to the three largest eigenvalues ​​to form the projection matrix. :

[0091]

[0092] Finally, the centralized data matrix Multiply by the projection matrix The three-dimensional hidden variables are obtained after dimensionality reduction. In this way, the PCA algorithm is used to reduce the dimensionality of the latent space Z to three-dimensional hidden variables. :

[0093]

[0094] refer to Figure 1 and Figure 2 In step 5, the number of clusters is 3. Step 7: Duplicate the first network model three times, and label them as follows: Duplicate the second network model three times, and denote them as follows: , Training separately and And completed the training. and , Training separately and And completed the training. and , Training separately and And completed the training. and In step 6, both the first and second network models are based on... The loss functions of the backbone network, the first network model, and the second network model. Similarly, steps 6 and 7 determine the completion of training based on the loss function. No further significant changes occurred.

[0095] Specifically, images and center mask The training set is used to train the first network model, resulting in the trained first network model, and the image. and mask The training set is used to train the second network model, resulting in the trained second network model. The loss functions of the first and second network models are also described. for:

[0096]

[0097] and These are the predicted mask and the actual mask, respectively. For intersection, union, and comparison, This represents the area of ​​the intersection between the predicted mask and the true mask. This represents the area of ​​the union of the predicted mask and the ground truth mask, when the two bounding boxes do not overlap at all. =0; when the two bounding boxes completely overlap, The value is 1; therefore, A larger value indicates a greater degree of overlap between the two bounding boxes, while a smaller value indicates a smaller degree of overlap. To evaluate the model's performance.

[0098] Specifically, the first network model is copied three times, and denoted as follows: network model Make three copies, and record them as follows: Then use clustering Training is for the purpose of improving the model. and model Fine-tuning was performed, and the first optimized model was obtained. Second optimization model .

[0099] Specifically, using the first optimization model Calculate the mask of the cell image. Using the second optimization model Calculate the cell image Then, the Watershed algorithm is used to achieve accurate cell segmentation and obtain the individual cell mask. The specific steps are as follows:

[0100] First, use a mask As a prospect, As seed points, calculate the mask. The distance transformation can be achieved by calculating the Euclidean distance:

[0101]

[0102] in (x) is the distance transformation function.

[0103] use For each seed point, assign a unique label; these labels will be used to distinguish different cells.

[0104]

[0105] To ensure the seed point is separated from the background, the background is set to 0, the seed point label is appropriately offset, and the seed point is marked. Increment the data within by 1.

[0106] Change the distance and seed point marking The cells are segmented using the Watershed algorithm, starting with seed point marking. Starting from each marker point, the algorithm follows the gradient descent direction of the distance transform until it encounters other cell boundaries or the background, segmenting the identified boundaries:

[0107]

[0108] in (x) is the watershed algorithm function.

[0109] After obtaining the segmented cell image, the labels are converted back to the original binary mask format by setting the value of each non-zero pixel to 1:

[0110]

[0111] This provides the Watershed algorithm, which enables precise segmentation of cells and yields individual cell masks.

[0112] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A rapid and high-precision cell identification and segmentation method, comprising the following steps: Step 1: Given any set of cell images and the mask corresponding to each cell image, perform affine and magnification transformations on the image set and the mask respectively to obtain a new image set and a corresponding new mask; Step 2: Calculate the pixel center of each cell in the new mask sequentially to obtain the center mask; Step 3: Construct a multi-coder-autoencoder model, where the new image group is the input to the encoder model and the output of the encoder model is the latent space; Step 4: Use the PCA algorithm to reduce the dimensionality of the latent space to obtain three-dimensional hidden variables; Step 5: Cluster the hidden variables using the self-clustering algorithm K-Means, turning the hidden variables into several clusters; Step 6: Use the new image set and new mask from Step 1 to train the cell mask recognition network and obtain the first network model after training. Use the new mask set from Step 1 and the mask before the change to train the cell mask recognition network and obtain the second network model after training. Step 7: Use the clustering methods from Step 5 to train the first network model and the second network model respectively, to obtain the optimized first and second optimized models; Step 8: Calculate the mask of the cell image to be segmented using the first optimization model. The mask for the cell image to be segmented is calculated using the second optimization model. Then mask Seed point, Using the foreground as a reference, precise segmentation of cells is achieved to obtain individual cell masks; In step 3, the multi-encoder-autoencoder model also handles different spatial variations. Different encoders were set. encoder Input is an image The output is a hidden vector. The multi-encoder-auto encoder also includes a decoder. decoder The input is a latent vector The output is the implicit space. .

2. The rapid and high-precision cell identification and segmentation method according to claim 1, characterized in that: In step 3, the multi-encoder-autoencoder model also sets different encoders for different spatial variations. The encoder takes the new image group as input and outputs a latent vector.

3. The rapid and high-precision cell identification and segmentation method according to claim 2, characterized in that: The multi-encoder-autoencoder model also includes a decoder, which takes the latent vector as input and outputs the latent space.

4. The rapid and high-precision cell identification and segmentation method according to claim 1, characterized in that: Step 4: Utilize the hidden space The specific steps of the algorithm for dimensionality reduction are as follows: Step 1: Calculate the average value of each dimension in the latent space; Step 2: Subtract the average value of the corresponding dimension from the data in each dimension of the latent space to obtain the centered data matrix; Step 3: Calculate the covariance matrix of the centered data matrix; Step 4: Calculate the eigenvalues ​​and eigenvectors of the covariance matrix; Step 5: Sort the eigenvalues ​​in descending order, and select the eigenvectors corresponding to the three largest eigenvalues ​​to form the projection matrix; Step 6: Multiply the centered data matrix by the projection matrix to obtain the dimensionality-reduced 3D hidden variables.

5. The rapid and high-precision cell identification and segmentation method according to claim 1, characterized in that: In step 5, the number of clusters is 3. Step 7: Duplicate the first network model three times, and label them as follows: Duplicate the second network model three times, and denote them as follows: .

6. The rapid and high-precision cell identification and segmentation method according to claim 5, characterized in that: Training separately and And completed the training. and , Training separately and And completed the training. and , Training separately and And completed the training. and .

7. The rapid and high-precision cell identification and segmentation method according to claim 1, characterized in that: In step 6, both the first and second network models are based on... As the backbone network, the loss functions of both the first and second network models are the same.

8. The rapid and high-precision cell identification and segmentation method according to claim 7, characterized in that: The basis for determining whether the model training is complete in steps 6 and 7 is that the loss function no longer changes significantly.

Images