Stem cell classification method and system based on image segmentation
Through image segmentation and deep learning technology, the noise, insufficient contrast and morphological complexity in stem cell classification are solved, and the stem cell classification with high accuracy and high efficiency is achieved. It is suitable for a variety of microscopic image types and provides intuitive visual results .
Patent Information
- Application Number
- CN202411237424.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-05
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-09-05
AI Technical Summary
The existing stem cell classification methods have problems with noise, insufficient contrast and cell morphology complexity, resulting in low classification accuracy and difficulty in capturing small differences in different types of stem cells.
Using an image segmentation-based method, noise is removed through mean filtering and adaptive histogram equalization, contrast is enhanced, and multi-scale cell features are extracted using multi-level deep learning networks, combined with conditional random field post-processing to achieve accurate segmentation, and high-dimensional Cell feature vectors, using dual-path deep learning classification and attention mechanism fusion, use conditions to generate adversarial networks to reconstruct multi-class cell images.
It improves the accuracy and efficiency of stem cell classification, can automatically process multiple types of microscopic images, has broad applicability, and provides intuitive classification visualization results, enhancing the interpretability of the method.
Smart Images

Figure CN118762362B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of cell image segmentation, and in particular to a stem cell classification method and system based on image segmentation. Background Art
[0002] Traditional stem cell classification methods mainly rely on manual observation and empirical judgment, which are highly subjective, inefficient, and difficult to apply on a large scale. With the development of computer vision and artificial intelligence technology, automated stem cell classification methods based on image analysis have gradually attracted attention.
[0003] However, existing stem cell classification methods based on image analysis still face many challenges. Stem cell microscopic images usually have problems such as noise and insufficient contrast, which affect the classification accuracy. Secondly, the complexity of cell morphology and structure makes accurate cell segmentation difficult. Furthermore, the subtle differences between different types of stem cells are difficult to be effectively captured by traditional algorithms, resulting in limited classification performance. Summary of the invention
[0004] The present application provides a stem cell classification method and system based on image segmentation, and the present application improves the accuracy of stem cell classification.
[0005] In a first aspect, the present application provides a stem cell classification method based on image segmentation, the stem cell classification method based on image segmentation comprising:
[0006] Perform mean filtering and equalization processing on stem cell microscopic images to obtain standardized high-resolution images;
[0007] Inputting the standardized high-resolution image into a multi-level deep learning network for feature extraction to obtain a multi-scale cell feature map;
[0008] Performing cell segmentation and post-processing on the multi-scale cell feature map to obtain a precisely segmented cell region mask;
[0009] Extracting geometric features, texture features and intensity features from the precisely segmented cell region mask to obtain a high-dimensional cell feature vector;
[0010] Performing dual-path deep learning classification and fusion on the high-dimensional cell feature vector and the stem cell microscopic image to obtain cell category probability distribution and segmentation results;
[0011] A multi-category cell image is reconstructed based on the cell category probability distribution and the segmentation result to obtain a classification visualization result.
[0012] In a second aspect, the present application provides a stem cell classification system based on image segmentation, the stem cell classification system based on image segmentation comprising:
[0013] An imaging module is used to perform mean filtering and equalization processing on stem cell microscopic images to obtain standardized high-resolution images;
[0014] An extraction module, used for inputting the standardized high-resolution image into a multi-level deep learning network for feature extraction to obtain a multi-scale cell feature map;
[0015] A segmentation module, used for performing cell segmentation and post-processing on the multi-scale cell feature map to obtain a precisely segmented cell region mask;
[0016] A processing module, used for extracting geometric features, texture features and intensity features from the accurately segmented cell region mask to obtain a high-dimensional cell feature vector;
[0017] A classification module, used for performing dual-path deep learning classification and fusion on the high-dimensional cell feature vector and the stem cell microscopic image to obtain a cell category probability distribution and segmentation result;
[0018] A visualization module is used to reconstruct multi-category cell images based on the cell category probability distribution and the segmentation results to obtain classification visualization results.
[0019] In the technical solution provided by the present application, the image noise is effectively removed and the contrast is enhanced by using technologies such as non-local mean filtering and adaptive histogram equalization. The deep learning network containing residual connection and feature pyramid structure can extract multi-scale cell features, which is conducive to capturing the subtle differences of different types of stem cells. Based on the automatic cell segmentation algorithm, combined with conditional random field post-processing, accurate cell region segmentation can be achieved, providing accurate spatial positioning for feature extraction. By extracting the geometric, texture and intensity features of the core area, edge area and transition area, a high-dimensional cell feature vector is constructed to comprehensively characterize the morphological characteristics of the cell. The dual-path deep learning classification and attention mechanism fusion are used to make full use of the feature vector and original image information to improve the classification accuracy. The conditional generative adversarial network is used to reconstruct multi-category cell images, providing intuitive classification visualization results and enhancing the interpretability of the method. The entire classification process from image preprocessing to result output is completed automatically, which greatly improves the efficiency of stem cell classification. It can process various types of stem cell microscopic images, including bright field, fluorescence, confocal and electron microscopic images, and has a wide range of applicability. Through a variety of image enhancement and noise reduction techniques, the method's adaptability to different imaging conditions and image qualities is improved, thereby improving the accuracy of stem cell classification. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.
[0021] Figure 1 This is a schematic diagram of an embodiment of a stem cell classification method based on image segmentation in an embodiment of the present application;
[0022] Figure 2 This is a schematic diagram of an embodiment of a stem cell classification system based on image segmentation in an embodiment of the present application. DETAILED DESCRIPTION
[0023] The embodiments of the present application provide a method and system for stem cell classification based on image segmentation. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described here can be implemented in an order other than that illustrated or described here. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0024] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 In the embodiments of the present application, an embodiment of the stem cell classification method based on image segmentation includes:
[0025] Step 101, performing mean filtering and equalization processing on the stem cell microscopic image to obtain a standardized high-resolution image;
[0026] It is understandable that the execution subject of the present application may be a stem cell classification system based on image segmentation, or a terminal or a server, which is not limited here. The present application embodiment is described by taking a server as the execution subject as an example.
[0027] Specifically, high-resolution imaging of stem cell microscopic images is performed to obtain various types of original images including bright field, fluorescence, confocal and electron microscopic images, each of which has unique optical properties and resolution. Non-local mean filtering is applied to various types of original images to effectively reduce noise while retaining image edge information to obtain filtered images. The filtered images are divided into non-overlapping rectangular areas, and histogram equalization is performed on each rectangular area to enhance the contrast of the image, making the tiny cell structure more obvious. Multi-scale Retinex analysis is performed on the contrast-enhanced image. The Retinex algorithm is an image enhancement method based on the human visual system. Through this method, the illumination non-uniformity in the image is eliminated to obtain an illumination-equalized image. Illumination equalization not only improves the visual effect of the image, but also makes the illumination conditions between different images more consistent. The illumination-equalized image is size-standardized to adjust it to a uniform size, eliminate the size differences caused by different microscopes or imaging conditions, and ensure the consistency and comparability of subsequent feature extraction and classification. The pixel intensity of the image of uniform size is normalized to normalize the pixel intensity value to a fixed range to obtain a standardized high-resolution image.
[0028] Step 102, inputting the standardized high-resolution image into a multi-level deep learning network for feature extraction to obtain a multi-scale cell feature map;
[0029] Specifically, the standardized high-resolution image is input into the multi-level deep learning network, and the image is convolved using a 3×3 convolution kernel to obtain the first initial feature map. The convolution operation helps to extract low-level features in the image, such as edges and textures. The first initial feature map is input into the residual block, and the residual feature map is obtained by executing F1=F0+ReLU(Conv(ReLU(Conv(F0)))), where F1 represents the residual feature map, Conv is the convolution operation, ReLU is the activation function, and F0 represents the first initial feature map. The introduction of the residual block can effectively solve the problem of gradient disappearance in the deep network, ensuring the effective transmission of features and the flow of gradients. The residual feature map is subjected to a maximum pooling operation to obtain the first feature map. The purpose of the maximum pooling operation is to reduce the size of the feature map while retaining the most important feature information, thereby reducing the amount of calculation and the number of parameters. After completing the first round of feature extraction and pooling, the residual block processing and maximum pooling operation are repeated three times to obtain the second feature map, the third feature map, and the fourth feature map of different scales, respectively. These feature maps represent image features at different scales, covering feature information from low level to high level. A 1×1 convolution kernel is applied to the fourth feature map for channel compression to obtain the top feature map. The most important feature information is extracted by reducing the number of channels in the feature map. The top feature map is upsampled and element-wise added and fused with the third feature map to generate the first fused feature map. Element-wise addition fusion can combine information from different scales to enrich the feature expression in the feature map. Similar upsampling and fusion operations are performed on the first fused feature map, which is fused with the second feature map and the first feature map respectively to obtain the second fused feature map and the third fused feature map. Through this series of operations, features from different scales are gradually fused to form a feature map set containing rich and diverse information. The third fused feature map, the second fused feature map, the first fused feature map and the top feature map are combined to construct a feature pyramid to form a multi-scale feature set. A 3×3 convolution operation is performed on each feature map in the multi-scale feature set, and the number of channels is adjusted to make the channel dimensions of all feature maps consistent and standardize the morphology of the feature maps. The feature map set of uniform dimension is concatenated in the channel dimension to obtain the final multi-scale cell feature map.
[0030] Step 103, performing cell segmentation and post-processing on the multi-scale cell feature map to obtain an accurately segmented cell region mask;
[0031] Specifically, a 3×3 convolution operation is performed on the multi-scale cell feature map, and an activation operation is performed through the ReLU activation function to obtain a second initial feature map. A downsampling operation is performed on the second initial feature map to reduce the size of the feature map step by step to obtain a first encoder feature map, a second encoder feature map, a third encoder feature map, and a fourth encoder feature map. Channel compression is performed on the fourth encoder feature map to reduce the computational complexity while retaining the most important feature information to generate a bridge feature map. An upsampling operation is performed on the bridge feature map to restore the size of the feature map to a larger scale, and it is spliced with the third encoder feature map to form a first decoder feature map. A 3×3 convolution operation and a ReLU activation function operation are performed on the first decoder feature map to enhance the nonlinear expression ability of the feature. The processed feature map is spliced with the second encoder feature map and the first encoder feature map to obtain a second decoder feature map and a third decoder feature map. Multiple splicing operations help to gradually restore the spatial resolution of the image in the decoding stage, while integrating feature information from different levels to improve segmentation accuracy. A 1×1 convolution operation is performed on the third decoder feature map to reduce the number of channels and generate an initial segmentation mask. The initial segmentation mask is activated by the Sigmoid function to generate a probability map P. The probability map P represents the probability distribution of each pixel belonging to the cell region. The probability map is segmented by threshold to generate a binary mask. The binary mask and the original stem cell microscopic image are subjected to conditional random field analysis to optimize the segmentation results. Conditional random field analysis can effectively correct misclassified pixels and improve segmentation accuracy by considering the relationship between pixels. Morphological operations such as dilation and erosion are performed on the optimized segmentation mask to eliminate small noise points and fine artifacts, and to repair possible segmentation breaks to obtain an accurately segmented cell region mask.
[0032] The optimal threshold of the probability map P is calculated by the Otsu algorithm. The Otsu algorithm is a widely used automated threshold selection method that determines the optimal threshold point by maximizing the inter-class variance, thereby effectively distinguishing the background and foreground in the image. The probability map is binarized according to the adaptive threshold to generate an initial binary mask, which preliminarily defines the boundaries of the cell region. The initial binary mask is subjected to connected domain analysis to mark different connected regions in the mask, effectively identify independent cell populations in the image, and obtain a marked mask. Based on the marked mask, the area of each connected region is calculated to form a set of region areas. This step helps to determine the size of each connected domain, and based on this information, the connected regions with an area smaller than the preset threshold are removed to achieve denoising of the initial binary mask. The denoised binary mask is combined with the original stem cell microscopy image to construct a conditional random field (CRF) graph model. CRF is a probabilistic framework based on a graph model that can well capture the contextual relationship between pixels. In this model, a unary potential energy is calculated for each node (i.e., pixel in the image), reflecting the possibility that each pixel belongs to a cell or background, and a unary potential energy map is formed. At the same time, for each pair of adjacent nodes in the graph model, their pairwise potential energy is calculated, which takes into account the relative position and color similarity between pixels, thereby generating a pairwise potential energy map. After obtaining the unary potential energy map and the pairwise potential energy map, the mean field inference algorithm is used for iterative update to gradually optimize the edge distribution of each node. The mean field inference algorithm reduces the inconsistent parts in the potential energy map through multiple iterations to obtain the optimized probability distribution. The maximum a posteriori probability inference is performed on the optimized probability distribution, and the most likely label assignment is selected from it to accurately distinguish cells and background, and generate an optimized segmentation mask.
[0033] Step 104, extracting geometric features, texture features and intensity features from the accurately segmented cell region mask to obtain a high-dimensional cell feature vector;
[0034] Specifically, the distance transformation is performed on the precisely segmented cell region mask to generate a distance map. Distance transformation is to calculate the distance from each pixel in the image to the nearest edge. Through the distance map, the cell region is divided into a core region, a transition region and an edge region, which correspond to the three regions of the center, transition and periphery, respectively, to obtain a three-region mask. Based on the obtained three-region mask, the area, perimeter and circularity of each region are calculated to obtain a geometric feature vector. The area and perimeter provide basic morphological information of the region, while the circularity measures the degree to which the shape of the region is close to a circle, which is helpful for cell morphological analysis. The grayscale co-occurrence matrix is calculated for the core region, transition region and edge region of the stem cell microscopic image, respectively, to obtain the texture matrix of each region. Through the texture matrix, the energy, contrast, correlation and entropy features are calculated to form a texture feature vector. Energy describes the uniformity of the image, contrast reflects the brightness difference of the image, correlation represents the linear dependence of the image, and entropy quantifies the complexity of the image. The average intensity, standard deviation, skewness and kurtosis of the three regions of the stem cell microscopic image are calculated to obtain the intensity feature vector. The average intensity provides the brightness information of the image, the standard deviation reflects the degree of discreteness of the brightness distribution, the skewness indicates the asymmetry of the brightness distribution, and the kurtosis measures the sharpness of the brightness distribution. These features can capture the brightness characteristics of the cell. The Zernike moment is calculated for the cell region mask to obtain the shape feature vector. The Zernike moment is a commonly used shape descriptor that can describe the shape characteristics of the image with high precision. The area ratio of the core area to the transition area and the area ratio of the core area to the edge area are calculated to obtain the regional ratio feature vector. The ratio feature reflects the relative area relationship between different regions inside the cell. The geometric feature vector, texture feature vector, intensity feature vector, shape feature vector and regional ratio feature vector are spliced to form a high-dimensional cell feature vector.
[0035] Step 105, performing dual-path deep learning classification and fusion on the high-dimensional cell feature vector and the stem cell microscopic image to obtain cell category probability distribution and segmentation results;
[0036] Specifically, a 3×3 convolution operation is performed on the stem cell microscopic image, and the ReLU activation function is applied to obtain the initial image features. The convolution operation extracts low-level features such as edges and textures in the image, while the ReLU activation function introduces nonlinear factors to increase the complexity of feature expression. The initial image features are input into the residual network for deep feature extraction. The residual network solves the gradient vanishing problem in deep network training through jump connections and effectively captures the high-level feature information of the image. The extracted deep image features are globally averaged pooled, and the high-dimensional feature map is reduced to a low-dimensional feature vector by calculating the average value of the feature map to form the global feature of the image. At the same time, the high-dimensional cell feature vector is analyzed by the fully connected layer and the ReLU activation function is used to obtain the transformed cell features, which contain information extracted from multiple aspects such as geometry, texture, intensity and shape. The global image features and the transformed cell features are spliced to generate fused image features. The self-attention mechanism is analyzed for the fused image features. By calculating the correlation between features, the weight of the features can be dynamically adjusted to highlight the key features and generate attention-weighted features. The attention weighted features are input into the fully connected layer for processing to generate output features. The Softmax function is applied to the output features to calculate the probability distribution of each category and obtain the probability distribution of cell categories. Based on these probability distributions, the category with the highest probability is selected as the classification result to determine the category of each cell. The classification result is matched with the accurately segmented cell area mask at the pixel level to obtain the segmentation result.
[0037] The global features of the image are input into the image path hidden layer, and the transformed cell features are input into the feature path hidden layer. The two hidden layers process feature data from different paths respectively. Feature concatenation is performed in the feature path hidden layer and the image path hidden layer to combine the two types of feature data to form fused image features. The fused image features are processed using a weight matrix to generate intermediate features. The weight matrix plays a role in selecting and enhancing important features in this process. A bias vector is added to the intermediate features to generate bias features. The introduction of additional degrees of freedom further adjusts the distribution of features to make them more suitable for subsequent processing. The hyperbolic tangent function (tanh) is performed on the bias features to generate activation features. The tanh function is a commonly used activation function that can effectively handle nonlinear relationships and limit feature values to a certain range, thereby improving the stability of the model. The weight vector analysis of the activated features is performed to calculate the attention score of each feature. The score reflects the importance of each feature and determines their weight in the final output. The Softmax function is applied to the attention score to obtain the normalized attention weight. The Softmax function can convert the score into a probability distribution to ensure that the sum of the weights of all features is 1. The attention weight is decomposed into the first path attention weight and the second path attention weight, which correspond to the attention weights of the cell features and the global image features, respectively. In the feature path hidden layer, the transformed cell features are subjected to attention weighted analysis according to the first path attention weight to generate weighted feature path features. Similarly, in the image path hidden layer, the global image features are subjected to attention weighted analysis according to the second path attention weight to generate weighted image path features. The weighted feature path features and the weighted image path features are element-wise added to obtain the attention weighted features by combining the weighted information of the two types of features.
[0038] Step 106: reconstruct multi-category cell images based on the cell category probability distribution and the segmentation results to obtain classification visualization results.
[0039] Specifically, the probability distribution of cell categories and the segmentation results are concatenated to form a comprehensive conditional vector. A random noise vector is concatenated with the conditional vector to form the input of the generator, so that the generator can generate diverse images while satisfying the information contained in the conditional vector. The input of the generator is subjected to the first layer of transposed convolution operation to generate the first layer of feature maps. The transposed convolution operation plays the role of upsampling and extracts the low-level features of the image through the convolution kernel. The first layer of feature maps is subjected to the second layer of transposed convolution operation to generate the second layer of feature maps, which gradually improves the resolution and details of the image. The second layer of feature maps is subjected to the third layer of transposed convolution operation to generate the third layer of feature maps, and then the fourth layer of transposed convolution operation is performed to generate cell images. Multiple transposed convolution operations gradually enlarge the image size and reconstruct multi-category cell images containing cell structures. The generated cell images are concatenated with the conditional vector to form the input of the discriminator. The discriminator provides feedback by judging whether the input image is real (from the dataset) or generated (from the generator). The discriminator input passes through the first layer of convolution operation to generate the first layer of discriminant features, capturing the basic morphological features of the image. The first layer of discriminant features passes through the second layer of convolution operation, the third layer of convolution operation, and the fourth layer of convolution operation in turn, gradually extracting higher-level features to generate discrimination results. Each layer of convolution operation further analyzes the details and features of the image to ensure the accuracy of the discrimination. Based on the generated cell image, discrimination results, and cell category probability distribution, a classification visualization result is generated. The result contains the classification information, segmentation results, and corresponding probability distribution of each cell, and is visualized in the form of an image.
[0040] In the embodiment of the present application, by using techniques such as non-local mean filtering and adaptive histogram equalization, image noise is effectively removed and contrast is enhanced. A deep learning network containing residual connections and feature pyramid structures is used to extract multi-scale cell features, which is conducive to capturing the subtle differences of different types of stem cells. Based on the automatic cell segmentation algorithm, combined with conditional random field post-processing, accurate cell region segmentation can be achieved, providing accurate spatial positioning for feature extraction. By extracting the geometric, texture and intensity features of the core area, edge area and transition area, a high-dimensional cell feature vector is constructed to comprehensively characterize the morphological characteristics of the cell. The dual-path deep learning classification and attention mechanism fusion are used to make full use of the feature vector and original image information, thereby improving the classification accuracy. The conditional generative adversarial network is used to reconstruct multi-category cell images, providing intuitive classification visualization results and enhancing the interpretability of the method. The entire classification process is automatically completed from image preprocessing to result output, which greatly improves the efficiency of stem cell classification. It can process various types of stem cell microscopic images, including bright field, fluorescence, confocal and electron microscopic images, and has a wide range of applicability. Through a variety of image enhancement and noise reduction techniques, the method's adaptability to different imaging conditions and image qualities is improved, thereby improving the accuracy of stem cell classification.
[0041] In a specific embodiment, the process of executing step 101 may specifically include the following steps:
[0042] Perform high-resolution imaging of stem cell microscopy images to obtain multiple types of original images including bright field, fluorescence, confocal, and electron microscopy images;
[0043] Perform non-local mean filtering on various types of original images to obtain filtered images;
[0044] The filtered image is divided into non-overlapping rectangular regions, and each rectangular region is subjected to histogram equalization to obtain a contrast-enhanced image;
[0045] Perform multi-scale Retinex analysis on the contrast-enhanced image to obtain an illumination-balanced image, and then standardize the size of the illumination-balanced image to obtain an image of uniform size.
[0046] The pixel intensity of images of uniform size is normalized to obtain standardized high-resolution images.
[0047] Specifically, high-resolution imaging of stem cell microscopic images was performed to obtain various types of original images, including bright field, fluorescence, confocal, and electron microscopic images. Bright field microscopy uses transmitted light to generate contrast images of samples and is one of the most common microscopy imaging methods; fluorescence microscopy produces light of specific colors by exciting fluorescent molecules in the sample, providing information about specific components inside the cell; confocal microscopy provides higher resolution and contrast through point scanning and optical sectioning, which is suitable for three-dimensional imaging of thick samples; electron microscopy uses electron beams to provide ultra-high resolution images of sample surface structures. Non-local mean filtering is performed on various types of original images to reduce noise and retain image details. Non-local mean filtering is a denoising method that smoothes each pixel by calculating the similarity between each pixel and surrounding pixels. The basic formula is:
[0048] ;
[0049] in, Represents pixels The filtered intensity value at Represents pixels The original intensity value, is a weight function, which represents the pixel With pixels The filtered image is divided into non-overlapping rectangular regions, and each rectangular region is subjected to histogram equalization to improve the contrast. Histogram equalization enhances the overall contrast of the image by adjusting the grayscale distribution of the image, making the details clearer. For each region, the formula for histogram equalization can be expressed as:
[0050] ;
[0051] in, It is pixel level The gray value after equalization, is the maximum value of the gray level, and are the number of rows and columns of the image, respectively. The gray level is The number of pixels. This process makes pixels of different gray levels more evenly distributed in the image, enhancing the contrast and visibility of the image. Perform multi-scale Retinex analysis on the histogram-equalized image. Multi-scale Retinex analysis adjusts the illumination balance of the image by simulating the human visual system's ability to perceive illumination and reflection components of different scales. By decomposing the image into multiple scales and performing Retinex processing at each scale, an illumination-balanced image is obtained, reducing the brightness changes caused by uneven illumination, making the illumination conditions of all areas in the image more consistent. The illumination-balanced image is then size-normalized to ensure that all images have a uniform size. Different microscopic images are scaled and cropped to meet the predetermined size standards. The formula for size normalization can be expressed as:
[0052] ;
[0053] in, is the normalized image, and are the number of rows and columns of the new image, and is the number of rows and columns of the original image. Pixel intensity normalization is performed on images of uniform size to eliminate intensity differences caused by different imaging conditions. Pixel intensity normalization usually adjusts each pixel value to a predetermined range, such as between 0 and 1, so that the brightness and contrast of different images are on the same level, ensuring that all images are comparable and consistent in subsequent analysis.
[0054] In a specific embodiment, the process of executing step 102 may specifically include the following steps:
[0055] S1: Input the standardized high-resolution image into the multi-stage deep learning network, and use a 3×3 convolution kernel to perform a convolution operation on the standardized high-resolution image to obtain the first initial feature map;
[0056] S2: Input the first initial feature map into the residual block, execute F1=F0+ReLU(Conv(ReLU(Conv(F0)))), and obtain the residual feature map, where F1 is the residual feature map, Conv is the convolution operation, ReLU is the activation function, and F0 represents the first initial feature map;
[0057] S3: Perform a maximum pooling operation on the residual feature map to obtain a first feature map;
[0058] S4: Repeat steps S2 and S3 three times to obtain a second feature map, a third feature map, and a fourth feature map of different scales respectively;
[0059] S5: Perform channel compression on the fourth feature map through a 1×1 convolution kernel to obtain a top feature map;
[0060] S6: up-sample the top feature map, and perform element-level addition fusion with the third feature map to obtain a first fused feature map;
[0061] S7: Repeat the operation of step S6 on the first fused feature map, and fuse it with the second feature map and the first feature map respectively to obtain a second fused feature map and a third fused feature map;
[0062] S8: combining the third fused feature map, the second fused feature map, the first fused feature map and the top feature map into a feature pyramid to obtain a multi-scale feature set;
[0063] S9: Perform a 3×3 convolution operation on each feature map in the multi-scale feature set, adjust the number of channels, and obtain a feature map set of uniform dimension;
[0064] S10: Concatenate the feature maps of uniform dimension in the channel dimension to obtain a multi-scale cell feature map.
[0065] Specifically, the standardized high-resolution image is input into the first layer of the multi-level deep learning network, and a 3×3 convolution kernel is used to perform a convolution operation on the image to extract low-level features of the image, such as edges and textures. The convolution operation can be described as: ,in, is the output feature map of the convolution operation, Represents the input standardized high-resolution image, and Conv represents the convolution operation. The size of the convolution kernel is 3×3, which slides on the input image to extract features in the local area. Enter the residual block. The residual block calculates the output feature map by doing the following: In this formula, Represents the residual feature map, Conv represents the convolution operation, and ReLU is the activation function (rectified linear unit). Apply convolution and ReLU activation, convolution and ReLU activation again, and finally combine the result with the original input feature map Add. This design helps alleviate the gradient vanishing problem in deep networks, ensures that gradients can be effectively propagated, and can capture more complex features. Perform a maximum pooling operation on the obtained residual feature map to obtain the first feature map. Maximum pooling is a downsampling operation that reduces the size of the feature map by taking the maximum value in the local area, while retaining important features, reducing the dimension of the data, improving computational efficiency, and helping to enhance the translation invariance of the feature map. Repeat steps S2 and S3 three times to obtain the second feature map, the third feature map, and the fourth feature map of different scales. Each repetition continues convolution, activation, and pooling on the basis of the previous layer of feature maps, gradually extracting higher-level features and capturing more complex and abstract image feature information. These feature maps represent the information of the image at different scales, providing multi-level feature expressions. Use a 1×1 convolution kernel to compress the channels of the fourth feature map to obtain the top feature map. 1×1 convolution is used to adjust the number of channels of the feature map without affecting its spatial dimension. In this way, the amount of calculation is reduced while retaining key information. The top feature map is upsampled and element-wise fused with the third feature map to obtain a first fused feature map. The upsampling operation is used to restore the spatial resolution of the feature map to ensure that the two feature maps match in the spatial dimension when fused. Repeat the operation of step S6 for the first fused feature map, and fuse it with the second feature map and the first feature map respectively to obtain a second fused feature map and a third fused feature map. Gradually fusion of deeper features with shallower features to form a complete feature representation, the multi-scale fusion method can significantly improve the richness of feature expression. The third fused feature map, the second fused feature map, the first fused feature map and the top feature map are combined into a feature pyramid to obtain a multi-scale feature set. The feature pyramid is a structured representation that contains feature maps at different scales, which helps to capture various information from global to local. Perform a 3×3 convolution operation on each feature map in the multi-scale feature set, adjust the number of channels, and obtain a feature map set of unified dimension, ensuring that the number of channels of all feature maps is consistent, which is convenient for subsequent processing and analysis. The feature map set of unified dimension is spliced in the channel dimension to obtain a multi-scale cell feature map.
[0066] In a specific embodiment, the process of executing step 103 may specifically include the following steps:
[0067] Perform 3×3 convolution operation and ReLU activation function operation on the multi-scale cell feature map to obtain the second initial feature map;
[0068] Performing a downsampling operation on the second initial feature map to obtain a first encoder feature map, a second encoder feature map, a third encoder feature map, and a fourth encoder feature map;
[0069] Perform channel compression on the fourth encoder feature map to obtain a bridge feature map, upsample the bridge feature map, and concatenate the bridge feature map with the third encoder feature map to obtain a first decoder feature map;
[0070] Perform a 3×3 convolution operation and a ReLU activation function operation on the first decoder feature map, and concatenate it with the second encoder feature map and the first encoder feature map to obtain a second decoder feature map and a third decoder feature map;
[0071] Perform a 1×1 convolution operation on the third decoder feature map to obtain an initial segmentation mask, and perform a Sigmoid activation function operation on the initial segmentation mask to obtain a probability map P;
[0072] Perform threshold segmentation on the probability map to obtain a binary mask, and perform conditional random field analysis on the binary mask and the stem cell microscopic image to obtain an optimized segmentation mask;
[0073] Morphological operations are performed on the optimized segmentation mask to obtain the accurately segmented cell region mask.
[0074] Specifically, a 3×3 convolution operation and a ReLU activation function operation are performed on the multi-scale cell feature map to extract more detailed features and introduce nonlinear changes to obtain the second initial feature map. By sliding a small-sized convolution kernel (3×3) on the feature map, the feature details of the local area can be captured. The formula is expressed as: ,in is the input multi-scale cell feature map, Conv represents the convolution operation, ReLU is the rectified linear unit activation function, Then it is the second initial feature map of the output. The second initial feature map is downsampled to reduce the size of the feature map to obtain the first encoder feature map, the second encoder feature map, the third encoder feature map and the fourth encoder feature map. The downsampling operation is usually implemented through maximum pooling. Each pooling extracts the maximum value from the feature map, reduces the size of the data and retains important feature information. This process helps to reduce computational complexity while enhancing the model's ability to abstract image features. After completing the encoding stage, the fourth encoder feature map is channel compressed to obtain a bridge feature map. The channel compression in this step uses Convolution is done, the purpose of which is to reduce the computational burden by reducing the number of channels in the feature map while retaining important feature information. The bridge feature map is upsampled to increase the spatial resolution of the feature map. Upsampling can be achieved by methods such as bilinear interpolation or deconvolution, which enlarges the low-resolution feature map to the same size as the shallower feature map. The upsampled bridge feature map is concatenated with the third encoder feature map to obtain the first decoder feature map. The concatenation operation combines the feature maps from the encoding stage and the decoding stage in the channel dimension, fusing low-level and high-level feature information so that the model can better recover the details of the image. The first decoder feature map is again convolved with a 3×3 convolution operation and a ReLU activation function to refine the information in the feature map, and then the result is concatenated with the second encoder feature map and the first encoder feature map to obtain the second decoder feature map and the third decoder feature map, respectively. The third decoder feature map is convolved with a 1×1 convolution operation to generate the initial segmentation mask. Similar to channel compression, the multi-channel feature map is converted into a single-channel segmentation map by reducing the number of channels. The Sigmoid activation function is then applied to the initial segmentation mask to generate a probability map P. The Sigmoid function compresses the output between 0 and 1 so that the value of each pixel represents the probability that it belongs to the cell region. Threshold segmentation is performed on the probability map P to determine the pixels that belong to the cell region. The process of threshold segmentation can be expressed as:
[0075] ;
[0076] in is the mask after binarization, is the probability map with coordinates The pixel value at is the set threshold. The choice of threshold directly affects the accuracy of the segmentation result. Conditional random field (CRF) analysis is performed on the binary mask and stem cell microscopy image. CRF can refine the segmentation results and repair misclassified pixels by considering the contextual relationship between pixels. In the CRF model, the unary potential function represents the class probability of a single pixel, and the pairwise potential function represents the consistency between adjacent pixels. These are iteratively updated to finally obtain the optimized segmentation mask. Morphological operations such as dilation and erosion are performed on the optimized segmentation mask to eliminate small noise points and detail defects, and obtain an accurately segmented cell region mask. For example, when segmenting stem cell microscopy images, the dilation operation can help expand the cell boundary to make it smoother, while the erosion operation can remove small artifacts and noise, thereby improving the segmentation accuracy.
[0077] In a specific embodiment, the execution step performs threshold segmentation on the probability map to obtain a binary mask, and performs conditional random field analysis on the binary mask and the stem cell microscopic image to obtain an optimized segmentation mask. The process may specifically include the following steps:
[0078] The optimal threshold of the probability map P is calculated by the Otsu algorithm to obtain an adaptive threshold, and the probability map is binarized according to the adaptive threshold to obtain an initial binary mask;
[0079] Perform connected domain analysis on the initial binary mask, mark different connected areas, and obtain a marked mask;
[0080] The area of each connected region of the marked mask is calculated to obtain a region area set, and the connected regions with an area smaller than a preset threshold are removed according to the region area set to obtain a denoised binary mask;
[0081] A conditional random field graph model was constructed for the denoised binary mask and stem cell microscopic images;
[0082] Calculate the unary potential energy for each node in the conditional random field graph model to obtain a unary potential energy graph, and calculate the pairwise potential energy for each pair of adjacent nodes in the graph model to obtain a pairwise potential energy graph;
[0083] Based on the unary potential energy graph and the pairwise potential energy graph, the mean field inference algorithm is used to iteratively update the edge distribution of each node to obtain the optimized probability distribution;
[0084] Perform maximum a posteriori probability inference on the optimized probability distribution to obtain the optimized segmentation mask.
[0085] Specifically, for the probability graph Perform the optimal threshold calculation of the Otsu algorithm to determine the adaptive threshold for binarization. The Otsu algorithm finds the optimal threshold by maximizing the inter-class variance. It assumes that the image consists of two classes, foreground and background, and calculates the maximum value of the inter-class variance under all possible thresholds. The goal of the Otsu algorithm is to find the threshold Maximize the following formula:
[0086] ;
[0087] in, and The thresholds are The probability of foreground and background, and are the average grayscale values of the foreground and background respectively. According to the calculated optimal threshold , for the probability graph Perform binarization to obtain the initial binary mask. Set all pixels in the image with values greater than or equal to Pixels with a value of 1 are marked as foreground (usually 1), and pixels smaller than The initial binary mask is marked as background (usually 0). Connected domain analysis is performed on the initial binary mask to mark different connected regions. The purpose of connected domain analysis is to identify interconnected groups of pixels in the image, which represent different independent objects or regions. In the process of connected domain analysis, a unique label is assigned to each connected domain to distinguish each independent region. The marked mask is further processed to calculate the area of each connected region and form a set of regional areas. The area size of these regions is obtained by calculating the number of pixels contained in each connected region. According to the data in the set of regional areas, the connected regions with an area less than a preset threshold are removed to obtain the denoised binary mask. This process helps to remove small noise points or irrelevant small areas in the image, ensuring that only sufficiently large areas that may be stem cells are retained. A conditional random field (CRF) graph model is constructed for the denoised binary mask and the stem cell microscopy image. CRF is a probabilistic graph model that is suitable for labeling pixels in sequence data or image data. The labeling sequence is predicted by maximizing the conditional probability. In this model, each pixel is regarded as a node, and the edges between nodes represent their mutual relationship. The unary potential energy is calculated for each node in the CRF graph model to obtain a unary potential energy map. The unary potential energy represents the probability that a single pixel belongs to a certain category (such as foreground or background), which is usually calculated by the difference between the pixel value and the expected value. The pairwise potential energy is calculated for each pair of adjacent nodes in the graph model to obtain a pairwise potential energy map. The pairwise potential energy reflects the relationship between adjacent pixels, such as edge consistency, which helps to maintain spatial continuity. In this model, the pairwise potential energy can be used to penalize adjacent pixels with inconsistent labels to enhance the smoothness of image segmentation. Based on the unary potential energy map and the pairwise potential energy map, the edge distribution of each node is iteratively updated using the mean field inference algorithm, and the edge distribution of each node is gradually optimized. The mean field inference repeatedly updates the class distribution of each pixel to make it tend to the optimal state. In each iteration, this algorithm updates the edge distribution of each node based on the current model parameters and the information of the neighboring nodes. The maximum a posteriori probability inference is performed on the optimized probability distribution to determine the final category of each pixel and obtain the optimized segmentation mask. The maximum a posteriori probability inference selects the class that maximizes the posterior probability as the final classification result. The formula is:
[0088] ;
[0089] in, is the final category label, Represents a given observation Post Category This method effectively combines the local information and global information of pixels, and makes the segmentation result more accurate through the optimization of the conditional random field model.
[0090] In a specific embodiment, the process of executing step 104 may specifically include the following steps:
[0091] Perform distance transformation on the accurately segmented cell region mask to obtain a distance map;
[0092] According to the distance map, the cell region is divided into the core region, the transition region and the edge region to obtain a three-region mask. The area, perimeter and circularity of the three-region mask are calculated respectively to obtain the geometric feature vector.
[0093] Grayscale co-occurrence matrix calculation was performed on three regions of stem cell microscopic images to obtain a texture matrix, and energy, contrast, correlation and entropy features were calculated based on the texture matrix to obtain a texture feature vector;
[0094] The mean intensity, standard deviation, skewness, and kurtosis were calculated for the three regions of the stem cell microscopic image to obtain the intensity feature vector;
[0095] The Zernike moment of the cell region mask is calculated to obtain the shape feature vector;
[0096] Calculate the area ratio between the core area and the transition area, and the area ratio between the core area and the edge area, and obtain the regional proportion feature vector;
[0097] The geometric feature vector, texture feature vector, intensity feature vector, shape feature vector and area ratio feature vector are concatenated to obtain a high-dimensional cell feature vector.
[0098] Specifically, the distance transform is performed on the accurately segmented cell region mask to generate a distance map. The distance from each pixel to the nearest background pixel (i.e., the edge of the cell region) is calculated. The commonly used method is the Euclidean distance transform, and the result is a distance map in which each pixel value represents the distance to the nearest edge. The formula is:
[0099] ;
[0100] in, is the position in the distance map The distance value at represents the coordinates of the boundary pixels, is a collection of boundary pixels. The value of each pixel in the distance map represents its distance from the nearest background edge. According to the distance map, the cell area is divided into a core area, a transition area, and an edge area, which is usually achieved by setting multiple thresholds. For example, one-third of the maximum distance is selected as the threshold of the core area, two-thirds as the threshold of the transition area, and the rest as the edge area, and three area masks are obtained, representing different cell parts. The three-area masks are subjected to geometric feature analysis, including calculation of area, perimeter, and circularity. Area calculation is to count the number of pixels in the area. The perimeter is to calculate the number of pixels at the edge of the area, or it is obtained by edge detection algorithm. Circularity is a measure of the degree to which the shape of the area is close to a circle. The value is 1 when it is completely circular and less than 1 when it is other shapes. Through calculation, the geometric feature vector describing the cell morphology is obtained. The grayscale co-occurrence matrix is calculated for the three areas of the stem cell microscopy image. The grayscale co-occurrence matrix describes the statistics of texture by calculating the grayscale relationship of pixel pairs in the image. According to the obtained grayscale co-occurrence matrix, the energy, contrast, correlation, and entropy features are calculated. Energy represents the uniformity of the image, contrast reflects the grayscale difference, correlation represents the linear relationship between pixels, and entropy measures the complexity of the image. The formula is as follows:
[0101] ;
[0102] ;
[0103] in Represents the gray level co-occurrence matrix Line The element values of the column. These texture features constitute the texture feature vector, which provides important information about the cell surface structure. The average intensity, standard deviation, skewness and kurtosis are calculated for the three regions of the stem cell microscopy image to form an intensity feature vector. The average intensity is the average of all pixel values, the standard deviation indicates the discreteness of the pixel values, and the skewness and kurtosis describe the symmetry and kurtosis of the distribution respectively. The Zernike moment is calculated for the cell region mask. The Zernike moment is a shape descriptor that can be expanded by a set of orthogonal Zernike polynomials to describe the shape characteristics of the region very accurately. The Zernike moment is rotationally invariant, which is very useful for image recognition and classification. The calculation formula of the Zernike moment is as follows:
[0104] ;
[0105] in, is the Zernike moment, is the image pixel value, is a Zernike polynomial, and * represents conjugation. In this way, the shape eigenvector is obtained. The area ratio of the core area to the transition area and the area ratio of the core area to the edge area are calculated to obtain the regional proportion eigenvector. These proportional features reflect the internal proportional relationship of the cell structure and are important indicators for understanding cell morphology. For example, in healthy cells, the core area usually accounts for a larger proportion, while in abnormal cells, these proportions may change significantly. The geometric eigenvector, texture eigenvector, intensity eigenvector, shape eigenvector and regional proportion eigenvector are concatenated to form a high-dimensional cell feature vector.
[0106] In a specific embodiment, the process of executing step 105 may specifically include the following steps:
[0107] Perform 3×3 convolution operation and ReLU activation function operation on the stem cell microscopic image to obtain the initial image features;
[0108] Perform residual feature extraction on the initial image features to obtain deep image features, and perform global average pooling on the deep image features to obtain global image features;
[0109] Perform full connection layer analysis and ReLU activation function operation on the high-dimensional cell feature vector to obtain the transformed cell features;
[0110] The global image features and the transformed cell features are concatenated to obtain fused image features, and the fused image features are analyzed by self-attention mechanism to obtain attention-weighted features;
[0111] The attention weighted features are processed by the fully connected layer to obtain the output features, and the output features are operated by the Softmax function to obtain the probability distribution of cell categories;
[0112] According to the probability distribution of cell categories, the category with the highest probability is selected as the classification result to obtain the classification result, and the classification result and the accurately segmented cell area mask are matched at the pixel level to obtain the segmentation result.
[0113] Specifically, a 3×3 convolution operation and a ReLU activation function operation are performed on the input stem cell microscopic image to extract the initial image features. The 3×3 convolution kernel slides on the image to extract local features such as edges and textures, and its formula is expressed as:
[0114] ;
[0115] in, represents the initial image features, is the input microscopic image, is the convolution kernel, Conv represents the convolution operation, and ReLU is the rectified linear unit activation function. The ReLU function introduces nonlinear characteristics by mapping negative values to zero, enabling the network to better learn complex features. Residual feature extraction is performed on the initial image features, and deep image features are extracted using residual blocks. Residual blocks alleviate the gradient vanishing problem in deep neural networks through jump connections. The basic operations of residual blocks are:
[0116] ;
[0117] in, represents deep image features, and is the convolution kernel. This structure allows the input to propagate directly through the block and be added to the output, which helps capture more complex features while accelerating the training process. Global average pooling is performed on deep image features to obtain global image features. Global average pooling reduces the dimension of the feature by averaging each channel of the feature map while retaining the important information of each channel. The formula is:
[0118] ;
[0119] in, Represents the global features of the image, and are the height and width of the feature map, respectively. While processing the image features, the high-dimensional cell feature vector is subjected to full connection layer analysis and ReLU activation function operation to obtain the transformed cell features. The fully connected layer is used to map the high-dimensional feature vector to a new feature space. The ReLU activation function introduces nonlinearity, which helps to capture more complex cell features. The global features of the image are concatenated with the transformed cell features to obtain the fused image features. The self-attention mechanism is analyzed on the fused image features, and the weights of the features are dynamically adjusted by calculating the correlation within the features, highlighting important features and suppressing irrelevant information. The self-attention operation includes calculating the attention score of each feature and normalizing these scores through the Softmax function to obtain the attention weight:
[0120] ;
[0121] in, is the query matrix, is the key matrix, is the value matrix, is the dimension of the key. Through this process, attention-weighted features are obtained, in which each feature is weighted according to its importance in the entire feature space. The attention-weighted features are processed by the fully connected layer to integrate the feature information and generate the final output features. The output features are operated using the Softmax function to calculate the probability distribution of each category. The Softmax function converts the output into a probability distribution, and its formula is:
[0122] ;
[0123] in, Represents input Belongs to category The probability of Yes Category The output score of is the total number of categories. In this way, the probability distribution of cell categories is obtained. According to the probability distribution of cell categories, the category with the highest probability is selected as the final classification result. For each input image model, its most likely cell type is predicted and the category is output as the classification result. The classification result is pixel-wise matched with the accurately segmented cell region mask to generate the final segmentation result. The classification result is mapped to the spatial position of the original image and the image is annotated according to the cell category.
[0124] In a specific embodiment, the execution step of splicing the global image features and the transformed cell features to obtain fused image features, and performing self-attention mechanism analysis on the fused image features to obtain the attention weighted features may specifically include the following steps:
[0125] The global features of the image are input into the image path hidden layer, and the transformed cell features are input into the feature path hidden layer, and a feature splicing operation is performed to obtain a fused image feature;
[0126] The fused image features are processed through the weight matrix to obtain intermediate features, and a bias vector is added to the intermediate features to obtain bias features;
[0127] Perform a hyperbolic tangent function operation on the bias feature to obtain the activation feature, and perform a weight vector analysis on the activation feature to obtain the attention score;
[0128] Perform Softmax function analysis on the attention score to obtain the attention weight, and decompose the attention weight into the first path attention weight and the second path attention weight;
[0129] In the feature path hidden layer, the transformed cell features are subjected to attention weighted analysis according to the first path attention weights to obtain weighted feature path features;
[0130] In the image path hidden layer, the global image features are subjected to attention weighted analysis according to the second path attention weights to obtain weighted image path features;
[0131] The weighted feature path features and the weighted image path features are element-wise added to obtain the attention weighted features.
[0132] Specifically, the global features of the image are input into the image path hidden layer, and the transformed cell features are input into the feature path hidden layer. These two hidden layers are responsible for processing feature information from different sources: the image path hidden layer processes the overall image features, and the feature path hidden layer processes the cell-specific high-dimensional features. Perform a feature concatenation operation to concatenate the two features in the channel dimension to form a fused image feature. The fused image features are processed by a weight matrix to obtain intermediate features. A linear transformation of the fused features, in which the weight matrix acts on each element of the input feature vector to adjust the weight and contribution of the features. The formula can be expressed as:
[0133] ;
[0134] in, represents the intermediate feature, is the weight matrix, is the fusion image feature. Then add a bias vector to the intermediate feature to obtain the bias feature. The role of the bias vector is to adjust the baseline value of the feature so that the model can adapt to changes in data distribution more flexibly. The formula is:
[0135] ;
[0136] in, represents the feature after adding the bias, is the bias vector. The hyperbolic tangent function (tanh) is operated on the bias feature to obtain the activation feature. The tanh function is a commonly used activation function that can compress the input value to between -1 and 1, thereby introducing nonlinearity and allowing the model to capture more complex feature relationships. The formula of the tanh function is:
[0137] ;
[0138] in, is the activated feature. The weight vector of the activated feature is analyzed to calculate the attention score. The attention score reflects the importance of each feature and can be achieved through a simple linear transformation or a more complex neural network. The formula is:
[0139] ;
[0140] in, represents the attention score, is the weight vector. The attention score is normalized by the Softmax function to obtain the attention weight, ensuring that the sum of the attention weights of all features is 1, thus forming a probability distribution. The formula is:
[0141] ;
[0142] in, Indicates The attention weight of each feature, is the attention score. The attention weight is decomposed into the first path attention weight and the second path attention weight, which correspond to the feature path and the image path respectively. In the feature path hidden layer, the transformed cell features are subjected to attention weighted analysis according to the first path attention weight. Each feature is weighted according to its corresponding attention weight to highlight important features while suppressing irrelevant features. Similarly, in the image path hidden layer, the global image features are subjected to attention weighted analysis according to the second path attention weight to obtain the weighted image path features. The weighted feature path features and the weighted image path features are element-wise added to obtain the final attention weighted features.
[0143] In a specific embodiment, the process of executing step 106 may specifically include the following steps:
[0144] Concatenate the cell category probability distribution and the segmentation result to obtain the conditional vector, and concatenate the random noise vector and the conditional vector to obtain the generator input;
[0145] Perform a first-layer transposed convolution operation on the generator input to obtain a first-layer feature map, and perform a second-layer transposed convolution operation on the first-layer feature map to obtain a second-layer feature map;
[0146] Performing a third-layer transposed convolution operation on the second-layer feature map to obtain a third-layer feature map, and performing a fourth-layer transposed convolution operation on the third-layer feature map to obtain a generated cell image;
[0147] The generated cell image and conditional vector are concatenated to obtain the discriminator input, and the first layer of convolution operation is performed on the discriminator input to obtain the first layer of discriminant features;
[0148] Apply the second layer convolution operation, the third layer convolution operation, and the fourth layer convolution operation to the first layer discriminant features in sequence to obtain the discriminant results;
[0149] Based on the generated cell images, discrimination results, and cell category probability distribution, classification visualization results are generated.
[0150] Specifically, the probability distribution of cell categories and the segmentation results are concatenated to form a conditional vector. The generated conditional vector is concatenated with a random noise vector to obtain the input of the generator. The introduction of the random noise vector helps to generate diverse cell images and prevent the generator from only generating copies of specific samples. The first layer of transposed convolution operation is performed on the generator input to obtain the first layer of feature maps. The transposed convolution (also known as deconvolution) operation is an upsampling operation that inversely expands the size of the input feature map by applying a convolution kernel. This operation allows the generation of higher resolution feature maps while maintaining the structure of the original features. The second layer of feature maps is subjected to a second layer of transposed convolution operation to increase the size of the feature map to obtain the second layer of feature maps. The process of upsampling layer by layer enables the generator to gradually build up the details of the cell image. The third layer of feature maps is subjected to a third layer of transposed convolution operation to obtain the third layer of feature maps, and the fourth layer of transposed convolution operation is continued on the third layer of feature maps to generate a complete cell image. The multi-layer transposed convolution network allows the model to gradually generate high-resolution cell images from low-resolution abstract features, and each layer refines the details and structure of the image. The generated cell images will usually have the same resolution and level of detail as the input conditional vector. The generated cell images are concatenated with the conditional vector again to obtain the input of the discriminator. The task of the discriminator is to distinguish between the generated images and the real cell images. Combining the image and the conditional vector together helps the discriminator better understand whether the image is consistent with the given conditions. The first layer of convolution operation is performed on the discriminator input to extract low-level image features, which usually contain basic edge and texture information. The second, third, and fourth layers of convolution operations are applied to the first layer of discriminative features in sequence to gradually extract higher-level feature information. Each layer of convolution operation not only extracts new features, but also reduces the spatial dimension of the feature map and focuses on key feature patterns. In this way, the discriminator can effectively capture the global and local features of the image to judge the authenticity of the image. Through the layer-by-layer processing of these features, the discriminator outputs a discrimination result, which indicates whether the input image is generated or real. The discrimination result is usually processed by a sigmoid function to provide a probability value between 0 and 1, indicating the authenticity of the image. Generate classification visualization results based on the generated cell images, discriminant results, and cell category probability distributions. This includes comparing the generated cell images with the actual cell categories to evaluate the performance of the generator and the quality of the generated images. For different categories of cells, a variety of cell images are generated and their similarity to the real data is evaluated by the discriminator.
[0151] The above describes the stem cell classification method based on image segmentation in the embodiment of the present application. The following describes the stem cell classification system based on image segmentation in the embodiment of the present application. Figure 2In the embodiments of the present application, an embodiment of a stem cell classification system based on image segmentation includes:
[0152] An imaging module 201 is used to perform mean filtering and equalization processing on the stem cell microscopic image to obtain a standardized high-resolution image;
[0153] Extraction module 202, used for inputting the standardized high-resolution image into a multi-level deep learning network for feature extraction to obtain a multi-scale cell feature map;
[0154] The segmentation module 203 is used to perform cell segmentation and post-processing on the multi-scale cell feature map to obtain an accurately segmented cell region mask;
[0155] The processing module 204 is used to extract geometric features, texture features and intensity features from the accurately segmented cell region mask to obtain a high-dimensional cell feature vector;
[0156] The classification module 205 is used to perform dual-path deep learning classification and fusion on the high-dimensional cell feature vector and the stem cell microscopic image to obtain the cell category probability distribution and segmentation results;
[0157] The visualization module 206 is used to reconstruct multi-category cell images based on the cell category probability distribution and the segmentation results to obtain classification visualization results.
[0158] Through the synergy of the above components, the image noise can be effectively removed and the contrast can be enhanced through techniques such as non-local mean filtering and adaptive histogram equalization. The deep learning network containing residual connections and feature pyramid structures can extract multi-scale cell features, which is conducive to capturing the subtle differences of different types of stem cells. Based on the automatic cell segmentation algorithm, combined with conditional random field post-processing, accurate cell region segmentation can be achieved, providing accurate spatial positioning for feature extraction. By extracting the geometric, texture and intensity features of the core area, edge area and transition area, a high-dimensional cell feature vector is constructed to comprehensively characterize the morphological characteristics of the cell. The dual-path deep learning classification and attention mechanism fusion are used to make full use of the feature vector and original image information to improve the classification accuracy. The conditional generative adversarial network is used to reconstruct multi-category cell images, providing intuitive classification visualization results and enhancing the interpretability of the method. The entire classification process from image preprocessing to result output is completed automatically, which greatly improves the efficiency of stem cell classification. It can process various types of stem cell microscopic images, including bright field, fluorescence, confocal and electron microscopic images, and has a wide range of applicability. Through a variety of image enhancement and noise reduction techniques, the method's adaptability to different imaging conditions and image qualities is improved, thereby improving the accuracy of stem cell classification.
[0159] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0160] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.
[0161] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A stem cell classification method based on image segmentation, characterized in that: The stem cell classification method based on image segmentation comprises: Perform mean filtering and equalization processing on stem cell microscopic images to obtain standardized high-resolution images; Inputting the standardized high-resolution image into a multi-level deep learning network for feature extraction to obtain a multi-scale cell feature map; Performing cell segmentation and post-processing on the multi-scale cell feature map to obtain a precisely segmented cell region mask; Extracting geometric features, texture features and intensity features from the precisely segmented cell region mask to obtain a high-dimensional cell feature vector; Performing dual-path deep learning classification and fusion on the high-dimensional cell feature vector and the stem cell microscopic image to obtain cell category probability distribution and segmentation results; A multi-category cell image is reconstructed based on the cell category probability distribution and the segmentation result to obtain a classification visualization result.
2. The stem cell classification method based on image segmentation according to claim 1, characterized in that: The step of performing mean filtering and equalization processing on the stem cell microscopic image to obtain a standardized high-resolution image includes: Perform high-resolution imaging of stem cell microscopy images to obtain multiple types of original images including bright field, fluorescence, confocal, and electron microscopy images; Performing non-local mean filtering on the multiple types of original images to obtain filtered images; Dividing the filtered image into non-overlapping rectangular regions, performing histogram equalization on each rectangular region to obtain a contrast-enhanced image; Performing multi-scale Retinex analysis on the contrast-enhanced image to obtain a light-equalized image, and performing size standardization on the light-equalized image to obtain an image of uniform size; The pixel intensity of the image of uniform size is normalized to obtain a standardized high-resolution image.
3. The stem cell classification method based on image segmentation according to claim 1, characterized in that: The standardized high-resolution image is input into a multi-level deep learning network for feature extraction to obtain a multi-scale cell feature map, including: S1: Input the standardized high-resolution image into a multi-stage deep learning network, and perform a convolution operation on the standardized high-resolution image using a 3×3 convolution kernel to obtain a first initial feature map; S2: Input the first initial feature map into the residual block, and execute F1=F0+ReLU(Conv(ReLU(Conv(F0))))) to obtain a residual feature map, where F1 is the residual feature map, Conv is the convolution operation, ReLU is the activation function, and F0 represents the first initial feature map; S3: Perform a maximum pooling operation on the residual feature map to obtain a first feature map; S4: Repeat steps S2 and S3 three times to obtain a second feature map, a third feature map, and a fourth feature map of different scales respectively; S5: performing channel compression on the fourth feature map through a 1×1 convolution kernel to obtain a top-level feature map; S6: Upsampling the top-level feature map, and performing element-level addition fusion with the third feature map to obtain a first fused feature map; S7: Repeat the operation of step S6 on the first fused feature map, and fuse it with the second feature map and the first feature map respectively to obtain a second fused feature map and a third fused feature map; S8: Combining the third fused feature map, the second fused feature map, the first fused feature map and the top-level feature map into a feature pyramid to obtain a multi-scale feature set; S9: performing a 3×3 convolution operation on each feature map in the multi-scale feature set, adjusting the number of channels, and obtaining a feature map set of uniform dimension; S10: Concatenate the feature map set of the unified dimension in the channel dimension to obtain a multi-scale cell feature map.
4. The stem cell classification method based on image segmentation according to claim 1, characterized in that: The step of performing cell segmentation and post-processing on the multi-scale cell feature map to obtain an accurately segmented cell region mask comprises: Performing a 3×3 convolution operation and a ReLU activation function operation on the multi-scale cell feature map to obtain a second initial feature map; Performing a downsampling operation on the second initial feature map to obtain a first encoder feature map, a second encoder feature map, a third encoder feature map, and a fourth encoder feature map; Performing channel compression on the fourth encoder feature map to obtain a bridge feature map, upsampling the bridge feature map, and concatenating the bridge feature map with the third encoder feature map to obtain a first decoder feature map; Performing a 3×3 convolution operation and a ReLU activation function operation on the first decoder feature map, and concatenating it with the second encoder feature map and the first encoder feature map to obtain a second decoder feature map and a third decoder feature map; Performing a 1×1 convolution operation on the third decoder feature map to obtain an initial segmentation mask, and performing a Sigmoid activation function operation on the initial segmentation mask to obtain a probability map P; Performing threshold segmentation on the probability map to obtain a binary mask, and performing conditional random field analysis on the binary mask and the stem cell microscopic image to obtain an optimized segmentation mask; Morphological operations are performed on the optimized segmentation mask to obtain an accurately segmented cell region mask.
5. The stem cell classification method based on image segmentation according to claim 4, characterized in that: The method of performing threshold segmentation on the probability map to obtain a binary mask, and performing conditional random field analysis on the binary mask and the stem cell microscopic image to obtain an optimized segmentation mask comprises: Performing an optimal threshold calculation on the probability map P by using the Otsu algorithm to obtain an adaptive threshold, and performing binarization processing on the probability map according to the adaptive threshold to obtain an initial binarization mask; Performing connected domain analysis on the initial binary mask, marking different connected regions, and obtaining a marked mask; The area of each connected region of the marked mask is calculated to obtain a region area set, and the connected regions whose area is smaller than a preset threshold are removed according to the region area set to obtain a denoised binary mask; constructing a conditional random field graph model for the denoised binary mask and the stem cell microscopic image; Calculating a unary potential energy for each node in the conditional random field graph model to obtain a unary potential energy graph, and calculating a pairwise potential energy for each pair of adjacent nodes in the graph model to obtain a pairwise potential energy graph; Based on the unary potential energy graph and the paired potential energy graph, using a mean field inference algorithm to iteratively update the edge distribution of each node to obtain an optimized probability distribution; Performing maximum a posteriori probability inference on the optimized probability distribution to obtain an optimized segmentation mask.
6. The stem cell classification method based on image segmentation according to claim 1, characterized in that: Extracting geometric features, texture features and intensity features from the precisely segmented cell region mask to obtain a high-dimensional cell feature vector includes: Performing distance transformation on the precisely segmented cell region mask to obtain a distance map; According to the distance map, the cell region is divided into a core region, a transition region and an edge region to obtain a three-region mask, and the area, perimeter and circularity of the three-region mask are calculated respectively to obtain a geometric feature vector; Performing grayscale co-occurrence matrix calculation on three regions of the stem cell microscopic image to obtain a texture matrix, and calculating energy, contrast, correlation and entropy features based on the texture matrix to obtain a texture feature vector; Calculating the average intensity, standard deviation, skewness and kurtosis of three regions of the stem cell microscopic image to obtain an intensity feature vector; Performing Zernike moment calculation on the cell region mask to obtain a shape feature vector; Calculating the area ratio of the core area to the transition area and the area ratio of the core area to the edge area to obtain a regional proportion feature vector; The geometric feature vector, the texture feature vector, the intensity feature vector, the shape feature vector and the area ratio feature vector are concatenated to obtain a high-dimensional cell feature vector.
7. The stem cell classification method based on image segmentation according to claim 1, characterized in that: The dual-path deep learning classification and fusion of the high-dimensional cell feature vector and the stem cell microscopic image to obtain cell category probability distribution and segmentation results include: Performing a 3×3 convolution operation and a ReLU activation function operation on the stem cell microscopic image to obtain initial image features; Performing residual feature extraction on the initial image features to obtain deep image features, and performing global average pooling on the deep image features to obtain image global features; Performing a fully connected layer analysis and a ReLU activation function operation on the high-dimensional cell feature vector to obtain a transformed cell feature; splicing the global image features and the transformed cell features to obtain fused image features, and performing self-attention mechanism analysis on the fused image features to obtain attention weighted features; Performing full connection layer processing on the attention weighted features to obtain output features, and performing Softmax function operation on the output features to obtain cell category probability distribution; According to the probability distribution of the cell categories, the category with the highest probability is selected as the classification result to obtain the classification result, and the classification result and the accurately segmented cell region mask are matched at the pixel level to obtain the segmentation result.
8. The stem cell classification method based on image segmentation according to claim 7, characterized in that: The global image features and the transformed cell features are concatenated to obtain fused image features, and the fused image features are analyzed using a self-attention mechanism to obtain attention-weighted features, including: Inputting the global features of the image into the image path hidden layer, and inputting the transformed cell features into the feature path hidden layer, performing a feature splicing operation, and obtaining a fused image feature; Processing the fused image features through a weight matrix to obtain intermediate features, and adding a bias vector to the intermediate features to obtain bias features; Performing a hyperbolic tangent function operation on the bias feature to obtain an activation feature, and performing a weight vector analysis on the activation feature to obtain an attention score; Performing a Softmax function analysis on the attention score to obtain an attention weight, and decomposing the attention weight into a first path attention weight and a second path attention weight; In the feature path hidden layer, performing attention weighted analysis on the transformed cell features according to the first path attention weights to obtain weighted feature path features; In the image path hidden layer, performing attention weighted analysis on the image global features according to the second path attention weights to obtain weighted image path features; Performing element-wise addition operation on the weighted feature path feature and the weighted image path feature to obtain an attention weighted feature.
9. The stem cell classification method based on image segmentation according to claim 1, characterized in that: The reconstructing a multi-category cell image based on the cell category probability distribution and the segmentation result to obtain a classification visualization result includes: Concatenating the cell category probability distribution and the segmentation result to obtain a conditional vector, and concatenating a random noise vector and the conditional vector to obtain a generator input; Performing a first-layer transposed convolution operation on the generator input to obtain a first-layer feature map, and performing a second-layer transposed convolution operation on the first-layer feature map to obtain a second-layer feature map; Performing a third-layer transposed convolution operation on the second-layer feature map to obtain a third-layer feature map, and performing a fourth-layer transposed convolution operation on the third-layer feature map to obtain a generated cell image; splicing the generated cell image and the conditional vector to obtain a discriminator input, and performing a first-layer convolution operation on the discriminator input to obtain a first-layer discriminant feature; Applying the second layer convolution operation, the third layer convolution operation, and the fourth layer convolution operation to the first layer discriminant features in sequence to obtain a discriminant result; A classification visualization result is generated according to the generated cell image, the discrimination result and the cell category probability distribution.
10. A stem cell classification system based on image segmentation, characterized in that: Used to perform the stem cell classification method based on image segmentation as described in any one of claims 1 to 9, the stem cell classification system based on image segmentation comprises: An imaging module is used to perform mean filtering and equalization processing on stem cell microscopic images to obtain standardized high-resolution images; An extraction module, used for inputting the standardized high-resolution image into a multi-level deep learning network for feature extraction to obtain a multi-scale cell feature map; A segmentation module, used for performing cell segmentation and post-processing on the multi-scale cell feature map to obtain a precisely segmented cell region mask; A processing module, used for extracting geometric features, texture features and intensity features from the accurately segmented cell region mask to obtain a high-dimensional cell feature vector; A classification module, used for performing dual-path deep learning classification and fusion on the high-dimensional cell feature vector and the stem cell microscopic image to obtain a cell category probability distribution and segmentation result; A visualization module is used to reconstruct multi-category cell images based on the cell category probability distribution and the segmentation results to obtain classification visualization results.
Citation Information
Patent Citations
Cervical cell classification method based on double attention mechanism and multi-scale feature fusion
CN113378791A
Deep learning-based squamous epithelial tumor cell image automatic classification and segmentation method
CN115423802A
Method, apparatus and recording medium for cell image segmentation using scribble labels and multi-scale contrastive loss function
KR102557748B1
