Cell counting method and apparatus, and computer device

Abstract

The embodiments of the present application are applicable to the technical field of biomedical technology and image processing. Provided are a cell counting method and apparatus, and a computer device. The method comprises: acquiring an image of adherent stem cells; using a trained detection segmentation model to segment the image of adherent stem cells, so as to obtain a plurality of stem cell masks; and, on the basis of the plurality of stem cell masks, determining the number of stem cells contained in the image of adherent stem cells, wherein the detection segmentation model comprises a backbone network structure, convolutional layers in the backbone network structure are multi-layer pyramid convolutions, and the multi-layer pyramid convolutions are used for extracting multi-scale feature information from the image of adherent stem cells. The method can be used for quickly and accurately counting adherent stem cells.

Description

Cell counting methods, apparatus and computer equipment

[0001] This application claims priority to Chinese Patent Application No. 202411896475.3, filed on December 20, 2024, entitled “Cell Counting Method, Apparatus and Computer Equipment”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the fields of biomedical technology and image processing technology, and in particular to a cell counting method, apparatus and computer equipment. Background Technology

[0003] Counting adherent stem cells is crucial for assessing cell growth status, proliferative capacity, and the effectiveness of experimental procedures. Regularly counting adherent stem cells allows for monitoring cell growth rate and status, facilitating the optimization of culture conditions and understanding of cell biological characteristics. Existing cell counting methods mainly include Coulter counting, manual statistical methods, and flow cytometry. However, these methods all have drawbacks. For example, Coulter counting is unsuitable for stem cell research; manual statistical or flow cytometry counting is difficult to implement with large datasets; and the counting results are not accurate to the nearest whole number, leading to low accuracy.

[0004] Therefore, there is an urgent need for a method that can quickly and accurately count cells to overcome the problems existing in the current technology. Technical issues

[0005] In view of this, embodiments of this application provide a cell counting method, apparatus, and computer device for rapidly and accurately counting cells. Technical solutions

[0006] A first aspect of this application provides a cell counting method, including:

[0007] Images of adherent stem cells were acquired;

[0008] The trained detection and segmentation model is used to segment the adherent stem cell image to obtain multiple stem cell masks;

[0009] The number of stem cells contained in the adherent stem cell image is determined based on multiple stem cell masks.

[0010] The detection and segmentation model includes a backbone network structure, in which the convolutional layers are multi-layer pyramidal convolutions. These multi-layer pyramidal convolutions are used to extract multi-scale feature information from the adherent stem cell image.

[0011] In this embodiment of the application, before segmenting the adherent stem cell image using the trained detection and segmentation model to obtain multiple stem cell masks, the method further includes:

[0012] Generate a dataset for training the detection and segmentation model, the dataset containing multiple sample stem cell images labeled with mask tags, the mask tags being used to indicate stem cell regions in the sample stem cell images;

[0013] The pre-built model is trained using the dataset to obtain the detection and segmentation model;

[0014] The pre-built model includes the backbone network structure, the neck structure, and the head structure. Each convolutional layer in the neck structure is followed by an attention mechanism layer. The neck structure is used to fuse and enhance the multi-scale feature information, and the head structure is used to output the stem cell mask in the sample stem cell image.

[0015] In one possible implementation of this application embodiment, generating the dataset for training the detection and segmentation model includes:

[0016] The pre-labeling model is trained by using a segmentation model to predict the stem cell mask of the sample stem cell image, obtaining the corresponding mask label, and then inputting the mask label of the sample stem cell image into the segmentation model for training.

[0017] The pre-labeling model is used to predict multiple stem cell images of the sample to obtain a mask label for each stem cell image of the sample.

[0018] The mask label of each of the sample stem cell images is matched one-to-one with the corresponding sample stem cell image to obtain the dataset, which is divided into training dataset, validation dataset and test dataset according to the proportion.

[0019] In this embodiment of the application, before training the pre-built model using the dataset to obtain the detection and segmentation model, the method further includes:

[0020] The dataset is preprocessed, which includes data augmentation of the sample stem cell images in the dataset and processing of the data-augmented sample stem cell images using a thermal diffusion algorithm.

[0021] In one possible implementation of this application embodiment, the processing of the data-enhanced sample stem cell image using a thermal diffusion algorithm includes:

[0022] The thermal diffusion algorithm is used in each region of interest of each sample stem cell image labeled with a mask to iteratively diffuse from the center of each region of interest to simulate the creation of a topological map of the stem cell region in each sample stem cell image.

[0023] The gradient vector fields in the x and y directions of each sample stem cell image are obtained based on the topological graph analysis, and the gradient vector fields are normalized.

[0024] This application embodiment uses a thermal diffusion algorithm during image processing, allowing the processed data to be used as input data for subsequent processing. By processing the input labels using the thermal diffusion algorithm to form a thermal diffusion vector field, it facilitates the subsequent learning of the star-shaped image shape of stem cell regions using a detection and segmentation model, thereby achieving accurate segmentation of stem cell regions in stem cell images.

[0025] In this embodiment of the application, the step of segmenting the adherent stem cell image using a trained detection and segmentation model to obtain multiple stem cell masks includes:

[0026] Multi-scale feature information in the adherent stem cell image is extracted using multi-pyramid convolution in the detection and segmentation model.

[0027] The multi-scale feature information is processed using the neck structure of the detection and segmentation model to obtain an output feature map;

[0028] The head structure of the detection and segmentation model is used to extract each stem cell mask from the output feature map.

[0029] In one possible implementation of this application embodiment, the step of processing the multi-scale feature information using the neck structure of the detection and segmentation model to obtain an output feature map includes:

[0030] The attention mechanism layer in the neck structure is used to perform global average pooling on the multi-scale feature information.

[0031] Perform a one-dimensional convolution operation on the multi-scale feature information after global average pooling, and determine the weights of each channel of the multi-scale feature information.

[0032] The weights are multiplied by the corresponding elements in the original multi-scale feature information to obtain the output feature map.

[0033] In one possible implementation of this application embodiment, after determining the number of stem cells contained in the adherent stem cell image based on multiple stem cell masks, the method further includes:

[0034] The area of ​​each stem cell and the total area of ​​stem cells in the adherent stem cell image are calculated based on each of the stem cell masks.

[0035] The outline of each stem cell is determined, and the perimeter and roundness of each stem cell are calculated based on the outline.

[0036] A second aspect of this application provides a cell counting device, comprising:

[0037] The acquisition module is used to acquire images of adherent stem cells;

[0038] The segmentation module is used to segment the adherent stem cell image using a trained detection and segmentation model to obtain multiple stem cell masks.

[0039] A determining module is used to determine the number of stem cells contained in the adherent stem cell image based on multiple stem cell masks;

[0040] The detection and segmentation model includes a backbone network structure, in which the convolutional layers are multi-layer pyramidal convolutions. These multi-layer pyramidal convolutions are used to extract multi-scale feature information from the adherent stem cell image.

[0041] A third aspect of this application provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the computer device performs the method as described in any of the first aspects above.

[0042] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a computer, implements the method described in any of the first aspects above.

[0043] A fifth aspect of this application provides a computer program product, including a computer program that, when the computer program is run, causes the method described in any of the first aspects above to be executed. Beneficial effects

[0044] Compared with the prior art, the embodiments of this application have the following beneficial effects:

[0045] In this embodiment, by improving a general or common detection network model, such as the YOlO-Seg model, the traditional convolutional layers in the YOlO-Seg model backbone are replaced with multi-layer pyramidal convolutions. This allows for the extraction of multi-scale feature information from the acquired adherent stem cell images during processing. Using this multi-scale feature information, the detection and segmentation model can output accurate stem cell masks. Based on these masks, the number of stem cells in the image can be determined, enabling rapid and accurate counting of adherent stem cells. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 is a schematic diagram of a cell counting method provided in an embodiment of this application;

[0048] Figure 2 is a schematic diagram of another cell counting method provided in an embodiment of this application;

[0049] Figure 3 is a schematic diagram of an optimization process for adherent stem cell images provided in an embodiment of this application;

[0050] Figure 4 is a schematic diagram of a possible implementation of S201 in a cell counting method provided in an embodiment of this application;

[0051] Figure 5 is a schematic diagram of a pyramid convolution structure provided in an embodiment of this application;

[0052] Figure 6 is a schematic diagram of the structure of an attention mechanism layer provided in an embodiment of this application;

[0053] Figure 7 is a schematic diagram of the prediction results of a stem cell mask provided in an embodiment of this application;

[0054] Figure 8 is a schematic flowchart of a cell counting and cell state analysis provided in an embodiment of this application;

[0055] Figure 9 is a schematic diagram of a cell counting device provided in an embodiment of this application;

[0056] Figure 10 is a schematic diagram of a computer device provided in an embodiment of this application. Embodiments of the present invention

[0057] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0058] In response to the various problems existing in current cell counting methods, the industry has conducted research on possible counting methods based on various technical means.

[0059] For example, stem cell counting can be achieved by segmenting stem cells using a basic segmentation neural network (FCN) in deep learning, and then counting the stem cells based on this segmentation. However, this technique belongs to an early semantic segmentation neural network with limited network performance. It can only classify each pixel in the image as foreground or background, and cannot assign foreground pixels to cells of a specific instance. When faced with stem cells that are adhered together, it cannot achieve accurate segmentation of the stem cells.

[0060] For example, a related study proposed a method for cell counting based on YOLOv3 and density estimation. This method addresses the phenomenon of sparse white blood cell distribution and dense red blood cell distribution in blood cell micrographs. First, the YOLOv3 network model is used to detect white blood cells, isolated red blood cells, and regions of adhered red blood cells in the micrograph. Then, a density estimation algorithm is used to count the red blood cells in the adhered regions. This method supplements the YOLOv3 detection algorithm with a density estimation algorithm for adhered red blood cells, mainly because YOLOv3 lacks the ability to accurately detect adhered cells. However, the density estimation algorithm used in this method only estimates the number of adhered cells, not accurately counts them. Neither YOLOv3 nor the density estimation algorithm involves cell region segmentation and cannot assess cell state.

[0061] For example, in algal cell statistics methods based on deep learning and image pattern recognition, a deep learning detection network can be used to detect the location of disk star algal cells, followed by a series of traditional algorithms to determine the integrity of the cells. However, because traditional algorithms lack generalization ability, multiple threshold parameters need to be adjusted during application, and the method may not be applicable after changing the microscope imaging scene. Furthermore, the process of first using deep learning detection and then segmenting with traditional algorithms is too cumbersome.

[0062] In response to the various problems existing in the prior art and the related studies mentioned in the foregoing examples, this application provides a cell counting method. By improving common detection network models, such as the YOlO-Seg model, pyramid convolution and attention mechanisms are introduced into the backbone network structure of the model to enhance the backbone network's ability to capture image features, thereby accurately segmenting the stem cell mask in adherent stem cell images and enabling rapid and accurate counting of adherent stem cells.

[0063] The technical solution of this application will be described below through specific embodiments.

[0064] Referring to Figure 1, a schematic diagram of a cell counting method provided in an embodiment of this application is shown, which may specifically include the following steps:

[0065] S101. Collect images of adherent stem cells.

[0066] It should be noted that this method can be implemented by a computer device, that is, the executing entity of this application embodiment can be a computer device. By executing the various steps in this method, the computer device can achieve rapid and accurate counting of adherent stem cells. The computer device in this application embodiment can be an electronic device with data processing capabilities, such as a desktop computer, a cloud server, etc. This application embodiment does not limit the specific type of electronic device.

[0067] The adherent stem cell images in this application embodiment can be acquired at one or more time points during the cell culture process. The process of acquiring adherent stem cell images can be achieved by a microscope or other devices with similar functions.

[0068] Taking the acquisition of adherent stem cell images via a microscope as an example, in one possible implementation of this application embodiment, the microscope can be integrated with the aforementioned computer equipment, meaning the microscope can be part of the computer equipment. After acquiring the adherent stem cell images, the data processing unit of the computer equipment processes them and outputs the corresponding counting results. Alternatively, the microscope can be independent of the aforementioned computer equipment. After acquiring the adherent stem cell images via the microscope, the images can be input into the computer equipment for processing and outputting the corresponding counting results. This application embodiment does not limit this approach.

[0069] S102. The trained detection and segmentation model is used to segment the adherent stem cell image to obtain multiple stem cell masks. The detection and segmentation model includes a backbone network structure, and the convolutional layers in the backbone network structure are multi-layer pyramid convolutions. The multi-layer pyramid convolutions are used to extract multi-scale feature information from the adherent stem cell image.

[0070] In the embodiments of this application, a trained detection and segmentation model can be used to segment adherent stem cells into images, resulting in multiple stem cell masks, each representing a region where a stem cell is located.

[0071] The detection and segmentation model in this application embodiment can be obtained by improving upon a general or common detection network model and training it. For example, the YOlO-Seg model can be improved and trained to obtain a detection and segmentation model that can be used for accurate image segmentation of adherent stem cell images.

[0072] Taking the YOlO-Seg model as an example, it mainly includes a backbone network structure, a neck structure, and a head structure. The backbone network primarily extracts features from the input image, including CBS (compound convolution), C2F (convolution to fully connected layers), and SPPF (spatial pyramid pooling). CBS consists of convolutional layers (Conv2d), batch normalization layers (BatchNorm2d), and activation functions.

[0073] One improvement of the YO1O-Seg model described above in this application embodiment may include replacing all traditional convolutional layers in CBS with pyramidal convolutions (Pyconv). Pyramid convolution is a pyramidal multi-layer structure proposed in recent years that can be used to extract multi-scale feature information. Pyramid convolution contains a kernel pyramid, and each layer contains different types of filters. The size and depth of each filter are variable, so when using pyramidal convolution to process images, detailed information at different scales can be extracted.

[0074] In this embodiment, the trained detection and segmentation model can be obtained by replacing the traditional convolutional layers in the backbone network of the general YO1O-Seg model with multi-layer pyramidal convolutions, as described above, and then training the improved model using training data. Thus, when processing adherent stem cells using the trained detection and segmentation model, the aforementioned replaced multi-layer pyramidal convolutions can be used to extract multi-scale feature information from the adherent stem cell image. Based on this, the neck structure in the model can further process the multi-scale feature information, ultimately outputting multiple stem cell masks from the head structure.

[0075] S103. Determine the number of stem cells contained in the adherent stem cell image based on the plurality of stem cell masks.

[0076] In this embodiment of the application, each stem cell mask output by the detection segmentation model can represent a stem cell region. Therefore, based on the multiple stem cell masks output by the model, the number of stem cells contained in the currently acquired adherent stem cell image can be determined, thereby achieving rapid and accurate counting of adherent stem cells.

[0077] This application improves upon common detection network models, such as the YOlO-Seg model, by replacing traditional convolutional layers in the YOlO-Seg backbone with multi-layer pyramidal convolutions. This allows for the extraction of multi-scale feature information from the acquired adherent stem cell images during processing. Based on this multi-scale feature information, the detection and segmentation model can output accurate stem cell masks. Furthermore, by using multiple stem cell masks, the number of stem cells in the image can be determined, enabling rapid and accurate counting of adherent stem cells.

[0078] Referring to Figure 2, a schematic diagram of another cell counting method provided in an embodiment of this application is shown, which may specifically include the following steps:

[0079] S201. Generate a dataset for training the detection and segmentation model, wherein the dataset contains multiple sample stem cell images labeled with mask tags.

[0080] In this embodiment, the dataset used to train the detection and segmentation model can consist of multiple sample stem cell images labeled with mask tags. These sample stem cell images are adherent stem cell images acquired using a microscope or related equipment. That is, adherent stem cell images can be used as samples to train the detection and segmentation model.

[0081] In one possible implementation of this application, a computer device can perform image optimization processing on the acquired adherent stem cell images used as samples to enhance the pixel gradient changes in the stem cell region image and highlight the stem cell image features.

[0082] For example, in the embodiments of this application, image optimization processing of adherent stem cell images may mainly include processing the original image based on opening and closing operations in morphological image processing algorithms, and then optimizing the original image by combining contrast enhancement, gamma correction and other brightness equalization methods.

[0083] In image processing algorithms, opening is a sequential operation that performs erosion followed by dilation on an image; closing, on the other hand, performs dilation first, followed by erosion. The order of erosion and dilation in opening and closing operations is reversed.

[0084] The opening operation involves subtracting the result of the opening operation from the original image to extract peak information. Opening is primarily used for bright objects on dark backgrounds, correcting for uneven lighting, removing most of the non-uniform background, and separating the foreground and background. The closing operation, on the other hand, involves subtracting the result of the closing operation from the original image to extract valley information. Closing can highlight dark details and separate any background information that may be present in the image.

[0085] In the embodiments of this application, in order to optimize the original image of adherent cells, the original image can be subjected to opening operation processing, or closing operation processing, or both opening and closing operations can be performed on the original image simultaneously.

[0086] For example, opening and closing operations can be performed sequentially on the adherent stem cell image to obtain a first image and a second image, respectively. That is, the first image is the image obtained after the computer device performs an opening operation on the adherent stem cell image, and the second image is the image obtained after the computer device performs a closing operation on the adherent stem cell image. Then, based on the original adherent stem cell image, the image obtained by subtracting the first image from the original image can be added, and the image obtained by subtracting the original image from the second image can be subtracted, thereby obtaining an optimized adherent stem cell image, which enhances the stem cell region information in the original image.

[0087] The above process can be represented as:

[0088]

[0089] in, The optimized image of adherent stem cells. This represents the original image of adherent stem cells. This indicates the original image of adherent stem cells. The image obtained after performing the opening operation is the first image. This indicates the original image of adherent stem cells. The image obtained after performing the closing operation is the second image. Therefore, This represents the image obtained by subtracting the first image from the original image. This represents the image obtained by subtracting the original image from the second image.

[0090] In one possible implementation of this application, after the original image is processed by opening and closing operations in a morphological image processing algorithm to obtain an optimized image of the wall-attached dry image, the computer device can perform subsequent processing on the image of the wall-attached dry image to create a dataset for training a detection and segmentation model.

[0091] In another possible implementation of this application, after optimizing the original image of adherent stem cells using opening and / or closing operations, the computer device can further optimize the image using contrast enhancement and brightness equalization techniques, so that the further optimized image can be used as the image for subsequent processing.

[0092] In this embodiment, when performing contrast enhancement processing on an image, contrast enhancement algorithms such as histogram equalization and linear stretching can be used, but are not limited to. When performing brightness equalization processing on an image, brightness adjustment can be achieved based on gamma correction. The gamma value (γ) is a non-linear parameter that describes the relationship between input and output. By using a power function formula to describe the relationship between the input pixel value (Iin) and the output pixel value (Iout), brightness equalization processing of the image can be achieved. The above processing can be expressed as:

[0093]

[0094] Figure 3 shows a schematic diagram of optimizing an adherent stem cell image according to an embodiment of this application. Figure 3(a) shows a schematic diagram of the original adherent stem cell image, and Figure 3(b) shows a schematic diagram of the adherent stem cell image obtained after optimizing the original image shown in Figure 3(a) using the aforementioned optimization methods. In other words, Figure 3(b) shows the effect of optimizing Figure 3(a). By comparing Figure 3(a) and (b), it can be seen that the image features of the stem cells are clearer and more prominent in the optimized adherent stem cell image.

[0095] This application embodiment can use optimized adherent stem cell images as samples to create a dataset that can be used to train a detection and segmentation model.

[0096] In one possible implementation of this application embodiment, as shown in FIG4, generating a dataset for training the detection and segmentation model in S201 may include the following steps S2011-S2013:

[0097] S2011. Training the pre-labeled model, wherein the pre-labeled model is obtained by using the segmentation model to predict the stem cell mask of the sample stem cell image, obtaining the corresponding mask label, and then inputting the mask label of the sample stem cell image into the segmentation model for training.

[0098] In this embodiment, the pre-labeling model can be a model used to predict the stem cell mask in each image of adherent stem cell images used as samples. The aforementioned pre-labeling model can be based on a general segmentation model, and through training and fine-tuning, make the general segmentation model more suitable for stem cell segmentation tasks in adherent stem cell images.

[0099] In one possible implementation of this application, a segmentation model can first be used to predict the stem cell mask of the sample stem cell image, obtaining the corresponding mask label. The aforementioned sample stem cell image is an adherent stem cell image serving as the sample, which can be a stem cell image optimized from the original adherent stem cell image according to the optimization processing method corresponding to Figure 3. The aforementioned mask label can be used to indicate stem cell regions in the sample stem cell image.

[0100] After obtaining the mask labels of the sample stem cell images, the incorrect stem cell outlines in the prediction results can be adjusted to generate new mask labels.

[0101] The mask labels of the adjusted sample stem cell images can be input into the segmentation model for training, and the segmentation model can be fine-tuned to be more suitable for stem cell segmentation tasks in stem cell images, that is, the pre-labeled model in the embodiments of this application.

[0102] S2012. The pre-labeling model is used to predict multiple sample stem cell images to obtain a mask label for each sample stem cell image.

[0103] After obtaining the pre-labeled model, it can be used to predict the acquired adherent stem cell images as samples, and obtain the mask label for each sample stem cell image.

[0104] S2013. Match the mask label of each of the sample stem cell images with the corresponding sample stem cell images to obtain the dataset. The dataset is divided into training dataset, validation dataset and test dataset according to the proportion.

[0105] By mapping the mask labels of each stem cell sample image to the corresponding stem cell sample image, a dataset can be created for training the detection and segmentation model.

[0106] In this embodiment of the application, the generated dataset can be divided into training dataset, validation dataset and test dataset according to a certain ratio, and used for training, validation and testing in the subsequent model training process.

[0107] For example, the resulting dataset can be divided into a training dataset, a validation dataset, and a test dataset in a 7:2:1 ratio. Therefore, the sample stem cell images in the training dataset account for 70% of the total sample stem cell images in the dataset, while the sample stem cell images in the validation dataset and the test dataset account for 20% and 10% of the total sample stem cell images in the dataset, respectively.

[0108] In one possible implementation of this application, before training the pre-built model using the dataset to obtain the detection and segmentation model, data preprocessing can be performed on the dataset. Data preprocessing in this application embodiment may include various processes such as performing data augmentation on the sample stem cell images in the dataset and processing the data-augmented sample stem cell images using a thermal diffusion algorithm.

[0109] For example, the generated dataset can be augmented and enhanced by performing data enhancement and augmentation operations on the sample stem cell images. These operations may include, but are not limited to, image transformations such as brightness, contrast, cropping, and translation.

[0110] In this embodiment, the data-enhanced sample stem cell images can be further processed using a thermal diffusion algorithm. The data processed by the thermal diffusion algorithm can be used as input data for the next processing step. For example, the thermal diffusion algorithm is used in each region of interest (ROI) of each sample stem cell image labeled with a mask, iteratively diffusing from the center of each ROI to simulate and create a topological map of the stem cell region in each sample stem cell image. Based on this topological map, the gradient vector fields in the x and y directions of each sample stem cell image can be obtained, and finally, the gradient vector fields are normalized. The image data obtained after normalization can be used for segmentation branches to learn the star-shaped image shape of stem cells, improving the ability of segmentation branches to accurately segment stem cell regions in stem cell images, thus achieving accurate segmentation of stem cell regions.

[0111] Normalization can include, but is not limited to, min-max normalization, Z-score normalization, mean normalization, logarithmic normalization, and tangent normalization. Taking min-max normalization as an example, the following formula illustrates the normalization of the gradient vector field in the x-direction:

[0112]

[0113] in, The normalized gradient vector field in the x-direction. This represents the gradient vector field in the x-direction before normalization. and These are the maximum and minimum values ​​of the gradient vector field in the x-direction, respectively.

[0114] Therefore, the dataset used to train the detection and segmentation model can be sample stem cell images obtained by processing images as shown in the foregoing embodiments.

[0115] S202. The pre-built model is trained using the dataset to obtain the detection and segmentation model; wherein the pre-built model includes a backbone network structure, a neck structure and a head structure, the convolutional layers in the backbone network structure are multi-layer pyramid convolutions, and each composite convolutional layer in the neck structure is followed by an attention mechanism layer.

[0116] In this embodiment, a model can be pre-built and trained using the dataset obtained after the aforementioned processing, thereby obtaining a detection and segmentation model that can be used to segment adherent stem cell images. The dataset obtained after the aforementioned processing can be data output after processing by a thermal diffusion algorithm. The pre-built model can include a backbone network structure, a neck structure, and a head structure. The convolutional layers in the backbone network structure can be multi-layer pyramidal convolutions used to extract multi-scale feature information from the adherent stem cell images; the neck structure can be used to fuse and enhance multi-scale feature information; and the head structure can be used to output a stem cell mask in the sample stem cell image. In the neck structure of the pre-built model in this embodiment, an attention mechanism layer can be configured after each composite convolutional layer (CBS). This helps the model better focus on key information and ignore irrelevant information, thereby improving detection accuracy.

[0117] In one possible implementation of this application, the pre-built model can be obtained by improving a general or common detection module, for example, by improving the YOlO-Seg model to obtain the model to be trained.

[0118] The YOlO-Seg model mainly consists of a backbone network structure, a neck structure, and a head structure. Among them:

[0119] The backbone network primarily performs feature extraction from the input image, including modules such as CBS, C2F, and SPPF. CBS consists of convolutional layers (Conv2d), batch normalization layers (BatchNorm2d), and activation functions (e.g., SiLU). Conv2d is responsible for extracting features from the image and gradually reduces the size of the feature map through downsampling while increasing the number of channels. C2F is a module for deep feature extraction. The SPPF module effectively reduces computation by employing a fast spatial pyramid pooling strategy. By fusing multi-scale feature information, the SPPF module allows the model to better adapt to scale changes, thereby improving detection robustness. In this embodiment, all traditional convolutional layers in CBS are replaced with pyramid convolutions, which can extract multi-scale feature information. Pyramid convolutions are multi-layered structures containing a kernel pyramid, with each layer containing different types of filters. The size and depth of the filters are variable, thus allowing for the extraction of detailed information at different scales. The structure of a pyramid convolution can be shown in Figure 5. Each layer in the multi-layer pyramid convolution can process the input feature map and output multi-scale feature information respectively.

[0120] The neck structure, located between the backbone network and the head structure, primarily functions to further fuse and enhance features. The YOlO-Seg model's neck structure employs a hybrid structure of FPN (Feature Pyramid Networks) and PAN (Path Aggregation Network) to achieve multi-scale feature fusion and propagation. In this embodiment, an attention mechanism layer is added after each CBS layer in the neck structure. This new attention mechanism helps the model better focus on key information and ignore irrelevant information, thereby improving detection accuracy. For example, the attention mechanism layer in this embodiment can be an Efficient Channel Attention Module (ECA), the structure of which is shown in Figure 6.

[0121] In one possible implementation of this application embodiment, as shown in FIG6, the process of applying the above attention mechanism may include the following steps:

[0122] ① Perform global average pooling on the input feature map;

[0123] ② Perform a one-dimensional convolution operation with a kernel size of k, and then use the Sigmoid activation function to obtain the weights of each channel; for example, k=5 in Figure 6;

[0124] ③ Multiply the weights by the corresponding elements of the original input feature map to obtain the final output feature map.

[0125] The head structure is the output part of the YOlO-Seg model, which is responsible for generating bounding boxes for object detection based on the extracted feature maps, and performing processing such as class prediction and mask segmentation.

[0126] For the model built in the aforementioned manner, the computer device can use the prepared dataset for model training. The model training steps may include:

[0127] ① Forward propagation: Input data is processed by the model to obtain the predicted output.

[0128] ② Calculate the loss: Use a loss function to calculate the difference between the predicted output and the true label.

[0129] ③ Backpropagation: Calculate the gradient based on the loss value and propagate it back to the network to update the weights.

[0130] ④ Weight update: Use the optimizer to update the model weights based on the gradient.

[0131] Once the pre-built model has been trained, it can be used as a detection and segmentation model to perform mask segmentation on adherent stem cell images in subsequent image processing, obtaining multiple stem cell masks in each adherent stem cell image. Each stem cell mask can represent a specific stem cell in the image.

[0132] S203. Collect images of adherent stem cells.

[0133] In this embodiment of the application, the adherent stem cell image is the stem cell image to be counted or to be analyzed for cell state, and the adherent stem cell image can be acquired by a microscope or other equipment.

[0134] S204. Extract multi-scale feature information from the adherent stem cell image using multi-layer pyramid convolution in the detection and segmentation model.

[0135] The detection and segmentation model in this embodiment is the model trained according to the aforementioned S201-S202. The convolutional layers in the backbone network of this detection and segmentation model are multi-layer pyramidal convolutions. The adherent stem cell images acquired in S203 can be input into the above-mentioned detection and segmentation model for processing. Among them, the multi-layer pyramidal convolutions in the backbone network can be used to extract multi-scale feature information from the input adherent stem cell images.

[0136] S205. The multi-scale feature information is processed using the neck structure of the detection and segmentation model to obtain an output feature map.

[0137] In this embodiment of the application, the extracted multi-scale feature information can be further fused and enhanced in the detection segmentation model to achieve the fusion and transfer of multi-scale features.

[0138] In one possible implementation of this application, an attention mechanism layer can be added after each CBS layer in the neck structure to focus on key information in the features and ignore irrelevant information, thereby improving detection accuracy. Specifically, the attention mechanism layer in the neck structure can be used to perform global average pooling on the multi-scale feature information, and then a one-dimensional convolution operation can be performed on the multi-scale feature information after global average pooling. The weights of each channel of the multi-scale feature information are determined, and the final output feature map can be obtained by multiplying the weights of each channel with the corresponding elements in the original multi-scale feature information. This output feature map is then processed by the head structure to output the feature maps of each stem cell mask.

[0139] S206. Using the head structure of the detection and segmentation model, extract each stem cell mask from the output feature map.

[0140] In this embodiment, the head structure in the detection segmentation model is responsible for generating bounding boxes for target detection, neck category prediction, and mask segmentation based on the extracted feature maps. Therefore, for the feature maps output by S205, corresponding stem cell masks can be output through the processing of the head structure, with each stem cell mask representing a stem cell region in the original adherent stem cell image.

[0141] The process described in S204-S206 above is the process of using the trained detection and segmentation model to predict the acquired adherent stem cell image and output a mask for a single stem cell region. Figure 7 shows a schematic diagram of the prediction result of a stem cell mask provided in an embodiment of this application. In Figure 7(a), the adherent stem cell image is shown, and Figure 7(b) shows a schematic diagram of the mask result output after predicting the adherent stem cell image shown in Figure 7(a) using the aforementioned detection and segmentation model.

[0142] S207. Determine the number of stem cells contained in the adherent stem cell image based on the plurality of stem cell masks.

[0143] In this embodiment, the output of the detection and segmentation model is n stem cell masks, each corresponding to a specific stem cell region in the image. Therefore, the number of stem cells contained in the originally acquired adherent stem cell image can be determined based on the number of stem cell masks output by the model.

[0144] In one possible implementation of this application, based on the stem cell mask output by the detection and segmentation model, the computer device can also analyze the state of the stem cells. That is, in addition to determining the number of stem cells, it can also output parameters such as the confluence degree of stem cells, the area, perimeter, and roundness of individual stem cells for analyzing the growth state of stem cells.

[0145] In the embodiments of this application, the area of ​​each stem cell and the total area of ​​stem cells in the adherent stem cell image can be calculated based on each stem cell mask, and the perimeter and roundness of each stem cell can be calculated based on the contour by determining the outline of each stem cell.

[0146] For example, each stem cell output by the detection and segmentation model can be traversed, and the number of pixels with a value of 1 (or any value representing the foreground) can be counted as the area of ​​a single stem cell. By summing the areas of individual stem cells, the total area of ​​stem cells in the stem cell image can be obtained, which is the stem cell confluence. In addition, the contour detection algorithm of image processing can be used to obtain the contour of a single stem cell, and the perimeter of the stem cell can be calculated based on the stem cell contour. The roundness parameter of the stem cell can be further calculated based on the stem cell perimeter. Roundness is an index describing how close an object's shape is to a circle. The roundness calculation formula can be expressed as follows:

[0147]

[0148] This application's embodiments, tailored to the characteristics of stem cell images, improve upon the common detection network model YO1O-Seg by introducing pyramidal convolution and attention mechanisms into the backbone network, effectively enhancing its ability to capture image features. Secondly, by using a thermal diffusion algorithm in the model's segmentation branch to process the input labels and form a thermal diffusion vector field, the segmentation branch is better able to learn the star-shaped contours of stem cell regions. This allows for accurate segmentation of stem cell regions in adherent stem cell images using the detection and segmentation network, effectively improving the model's segmentation performance. Thirdly, this application's embodiments utilize morphological image processing algorithms such as closing and opening operations to process the image, combined with contrast enhancement and gamma correction for brightness equalization, to optimize the original image. This strengthens the pixel gradient changes in the stem cell region image, highlighting stem cell image features. Fine-tuning the pre-labeled model using the segmentation model also reduces the workload of labeling. Fourth, after obtaining the stem cell outline through the detection and segmentation model, the stem cell outline can be analyzed. Image processing technology can be used to calculate parameters such as the area, roundness, and perimeter of the outline, which enriches the types of parameters that the model can output. This can help users such as cell culture engineers to make more accurate judgments on the cell status.

[0149] To facilitate understanding, a complete example is provided below to illustrate the cell counting and cell state analysis process provided in this application embodiment. Figure 8 shows a schematic flowchart of a cell counting and cell state analysis process provided in this application embodiment. The flowchart shown in Figure 8 mainly includes:

[0150] Acquiring images of adherent stem cells: Images of the adherent stem cells to be identified can be obtained using a microscope at one or more time points during the cell culture process.

[0151] Stem cell image optimization: For the adherent stem cell images acquired in the aforementioned steps, the original image can be processed based on opening and closing operations in morphological image processing algorithms. Then, combined with brightness equalization techniques such as contrast enhancement and gamma correction, the original image can be optimized to enhance the pixel gradient changes in the stem cell region and highlight the stem cell image features.

[0152] Training the pre-labeled model: Based on commonly used general segmentation models, the mask of stem cell region in stem cell images can be predicted. The incorrect stem cell contours in the prediction results can be adjusted to generate new mask labels. The mask labels are then input into the general segmentation model for training, and the general segmentation model can be fine-tuned into a pre-labeled model that is more suitable for stem cell segmentation tasks in stem cell images.

[0153] Dataset Creation: The pre-labeled model trained in the previous step can be used to predict all the collected stem cell images, obtaining mask labels. The network training dataset is then created by mapping these mask labels one-to-one with the original images. The dataset can be divided into training, validation, and test datasets in a 7:2:1 ratio.

[0154] Dataset preprocessing: Before feeding the dataset into the detection and segmentation model, augmentation operations can be performed on the data, including but not limited to image transformations such as brightness, contrast, cropping, and translation. The augmented data can be processed using a thermal diffusion algorithm. In the labeled mask, the thermal diffusion algorithm is used for each ROI, iteratively diffusing from the ROI center to simulate and create a topology map, resolving the gradient vector fields in the x and y directions, and then normalizing the gradient vector fields.

[0155] Detection and segmentation model construction: It can be improved based on the classic detection neural network YOlO-Seg. All traditional convolutional layers in the CBS of the backbone network are replaced with multi-layer pyramid convolutions to extract multi-scale feature information in the image. A feature attention mechanism layer is added after each CBS layer of the neck structure to help the model better focus on key information and ignore irrelevant information, thereby improving detection accuracy.

[0156] Detection and segmentation model training: The aforementioned dataset can be used to train the constructed detection and segmentation model.

[0157] Mask prediction and related parameter calculation: A trained detection and segmentation model is used to predict stem cell regions in the stem cell image, obtaining individual stem cell region masks, and thus determining the number of stem cells in the image. Based on this, each mask is traversed, and the number of pixels with a value of 1 (or any value representing the foreground) is used as the calculated area of ​​a single stem cell. Summing up the areas of individual stem cells yields the total area of ​​all stem cells in the image, i.e., the stem cell confluence. An image processing contour detection algorithm is used to obtain the contour of a single stem cell, and based on this contour, the stem cell perimeter and roundness can be calculated. The aforementioned stem cell image refers to an image obtained from the acquisition of stem cells for cell counting or cell state analysis.

[0158] Output relevant parameters: The detection and segmentation model can output parameters such as the number of stem cells in the stem cell image, confluence degree, area, perimeter, and roundness of a single stem cell, so that relevant personnel can analyze the growth status of stem cells.

[0159] Referring to Figure 9, a schematic diagram of a cell counting device provided in an embodiment of this application is shown, which may specifically include a collection module 901, a segmentation module 902, and a determination module 903, wherein:

[0160] Acquisition module 901 is used to acquire images of adherent stem cells;

[0161] The segmentation module 902 is used to segment the adherent stem cell image using a trained detection and segmentation model to obtain multiple stem cell masks;

[0162] The determining module 903 is used to determine the number of stem cells contained in the adherent stem cell image based on the plurality of stem cell masks;

[0163] The detection and segmentation model includes a backbone network structure, in which the convolutional layers are multi-layer pyramidal convolutions. These multi-layer pyramidal convolutions are used to extract multi-scale feature information from the adherent stem cell image.

[0164] In this embodiment, the apparatus further includes a dataset generation module and a detection and segmentation model training module. Wherein:

[0165] A dataset generation module is used to generate a dataset for training the detection and segmentation model. The dataset contains multiple sample stem cell images labeled with mask tags, which are used to indicate stem cell regions in the sample stem cell images.

[0166] The detection and segmentation model training module is used to train the pre-built model using the dataset to obtain the detection and segmentation model;

[0167] The pre-built model includes the backbone network structure, the neck structure, and the head structure. Each convolutional layer in the neck structure is followed by an attention mechanism layer. The neck structure is used to fuse and enhance the multi-scale feature information, and the head structure is used to output the stem cell mask in the sample stem cell image.

[0168] In one possible implementation of this application embodiment, the dataset generation module can specifically be used for:

[0169] The pre-labeling model is trained by using a segmentation model to predict the stem cell mask of the sample stem cell image, obtaining the corresponding mask label, and then inputting the mask label of the sample stem cell image into the segmentation model for training.

[0170] The pre-labeling model is used to predict multiple stem cell images of the sample to obtain a mask label for each stem cell image of the sample.

[0171] The mask label of each of the sample stem cell images is matched one-to-one with the corresponding sample stem cell image to obtain the dataset, which is divided into training dataset, validation dataset and test dataset according to the proportion.

[0172] In another possible implementation of this application embodiment, the dataset generation module may also be used for:

[0173] The dataset is preprocessed, which includes data augmentation of the sample stem cell images in the dataset and processing of the data-augmented sample stem cell images using a thermal diffusion algorithm.

[0174] In this embodiment of the application, the dataset generation module can also be used for:

[0175] The thermal diffusion algorithm is used in each region of interest of each sample stem cell image labeled with a mask to iteratively diffuse from the center of each region of interest to simulate the creation of a topological map of the stem cell region in each sample stem cell image.

[0176] The gradient vector fields in the x and y directions of each sample stem cell image are obtained based on the topological graph analysis, and the gradient vector fields are normalized.

[0177] In this embodiment of the application, the segmentation module 902 can specifically be used for:

[0178] Multi-scale feature information in the adherent stem cell image is extracted using multi-pyramid convolution in the detection and segmentation model.

[0179] The multi-scale feature information is processed using the neck structure of the detection and segmentation model to obtain an output feature map;

[0180] The head structure of the detection and segmentation model is used to extract each stem cell mask from the output feature map.

[0181] In one possible implementation of this application embodiment, the segmentation module 902 may also be used for:

[0182] The attention mechanism layer in the neck structure is used to perform global average pooling on the multi-scale feature information.

[0183] Perform a one-dimensional convolution operation on the multi-scale feature information after global average pooling, and determine the weights of each channel of the multi-scale feature information.

[0184] The weights are multiplied by the corresponding elements in the original multi-scale feature information to obtain the output feature map.

[0185] In this embodiment of the application, the determining module 903 can also be used for:

[0186] The area of ​​each stem cell and the total area of ​​stem cells in the adherent stem cell image are calculated based on each of the stem cell masks.

[0187] The outline of each stem cell is determined, and the perimeter and roundness of each stem cell are calculated based on the outline.

[0188] This application provides a cell counting device, which can be used to implement the steps in the aforementioned method embodiments.

[0189] As the apparatus embodiments are basically similar to the method embodiments, they are described in a relatively simple manner. For relevant details, please refer to the description in the method embodiment section.

[0190] Referring to FIG10, a schematic diagram of a computer device provided in an embodiment of this application is shown. As shown in FIG10, the computer device 1000 in this embodiment includes: a processor 1010, a memory 1020, and a computer program 1021 stored in the memory 1020 and executable on the processor 1010. When the processor 1010 executes the computer program 1021, it implements the steps in the various embodiments of the cell counting method described above, such as steps S101 to S103 shown in FIG1. ​​Alternatively, when the processor 1010 executes the computer program 1021, it implements the functions of each module / unit in the various device embodiments described above, such as the functions of modules 901 to 903 shown in FIG9.

[0191] For example, the computer program 1021 can be divided into one or more modules / units, which are stored in the memory 1020 and executed by the processor 1010 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which can be used to describe the execution process of the computer program 1021 in the computer device 1000. For example, the computer program 1021 can be divided into an acquisition module, a segmentation module, and a determination module, with the specific functions of each module as follows:

[0192] The acquisition module is used to acquire images of adherent stem cells;

[0193] The segmentation module is used to segment the adherent stem cell image using a trained detection and segmentation model to obtain multiple stem cell masks.

[0194] A determining module is used to determine the number of stem cells contained in the adherent stem cell image based on multiple stem cell masks;

[0195] The detection and segmentation model includes a backbone network structure, in which the convolutional layers are multi-layer pyramidal convolutions. These multi-layer pyramidal convolutions are used to extract multi-scale feature information from the adherent stem cell image.

[0196] The computer device 1000 may be a device capable of implementing the steps in the foregoing embodiments. The computer device 1000 may be a desktop computer, a cloud server, or other similar device. The computer device 1000 may include, but is not limited to, a processor 1010 and a memory 1020. Those skilled in the art will understand that FIG10 is merely an example of the computer device 1000 and does not constitute a limitation on the computer device 1000. It may include more or fewer components than illustrated, or combine certain components, or use different components. For example, the computer device 1000 may also include input / output devices, network access devices, buses, etc.

[0197] The processor 1010 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0198] The memory 1020 may be an internal storage unit of the computer device 1000, such as a hard disk or memory of the computer device 1000. The memory 1020 may also be an external storage device of the computer device 1000, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 1000. Furthermore, the memory 1020 may include both internal storage units and external storage devices of the computer device 1000. The memory 1020 is used to store the computer program 1021 and other programs and data required by the computer device 1000. The memory 1020 can also be used to temporarily store data that has been output or will be output.

[0199] This application also discloses a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the methods described in the foregoing embodiments.

[0200] This application also discloses a computer-readable storage medium storing a computer program that, when executed by a computer, implements the methods described in the foregoing embodiments.

[0201] This application also discloses a computer program product, including a computer program that, when run on a computer, causes the computer to perform the methods described in the foregoing embodiments.

[0202] The embodiments described above are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method of cell counting, characterized by, include: Images of adherent stem cells were acquired; The trained detection and segmentation model is used to segment the adherent stem cell image to obtain multiple stem cell masks; The number of stem cells contained in the adherent stem cell image is determined based on multiple stem cell masks. The detection and segmentation model includes a backbone network structure, in which the convolutional layers are multi-layer pyramidal convolutions. These multi-layer pyramidal convolutions are used to extract multi-scale feature information from the adherent stem cell image.

2. The method of claim 1, wherein, Before segmenting the adherent stem cell image using the trained detection and segmentation model to obtain multiple stem cell masks, the process further includes: Generate a dataset for training the detection and segmentation model, the dataset containing multiple sample stem cell images labeled with mask tags, the mask tags being used to indicate stem cell regions in the sample stem cell images; The pre-built model is trained using the dataset to obtain the detection and segmentation model; The pre-built model includes the backbone network structure, the neck structure, and the head structure. Each convolutional layer in the neck structure is followed by an attention mechanism layer. The neck structure is used to fuse and enhance the multi-scale feature information, and the head structure is used to output the stem cell mask in the sample stem cell image.

3. The method of claim 2, wherein, The dataset generated for training the detection and segmentation model includes: The pre-labeling model is trained by using a segmentation model to predict the stem cell mask of the sample stem cell image, obtaining the corresponding mask label, and then inputting the mask label of the sample stem cell image into the segmentation model for training. The pre-labeling model is used to predict multiple stem cell images of the sample to obtain a mask label for each stem cell image of the sample. The mask label of each of the sample stem cell images is matched one-to-one with the corresponding sample stem cell image to obtain the dataset, which is divided into training dataset, validation dataset and test dataset according to the proportion.

4. The method according to claim 2 or 3, characterized in that, Before training the pre-built model using the dataset to obtain the detection and segmentation model, the process further includes: The dataset is preprocessed, which includes data augmentation of the sample stem cell images in the dataset and processing of the data-augmented sample stem cell images using a thermal diffusion algorithm.

5. The method of claim 4, wherein, The process of processing the data-enhanced stem cell image of the sample using a thermal diffusion algorithm includes: The thermal diffusion algorithm is used in each region of interest of each sample stem cell image labeled with a mask to iteratively diffuse from the center of each region of interest to simulate the creation of a topological map of the stem cell region in each sample stem cell image. The gradient vector fields in the x and y directions of each sample stem cell image are obtained based on the topological graph analysis, and the gradient vector fields are normalized.

6. The method according to any of claims 1 to 3 or 5, characterized in that, The trained detection and segmentation model is used to segment the adherent stem cell image to obtain multiple stem cell masks, including: Multi-scale feature information in the adherent stem cell image is extracted using multi-pyramid convolution in the detection and segmentation model. The multi-scale feature information is processed using the neck structure of the detection and segmentation model to obtain an output feature map; The head structure of the detection and segmentation model is used to extract each stem cell mask from the output feature map.

7. The method of claim 6, wherein, The process of using the neck structure of the detection and segmentation model to process the multi-scale feature information to obtain an output feature map includes: The attention mechanism layer in the neck structure is used to perform global average pooling on the multi-scale feature information. Perform a one-dimensional convolution operation on the multi-scale feature information after global average pooling, and determine the weights of each channel of the multi-scale feature information. The weights are multiplied by the corresponding elements in the original multi-scale feature information to obtain the output feature map.

8. The method according to any one of claims 1 to 3, 5, or 7, characterized in that, After determining the number of stem cells contained in the adherent stem cell image based on multiple stem cell masks, the method further includes: The area of ​​each stem cell and the total area of ​​stem cells in the adherent stem cell image are calculated based on each of the stem cell masks. The outline of each stem cell is determined, and the perimeter and roundness of each stem cell are calculated based on the outline.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it causes the computer device to implement the method as described in any one of claims 1 to 8.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is run, the method as described in any one of claims 1 to 8 is performed.

Images