Electron microscope red blood cell segmentation and positioning method based on UNet network and watershed algorithm

Abstract

The present invention relates to the technical field of electron microscope cell diagnosis, and specifically discloses a method for electron microscope red blood cell segmentation and positioning based on a UNet network and a watershed algorithm, comprising the steps of: S1, constructing a multi-attention segmentation model based on UNet to segment the foreground and background of electron microscope red blood cells; S2, positioning and segmenting the cells on the segmented graphics based on a watershed algorithm. The present invention first adopts methods such as data enhancement to expand the richness of the data set, and then adopts the designed multi-attention segmentation model Aca‑UNet to segment the foreground and background of cells. Finally, the segmented graphics are subjected to distance transformation and watershed correction, and then the single cell area is found to complete the positioning and segmentation of the cells. This method applies a computer-assisted algorithm to an electron microscope data set for the first time. In the experiment, the dice coefficient of dense cell segmentation reached 88.94%, and the dice coefficient of high-magnification cell segmentation reached 93.10%.

Description

Technical Field

[0001] The present invention relates to the technical field of electron microscope cell diagnosis, and in particular to an electron microscope red blood cell segmentation and positioning method based on a UNet network and a watershed algorithm. Background Art

[0002] Electron microscopy (EM) images have become increasingly popular in recent years as an important tool for diagnosing human diseases. Traditional EM pathology diagnosis relies primarily on pathologists determining and counting cell morphology and size in EM images. This significantly reduces physician efficiency and results in highly subjective results. Therefore, utilizing computer technology to assist physicians in EM image analysis is of great significance.

[0003] On the other hand, with the development of deep learning and computer hardware, an increasing number of deep learning methods are being applied to medical image processing. Some researchers have successfully solved the problem of fundus vessel segmentation using the Weighted Res-UNet model, which uses different weighted background and foreground loss calculations, and a residual network-based UNet model. Other researchers have used the UNet model for T cell spot detection, achieving significant improvements in segmentation accuracy compared to traditional region growing-based segmentation methods, validating the feasibility of deep learning models in medical imaging. In recent years, the application of deep learning models in medicine has been booming. Some researchers have proposed a multimodal feature fusion image segmentation network for brain tumor image segmentation, demonstrating that the addition of inter-level fusion and inter-layer skip connections improves the segmentation performance of multimodal medical images. They have also proposed a medical image segmentation network based on attention and pyramid fusion, effectively improving the edge blurring problem in segmented images. They have also proposed an attention mechanism module that incorporates edge enhancement to capture position-aware cross-channel information and more defined edge contours, thereby improving segmentation accuracy. These works provide practical insights for solving medical data problems using deep learning models.

[0004] Most existing research on abnormal red blood cell segmentation and counting focuses on blood smear datasets. However, these studies rely on the high-contrast and abundant data of blood smear datasets. Electron microscope images, however, have low contrast and limited data, making accurate and rapid segmentation and localization of blood cells a key challenge. Summary of the Invention

[0005] The present invention provides an electron microscope red blood cell segmentation and positioning method based on a UNet network and a watershed algorithm, which solves the technical problem of how to accurately and quickly segment and position blood cells.

[0006] To solve the above technical problems, the present invention provides an electron microscope red blood cell segmentation and positioning method based on the UNet network and the watershed algorithm, comprising the following steps:

[0007] S1. Construct a multi-attention segmentation model based on the UNet network to segment the foreground and background of red blood cells under electron microscopy;

[0008] S2. Based on the watershed algorithm, cells are located and segmented in the segmented graphics.

[0009] Furthermore, in step S1, the multi-attention segmentation model includes an encoder network and a decoder network designed in imitation of the UNet network, the encoder network includes a first residual block, a second residual block, a third residual block, a first self-attention mechanism block, and a second self-attention mechanism block from the first layer to the fifth layer, the decoder network includes a first depthwise separable convolution block, a second depthwise separable convolution block, a first decoder block, and a second decoder block from the fourth layer to the first layer, and the multi-attention segmentation model also includes four feature selection modules arranged in the jump connection stage of the corresponding layer;

[0010] The first and second self-attention mechanism blocks use convolution operations to extract low-dimensional features of the image, and at the same time perform long-range modeling of high-dimensional features of the image in the same way as Transformer.

[0011] The first depth-wise separable convolution block and the second depth-wise separable convolution block maintain the same size and resolution throughout the network, and separate the mixed spatial dimension and channel dimension by means of depth-wise separable convolution and point-wise convolution;

[0012] The feature selection module performs feature selection during feature fusion.

[0013] Furthermore, the feature selection module includes a channel attention mechanism module and a spatial attention mechanism module;

[0014] In the channel attention mechanism module, after the image is extracted by CNN features, the feature layer first obtains a feature map of size C*1*1 through an adaptive average pooling layer, where C is the number of feature channels. Then, convolution weighting is performed through a 1-dimensional convolution kernel with a convolution kernel size of k to achieve information interaction between different channels. The weight of each feature channel is then obtained through a sigmoid function, and the overall feature map is weighted to obtain a channel weighted map. Finally, the channel-weighted feature map is input into the spatial attention mechanism module.

[0015] In the spatial attention mechanism module, maximum pooling and average pooling operations are performed on the feature dimensions of the input feature map, and the feature dimensions are converted into 1*H*W, where H and W are the height and width of the original image, respectively. After the two feature maps are spliced, they are sent to a convolution kernel with an adjustable convolution kernel size for information interaction and feature reconstruction, and finally the spatial feature weight is obtained. It is weighted with the feature map processed by channel attention to obtain the final feature map.

[0016] Furthermore, in the first self-attention mechanism block or the second self-attention mechanism block, the feature map is first feature encoded, and the size of each word vector after encoding is P*P, and the number is H*W / (P*P), and then a depth-wise separable convolution operation is used to encode the position information.

[0017] Furthermore, in step S1, the loss function used in the process of training the multi-attention segmentation model is:

[0018]

[0019] in, represents BCE loss, represents the DICE loss, and has:

[0020]

[0021]

[0022] Among them, p and q represent the predicted pixel value and its actual pixel value respectively, w1 and w2 represent the weights of foreground and background loss respectively, k represents the kth pixel, n is the total number of pixels, X and Y represent the predicted element and the real element respectively, |X∩Y| represents the intersection of the predicted element and the real element, and |X|, |Y| represent the number of elements respectively.

[0023] Preferably, w1:w2=2:1.

[0024] Preferably, the data set used for training and testing the multi-attention segmentation model includes 48 dense cell images and 42 high-magnification images as well as images after data amplification of these images, and the corresponding label image is a binary image obtained by filling the target area of the original image with an image tool and then performing binarization and denoising processing.

[0025] Furthermore, the step S2 specifically includes the steps of:

[0026] S21, performing a rough segmentation of the foreground and background of the electron microscope image obtained in step S1 using a deep learning model, and converting the segmentation result into a binary image of 0 or 1, which is then grayscaled and displayed as a black and white image, where black represents background areas or non-cellular objects unrelated to the electron microscope cells, and white represents the target cell area;

[0027] S22, dilating the original coarsely segmented image to obtain a background area as the background of the image, and performing distance transformation on the foreground area to obtain a suitable distance value as the foreground of the image;

[0028] S23, marking the area between the foreground and background of the image as an unknown area, and using a watershed algorithm to divide the foreground and background of the unknown area;

[0029] S24: Perform connected area determination on the divided image, segment out individual cell areas and obtain position information.

[0030] The present invention provides a method for electron microscopy red blood cell segmentation and localization based on a UNet network and a watershed algorithm. The method first uses methods such as data augmentation to expand the richness of the dataset. Then, a designed multi-attention segmentation model, Aca-UNet, is used to segment the cell foreground and background. Finally, the segmented image is subjected to distance transformation and watershed correction to locate individual cell regions, completing cell localization and segmentation. This method is the first to apply a computer-assisted algorithm to electron microscopy datasets. In experiments, the DICE coefficient for dense cell segmentation reached 88.94%, and the DICE coefficient for high-magnification cell segmentation reached 93.10%. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 2 are examples of two types of electron microscope image data sets provided by embodiments of the present invention;

[0032] Figure 2 This is an image preprocessing effect diagram provided by an embodiment of the present invention;

[0033] Figure 3 This is a diagram of the Aca-UNet network structure provided by an embodiment of the present invention;

[0034] Figure 4 It is a UNet skeleton network diagram provided by an embodiment of the present invention;

[0035] Figure 5 is a structural diagram of a feature selection module provided by an embodiment of the present invention;

[0036] Figure 6 is a structural diagram of the self-attention mechanism block provided by an embodiment of the present invention;

[0037] Figure 7 is a structural diagram of a depthwise separable convolution module provided by an embodiment of the present invention;

[0038] Figure 8 This is a diagram showing the steps of performing cell segmentation and counting based on the watershed algorithm provided by an embodiment of the present invention;

[0039] Figure 9 This is an example diagram of dense cell segmentation and counting provided by an embodiment of the present invention;

[0040] Figure 10 This is a high-magnification cell segmentation and counting diagram provided by an embodiment of the present invention;

[0041] Figure 11 This is a comparison chart of the effects of different model test sets provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The following describes the embodiments of the present invention in detail with reference to the accompanying drawings. The embodiments are provided for illustrative purposes only and are not to be construed as limiting the present invention. The accompanying drawings are provided for reference and illustration only and do not constitute a limitation on the scope of protection of the present invention. Many changes may be made to the present invention without departing from the spirit and scope of the present invention.

[0043] In view of the low contrast and small amount of data in electron microscope red blood cell images, an embodiment of the present invention provides an electron microscope red blood cell segmentation and positioning method based on a UNet network and a watershed algorithm, comprising the following steps:

[0044] S1. Build a multi-attention segmentation model based on the UNet network (called Aca-UNet network in this example) to segment the foreground and background of red blood cells under electron microscopy;

[0045] S2. Based on the watershed algorithm, cells are located and segmented in the segmented graphics.

[0046] The dataset used in this article comes from the First Affiliated Hospital of Chongqing Medical University. Unlike traditional blood images, electron microscope images have low contrast and high resolution. The images are presented in grayscale, and the foreground and background are seriously mixed. Therefore, they are not suitable for segmentation using traditional segmentation methods such as Otsu threshold (OTSU). This article designs a suitable deep learning model for cell positioning, segmentation and counting. The electron microscope image datasets used in this article can be divided into dense cell datasets and high-magnification datasets, among which some image data examples are as follows: Figure 1 As shown, Figure 1 (a) is a high-magnification image, and (b) is a dense cell image.

[0047] for Figure 2(a) The original image of the example also needs to generate the corresponding label image. In order to ensure that as much boundary information as possible is retained, this paper does not use the traditional anchor point demarcation method, but uses image tools such as Photoshop to fill the target area (the area of interest, i.e., the red blood cell area, is filled with white). However, the final filled image is not a simple binary image, such as Figure 2 (b) There are many noise points with unclear brightness and darkness, so it is necessary to perform binary denoising on the manually filled image. The resulting label map is as follows: Figure 2 (c) shown.

[0048] Due to the specific nature of electron microscopy images, the data provided by the collaborating institutions was limited, consisting of only 48 dense cell images and 42 high-magnification images. Therefore, this paper used spatial transformations such as flipping and rotation to expand the semantic richness of the dataset. To preserve the relevant information of the original electron microscopy images, this paper avoided using color transformations and other related operations. After data augmentation, this paper selected 20% of the data as a test set, and the rest as a training set, for data feature learning and model validation of electron microscopy images.

[0049] The Aca-UNet network proposed in this paper is based on the encoder and decoder structure of the UNet network, which mainly includes four parts: (1) UNet basic skeleton; (2) feature selection module; (3) self-attention mechanism module for high-level semantic features (including self-attention mechanism block 1 and self-attention mechanism block 2); (4) depth-wise separable convolution module (including depth-wise separable convolution block 1 and depth-wise separable convolution block 2). The overall simple structure of the model is as follows Figure 3 Specifically, the multi-attention segmentation model includes an encoder network and a decoder network designed based on the UNet network. The encoder network includes the first residual block (i.e., residual block 1), the second residual block (i.e., residual block 2), the third residual block (i.e., residual block 3), the first self-attention mechanism block (i.e., self-attention mechanism block 1), and the second self-attention mechanism block (i.e., self-attention mechanism block 2) from the first to the fifth layers. The decoder network includes the first depthwise separable convolution block (i.e., depthwise separable convolution block 1), the second depthwise separable convolution block (i.e., depthwise separable convolution block 2), the first decoder block (i.e., decoder block 1), and the second decoder block (i.e., decoder block 2) from the fourth to the first layers. The multi-attention segmentation model also includes four feature selection modules (referred to as FS) set in the jump connection stage of the corresponding layers.

[0050] The UNet framework is the backbone of the Aca-UNet network. Its core components are the downsampling module for extracting image features and the upsampling module for restoring image resolution. The feature selection module is designed to fully integrate image channel and spatial feature information. The self-attention mechanism module globally models the image using high-level semantic information. The depthwise separable convolution module can improve model performance while reducing the number of model parameters.

[0051] Specifically, the UNet basic skeleton, as a classic network in the field of medical image segmentation, has also derived many variants in recent years. However, its core idea has remained unchanged. Its structure mainly consists of a downsampling path (encoder network) that captures global information and an upsampling path (decoder network) that gradually restores the pixel scale. It has performed well in many segmentation tasks in recent years. In this article, this article follows the basic architecture of the UNet network and makes some adjustments on this basis. In addition, due to the particularity and specificity of the task of segmenting cell images in this article, this article adjusts the number of relevant feature channels of the network to reduce model overhead. The UNet network skeleton structure used in this article is as follows Figure 4 As shown. Combined Figure 3 and Figure 4 It can be seen that the input image of the Aca-UNet network in this paper is a 3×512×512 image. After downsampling by residual block 1, the number of channels of the image becomes 32. After downsampling by residual block 2, the number of channels of the image becomes 64, and the height and width become 1 / 2 of the original image respectively. After downsampling by residual block 3, the number of channels of the image becomes 128, and the height and width become 1 / 4 of the original image respectively. After downsampling by self-attention mechanism block 1, the number of channels of the image becomes 256, and the height and width become 1 / 8 of the original image respectively. After downsampling by self-attention mechanism block 2, the number of channels of the image becomes 512, and the height and width become 1 / 16 of the original image respectively. The specific structure of the residual block is as follows Figure 3 Shown in the right half.

[0052] In order to solve the problem of information loss during downsampling of the UNet module, this paper designs a feature selection module in the UNet jump connection stage to perform feature selection during feature fusion. Figure 5As shown in the figure, it includes a channel attention mechanism module and a spatial attention mechanism module. First, after the image is extracted through CNN features, the feature layer first passes through an adaptive average pooling layer to obtain a feature map of size C*1*1, where C is the number of feature channels. Then, convolution weighting is performed using a 1-dimensional convolution kernel with a kernel size of k to achieve information exchange between different channels. A sigmoid function is then applied to each feature channel to obtain the weight of the entire feature map, and the overall feature map is weighted to obtain a channel-weighted map. The channel-weighted feature map is then input into the spatial attention mechanism module. First, max pooling and average pooling (taking the mean) are performed on the feature dimensions of the feature map, converting the feature dimensions to 1*H*W (H and W are the height and width of the original image, respectively). These two feature maps are concatenated and fed into a convolution kernel with an adjustable kernel size for information exchange and feature reconstruction, ultimately obtaining the spatial feature weights. This weighted feature map is then weighted with the feature map processed by channel attention to obtain the final feature map.

[0053] In order to model the long-distance semantic dependency of feature maps and better extract the semantic features of images, this paper uses convolution operations to extract low-dimensional features of images, and at the same time imitates Transformer to build a self-attention mechanism module to perform long-distance modeling of high-dimensional features of images. Figure 6 As shown in the figure, the feature map is first encoded. After encoding, each word vector has a size of P*P and a number of H*W / P*P. Regarding positional information encoding, this paper uses a depthwise separable convolution (DW) to encode positional information. This module can effectively encode position and has a small number of parameters. In specific implementation, the image is first subjected to feature extraction by the corresponding convolution block of the UNet network to form a feature map. This feature map is then fed into a convolution module with a kernel size of k and a stride of s for convolution and word embedding. Next, the word-embedded feature map undergoes dimensionality transformation, transforming the C*H*W feature map into a P*C word block, where P is the product of H and W. Subsequently, the word-vectorized word blocks are normalized. To fully explore the relationships between each word block, a multi-head attention mechanism is used for long-range modeling. The core idea is to form a q, k, and v matrix for each word block. The q value of each word block is used to calculate the similarity with the k values of the remaining word blocks, and then multiply it with the corresponding v value to obtain the final result. Next, residual connection and normalization are used in sequence, and the latent features are learned by the fully connected layer. The convolution position encoding is performed by the depthwise separable convolution. The hidden features are learned again by the fully connected layer, and the dimension of the word block is restored to the C*H*W image feature block to obtain the final result.

[0054] In order to alleviate the problem that the segmentation model parameters are too large and difficult to deploy, this paper uses a depthwise separable convolution module to model the feature map. This module can separate the mixed spatial dimensions and channel dimensions with the help of Depthwise convolution (DW convolution) and Pointwise convolution while maintaining the same size and resolution throughout the network. However, while reducing the number of model parameters, this module will also reduce the model's inference speed and training speed. Therefore, this paper only uses DW convolution when upsampling the model. The specific structure of the depthwise separable convolution module is as follows Figure 7 As shown. The specific implementation is to use a convolution module with a convolution kernel size of k and a stride of s to perform a convolution operation on the input image features. After convolution, the image feature size becomes 1 / s of the original. The convolution image is subjected to a depthwise convolution operation. If the number of channels of the input image feature is C, the number of convolution kernels for depthwise convolution is C, and each convolution kernel is convolved with one channel of the input feature to learn related features. After the GELU activation function and batch normalization operation, the image features are then convolved by pointwise convolution. The pointwise convolution is a convolution kernel with a kernel size of 1, and its number determines the number of feature channels of the output image. Finally, the final image features are obtained after the GELU activation function and batch normalization operation again.

[0055] This paper adopts a hybrid loss function, using the sum of the DICE Loss and BCE Loss to measure the gap between the predicted feature map and the true feature map. The overall loss function formula is shown in formula (3). This paper takes the sum of BCE Loss and DICE Loss as the total loss. Where DICE Loss is shown in formula (1), p and q represent the predicted pixel value and its actual pixel value respectively. w1 and w2 represent the weights when calculating the loss of foreground and background respectively. This paper is a two-class foreground and background problem. In order to better distinguish individual cells, this paper increases the weight of the model segmentation background so that the model of this paper pays more attention to the image background and the boundaries between each cell. BCE Loss is shown in formula (2), where |X∩Y| represents the intersection of the predicted element and the true element, and |X|, |Y| represent the number of elements respectively.

[0056]

[0057]

[0058]

[0059] The watershed algorithm is a regional image segmentation algorithm based on mathematical morphology theory and is widely used in many image segmentation post-processing. In this paper, after the electron microscope cell image is preliminarily segmented, the watershed algorithm is used to locate and count the segmented image. The flow chart is as follows Figure 8 As shown, it includes rough segmentation of the foreground and background of the electron microscope image through a deep learning model, opening operation, distance transformation, foreground and background determination, unknown area marking, watershed algorithm transformation, contour search and other operations.

[0060] 1) After appropriate training, the proposed deep learning model structure is used to perform preliminary segmentation of the foreground and background of the electron microscope image. The segmentation result is a binary image of 0 or 1, which is then grayscaled and displayed as a black and white image, where black represents background areas or non-cellular objects unrelated to the electron microscope cells, and white represents the target cell area;

[0061] 2) Opening is a widely used method in image processing. It removes isolated points and separates two objects that are slightly connected by performing an erosion operation followed by a dilation operation.

[0062] 3) Distance transform was first proposed by scholars in 1966 and is now widely used in the field of imaging. It transforms the binary images of foreground and background into grayscale images, where the grayscale value of a pixel in the image is the distance between the pixel and the nearest background pixel. By selecting the appropriate distance value, this paper obtains the foreground area of the image cell.

[0063] 4) In this paper, the background area obtained by dilating the original coarse segmentation image is used as the background of the image, and the foreground image obtained by screening the appropriate distance value and performing distance transformation is used as the foreground of the image;

[0064] 5) To finely segment the foreground and background of cells, the area between the foreground and background of the selected image is the unknown area. In this paper, the watershed algorithm is used to divide the foreground and background of the unknown area;

[0065] 6) The segmented image is subjected to connected area determination, and individual cell areas are segmented and position information is obtained.

[0066] Correspondingly, step S2 generally includes the following steps:

[0067] S21, performing a rough segmentation of the foreground and background of the electron microscope image obtained in step S1 using a deep learning model, and converting the segmentation result into a binary image of 0 or 1, which is then grayscaled and displayed as a black and white image, where black represents background areas or non-cellular objects unrelated to the electron microscope cells, and white represents the target cell area;

[0068] S22, dilating the original coarsely segmented image to obtain a background area as the background of the image, and performing distance transformation on the foreground area to obtain a suitable distance value as the foreground of the image;

[0069] S23, marking the area between the foreground and background of the image as an unknown area, and using a watershed algorithm to divide the foreground and background of the unknown area;

[0070] S24: Perform connected area determination on the divided image, segment out individual cell areas and obtain position information.

[0071] The segmentation counting example is shown in the figure Figure 9 、 10 As shown, Figure 9 Corresponding to the dense cell map, Figure 10 Corresponding high-magnification image.

[0072] The following is the experiment and analysis of its results.

[0073] (1) Preliminary image segmentation

[0074] After denoising and data enhancement of the image dataset, this paper sets the initial learning rate to 0.001, the batch size to 4, the number of training rounds to 40, and uses the adam optimizer for gradient update. After completing all rounds of training, this paper obtains the final training model. This paper compares the performance of the proposed model with the recently popular segmentation model. The comparison results of the dense cell dataset, i.e., the magnified cell dataset, are shown in Table 1 and Table 2, respectively. Among them, the model proposed in the present invention performs poorly on the high-magnification training dataset. It is speculated that this is because the single cell area of the high-magnification dataset is too large, and the distance between single cells is too small, and the self-attention mechanism cannot well construct global context information through the connection between the encoding word blocks. Therefore, the present invention uses DW convolution to extract image features, and the comparison results are shown in Table 2.

[0075] Table 1 Comparison of dense cell training effects

[0076]

[0077]

[0078] Table 2 Comparison of enlarged cell training effects

[0079]

[0080] exist Figure 11 ,This paper compares the test set effects of different models. It can be seen that the model in this paper has strong expressiveness in over-segmentation, under-segmentation and detail processing, and is closer to the real label.

[0081] (2) Watershed segmentation post-processing and counting

[0082] In measuring the accuracy of segmentation and counting, this paper uses TP, TN, FP, and FN to measure the performance of the algorithm. Among them, TP represents the number of cells that are correctly segmented (i.e., a single cell is segmented into a single cell), TN represents the number of cells that are incorrectly segmented (i.e., a single cell is segmented into multiple cells or not segmented), FT represents the number of objects that are not cells that are correctly segmented, and FN represents the number of objects that are not cells that are segmented into cells. Because objects that are not cells are mainly filtered during the initial segmentation, the values of FN and FT are determined by the performance of the model proposed in this paper. This paper uses the proportion of correctly segmented cells to all cells (ACC) to measure the overall segmentation effect. This paper randomly selects several images for testing, and some test images are as follows Figure 11 The test indicators are shown in Table 3 (No. 1 to 3 correspond to Figure 11 Dense cells in Figures 1 to 3 , numbers 4 and 5 correspond to Figure 11 High magnification in Figure 1 、 2 ).

[0083] Table 3

[0084]

[0085] From Table 3 and Figure 11 It can be seen that the segmentation effect of dense cells is perfect, and the segmentation effect of high-magnification cell images is poor due to the high degree of cell overlap, but the overall segmentation effect is good, which meets the segmentation requirements of this article.

[0086] (3) Ablation experiment of attention mechanism

[0087] This paper conducts ablation experiments on the proposed model to verify the effectiveness of the model. Figure 3 As shown in Table 4, this paper removes the self-attention mechanism module (ACa-UNet_mini1) and the feature selection module (ACa-UNet_mini2) in turn, and observes the transformation of the model performance. The overall effect is shown in Table 4.

[0088] Table 4 Ablation experiment of attention mechanism

[0089]

[0090] It can be seen that the Aca-UNet proposed in the present invention is optimal in all aspects, and the self-attention mechanism module and feature selection module have made substantial contributions to the effect of the present invention.

[0091] (4) Ablation experiment of loss function weight

[0092] To explore the impact of the loss function on the performance of our model, we conducted ablation experiments on the loss function. We set the background-to-foreground ratio to 1:1, 1:2, and 2:1, respectively, and used the proposed model to train on the dataset. The training results are shown in Table 5 below, using Loss 1, Loss 2, and Loss 3, respectively.

[0093] Table 5 Loss function ablation experiment

[0094]

[0095] As shown in Table 5, when the foreground-background weight ratio is 2:1 (i.e., w1:w2 = 2:1), the training effect is the best, which verifies the rationality of focusing more on the background area in order to segment single cells in this paper.

[0096] In summary, this paper proposes its own solution to the problem of automatic segmentation and counting of electron microscopy cells, filling a gap in this field. For dense electron microscopy cell images, this paper uses feature channel selection fusion and self-attention mechanism long-distance modeling to improve the effectiveness of image semantic feature extraction. High-magnification images have limitations in their own semantic features, namely, the large area of a single cell and the small cell spacing. This paper uses channel selection fusion and depthwise separable convolution to extract semantic features, and then combines it with the watershed algorithm to segment and count cells. Among them, the DICE coefficient for dense cell segmentation reached 88.94%, and the DICE coefficient for high-magnification cell segmentation reached 93.10%. The counting and watershed processing indicators are in line with expectations.

[0097] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. The electron microscope red blood cell segmentation and positioning method based on UNet network and watershed algorithm is characterized by: Including steps: S1. Construct a multi-attention segmentation model based on the UNet network to segment the foreground and background of red blood cells under electron microscopy; In step S1, the multi-attention segmentation model includes an encoder network and a decoder network designed in imitation of the UNet network, the encoder network includes a first residual block, a second residual block, a third residual block, a first self-attention mechanism block, and a second self-attention mechanism block from the first layer to the fifth layer, the decoder network includes a first depth-separable convolution block, a second depth-separable convolution block, a first decoder block, and a second decoder block from the fourth layer to the first layer, and the multi-attention segmentation model also includes four feature selection modules arranged in the jump connection stage of the corresponding layer; The first and second self-attention mechanism blocks use convolution operations to extract low-dimensional features of the image, and at the same time perform long-range modeling of high-dimensional features of the image in the same way as Transformer. The first depth-wise separable convolution block and the second depth-wise separable convolution block maintain the same size and resolution throughout the network, and separate the mixed spatial dimension and channel dimension by means of depth-wise separable convolution and point-wise convolution; The feature selection module performs feature selection when performing feature fusion; The feature selection module includes a channel attention mechanism module and a spatial attention mechanism module; In the channel attention mechanism module, after the image is extracted by CNN features, the feature layer first obtains a feature map of size C*1*1 through an adaptive average pooling layer, where C is the number of feature channels. Then, convolution weighting is performed through a 1-dimensional convolution kernel with a convolution kernel size of k to achieve information interaction between different channels. The weight of each feature channel is then obtained through a sigmoid function, and the overall feature map is weighted to obtain a channel weighted map. Finally, the channel-weighted feature map is input into the spatial attention mechanism module. In the spatial attention mechanism module, the feature dimensions of the input feature map are subjected to maximum pooling and average pooling operations respectively, and the feature dimensions are converted to 1*H*W, where H and W are the height and width of the original image respectively. After the two feature maps are spliced, they are sent to a convolution kernel with an adjustable kernel size for information interaction and feature reconstruction. Finally, the spatial feature weight is obtained, which is weighted with the feature map processed by channel attention to obtain the final feature map; S2. Position and segment cells in the segmented graph based on the watershed algorithm; The step S2 specifically includes the following steps: S21, performing a rough segmentation of the foreground and background of the electron microscope image obtained in step S1 using a deep learning model, and converting the segmentation result into a binary image of 0 or 1, which is then grayscaled and displayed as a black and white image, where black represents background areas or non-cellular objects unrelated to the electron microscope cells, and white represents the target cell area; S22, dilating the original coarsely segmented image to obtain a background area as the background of the image, and performing distance transformation on the foreground area to obtain a suitable distance value as the foreground of the image; S23, marking the area between the foreground and background of the image as an unknown area, and using a watershed algorithm to divide the foreground and background of the unknown area; S24: Perform connected area determination on the divided image, segment out individual cell areas and obtain position information.

2. The electron microscope red blood cell segmentation and positioning method based on the U Net network and the watershed algorithm according to claim 1, characterized in that: In the first self-attention mechanism block or the second self-attention mechanism block, the feature map is first feature encoded. The size of each word vector after encoding is P*P, and the number is H*W / (P*P). Then, a depth-wise separable convolution operation is used to encode the position information.

3. The electron microscope red blood cell segmentation and positioning method based on UNet network and watershed algorithm according to claim 2 is characterized in that: In step S1, the loss function used in the process of training the multi-attention segmentation model is: ， in, represents BCE loss, represents the DICE loss, and has: ， ， in, p and q Represent the predicted pixel value and its actual pixel value respectively, Represent the weights of foreground and background when calculating losses, k Indicates the k pixels, n is the total number of pixels, Represent the predicted elements and the real elements respectively, represents the intersection of the predicted element and the true element, Indicates the number of elements in each.

4. The electron microscope red blood cell segmentation and positioning method based on UNet network and watershed algorithm according to claim 3 is characterized in that: 。 5. The electron microscope red blood cell segmentation and positioning method based on the UNet network and the watershed algorithm according to any one of claims 1 to 4, characterized in that: The dataset used for training and testing the multi-attention segmentation model includes 48 dense cell images and 42 high-magnification images, as well as images after data amplification of these images. The corresponding label image is a binary image obtained by filling the target area of the original image with an image tool and then performing binarization and denoising.

Images