A cervical cell nucleus segmentation method based on shallow semantic feature enhancement

Abstract

The application provides a cervical cell nucleus segmentation method based on shallow semantic feature strengthening, and belongs to the technical field of computer vision. The application comprises the following steps: preparing sample data sets; using an Opencv library to perform data enhancement; data set labeling; learning a cell nucleus and non-cell nucleus by using a model; using the Opencv library to draw a cell nucleus boundary; and finally displaying a segmentation result on an original image. The application optimizes the disadvantages of a traditional U-net network model encoder structure, weakens the influence of false positive impurities on segmentation accuracy, and has good segmentation effect for cell nucleus segmentation.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method for cervical cell nucleus segmentation based on shallow semantic feature enhancement. Background Technology

[0002] Currently, methods for segmenting cervical cell nuclei can be broadly categorized into two main types: those based on traditional methods and those based on deep learning methods. Traditional methods for cervical cell segmentation are further divided into region-based segmentation methods and edge-based segmentation methods.

[0003] Region-based segmentation methods are segmentation techniques that rely on identifying regions and merging regions with similar characteristics to achieve segmentation. These methods include seed region growing, watershed methods, and region splitting and merging. While these methods fully utilize the image features of the samples (grayscale, texture, intensity, etc.), cervical cell samples contain impurities such as cell debris and blood. These impurities are highly similar to cell nuclei in image features, making it difficult for these methods to identify them, resulting in false positives.

[0004] Edge-based segmentation methods utilize both the direction and amplitude of edges for segmentation. Pixel grayscale changes slowly or not at all along the cell edge direction, while pixel grayscale changes drastically perpendicular to the edge. Common methods include gradient operator methods, active contour models, and balloon theory models. These methods rely on the direction and amplitude attributes of the image's edges. However, for overlapping cervical cell nuclei, the edges are not distinct, and their amplitude and direction attributes are not obvious, making it difficult to improve the segmentation accuracy of these methods for overlapping cervical cell nuclei. Among deep learning-based segmentation methods, the U-Net network is widely used. The unique skip connection structure of the U-Net network better utilizes the semantic information of the image, resulting in higher segmentation accuracy compared to traditional methods. However, its simple skip connection structure leads to low utilization of image semantic information, resulting in a certain degree of false positive segmentation. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a cervical cell nucleus segmentation method based on shallow semantic feature enhancement. This method can accurately segment single-nucleus and overlapping nuclei, avoiding false-positive segmentation.

[0006] The technical solution of this invention is: A cervical cell nucleus segmentation method based on shallow semantic feature enhancement, comprising: Creating a sample dataset Data augmentation using the OpenCV library; Dataset labeling; The model learns about both the cell nucleus and non-cell nuclei; Use the OpenCV library to draw the cell nucleus boundary; Finally, the segmentation results are displayed on the original image.

[0007] Furthermore, The dataset consists of two main parts: a public dataset and a self-built dataset. The public dataset is derived from publicly available datasets. The self-built dataset includes collected cell smear images.

[0008] Using a self-built dataset and a public dataset, slices were created to extract impurity samples that do not contain cell nuclei but have similar morphology, grayscale, and other characteristics to cell nuclei. These impurity samples are false positive samples.

[0009] Furthermore, Use OpenCV libraries to add Gaussian noise, rotate and flip (horizontally and vertically), and make the samples transparent. The publicly available dataset already contains cell nucleus annotations, and the annotations also need to be expanded in the same way after the OpenCV library is expanded.

[0010] The network integrates feature layers of different scales and performs full-scale supervision to enhance feature learning; a false positive impurity guidance module is set to reduce false positive segmentation. The single-scale skip connections are transformed into multi-scale skip connections; the source of the network encoder feature layers is: encoder feature layers of the same depth and all encoder feature layers of smaller depth; the encoder feature map of this network contains several decoder feature layers, which is the multi-scale feature fusion mechanism.

[0011] Meanwhile, the network uses a full-scale supervision mode, which allows the network to focus on features at every depth to enhance feature learning.

[0012] The false positive impurity guidance module provides constraints for the network optimization process. Before loss calculation, a binary classification task is performed to determine whether the feature map is a cell nucleus. For feature maps that are cell nuclei, loss function calculation is allowed to optimize the network in reverse. For feature maps that are not cell nuclei, i.e. impurities, loss calculation is rejected to correct the network. This design aims to reduce false positive segmentation.

[0013] Furthermore, The false positive guidance module consists of three main parts. The first part contains a 3x3 convolutional layer, a normalization layer, and a ReLU activation function layer. The second part is an adaptive max pooling layer. The last part is a dropout layer. The EN5 layer, which contains the most features, is selected for classification task prediction. After passing through the false positive guidance module, the EN5 layer will output two weights, probability, representing the weights of cell nuclei and non-cell nuclei. With the help of the argmax function, the weights can be converted into two tensors {0,1}, where 1 represents cell nuclei and 0 represents non-cell nuclei. This is a binary classification task. The classification task result and the segmentation task are multiplied. The segmentation result of non-cell nuclei will be suppressed after being multiplied with the classification task result (0) and will not be subject to loss function calculation, thus correcting the network. The cell nucleus segmentation result and the classification task result (1) will continue to be multiplied and the loss function will be calculated to optimize the network.

[0014] The beneficial effects of this invention are The method proposed in this invention has higher segmentation accuracy than traditional methods, while optimizing the drawbacks of the encoder structure of the traditional U-net network model, weakening the impact of false positives on segmentation accuracy, and achieving good segmentation results for cervical cell nuclei. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the workflow steps of the present invention; Figure 2 This is a schematic diagram of the FalsePositiveGuide module. Implementation

[0016] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0017] This invention proposes a novel cell nucleus segmentation method based on the U-net network in deep learning. In cervical cell nucleus segmentation, the overlapping nature of cervical cells leads to blurred nucleus boundaries, a problem termed the "weak edge problem." This invention addresses this weak edge problem by improving the skip connection mechanism of the U-net network and incorporating more shallow feature maps rich in semantic information. Furthermore, blood clots and cell fragments on cervical smears exhibit high feature similarity to cervical cell nuclei, easily resulting in false-positive segmentation. This invention proposes a segmentation method based on shallow semantic feature enhancement to address the problem of segmenting overlapping cervical cell nuclei. Specifically, the method incorporates more false positive samples in the dataset to improve the network's learning of sample features, and uses the OpenCV library for data broadening to enhance the generalization ability of the scheme. Furthermore, the network structure design strengthens the utilization rate of the semantic feature layer. Ultimately, the network achieves the goal of segmenting overlapping cervical cell nuclei through learning.

[0018] Dataset creation The dataset used in this invention comprises two main parts: a publicly available dataset and a self-built dataset. The publicly available dataset originates from the Overlapping Cervical Cytology Image Segmentation Challenge (ISBI 2014). For the self-built dataset, this invention collected cervical cell smear images from hospitals. Furthermore, to broaden the dataset, we used both the self-built and publicly available datasets to slice samples, extracting impurity samples (blood, cytoplasmic fragments, bacteria, etc.) that do not contain cell nuclei but whose morphology, grayscale, and other features are extremely similar to cell nuclei. This invention refers to these impurity samples as false positive samples. This design enhances the network's learning of false positive sample features from the perspective of the dataset, thereby better distinguishing cell nuclei from impurities and improving segmentation accuracy.

[0019] Data augmentation using the OpenCV library To improve the network's ability to learn features, the OpenCV library was used to expand the dataset by adding Gaussian noise (standard deviation of 5 and 10), rotating (45° and 90°), flipping (horizontal and vertical), perspective transformation, brightness adjustment (brightening by 30% and 50%), and image stitching. This expanded the dataset collected in the first stage by 10 times, thereby improving the model's generalization ability.

[0020] Dataset labeling The publicly available dataset used in this invention already contains annotations of cell nuclei; therefore, after expanding the OpenCV library, the annotations also need to be expanded in the same way. The annotation of the self-built dataset and its expanded images was completed under the guidance of professional doctors.

[0021] Network model design This paper proposes an improved U-shaped cervical cell nucleus segmentation network based on shallow feature enhancement, building upon the U-net network. The specific network structure is shown below. The network integrates feature layers of different scales and performs full-scale supervision to enhance feature learning; a false positive impurity guidance module is designed to reduce false positive segmentation.

[0022] Cell nuclei and false positive impurities possess different semantic information. To distinguish between them, we enhance feature fusion at different depths. Decoder features are fused with decoder feature maps of the same depth and encoder feature maps of all higher depths, then skip connections are made to learn semantic information at different depths. Compared to traditional methods, the proposed method not only focuses on image features but also fully utilizes semantic features, enabling better segmentation of overlapping cervical cell nuclei. Specifically, this invention changes the skip connection method of the Unet network, transforming single-scale skip connections into multi-scale skip connections. The encoder feature layers proposed in this invention are derived from: encoder feature layers of the same depth and all encoder feature layers of lower depth. Taking the DE4 feature layer as an example: EN1, EN2, EN3, and EN4 have different sizes, so firstly, max pooling is used to adjust all feature layers from the encoder to the same size as the EN4 feature layer. Then, the EN5 feature layer is upsampled to the same size as EN4. Subsequently, convolution is performed to maintain a consistent number of features across the feature layers. Finally, the transformed feature layers are concatenated to obtain the DE4 layer. Other decoder feature layers are constructed in a similar manner. Compared to the U-net network, the encoder feature map of this network contains multiple decoder feature layers, which is the multi-scale feature fusion mechanism proposed in this invention. Accurate segmentation of cell nuclei requires more guidance from semantic information, which often resides in the shallow encoder feature layers. This invention, through the design of multi-scale (different depths) feature layer fusion, allows the network to learn semantic information at more scales (depths), thereby enhancing feature learning.

[0023] Simultaneously, the network employs a full-scale supervised mode, enabling it to focus on features at every depth level and thus enhance feature learning. The network proposed in this invention learns cell nucleus features at a deeper level, thereby significantly improving segmentation accuracy. Furthermore, this invention guides the network to learn overlapping cervical cell nuclei boundaries by manually annotating cytoplasmic boundaries.

[0024] This invention also proposes a FalsePositiveGuide module to provide constraints for the network optimization process. Specifically, before loss calculation, a binary classification task is performed to determine whether the feature map represents a cell nucleus. For feature maps that are cell nuclei, loss function calculation is allowed for backpropagation of the network; for feature maps that are not cell nuclei (i.e., impurities), loss calculation is rejected for network correction. This design aims to reduce false positive segmentation. The structure of the FalsePositiveGuide module is shown below. Figure 2 As shown.

[0025] The false positive guidance module consists of three main parts. The first part includes a 3*3 convolutional layer, a normalization layer, and a ReLU activation function layer. The second part is an adaptive max pooling layer. The last part is a dropout layer. In this invention, the EN5 layer, which contains the most features, is selected for classification task prediction. After passing through the false positive guidance module, the EN5 layer outputs two weights, probability, representing the weights of cell nuclei and non-cell nuclei. With the help of the argmax function, the weights can be converted into two tensors {0,1}, where 1 represents a cell nucleus and 0 represents a non-cell nucleus. This is a binary classification task. This invention uses the classification task to guide the segmentation task. The classification task result and the segmentation task are multiplied. The segmentation result of non-cell nuclei will be suppressed after multiplying with the classification task result (0) and will not be subject to loss function calculation to correct the network. The segmentation result of cell nuclei will continue to be subject to loss function calculation after multiplying with the classification task result (1) to optimize the network. This design achieves the purpose of suppressing false positive segmentation.

[0026] Results visualization Finally, the segmentation results are displayed on the original image.

[0027] To improve the network's learning of sample features, more false positive samples were incorporated into the dataset. Meanwhile, OpenCV was used to broaden the data and enhance the generalization ability of the solution. The network structure was designed to maximize the utilization of semantic feature layers. Ultimately, the network's learning process achieved the goal of segmenting overlapping cervical cell nuclei.

[0028] The above description is merely a preferred embodiment of the present invention and is used only to illustrate the technical solution of the present invention, and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for cervical cell nucleus segmentation based on shallow semantic feature enhancement, characterized in that, include: Create a sample dataset; Data augmentation using the OpenCV library; Dataset labeling; Network learning of cell nuclei and non-cell nuclei; Use the OpenCV library to draw the cell nucleus boundary; Finally, the segmentation results are displayed on the original image; The dataset consists of two main parts: a public dataset and a self-built dataset. The public dataset is derived from publicly available datasets. The self-built dataset includes collected cell smear images. Slicing was performed using a self-built dataset and a public dataset. Impurity samples that do not contain cell nuclei but have similar morphology, grayscale features and cell nuclei were sliced ​​out. These impurity samples were false positive samples. The dataset can be expanded using OpenCV library techniques such as adding Gaussian noise, rotation, horizontal and vertical flipping, perspective transformation, brightness adjustment, and image stitching. The publicly available dataset already contains cell nucleus annotations, and the annotations were also expanded in the same way after the OpenCV library was used for expansion. The network integrates feature layers of different scales and performs full-scale supervision to enhance feature learning; a false positive impurity guidance module is set to reduce false positive segmentation. The single-scale skip connections are transformed into multi-scale skip connections; wherein, the input feature map of each feature layer of the decoder comes from: the encoder feature layer with the same depth index as the decoder feature layer, all encoder feature layers with depth indices less than the decoder feature layer, and all encoder feature layers with depth indices greater than the decoder feature layer; each feature layer of the decoder fuses feature maps of several depth scales output by the encoder to achieve multi-scale feature fusion. The false positive impurity guidance module provides constraints for the network optimization process. Before loss calculation, a binary classification task is performed to determine whether the feature map is a cell nucleus. For feature maps that are cell nuclei, loss function calculation is allowed to optimize the network in reverse. For feature maps that are not cell nuclei, i.e. impurities, loss calculation is rejected to correct the network. This design aims to reduce false positive segmentation.

2. The method according to claim 1, characterized in that, Meanwhile, the network uses a full-scale supervision mode, which allows the network to focus on features at every depth to enhance feature learning.

3. The method according to claim 1, characterized in that, The false positive guidance module consists of three main parts. The first part contains a 3x3 convolutional layer, a normalization layer, and a ReLU activation function layer. The second part is an adaptive max pooling layer. The last part is a dropout layer. The EN5 layer, which contains the most features, is selected for classification prediction. After passing through the false positive guidance module, the EN5 layer outputs two weights: probability, representing the weights of cells that are nuclei and those that are not. This is achieved using the argmax function. The weights are converted into a binary tensor, which includes a first identifier for indicating non-nuclei and a second identifier for indicating nuclei. The segmentation task result is multiplied element-wise with the binary tensor to suppress the segmentation results of non-nuclei regions from participating in the loss function calculation through the first identifier, and to retain the segmentation results of nuclei regions from participating in the loss function calculation through the second identifier.

Images