Cell nucleus segmentation method based on region enhancement

Abstract

This application discloses a cell nucleus segmentation method based on region enhancement, belonging to the field of intelligent pathological diagnosis. The method includes preparing a training data set to obtain a cell nucleus image; building a cell nucleus segmentation network model, loading the cell nucleus image, and obtaining highlighted salient features, wherein the cell nucleus segmentation network model includes a contour branch for detecting the cell nucleus edge and generating contour features, a coarse segmentation branch for completing the semantic segmentation task of the nucleus and combining the contour branch to predict the cell nucleus proposed region, and a fine segmentation branch for processing the image after region enhancement; automatic recognition operations are performed based on the salient features of the cell nucleus segmentation network model; training and testing the cell nucleus segmentation network model to segment the cell nucleus. This application can accurately segment the boundaries of the cell nucleus.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent pathological diagnosis, and in particular to a cell nucleus segmentation method based on regional enhancement. Background Art

[0002] Cervical cancer poses a significant threat to the health of women worldwide, resulting in numerous deaths. To study cervical cancer, cytopathologists must screen for abnormal cells. However, screening cervical cytology specimens for abnormal cells is a tedious and laborious task. Therefore, it is necessary to develop automated screening technologies to assist cytopathologists in diagnosing cervical smears. Cell nucleus segmentation is a key task in automated screening and diagnosis, as the nucleus provides information on chromatin richness and nuclear morphology.

[0003] Image segmentation technology based on deep learning is widely used in various fields of medical images, such as CT, MRI, pathological images, etc. In the field of medical image segmentation, the most famous semantic segmentation network method is U-Net. Inspired by FCN, U-Net preserves contextual information by connecting the intermediate downsampling and upsampling paths, but the kip-connection in U-Net directly fuses low-level features and high-level features, which easily causes a semantic gap; therefore, in Unet++ and Unet 3+, the skip connection is redesigned to combine low-level features with high-level features from feature maps of different scales; in order to enhance the feature learning ability of U-Net, some new modules are proposed to replace the original blocks; Deep Residual U-Net (ResUNet) is proposed, which is built by residual units and has a similar architecture to U-Net; CE-Net model is proposed, which replaces the encoding module of U-Net with pre-trained Resnet and adds a context to the model extractor module to enhance the model feature extraction capability; someone added multi-scale input and deep supervision to the U-Net architecture and proposed an improved U-Net framework called M-Net for joint optic disc and cup segmentation; someone proposed the ResUNet++ model, which is an enhanced version of the standard ResUNet that integrates additional layers into the network; someone proposed a U-Net consisting of two layers of U-shaped structures nested together. 2 -Net and combined with ReSidual U-blocks to capture more contextual information at different scales; however, due to differences in domains, these methods cannot be directly applied to the task of cell nucleus segmentation. Current cell nucleus segmentation methods include those based on handcrafted features. Most existing handcrafted feature-based cell nucleus segmentation methods combine thresholding, morphological operations, region accumulation, and labeled watershed.

[0004] In the process of implementing this application, the inventors discovered that there are at least the following problems in this technology: traditional cell nucleus segmentation methods rely on the difference between the cell nucleus and the background or on the predefined geometric shape of the cell nucleus to generate labels; if they rely on the difference between the cell nucleus and the background, they will produce unreliable results and are not suitable for images with challenging scenes such as crowded, overlapping, and occluded cell nuclei, making it difficult to accurately segment the boundaries of the cell nucleus; if they rely on the predefined geometric shape of the cell nucleus to generate labels, due to the large differences in shape, size, and chromatin pattern of cell nuclei of different cell types or different disease types, the labeling accuracy is low, making it more difficult to accurately segment the cell nucleus boundaries. In this regard, further improvement is needed. Summary of the Invention

[0005] In order to accurately segment the boundaries of cell nuclei, the present application provides a cell nucleus segmentation method based on region enhancement.

[0006] The present application provides a cell nucleus segmentation method based on region enhancement using the following technical solutions:

[0007] A cell nucleus segmentation method based on region enhancement, the method comprising the following steps:

[0008] Prepare a training data set and obtain cell nucleus images;

[0009] Build a cell nucleus segmentation network model, load the cell nucleus image, and obtain the highlighted salient features, wherein the cell nucleus segmentation network model includes a contour branch for detecting the cell nucleus edge and generating contour features, a coarse segmentation branch for completing the semantic segmentation task of the nucleus and combining the contour branch prediction to obtain the cell nucleus proposal region, and a fine segmentation branch for processing the image after region enhancement;

[0010] performing an automatic recognition operation based on the salient features of the cell nucleus segmentation network model;

[0011] The network model for cell nucleus segmentation is trained and tested to segment the cell nucleus.

[0012] By adopting the above technical solution, it is possible to detect the edge of the cell nucleus and generate contour features, complete the semantic segmentation task and combine the prediction to obtain the cell nucleus proposal area, and use the cell nucleus proposal area to enhance the relevant area of ​​the cell nucleus image, thereby accurately segmenting the boundary of the cell nucleus and achieving the purpose of obtaining fine segmentation results.

[0013] Optionally, the preparing of the training data set to obtain the cell nucleus image specifically includes:

[0014] Generate a cell nuclear image: select a cell nuclear image, wherein the cell nuclear image is derived from a cell image including exfoliated cells, wherein the exfoliated cells are collected from people of different ages and conditions, and include all lesion levels of the TBS diagnostic criteria;

[0015] Create a labeling file, mark the cell nucleus, circle the cell nucleus outline, obtain the outline pixel position information used to represent the cell nucleus outline position, and save the outline pixel position information to the labeling file;

[0016] Generate a labeled image: Create a single-channel image with a grayscale value of 0;

[0017] Reading the contour pixel position information in the annotation file;

[0018] The grayscale value of each cell nucleus region was set to 255;

[0019] The cell nucleus image and the labeling map are cropped until the pixels of the cell nucleus image and the labeling map are consistent.

[0020] By adopting the above technical solution, a cell nucleus image and a labeling map can be generated, and the pixels of the cell nucleus image and the labeling map can be ensured to be consistent, thereby facilitating accurate segmentation of the cell nucleus boundary.

[0021] Optionally, the coarse segmentation branch includes an encoder E1 and a decoder D1, the contour branch includes an encoder E2 and a decoder D2, and the fine segmentation branch includes an encoder E3 and a decoder D3. The steps of building a cell nucleus segmentation network model, loading a cell nucleus image, and obtaining highlighted salient features specifically include:

[0022] Build an ATU-Net model including contour branch, coarse segmentation branch and fine segmentation branch, and add ASPP modules to the coarse segmentation branch, contour branch and fine segmentation branch. The ASPP module includes a 1×1 convolution and three 3×3 dilated convolutions. The interval spans of the three 3×3 dilated convolutions are 4, 8 and 12 respectively.

[0023] Load the cell nucleus image into the ATU-Net model, output the prediction result of the cell nucleus through decoder D1, and output the prediction result of the cell nucleus outline through decoder D2;

[0024] Perform element-wise multiplication operations on the output of decoder D1 and the output of decoder D2 with the cell nucleus image to obtain the enhanced cell nucleus image;

[0025] inputting the enhanced cell nucleus image and the original cell nucleus image into the fine segmentation branch;

[0026] The output is spliced ​​through the input features x of encoder E1, encoder E2 and encoder E3 to obtain the features after the splicing operation, and the number of feature channels is reduced to the original number of feature channels through 1×1 convolution.

[0027] The features after the splicing operation are filtered through the attention gate to obtain the highlighted salient features, and are connected with the decoder D3 through a skip connection operation to obtain edge contour features and segmentation semantic features.

[0028] By adopting the above technical solution, rich edge contour features and segmentation semantic features can be obtained, and irrelevant regions in the input feature x can be implicitly suppressed. At the same time, significant features can be highlighted, thereby improving the accuracy of segmenting cell nuclei.

[0029] Optionally, the automatic recognition operation based on the salient features of the cell nucleus segmentation network model specifically includes: replacing the ordinary convolution operation between the encoder and the decoder in the ATU-Net model by the ASPP module to obtain spatial information; obtaining context information by the gating signal g, and selecting the spatial region by analyzing the context information provided by the gating signal g; obtaining an attention coefficient α by attention gate calculation, wherein the attention coefficient α is used to identify the image region and prune the feature response to retain the activation related to the task of segmenting the cell nucleus;

[0030] The input feature x is processed by the attention coefficient α l Scaling is performed, where α∈[0,1], x l Represents the input features of the lth layer of x;

[0031] By adopting ASPP to resample features at different scales, we can classify regions of any scale, thus achieving automatic recognition.

[0032] By adopting the above technical solution, it is possible to accurately and effectively classify regions of any scale, thereby further improving the accuracy of cell nucleus segmentation.

[0033] Optionally, the ASPP further includes global average pooling, and the classification of regions of any scale by resampling features at different scales using ASPP specifically includes:

[0034] Obtain image-level features and process the image-level features through 1×1 convolution and batch normalization;

[0035] Upsampling the image-level features to the required spatial dimensions through feature bilinearity;

[0036] The resulting features of all branches of the ASPP are concatenated and the output features are generated through 1×1 convolution.

[0037] By adopting the above technical solution, it is possible to prevent fitting between features, thereby further improving the accuracy of cell nucleus boundary segmentation.

[0038] Optionally, when generating output features through 1×1 convolution, the output parameters of the attention gate are calculated using the following formula:

[0039]

[0040] in, represents the output features of the attention gate, represents the input feature of position i in layer l, F l represents the number of feature maps corresponding to the lth layer, represents the value of the attention coefficient α at position i in layer l;

[0041] The attention coefficient is calculated by the following formula

[0042]

[0043]

[0044] Among them, σ1 and σ2 represent two identical sigmoid activation functions, Θ att Represents the characteristics of the attention gate, linear transformation Bias term b Ψ ∈R,

[0045] By adopting the above technical solution, the output features of the attention gate can be calculated This facilitates accurate segmentation of cell nucleus boundaries.

[0046] Optionally, the training and testing of the network model for cell nucleus segmentation to segment the cell nucleus specifically includes: calculating L by combining DiceLoss and cross entropy loss with the following formula: fine , L rough and L contour , to train and test the network model for cell nucleus segmentation:

[0047]

[0048] Among them, L fine represents the loss function of the fine segmentation branch, L rough represents the loss function of the coarse segmentation branch, L contour represents the loss function of the contour branch, g i Indicates a labeled image, p iRepresents the predicted probability, i represents the pixel value, N represents the number of categories, and ε represents the regularization term used to filter abnormal data;

[0049] The overall objective function of the cell nucleus segmentation network model is calculated by the following formula, and the cell nucleus is segmented based on the overall objective function:

[0050] L total =λ1L fine +λ2L rough +λ3L contour ,

[0051] Among them, L total represents the overall objective function, λ i A scalar representing the weight assigned by the corresponding loss function.

[0052] By adopting the above technical solution, the overall objective function of the cell nucleus segmentation network model can be calculated, which is conducive to achieving accurate segmentation of the cell nucleus boundary.

[0053] In summary, this application includes at least one of the following beneficial technical effects:

[0054] 1. It can detect the edge of the cell nucleus and generate contour features, complete the semantic segmentation task and combine the prediction to obtain the cell nucleus proposal region. The cell nucleus proposal region is used to enhance the relevant area of ​​the cell nucleus image, thereby accurately segmenting the cell nucleus boundary and achieving the purpose of obtaining fine segmentation results.

[0055] 2. It can obtain rich edge contour features and segmentation semantic features, and implicitly suppress irrelevant areas in the input feature x, while highlighting significant features, thereby improving the accuracy of cell nucleus segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a schematic diagram of a method flow in an embodiment of the present application.

[0057] Figure 2 It is a sub-flowchart of S1 of an embodiment of the present application.

[0058] Figure 3 This is a sub-flowchart of S13 of an embodiment of the present application.

[0059] Figure 4 It is a sub-flowchart of S2 of an embodiment of the present application.

[0060] Figure 5 Schematic diagram of the ATU-Net model according to an embodiment of the present invention.

[0061] Figure 6 It is a schematic diagram of a branch of an embodiment of the present application.

[0062] Figure 7 It is a schematic diagram of a branch of an embodiment of the present application.

[0063] Figure 8 It is a sub-flowchart of S3 of an embodiment of the present application.

[0064] Figure 9 It is a sub-flowchart of S35 of an embodiment of the present application.

[0065] Figure 10 Schematic diagram of an attention gate according to an embodiment of the present application.

[0066] Figure 11 It is a schematic diagram of the element-wise product of an embodiment of the present application.

[0067] Figure 12 It is a sub-flowchart of S4 of an embodiment in this application. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solutions and advantages of this application more clear, the following Figure 1-12 It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0069] The embodiments of the present application disclose a cell nucleus segmentation method based on region enhancement.

[0070] like Figure 1 As shown, the method includes the following steps:

[0071] S1, prepare training data set and obtain cell nucleus map;

[0072] like Figure 1 and Figure 2 As shown, S1 includes three sub-steps: S11, S12, S13 and S14, specifically including:

[0073] S11, generate cell nucleus images;

[0074] Specifically, cell nucleus images are selected from 100 cell images from cervical cell stained slides containing exfoliated cells; the exfoliated cells are collected from people of different ages and different conditions. In this embodiment, different ages refer to those between 25 and 65 years old. In other embodiments, they can also refer to age groups older than 65 years old; different conditions refer to normal, mild, moderate and severe conditions, including all lesion levels of the TBS diagnostic criteria, and TNS diagnosis refers to the classification and reporting details of vaginal cytology.

[0075] S12, creating a labeling file, marking the cell nucleus, circling the cell nucleus outline, obtaining outline pixel position information, and saving the outline pixel position information to the labeling file;

[0076] Specifically, the outline of the cell nucleus is manually circled, and the obtained outline pixel position information can represent the outline position of the cell nucleus.

[0077] S13, generating a labeling map;

[0078] like Figure 2 and Figure 3 As shown, S13 includes three sub-steps, S131, S132 and S133, which specifically include:

[0079] S131, create a single-channel image;

[0080] Specifically, create a single-channel image with a grayscale value of 0. A single-channel image with a grayscale value of 0 is black.

[0081] S132, reading the contour pixel position information;

[0082] Specifically, the contour pixel position information stored in the annotation file is read.

[0083] S133, set the grayscale value of the cell nucleus region to 255;

[0084] Specifically, the grayscale value of each cell nucleus region is set to 255, and the cell nucleus region with a grayscale value of 255 appears white.

[0085] S14, cropped nucleus image and labeling;

[0086] Specifically, the cell nucleus image and the marker image are cropped simultaneously until the sizes of the cell nucleus image and the marker image are both 256×256 pixels, and the pixels of the cell nucleus image and the marker image are consistent.

[0087] S2, building a cell nucleus segmentation network model;

[0088] Specifically, the cell nucleus segmentation network model includes three branches with similar structures, namely the contour branch, the coarse segmentation branch and the fine segmentation branch. The contour branch can detect the edge of the cell nucleus and generate contour features. The contour branch includes encoder E2 and decoder D2; the coarse segmentation branch can complete the semantic segmentation task of the nucleus and combine with the contour branch to predict the cell nucleus proposal area. The coarse segmentation branch includes encoder E1 and decoder D1; at the same time, the fine segmentation branch can process the image after regional enhancement. The fine segmentation branch includes encoder E3 and decoder D3.

[0089] like Figure 1 and Figure 4As shown, S2 includes six sub-steps: S21, S22, S23, S24, S25 and S26, specifically including:

[0090] S21, build the ATU-Net model, add ASPP modules to the coarse segmentation branch, contour branch and fine segmentation branch; specifically, Figure 5 As shown in Figure 2, the ATU-Net model includes a contour branch, a coarse segmentation branch, and a fine segmentation branch. After adding the ASPP module to the coarse segmentation branch, the contour branch, and the fine segmentation branch, the coarse segmentation branch includes an encoder E1, a decoder D1, and ASPP, the contour branch includes an encoder E2, a decoder D2, and ASPP, and the fine segmentation branch includes an encoder E3, a decoder D3, and ASPP. The ATU-Net model uses aggregated features within the branch and between different branches. Figure 5 In , MULTIPLY represents element-wise multiplication, concatenation represents feature concatenation, solid lines represent data flow, and dotted lines represent data flow within the module. Figure 6 and Figure 7 As shown in the figure, rate represents the interval span of the dilated convolution, Contact represents the splicing operation, and ASPP includes a 1×1 convolution and three 3×3 dilated convolutions. Among them, the three 3×3 dilated convolutions all use dilated convolutions, and the interval spans of the three 3×3 dilated convolutions are 4, 8, and 12 respectively. The number of dilated convolutions gradually increases, so that the receptive field of each branch continues to increase.

[0091] S22, loading the cell nucleus image into the ATU-Net model to obtain the prediction results of the cell nucleus and the cell nucleus contour;

[0092] Specifically, the cell nucleus image is loaded and input into the ATU-Net model; then, the decoder D1 of the coarse segmentation branch outputs the prediction result of the cell nucleus, and the decoder D2 of the contour branch outputs the prediction result of the cell nucleus contour.

[0093] S23, performing element-wise multiplication operation on the output of decoder D1 and the output of decoder D2 and the cell nucleus image respectively to obtain an enhanced cell nucleus image;

[0094] Specifically, in the enhanced cell nucleus image, the contour edges of the cell nucleus and the cell nucleus region are more prominent.

[0095] S24, inputting the enhanced cell nucleus image and the original cell nucleus image into the fine segmentation branch;

[0096] Specifically, the enhanced cell nucleus image and the original cell nucleus image are input into the fine segmentation branch to facilitate subsequent segmentation operations.

[0097] S25, performing splicing processing on the output to obtain the features after the splicing operation and reducing the number of feature channels;

[0098] Specifically, the output is spliced ​​through the input features x of the encoder E1 of the coarse segmentation branch, the encoder E2 of the contour branch, and the encoder E3 of the fine segmentation branch, and then the number of feature channels is reduced to 1 / 1 of the original number of feature channels through 1×1 convolution.

[0099] S26, filters the features after the splicing operation and connects them to the decoder D3 through a skip connection operation;

[0100] Specifically, the attention gate is also called the attention gate. The features after the splicing operation are first filtered through the attention gate. The attention gate can implicitly suppress irrelevant areas in the input feature x, and at the same time, highlight significant features that are useful for a specific task. In this embodiment, the specific task represents the task of segmenting cell nuclei; then it is connected to the decoder D3 through a skip connection operation, so that the fine segmentation branch obtains edge contour features and segmentation semantic features.

[0101] S3, automatic recognition operation based on the salient features of the cell nucleus segmentation network model;

[0102] like Figure 1 and Figure 8 As shown, S3 includes five sub-steps: S31, S32, S33, S34 and S35, specifically including:

[0103] S31, replaces the ordinary convolution operation between the encoder and decoder in the ATU-Net model with the ASPP module to obtain spatial information;

[0104] Specifically, the ordinary convolution operation between the encoder and decoder in the ATU-Net model is replaced by the ASPP module to obtain richer spatial information.

[0105] S32, obtaining context information through the gate signal g and selecting a spatial region;

[0106] Specifically, context information is obtained through the gating signal g, and the spatial region is selected by analyzing the context information provided by the gating signal g.

[0107] S33, calculating and obtaining the attention coefficient α;

[0108] Specifically, the attention coefficient α is obtained by calculating the attention gate. The attention coefficient α can identify the image area and prune the feature response, thereby retaining the activation related to the task of segmenting the cell nucleus. The activation represents the operation of the attention gate to enhance the filtering of the feature.

[0109] S34, the input feature x is analyzed by the attention coefficient α l Zoom in or out;

[0110] Specifically, the input feature x is processed by the attention coefficient α l Scaling is performed, where α∈[0,1], when α is the α at position i, a i ∈[0,1]; x l Represents the input features of the lth layer of x.

[0111] S35, classifies regions of arbitrary scales by resampling features at different scales using ASPP;

[0112] Specifically, if Figure 6 and Figure 7 As shown in Figure 1, ASPP also includes a global average pooling, that is, ASPP contains 5 cascade branches; by using ASPP to resample features at different scales to accurately and effectively classify regions of any scale, automatic recognition operations are achieved.

[0113] like Figure 8 、 Figure 9 As shown, S35 includes three sub-steps: S351, S352, and S353, specifically including:

[0114] S351, obtaining image-level features and processing image-level features;

[0115] Specifically, since ASPP also includes a global average pooling, global average pooling can process the obtained image-level features through 1×1 convolution and batch normalization.

[0116] S352, upsampling the image-level features to the required spatial dimensions;

[0117] Specifically, if Figure 10 As shown, the image-level features are upsampled to the required spatial dimensions by feature bilinearity. Figure 10 In the figure, the triangle arrow with the A label indicates that the feature size becomes the original size after the maximum pooling. The triangle arrow with the A label indicates that the feature size becomes the original size after the maximum pooling. times; the triangle arrow with the B label indicates that the feature size becomes twice as large as the original after upsampling; the triangle arrow with the C label indicates the gating signal; the triangle arrow with the D label indicates the skip connection.

[0118] S353, connecting the result features of all ASPP branches to generate output features;

[0119] Specifically, if Figure 7 and Figure 11As shown in , the result features of all ASPP branches are connected, and the output features are generated by 1×1 convolution. Figure 11 , the circle with “+” represents element addition, the circle with cross represents element multiplication, ReL U represents the Rel U activation function in deep learning, Sigmoid represents the Sigmoid function, and Resampler represents the resampler.

[0120] In step S353, the output parameters of the attention gate are calculated by the following formula:

[0121]

[0122] in, Represents the output features of the attention gate; represents the input feature of position i in layer l, F l Indicates the number of feature maps corresponding to the lth layer; represents the value of the attention coefficient α at position i in layer l;

[0123] It is calculated by the following formula

[0124]

[0125] in, σ1 and σ2 represent two identical sigmoid activation functions; Θ att Represents the characteristics of the attention gate; linear transformation Bias term

[0126] S4, train and test the network model for cell nucleus segmentation to segment the cell nucleus;

[0127] like Figure 1 、 Figure 12 As shown, S4 includes two sub-steps, S41 and S42, which specifically include:

[0128] S41, calculate and obtain L fine , L rough and L contour ;

[0129] Specifically, L fine represents the loss function of the fine segmentation branch, L rough represents the loss function of the coarse segmentation branch, L contour Represents the loss function of the contour branch; in order to improve the network's ability to identify cell nuclei, L fine , L rough and L contourUse DiceLoss and cross entropy loss, combined with the following formula to calculate L fine , L rough and L contour , train and test the network model for cell nucleus segmentation to achieve accurate cell nucleus segmentation:

[0130]

[0131] Among them, g i Indicates a marked picture; p i represents the predicted probability; i represents the pixel value; N represents the number of categories; ε represents the regularization term used to filter abnormal data. In this embodiment, ε is set to 1.0e -5 ;

[0132] S42, calculate the overall objective function and segment the cell nucleus;

[0133] Specifically, the overall objective function of the cell nucleus segmentation network model is calculated by the following formula, and the cell nucleus is segmented by combining the overall objective function:

[0134] L total =λ1L fine +λ2L rough +λ3L contour ,

[0135] The overall objective function of the cell nucleus segmentation network model comes from the weighted sum of the losses of the coarse segmentation branch, the contour branch, and the fine segmentation branch, where L total represents the overall objective function, λ i A scalar representing the weight assigned by the corresponding loss function; in this embodiment, λ1 is set to 2, λ2 is set to 1, and λ3 is set to 1.

[0136] The implementation principle of a cell nucleus segmentation method based on regional enhancement in an embodiment of the present application is: prepare a training data set to obtain a cell nucleus image; build a cell nucleus segmentation network model, load the cell nucleus image, and obtain highlighted salient features; perform automatic recognition operations based on the salient features of the cell nucleus segmentation network model; train and test the cell nucleus segmentation network model to segment the cell nucleus.

[0137] The inventors conducted a series of tests on the segmentation method in the application. First, a comparative experiment was conducted on three public datasets, namely MoNuSeg, CoNSep and CPM-17. The selected comparison models are relatively prominent models in the field of cell nucleus segmentation in recent years, including: CNN3 (kumar), U-Net, Mask R-CNN, DIST, DCAN, HoVer-Net, CIA-Net, Bending Loss, PDAM, PFFNet, and Micro-net.

[0138] From Table 1, we can see the advantages of the ATU-Net proposed in this application, and further verify that the ATU-Net proposed in this application has the same advantages in other public data.

[0139] Table 1

[0140] Method test data:

[0141]

[0142] The above are all preferred embodiments of the present application, and are not intended to limit the scope of protection of the present application. Therefore, any equivalent changes made based on the structure, shape, and principle of the present application should be included in the scope of protection of the present application.

Claims

1. A cell nucleus segmentation method based on region enhancement, characterized in that: The method comprises the following steps: Prepare a training data set and obtain cell nucleus images; Build a cell nucleus segmentation network model, load the cell nucleus image, and obtain the highlighted salient features, wherein the cell nucleus segmentation network model includes a contour branch for detecting the cell nucleus edge and generating contour features, a coarse segmentation branch for completing the semantic segmentation task of the nucleus and combining the contour branch prediction to obtain the cell nucleus proposal region, and a fine segmentation branch for processing the image after region enhancement; The coarse segmentation branch includes an encoder E1 and a decoder D1, the contour branch includes an encoder E2 and a decoder D2, and the fine segmentation branch includes an encoder E3 and a decoder D3. The cell nucleus segmentation network model is constructed, the cell nucleus image is loaded, and the highlighted salient features are obtained, specifically including: Build an ATU-Net model including contour branch, coarse segmentation branch and fine segmentation branch, and add ASPP modules to the coarse segmentation branch, contour branch and fine segmentation branch. The ASPP module includes a 1×1 convolution and three 3×3 dilated convolutions. The interval spans of the three 3×3 dilated convolutions are 4, 8 and 12 respectively. Load the cell nucleus image into the ATU-Net model, output the prediction result of the cell nucleus through decoder D1, and output the prediction result of the cell nucleus outline through decoder D2; Perform element-wise multiplication of the output of decoder D1 and the output of decoder D2 with the cell nucleus image to obtain the enhanced cell nucleus image; inputting the enhanced cell nucleus image and the original cell nucleus image into the fine segmentation branch; The output is spliced ​​through the input features x of encoder E1, encoder E2 and encoder E3 to obtain the features after the splicing operation, and the number of feature channels is reduced to the original number of feature channels through 1×1 convolution. The features after the splicing operation are filtered through the attention gate to obtain the highlighted salient features, and are connected to the decoder D3 through the skip connection operation to obtain the edge contour features and segmentation semantic features; performing an automatic recognition operation based on the salient features of the cell nucleus segmentation network model; Training and testing the network model for cell nucleus segmentation to segment the cell nucleus includes: calculating L by combining DiceLoss and cross entropy loss with the following formula fine , L rough and L contour , to train and test the network model for cell nucleus segmentation: Among them, L fine represents the loss function of the fine segmentation branch, L rough represents the loss function of the coarse segmentation branch, L contour represents the loss function of the contour branch, g i Indicates a labeled image, p i Represents the predicted probability, i represents the pixel value, N represents the number of categories, and ε represents the regularization term used to filter abnormal data: The overall objective function of the cell nucleus segmentation network model is calculated by the following formula, and the cell nucleus is segmented based on the overall objective function: L total =λ1L fine +λ2L rough +λ3L contour , Among them, L total Represents the overall objective function, and L represents the scalar weighted by the corresponding loss function.

2. The cell nucleus segmentation method based on region enhancement according to claim 1, characterized in that: The preparing of the training data set to obtain the cell nucleus image specifically includes: Generate a cell nuclear image: select a cell nuclear image, wherein the cell nuclear image is derived from a cell image including exfoliated cells, wherein the exfoliated cells are collected from people of different ages and conditions, and include all lesion levels of the TBS diagnostic criteria; Create a labeling file, mark the cell nucleus, circle the cell nucleus outline, obtain the outline pixel position information used to represent the cell nucleus outline position, and save the outline pixel position information to the labeling file; Generate a labeled image: Create a single-channel image with a grayscale value of 0; Reading the contour pixel position information in the annotation file; The grayscale value of each cell nucleus region was set to 255; The cell nucleus image and the labeling map are cropped until the pixels of the cell nucleus image and the labeling map are consistent.

3. The cell nucleus segmentation method based on region enhancement according to claim 1, characterized in that: The automatic recognition operation based on the significant features of the cell nucleus segmentation network model specifically includes: The ASPP module replaces the ordinary convolution operation between the encoder and decoder in the ATU-Net model to obtain spatial information; the gating signal g is used to obtain contextual information, and the spatial region is selected by analyzing the contextual information provided by the gating signal g; the attention coefficient α is obtained by the attention gate calculation, and the attention coefficient α is used to identify the image region and prune the feature response to retain the activation related to the task of segmenting the cell nucleus; The input feature x is processed by the attention coefficient α l Scaling is performed, where α∈[0,1], x l Represents the input features of the lth layer of x; by using ASPP to resample features at different scales to classify regions of any scale, automatic recognition operations are achieved.

4. The cell nucleus segmentation method based on region enhancement according to claim 3, characterized in that: The ASPP also includes global average pooling, which classifies regions of any scale by resampling features at different scales using ASPP, specifically including: Obtain image-level features and process the image-level features through 1×1 convolution and batch normalization; Upsampling the image-level features to the required spatial dimensions through feature bilinearity; The resulting features of all branches of the ASPP are concatenated and the output features are generated through 1×1 convolution.

5. The cell nucleus segmentation method based on region enhancement according to claim 4, characterized in that: In generating output features through 1×1 convolution, the output parameters of the attention gate are calculated by the following formula in, represents the output features of the attention gate, represents the input feature of position i in layer 1, F l represents the number of feature maps corresponding to the lth layer, represents the value of the attention coefficient α at position i in layer l; The attention coefficient is calculated by the following formula in, σ1 and σ2 represent two identical sigmid activation functions, Θ att Represents the characteristics of the attention gate, linear transformation Bias term b Ψ ∈R,

Images