Double-branch cell nucleus instance segmentation method based on shape prior

Abstract

The invention discloses a double-branch cell nucleus instance segmentation method based on shape prior. According to the method, a U-Net decoder is mainly divided into two branches, namely a mask branch and an edge branch, one branch is used for predicting a cell nucleus mask, the other branch is used for predicting a cell nucleus boundary, a dense cavity convolution module is provided for assisting cell nucleus boundary generation, and a multi-scale convolution module is provided for assisting cell nucleus mask generation. And then features obtained by the two branches are fused together, and a corresponding verification example is obtained by using post-processing operation. The method comprises the following steps of: firstly, designing an improved loss function by utilizing priori knowledge, respectively selecting proper loss functions for a mask branch and an edge branch, and finally, guiding model training by taking the weighted sum of the two task loss functions as network loss, and effectively solving the problems of unsmooth and special-shaped cell nucleus contour and the like by embedding the priori knowledge into the loss functions.

Description

Technical Field

[0001] The invention discloses a double-branched cell nucleus instance segmentation method based on shape prior, and relates to the technical field of image segmentation. Background Art

[0002] Cancer is a serious disease that endangers human life and health. The key to reducing cancer mortality is timely screening and diagnosis, early detection and early treatment. Traditional cancer screening mainly relies on cytological examinations such as Pap smear examinations and liquid-based thin-layer cell tests. However, these methods are time-consuming and labor-intensive, and are easily affected by personal subjective factors. Therefore, in recent years, many studies have focused on using machine-based screening methods, namely computer-aided diagnosis (CAD) systems, to help experts reduce errors and workload, speed up the screening process, and reduce screening costs. Cell image segmentation plays a key role in computer-aided diagnosis systems. It is the first step in the automated analysis of cell images, and it provides the necessary basic preparation for the next step of feature extraction and classification.

[0003] In cell image segmentation, cell nucleus segmentation is the top priority of research. Cell nuclei are crucial for assessing the presence and severity of diseases. The distribution, size, morphology, and aggregation density of cell nuclei can reflect important information of histopathological images. For example, the infiltration of lymphocyte nuclei in breast cancer is related to the survival and prognosis of patients, and nuclear polymorphism has diagnostic value for cancer grading. Usually, one cell nucleus corresponds to one cell. As an indicator of cytoplasm detection, the accuracy of its detection also directly affects the subsequent identification and segmentation of cytoplasm. Moreover, the analysis, diagnosis, classification, and grading of cancer depend to a large extent on the quality of cell nucleus segmentation. Accurate cell nucleus segmentation can help researchers quickly analyze a large amount of cell image data, identify information such as the number, morphology, and location of cell nuclei, and help doctors and researchers better understand cell structure and function, thereby promoting early diagnosis and treatment research of diseases. Therefore, it is necessary to study the segmentation of cell nuclei in cell images. In addition, in cell images, cells often adhere to each other. How to distinguish different cell instances that adhere to each other is a very challenging topic. Due to the imaging conditions and distribution characteristics of cell nuclei under a microscope, there are still many difficulties and challenges in the field of instance segmentation of cell nuclei. Cells in some nucleus images tend to cluster with each other, leading to overlapping issues. As a result, under low-contrast staining, the translucent cytoplasm of cells occludes each other, leading to blurred cell boundary predictions. In addition, complex factors such as different cell types having various characteristics, including shape, size, color, and texture, make it more difficult to accurately segment cell nuclei.

[0004] At present, instance segmentation methods are divided into two-stage and one-stage methods. The two-stage method includes a detection stage and a segmentation stage. The detection stage is used to locate the nuclear instance, and the segmentation stage is used to predict the foreground mask of each instance. The representative method is Mask R-CNN and its variant models, which use bounding boxes to detect nuclear instances. However, the shape of the nuclear tends to be elliptical, and there is typically severe occlusion between instances, which means that each bounding box can contain pixels representing two or more instances, indicating that bounding boxes may ultimately be suboptimal for nuclear segmentation. In addition, the complexity of the two-stage network is high. At present, some of the latest studies use a one-stage method, that is, a single network prediction is used, and post-processing operations are used to obtain nuclear instances. Distance-based and boundary-based methods are often used in one-stage methods. Distance-based methods have two shortcomings: first, the centroid itself lacks contextual information, and second, for nuclei of different sizes, the centroid is at different distances from the boundary pixels, resulting in a decrease in distance prediction accuracy. In contrast, boundary-based methods have better overall performance, and the prior knowledge of the nucleus is not considered in the nucleus segmentation network, resulting in uneven contours and heteromorphism of some nuclei. To address the above problems, we proposed a dual-branch nucleus instance segmentation method based on shape priors according to the shape features of the nucleus. This method mainly divides the U-Net decoder into two branches, namely the mask and edge branches, one for predicting the nucleus mask and the other for predicting the nucleus boundary. A dense hole convolution module is proposed to assist in the generation of the nucleus boundary and a multi-scale convolution module is proposed to assist in the generation of the nucleus mask. The features obtained from the two branches are then fused together, and the corresponding nucleus instances are obtained by post-processing operations. Then, an improved loss function is designed using prior knowledge, and appropriate loss functions are selected for the mask branch and the edge branch respectively. Finally, the weighted sum of the two task loss functions is used as the network loss to guide model training. By embedding prior knowledge into the loss function, the problems of uneven nucleus contours and irregular shapes are effectively solved. Summary of the invention

[0005] In view of the above problems or deficiencies, in order to reduce the problems of complex background, different nucleus sizes, blurred edges, etc. in the cell nucleus instance segmentation, the present invention proposes a double-branched cell nucleus instance segmentation method based on shape prior. The present invention adopts the following technical solutions to solve the above technical problems:

[0006] The steps include:

[0007] Step 1: Preprocess the cell images and annotated images, and divide the preprocessed cell images and their corresponding annotated images into a training set, a validation set, and a test set;

[0008] Step 2: Build the cell nucleus segmentation network, including the encoder part, DAC block, MSC block, and the double-branch network construction and post-processing operations of obtaining instances;

[0009] Step 3: Use the training set obtained in step 1 to input the segmentation network in batches for training;

[0010] Step 4: For the mask branch, a hybrid loss function is formed by combining the binary cross entropy BCE loss function and the DICE loss function to calculate the loss between the predicted result and the true label. For the edge branch, a loss function based on prior knowledge is used to calculate the cell nucleus boundary loss. Both branches perform back propagation to update the weights.

[0011] Step 5: At the end of each training, evaluate the model obtained from step 3 using the validation set and save the model with the best evaluation result;

[0012] Step 6: Use the test set obtained in step 1 and input it into the model with the best evaluation result in step 5 for segmentation to obtain the predicted image; use the Dice coefficient, panoramic quality (PQ) and aggregate Jaccard index as evaluation criteria to evaluate the segmentation effect.

[0013] As a preferred embodiment of the present invention, the specific content of step 2 is:

[0014] 1.1 Encoder: In the original U-Net network, the encoding part is a downsampling module composed of 3×3 convolution (RELU) and a 2×2 maxpooling layer. In the proposed method, we replace it with a pre-trained ResNet-34 in the feature encoder module to increase the network depth and solve the problem of gradient vanishing;

[0015] 1.2DAC block implementation: The DAC block has four cascaded branches. In each hole branch, a combination of convolutions with different dilation rates is used, and a 1×1 convolution is applied for linear activation. Afterwards, a mixed attention (CBAM) layer is introduced to model the importance of each branch. Finally, the original features are added to the features extracted by the CBAM layer, similar to the shortcut mechanism in ResNet. Generally, convolutions with large dilation rates can extract and generate more abstract features of large objects, while convolutions with small dilation rates are more suitable for small objects. By combining convolutions with different dilation rates and the use of CBAM layers, the DAC block is able to better extract cell nucleus boundary features;

[0016] 1.3MSC block implementation: Four different sizes of convolution kernels are used to encode global context information: 2×2, 3×3, 5×5, and 6×6. In order to reduce the dimensionality and computational cost, we use 1×1 convolution after each level of pooling. Finally, the low-dimensional feature map is upsampled by bilinear interpolation to obtain features of the same size as the original feature map and concatenated with the original features. The MSC block uses multi-scale convolution operations to obtain further context information of the cell kernel;

[0017] 1.4 Dual-branch network: In the original U-Net network, the decoding part is composed of up-conv 2×2 and 3×3 convolution (RELU) layers. This paper adds a dedicated boundary prediction branch in the decoder part to enable the network to better perceive the boundary information of the cell nucleus. This branch is parallel to the semantic segmentation branch and uses the feature map of the decoder stage of U-Net to predict the outline of the cell nucleus. The semantic segmentation branch is still responsible for generating pixel-level cell nucleus masks, while the boundary prediction branch focuses on extracting boundary details. At the same time, the DAC block assists in the generation of cell nucleus boundaries and the MSC block assists in the generation of cell nucleus masks.

[0018] 1.5 Post-processing: The method proposed in this paper is an instance segmentation method. In order to obtain the cell nucleus instance, we proposed a new post-processing method. The specific steps are as follows: First, subtract the contour from the foreground and mark each disconnected region as an instance. Then, diffuse the instance result outward until the foreground information is fully filled to generate the final segmentation mask. This method is simple and effective, which can effectively use the contour information to separate overlapping cells and restore the cell instance using the high-precision foreground information.

[0019] As a preferred embodiment of the present invention, the specific content of step 4 is:

[0020] 2.1 Calculate the loss functions of the mask branch and edge branch outputs respectively. The calculation formulas are shown in formulas (1) and (2). Both formulas contain the common bce loss function and dice loss function. α and β in the two formulas are weight coefficients. Since the area occupied by the cell nucleus and the cell boundary in the image is different, the values ​​of α and β will be different; P is the penalty term, which is used to estimate the weight. The weight represents the influence of the error area on the shape of the cell nucleus:

[0021] L m =αL bce +(1-α)L dice (1)

[0022] L e =βL bce +(1-β)L dice +P (2)

[0023] 2.2 To calculate the penalty term P in the edge branch loss function, we need to first calculate the weight of the pixels around the cell nucleus contour in the edge prediction image that affect its shape feature value F. First, we select rectangular areas of equal size with some points on the real edge as the center, and calculate the weight of the predicted area in each shape box that affects F according to formula (3), where A represents the area of ​​the real area in the rectangular box, B represents the area of ​​the predicted area in the rectangular box, and I i The closer the value of is to 1, the closer the predicted area is to the true label. Then, the shape feature F (ellipticity) of the cell nucleus is calculated. The closer F is to 1, the better the effect. The penalty term P is calculated by formula (4). The penalty value of the predicted area in the i-th box is set to P i , the penalty values ​​of other pixels are set to 0:

[0024]

[0025] P i =(1-I i )(1-F) (4)

[0026] 2.3 Calculate the overall loss of the network, which consists of two parts. The calculation formula is shown in formula (5):

[0027] L=L m +L e (5) BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a dual-branch instance segmentation model diagram based on shape prior proposed by the present invention.

[0029] Figure 2 It is a schematic diagram of the post-processing part of the present invention.

[0030] Figure 3 Schematic diagram of the residual module ResNet-34 used in the encoding part of the present invention.

[0031] Figure 4 Schematic diagram of the DAC module of the example segmentation model of the present invention.

[0032] Figure 5 It is a schematic diagram of the MSC module of the example segmentation model of the present invention. DETAILED DESCRIPTION

[0033] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0034] like Figure 1As shown, the present invention integrates shape prior knowledge and a dual-branch network, including the following steps:

[0035] 1. Preprocess the cell images and annotated images: Taking the CPM-17 dataset as an example, the mat file provided in the dataset is first converted into the corresponding contour and segmentation images to form the corresponding edge and mask folders. After that, data enhancement is performed, mainly mirroring and flipping, and the images in the original train folder are added. In addition to enhancement, the edge contours are expanded; finally, the preprocessed cell images and their corresponding annotated images are divided into training sets, validation sets, and test sets according to the ratios of 70%, 10%, and 20%;

[0036] 2. Build a segmentation network structure, as follows: Figure 1 The figure shows the structure of the segmentation network proposed in this paper. The segmentation network consists of an encoding part and a decoding part. The encoding part consists of Figure 3 The residual module ResNet-34 increases the network depth while preventing gradient disappearance, which is conducive to better extraction of image features. The decoder is divided into two branches, one for predicting the cell nucleus and the other for predicting the cell nucleus boundary. The dense hole convolution module assists in the generation of the cell nucleus boundary and the multi-scale convolution module assists in the generation of the cell nucleus mask. Figure 1 The structure diagram of DAC block and MSC block is as follows Figure 4 and Figure 5 As shown in the figure, the dilated convolutions in the DAC block are stacked in a cascade mode. In this case, the DAC has four cascaded branches. In each dilated branch, we apply a 1×1 convolution for linear activation. After that, a mixed attention (CBAM) layer is introduced to model the importance of each branch. Finally, the original features are added to the features extracted by the CBAM layer, similar to the shortcut mechanism in ResNet. By combining dilated convolutions with different dilation rates and the use of CBAM layers, the DAC block is able to better extract the boundary features of the cell nucleus. The MSC block encodes global context information with four different sizes of convolution kernels: 2×2, 3×3, 5×5, and 6×6. To reduce the dimensionality and computational cost, we use 1×1 convolution after each level of pooling. Finally, the low-dimensional feature map is upsampled by bilinear interpolation to obtain features of the same size as the original feature map and concatenated with the original features. The MSC block is able to better extract the features of the cell nucleus. Taking the obtained feature map of 14×14×512 as an example, after the MSC module, the size of the feature map becomes 14×14×516. In order to obtain instantiation results, the fused nucleus mask and boundary edge prediction obtained by the two branches are post-processed as follows Figure 2As shown in the figure. First, the contour is subtracted from the foreground and each disconnected region is labeled as an instance. Then, the instance result is diffused outward until the foreground information is fully filled to generate the final segmentation mask. This method is simple and effective because it can effectively use the contour information to separate overlapping cells and use the high-precision foreground information to restore the cell instance;

[0037] 3. Use the training set obtained in 1 to input the segmentation network in batches for training; input the prepared training set images into the segmentation network in batches, and each layer of features output by the encoding part will be skipped and fused with the features of the same resolution in the decoding part. The decoding part extracts the cell nucleus and cell boundary features respectively, and then performs fusion upsampling until it is upsampled to the original image size;

[0038] 4. Use different loss functions to calculate the loss between the mask prediction result and the true label and the cell nucleus boundary loss, and perform back propagation to update the weights respectively. The bilateral U-Net structure has two task-specific decoders, so the overall loss of the network is composed of two parts. The calculation formula is shown in formula (1):

[0039] L=L m +L e (1)

[0040] The loss functions for calculating the output of the mask branch and the edge branch both contain the bce loss function and the dice loss function. The loss functions for calculating the output of the edge branch and the loss functions for calculating the output of the mask branch are shown in equations (2) and (3). α and β in the two formulas are weight coefficients. Since the areas occupied by the cell nucleus and the cell boundary in the image are different in size, the values ​​of α and β will be different. The role of the weight coefficients α and β is to adjust the proportion of the dice loss function and the binary cross entropy bce loss function in the two branches respectively when training the model, so as to eliminate the impact of the imbalance in the proportion of positive and negative data in the training data. For the edge branch, we use the loss function L based on prior knowledge e , a shape loss penalty term is added to each cell nucleus to increase the loss of the error region. The penalty term P is added to the loss function, and the penalty term is used to estimate the weight, which represents the degree of influence of the error region on the shape of the cell nucleus. By minimizing the weight, the prediction result is close to the true label.

[0041] L m =αL bce +(1-α)L dice (2)

[0042] L e =βL bce +(1-β)L dice +P (3)

[0043] To calculate the shape loss penalty term P, we need to first calculate the weight of the pixels around the cell nucleus contour in the edge prediction image that affect its shape feature value F. First, select rectangular areas of equal size with some points on the real edge as the center; calculate the weight of the predicted area in each shape box that affects F according to formula (4);

[0044]

[0045] Where A represents the area of ​​the real area in the rectangular box, and B represents the area of ​​the predicted area in the rectangular box; i The closer the value of is to 1, the closer the predicted area is to the true label. Then, the shape feature F (ellipticity) of the cell nucleus is calculated. The closer F is to 1, the better the effect. The penalty term P is calculated as follows:

[0046] P i =(1-I i )(1-F) (5)

[0047] The penalty value of the predicted area in the i-th box is set to P i , the penalty values ​​of other pixels are set to 0. In this way, the prior knowledge of the shape of the cell nucleus is added to the loss function, L e The function can effectively control the shape of the contour and make the contour smoother.

[0048] 5. At the end of each training, evaluate the model obtained from 3 with the validation set and save the model with the best evaluation result: At the end of each training, evaluate the score of the model with the validation set and compare it with the score of the previous evaluation, so as to save the model with the highest score.

[0049] 6. Use the test set obtained in 1 to input the model with the best evaluation result in step 5 for segmentation to obtain the predicted image. Use the Dice coefficient, panoramic quality (PQ) and aggregate Jaccard index as performance evaluation indicators to evaluate the segmentation effect.

[0050] The Dice similarity coefficient is a set similarity measurement function used to measure the degree of overlap between two binary samples. The measurement value is between [0,1]. A Dice similarity coefficient of 1 indicates complete overlap. The formula is as follows:

[0051]

[0052] Among them, TP (True Positive) represents the correctly segmented area, FP (False Positive) represents the area that is incorrectly segmented as a positive example, and FN (False Negative) represents the area that is incorrectly segmented as a negative example.

[0053] Panoramic Quality (PQ): is a metric used to evaluate the performance of semantic segmentation and instance segmentation tasks. It comprehensively considers the accuracy of semantic segmentation and the connectivity of instance segmentation to provide a more comprehensive evaluation. The calculation of PQ involves three important components: the accuracy of semantic segmentation (Segmentation Quality), the accuracy of instance segmentation (Detection Quality), and the connectivity of instance segmentation (Connectivity). Among them, Segmentation Quality measures the degree of match between the predicted category labels in semantic segmentation and the true labels, Detection Quality measures the degree of match between the predicted instances in instance segmentation and the true instances, and Connectivity measures the connectivity of the predicted instances in instance segmentation. The formula is as follows:

[0054]

[0055] The first part corresponds to the precision-recall component, which is used to measure the accuracy of the segmentation results. The definitions of TP, FP, and FN are the same as the Dice coefficient. The second part represents the IoU (intersection over union) component, which is used to evaluate the accuracy of the segmentation results. ∑IoU(x,y)(x,y)∈TP represents the average IoU value of correctly matched objects, and TP represents the number of correctly matched objects.

[0056] Aggregate Jaccard Index (AJI): is an indicator used to evaluate cell segmentation results, which comprehensively considers the accuracy and consistency of segmentation. The aggregate Jaccard index obtains the final evaluation indicator by multiplying the Jaccard index of each cell by its corresponding area and averaging all cells. This means that large cell areas contribute more to the aggregate Jaccard index. The value range of the aggregate Jaccard index is between 0 and 1. The higher the value, the higher the overlap between the segmentation result and the true cell area, that is, the more accurate and consistent the segmentation result. The specific formula is as follows:

[0057]

[0058] In the AJI formula, N represents the number of instances in the sample. i represents the true annotation mask of the i-th instance, represents the predicted mask of the i-th instance. Represents the total size of the union of the matched predicted mask and the true annotation mask. The second term ∑ F∈U |P F | represents the total number of pixels in the predicted mask that are not matched. The numerator of the formula Represents the total number of correctly matched pixels.

[0059] The present invention is a new dual-branch cell nucleus instance segmentation method based on shape prior, which enhances the original nucleus mask and nucleus boundary generation through DAC and MSC modules, and designs a specific loss function based on prior knowledge to enhance the performance and result quality of the model. The method predicts through a dual-branch network and obtains nucleus instances through post-processing methods. Experiments show that the network has good robustness.

[0060] The above embodiments are only for illustrating the technical idea of ​​the present invention, and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the invention.

Claims

1. A double-branched cell nucleus instance segmentation method based on shape prior, characterized in that: The steps include: Step 1: Preprocess the cell images and annotated images, and divide the preprocessed cell images and their corresponding annotated images into a training set, a validation set, and a test set; Step 2: Build the cell nucleus segmentation network, including the encoder part, DAC block, MSC block, and the double-branch network construction and post-processing operations of obtaining instances; Step 3: Use the training set obtained in step 1 to input the segmentation network in batches for training; Step 4: For the mask branch, a hybrid loss function is formed by combining the binary cross entropy BCE loss function and the DICE loss function to calculate the loss between the predicted result and the true label. For the edge branch, a loss function based on prior knowledge is used to calculate the cell nucleus boundary loss. Both branches perform back propagation to update the weights. Step 5: At the end of each training, evaluate the model obtained from step 3 using the validation set and save the model with the best evaluation result; Step 6: Use the test set obtained in step 1 and input it into the model with the best evaluation result in step 5 for segmentation to obtain the predicted image; use the Dice coefficient, panoramic quality (PQ) and aggregate Jaccard index as evaluation criteria to evaluate the segmentation effect.

2. The method for segmenting double-branched cell nuclei based on shape prior according to claim 1, characterized in that: The specific content of step 2 is: 1.1 Encoder: In the original U-Net network, the encoding part is a downsampling module composed of 3×3 convolution (RELU) and a 2×2 maxpooling layer. In the proposed method, we replace it with a pre-trained ResNet-34 in the feature encoder module to increase the network depth and solve the problem of gradient vanishing; 1.2 DAC block implementation: The DAC block has four cascaded branches. In each hole branch, a combination of convolutions with different dilation rates is used, and a 1×1 convolution is applied for linear activation. Afterwards, a mixed attention (CBAM) layer is introduced to model the importance of each branch. Finally, the original features are added to the features extracted by the CBAM layer, similar to the shortcut mechanism in ResNet. Generally, convolutions with large dilation rates can extract and generate more abstract features of large objects, while convolutions with small dilation rates are more suitable for small objects. By combining convolutions with different dilation rates and the use of CBAM layers, the DAC block is able to better extract cell nucleus boundary features; 1.3 MSC block implementation: Four different sizes of convolution kernels are used to encode global context information: 2×2, 3×3, 5×5, and 6×6. To reduce the dimensionality and computational cost, we use 1×1 convolution after each level of pooling. Finally, the low-dimensional feature map is upsampled by bilinear interpolation to obtain features of the same size as the original feature map and concatenated with the original features. The MSC block uses multi-scale convolution operations to obtain further context information of the cell kernel; 1.4 Dual-branch network: In the original U-Net network, the decoding part is composed of up-conv 2×2 and 3×3 convolution (RELU) layers. This paper adds a dedicated boundary prediction branch in the decoder part to enable the network to better perceive the boundary information of the cell nucleus. This branch is parallel to the semantic segmentation branch and uses the feature map of the decoder stage of U-Net to predict the outline of the cell nucleus. The semantic segmentation branch is still responsible for generating pixel-level cell nucleus masks, while the boundary prediction branch focuses on extracting boundary details. At the same time, the DAC block assists in the generation of cell nucleus boundaries and the MSC block assists in the generation of cell nucleus masks. 1.5 Post-processing: The method proposed in this paper is an instance segmentation method. In order to obtain the cell nucleus instance, we proposed a new post-processing method. The specific steps are as follows: First, subtract the contour from the foreground and mark each disconnected region as an instance. Then, diffuse the instance result outward until the foreground information is fully filled to generate the final segmentation mask. This method is simple and effective, which can effectively use the contour information to separate overlapping cells and restore the cell instance using the high-precision foreground information.

3. The method for segmenting double-branched cell nucleus instances based on shape prior according to claim 1, characterized in that: The specific process of step 4 is as follows: 2.1 Calculate the loss functions of the mask branch and edge branch outputs respectively. The calculation formulas are shown in formulas (1) and (2). Both formulas contain the common bce loss function and dice loss function. α and β in the two formulas are weight coefficients. Since the area occupied by the cell nucleus and the cell boundary in the image is different, the values ​​of α and β will be different; P is the penalty term, which is used to estimate the weight. The weight represents the influence of the error area on the shape of the cell nucleus: L m =αL bce +(1-α)L dice (1)L e =βL bce +(1-β)L dice +P (2) 2.2 To calculate the penalty term P in the edge branch loss function, we need to first calculate the weight of the pixels around the cell nucleus contour in the edge prediction image that affect its shape feature value F. First, we select rectangular areas of equal size with some points on the real edge as the center, and calculate the weight of the predicted area in each shape box that affects F according to formula (3), where A represents the area of ​​the real area in the rectangular box, B represents the area of ​​the predicted area in the rectangular box, and I i The closer the value of is to 1, the closer the predicted area is to the true label. Then, the shape feature F (ellipticity) of the cell nucleus is calculated. The closer F is to 1, the better the effect. The penalty term P is calculated by formula (4). The penalty value of the predicted area in the i-th box is set to P i , the penalty values ​​of other pixels are set to 0: P i =(1-I i )(1-F) (4) 2.3 Calculate the overall loss of the network, which consists of two parts. The calculation formula is shown in formula (5): L=L m +L e (5)

Images