Cell nucleus instance segmentation method based on multi-scale feature fusion

Abstract

A cell nucleus instance segmentation method based on multi-scale feature fusion is proposed, relating to the field of medical image processing. Point features can accurately locate the cell nucleus, line features can describe the shape and contour of the cell nucleus, while surface features provide a holistic description of the cell nucleus region. By comprehensively utilizing point, line, and surface features, a comprehensive description of the cell nucleus is achieved, increasing the network's representational capability. First, a mask optimization algorithm module is proposed to effectively merge point and boundary features. Second, a post-processing method is introduced to generate an instance segmentation image by combining target information and the cell nucleus mask image. The effectiveness of each component in the proposed cell nucleus instance segmentation method based on multi-scale feature fusion is then evaluated.

Description

Technical Field

[0001] This invention relates to the field of medical image processing, specifically to a cell nucleus instance segmentation method based on multi-scale feature fusion. Background Technology

[0002] In pathology, the morphology and distribution characteristics of cell nuclei are crucial for disease diagnosis and treatment. Accurate segmentation of cell nuclei allows physicians to perform more precise pathological analysis, identify abnormal cell morphologies and structures, and thus help determine the type, grade, and prognosis of the disease. Furthermore, cell nucleus segmentation can be used to assess the effects of drugs on cells and tissues, and to monitor treatment efficacy. Due to the complexity and diversity of cell nuclei in pathological images, cell nucleus segmentation is a challenging task. Cell nuclei often have irregular shapes, varying sizes and colors, and may overlap or contact. In addition, pathological images are often accompanied by noise, artifacts, and uneven lighting, further increasing the difficulty of the segmentation task. Summary of the Invention

[0003] In order to overcome the shortcomings of the above technologies, this invention provides a cell nucleus instance segmentation method based on multi-scale feature fusion that improves the accuracy of cell nucleus instance segmentation.

[0004] The technical solution adopted by the present invention to overcome the technical problems is:

[0005] A cell nucleus instance segmentation method based on multi-scale feature fusion, comprising:

[0006] a) Obtain n cell nucleus images and their corresponding labels to obtain a cell nucleus image set Y and a label set G, where Y = {Y1, Y2, ..., Y}. i ,...,Y n}, Y i Let G be the i-th cell nucleus image, i∈{1,...,n}, and G={G1,G2,...,G i ,...,G n}, G i For the i-th cell nucleus image Y i Corresponding tags;

[0007] b) Divide the cell nucleus image set Y into a training set P and a test set T, where P = {P1, P2, ..., P2}. i ,...,P o}, P i Let T be the i-th cell nucleus image in the training set P, i∈{1,...,o}, and o be the total number of cell nucleus images in the training set P, where T={T1,T2,…,T…}. i ,…,T q}, T iLet i be the i-th cell nucleus image in the test set T, where i∈{1,...,q} and q is the total number of cell nucleus images in the test set T.

[0008] c) Take the i-th cell nucleus image P from the training set P. i The corresponding tag G i Convert to cell nucleus edge contour label and heat map labels All cell nucleus edge contour labels in the training set P constitute the cell nucleus edge contour label set. All heatmap labels in the training set P constitute the heatmap label set.

[0009] d) Establish a segmentation network, which consists of a feature extraction module, a mask optimization algorithm module, and a post-processing algorithm module;

[0010] e) Take the i-th cell nucleus image P from the training set P. i The input is fed into the feature extraction module of the segmentation network, and the output is a feature map. Feature map Feature map

[0011] f) Utilizing feature maps Obtain key point map of cell nucleus

[0012] g) Feature map Feature map Key points of the cell nucleus The input is fed into the mask optimization algorithm module of the segmentation network, and the output is the feature map.

[0013] h) Feature map With feature map The input is fed into the post-processing algorithm module of the segmentation network, and the output is the final cell nucleus instance segmentation map. i) Use the SGD optimizer to utilize the total loss function L total Train the segmentation network to obtain the optimized segmentation network; j) Take the i-th cell nucleus image P from the test set T. i The input is fed into the optimized segmentation network, and the output is a segmentation map of cell nuclei instances. Further, in step a), n cell nucleus images and their corresponding labels are obtained from the MoNuSeg dataset. Further, in step b), the cell nucleus image set Y is divided into a training set P and a test set T in a 7:3 ratio.

[0014] Furthermore, step c) includes the following steps:

[0015] c-1) Use the cv2.boundingRect() function to obtain the i-th cell nucleus image P in the training set P. i The corresponding tag G i The boundary rectangle information includes the center point coordinates, the height of the boundary rectangle, and the width of the boundary rectangle;

[0016] c-2) Input the height and width of the boundary rectangle into the gaussian_radius() function to calculate the radius of the Gaussian kernel.

[0017] c-3) The diameter Dia of the Gaussian kernel is calculated by Dia = 2*(radius+1);

[0018] c-4) Input the diameter Dia into the gussian2D() function of the astropy library to obtain the Gaussian kernel Gk; c-5) Place the Gaussian kernel Gk at the center point coordinates of the boundary rectangle to obtain the heatmap labels. c-6) Take the i-th cell nucleus image P from the training set P. i The corresponding tag G i Convert to a NumPy array using the np.array() function;

[0019] c-7) Use cv2.Canny() to perform edge detection on the NumPy array and convert it to a uint8 type edge detection image using the np.uint8() function;

[0020] c-8) Set the values ​​of non-zero pixels in the edge detection image to 255 and the values ​​of the remaining pixels to 0 to obtain the cell nucleus edge contour labels. Furthermore, step e) includes the following steps:

[0021] e-1) The feature extraction module of the segmentation network consists of a U-Net model, a first two-dimensional deconvolution layer, a second two-dimensional deconvolution layer, and a third two-dimensional deconvolution layer, which extracts the i-th cell nucleus image P from the training set P. i The input is fed into the U-Net model of the feature extraction module, and the output is the feature map M. i ;

[0022] e-2) The feature map M i The input is fed into the first two-dimensional deconvolution layer of the feature extraction module, and the output is the feature map.

[0023] e-3) The feature map M i The input is fed into the second two-dimensional deconvolution layer of the feature extraction module, and the output is the feature map.

[0024] e-4) The feature map M i The input is fed into the third two-dimensional deconvolution layer of the feature extraction module, and the output is the feature map.

[0025] Furthermore, in step f), the peak_local_max() function from the skimage package is used to find the feature map. Local peaks were observed to obtain a key point map of the cell nucleus. Furthermore, step g) includes the following steps:

[0026] g-1) feature map The input is fed into the mask optimization algorithm module of the segmentation network, and an image is initialized using the Image.new() function, which is similar to the feature map. Blank feature maps of the same size g-2) Map of key points of the cell nucleus The input is fed into the mask optimization algorithm module of the segmentation network, and the cell nucleus key point map is obtained through the np.column_stack(np.where()) function. Given n key points in the cell nuclei, we obtain a key point set K, K = {k1, k2, ..., kn}. i ,…,k n}, k i Let i be the key point of the i-th cell nucleus, where i∈{1,…,n};

[0027] g-3) feature map The input is fed into the mask optimization algorithm module of the segmentation network, and the feature map is obtained through the cv2.findContours() function. Given m cell nuclear boundaries, we obtain the cell nuclear boundary set E, E = {e1, e2, ..., e...} i ,...,e n}, e i This represents the boundary of the i-th cell nucleus;

[0028] g-4) Use the sklearn.neighbors.NearestNeighbors() function to analyze the keypoints k of the i-th cell nucleus. i Perform a nearest neighbor search; when the corresponding boundary of the i-th cell nucleus e is found... i Stop the search in the blank feature map The corresponding position in the map is marked as 1, and the feature map is blank. After all corresponding positions are marked, the feature map is obtained. Furthermore, step h) includes the following steps:

[0029] h-1) feature map With feature map The input is fed into the post-processing algorithm module of the segmentation network for subtraction to obtain the difference map. h-2) Using the cv2.connectedComponentsWithStats() function to compare the plots Perform connected component analysis to output a set C for each connected region, C = {c1, c2, ..., c3}. i ,...,c n}, c i For difference map In the j-th subregion, if the j-th subregion c i If the pixel value is less than 5 pixels, delete it; if the j-th subregion c i If the pixel value is greater than or equal to 5 pixels, it is retained to obtain the feature map. h-3) Use the cv2.dilate() function to process the feature map. Perform a dilation operation to obtain the final segmentation map of cell nuclei. The convolution kernel used in the dilation operation is a kernel. Furthermore, in step i), formula L total =L p +L edge +L m The total loss function L is calculated. total In the formula L p Mean squared error loss function, heatmap labels The mean squared error loss function L p Label map, feature map The mean squared error loss function L p The original image, L edge The binary cross-entropy loss function and feature map are shown. The binary cross-entropy loss function L edge The original image, cell nucleus edge outline label The binary cross-entropy loss function L edge The label image, L m The cross-entropy loss function and feature map are given. The cross-entropy loss function L m The original image, the i-th cell nucleus image Y i The corresponding tag G i The cross-entropy loss function L m The tag image.

[0030] The beneficial effects of this invention are: point features can accurately locate the cell nucleus, line features can describe the shape and contour of the cell nucleus, and surface features provide a holistic description of the cell nucleus region. By comprehensively utilizing point, line, and surface features, a comprehensive description of the cell nucleus is achieved, increasing the network's representational capabilities. First, a mask optimization algorithm module is proposed to effectively merge point and boundary features. Second, a post-processing method is introduced to generate an instance segmentation image by combining target information and the cell nucleus mask image. The effectiveness of each component in the proposed cell nucleus instance segmentation method based on multi-scale feature fusion is then evaluated. Attached Figure Description

[0031] Figure 1 This is a structural diagram of the segmentation network of the present invention;

[0032] Figure 2 The images show the segmentation results for the MoNuSeg, Kumar, and CoNSep datasets. Detailed Implementation

[0033] The following is in conjunction with the appendix Figure 1 Appendix Figure 2 The present invention will be further described below.

[0034] A cell nucleus instance segmentation method based on multi-scale feature fusion, comprising:

[0035] a) Obtain n cell nucleus images and their corresponding labels to obtain a cell nucleus image set Y and a label set G, where Y = {Y1, Y2, ..., Y}. i ,...,Y n}, Y i Let G be the i-th cell nucleus image, i∈{1,...,n}, and G={G1,G2,...,G i ,...,G n}, G i For the i-th cell nucleus image Y i The corresponding tags.

[0036] b) Divide the cell nucleus image set Y into a training set P and a test set T, where P = {P1, P2, ..., P2}. i ,...,P o}, P i Let T be the i-th cell nucleus image in the training set P, i∈{1,...,o}, and o be the total number of cell nucleus images in the training set P, where T={T1,T2,...,T i ,...,T q}, T i Let be the i-th cell nucleus image in the test set T, where i∈{1,…,q} and q is the total number of cell nucleus images in the test set T.

[0037] c) Take the i-th cell nucleus image P from the training set P. i The corresponding tag G i Convert to cell nucleus edge contour label and heat map labels All cell nucleus edge contour labels in the training set P constitute the cell nucleus edge contour label set. All heatmap labels in the training set P constitute the heatmap label set.

[0038] d) Establish a segmentation network, which consists of a feature extraction module, a mask optimization algorithm module, and a post-processing algorithm module.

[0039] e) Take the i-th cell nucleus image P from the training set P. i The input is fed into the feature extraction module of the segmentation network, and the output is a feature map. Feature map Feature map

[0040] f) Utilizing feature maps Obtain key point map of cell nucleus

[0041] g) Feature map Feature map Key points of the cell nucleus The input is fed into the mask optimization algorithm module of the segmentation network, and the output is the feature map.

[0042] h) Feature map With feature map The input is fed into the post-processing algorithm module of the segmentation network, and the output is the final cell nucleus instance segmentation map. i) Use the SGD optimizer to utilize the total loss function L total Train the segmentation network to obtain the optimized segmentation network. j) Take the i-th cell nucleus image P from the test set T. i The input is fed into the optimized segmentation network, and the output is a segmentation map of cell nuclei instances. In one embodiment of the present invention, step a) involves obtaining n cell nucleus images and their corresponding labels from the MoNuSeg dataset.

[0043] In one embodiment of the present invention, in step b), the cell nucleus image set Y is divided into a training set P and a test set T in a ratio of 7:3.

[0044] In one embodiment of the present invention, step c) includes the following steps:

[0045] c-1) Use the cv2.boundingRect() function to obtain the i-th cell nucleus image P in the training set P. i The corresponding tag G i The boundary rectangle information includes the center point coordinates, the height of the boundary rectangle, and the width of the boundary rectangle.

[0046] c-2) Input the height and width of the boundary rectangle into the gaussian_radius() function to calculate the radius of the Gaussian kernel.

[0047] c-3) The diameter Dia of the Gaussian kernel is calculated by Dia = 2*(radius+1).

[0048] c-4) Input the diameter Dia into the gussian2D() function of the astropy library to obtain the Gaussian kernel Gk. c-5) Place the Gaussian kernel Gk at the center point coordinates of the boundary rectangle to obtain the heatmap labels. c-6) Take the i-th cell nucleus image P from the training set P. i The corresponding tag G i Convert the array to NumPy format using the np.array() function.

[0049] c-7) Use cv2.Canny() to perform edge detection on the NumPy array and convert it to a uint8 type edge detection image using the np.uint8() function.

[0050] c-8) Set the values ​​of non-zero pixels in the edge detection image to 255 and the values ​​of the remaining pixels to 0 to obtain the cell nucleus edge contour labels. In one embodiment of the present invention, step e) includes the following steps:

[0051] e-1) The feature extraction module of the segmentation network consists of a U-Net model, a first two-dimensional deconvolution layer, a second two-dimensional deconvolution layer, and a third two-dimensional deconvolution layer, which extracts the i-th cell nucleus image P from the training set P. i The input is fed into the U-Net model of the feature extraction module, and the output is the feature map M. i .

[0052] e-2) The feature map M i The input is fed into the first two-dimensional deconvolution layer of the feature extraction module, and the output is the feature map.

[0053] e-3) The feature map M i The input is fed into the second two-dimensional deconvolution layer of the feature extraction module, and the output is the feature map.

[0054] e-4) The feature map M i The input is fed into the third two-dimensional deconvolution layer of the feature extraction module, and the output is the feature map.

[0055] In one embodiment of the present invention, step f) uses the peak_local_max() function in the skimage package to find the feature map. Local peaks were observed to obtain a key point map of the cell nucleus. In one embodiment of the present invention, step g) includes the following steps:

[0056] g-1) feature map The input is fed into the mask optimization algorithm module of the segmentation network, and an image is initialized using the Image.new() function, which is similar to the feature map. Blank feature maps of the same size g-2) Map of key points of the cell nucleus The input is fed into the mask optimization algorithm module of the segmentation network, and the cell nucleus key point map is obtained through the np.column_stack(np.where()) function. Given n key points in the cell nuclei, we obtain a key point set K, K = {k1, k2, ..., kn}. i ,…,k n}, k i Let i be the key point of the i-th cell nucleus, where i∈{1,...,n}.

[0057] g-3) feature map The input is fed into the mask optimization algorithm module of the segmentation network, and the feature map is obtained through the cv2.findContours() function. Given m cell nuclear boundaries, we obtain the cell nuclear boundary set E, E = {e1, e2, ..., e...} i ,...,e n}, e i This represents the boundary of the i-th cell nucleus.

[0058] g-4) Use the sklearn.neighbors.NearestNeighbors() function to analyze the keypoints k of the i-th cell nucleus. i Perform a nearest neighbor search; when the corresponding boundary of the i-th cell nucleus e is found... i Stop the search in the blank feature map The corresponding position in the map is marked as 1, and the feature map is blank. After all corresponding positions are marked, the feature map is obtained. In one embodiment of the present invention, step h) includes the following steps:

[0059] h-1) feature map With feature map The input is fed into the post-processing algorithm module of the segmentation network for subtraction to obtain the difference map. h-2) Using the cv2.connectedComponentsWithStats() function to compare the plots Perform connected component analysis and output the set C of each connected component, C = {c1, c2, ..., c3}. i ,...,c n}, c i For difference map In the j-th subregion, if the j-th subregion c i If the pixel value is less than 5 pixels, delete it; if the j-th subregion c i If the pixel value is greater than or equal to 5 pixels, it is retained to obtain the feature map. h-3) Use the cv2.dilate() function to process the feature map. Perform a dilation operation to obtain the final segmentation map of cell nuclei. The convolution kernel used in the dilation operation is a kernel. In one embodiment of the present invention, step i) is performed using formula L total =L p +L edge +L m The total loss function L is calculated. total In the formula L p Mean squared error loss function, heatmap labels The mean squared error loss function L p Label map, feature map The mean squared error loss function L p The original image, L edge The binary cross-entropy loss function and feature map are shown. The binary cross-entropy loss function L edge The original image, cell nucleus edge outline label The binary cross-entropy loss function L edge The label image, L m The cross-entropy loss function and feature map are given. The cross-entropy loss function L m The original image, the i-th cell nucleus image Y i The corresponding tag G i The cross-entropy loss function L m The tag image.

[0060] We implemented these models on an NVIDIA Tesla V100 system using PyTorch 3.7. During training, we maintained the original dimensions of all datasets and applied data augmentation techniques such as random horizontal and vertical flips and random rotations. We optimized using stochastic gradient descent (SGD) with a learning rate of 0.00001 and an adaptive learning rate decay strategy.

[0061] For the MoNuSeg dataset, the input size is 1000×1000 and the batch size is 12; for the Kumar dataset, the input size is 1000×1000 and the batch size is 8; for the CoNSep dataset, the input size is 512×512 and the batch size is 12. We save a checkpoint every 20 epochs of training, for a total of 200 epochs. Furthermore, we employ a specific method in the post-processing stage to address the discontinuity problem in cell nucleus segmentation.

[0062] A qualitative analysis of the proposed method is attached. Figure 2 As shown, the images, from top to bottom, are cropped from the test images of the MoNuSeg, Kumar, and CoNSep datasets. The original image and its corresponding segmentation kernel are listed in rows. (See attached...) Figure 2 As can be seen, the proposed model can successfully segment overlapping cell nuclei, showing a significant advantage over the other three models.

[0063] Table 1 shows the comparison results between the MoNuSeg dataset and the SOTAs models.

[0064] Table 1 shows the comparison results between the MoNuSeg dataset and the SOTAs models.

[0065]

[0066] For quantitative analysis, Table 1 shows that our model achieved good performance in the cell nucleus segmentation task. Specifically, it achieved 0.83711 Dice and 0.7213 AJI on the MoNuSeg dataset, which is a significant improvement over Hover-Net and CIA-Net.

[0067] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A cell nucleus instance segmentation method based on multi-scale feature fusion, characterized in that, include: a) Obtain n cell nucleus images and their corresponding labels to obtain a cell nucleus image set Y and a label set G, where Y = {Y1, Y2, ..., Y}. i ,...,Y n }, Y i Let G be the i-th cell nucleus image, i∈{1,...,n}, and G={G1,G2,...,G i ,...,G n }, G i For the i-th cell nucleus image Y i Corresponding tags; b) Divide the cell nucleus image set Y into a training set P and a test set T, where P = {P1, P2, ..., P2}. i ,...,P o }, P i Let T be the i-th cell nucleus image in the training set P, i∈{1,...,o}, and o be the total number of cell nucleus images in the training set P, where T={T1,T2,...,T i ,...,T q }, T i Let i be the i-th cell nucleus image in the test set T, i∈{1,…,q}, and q be the total number of cell nucleus images in the test set T. c) Take the i-th cell nucleus image P from the training set P. i The corresponding tag G i Convert to cell nucleus edge contour label and heat map labels All cell nucleus edge contour labels in the training set P constitute the cell nucleus edge contour label set. All heatmap labels in the training set P constitute the heatmap label set. d) Establish a segmentation network, which consists of a feature extraction module, a mask optimization algorithm module, and a post-processing algorithm module; e) Take the i-th cell nucleus image P from the training set P. i The input is fed into the feature extraction module of the segmentation network, and the output is a feature map. Feature map Feature map f) Utilizing feature maps Obtain key point map of cell nucleus g) Feature map Feature map Key points of the cell nucleus The input is fed into the mask optimization algorithm module of the segmentation network, and the output is the feature map. h) Feature map With feature map The input is fed into the post-processing algorithm module of the segmentation network, and the output is the final cell nucleus instance segmentation map. i) Use the SGD optimizer to utilize the total loss function L total Train the segmentation network to obtain the optimized segmentation network; j) Take the i-th cell nucleus image P from the test set T. i The input is fed into the optimized segmentation network, and the output is a segmentation map of cell nuclei instances.

2. The cell nucleus instance segmentation method based on multi-scale feature fusion according to claim 1, characterized in that: In step a), n cell nucleus images and their corresponding labels are obtained from the MoNuSeg dataset.

3. The cell nucleus instance segmentation method based on multi-scale feature fusion according to claim 1, characterized in that: In step b), the cell nucleus image set Y is divided into a training set P and a test set T in a 7:3 ratio.

4. The cell nucleus instance segmentation method based on multi-scale feature fusion according to claim 1, characterized in that, Step c) includes the following steps: c-1) Use the cv2.boundingRect() function to obtain the i-th cell nucleus image P in the training set P. i The corresponding tag G i The boundary rectangle information includes the center point coordinates, the height of the boundary rectangle, and the width of the boundary rectangle; c-2) Input the height and width of the boundary rectangle into the gaussian_radius() function to calculate the radius of the Gaussian kernel. c-3) The diameter Dia of the Gaussian kernel is calculated by Dia = 2*(radius+1); c-4) Input the diameter Dia into the gussian2D() function in the astropy library to obtain the Gaussian kernel Gk; c-5) Place the Gaussian kernel Gk at the center point coordinates of the boundary rectangle to obtain the heatmap label. c-6) Take the i-th cell nucleus image P from the training set P. i The corresponding tag G i Convert to a NumPy array using the np.array() function; c-7) Use cv2.Canny() to perform edge detection on the NumPy array and convert it to a uint8 type edge detection image using the np.uint8() function; c-8) Set the values ​​of non-zero pixels in the edge detection image to 255 and the values ​​of the remaining pixels to 0 to obtain the cell nucleus edge contour labels.

5. The cell nucleus instance segmentation method based on multi-scale feature fusion according to claim 1, characterized in that, Step e) includes the following steps: e-1) The feature extraction module of the segmentation network consists of a U-Net model, a first two-dimensional deconvolution layer, a second two-dimensional deconvolution layer, and a third two-dimensional deconvolution layer, which extracts the i-th cell nucleus image P from the training set P. i The input is fed into the U-Net model of the feature extraction module, and the output is the feature map M. i ; e-2) The feature map M i The input is fed into the first two-dimensional deconvolution layer of the feature extraction module, and the output is the feature map. e-3) The feature map M i The input is fed into the second two-dimensional deconvolution layer of the feature extraction module, and the output is the feature map. e-4) The feature map M i The input is fed into the third two-dimensional deconvolution layer of the feature extraction module, and the output is the feature map.

6. The cell nucleus instance segmentation method based on multi-scale feature fusion according to claim 1, characterized in that: In step f), the peak_local_max() function from the skimage package is used to find feature maps. Local peaks were observed to obtain a key point map of the cell nucleus.

7. The cell nucleus instance segmentation method based on multi-scale feature fusion according to claim 1, characterized in that, Step g) includes the following steps: g-1) feature map The input is fed into the mask optimization algorithm module of the segmentation network, and an image is initialized using the Image.new() function, which is similar to the feature map. Blank feature maps of the same size g-2) Map of key points of the cell nucleus The input is fed into the mask optimization algorithm module of the segmentation network, and the cell nucleus key point map is obtained through the np.column_stack(np.where()) function. Given n key points in the cell nuclei, we obtain a key point set K, K = {k1, k2, ..., kn}. i ,...,k n }, k i Let i be the key point of the i-th cell nucleus, where i∈{1,...,n}; g-3) feature map The input is fed into the mask optimization algorithm module of the segmentation network, and the feature map is obtained through the cv2.findContours() function. Given m cell nuclei, we obtain the cell nucleus boundary set E, E = {e1, e2, ..., e...} i ,...,e n }, e i This represents the boundary of the i-th cell nucleus; g-4) Use the sklearn.neighbors.NearestNeighbors() function to analyze the keypoints k of the i-th cell nucleus. i Perform a nearest neighbor search; when the corresponding boundary of the i-th cell nucleus e is found... i Stop the search in the blank feature map The corresponding position in the map is marked as 1, and the feature map is blank. After all corresponding positions are marked, the feature map is obtained.

8. The cell nucleus instance segmentation method based on multi-scale feature fusion according to claim 1, characterized in that, Step h) includes the following steps: h-1) feature map With feature map The input is fed into the post-processing algorithm module of the segmentation network for subtraction to obtain the difference map. h-2) Using the cv2.connectedComponentsWithStats() function to compare the plots Perform connected component analysis to output a set C for each connected region, C = {c1, c2, ..., c3}. i ,...,c n }, c i For difference map In the j-th subregion, if the j-th subregion c i If the pixel value is less than 5 pixels, delete it; if the j-th subregion c i If the pixel value is greater than or equal to 5 pixels, it is retained to obtain the feature map. h-3) Use the cv2.dilate() function to process the feature map. Perform a dilation operation to obtain the final segmentation map of cell nuclei. The convolution kernel used in the dilation operation is a kernel.

9. The cell nucleus instance segmentation method based on multi-scale feature fusion according to claim 1, characterized in that: In step i), formula L total =L p +L edge +L m The total loss function L is calculated. total In the formula L p Mean squared error loss function, heatmap labels The mean squared error loss function L p Label map, feature map The mean squared error loss function L p The original image, L edge The binary cross-entropy loss function and feature map are shown. The binary cross-entropy loss function L edge The original image, cell nucleus edge outline label The binary cross-entropy loss function L edge The label image, L m The cross-entropy loss function and feature map are given. The cross-entropy loss function L m The original image, the i-th cell nucleus image Y i The corresponding tag G i The cross-entropy loss function L m The tag image.

Images