A cell nucleus image segmentation method based on an ultra-light medical image segmentation network

Abstract

The application discloses a kind of based on ultra-light medical image segmentation network's cell nucleus image segmentation method, constructs ultra-light medical image segmentation network, and ultra-light medical image segmentation network adopts the coding-decoding structure of U type, including ultra-light encoder module and ultra-light decoder module;Cell image is normalized;The cell image after pretreatment is input into ultra-light medical image segmentation network, and the deep semantic feature of cell image is extracted by ultra-light encoder module, obtains high-level feature map and inputs high-level feature map into ultra-light decoder module, recovers resolution layer by layer, finally outputs the cell nucleus segmentation result graph that resolution and channel number are same with input;The present application can accurately determine the outline position of cell nucleus, realize efficient cell nucleus segmentation, and has wide application prospect.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing and computer vision, and particularly relates to a cell nucleus image segmentation method based on a super-light medical image segmentation network. BACKGROUND

[0002] Currently, cell cancer has become one of the most common diseases in the world and has a greater impact on the quality of human life, and even seriously threatens human life and health. Rapid and accurate segmentation on the cell nucleus image has important value for assisting doctors to improve the diagnosis efficiency and treatment success rate. However, the cell nucleus image usually has the characteristics of large noise and low contrast, which makes the cell nucleus segmentation difficult; in addition, manual segmentation of the cell nucleus image is not only tedious and inefficient, but also easily affected by subjective factors, resulting in low segmentation accuracy. Therefore, the precise and rapid cell nucleus automatic segmentation method is one of the important research targets in the field of medical image analysis.

[0003] In recent years, the medical image segmentation technology based on deep learning has made significant progress and has attracted widespread attention from researchers. The powerful segmentation network adopts the idea of extracting pixel-level features, and many networks are based on the classic U-Net for improvement. The above segmentation network is mainly composed of two parts of encoding and decoding. First, the encoder capable of fully extracting features is the basis for the network to achieve good performance. From VGG to Res-Net, the medical image segmentation network usually chooses to hire the most popular feature extractor, but the accuracy is improved at the same time, which usually leads to an increase in network complexity. Second, the decoder is indispensable for restoring image features, and most of the outstanding decoder modules are based on simple convolution and upsampling operations. Third, compared with natural image segmentation, the medical image segmentation network is mostly expanded on the basis of U-Net. Since U-Net uses a lot of skip connections in the decoding stage, it combines low-resolution information that provides object class identification and high-resolution information that provides accurate segmentation positioning, which can effectively segment medical images with fuzzy boundaries. It is worth noting that although the use of simple skip connections supplements the segmentation map with multi-scale information, there is still room for improvement. On the other hand, while ensuring accuracy, model lightweight is the premise of model deployment, but most of these segmentation networks ignore network parameter quantity and computational cost in order to pursue segmentation accuracy.

[0004] The cell nucleus in the cell nucleus image has high complexity and shape difference, and the boundary with the surrounding adjacent tissue structure is difficult to distinguish, and the traditional encoding and decoding network usually increases the network complexity to improve the segmentation accuracy, ignores the network parameter quantity and computational cost, and the traditional convolution operation cannot capture a large receptive field, so it is difficult to realize the automatic, rapid and accurate segmentation of the cell nucleus.

[0005] Therefore, it has important research significance and application value to study a light medical image segmentation network that can balance segmentation accuracy. SUMMARY

[0006] In order to overcome the shortcomings of the prior art, the purpose of the present application is to provide a cell nucleus image segmentation method based on a super-light medical image segmentation network, which can quickly and accurately determine the contour position of the cell nucleus and realize more efficient cell nucleus segmentation.

[0007] In order to achieve the above purpose, the technical scheme adopted by the present application is:

[0008] A cell nucleus image segmentation method based on a super-light medical image segmentation network, comprising the following steps:

[0009] 1) Construct a super-light medical image segmentation network:

[0010] The super-light medical image segmentation network adopts a U-shaped encoding-decoding structure, including a super-light encoder module and a super-light decoder module; the super-light encoder module adopts a Ghost bottleneck and a Tokenized MLP block to extract deep semantic features of the cell image and obtain a high-level feature map; the super-light decoder has a Tokenized MLP block in front and a Ghost bottleneck in back, which is used to restore the resolution of the high-level feature map layer by layer and output a cell nucleus segmentation result map;

[0011] 2) Preprocessing: normalizing the cell image;

[0012] 3) Input the preprocessed cell image into the super-light medical image segmentation network, extract the deep semantic features of the cell image through the super-light encoder module to obtain a high-level feature map, and input the high-level feature map into the super-light decoder module to restore the resolution layer by layer, finally output a cell nucleus segmentation result map with the same resolution and channel number as the input.

[0013] Further, the first three modules in the super-light encoder module are Ghost bottlenecks, and the last two modules are Tokenized MLP blocks, each module is followed by a max-pooling operation to reduce the resolution of the feature map by half; the first two modules of the super-light decoder module are Tokenized MLP blocks, and the last three modules are Ghost bottlenecks, each module is followed by a bilinear interpolation for up-sampling operation to increase the resolution of the feature map by 2 times.

[0014] Further, in step 1), the cell image is used as input to set the network operation parameters, and the constructed super-light medical image segmentation network is trained until the network converges.

[0015] Further, the network training parameters are set, the maximum iteration number is set to 30, the training batch is set to 8, the initial learning rate is set to 0.001, the learning rate automatic decay strategy is used, wherein the step is 1, the learning rate decay factor is 0.95, and the Adam algorithm is used to optimize the network.

[0016] Further, the X-shaped structure is designed in the super-light decoder module, which performs down-sampling on the original image and superimposes the 1 / 4 resolution feature map obtained by up-sampling in the decoder, and supplements the intermediate features to the final segmentation map.

[0017] Further, the super-light decoder module adopts a complete skip connection and an X-shaped structure for feature decoding, fully fuses the features from the input cell image through the skip connection, and supplements the detail features and real features through the X-shaped connection to restore the segmentation result map of the cell nucleus.

[0018] Further, the normalization processing in the step 2) includes normalizing the gray pixels of the cell image to the range of [0, 255] and deriving the image with a resolution size of 512*512.

[0019] Compared with the prior art, the beneficial effects of the present application are:

[0020] The present application applies a U-shaped frame to the super-light medical image segmentation network for cell nucleus segmentation, and the symmetrical structure fits the medical image features, thereby realizing efficient segmentation of the cell nucleus. Secondly, the present application also designs a super-light encoder, and the advantages of the lightweight Ghost module and the Tokenized MLP block are complementary for feature extraction. The encoder combines the lightweight convolution module Ghost module and Tokenized MLP block for the first time, takes into account the network's detail capture and global modeling capability, fully extracts the image features with a super-light encoding structure, and better performs cell nucleus segmentation. In addition, the present application also designs a multi-scale super-light decoder completely symmetrical with the encoder, which combines a complete skip connection and an X-shaped structure, more fully fuses the original features and detail information of the segmentation target, and can effectively improve the segmentation accuracy of the network. On the basis of the U-shaped structure, the X-shaped structure is fused, which not only can fully fuse information of different scales, but also can retain more real target features, realize more complete detail information recovery. The present application can accurately determine the contour position of the cell nucleus, realize efficient cell nucleus segmentation, and has a wide application prospect.

[0021] The super-light medical image segmentation network GMULNet proposed in the application adopts a brand-new super-light encoder, which combines the design ideas of multi-layer perceptron and convolution, can extract local detail information by using the Ghost module, extract global information by using the Tokenized MLP, and better improve the segmentation accuracy by combining the two. Moreover, the GMULNet can more effectively extract spatial context information while retaining high-level features.

[0022] The super-light medical image segmentation network GMULNet proposed in the application designs a brand-new super-light decoder, which adopts complete skip connection to realize the supplement of image features at multiple levels; meanwhile, the network adopts an X-shaped design, which takes into account more detailed information, so that the segmentation effect is more accurate.

[0023] The application solves the problem that the boundary between the cell nucleus and the surrounding adjacent tissue structure is difficult to distinguish, and the traditional encoding and decoding network usually ignores the network parameter quantity and the calculation cost, so it is difficult to realize the automatic, rapid and accurate segmentation of the cell nucleus. The network proposed in the application can effectively solve the problem of cell nucleus segmentation, and has wide application prospect. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 is a method flowchart of the application;

[0025] Figure 2 is a structure schematic diagram of the GMULNet network of the application;

[0026] Fig. 3(a) is a structure schematic diagram of the Ghost bottleneck module of the application;

[0027] Fig. 3(b) is a structure schematic diagram of the Ghost module;

[0028] Figure 4 is a structure schematic diagram of the Tokenized MLP of the application;

[0029] Figure 5 is a structure schematic diagram of the super-light decoder module of the application;

[0030] Figure 6 is a visual effect comparison diagram obtained by respectively using the network of the application and other different segmentation networks to perform cell nucleus segmentation on the public data set DSB2018. DETAILED DESCRIPTION

[0031] The application will be further described in detail below in combination with the drawings and specific embodiments.

[0032] The embodiment of the present application provides a kind of ultra-light medical image segmentation network GMULNet, it is related to the theoretical knowledge of depth learning, image segmentation field.The present application can be effectively applied to the automatic segmentation of cell nucleus in cell nucleus image, the problem that the precision and network parameter quantity cannot be considered when traditional coding and decoding network is segmented in the cell nucleus image data with fuzzy cell nucleus boundary is solved, a new research idea and technical means are provided for the automatic segmentation of cell nucleus.The segmentation network GMULNet of the embodiment of the present application can also be applied to other image segmentation, and is not limited to cell nucleus segmentation.

[0033] Specifically, the ultra-light medical image segmentation network GMULNet of the embodiment of the present application adopts U type coding and decoding structure, including ultra-light encoder module and ultra-light decoder module;The ultra-light encoder module adopts Ghost bottleneck and Tokenized MLP block to extract deep semantic features of cell image, obtains high-level feature map and inputs high-level feature map into the ultra-light encoder module;With image X as an example, its resolution is 512x512, and the number of channels is 3, original feature map X enters the ultra-light encoder after pretreatment, wherein the first 3 modules are Ghost bottleneck, and the last 2 modules are Tokenized MLP block, and the original feature map X is coded into high-level feature map with resolution of 32x32 and channel number of 256, i.e. the output of the ultra-light encoder;The ultra-light decoder also has 2 Tokenized MLP blocks, followed by 3 Ghost bottlenecks.The high-level feature map enters the ultra-light decoder, and the resolution is recovered layer by layer, and finally the cell nucleus segmentation result graph with the same input resolution and channel number is output.

[0034] Preferably, each module of the ultra-light encoder is followed by maximum pooling operation to reduce the resolution of feature map by half, each module of the ultra-light decoder is followed by bilinear interpolation for up-sampling operation to increase the resolution of feature map by 2 times, and each branch of X type decoding structure changes the resolution of feature map output by encoding module into the size required by decoder module.

[0035] Preferably, the network of the embodiment of the present application uses cell nucleus image as input, sets network running parameters for training until the network converges.In the training of network, the maximum number of iterations is set to 30, the training batch is set to 8, the initial learning rate is set to 0.001, the learning rate automatic decay strategy is used, wherein the step is 1, the learning rate decay factor is 0.95, and the Adam algorithm is used to optimize the network.

[0036] Preferably, in the ultra-light decoder module, an X-shaped structure is designed. The structure directly down-samples the original image and superimposes the 1 / 4 resolution feature map obtained by up-sampling in the decoder, and at the same time, the intermediate features are supplemented to the final segmentation map.

[0037] As shown in Figure 1 , the embodiment of the present application also provides a method for segmenting a nucleus image by using the above-mentioned ultra-light medical image segmentation network GMULNet, comprising:

[0038] 1) normalizing the nucleus image;

[0039] 2) segmenting the nucleus image by using the ultra-light medical image segmentation network GMULNet to obtain the segmentation result of the nucleus. The present application first designs an ultra-light encoder to further expand the receptive field of the encoder, helping it to learn effective features in the nucleus image; secondly, the present application designs an ultra-light decoder for supplementing detailed information, which is used to restore the resolution of the image. The experimental results prove that the present application has precise and rapid segmentation performance for the nucleus in the nucleus image, which can effectively improve the level of intelligent medical treatment.

[0040] Specifically, the embodiment of the present application comprises the following steps:

[0041] 1) preprocessing: normalizing the nucleus image, and normalizing the gray pixels of the nucleus image to the range of [0, 255].

[0042] The normalization process specifically includes: normalizing the gray pixels of the nucleus image to the range of [0, 255], and then exporting it as an image with a resolution size of 512x512.

[0043] Referring to Figure 2 , the preprocessed nucleus image is input into the ultra-light medical image segmentation network GMULNet, the ultra-light encoder module uses Ghost bottleneck combined with Tokenized MLP block to extract deep semantic features of the nucleus image, obtains high-level feature maps and inputs the high-level feature maps into the ultra-light decoder module; the ultra-light decoder module uses full jump connection and X-shaped structure for feature decoding, fully fuses the features from the input cell image through jump connection, and supplements detailed features and real features through X-shaped connection to restore the segmentation result image of the nucleus.

[0044] 2) referring to Figure 5 , the X-shaped structure specifically includes:

[0045] a) down-sampling the original input image to supplement the 1 / 4 resolution feature map in the decoder;

[0046] b) supplementing the intermediate features to the final segmentation map;

[0047] c) up-sampling the fused feature map to output a final segmentation map.

[0048] The super-light encoder designed on the basis of the U-shaped coding structure adopts the Ghost bottleneck structure good at capturing local information and the Tokenized MLP block good at capturing global information to extract deep semantic features. Figure 2 Figure 4 The super-light decoder structure is shown in Figure 5 .

[0049] The Ghost bottleneck is a convolutional neural network structure based on the Ghost module, which is similar to the basic residual block (Basic Residual Block) in ResNet. The Ghost bottleneck mainly consists of two stacked Ghost modules, the first Ghost module is used as an expansion layer to increase the number of channels, and the ratio of the output channel number to the input channel number is called expansion ratio here; the second Ghost module reduces the number of channels to match the shortcut path, and then uses the shortcut to connect the input and output of the two Ghost modules.

[0050] The present application uses a super-light medical image segmentation network, which aims to automatically and quickly and accurately segment the cell nucleus. Compared with existing automatic cell nucleus segmentation methods, the present application uses a designed super-light encoder to learn deep semantic features, combines a lightweight convolutional module Ghost module and a Tokenized MLP block for the first time, balances the network's detail capture and global modeling capabilities, fully extracts the features of the cell nucleus image with a super-light encoding structure, and designs a multi-scale super-light decoder completely symmetrical to the encoder. The decoder fuses the X-shaped structure on the basis of the U-shaped structure, not only can fully fuse information of different scales, but also can retain more real target features, realize more complete detail information recovery, realize more rapid and accurate cell nucleus segmentation, and has a broad application prospect.

[0051] In order to test the accuracy and superiority of the present application, the following experiments are further described.

[0052] Hardware experiment platform: GPU is NVIDIA GeForce RTX 3090Ti, and display memory is 24GB; software platform: PyTorch.​

[0053] The network is trained and tested using the nucleus images in the DSB2018 public dataset, and the training set and test set given by DSB2018 are used.

[0054] The application evaluates the segmentation performance by calculating the following two index parameters, which are respectively:

[0055]

[0056]

[0057] Wherein, TP, TN, FP and FN represent the number of true, true negative, false positive and false negative samples respectively. Acc represents the accuracy, that is, the ratio of target results in all prediction results; IoU represents the ratio of predicted target and true label, which represents the similarity or overlap degree of two samples, the value range of Acc and IoU is [0, 1], the value is closer to 1, which indicates that the segmentation effect is better;

[0058] Firstly, the role of the components (Tokenized MLP block, Ghost bottleneck) in the GMULNet and the ultra-light decoder proposed in the application is verified through ablation experiments, and the experimental results are shown in Table 1.

[0059] In the table, "+" represents that the corresponding module is applied, TMB represents Tokenized MLP block; GB represents Ghost bottleneck, otherwise the conventional convolution is used; GX represents the ultra-light decoder, otherwise the decoder of UNet is the same. The first row represents the network structure of the original UNet as the baseline network.

[0060] As can be seen from Table 1, under the premise of reducing the depth of the network, the use of Tokenized MLP block makes the segmentation accuracy slightly decrease, but greatly reduces the complexity and parameter amount of the network. The application of Ghost bottleneck not only makes the segmentation accuracy slightly improve, but also reduces the parameter amount of the network to less than 1M. When the network increases the ultra-light decoder combined with the X-shaped structure designed in this chapter, the segmentation accuracy is greatly improved, especially the Acc is obviously improved. Finally, the complexity of the overall network is only 0.57G FLOPs, and the parameter amount is only 1M, which effectively completes the network lightening compared with the baseline, and achieves the best balance between segmentation accuracy and network complexity.

[0061] Table 1 Ablation experiment of each component of GMULNet

[0062]

[0063] To verify the role of the GMULNet proposed in the present application in the nucleus segmentation as a whole, the GMULNet proposed in the present application is compared with a plurality of mainstream medical image segmentation networks. The selected comparison networks include UNet, DeepLabv3+, CENet and other classic networks, and also include lightweight medical image segmentation networks UNet++, GhostUNet and UNeXt. The lightweight evaluation indexes involved are parameter quantity Params and FLOPs, and the segmentation accuracy evaluation indexes are IoU and Acc.

[0064] To intuitively show the lightweight of the GMULNet proposed in the present application, the parameter quantity and complexity of the above network are calculated, as shown in Table 2. As can be seen from Table 2, the parameter quantity of the GMULNet is only 1M. When the input image size is 256x256 and 512x512, the model complexity of the GMULNet is 0.57G FLOPs and 2.26G FLOPs, respectively. Compared with the six comparison networks, the GMULNet has achieved great advantages in lightweight. Among them, compared with the currently popular lightweight medical image segmentation network UNet++, the parameter quantity of the GMULNet is only 1 / 9 of that of UNet++. When the segmentation size is 512x512, the network complexity of the GMULNet is only 1 / 50 of that of UNet++. Compared with the classic method UNet, the GMULNet not only greatly reduces the parameter quantity, but also reduces the network complexity by more than 100 times.

[0065] Table 2 Parameter quantity, calculation cost and inference time of GMULNet and the most advanced method

[0066]

[0067] The accuracy quantitative results of the above seven segmentation networks on the nucleus data set are shown in Table 3. As can be seen from Table 3, the IoU and Acc values of the GMULNet achieve the highest segmentation accuracy. Among them, the IoU value of the GMULNet is improved by 4.94% compared with the popular lightweight segmentation network UNet++, and is improved by 13.92% compared with the IoU value of the classic model CENet. This is because the comparison method only simply transmits the low-level features to the decoder through the jump connection, while the super-light decoder designed in the present application fuses the X-shaped structure, and applies the Ghost module and Tokenized MLP block in the whole network to balance the local and global information of the feature map, thereby improving the nucleus segmentation accuracy. In summary, the performance of the GMULNet in the nucleus segmentation is obviously better than that of other segmentation networks, and the accuracy of the nucleus segmentation can be effectively improved.

[0068] Table 3 Comparison of segmentation accuracy of seven networks on the nucleus data set

[0069]

[0070] Through Figure 6 The comprehensive analysis of Table 2 and Table 3 proves that the GMULNet proposed in this invention is more lightweight and can achieve more accurate cell nucleus segmentation, and has broad application prospects in the fields of auxiliary diagnosis of cell diseases.

Claims

1.A method for nucleus image segmentation based on an ultra-light medical image segmentation network, characterized by Comprising the following steps: 1) Constructing a super-light medical image segmentation network: The super-light medical image segmentation network adopts a U-shaped encoding-decoding structure, including a super-light encoder module and a super-light decoder module; the super-light encoder module adopts a Ghost bottleneck and a Tokenized MLP block to extract deep semantic features of the cell image and obtain a high-level feature map; the super-light decoder is preceded by a Tokenized MLP block and followed by a Ghost bottleneck, which is used to restore the resolution of the high-level feature map layer by layer and output a cell nucleus segmentation result map; An X-shaped structure is designed in the super-light decoder module, which stacks the original image after downsampling with the 1 / 4 resolution feature map obtained by upsampling in the super-light decoder, while supplementing the intermediate features, i.e., the 1 / 4 resolution feature map of the super-light encoder, to the final segmentation map; the X-shaped structure directly supplements the feature map output by the first layer of the super-light encoder to the third layer of the super-light decoder, and directly supplements the feature map output by the third layer of the super-light encoder to the first layer of the super-light decoder; the super-light decoder module uses full skip connection and X-shaped structure for feature decoding; the full skip connection directly supplements the feature map output by each layer of the super-light encoder to the corresponding layer of the super-light decoder and performs feature fusion, and this process is a direct skip connection without any processing operation; the features from the input cell image are fused through full skip connection, and the details and true features are supplemented through the X-shaped structure to restore the cell nucleus segmentation result map; 2) Preprocessing: normalizing the cell image; 3) inputting the preprocessed cell image into the super-light medical image segmentation network, extracting deep semantic features of the cell image through the super-light encoder module to obtain a high-level feature map, and inputting the high-level feature map into the super-light decoder module to restore the resolution layer by layer, and finally outputting a cell nucleus segmentation result map with the same resolution and channel number as the input. 2.The method of claim 1, wherein the method comprises: The first three modules in the super-light encoder module are Ghost bottleneck, and the last two modules are Tokenized MLP block, followed by a max pooling operation to reduce the feature map resolution by half; the first two modules in the super-light decoder module are Tokenized MLP block, and the last three modules are Ghost bottleneck, followed by a bilinear interpolation for upsampling operation to increase the resolution of the feature map by 2 times. 3.The method of claim 1, wherein the method further comprises: determining a plurality of segmentation masks for the plurality of nuclei images based on the plurality of segmentation masks for the plurality of training images. In step 1), the cell image is used as input to set the network running parameters, and the constructed super-light medical image segmentation network is trained until the network converges. 4.The method of claim 3, wherein the method further comprises: determining a plurality of segmentation masks for the plurality of nuclei images based on the plurality of segmentation masks for the plurality of training images. The network training parameters are set as follows: the maximum number of iterations is set to 30, the training batch is set to 8, the initial learning rate is set to 0.001, the learning rate automatic decay strategy is used, the step is 1, the learning rate decay factor is 0.95, and the Adam algorithm is used to optimize the network. 5.The method of claim 1, wherein the method further comprises: determining a plurality of segmentation masks for the plurality of nuclei images based on the segmentation network; and determining a plurality of segmentation masks for the plurality of nuclei images based on the segmentation network. The normalization processing used in step 2) includes normalizing the gray pixels of the cell image to the range of [0, 255], and exporting it as an image with a resolution size of 512x512.

Images