A liver segmentation method based on multi-scale semantic feature network

Abstract

The present invention discloses a liver segmentation method based on a multi-scale semantic feature network, comprising the following steps: selecting an existing abdominal CT public dataset image and preprocessing the images in the dataset; building a network model based on the multi-scale semantic feature network structure; sending the preprocessed images in batches to the built multi-scale semantic feature network model to train the model; preprocessing the abdominal CT images to be segmented; inputting the preprocessed abdominal CT images to be segmented into the trained multi-scale semantic feature network model, which outputs a segmented image of the liver region. The present invention employs PLSANet to extract multi-scale semantic features, thereby resolving the problem of the large scale of the network model and improving the operational efficiency of the model; the present invention integrates multi-scale semantic feature information, which is beneficial for reducing information loss generated during the convolution calculation process, making the network more sensitive to small target areas, and improving the segmentation accuracy of the liver region.

Description

Technical Field

[0001] The present invention relates to computer vision, and in particular to a liver segmentation method based on a multi-scale semantic feature network. Background Art

[0002] Rapidly and accurately segmenting the liver region from abdominal CT images is a critical task in liver disease diagnosis and treatment planning. A single abdominal CT image typically contains hundreds to thousands of images, making segmenting the liver layer by layer an extremely time-consuming and laborious task for doctors. Furthermore, manual segmentation by doctors is subjective and highly dependent on their experience, increasing the likelihood of misjudgment. Therefore, using computer vision technology to assist doctors in liver segmentation is crucial for improving the efficiency of liver disease diagnosis.

[0003] Computer vision methods commonly used for liver segmentation include traditional methods and deep learning methods. Traditional liver segmentation methods are generally based on grayscale, such as the OTSU threshold-based liver segmentation method proposed in Patent No. CN201910526937.5. Deep learning methods for liver segmentation generally train a convolutional neural network model and then input the image to be segmented into the network model to obtain a segmented image, such as the liver segmentation method based on residual-attention deep neural network proposed in Patent No. CN201811045104.9.

[0004] Traditional liver segmentation methods generally require manual setting of numerous parameters, which play a decisive role in the accuracy of liver segmentation, resulting in the accuracy of traditional liver segmentation methods being limited by human subjectivity. Deep learning methods can automatically learn parameters, but deep learning network models are prone to loss of image feature information during the transmission process. As a result, deep learning network models are generally more accurate in segmenting large-scale objects, but less accurate in segmenting small-scale objects. Deep learning network models commonly used for liver segmentation are generally large in scale and slow to train and run. Summary of the Invention

[0005] Purpose of the invention: The purpose of the present invention is to provide a liver segmentation method based on a multi-scale semantic feature network, so as to solve the problem of low accuracy of small-scale liver region segmentation and improve the efficiency of liver segmentation.

[0006] Technical solution: The liver segmentation method based on a multi-scale semantic feature network described in the present invention is characterized by comprising the following steps:

[0007] (1) Select existing abdominal CT public dataset images and preprocess the images in the dataset;

[0008] (2) Build a network model based on the multi-scale semantic feature network structure;

[0009] (3) The preprocessed images are fed into the built multi-scale semantic feature network model in batches to train the model;

[0010] (4) Preprocessing the abdominal CT image to be segmented;

[0011] (5) The preprocessed abdominal CT image to be segmented is input into the trained multi-scale semantic feature network model, and the model outputs a segmented image of the liver area.

[0012] The step (1) is specifically as follows:

[0013] (1.1) Hu value truncation is performed on the image to enhance the contrast of the abdominal CT image;

[0014] (1.2) Flip the image and expand the abdominal CT image dataset.

[0015] The step (2) is specifically as follows:

[0016] (2.1) The multi-scale semantic feature network structure includes two parts: feature extraction and feature restoration;

[0017] (2.2) The feature extraction part uses PLSANet to extract multi-scale semantic features of abdominal CT images;

[0018] (2.3) The feature restoration part uses the RFB module, SA module and upsampling operation to fuse the high-level semantic features of the abdominal CT image and generate a segmented image of the liver area.

[0019] The PLSANet described in step (2.2) is specifically as follows: PP-LCNet is used as the basic network structure, the last classification network part in PP-LCNet is removed, and the channel attention SE module is replaced by the compression attention SA module to obtain PLSANet.

[0020] The high-level semantic features of the fused abdominal CT image described in step (2.3) are specifically as follows: there are five semantic features extracted in the feature extraction part, and the semantic features after each scale reduction are calculated in the order of low-level semantic feature 1, low-level semantic feature 2, high-level semantic feature 1, high-level semantic feature 2, and high-level semantic feature 3, denoted as L1, L2, H1, H2, and H3; among them, H3 is added to H2 calculated by the RFB module after upsampling, and then input into the SA module to obtain S2; S2 is added to H1 calculated by the RFB module after upsampling, and then input into the SA module to obtain S1, which is the fused semantic feature.

[0021] The generation of the segmentation image of the liver region described in step (2.3) is specifically as follows: the semantic features obtained by fusing the high-level semantic features of the abdominal CT image are subjected to upsampling operation and Sigmoid function calculation to obtain the segmentation image of the liver region.

[0022] The step (3) is specifically as follows:

[0023] The preprocessed images are sent in batches to the built multi-scale semantic feature network model to train the model. During network training, the learning rate update method is set to segmented update.

[0024] The step (4) specifically includes: performing Hu value truncation on the abdominal CT image to be segmented.

[0025] A computer storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned liver segmentation method based on a multi-scale semantic feature network.

[0026] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the liver segmentation method based on a multi-scale semantic feature network is implemented.

[0027] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0028] 1. The present invention adopts PLSANet to extract multi-scale semantic features, which can extract more feature information with very few parameters, solves the problem of large network model size, and improves the operation efficiency of the model;

[0029] 2. The present invention integrates multi-scale semantic feature information, which is beneficial to reducing the information loss generated during the convolution calculation process, making the network more sensitive to small target areas, and improving the segmentation accuracy of the liver area, especially when segmenting small-scale liver areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the SA module structure of the present invention;

[0031] Figure 2 It is a schematic structural diagram of the RFB module in the present invention;

[0032] Figure 3 Schematic diagram of the PLSANet network structure in the present invention;

[0033] Figure 4 Schematic diagram of the multi-scale semantic feature network structure in the present invention;

[0034] Figure 5is the original abdominal CT image in LiTS; Figure 5 (a) is the original abdominal CT image example 1 in LiTS, Figure 5 (b) is the original abdominal CT image example 2 in LiTS, Figure 5 (c) is the original abdominal CT image example 3 in LiTS;

[0035] Figure 6 is the pre-processed abdominal CT image; Figure 6 (a) is the abdominal CT image example 1 after preprocessing, Figure 6 (b) is the abdominal CT image example 2 after preprocessing. Figure 6 (c) is the abdominal CT image example 3 after preprocessing;

[0036] Figure 7 is the segmented image of the liver region; where Figure 7 (a) is the segmented image example 1 of the liver area. Figure 7 (b) is the segmented image example 2 of the liver area. Figure 7 (c) is the segmented image example 3 of the liver area;

[0037] Figure 8 It is an operational flow chart of the present invention. DETAILED DESCRIPTION

[0038] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0039] A liver segmentation method based on a multi-scale semantic feature network. The specific steps are as follows:

[0040] (1) Select existing abdominal CT public dataset images and preprocess the images in the dataset;

[0041] Specifically, we first obtained the LiTS abdominal CT image dataset, which includes 131 abdominal CT images with livers labeled by professional physicians. The Hu value range of abdominal CT images is generally [-1000, 1000]. Truncating the images within the Hu value range of [-200, 200] can enhance the image contrast, make the image clearer, and speed up network training. Then, we flip the images upside down and left to right to increase the number of images and expand the dataset, which is conducive to improving the robustness of the network and the segmentation accuracy.

[0042] (2) Build a network model based on the multi-scale semantic feature network structure;

[0043] The multi-scale semantic feature network structure includes two parts: feature extraction and feature restoration. The feature extraction part uses PLSANet to extract multi-scale semantic features of abdominal CT images. PLSANet uses PP-LCNet as the basic network structure, removes the last classification network part in PP-LCNet, and replaces the channel attention SE module with the compressed attention SA module. PLSANet can extract more semantic feature information with a very small number of parameters, avoid the problem of excessive network scale, and improve the learning speed and operation efficiency of the network; the feature restoration part uses the RFB module, SA attention module and upsampling operation to fuse the high-level semantic features of the abdominal CT image and generate a segmented image of the liver area. The fusion of multi-scale semantic features can reduce the loss of semantic feature information and improve the segmentation accuracy of the network;

[0044] In this embodiment, in the feature extraction part, the input image first passes through a normal 3*3 convolution with a stride of 2 and a 3*3 depth-separable convolution with a stride of 1 to obtain a low-level semantic feature L1, and then passes through a 3*3 depth-separable convolution with a stride of 2 and a 3*3 depth-separable convolution with a stride of 1 to obtain a low-level semantic feature L2, and then passes through a 3*3 depth-separable convolution with a stride of 2 and a 3*3 depth-separable convolution with a stride of 1 to obtain a high-level semantic feature H1, and then passes through a 3*3 depth-separable convolution with a stride of 2 and five 3*3 depth-separable convolutions with a stride of 1 to obtain a high-level semantic feature H2. The high-level semantic feature H2 is obtained by a 5*5 depth-wise separable convolution with a stride of 2, an SA module, a 5*5 depth-wise separable convolution with a stride of 1, and an SA module. In the feature restoration part, H3 is upsampled and added to H2 calculated by the RFB module, and then input into the SA module to obtain S2. S2 is upsampled and added to H1 calculated by the RFB module, and then input into the SA module to obtain S1. After upsampling and Sigmoid function calculation, S1 obtains the output image.

[0045] In addition, each convolution used in the PLSANet in the feature extraction part is followed by a batch normalization (BN) layer and a Hardswish activation function layer to standardize the data distribution of feature information in the network, which is beneficial to improving the learning efficiency of the network, reducing the impact of large data distribution gaps on the network, enhancing the network's generalization ability, and alleviating the gradient disappearance problem.

[0046] (3) The preprocessed images are fed into the built multi-scale semantic feature network model in batches to train the model;

[0047] In this embodiment, the preprocessed images are sent in batches to the built multi-scale semantic feature network model, and the parameters set are specifically: during network training, the network training batch processing number is set to 16, the number of iterations is 500, the optimizer is Adam, the initial learning rate is 0.0001, and the learning rate update method is segmented update. When the training iterations reach 30 times, 60 times, and 80 times, the learning rate decays to 1 / 3 of the original, and the loss function uses the binary cross entropy loss function.

[0048] After the training image is input into the network, the network outputs the segmented image, calculates the binary cross entropy loss value between the segmented image and the label image, and then passes the loss value to the Adam optimizer. The Adam optimizer adjusts the various parameters in the network structure according to the loss value.

[0049] (4) Preprocessing the abdominal CT image to be segmented;

[0050] The preprocessing of the abdominal CT image to be segmented is specifically: truncating the abdominal CT image to be segmented within the Hu value range of [-200, 200], and setting the image to be segmented to the same format as the image used during training, which is beneficial to the network's segmentation of the image and improves the accuracy of segmentation.

[0051] (5) The preprocessed abdominal CT image to be segmented is input into the trained multi-scale semantic feature network model, and the model outputs a segmented image of the liver area.

[0052] The hardware system used in this embodiment is Windows 11, the processor is Intel (R) Core (TM) i5-11400F CPU @ 2.60GHz 2.59GHz, the machine has 16G of RAM, the software operating environment is Pycharm, the programming language is Python 3.7, and the Pytorch framework is used to build the network.

[0053] The present invention uses PLSANet to extract multi-scale semantic features, which can extract more feature information with a very small number of parameters, so that the network will not be too large in scale, solving the problem of low operating efficiency; at the same time, it integrates multi-scale semantic feature information, which is conducive to reducing the information loss problem generated during the convolution calculation process, making the network more sensitive to small target areas, and solving the problem of low accuracy in small target segmentation. Through experiments, the time cost of training and prediction of the present invention has been reduced, and the segmentation accuracy has been improved, especially the accuracy of segmentation of small target areas has been improved. The segmented image generated by the network is as follows: Figure 7 shown.

Claims

1. A liver segmentation method based on a multi-scale semantic feature network, characterized in that: The following steps are involved: (1) Select existing abdominal CT public dataset images and preprocess the images in the dataset; (2) Build a network model based on the multi-scale semantic feature network structure; (3) The preprocessed images are fed into the built multi-scale semantic feature network model in batches to train the model; (4) Preprocessing the abdominal CT image to be segmented; (5) The preprocessed abdominal CT image to be segmented is input into the trained multi-scale semantic feature network model, and the model outputs a segmented image of the liver region; The step (2) is specifically as follows: (2.1) The multi-scale semantic feature network structure includes two parts: feature extraction and feature restoration; (2.2) The feature extraction part uses PLSANet to extract multi-scale semantic features of abdominal CT images. The PLSANet is specifically: PP-LCNet is used as the basic network structure, the last classification network part of PP-LCNet is removed, and the channel attention SE module is replaced by the compressed attention SA module to obtain PLSANet; (2.3) The feature restoration part uses the RFB module, the SA module and the upsampling operation to fuse the high-level semantic features of the abdominal CT image and generate a segmented image of the liver area; the high-level semantic features of the fused abdominal CT image are specifically as follows: there are five semantic features extracted by the feature extraction part, and the semantic features after each scale reduction are calculated in the order of low-level semantic feature 1, low-level semantic feature 2, high-level semantic feature 1, high-level semantic feature 2, and high-level semantic feature 3, which are denoted as L1, L2, H1, H2, and H3; among them, H3 is added to H2 calculated by the RFB module after upsampling, and then input into the SA module to obtain S2; S2 is added to H1 calculated by the RFB module after upsampling, and then input into the SA module to obtain S1, which is the fused semantic feature.

2. The liver segmentation method based on a multi-scale semantic feature network according to claim 1, characterized in that: The step (1) is specifically as follows: (1.1) Hu value truncation is performed on the image to enhance the contrast of the abdominal CT image; (1.2) Flip the image and expand the abdominal CT image dataset.

3. The liver segmentation method based on a multi-scale semantic feature network according to claim 1, characterized in that: The generation of the segmentation image of the liver region described in step (2.3) is specifically as follows: the semantic features obtained by fusing the high-level semantic features of the abdominal CT image are subjected to upsampling operation and Sigmoid function calculation to obtain the segmentation image of the liver region.

4. The liver segmentation method based on a multi-scale semantic feature network according to claim 1, characterized in that: The step (3) is specifically as follows: The preprocessed images are sent in batches to the built multi-scale semantic feature network model to train the model. During network training, the learning rate update method is set to segmented update.

5. The liver segmentation method based on a multi-scale semantic feature network according to claim 1, characterized in that: The step (4) specifically includes: performing Hu value truncation on the abdominal CT image to be segmented.

6. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the liver segmentation method based on a multi-scale semantic feature network as described in any one of claims 1 to 5 is implemented.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, it implements a liver segmentation method based on a multi-scale semantic feature network according to any one of claims 1 to 5.

Images