A three-channel cascade SEU-Nets liver tumor segmentation method

Through the three-channel cascade SEU-Nets method, the multimodal features of enhanced CT images are fusion and extraction of multimodal features of enhanced CT images by using the SE structure, 3D convolution and C-LSTM modules, and the problem of insufficient segmentation accuracy in the prior art is solved, and the problem of insufficient liver tumor segmentation accuracy is realized. High precision and efficient liver tumor segmentation.

CN113935951BActive Publication Date: 2024-11-01NANJING UNIV OF POSTS & TELECOMM
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202111066977.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-09-13
Publication Date
2024-11-01
Estimated Expiration
2041-09-13

AI Technical Summary

Technical Problem

The prior art is difficult to fully utilize the multimodal feature information of enhanced CT images in liver tumor segmentation, resulting in insufficient segmentation accuracy.

Method used

Using the three-channel cascaded SEU-Nets method, the traditional U-Net network is improved through the SE structure, combined with the 3D convolution module and the C-LSTM module, the feature maps of all slices are fused and the feature information between different slices is extracted, making full use of the enhanced type Multimodal features of CT images.

Benefits of technology

It significantly improves the segmentation accuracy and segmentation accuracy of liver tumors, optimizes the network training process, reduces parameters and operation complexity, speeds up convergence speed, and improves segmentation efficiency and reliability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113935951B_ABST
    Figure CN113935951B_ABST
Patent Text Reader

Abstract

The present invention discloses a three-channel cascade SEU-Nets liver tumor segmentation method. The first-level liver segmentation network segments a liver area from a venous phase CT image, and converts the liver area into a binary liver mask according to a preset threshold value; the arterial phase CT image, the venous phase CT image and the delayed phase CT image are subjected to mask operation with the liver mask to obtain an arterial phase liver region of interest, a venous phase liver region of interest and a delayed phase liver region of interest; the second-level tumor segmentation network processes the arterial phase liver region of interest, the venous phase liver region of interest and the delayed phase liver region of interest to obtain an arterial phase deep image feature map, a venous phase deep image feature map and a delayed phase deep image feature map; the arterial phase deep image feature map, the venous phase deep image feature map and the delayed phase deep image feature map are subjected to feature fusion by using feature cascade fusion, and a liver tumor segmentation probability map is output. The present invention improves the overall segmentation accuracy of liver tumors.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a three-channel cascade SEU-Nets liver tumor segmentation method, belonging to the technical field of medical image processing. Background Art

[0002] Cancer is a major disease that affects human health, and liver cancer has the largest number of patients among all cancers. The characteristics of the liver lesion area in the arterial phase, venous phase, and delayed phase of enhanced CT images are different. In enhanced CT images, the density characteristics of the liver in the arterial phase are obvious, and the contrast with the surrounding organs is large. Then the contrast difference reaches its peak in the venous phase, but the contrast with the organs decreases. Until the delayed phase, the density of the liver lesion area continues to decrease. Summary of the invention

[0003] The technical problem to be solved by the present invention is to overcome the defects of the prior art and provide a three-channel cascade SEU-Nets liver tumor segmentation method. The SE structure is used to improve the traditional U-Net network structure to improve the feature extraction ability of the convolution layer in the U-Net network, and then a 3D convolution module and a C-LSTM module are used to perform multimodal feature extraction. Specifically, the 3D convolution module is used to fuse the feature maps of three periods of the same slice, and then the C-LSTM is used to extract the feature information between different slices, so as to finally achieve the purpose of being able to fully utilize the multimodal feature information of enhanced CT.

[0004] To achieve the above object, the present invention provides a three-channel cascade SEU-Nets liver tumor segmentation method, comprising:

[0005] acquiring a venous phase CT image containing a liver tumor, an arterial phase CT image containing a liver tumor, and a delayed phase CT image containing a liver tumor;

[0006] The liver region is segmented from the venous phase CT image using the first-level liver segmentation network, and the liver region is converted into a binary liver mask according to a preset threshold;

[0007] Using the arterial phase CT image, the venous phase CT image and the delayed phase CT image to perform mask operations with the liver mask, respectively, to obtain the liver region of interest in the arterial phase, the liver region of interest in the venous phase and the liver region of interest in the delayed phase;

[0008] The second-level tumor segmentation network is used to process the liver region of interest in the arterial phase, the liver region of interest in the venous phase, and the liver region of interest in the delayed phase to obtain the deep image feature map in the arterial phase, the deep image feature map in the venous phase, and the deep image feature map in the delayed phase;

[0009] Feature cascade fusion is used to fuse the feature maps of the arterial phase deep image, the venous phase deep image and the delayed phase deep image, and the liver tumor segmentation probability map is output.

[0010] Preferably, the second-level tumor segmentation network includes an arterial phase segmentation network for processing an arterial phase liver region of interest, a venous phase segmentation network for processing a venous phase liver region of interest, and a delayed phase segmentation network for processing a delayed phase liver region of interest;

[0011] The first-level liver segmentation network, arterial phase segmentation network, venous phase segmentation network and delayed phase segmentation network are all SEU-Net networks. The SEU-Net network includes input port input, convolution layer Conv 1-1, SE layer 1-1, convolution layer Conv 1-2, SE layer 1-2, pooling layer Maxpool-1, convolution layer Conv 2-1, SE layer 2-1, convolution layer Conv 2-1, SE layer 2-2, pooling layer Maxpool-2, convolution layer Conv 3-1, SE layer 3-1, convolution layer Conv 3-2, SE layer 3-2, pooling layer Maxpool-3, convolution layer Conv 4-1, SE layer 4-1, convolution layer Conv 4-2, SE layer 4-2, pooling layer Maxpool-4, convolution layer Conv 5-1, SE layer 5-1, convolution layer Conv 5-2, SE layer 5-2, up convolution layer Upconv-1, convolution layer Conv 6_1, convolution layer Conv6-2, up convolution layer Upconv-2, convolution layer Conv 7-1, convolution layer Conv 7-2, up convolution layer Upconv-3, convolution layer Conv 8-1, convolution layer Conv 8-2, up convolution layer Upconv-4, convolution layer Conv 9-1, convolution layer Conv 9-2, convolution layer Conv 10-1 and output port output, input port input, convolution layer Conv 1-1, SE layer 1-1, convolution layer Conv1-2, SE layer 1-2, pooling layer Maxpool-1, convolution layer Conv 2-1, SE layer 2-1, convolution layer Conv 2-1, SE layer 2-2, pooling layer Maxpool-2, convolution layer Conv 3-1, SE layer 3-1, convolution layer Conv 3-2, SE layer 3-2, pooling layer Maxpool-3, convolution layer Conv 4-1, SE layer 4-1, convolution layer Conv 4-2, SE layer 4-2, pooling layer Maxpool-4, convolution layer Conv 5-1, SE layer 5-1, convolution layer Conv 5-2, SE layer 5-2, upper convolution layer Upconv-1, convolution layer Conv6_1, convolution layer Conv 6-2, upper convolution layer Upconv-2, convolution layer Conv 7-1, convolution layer Conv 7-2, upper convolution layer Upconv-3, convolution layer Conv 8-1, convolution layer Conv 8-2, upper convolution layer Upconv-4, convolution layer Conv 9-1, convolution layer Conv 9-2, convolution layer Conv 10-1 and output port output are connected in sequence.

[0012] Preferably, the convolutional layers in the first-level liver segmentation network, the arterial phase segmentation network, the venous phase segmentation network, and the delayed phase segmentation network all use a 3×3 convolution kernel, and the maximum pooling layers in the first-level liver segmentation network, the arterial phase segmentation network, the venous phase segmentation network, and the delayed phase segmentation network all use a 2×2 convolution kernel.

[0013] Preferably, the feature fusion of the arterial phase deep image feature map, the venous phase deep image feature map and the delayed phase deep image feature map is performed using feature cascade fusion, and includes:

[0014] The 3D convolution module is used to fuse the deep image feature map of the arterial phase, the deep image feature map of the venous phase, and the deep image feature map of the delayed phase from the same slice;

[0015] The C-LSTM module is used to extract the feature information of the continuously acquired feature maps.

[0016] Prioritize, the C-LSTM module is used to extract the feature information of the continuously acquired feature graph, including:

[0017] The first N feature maps only output the current feature map, where N is a positive integer;

[0018] After N feature maps, use several previous consecutive feature maps to predict the current feature map;

[0019] The characteristic information is the range of the lesion area in the liver region.

[0020] Preferably, the venous phase CT image, the arterial phase CT image, and the delayed phase CT image are set as follows:

[0021] The window level of the HU value is set to 20, and the window width is set to 200.

[0022] Preferably, the 3D convolution module uses a 3D convolution kernel of 3×3×3.

[0023] Preferably, training a first-level liver segmentation network and a second-level tumor segmentation network comprises:

[0024] Initialize the network parameters of the SEU-Net network;

[0025] Input the venous phase CT images of the acquired training samples into the first-level liver segmentation network to segment the liver area;

[0026] Using the Adam optimizer with self-adjusting learning rate, the network parameters of the SEU-Net network were iteratively updated based on the liver region. After several iterations, the network loss function was calculated based on the latest liver region.

[0027] Convergence is judged based on the network loss function. When the network loss function converges, the iteration ends and the trained first-level liver segmentation network and second-level tumor segmentation network are output;

[0028] Input the acquired test set into the trained first-level liver segmentation network and the second-level tumor segmentation network, and output a liver tumor segmentation probability map;

[0029] The liver tumor segmentation probability map is judged to be qualified based on the Dice similarity coefficient, VOE volume overlap error, RVD relative volume error, accuracy and recall rate. If it is unqualified, the first-level liver segmentation network and the second-level tumor segmentation network are retrained until they are qualified.

[0030] Preferably, the network loss function is the Dice loss function, and the formula is:

[0031]

[0032] Among them, S represents the segmentation result of the first-level liver segmentation network, and G represents our labeled gold standard.

[0033] The hyperparameters β1 and β2 are the power values ​​of the Adam optimizer, which are 0.9 and 0.999 respectively.

[0034] Preferably, the network parameters of the SEU-Net network include learning rate, descent rate and momentum parameters.

[0035] The beneficial effects achieved by the present invention are:

[0036] The present invention uses SE structure to improve the traditional U-Net network structure to improve the feature extraction ability of the convolution layer in the U-Net network, and then uses 3D convolution module and C-LSTM module to extract multimodal features. Specifically, the 3D convolution module is used to fuse the feature maps of three periods of the same slice, and then the C-LSTM is used to extract the feature information between different slices, so as to finally achieve the purpose of being able to fully utilize the multimodal feature information of enhanced CT; the overall segmentation accuracy of liver tumors is improved. The liver tumor segmentation method of the present invention has high liver tumor segmentation accuracy, accurate and reliable segmentation results, and the liver tumor segmentation effect is much higher than the segmentation effect of single-channel / single-phase CT images. On the other hand, the network training process is optimized, the network parameters are reduced, the complexity of segmentation operations is reduced, the convergence speed of the network is accelerated, the reliability and effectiveness of the network are improved, and the segmentation efficiency is improved and the segmentation accuracy is further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a flow chart in an embodiment of the present invention;

[0038] Figure 2 This is a schematic diagram of the SE structure in an embodiment of the present invention;

[0039] Figure 3 Schematic diagram of compression operation and excitation operation in the SE structure in an embodiment of the present invention;

[0040] Figure 4 : is a structural diagram of a basic U-Net structure used in an embodiment of the present invention;

[0041] Figure 5 Schematic diagram of the SEU-Net network structure in an embodiment of the present invention;

[0042] Figure 6 Schematic diagram of a three-channel cascaded SEU-Nets network structure in an embodiment of the present invention;

[0043] Figure 7 Schematic diagram of a multimodal SEU-Nets network structure in an embodiment of the present invention;

[0044] Figure 8 Flowchart of the 3D convolution module and the C-LSTM module in the embodiment of the present invention;

[0045] Fig. 9 A schematic diagram of an arterial phase CT image used in an experiment in an embodiment of the present invention;

[0046] Fig.10 A schematic diagram of a venous phase CT image used in an experiment in an embodiment of the present invention;

[0047] Fig.11 A schematic diagram of a delayed phase CT image used in an experiment in an embodiment of the present invention;

[0048] Fig.12 Schematic diagram of the CT image gold standard used in the experiment in the embodiment of the present invention;

[0049] Fig.13 It is a schematic diagram of the segmentation result of the comparison algorithm H-DenseUNet in the prior art;

[0050] Fig.14 A schematic diagram of the segmentation results of the comparison algorithm DC-CUNets in the prior art;

[0051] Fig.15 is a schematic diagram of the segmentation result of the second-level tumor segmentation network in an embodiment of the present invention;

[0052] Fig.16 Schematic diagram of the segmentation results of the second-level tumor segmentation network based on 3D convolution and C-LSTM in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The following embodiments are only used to more clearly illustrate the technical solutions of the present invention, and are not intended to limit the protection scope of the present invention.

[0054] The characteristics of the liver lesion area in the arterial phase, venous phase and delayed phase of the enhanced CT image are different. When radiologists make a diagnosis based on the enhanced CT images, they will also make judgments based on the differences in the characteristics of the three periods. In the enhanced CT images, the density characteristics of the liver in the arterial phase are obvious, and the difference contrast with the surrounding organs is large. Then the contrast difference reaches its peak in the venous phase, but the contrast with the organs decreases. Until the delayed phase, the density of the liver lesion area continues to decrease. Liver cancer is mainly manifested in the enhanced CT image as unclear edges of liver lesions, and the characteristic brightness of the liver shadow lesion area in the arterial phase is low, and the contrast with the surrounding organs is low. However, as the inflow time of the contrast agent changes, the difference between the lesion area and the liver substance in the venous phase and delayed phase becomes larger, which makes it easier for doctors to make a reasonable diagnosis. Through professional research, it was found that mixed liver cancer has different characteristics due to the different volume ratios of hepatocellular carcinoma and cholangiocarcinoma in the lesion area. Therefore, in daily diagnosis, doctors can quickly determine the type of liver cancer based on the characteristic differences of the liver lesion area on the enhanced CT images. Therefore, if we can combine the feature information between the three periods in the liver tumor segmentation network, the segmentation accuracy will be significantly improved.

[0055] This paper proposes a three-channel cascade SEU-Nets based on the characteristic differences of different types of liver tumors in three phases of enhanced CT. First, the characteristics of three-phase enhanced CT are introduced, and the effectiveness of using a three-channel network to improve the accuracy of liver tumor segmentation is demonstrated. In the training process, the superiority of the cascade network is emphasized. The specific flowchart is as follows Figure 1 shown.

[0056] The specific implementation steps are as follows:

[0057] Step 1, obtaining a venous phase CT image, an arterial phase CT image, and a delayed phase CT image containing a liver tumor;

[0058] Step 2: Segment the liver region from the venous phase CT image using the first-level liver segmentation network, and convert the liver region into a binary liver mask according to a preset threshold;

[0059] Step 3, using the arterial phase CT image, the venous phase CT image and the delayed phase CT image to perform mask operations with the liver mask respectively, to obtain the liver region of interest in the arterial phase, the liver region of interest in the venous phase and the liver region of interest in the delayed phase; Step 4, using the second-level tumor segmentation network to process the liver region of interest in the arterial phase, the liver region of interest in the venous phase and the liver region of interest in the delayed phase, to obtain the deep image features in the arterial phase, the deep image features in the venous phase and the deep image features in the delayed phase;

[0060] Step 5: Use feature cascade fusion to fuse the arterial phase deep image features, venous phase deep image features and delayed phase deep image features, and output a liver tumor segmentation probability map.

[0061] SE structure (Squeeze compression, Excitation excitation) such as Figure 2 and Figure 3 As shown, the basic U-Net network structure is as follows Figure 4 As shown, the SEU-Net network structure is as follows Figure 5 As shown, the three-channel cascade SEU-Nets network structure is shown in Figure 6 Table 1 shows the layout of each layer of the three-channel SEU-Nets segmentation network in detail.

[0062] Table 1 Layout of each layer of the three-channel SEU-Nets segmentation network

[0063]

[0064]

[0065] Based on the first-level liver segmentation network using the venous phase CT image to segment the liver area Mask, the second-level tumor segmentation network uses three channels to segment the liver tumors in the three periods of enhanced CT arterial phase, venous phase and delayed phase respectively. The network structure used in each channel is also the SEU-Net network. The encoder is used to extract the feature information of the three periods of enhanced CT, and the feature map is used to perform feature cascade addition on the feature channel to fully combine the feature information of the three periods. Finally, the feature map extracted by the encoder is gradually restored to the same size as the input image through the decoder.

[0066] like Figure 7As shown, the present invention uses 3D convolution and C-LSTM (Convolution Long short-term memory) to extract multimodal features, specifically using 3D convolution to fuse the feature maps of the same slice in three periods, and then using C-LSTM to extract feature information between different slices, ultimately achieving the purpose of being able to fully utilize the multimodal feature information of enhanced CT. In addition, the feature information between the three periods of the enhanced CT image is fully utilized, significantly improving the segmentation accuracy and segmentation accuracy of liver tumors.

[0067] like Figure 7 As shown, the three encoders of the second-level tumor segmentation network are used for different modal feature extraction processes respectively. The proposed 3D convolution module effectively fuses the feature information of three modalities in three periods of enhanced CT. Then, the C-LSTM module is used to better utilize space and sequence, and the spatial and sequential correlations between consecutive slices are modeled using the C-LSTM module, and multimodal fusion is jointly optimized in an end-to-end manner. The process of the network structure is shown below, which is specifically divided into three parts. The 3D convolution kernel of the size of 3×3×3 is used in the present invention.

[0068] (1) Encoder feature extraction process

[0069] The encoder is a downsampling feature extraction process of the second-level tumor segmentation network, in which the input image size is H×W, the convolution layer uses a convolution kernel of size 3×3, and the maximum pooling layer uses a convolution kernel of size 2×2 to generate a feature map. In the decoder, the deconvolution layer is mainly used to gradually restore the feature map to a size consistent with the input image. The slice CT images of the three periods are first passed through the first-level liver segmentation network to obtain the liver mask, and then enter the second-level tumor segmentation network for a downsampling process to obtain feature maps of the three periods, with a size of H×W×C. For the encoder, the present invention adopts the SEU-Net architecture, and each convolution layer uses a convolution kernel size of 3×3 to generate a set of feature maps, and then obtains the output.

[0070] (2) Multimodal feature extraction process

[0071] The feature maps extracted by the second-level tumor segmentation network are fused by using the feature cascade addition method, so as to better combine the feature information of the three periods of the enhanced CT image. In order to better combine the feature information of the three periods, a 3D convolution module is used to fuse the feature maps of the three periods. After the encoder extracts the features, the feature map of each period of the three periods becomes a feature space of H×W×C, and then the feature maps of the three periods are superimposed on the number of channels to become a feature space of C×3×H×W. The 3D convolution kernel size is 3×3×3. After passing through the 3D convolution module, the feature map is H×W×C. When the 3D kernel convolves across different stacks, it assigns different weights to each modality and sums the eigenvalues ​​in the output feature map. The proposed multimodal feature extraction fuses the feature map information of the three periods and extracts the correlation between different slices of the enhanced CT image, thereby achieving the process of multimodal feature extraction.

[0072] (3) Decoder restores feature map size process

[0073] The decoder continuously upsamples the feature map after multimodal feature extraction through deconvolution layers, slowly restoring the feature map to an image of the same size as the input image, while also achieving pixel-level segmentation.

[0074] The convolution layer, pooling layer and SE layer are existing technologies, and are not given as examples one by one in this embodiment.

[0075] The training set used in the present invention is 40 sets of CT images of patients, of which 35 sets of data are used as training sets and 5 sets are used as test sets. Each patient's CT image is divided into three periods, and each period has 50 slices. Each slice is feature extracted by the SEU-Nets encoder and downsampled. The three channels obtain a feature map of 3×32×32×512 in size. The slices at the same position in the three periods are superimposed with the number of channels, and the feature map is reshaped to 512×32×32×3, and then through the 3D convolution kernel, the size is 3×3×3, and the feature extraction of the slices at the same position in three periods is performed to obtain a feature map of 512×32×32×1. The same process is performed on 50 slices to obtain a 50×512×32×32 feature map, which means that each of the 50 slices corresponds to 512 feature maps of size 32×32. The present invention then uses C-LSTM to extract features between slices, and reshapes the features to 512×50×32×32, which means that 512 sequential feature maps of size 50×32×32 are used as input to C-LSTM, and the 4 previous feature maps are used to predict the current feature map. For the 1st, 2nd, and 3rd feature maps, only the current feature map is output, and the prediction is continued for the subsequent ones, so that the input and output dimensions can be guaranteed to be consistent, that is, the output is also 50×32×32, but the feature correlation between slices has been fully extracted. Finally, the feature map is reshaped to 50×512×32×32 and used as the input of the decoder.

[0076] Training the first-level liver segmentation network and the second-level tumor segmentation network includes:

[0077] Initialize the network parameters of the SEU-Net network;

[0078] Input the venous phase CT images of the acquired training samples into the first-level liver segmentation network to segment the liver area;

[0079] Using the Adam optimizer with self-adjusting learning rate, the network parameters of the SEU-Net network were iteratively updated based on the liver region. After several iterations, the network loss function was calculated based on the latest liver region.

[0080] Convergence is judged based on the network loss function. When the network loss function converges, the iteration ends and the trained first-level liver segmentation network and second-level tumor segmentation network are output;

[0081] Input the acquired test set into the trained first-level liver segmentation network and the second-level tumor segmentation network, and output a liver tumor segmentation probability map;

[0082] The liver tumor segmentation probability map is judged to be qualified based on the Dice similarity coefficient, VOE volume overlap error, RVD relative volume error, accuracy and recall rate. If it is unqualified, the first-level liver segmentation network and the second-level tumor segmentation network are retrained until they are qualified.

[0083] The calculation process of the Dice similarity coefficient, VOE volume overlap error, RVD relative volume error, accuracy and recall rate and the adopted calculation formulas belong to the prior art and will not be given in detail in the present invention.

[0084] The network parameters of the SEU-Net network include learning rate, descent rate, momentum parameter and Batch_size. BatchSize definition: the number of samples selected for one training, which is pre-set.

[0085] The present invention sets the window level of the HU value of the CT image to 20 and the window width to 200. Therefore, when processing the original CT image, the present invention uses each pixel in the CT image as a basic unit and subtracts the average value of the image. The new CT image thus obtained is the CT image after the normalization operation of the present invention. Normalizing the CT image has the following advantages. The batch normalization layer is used in many network structures. Its advantage is that it can preprocess the CT image and normalize the values ​​of the entire CT image. It can also accelerate the convergence speed of the loss function in the network model training. The liver and liver tumor segmentation models all use a linear normalization preprocessing method, where Y represents the normalized image, X is the current pixel value, X_min is the minimum value of all pixels in the image, and X_max is the maximum value of all pixels in the image, as shown in the following formula.

[0086]

[0087] During training, the Adam optimizer is used first, which can adjust the learning rate size autonomously. It has the advantage of fast convergence under unstable objective functions, and can adaptively reduce the learning rate, making the convergence faster and preventing overfitting of parameters. The hyperparameters β1 and β2 are the power values ​​of the Adam optimizer, which are 0.9 and 0.999 respectively. The specific network parameter settings are shown in Table 2.

[0088] Table 2 Network model parameter settings

[0089] Learning Rate 1e-6 Descent speed 1e-3 Momentum parameter 0.9 Batch_size 5

[0090] The Dice loss function used in the present invention is specifically expressed as follows:

[0091]

[0092] Among them, S represents the segmentation result of our network model, and G represents our labeled gold standard.

[0093] The implementation environment of the present invention is TensorFlow and Keras. The specific experimental environment is shown in Table 3.

[0094] Table 3 Experimental hardware and software simulation environment

[0095]

[0096]

[0097] The experiment of the present invention mainly evaluates the superior performance of our algorithm from five indicators. In the field of liver tumor segmentation, the most commonly used indicator for evaluating the superiority of the algorithm is the Dice similarity coefficient. The Dice similarity coefficient mainly represents the similarity between the results of the liver tumor segmentation network and the labeled gold standard pixels. In addition, there are some commonly used indicators to evaluate the superiority of the model from different aspects. These indicators are commonly used in the field of liver tumor segmentation and can effectively and accurately illustrate the superior performance of our algorithm. Among them, our S represents the liver tumor segmentation map of our experiment, and G represents the labeled gold standard. The present invention mainly introduces five commonly used indicators and the corresponding calculation formulas.

[0098] (1) Dice similarity coefficient. The definition is shown in formula (3).

[0099]

[0100] The Dice coefficient refers to the degree of overlap between the segmentation result and the gold standard. The larger the Dice coefficient value, the greater the degree of overlap.

[0101] (2) VOE volume overlap error. It is defined as shown in formula (4).

[0102]

[0103] This indicator indicates the error between the segmentation result and the corresponding annotated volume. The larger the VOE value, the more consistent our segmentation result is with the annotated image, and the better the segmentation effect.

[0104] (3) RVD relative volume error. The definition is shown in formula (5).

[0105]

[0106] Intuitively speaking, the most important function of this indicator is to explain whether the segmentation network causes under-segmentation or over-segmentation of liver tumors.

[0107] (4) Precision. The definition is as shown in formula (6).

[0108]

[0109] (5) Recall. The definition is as shown in formula (7).

[0110]

[0111] The main comparison algorithms of the experiment of the present invention are H-DenseUNet, which has achieved excellent results in the liver tumor segmentation competition, and DC-CUNets, the previous research results of this research group. Compared with the DC-CUNets liver tumor segmentation algorithm, the segmentation algorithm of the SEU-Nets of the present invention improves the liver tumor segmentation accuracy by 1.82%, and the edge segmentation effect is also very good, with high accuracy. The superiority and accuracy of the SEU-Nets liver tumor segmentation algorithm proposed in the present invention are proved by experiments. The segmentation results of several comparison algorithms are shown in Table 3. The Dice coefficient of the liver segmentation of the multimodal SEU-Nets proposed in the present invention is 95.87%, the accuracy is 93.86%, and the Dice coefficient of the liver tumor is 93.19%. It can be seen from the table that the multimodal SEU-Nets liver tumor segmentation algorithm is 3.07% higher than the proposed SEU-Nets algorithm in tumor segmentation accuracy, which means that the multimodal feature extraction module adding 3D convolution and C-LSTM greatly improves the accuracy of liver tumor segmentation. In other indicators, the accuracy rate is also improved by 4.17% compared with SEU-Nets. However, due to the multimodal feature extraction process, too many operations are performed on the intermediate feature map, which will reduce the segmentation speed of each slice. However, the experimental results still prove that the multimodal SEU-Nets algorithm proposed in this invention can improve the segmentation accuracy of liver tumors.

[0112] Table 3 Comparison of liver tumor segmentation results

[0113]

[0114] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A three-channel cascade SEU-Nets liver tumor segmentation method, characterized in that: include: acquiring a venous phase CT image containing a liver tumor, an arterial phase CT image containing a liver tumor, and a delayed phase CT image containing a liver tumor; The liver region is segmented from the venous phase CT image using the first-level liver segmentation network, and the liver region is converted into a binary liver mask according to a preset threshold; Using the arterial phase CT image, the venous phase CT image and the delayed phase CT image to perform mask operations with the liver mask, respectively, to obtain the liver region of interest in the arterial phase, the liver region of interest in the venous phase and the liver region of interest in the delayed phase; The second-level tumor segmentation network is used to process the liver region of interest in the arterial phase, the liver region of interest in the venous phase, and the liver region of interest in the delayed phase to obtain the deep image feature map in the arterial phase, the deep image feature map in the venous phase, and the deep image feature map in the delayed phase; The feature cascade fusion is used to fuse the feature map of the deep image in the arterial phase, the feature map of the deep image in the venous phase and the feature map of the deep image in the delayed phase, and the liver tumor segmentation probability map is output; The second-level tumor segmentation network includes an arterial phase segmentation network for processing an arterial phase liver region of interest, a venous phase segmentation network for processing a venous phase liver region of interest, and a delayed phase segmentation network for processing a delayed phase liver region of interest; The first-level liver segmentation network, arterial phase segmentation network, venous phase segmentation network and delayed phase segmentation network are all SEU-Net networks. The SEU-Net network includes input port input, convolution layer Conv 1-1, SE layer 1-1, convolution layer Conv 1-2, SE layer 1-2, pooling layer Maxpool-1, convolution layer Conv 2-1, SE layer 2-1, convolution layer Conv 2-1, SE layer 2-2, pooling layer Maxpool-2, convolution layer Conv 3-1, SE layer 3-1, convolution layer Conv 3-2, SE layer 3-2, pooling layer Maxpool-3, convolution layer Conv 4-1, SE layer 4-1, convolution layer Conv 4-2, SE layer 4-2, pooling layer Maxpool-4, convolution layer Conv 5-1, SE layer 5-1, convolution layer Conv 5-2, SE layer 5-2, up convolution layer Upconv-1, convolution layer Conv 6_1, convolution layer Conv 6-2, upper convolution layer Upconv-2, convolution layer Conv 7-1, convolution layer Conv 7-2, upper convolution layer Upconv-3, convolution layer Conv 8-1, convolution layer Conv 8-2, upper convolution layer Upconv-4, convolution layer Conv 9-1, convolution layer Conv 9-2, convolution layer Conv 10-1 and output port output, input port input, convolution layer Conv 1-1, SE layer 1-1, convolution layer Conv 1-2, SE layer 1-2, pooling layer Maxpool-1, convolution layer Conv 2-1, SE layer 2-1, convolution layer Conv 2-1, SE layer 2-2, pooling layer Maxpool-2, convolution layer Conv 3-1, SE layer 3-1, convolution layer Conv 3-2, SE layer 3-2, pooling layer Maxpool-3, convolution layer Conv 4-1, SE layer 4-1, convolution layer Conv 4-2, SE layer 4-2, pooling layer Maxpool-4, convolution layer Conv 5-1, SE layer 5-1, convolution layer Conv 5-2, SE layer 5-2, upper convolution layer Upconv-1, convolution layer Conv 6_1, convolution layer Conv 6-2, upper convolution layer Upconv-2, convolution layer Conv 7-1, convolution layer Conv 7-2, upper convolution layer Upconv-3, convolution layer Conv 8-1, convolution layer Conv 8-2, upper convolution layer Upconv-4, convolution layer Conv 9-1, convolution layer Conv 9-2, convolution layer Conv 10-1 and output port output are connected in sequence.

2. A three-channel cascade SEU-Nets liver tumor segmentation method according to claim 1, characterized in that: The convolutional layers in the first-level liver segmentation network, arterial phase segmentation network, venous phase segmentation network and delayed phase segmentation network all use 3×3 convolution kernels, and the maximum pooling layers in the first-level liver segmentation network, arterial phase segmentation network, venous phase segmentation network and delayed phase segmentation network all use 2×2 convolution kernels.

3. A three-channel cascade SEU-Nets liver tumor segmentation method according to claim 1, characterized in that: The feature cascade fusion is used to perform feature fusion on the feature map of the deep image in the arterial phase, the feature map of the deep image in the venous phase and the feature map of the deep image in the delayed phase, and includes: The 3D convolution module is used to fuse the deep image feature map of the arterial phase, the deep image feature map of the venous phase, and the deep image feature map of the delayed phase from the same slice; The C-LSTM module is used to extract the feature information of the continuously acquired feature maps.

4. A three-channel cascade SEU-Nets liver tumor segmentation method according to claim 3, characterized in that: The C-LSTM module is used to extract the feature information of the continuously acquired feature graphs, including: The first N feature maps only output the current feature map, where N is a positive integer; After N feature maps, use several previous consecutive feature maps to predict the current feature map; The characteristic information is the range of the lesion area in the liver region.

5. The three-channel cascade SEU-Nets liver tumor segmentation method according to claim 1, characterized in that: The settings of venous phase CT images, arterial phase CT images, and delayed phase CT images are as follows: The window level of the HU value is set to 20, and the window width is set to 200.

6. A three-channel cascade SEU-Nets liver tumor segmentation method according to claim 3, characterized in that: The 3D convolution module uses a 3×3×3 3D convolution kernel.

7. The three-channel cascade SEU-Nets liver tumor segmentation method according to claim 1, characterized in that: Training the first-level liver segmentation network and the second-level tumor segmentation network includes: Initialize the network parameters of the SEU-Net network; Input the venous phase CT images of the acquired training samples into the first-level liver segmentation network to segment the liver area; Using the Adam optimizer with self-adjusting learning rate, the network parameters of the SEU-Net network were iteratively updated based on the liver region. After several iterations, the network loss function was calculated based on the latest liver region. Convergence is judged based on the network loss function. When the network loss function converges, the iteration ends and the trained first-level liver segmentation network and second-level tumor segmentation network are output; Input the acquired test set into the trained first-level liver segmentation network and the second-level tumor segmentation network, and output a liver tumor segmentation probability map; The liver tumor segmentation probability map is judged to be qualified based on the Dice similarity coefficient, VOE volume overlap error, RVD relative volume error, accuracy and recall rate. If it is unqualified, the first-level liver segmentation network and the second-level tumor segmentation network are retrained until they are qualified.

8. A three-channel cascade SEU-Nets liver tumor segmentation method according to claim 7, characterized in that: The network loss function is the Dice loss function, and the formula is: (2), Among them, S represents the segmentation result of the first-level liver segmentation network, and G represents our labeled gold standard; The hyperparameters β1 and β2 are the power values ​​of the Adam optimizer, which are 0.9 and 0.999 respectively.

9. The three-channel cascade SEU-Nets liver tumor segmentation method according to claim 1, characterized in that: The network parameters of the SEU-Net network include learning rate, descent rate and momentum parameters.

Citation Information

Patent Citations

  • Liver tumor segmentation method and device based on multi-stage CT image guidance

    CN109961443A

  • Tumor region segmentation method and system for liver CT image based on cascaded full convolutional network

    CN110599500A

  • Multi-scale DC-CUNets liver tumor segmentation method based on bottleneck structure

    CN112712532A

  • Multi-temporal CT image liver tumor segmentation method based on semantic migration

    CN112785605A