A deep learning-based liver tumor segmentation method for CT sequence images

Abstract

The present invention discloses a method for liver tumor segmentation in CT sequence images based on deep learning. The method mainly comprises: (1) constructing a U-shaped 2D convolutional network based on dilated spatial pyramid convolution, and using the network to perform two-dimensional slice segmentation of CT sequence images from three viewing directions: sagittal, coronal, and transverse; (2) using a lightweight 3D convolutional network to fuse the segmentation results obtained from different viewing directions to obtain the probability that each pixel in the CT sequence belongs to the target and the three-dimensional segmentation results of the CT sequence liver tumor; (3) constructing a graph cut energy function based on the obtained probabilities and three-dimensional segmentation results to further optimize the segmentation results. By combining 2D and 3D convolutional networks and a graph cut algorithm, the present invention can effectively extract three-dimensional spatial information from CT sequences while using a lightweight network, thereby improving the accuracy of liver tumor segmentation.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a method for segmenting liver tumors in CT sequence images based on deep learning. Background Art

[0002] Liver cancer has become one of the most serious cancers threatening the health of the Chinese people. According to data from the National Cancer Center, the incidence of liver cancer in my country ranks fifth among all malignant tumors, and its mortality rate ranks second. Early diagnosis, staging, and treatment of liver cancer play a vital role in improving patients' survival rates and quality of life. Computed tomography (CT) imaging has the advantages of high resolution, fast speed, and low cost, and is widely used in the clinical diagnosis of liver tumors. Accurate segmentation of liver tumor regions in CT sequence images is an important basis for computer-assisted diagnosis and treatment of liver cancer. Oncologists can obtain detailed information such as tumor size, shape, and location based on the segmentation results to formulate appropriate treatment plans for patients. Currently, liver tumor segmentation is usually performed manually by professionals in clinical practice. Due to the large number of slices generated by CT scans, manual segmentation is time-consuming and labor-intensive, and the quality of the segmentation results is highly dependent on the operator's experience. Therefore, there is an urgent need to develop an automatic and accurate segmentation method for liver tumor regions in abdominal CT sequence images.

[0003] Liver tumors in CT images are characterized by low contrast, blurred boundaries, and irregular shapes. Traditional segmentation methods, such as thresholding, region growing, level set, and graph cuts, typically require manual interaction, making automatic tumor segmentation difficult. In recent years, with the rapid development and widespread application of artificial intelligence technology in the field of intelligent healthcare, experts and scholars at home and abroad have proposed a number of liver tumor segmentation methods based on deep learning. These methods can be mainly divided into 2D and 3D convolution-based methods. 2D convolution-based segmentation networks do not consider inter-slice information in CT sequence images, cannot effectively extract three-dimensional spatial features, and have limited segmentation accuracy. 3D convolution-based segmentation networks, on the other hand, have a large number of parameters. Due to limited computing resources, existing 3D convolution-based segmentation networks are generally unable to directly process the original CT sequence. They need to pre-downsample or truncate the CT sequence into small-sized three-dimensional data, which will result in loss of image detail information and reduced network segmentation accuracy. Summary of the Invention

[0004] To address the shortcomings and deficiencies of existing technologies, this paper combines a 2D deep convolutional network with a 3D lightweight network and a graph cut algorithm to provide a method for effectively segmenting liver tumor regions in CT image sequences. This method effectively addresses the inability of 2D convolution alone to extract 3D features and the high memory overhead of 3D convolution. By combining deep learning with the graph cut algorithm, it significantly improves tumor segmentation accuracy. The results of this invention can provide technical support and decision-making services for the clinical diagnosis and treatment of liver tumors and pathological research.

[0005] A method for liver tumor segmentation in CT sequence images based on deep learning, comprising the following steps:

[0006] (1) Establish original training datasets A and B containing original CT sequence images and manual segmentation results of the liver region;

[0007] (2) Construct a U-shaped 2D convolutional network based on dilated spatial pyramid convolution, denoted as ASPP-UNet, which specifically includes:

[0008] (2-a) A U-shaped network is used as the backbone network, which includes three encoding layers, two skip connections, a dilated spatial pyramid convolution layer, three decoding layers and a 1×1 convolution layer, wherein: the output of the first encoding layer is not only used as the input of the second encoding layer, but also connected to the second decoding layer through the first skip connection as the input of the decoding layer; the output of the second encoding layer is not only used as the input of the third encoding layer, but also connected to the first decoding layer through the second skip connection as the input of the decoding layer; the output of the third encoding layer is used as the input of the dilated spatial pyramid convolution layer, and the output of the dilated spatial pyramid convolution layer is used as the input of the first decoding layer; in addition, the output of the previous decoding layer is used as the input of the next decoding layer; in order to obtain the segmentation result, the last decoding layer is connected to a 1×1 convolution layer, wherein the output of the last decoding layer is used as the input of the 1×1 convolution layer, and the output of the 1×1 convolution layer is the probability that each pixel belongs to the target. By introducing a threshold ε1, the segmentation result can be obtained; the ε1 is preferably a constant of 0.3 to 0.7;

[0009] (2-b) In the backbone network described in step (2-a), each encoding layer is composed of two 2D convolutional modules connected together, namely, 2D double convolutional modules, where each 2D convolutional module contains a 3×3 convolutional layer, a batch normalization layer, and a Relu activation layer; in order to downsample the image, a 2×2 maximum pooling layer is added at the end of the 2D double convolutional module in the second and third encoding layers;

[0010] (2-c) In the backbone network described in step (2-a), the dilated spatial pyramid convolution layer specifically includes: using n different sampling radii {r iThe 3×3 convolution kernels of |i=1,2,...,n} perform dilated convolution on the input feature maps respectively, and concatenate the dilated convolution results as the output of the dilated spatial pyramid convolution layer, where n is a natural number greater than 1. In order to expand the receptive field of the convolution kernel and obtain multi-scale context information, the sampling radius is set to r i =k×i+1, wherein k is a natural number greater than 0; n is preferably a natural number from 2 to 10, and k is preferably a natural number from 1 to 8;

[0011] (2-d) In the backbone network described in step (2-a), the first and second decoding layers are each composed of a 2D double convolution module described in step (2-b), a 2×2 deconvolution layer, and a splicing operation, and the third decoding layer is composed of only one 2D double convolution module described in step (2-b), wherein: the input of the 2D double convolution module in the first decoding layer is the output of the void spatial pyramid convolution layer, and thereafter, the input of the 2D double convolution module in the next decoding layer is the output of the previous decoding layer; the splicing operation in the first decoding layer is used to splice the deconvolution result in the decoding layer and the output of the second encoding layer, and the splicing result is used as the output of the decoding layer; the splicing operation in the second decoding layer is used to splice the deconvolution result in the decoding layer and the output of the first encoding layer, and the splicing result is used as the output of the decoding layer;

[0012] (3) Construct a lightweight 3D convolutional network, denoted as LW-3DNet, which involves three inputs and one output. The specific structure of the network includes: first, three 3D double convolution modules are used to convolve the three inputs respectively, then the convolution results are spliced ​​by splicing operation, and a 3D double convolution module is used to convolve the spliced ​​results to obtain a feature map F, finally, a 1×1×1 convolution layer is used to convolve the feature map F, and the output of the 1×1×1 convolution layer is the probability that each voxel belongs to the target. By introducing a threshold ε2, a three-dimensional segmentation result can be obtained, and the ε2 is preferably a constant of 0.3 to 0.7; the 3D double convolution modules described in step (3) are composed of two 3D convolution modules connected together, where each 3D convolution module contains a convolution layer of size 3×3×3, a batch normalization layer and a Relu activation layer;

[0013] (4) Using ASPP-UNet, multiple network models are trained to segment 2D slices in different viewing directions. The specific steps include: for each CT sequence in the training dataset A, first reconstruct the 2D slices from the sagittal, coronal and transverse viewing directions respectively, and obtain the 2D slices in different viewing directions, which are recorded as and Then, the 2D slices in the sagittal view direction obtained in the training dataset are The corresponding two-dimensional slice manual segmentation results are input into the ASPP-UNet network for training to obtain the network model ASPP-UNet that can be used to segment sagittal two-dimensional slices. X , the two-dimensional slices of the coronal view direction obtained in the training dataset The corresponding two-dimensional slice manual segmentation results are input into the ASPP-UNet network for training to obtain the network model ASPP-UNet that can be used to segment the coronal two-dimensional slices. Y , the two-dimensional slices of the cross-sectional view direction obtained in the training dataset The corresponding two-dimensional slice manual segmentation results are input into the ASPP-UNet network for training to obtain the network model ASPP-UNet that can be used to segment the cross-sectional two-dimensional slices. Z ; Train to obtain the network model ASPP-UNet X 、ASPP-UNet Y and ASPP-UNet Z When , the loss function is preferably based on a hybrid loss function of cross entropy and Dice, which is defined as follows:

[0014] l=l c +η·l d

[0015]

[0016]

[0017] Among them, l c and l d Represent cross entropy and Dice loss respectively, η is a weight parameter, preferably a constant of 0.5 to 2, represents the expert manual segmentation result of the w-th pixel in the CT image, where the background is marked as 0 and the target is marked as 1. represents the probability that the network model predicts that the w-th pixel belongs to the target, and N is the number of pixels in the CT image;

[0018] (5) Using LW-3DNet, a network model is trained to fuse segmentation results from different view directions, including:

[0019] (5-a) Construct the training dataset C of the LW-3DNet network, specifically including: First, for each CT sequence in the original training dataset B, reconstruct the two-dimensional slices from the three viewing directions of sagittal, coronal and transverse planes, and obtain the two-dimensional slices in different viewing directions, which are respectively recorded as and Then, respectively and Input to the trained network model ASPP-UNet X 、ASPP-UNet Y and ASPP-UNet Z The test is carried out in , and the two-dimensional slice segmentation results S in different view directions are obtained X 、S Y and S Z ; Finally, the network predicted S X 、S Y and S Z As the input of LW-3DNet network training, the 3D manual segmentation results of CT sequences in training dataset B are used as labels to construct the training dataset C of LW-3DNet network;

[0020] (5-b) Input the training data set C into the LW-3Dnet network for training. The loss function is preferably the Dice loss function, and the trained network model LW-3DNet is obtained. F ;

[0021] (6) For the CT sequence T to be tested, firstly, two-dimensional slice reconstruction is performed from the three viewing directions of sagittal, coronal and transverse planes to obtain two-dimensional slices of the CT sequence image in different viewing directions, which are respectively denoted as T X 、T Y and T Z ; Then, T X 、T Y and T Z Input to the trained network model ASPP-UNet X 、ASPP-UNet Y and ASPP-UNet Z The test is carried out in the , and the 2D slice segmentation results F in different view directions are obtained. X 、F Y and F Z ; Finally, F X 、F Y and F Z Enter LW-3DNet F The network model is tested to obtain the probability P of each voxel in the CT sequence T belonging to the target obj , and the liver tumor 3D segmentation result S obj ;

[0022] (7) Constructing a graph cut energy function to further optimize the segmentation results and obtain accurate liver tumor segmentation results, including:

[0023] (7-a) For the CT sequence T to be detected, the following graph cut energy function is constructed:

[0024]

[0025] Where v represents the voxel in the CT sequence image T, L v represents the neighborhood voxel set of voxel v, the L v It is preferably a neighborhood voxel set with the current voxel v as the center and a size of 3×3×3, and u represents the neighborhood voxel set L v In the voxel, R(v) and B(v,u) represent the regional penalty term and boundary penalty term in the graph cut energy function respectively. α is a weight parameter with a constant value between 0 and 1. The regional penalty term is mainly used to penalize the label of “target” or “background” for a single voxel. The calculation formula is as follows:

[0026] R(v)=β·R1(v)+(1-β)·R2(v)

[0027]

[0028]

[0029] Among them, P obj (v) represents the probability that voxel v in the CT sequence T belongs to a liver tumor, which is obtained by the method described in step (6). The greater the probability that voxel v belongs to the "target" or "liver tumor", the smaller the penalty for assigning the voxel as "target", and the greater the penalty for assigning the voxel as "background". β is a weight parameter, which is a constant between 0 and 1. g v represents the gray value of voxel v in CT sequence T, and γ represent the segmentation results S obtained according to the method described in step (6) obj The calculated grayscale mean and standard deviation of the liver tumor area are calculated as follows:

[0030]

[0031]

[0032] Among them, R o Represents the segmentation result S obj The liver tumor region voxel set in m represents the voxel set R o The voxels in g m represents the grayscale value of voxel m, N o Represents the voxel set R o The boundary term B(v,u) is used to penalize the grayscale inconsistency between adjacent voxels, and the calculation formula is as follows:

[0033]

[0034] in

[0035]

[0036] D(v,u) represents the Euclidean distance between voxels v and u, N1 represents the number of voxels in the CT sequence T, g v and g u Represents the grayscale value of voxel v and u respectively, |g v -g u | means to find g v With g u The absolute value of the difference;

[0037] (7-b) The maximum flow / minimum cut algorithm is used to minimize the energy function E(T) to obtain accurate liver tumor segmentation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 ASPP-UNet network structure diagram

[0039] Figure 2 Schematic diagram of the dilated spatial pyramid convolutional layer structure

[0040] Figure 3 LW-3DNet network structure diagram

[0041] Figure 4 Schematic diagram of the liver tumor segmentation process of CT sequence images according to an embodiment of the present invention

[0042] Figure 5 Examples of liver tumor segmentation results according to an embodiment of the present invention, where 5(a) to (c) are examples of 2D slice results of 3D segmentation results in the sagittal, coronal, and transverse planes, respectively. DETAILED DESCRIPTION

[0043] A deep learning-based liver tumor segmentation method for CT sequence images. The specific implementation steps are as follows:

[0044] (1) Randomly select 100 abdominal CT original sequence images and their corresponding liver tumor region manual segmentation results from the LiTS public database. 50 cases, i.e., 50 abdominal CT original sequence images and their corresponding liver tumor region manual segmentation results, are selected as training dataset A, and the remaining 50 cases are selected as training dataset B. In the manual segmentation results, the liver tumor region (i.e., the target region) is marked as "1" and the background region is marked as "0".

[0045] (2) Construct a U-shaped 2D convolutional network based on dilated spatial pyramid convolution, denoted as ASPP-UNet, with the following structure: Figure 1 As shown, specifically including:

[0046] (2-a) A U-shaped network is used as the backbone network, which includes three encoding layers, two skip connections, a dilated spatial pyramid convolution layer, three decoding layers and a 1×1 convolution layer, wherein: the output of the first encoding layer is not only used as the input of the second encoding layer, but also connected to the second decoding layer through the first skip connection as the input of the decoding layer; the output of the second encoding layer is not only used as the input of the third encoding layer, but also connected to the first decoding layer through the second skip connection as the input of the decoding layer; the output of the third encoding layer is used as the input of the dilated spatial pyramid convolution layer, and the output of the dilated spatial pyramid convolution layer is used as the input of the first decoding layer; in addition, the output of the previous decoding layer is used as the input of the next decoding layer; in order to obtain the segmentation result, the last decoding layer is connected to a 1×1 convolution layer, wherein the output of the last decoding layer is used as the input of the 1×1 convolution layer, and the output of the 1×1 convolution layer is the probability that each pixel belongs to the target. By introducing the threshold ε1, the segmentation result can be obtained; in this embodiment, ε1=0.5 is preferred;

[0047] (2-b) In the backbone network described in step (2-a), each encoding layer is composed of two 2D convolutional modules connected together, namely, 2D double convolutional modules, where each 2D convolutional module contains a 3×3 convolutional layer, a batch normalization layer, and a Relu activation layer; in order to downsample the image, a 2×2 maximum pooling layer is added at the end of the 2D double convolutional module in the second and third encoding layers;

[0048] (2-c) In the backbone network described in step (2-a), the dilated spatial pyramid convolution layer has the following structure: Figure 2 As shown, specifically including: using n different sampling radii {r i The 3×3 convolution kernels of |i=1,2,...,n} perform dilated convolution on the input feature maps respectively, and concatenate the dilated convolution results as the output of the dilated spatial pyramid convolution layer, where n is a natural number greater than 1. In order to expand the receptive field of the convolution kernel and obtain multi-scale context information, the sampling radius is set to r i =k×i+1, where k is a natural number greater than 0; in this embodiment, preferably n=5, k=2;

[0049] (2-d) In the backbone network described in step (2-a), the first and second decoding layers are each composed of a 2D double convolution module described in step (2-b), a 2×2 deconvolution layer, and a splicing operation, and the third decoding layer is composed of only one 2D double convolution module described in step (2-b), wherein: the input of the 2D double convolution module in the first decoding layer is the output of the void spatial pyramid convolution layer, and thereafter, the input of the 2D double convolution module in the next decoding layer is the output of the previous decoding layer; the splicing operation in the first decoding layer is used to splice the deconvolution result in the decoding layer and the output of the second encoding layer, and the splicing result is used as the output of the decoding layer; the splicing operation in the second decoding layer is used to splice the deconvolution result in the decoding layer and the output of the first encoding layer, and the splicing result is used as the output of the decoding layer;

[0050] (3) Construct a lightweight 3D convolutional network, denoted as LW-3DNet, with the following structure: Figure 3 As shown, the network involves three inputs and one output. The specific structure of the network includes: first, three 3D double convolution modules are used to convolve the three inputs respectively, then the convolution results are spliced ​​by splicing operation, and a 3D double convolution module is used to convolve the spliced ​​results to obtain a feature map F, and finally, a 1×1×1 convolution layer is used to convolve the feature map F, and the output of the 1×1×1 convolution layer is the probability that each voxel belongs to the target. By introducing the threshold ε2, a three-dimensional segmentation result can be obtained; the 3D double convolution module described in step (3) is composed of two 3D convolution modules connected, where each 3D convolution module contains a convolution layer of size 3×3×3, a batch normalization layer and a Relu activation layer; in this embodiment, ε2=0.5 is preferred;

[0051] (4) Using ASPP-UNet, multiple network models are trained to segment 2D slices in different viewing directions. The specific steps include: for each CT sequence in the training dataset A, first reconstruct the 2D slices from the sagittal, coronal and transverse viewing directions respectively, and obtain the 2D slices in different viewing directions, which are recorded as and Then, the 2D slices in the sagittal view direction obtained in the training dataset are The corresponding two-dimensional slice manual segmentation results are input into the ASPP-UNet network for training to obtain the network model ASPP-UNet that can be used to segment sagittal two-dimensional slices. X , the two-dimensional slices of the coronal view direction obtained in the training dataset The corresponding two-dimensional slice manual segmentation results are input into the ASPP-UNet network for training to obtain the network model ASPP-UNet that can be used to segment the coronal two-dimensional slices. Y, the two-dimensional slices of the cross-sectional view direction obtained in the training dataset The corresponding two-dimensional slice manual segmentation results are input into the ASPP-UNet network for training to obtain the network model ASPP-UNet that can be used to segment the cross-sectional two-dimensional slices. Z ; Train to obtain the network model ASPP-UNet X 、ASPP-UNet Y and ASPP-UNet Z When , the loss function is preferably based on a hybrid loss function of cross entropy and Dice, which is defined as follows:

[0052] l=l c +η·l d

[0053]

[0054]

[0055] Among them, l c and l d They represent cross entropy and Dice loss respectively, η is a weight parameter, and in this embodiment, η=1 is preferred. represents the expert manual segmentation result of the w-th pixel in the CT image, where the background is marked as 0 and the target is marked as 1. represents the probability that the network model predicts that the w-th pixel belongs to the target, and N is the number of pixels in the CT image;

[0056] (5) Using LW-3DNet, a network model is trained to fuse segmentation results from different view directions, including:

[0057] (5-a) Construct the training dataset C of the LW-3DNet network, specifically including: First, for each CT sequence in the original training dataset B, reconstruct the two-dimensional slices from the three viewing directions of sagittal, coronal and transverse planes, and obtain the two-dimensional slices in different viewing directions, which are respectively recorded as and Then, respectively and Input to the trained network model ASPP-UNet X 、ASPP-UNet Y and ASPP-UNet Z The test is carried out in , and the two-dimensional slice segmentation results S in different view directions are obtained X 、S Y and S Z ; Finally, the network predicted S X 、S Y and S ZAs the input of LW-3DNet network training, the 3D manual segmentation results of CT sequences in training dataset B are used as labels to construct the training dataset C of LW-3DNet network;

[0058] (5-b) Input the training data set C into the LW-3Dnet network for training. The loss function is preferably the Dice loss function, and the trained network model LW-3DNet is obtained. F ;

[0059] (6) For the CT sequence T to be tested, firstly, two-dimensional slice reconstruction is performed from the three viewing directions of sagittal, coronal and transverse planes to obtain two-dimensional slices of the CT sequence image in different viewing directions, which are respectively denoted as T X 、T Y and T Z ; Then, T X 、T Y and T Z Input to the trained network model ASPP-UNet X 、ASPP-UNet Y and ASPP-UNet Z The test is carried out in the , and the 2D slice segmentation results F in different view directions are obtained. X 、F Y and F Z ; Finally, F X 、F Y and F Z Enter LW-3DNet F The network model is tested to obtain the probability P of each voxel in the CT sequence T belonging to the target obj , and the liver tumor 3D segmentation result S obj ;

[0060] (7) Constructing a graph cut energy function to further optimize the segmentation results and obtain accurate liver tumor segmentation results, including:

[0061] (7-a) For the CT sequence T to be detected, the following graph cut energy function is constructed:

[0062]

[0063] Where v represents the voxel in the CT sequence image T, L v represents the neighborhood voxel set of voxel v, and in this embodiment, L v is the neighborhood voxel set with the current voxel v as the center and a size of 3×3×3, that is, the 26-neighborhood of the three-dimensional voxel, and u represents the neighborhood voxel set L vThe voxels in the graph cut energy function are R(v) and B(v,u), respectively, and α is a weight parameter, which is a constant between 0 and 1. In this embodiment, α = 0.5 is preferred. The regional penalty term is mainly used to penalize a single voxel for assigning a label of "target" or "background". The probability P obtained by the method described in step (6) is obj And the segmentation result S obj The specific calculation formula is as follows:

[0064] R(v)=β·R1(v)+(1-β)·R2(v)

[0065]

[0066]

[0067] Among them, P obj (v) represents the probability that voxel v in the CT sequence T belongs to a liver tumor, which is obtained by the method described in step (6). The greater the probability that voxel v belongs to the "target" or "liver tumor", the smaller the penalty for assigning the voxel to the "target", and the greater the penalty for assigning the voxel to the "background". β is a weight parameter, which is a constant between 0 and 1. In this embodiment, β = 0.5 is preferred. v represents the gray value of voxel v in CT sequence T, and γ represent the segmentation results S obtained according to the method described in step (6) obj The calculated grayscale mean and standard deviation of the liver tumor area are calculated as follows:

[0068]

[0069]

[0070] Among them, R o Represents the segmentation result S obj The liver tumor region voxel set in m represents the voxel set R o The voxels in g m represents the grayscale value of voxel m, N o Represents the voxel set R o The boundary term B(v,u) is used to penalize the grayscale inconsistency between adjacent voxels, and the calculation formula is as follows:

[0071]

[0072] in

[0073]

[0074] D(v,u) represents the Euclidean distance between voxels v and u, N1 represents the number of voxels in the CT sequence T, g v and g u Represents the grayscale value of voxel v and u respectively, |g v -g u | means to find g v and g u The absolute value of the difference;

[0075] (7-b) The maximum flow / minimum cut algorithm is used to minimize the energy function E(T) to obtain accurate liver tumor segmentation results. Figure 4 Schematic diagram of the liver tumor segmentation process of CT sequence images according to an embodiment of the present invention.

[0076] Figure 5 Examples of 2D slice results of the 3D segmentation results of the liver tumor in the sagittal, coronal, and transverse view directions are given. The area indicated by the white closed curve is the liver tumor segmentation result obtained in this example. It can be seen that the liver tumor area in the image is effectively segmented.

[0077] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for liver tumor segmentation in CT sequence images based on deep learning, characterized in that: The following steps are involved: (1) Establish original training datasets A and B containing original CT sequence images and manual segmentation results of the liver region; (2) Construct a U-shaped 2D convolutional network based on dilated spatial pyramid convolution, denoted as ASPP-UNet, which specifically includes: (2-a) A U-shaped network is used as the backbone network, which consists of three encoding layers, two skip connections, a dilated spatial pyramid convolution layer, three decoding layers and a 1×1 convolution layer, wherein: the output of the first encoding layer is not only used as the input of the second encoding layer, but also connected to the second decoding layer through the first skip connection as the input of the decoding layer; the output of the second encoding layer is not only used as the input of the third encoding layer, but also connected to the first decoding layer through the second skip connection as the input of the decoding layer; the output of the third encoding layer is used as the input of the dilated spatial pyramid convolution layer, and the output of the dilated spatial pyramid convolution layer is used as the input of the first decoding layer; in addition, the output of the previous decoding layer is used as the input of the next decoding layer; in order to obtain the segmentation result, the last decoding layer is connected to a 1×1 convolution layer, where the output of the last decoding layer is used as the input of the 1×1 convolution layer, and the output of the 1×1 convolution layer is the probability that each pixel belongs to the target. By introducing the threshold ε1, the segmentation result can be obtained; (2-b) In the backbone network described in step (2-a), each encoding layer is composed of two 2D convolutional modules connected together, namely, 2D double convolutional modules, where each 2D convolutional module contains a 3×3 convolutional layer, a batch normalization layer, and a Relu activation layer; in order to downsample the image, a 2×2 maximum pooling layer is added at the end of the 2D double convolutional module in the second and third encoding layers; (2-c) In the backbone network described in step (2-a), the dilated spatial pyramid convolution layer specifically includes: using n different sampling radii {r i The 3×3 convolution kernels of |i=1,2,...,n} perform dilated convolution on the input feature maps respectively, and concatenate the dilated convolution results as the output of the dilated spatial pyramid convolution layer, where n is a natural number greater than 1. In order to expand the receptive field of the convolution kernel and obtain multi-scale context information, the sampling radius is set to r i =k×i+1, where k is a natural number greater than 0; (2-d) In the backbone network described in step (2-a), the first and second decoding layers are each composed of a 2D double convolution module described in step (2-b), a 2×2 deconvolution layer, and a splicing operation, and the third decoding layer is composed of only one 2D double convolution module described in step (2-b), wherein: the input of the 2D double convolution module in the first decoding layer is the output of the void spatial pyramid convolution layer, and thereafter, the input of the 2D double convolution module in the next decoding layer is the output of the previous decoding layer; the splicing operation in the first decoding layer is used to splice the deconvolution result in the decoding layer and the output of the second encoding layer, and the splicing result is used as the output of the decoding layer; the splicing operation in the second decoding layer is used to splice the deconvolution result in the decoding layer and the output of the first encoding layer, and the splicing result is used as the output of the decoding layer; (3) Construct a lightweight 3D convolutional network, denoted as LW-3DNet, which involves three inputs and one output. The specific structure of the network includes: first, three 3D double convolution modules are used to convolve the three inputs respectively, then the convolution results are spliced ​​by splicing operation, and a 3D double convolution module is used to convolve the spliced ​​results to obtain a feature map F, finally, a 1×1×1 convolution layer is used to convolve the feature map F, and the output of the 1×1×1 convolution layer is the probability that each voxel belongs to the target. By introducing the threshold ε2, the three-dimensional segmentation result can be obtained; the 3D double convolution modules described in step (3) are composed of two 3D convolution modules connected together, where each 3D convolution module contains a convolution layer of size 3×3×3, a batch normalization layer and a Relu activation layer; (4) Using ASPP-UNet, multiple network models are trained to segment 2D slices in different viewing directions. The specific steps include: for each CT sequence in the training dataset A, first reconstruct the 2D slices from the sagittal, coronal and transverse viewing directions respectively, and obtain the 2D slices in different viewing directions, which are recorded as and Then, the 2D slices in the sagittal view direction obtained in the training dataset are The corresponding two-dimensional slice manual segmentation results are input into the ASPP-UNet network for training to obtain the network model ASPP-UNet that can be used to segment sagittal two-dimensional slices. X , the two-dimensional slices of the coronal view direction obtained in the training dataset The corresponding two-dimensional slice manual segmentation results are input into the ASPP-UNet network for training to obtain the network model ASPP-UNet that can be used to segment the coronal two-dimensional slices. Y , the two-dimensional slices of the cross-sectional view direction obtained in the training dataset The corresponding two-dimensional slice manual segmentation results are input into the ASPP-UNet network for training to obtain the network model ASPP-UNet that can be used to segment the cross-sectional two-dimensional slices. Z ; (5) Using LW-3DNet, a network model is trained to fuse segmentation results from different view directions, including: (5-a) Construct the training dataset C of the LW-3DNet network, specifically including: First, for each CT sequence in the original training dataset B, reconstruct the two-dimensional slices from the three viewing directions of sagittal, coronal and transverse planes, and obtain the two-dimensional slices in different viewing directions, which are respectively recorded as and Then, respectively and Input to the trained network model ASPP-UNet X 、ASPP-UNet Y and ASPP-UNet Z The test is carried out in , and the two-dimensional slice segmentation results S in different view directions are obtained X 、S Y and S Z ; Finally, the network predicted S X 、S Y and S Z As the input of LW-3DNet network training, the 3D manual segmentation results of CT sequences in training dataset B are used as labels to construct the training dataset C of LW-3DNet network; (5-b) Input the training data set C into the LW-3Dnet network for training, and obtain the trained network model LW-3DNet F ; (6) For the CT sequence T to be tested, firstly, two-dimensional slice reconstruction is performed from the three viewing directions of sagittal, coronal and transverse planes to obtain two-dimensional slices of the CT sequence image in different viewing directions, which are respectively denoted as T X 、T Y and T Z ; Then, T X 、T Y and T Z Input to the trained network model ASPP-UNet X 、ASPP-UNet Y and ASPP-UNet Z The test is carried out in the , and the 2D slice segmentation results F in different view directions are obtained. X 、F Y and F Z ; Finally, F X 、F Y and F Z Input the LW-3DNetF network model for testing and obtain the probability P of each voxel in the CT sequence T belonging to the target obj , and the liver tumor 3D segmentation result S obj ; (7) Constructing a graph cut energy function to further optimize the segmentation results and obtain accurate liver tumor segmentation results, including: (7-a) For the CT sequence T to be detected, the following graph cut energy function is constructed: Where v represents the voxel in the CT sequence image T, L v represents the neighborhood voxel set of voxel v, and u represents the voxel set L belonging to the neighborhood voxel set v In the voxel, R(v) and B(v,u) represent the regional penalty term and boundary penalty term in the graph cut energy function respectively. α is a weight parameter with a constant value between 0 and 1. The regional penalty term is mainly used to penalize a single voxel for assigning a label of "target" or "background". The calculation formula is as follows: R(v)=β·R1(v)+(1-β)·R2(v) Among them, P obj (v) represents the probability that voxel v in the CT sequence T belongs to a liver tumor, which is obtained by the method described in step (6). The greater the probability that voxel v belongs to the "target" or "liver tumor", the smaller the penalty for assigning the voxel to the "target", and the greater the penalty for assigning the voxel to the "background". β is a weight parameter, which is a constant between 0 and 1. g v represents the gray value of voxel v in CT sequence T, and γ represent the segmentation results S obtained according to the method described in step (6) obj The calculated grayscale mean and standard deviation of the liver tumor area are calculated as follows: Among them, R o Represents the segmentation result S obj The liver tumor region voxel set in m represents the voxel set R o The voxels in g m represents the grayscale value of voxel m, N o Represents the voxel set R o The number of voxels in ; the boundary term B(v,u) is used to penalize the grayscale inconsistency between adjacent voxels, and the calculation formula is as follows: in D(v,u) represents the Euclidean distance between voxels v and u, N1 represents the number of voxels in the CT sequence T, g v and g u Represents the grayscale value of voxel v and u respectively, |g v -g u | means to find g v With g u The absolute value of the difference; (7-b) The maximum flow minimum cut algorithm is used to minimize the energy function E(T) to obtain accurate liver tumor segmentation results.

2. The method for liver tumor segmentation in CT sequence images based on deep learning according to claim 1, characterized in that: In step (4), the network model ASPP-UNet is trained and obtained X 、ASPP-UNet Y and ASPP-UNet Z When , the loss function is a hybrid loss function based on cross entropy and Dice, which is defined as follows: l=l c +η·l d Among them, l c and l d Represent cross entropy and Dice loss respectively, η is the weight parameter, represents the expert manual segmentation result of the w-th pixel in the CT image, where the background is marked as 0 and the target is marked as 1. It represents the probability that the network model predicts that the w-th pixel belongs to the target, and N is the number of pixels in the CT image.

3. The method for liver tumor segmentation in CT sequence images based on deep learning according to claim 1, characterized in that: In step (5-b), train and obtain the network model LW-3DNet F When , the loss function is the Dice loss function.

4. The method for liver tumor segmentation in CT sequence images based on deep learning according to claim 1, characterized in that: In step (7-a), the L v is a neighborhood voxel set centered on the current voxel v and of size 3×3×3.

5. The method for liver tumor segmentation in CT sequence images based on deep learning according to claim 1, characterized in that: The ε1 is a constant of 0.3 to 0.7, n is a natural number of 2 to 10, k is a natural number of 1 to 8, ε2 is a constant of 0.3 to 0.7, α is a constant of 0.3 to 0.7, parameter β is a constant of 0.3 to 0.7, and η is a constant of 0.5 to 2.

Images