An automatic and accurate method for liver region segmentation in abdominal CT sequence images

Abstract

The present invention discloses a method for automatic and accurate segmentation of the liver region in abdominal CT sequence images, which mainly includes: (1) for the CT sequence to be detected, first reconstructing two-dimensional slices from three viewing directions: sagittal, coronal, and transverse; (2) using a U-shaped 2D convolutional network based on dilated spatial pyramid convolution to segment the two-dimensional slices in different viewing directions; (3) using a lightweight 3D convolutional network to fuse the segmentation results from different viewing directions to obtain the probability that each voxel in the CT sequence belongs to the target; (5) constructing a fully connected conditional random field energy function based on the obtained probability, and obtaining an accurate liver segmentation result by minimizing the energy function. The present invention extracts the three-dimensional features of the CT sequence by fusing information from different viewing directions, and obtains a three-dimensional liver segmentation result with high accuracy and strong robustness by introducing a fully connected conditional random field.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a method for automatically and accurately segmenting a liver region in abdominal CT sequence images. Background Art

[0002] The liver, the largest solid organ in the human body, is richly vascularized, has a complex structure, and plays a vital role in detoxification. Liver diseases are numerous and pose a serious threat to human health. Currently, approximately 33% of the world's population suffers from various forms of liver disease. Treatments for liver disease primarily include chemotherapy, surgery, and radiotherapy. Both surgery and radiotherapy require accurate segmentation of liver tissue from medical images to obtain pathological, physical, and anatomical information, providing a theoretical basis for treatment plans. The liver has a complex structure, fuzzy boundaries, and a variety of shapes. Even manual delineation by experienced experts is time-consuming, labor-intensive, and susceptible to subjective factors. Therefore, studying automatic liver segmentation has become a current research hotspot with significant clinical application value.

[0003] Many researchers have proposed different approaches to the liver segmentation problem in CT image sequences, primarily categorized as those based on artificial features and deep learning. Artificial feature-based methods primarily include region growing, thresholding, model-based methods, and machine learning-based methods. These methods manually extract input image features, such as intensity, shape, edges, or texture, and then generate liver contours or regions based on these features. Most of these methods are semi-automatic and require manual selection of seed points or regions of interest. Their segmentation accuracy is susceptible to feature selection and generalization is poor. In recent years, deep learning technology has been widely used in medical image segmentation due to its powerful feature extraction capabilities. Considering spatial and temporal efficiency, most deep learning-based liver segmentation methods employ 2D segmentation networks, sequentially segmenting the liver region for each slice in the CT sequence to obtain the final patient liver segmentation result. These methods fail to consider inter-slice correlation information, resulting in limited segmentation accuracy. To extract three-dimensional feature information from CT sequences, some experts and researchers have proposed 3D segmentation networks for CT image sequences. Due to limited computing resources, these three-dimensional networks usually need to downsample or truncate the CT sequences into small-sized three-dimensional data in advance, which will lead to the loss of image detail information and reduced network segmentation accuracy. Summary of the Invention

[0004] In view of the shortcomings and deficiencies of the existing technology, the present invention aims to provide a method for automatic and accurate segmentation of the liver area in abdominal CT sequence images. By combining 2D deep convolutional networks, 3D lightweight convolutional networks, and fully connected conditional random fields, accurate and effective segmentation of the liver area in CT sequence images can be achieved, thereby improving the accuracy and efficiency of computer-assisted diagnosis and treatment.

[0005] A method for automatically and accurately segmenting the liver region in abdominal CT sequence images comprises the following steps:

[0006] (1) Establish original training datasets A and B containing original CT sequence images and manual segmentation results of the liver region;

[0007] (2) Construct a U-shaped 2D convolutional network based on dilated spatial pyramid convolution, denoted as ASPP-UNet, which specifically includes:

[0008] (2-a) A U-shaped network is used as the backbone network, which includes three encoding layers, two skip connections, a dilated spatial pyramid convolution layer, three decoding layers and a 1×1 convolution layer, wherein: the output of the first encoding layer is not only used as the input of the second encoding layer, but also connected to the second decoding layer through the first skip connection as the input of the decoding layer; the output of the second encoding layer is not only used as the input of the third encoding layer, but also connected to the first decoding layer through the second skip connection as the input of the decoding layer; the output of the third encoding layer is used as the input of the dilated spatial pyramid convolution layer, and the output of the dilated spatial pyramid convolution layer is used as the input of the first decoding layer; in addition, the output of the previous decoding layer is used as the input of the next decoding layer; in order to obtain the segmentation result, the last decoding layer is connected to a 1×1 convolution layer, wherein the output of the last decoding layer is used as the input of the 1×1 convolution layer, and the output of the 1×1 convolution layer is the probability that each pixel belongs to the target. By introducing a threshold ε1, the segmentation result can be obtained; the ε1 is preferably a constant of 0.3 to 0.7;

[0009] (2-b) In the backbone network described in step (2-a), each encoding layer is composed of two 2D convolutional modules connected together, namely, 2D double convolutional modules, where each 2D convolutional module contains a 3×3 convolutional layer, a batch normalization layer, and a Relu activation layer; in order to downsample the image, a 2×2 maximum pooling layer is added at the end of the 2D double convolutional module in the second and third encoding layers;

[0010] (2-c) In the backbone network described in step (2-a), the dilated spatial pyramid convolution layer specifically includes: using n different sampling radii {r v The 3×3 convolution kernels of |v=1,2,...,n} perform dilated convolution on the input feature maps respectively, and concatenate the dilated convolution results as the output of the dilated spatial pyramid convolution layer, where n is a natural number greater than 1, preferably a natural number between 2 and 10; in order to expand the receptive field of the convolution kernel and obtain multi-scale context information, the sampling radius is set to r v =k×v+1, where k is a natural number greater than 0, preferably a natural number from 1 to 8;

[0011] (2-d) In the backbone network described in step (2-a), the first and second decoding layers are each composed of a 2D double convolution module described in step (2-b), a 2×2 deconvolution layer, and a splicing operation, and the third decoding layer is composed of only one 2D double convolution module described in step (2-b), wherein: the input of the 2D double convolution module in the first decoding layer is the output of the void spatial pyramid convolution layer, and thereafter, the input of the 2D double convolution module in the next decoding layer is the output of the previous decoding layer; the splicing operation in the first decoding layer is used to splice the deconvolution result in the decoding layer and the output of the second encoding layer, and the splicing result is used as the output of the decoding layer; the splicing operation in the second decoding layer is used to splice the deconvolution result in the decoding layer and the output of the first encoding layer, and the splicing result is used as the output of the decoding layer;

[0012] (3) Construct a lightweight 3D convolutional network, denoted as LW-3DNet, which involves three inputs and one output. The specific structure of the network includes: first, three 3D double convolution modules are used to convolve the three inputs respectively, then the convolution results are spliced by splicing operation, and a 3D double convolution module is used to convolve the spliced results to obtain a feature map F, and finally, a 1×1×1 convolution layer is used to convolve the feature map F, and the output of the 1×1×1 convolution layer is the probability that each voxel belongs to the target; the 3D double convolution module described in step (3) is composed of two 3D convolution modules connected, where each 3D convolution module contains a convolution layer of size 3×3×3, a batch normalization layer and a Relu activation layer;

[0013] (4) Using ASPP-UNet, multiple network models are trained to segment 2D slices in different viewing directions. The specific steps include: for each CT sequence in the training dataset A, first reconstruct the 2D slices from the sagittal, coronal and transverse viewing directions respectively, and obtain the 2D slices in different viewing directions, which are recorded as and Then, the 2D slices in the sagittal view direction obtained in the training dataset are The corresponding two-dimensional slice manual segmentation results are input into the ASPP-UNet network for training to obtain the network model ASPP-UNet that can be used to segment sagittal two-dimensional slices. X , the two-dimensional slices of the coronal view direction obtained in the training dataset The corresponding two-dimensional slice manual segmentation results are input into the ASPP-UNet network for training to obtain the network model ASPP-UNet that can be used to segment the coronal two-dimensional slices. Y , the two-dimensional slices of the cross-sectional view direction obtained in the training dataset The corresponding two-dimensional slice manual segmentation results are input into the ASPP-UNet network for training to obtain the network model ASPP-UNet that can be used to segment the cross-sectional two-dimensional slices. Z ; Train to obtain the network model ASPP-UNet X 、ASPP-UNet Y and ASPP-UNet Z When , the loss function is preferably a hybrid loss function based on cross entropy and Dice, which is defined as follows:

[0014] l=l c +η·l d

[0015]

[0016]

[0017] Among them, l c and l d They represent cross entropy and Dice loss respectively, η is a weight parameter, preferably a constant of 0.5 to 2, g w represents the expert manual segmentation result of the w-th pixel in the CT image, where the background is marked as 0 and the target is marked as 1, p w represents the probability that the network model predicts that the w-th pixel belongs to the target, and T is the number of pixels in the CT image;

[0018] (5) Using LW-3DNet, a network model is trained to fuse segmentation results from different view directions, including:

[0019] (5-a) Construct the training dataset C of the LW-3DNet network, specifically including: First, for each CT sequence in the original training dataset B, reconstruct the two-dimensional slices from the three viewing directions of sagittal, coronal and transverse planes, and obtain the two-dimensional slices in different viewing directions, which are respectively recorded as and Then, respectively and Input to the trained network model ASPP-UNet X 、ASPP-UNet Y and ASPP-UNet Z The test is carried out in , and the two-dimensional slice segmentation results S in different view directions are obtained X 、S Y and S Z ; Finally, the network predicted S X 、S Y and S ZAs the input of LW-3DNet network training, the 3D manual segmentation results of CT sequences in training dataset B are used as labels to construct the training dataset C of LW-3DNet network;

[0020] (5-b) Input the training data set C into the LW-3Dnet network for training. The Dice loss function is preferred as the loss function to obtain the trained network model LW-3DNet F ;

[0021] (6) For the CT sequence to be detected, firstly, two-dimensional slice reconstruction is performed from the three viewing directions of sagittal, coronal and transverse planes to obtain two-dimensional slices of the CT sequence image in different viewing directions, which are respectively denoted as T X 、T Y and T Z ; Then, T X 、T Y and T Z Input to the trained network model ASPP-UNet X 、ASPP-UNet Y and ASPP-UNet Z The test is carried out in the , and the 2D slice segmentation results F in different view directions are obtained. X 、F Y and F Z ; Finally, F X 、F Y and F Z Input the LW-3DNetF network model for testing to obtain the probability that each voxel in the CT sequence belongs to the target in represents the probability that the i-th voxel belongs to the liver, and N is the number of voxels in the CT sequence to be detected;

[0022] (7) Using fully connected conditional random fields to obtain accurate liver segmentation results, specifically including:

[0023] (7-a) For the CT sequence to be detected, construct a fully connected conditional random field energy function:

[0024]

[0025] Where x={x i |i=1,...,N},x i represents the label assigned to the i-th voxel, φ u (x i ) and φ p (x i ,x j ) are the first-order energy term and the second-order energy term respectively. u (x i) represents the label x assigned to the i-th voxel i The cost is calculated as follows:

[0026] φ u (x i )=-log(P(x i ))

[0027] Among them, P(x i ) means assigning the label of the i-th voxel to x i The probability is calculated as follows:

[0028]

[0029] in, represents the probability that the i-th voxel belongs to the liver, obtained by the method described in step (6), φ p (x i ,x j ) indicates that the labels x are assigned to the i-th and j-th voxels respectively. i and x j The cost is calculated as follows:

[0030] φ p (x i ,x j )=μ(x i ,x j )·G(f i ,f j )

[0031] Among them, f i and f j Represents the feature vectors of the i-th and j-th voxels, including position and intensity features, G(f i ,f j ) represents the function applied to the feature vector f i and f j Gaussian potential energy function on μ(x i ,x j ) is a class compatibility function that constrains energy transfer only between voxel pairs with the same class label, that is, voxel pairs with the same class label can influence each other. For the binary classification problem of liver segmentation in CT sequence images, a contrast-sensitive dual-kernel Gaussian potential energy function is used:

[0032] G(f i ,f j )=G a (f i ,f j )+G s (f i ,f j )

[0033] Among them, G a and G s They are surface kernel and smoothing kernel respectively. The surface kernel is used to assign the same label to pixels with similar positions and intensities, and the smoothing kernel is used to remove isolated small areas. The specific calculation formula is as follows:

[0034]

[0035]

[0036] Among them, β1 and β2 are the weight parameters of the surface kernel and smoothing kernel respectively, L i and L j are the spatial positions of the i-th and j-th voxels, I i and I j are the intensities of the i-th and j-th voxels, respectively, ||L i -L j || means finding L i and L j The Euclidean distance of |I i -I j | means asking for I i and I j The absolute value of the difference, parameter σ α and σ β The parameters σ are used to control the spatial proximity and intensity similarity between voxels assigned to the same class label, respectively. γ Used to control the smoothness of the region; the β1 is preferably a constant of 0.8 to 1.2, the β2 is preferably a constant of 0.8 to 1.2, and σ α The constant σ is preferably between 3.0 and 7.0. β The constant σ is preferably between 1.0 and 4.0. γ A constant of 3.0 to 7.0 is preferred;

[0037] (7-b) The mean field approximation method is used to minimize the fully connected conditional random field energy function E(x) to obtain the optimal label assignment result, which is the final liver segmentation result. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 ASPP-UNet network structure diagram

[0039] Figure 2 Schematic diagram of the dilated spatial pyramid convolutional layer structure according to an embodiment of the present invention

[0040] Figure 3 LW-3DNet network structure diagram

[0041] Figure 4Schematic diagram of the network model training process for segmenting 2D slices in different viewing directions

[0042] Figure 5 Schematic diagram of the liver segmentation process of CT sequence images according to an embodiment of the present invention

[0043] Figure 6 3D example of liver segmentation results according to an embodiment of the present invention DETAILED DESCRIPTION

[0044] A method for automatic and accurate segmentation of the liver region in abdominal CT sequence images, the specific implementation steps are as follows:

[0045] (1) Randomly select 100 abdominal CT original sequence images and their corresponding liver region manual segmentation results from the LiTS public database. 50 cases, i.e., 50 abdominal CT original sequence images and their corresponding liver region manual segmentation results, are selected as training dataset A, and the remaining 50 cases are selected as training dataset B. In the manual segmentation results, the liver region (i.e., the target region) is marked as "1" and the background region is marked as "0".

[0046] (2) Construct a U-shaped 2D convolutional network based on dilated spatial pyramid convolution, denoted as ASPP-UNet, with the following structure: Figure 1 As shown, specifically including:

[0047] (2-a) A U-shaped network is used as the backbone network, which includes three encoding layers, two skip connections, a dilated spatial pyramid convolution layer, three decoding layers and a 1×1 convolution layer, wherein: the output of the first encoding layer is not only used as the input of the second encoding layer, but also connected to the second decoding layer through the first skip connection as the input of the decoding layer; the output of the second encoding layer is not only used as the input of the third encoding layer, but also connected to the first decoding layer through the second skip connection as the input of the decoding layer; the output of the third encoding layer is used as the input of the dilated spatial pyramid convolution layer, and the output of the dilated spatial pyramid convolution layer is used as the input of the first decoding layer; in addition, the output of the previous decoding layer is used as the input of the next decoding layer; in order to obtain the segmentation result, the last decoding layer is connected to a 1×1 convolution layer, wherein the output of the last decoding layer is used as the input of the 1×1 convolution layer, and the output of the 1×1 convolution layer is the probability that each pixel belongs to the target. By introducing the threshold ε1, the segmentation result can be obtained; in this embodiment, ε1=0.5 is preferred;

[0048] (2-b) In the backbone network described in step (2-a), each encoding layer is composed of two 2D convolutional modules connected together, namely, 2D double convolutional modules, where each 2D convolutional module contains a 3×3 convolutional layer, a batch normalization layer, and a Relu activation layer; in order to downsample the image, a 2×2 maximum pooling layer is added at the end of the 2D double convolutional module in the second and third encoding layers;

[0049] (2-c) In the backbone network described in step (2-a), the dilated spatial pyramid convolution layer specifically includes: using n convolution layers with different sampling radii {r v The 3×3 convolution kernels of |v=1,2,...,n} perform dilated convolution on the input feature maps respectively, and concatenate the dilated convolution results as the output of the dilated spatial pyramid convolution layer, where n is a natural number greater than 1. In order to expand the receptive field of the convolution kernel and obtain multi-scale context information, the sampling radius is set to r v =k×v+1, where k is a natural number greater than 0; in this embodiment, n=4 and k=2 are preferred. The specific structure of the dilated spatial pyramid convolution layer in this embodiment is as follows Figure 2 As shown in the figure, four 3×3 convolution kernels with radii of 3, 5, 7, and 9 are first used to perform dilated convolution on the input features respectively, and then the dilated convolution results are concatenated as the output of the dilated spatial pyramid convolution layer;

[0050] (2-d) In the backbone network described in step (2-a), the first and second decoding layers are each composed of a 2D double convolution module described in step (2-b), a 2×2 deconvolution layer, and a splicing operation, and the third decoding layer is composed of only one 2D double convolution module described in step (2-b), wherein: the input of the 2D double convolution module in the first decoding layer is the output of the void spatial pyramid convolution layer, and thereafter, the input of the 2D double convolution module in the next decoding layer is the output of the previous decoding layer; the splicing operation in the first decoding layer is used to splice the deconvolution result in the decoding layer and the output of the second encoding layer, and the splicing result is used as the output of the decoding layer; the splicing operation in the second decoding layer is used to splice the deconvolution result in the decoding layer and the output of the first encoding layer, and the splicing result is used as the output of the decoding layer;

[0051] (3) Construct a lightweight 3D convolutional network, denoted as LW-3DNet, with the following structure: Figure 3As shown in FIG, the network involves three inputs and one output. The specific structure of the network includes: first, three 3D double convolution modules are used to convolve the three inputs respectively, and then the convolution results are spliced by splicing operation, and a 3D double convolution module is used to convolve the spliced results to obtain a feature map F. Finally, a 1×1×1 convolution layer is used to convolve the feature map F. The output of the 1×1×1 convolution layer is the probability that each voxel belongs to the target. The 3D double convolution module described in step (3) is composed of two 3D convolution modules connected together, where each 3D convolution module contains a convolution layer of size 3×3×3, a batch normalization layer and a Relu activation layer.

[0052] (4) Using ASPP-UNet, multiple network models that can be used to segment two-dimensional slices in different viewing directions are trained. The training process is as follows: Figure 4 As shown in the figure, the specific steps include: for each CT sequence in the training data set A, first reconstruct the two-dimensional slices from the three viewing directions of sagittal, coronal and transverse planes, and obtain the two-dimensional slices in different viewing directions, which are respectively recorded as and Then, the 2D slices of different view directions obtained in the training dataset are The corresponding two-dimensional slice manual segmentation results are input into the ASPP-UNet network for training, and three network models that can be used to segment two-dimensional slices in different view directions are obtained, which are respectively denoted as ASPP-UNet X 、ASPP-UNet Y and ASPP-UNet Z During the training process, the loss function is preferably a hybrid loss function based on cross entropy and Dice, which is specifically defined as follows:

[0053] l=l c +η·l d

[0054]

[0055]

[0056] Among them, l c and l d Denote cross entropy and Dice loss respectively, η is a weight parameter, and in this embodiment, η=1 is preferred, g w represents the expert manual segmentation result of the w-th pixel in the CT image, where the background is marked as 0 and the target is marked as 1, p w represents the probability that the network model predicts that the w-th pixel belongs to the target, and T is the number of pixels in the CT image;

[0057] (5) Using LW-3DNet, a network model is trained to fuse segmentation results from different view directions, including:

[0058] (5-a) Construct the training dataset C of the LW-3DNet network, specifically including: First, for each CT sequence in the original training dataset B, reconstruct the two-dimensional slices from the three viewing directions of sagittal, coronal and transverse planes, and obtain the two-dimensional slices in different viewing directions, which are respectively recorded as and Then, respectively and Input to the trained network model ASPP-UNet X 、ASPP-UNet Y and ASPP-UNet Z The test is carried out in , and the two-dimensional slice segmentation results S in different view directions are obtained X 、S Y and S Z ; Finally, the network predicted S X 、S Y and S Z As the input of LW-3DNet network training, the 3D manual segmentation results of CT sequences in training dataset B are used as labels to construct the training dataset C of LW-3DNet network;

[0059] (5-b) Input the training data set C into the LW-3Dnet network for training. The Dice loss function is preferred as the loss function to obtain the trained network model LW-3DNet F ;

[0060] (6) For the CT sequence to be detected, firstly, two-dimensional slice reconstruction is performed from the three viewing directions of sagittal, coronal and transverse planes to obtain two-dimensional slices of the CT sequence image in different viewing directions, which are respectively denoted as T X 、T Y and T Z ; Then, T X 、T Y and T Z Input to the trained network model ASPP-UNet X 、ASPP-UNet Y and ASPP-UNet Z The test is carried out in the , and the 2D slice segmentation results F in different view directions are obtained. X 、F Y and F Z ; Finally, F X 、F Y and F ZInput the LW-3DNetF network model for testing to obtain the probability that each voxel in the CT sequence belongs to the target in represents the probability that the i-th voxel belongs to the target (i.e., liver), and N is the number of voxels in the CT sequence;

[0061] (7) Using fully connected conditional random fields to obtain accurate liver segmentation results, specifically including:

[0062] (7-a) For the CT sequence to be detected, construct a fully connected conditional random field energy function:

[0063]

[0064] Where x={x i |i=1,...,N},x i represents the label assigned to the i-th voxel, φ u (x i ) and φ p (x i ,x j ) are the first-order energy term and the second-order energy term respectively. u (x i ) represents the label x assigned to the i-th voxel i The cost is calculated as follows:

[0065] φ u (x i )=-log(P(x i ))

[0066] Among them, P(x i ) means assigning the label of the i-th voxel to x i The probability is calculated as follows:

[0067]

[0068] in, represents the probability that the i-th voxel belongs to the target (i.e., liver), obtained by the method described in step (6), φ p (x i ,x j ) indicates that the labels x are assigned to the i-th and j-th voxels respectively. i and x j The cost is calculated as follows:

[0069] φ p (x i ,x j )=μ(x i ,x j )·G(f i ,fj )

[0070] Among them, f i and f j Represents the feature vectors of the i-th and j-th voxels, including position and intensity features, G(f i ,f j ) represents the function applied to the feature vector f i and f j Gaussian potential energy function on μ(x i ,x j ) is a class compatibility function that constrains energy transfer only between voxel pairs with the same class label, that is, voxel pairs with the same class label can influence each other. For the binary classification problem of liver segmentation in CT sequence images, a contrast-sensitive dual-kernel Gaussian potential energy function is used:

[0071] G(f i ,f j )=G a (f i ,f j )+G s (f i ,f j )

[0072] Among them, G a and G s They are surface kernel and smoothing kernel respectively. The surface kernel is used to assign the same label to pixels with similar positions and intensities, and the smoothing kernel is used to remove isolated small areas. The specific calculation formula is as follows:

[0073]

[0074]

[0075] Among them, β1 and β2 are the weight parameters of the surface kernel and smoothing kernel respectively, L i and L j are the spatial positions of the i-th and j-th voxels, I i and I j are the intensities of the i-th and j-th voxels, respectively, ||L i -L j || means finding L i and L j The Euclidean distance of |I i -I j | means asking for I i and I j The absolute value of the difference, parameter σ α and σ β The parameters σ are used to control the spatial proximity and intensity similarity between voxels assigned to the same class label, respectively.γ Used to control the smoothness of the region; in this embodiment, β1 = 1.0, β2 = 1.0, σ α =0.5,σ β =2.5,σ γ =5.0;

[0076] (7-b) The mean field approximation method is used to minimize the fully connected conditional random field energy function E(x) to obtain the optimal label assignment result, which is the final liver segmentation result.

[0077] Figure 5 A schematic diagram of the liver segmentation process of CT sequence images according to an embodiment of the present invention is given. For the original CT sequence images to be segmented, two-dimensional slice reconstruction is first performed from three viewing directions: sagittal, coronal, and transverse. Two-dimensional slices T of different viewing directions of the CT sequence images are obtained. X 、T Y and T Z ; Then, T X 、T Y and T Z Input to the trained network model ASPP-UNet X 、ASPP-UNet Y and ASPP-UNet Z The test is carried out in the , and the 2D slice segmentation results F in different view directions are obtained. X 、F Y and F Z ; Then, F X 、F Y and F Z Enter LW-3DNet F The network model is tested to obtain the probability P of each voxel in the CT sequence belonging to the target obj ; Finally, using CT sequence images and probability P obj , a fully connected conditional random field energy function is constructed, and accurate liver segmentation results are obtained by minimizing the energy function.

[0078] Figure 6 A three-dimensional example of the liver segmentation results obtained in this embodiment is given. The figure uses a grayscale scale bar to represent the minimum distance between the surface voxels of the segmentation result and the real liver surface voxels. The smaller the distance, the lower the grayscale value, indicating a more accurate segmentation result. It can be seen that the method of the present invention can effectively and accurately segment the liver area in the CT sequence. In the obtained segmentation results, the error values of the liver surface distance are small and close to 0.

[0079] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for automatic and accurate segmentation of the liver region in abdominal CT sequence images, characterized in that: The following steps are involved: (1) Establish original training datasets A and B containing original CT sequence images and manual segmentation results of the liver region; (2) Construct a U-shaped 2D convolutional network based on dilated spatial pyramid convolution, denoted as ASPP-UNet, which specifically includes: (2-a) A U-shaped network is used as the backbone network, which consists of three encoding layers, two skip connections, a dilated spatial pyramid convolution layer, three decoding layers and a 1×1 convolution layer, wherein: the output of the first encoding layer is not only used as the input of the second encoding layer, but also connected to the second decoding layer through the first skip connection as the input of the decoding layer; the output of the second encoding layer is not only used as the input of the third encoding layer, but also connected to the first decoding layer through the second skip connection as the input of the decoding layer; the output of the third encoding layer is used as the input of the dilated spatial pyramid convolution layer, and the output of the dilated spatial pyramid convolution layer is used as the input of the first decoding layer; in addition, the output of the previous decoding layer is used as the input of the next decoding layer; in order to obtain the segmentation result, the last decoding layer is connected to a 1×1 convolution layer, where the output of the last decoding layer is used as the input of the 1×1 convolution layer, and the output of the 1×1 convolution layer is the probability that each pixel belongs to the target. By introducing the threshold ε1, the segmentation result can be obtained; (2-b) In the backbone network described in step (2-a), each encoding layer is composed of two 2D convolutional modules connected together, namely, 2D double convolutional modules, where each 2D convolutional module contains a 3×3 convolutional layer, a batch normalization layer, and a Relu activation layer; in order to downsample the image, a 2×2 maximum pooling layer is added at the end of the 2D double convolutional module in the second and third encoding layers; (2-c) In the backbone network described in step (2-a), the dilated spatial pyramid convolutional layer specifically includes: Using n different sampling radii {r v The 3×3 convolution kernels of |v=1,2,...,n} perform dilated convolution on the input feature maps respectively, and concatenate the dilated convolution results as the output of the dilated spatial pyramid convolution layer, where n is a natural number greater than 1. In order to expand the receptive field of the convolution kernel and obtain multi-scale context information, the sampling radius is set to r v =k×v+1, where k is a natural number greater than 0; (2-d) In the backbone network described in step (2-a), the first and second decoding layers are each composed of a 2D double convolution module described in step (2-b), a 2×2 deconvolution layer, and a splicing operation, and the third decoding layer is composed of only one 2D double convolution module described in step (2-b), wherein: the input of the 2D double convolution module in the first decoding layer is the output of the void spatial pyramid convolution layer, and thereafter, the input of the 2D double convolution module in the next decoding layer is the output of the previous decoding layer; the splicing operation in the first decoding layer is used to splice the deconvolution result in the decoding layer and the output of the second encoding layer, and the splicing result is used as the output of the decoding layer; the splicing operation in the second decoding layer is used to splice the deconvolution result in the decoding layer and the output of the first encoding layer, and the splicing result is used as the output of the decoding layer; (3) Construct a lightweight 3D convolutional network, denoted as LW-3DNet, which involves three inputs and one output. The specific structure of the network includes: first, three 3D double convolution modules are used to convolve the three inputs respectively, then the convolution results are spliced by splicing operation, and a 3D double convolution module is used to convolve the spliced results to obtain a feature map F, and finally, a 1×1×1 convolution layer is used to convolve the feature map F, and the output of the 1×1×1 convolution layer is the probability that each voxel belongs to the target; the 3D double convolution module described in step (3) is composed of two 3D convolution modules connected, where each 3D convolution module contains a convolution layer of size 3×3×3, a batch normalization layer and a Relu activation layer; (4) Using ASPP-UNet, multiple network models are trained to segment 2D slices in different viewing directions. The specific steps include: for each CT sequence in the training dataset A, first reconstruct the 2D slices from the sagittal, coronal and transverse viewing directions respectively, and obtain the 2D slices in different viewing directions, which are recorded as and Then, the 2D slices in the sagittal view direction obtained in the training dataset are The corresponding two-dimensional slice manual segmentation results are input into the ASPP-UNet network for training to obtain the network model ASPP-UNet that can be used to segment sagittal two-dimensional slices. X , the two-dimensional slices of the coronal view direction obtained in the training dataset The corresponding two-dimensional slice manual segmentation results are input into the ASPP-UNet network for training to obtain the network model ASPP-UNet that can be used to segment the coronal two-dimensional slices. Y , the two-dimensional slices of the cross-sectional view direction obtained in the training dataset The corresponding two-dimensional slice manual segmentation results are input into the ASPP-UNet network for training to obtain the network model ASPP-UNet that can be used to segment the cross-sectional two-dimensional slices. Z ; (5) Using LW-3DNet, a network model is trained to fuse segmentation results from different view directions, including: (5-a) Construct the training dataset C of the LW-3DNet network, specifically including: First, for each CT sequence in the original training dataset B, reconstruct the two-dimensional slices from the three viewing directions of sagittal, coronal and transverse planes, and obtain the two-dimensional slices in different viewing directions, which are respectively recorded as and Then, respectively and Input to the trained network model ASPP-UNet X 、ASPP-UNet Y and ASPP-UNet Z The test is carried out in , and the two-dimensional slice segmentation results S in different view directions are obtained X 、S Y and S Z ; Finally, the network predicted S X 、S Y and S Z As the input of LW-3DNet network training, the 3D manual segmentation results of CT sequences in training dataset B are used as labels to construct the training dataset C of LW-3DNet network; (5-b) Input the training data set C into the LW-3Dnet network for training, and obtain the trained network model LW-3DNet F ; (6) For the CT sequence to be detected, firstly, two-dimensional slice reconstruction is performed from the three viewing directions of sagittal, coronal and transverse planes to obtain two-dimensional slices of the CT sequence image in different viewing directions, which are respectively denoted as T X 、T Y and T Z ; Then, T X 、T Y and T Z Input to the trained network model ASPP-UNet X 、ASPP-UNet Y and ASPP-UNet Z The test is carried out in the , and the 2D slice segmentation results F in different view directions are obtained. X 、F Y and F Z ; Finally, F X 、F Y and F Z Enter LW-3DNet F The network model is tested to obtain the probability that each voxel in the CT sequence belongs to the target in represents the probability that the i-th voxel belongs to the liver, and N is the number of voxels in the CT sequence to be detected; (7) Using fully connected conditional random fields to obtain accurate liver segmentation results, specifically including: (7-a) For the CT sequence to be detected, construct a fully connected conditional random field energy function: Where x={x i |i=1,...,N},x i represents the label assigned to the i-th voxel, φ u (x i ) and φ p (x i ,x j ) are the first-order energy term and the second-order energy term respectively; φ u (x i ) represents the label x assigned to the i-th voxel i The cost is calculated as follows: φ u (x i )=-log(P(x i )) Among them, P(x i ) means assigning the label of the i-th voxel to x i The probability is calculated as follows: in, represents the probability that the i-th voxel belongs to the liver, obtained by the method described in step (6), φ p (x i ,x j ) indicates that the labels x are assigned to the i-th and j-th voxels respectively. i and x j The cost is calculated as follows: φ p (x i ,x j )=μ(x i ,x j )·G(f i ,f j ) Among them, f i and f j Represents the feature vectors of the i-th and j-th voxels, including position and intensity features, G(f i ,f j ) represents the function applied to the feature vector f i and f j Gaussian potential energy function on μ(x i ,x j ) is a category compatibility function, which is used to constrain energy transfer only between voxel pairs with the same category label, that is, voxel pairs with the same category label can influence each other; for the binary classification problem of liver segmentation in CT sequence images, a contrast-sensitive dual-kernel Gaussian potential energy function is used: G(f i ,f j )=G a (f i ,f j )+G s (f i ,f j ) Among them, G a and G s They are surface kernel and smoothing kernel respectively; the surface kernel is used to assign the same label to pixels with adjacent positions and similar intensities, and the smoothing kernel is used to remove isolated small areas. The specific calculation formula is as follows: Among them, β1 and β2 are the weight parameters of the surface kernel and smoothing kernel respectively, L i and L j are the spatial positions of the i-th and j-th voxels, I i and I j are the intensities of the i-th and j-th voxels, respectively, ||L i -L j || means finding L i and L j The Euclidean distance of |I i -I j | means asking for I i and I j The absolute value of the difference, parameter σ α and σ β The parameters σ are used to control the spatial proximity and intensity similarity between voxels assigned to the same class label, respectively. γ Used to control the smoothness of the area; (7-b) The mean field approximation method is used to minimize the fully connected conditional random field energy function E(x) to obtain the optimal label assignment result, which is the final liver segmentation result.

2. The method for automatically and accurately segmenting the liver region in abdominal CT sequence images according to claim 1, wherein: In step (4), the network model ASPP-UNet is trained and obtained X 、ASPP-UNet Y and ASPP-UNet Z When , the loss function is preferably a hybrid loss function based on cross entropy and Dice, which is defined as follows: l=l c +η·l d Among them, l c and l d Represents cross entropy and Dice loss respectively, η is the weight parameter, g w represents the expert manual segmentation result of the w-th pixel in the CT image, where the background is marked as 0 and the target is marked as 1, p w It represents the probability that the network model predicts that the w-th pixel belongs to the target, and T is the number of pixels in the CT image.

3. The method for automatically and accurately segmenting the liver region in abdominal CT sequence images according to claim 1, wherein: In step (5-b), train and obtain the network model LW-3DNet F When , the loss function is preferably the Dice loss function.

4. The method for automatically and accurately segmenting the liver region in abdominal CT sequence images according to claim 1, wherein: The ε1 is preferably a constant of 0.3 to 0.7, n is preferably a natural number of 2 to 10, k is preferably a natural number of 1 to 8, η is preferably a constant of 0.5 to 2, β1 is preferably a constant of 0.8 to 1.2, β2 is preferably a constant of 0.8 to 1.2, σ α The constant σ is preferably between 3.0 and 7.

0. β The constant σ is preferably between 1.0 and 4.

0. γ A constant of 3.0 to 7.0 is preferred.

Images