Abdomen multi-organ CT image automatic segmentation method based on deep learning

Abstract

The invention discloses an abdominal multi-organ CT image automatic segmentation method based on deep learning. The method comprises the following steps: establishing a training sample set; constructing an improved encoder; an improved decoder is constructed; a PCE-TransUNet segmentation network model is established, and the PCE-TransUNet segmentation network model is Training the PCE-TransUNet segmentation network model by using the training set, and optimizing by using a joint loss function of cross entropy loss and Dice loss to obtain a trained PCE-TransUNet model; and inputting the test set into the trained PCE-TransUNet model, and outputting a segmented image by the PCE-TransUNet model. According to the method, partial convolution and an efficient channel attention mechanism are introduced, the ability of the model to extract image details is enhanced, the problem that feature extraction is insufficient in a traditional method is solved, and especially when small organs and complex boundaries are processed, the segmentation precision is remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology, specifically involving an automatic segmentation method for abdominal multi-organ CT images based on deep learning. Background Technology

[0002] Abdominal organs are vital physiological carriers in the human body, including key organs such as the liver, spleen, pancreas, and kidneys. Diseases in these organs often pose a serious threat to human health. Malignant tumors of organs such as the liver, pancreas, and kidneys have high incidence and mortality rates in clinical diagnosis and treatment. According to the "Global Cancer Data Report 2020" released by the International Agency for Research on Cancer (IARC) of the World Health Organization, cancers related to abdominal organs account for a high proportion of global cancer incidence and mortality. Early detection rates for liver cancer and pancreatic cancer are particularly low, with most patients being diagnosed at middle or late stages, severely impacting survival rates and quality of life.

[0003] Abdominal multi-organ imaging is an important tool for clinical diagnosis and treatment, with CT being widely used due to its advantages such as high image resolution, fast scanning speed, and intuitive imaging. When interpreting abdominal CT images, physicians often need to regionally delineate and precisely segment the target organs to provide a reliable basis for lesion identification, lesion localization, and subsequent diagnosis and treatment. However, due to the large number of abdominal organs, their significant morphological differences, and the frequent existence of blurred boundaries and signal interference between adjacent organs, manual segmentation is not only time-consuming and labor-intensive but also relies heavily on the physician's professional experience, making it highly subjective and prone to segmentation errors.

[0004] Traditional medical image segmentation methods mainly include region growing, level set methods, superpixel methods, and machine learning methods such as support vector machines (SVM). These methods often rely on low-level image features for segmentation. Due to the lack of support from high-level semantic information, these methods suffer from low segmentation accuracy and insufficient robustness when faced with situations involving blurred boundaries, significant morphological differences, and adhesion between organs in the abdominal region, making it difficult to meet the actual needs of clinical applications.

[0005] In recent years, with the rapid development of deep learning technology, deep learning-based medical image segmentation methods have gradually become a research hotspot. Compared with traditional methods, deep learning methods can automatically extract high-level semantic features, achieving more accurate segmentation results and reducing the complexity of manually designed features. Existing research has shown that using deep learning for automatic segmentation of organs such as the liver, pancreas, spleen, and kidneys can improve segmentation accuracy to a certain extent. However, existing methods still have the following shortcomings:

[0006] (1) Limited feature extraction capability, making it difficult to accurately capture details. Although existing U-Net and TransUNet methods have shown some performance in feature extraction, they still have insufficient detail extraction when processing abdominal multi-organ CT images, especially in the segmentation of small organs, where they cannot effectively recover complex boundary information.

[0007] (2) Insufficient context modeling capability, local information cannot be fully utilized. Traditional convolutional network structures fail to fully combine global context information with local details, resulting in the inability to accurately model the spatial relationships and boundary information between multiple organs, leading to a decrease in segmentation accuracy, especially when dealing with complex scenes where organs are interdependent and overlapping.

[0008] (3) Low segmentation accuracy, especially in the recovery of small organ boundaries. Existing methods are insufficient when segmenting small organs and cannot effectively recover the boundaries of small organs, resulting in significant deviations in segmentation results and affecting the final diagnostic effect. Summary of the Invention

[0009] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide an automatic segmentation method for abdominal multi-organ CT images based on deep learning, which has strong feature extraction ability, high segmentation accuracy and strong detail restoration ability.

[0010] The technical solution adopted to solve the above-mentioned technical problems is: an automatic segmentation method for abdominal multi-organ CT images based on deep learning, including the following steps:

[0011] Step 1. Establish the training sample set

[0012] Read the original files of the multi-organ CT dataset, create a multi-organ segmentation dataset, unify the dataset size to a fixed size, and divide it into a test set and a training set according to the proportion;

[0013] Step 2. Build an improved encoder

[0014] The improved encoder consists of a partial convolution module, an efficient channel attention module, a convolutional neural network downsampling module, and a Transformer module. The partial convolution module performs convolution calculations on a subset of the input feature map, preserving the original features of unprocessed channels, reducing redundant computation, and enhancing local feature extraction capabilities. The efficient channel attention module generates channel attention weights, weighting the channels of the feature map to enhance the features of important channels and suppress the features of unimportant channels. The convolutional neural network downsampling module progressively reduces the resolution of the feature map using convolutional layers, extracting features at different scales to obtain multi-scale feature maps. The Transformer module utilizes a self-attention mechanism to model global contextual information, capturing long-term dependencies between different locations in the image and enhancing the understanding of relationships between complex organs.

[0015] Step 3. Build an improved decoder

[0016] The improved decoder consists of a spatial location information reconstruction module, a fusion module, and a transposed convolutional upsampling module. The spatial location information reconstruction module is used to correct the spatial location correlation lost after feature map serialization by the Transformer module, ensuring that the spatial distribution of the feature map corresponds to the anatomical structure position of the original input image. The fusion module is used for accurate segmentation of small organ features in multiple organs. The transposed convolutional upsampling module gradually restores the spatial resolution of the feature map by learning adjustable convolution kernel parameters, restores the high-frequency details of the image, and fuses it with the corresponding feature map in the encoder to enhance the semantic information of cross-scale feature transmission.

[0017] Step 4. Establish the PCE-TransUNet segmentation network model

[0018] The PCE-TransUNet segmentation network model is a symmetric encoder and decoder structure built on the TransUNet framework, including an improved encoder, an improved decoder, skip connection layers, and a segmentation head. The improved encoder performs partial convolutional preprocessing on the input multi-organ CT images to enhance local detail features while retaining the original information. Through three stages of downsampling in the convolutional neural network downsampling module, it gradually extracts the image's detailed features, mid-level semantics, and high-level semantics, thereby gradually reducing the feature map size and increasing the number of channels. The Transformer module divides the feature map extracted by the convolutional neural network downsampling module into image blocks, converts them into embedding vectors, and captures long-distance dependencies through a multi-head self-attention mechanism to model global features.

[0019] The skip connection layer transmits features from different levels of the improved encoder to the corresponding levels of the improved decoder, ensuring that local detail information is not lost during upsampling and is spliced ​​and fused with the features of the improved decoder.

[0020] The improved decoder first corrects the spatial location information of the features output by the Transformer module through the spatial location information reconstruction module, ensuring that the spatial distribution of the feature map corresponds one-to-one with the anatomical structure position of the original image. Then, it gradually restores the spatial resolution of the feature map through the transposed convolution upsampling module, learns and optimizes the feature space mapping end-to-end, and efficiently restores the high-frequency details of the image. Through the fusion module, it combines the features passed by skip connections, extracts global spatial features through convolution operations, performs differential enhancement on some channels through partial convolution, and introduces nonlinear activation functions to improve the feature representation capability.

[0021] The segmentation head receives the feature map output by the decoder, optimizes the feature representation through 3×3 convolution, restores the original image resolution through bilinear upsampling, performs pixel-level classification using the Softmax activation function, generates a segmentation mask for multiple abdominal organs, and completes the image segmentation task.

[0022] Step 5. Train the PCE-TransUNet segmentation network model using the training set, and optimize it using a joint loss function of cross-entropy loss and Dice loss to obtain the trained PCE-TransUNet model;

[0023] Step 6. Input the test set into the trained PCE-TransUNet model, and the PCE-TransUNet model will output the segmented image.

[0024] As a preferred technical solution, the efficient channel attention module includes a global average pooling unit, a one-dimensional convolution, a sigmoid activation function unit, and a channel weighting unit;

[0025] The global average pooling unit is used to perform global average pooling on the input feature map in the spatial dimension to obtain the global feature vector of each channel.

[0026] The one-dimensional convolution is used to perform a one-dimensional convolution operation on the global feature vector to capture the local correlation between channels;

[0027] The Sigmoid activation function unit is used to generate channel attention weights by passing the feature vector after one-dimensional convolution through the Sigmoid activation function.

[0028] The channel weighting unit is used to multiply the generated channel attention weights with the original input feature map channel by channel, thereby enhancing the features of important channels and suppressing the features of unimportant channels to obtain the final output feature map.

[0029] As a preferred technical solution, the fusion module is composed of partially convolutional units, regular convolutional units, and ReLU nonlinear activation function units connected in sequence; the partially convolutional units are used to enhance local features and retain original information, reducing computational overhead; the regular convolutional units are used to extract global spatial features and compensate for insufficient local information; the ReLU nonlinear activation function units are used to introduce nonlinearity and improve the expressive ability of complex abdominal organ features.

[0030] As a preferred technical solution, the joint loss function of the cross-entropy loss and the Dice loss is:

[0031] Loss = 0.5 × (CE) Loss +Dice Loss )

[0032]

[0033] In the formula, Loss is the joint loss function. For the segmentation result of the i-th prediction, y i Let N be the i-th ground truth image, and N be the total number of sample labels.

[0034] The beneficial effects of this invention are as follows:

[0035] This invention introduces partial convolution and efficient channel attention mechanisms, which enhances the model's ability to extract image details and solves the problem of insufficient feature extraction in traditional methods. In particular, it significantly improves segmentation accuracy when dealing with small organs and complex boundaries.

[0036] This invention uses transposed convolution upsampling to replace the traditional bilinear interpolation method, and combines skip connections to fuse multi-scale features, which improves the accuracy of spatial feature restoration and the ability to recover local details, making the segmentation results more accurate, especially in the restoration of small organ boundaries.

[0037] This invention achieves global context modeling by introducing a Transformer module into the encoder, which can better capture the spatial relationships and interdependencies between abdominal organs and improve the model's ability to handle complex scenes.

[0038] This invention features a small network parameter scale, offering good practicality and deployability, and can meet the real-time and resource-constrained requirements of clinical applications. It achieves automatic segmentation of abdominal CT images, reducing manual intervention, improving segmentation efficiency, and providing a rapid and accurate reference for clinical diagnosis and treatment. Attached Figure Description

[0039] Figure 1 This is a flowchart illustrating the automatic segmentation method for abdominal multi-organ CT images based on deep learning, as proposed in this invention.

[0040] Figure 2 This is a sample diagram illustrating the creation of an abdominal training test set.

[0041] Figure 3 This is a schematic diagram of the structure of a partial convolutional module.

[0042] Figure 4 This is a schematic diagram of the structure of the high-efficiency channel attention module.

[0043] Figure 5 This is a schematic diagram of the PCE-TransUNet segmentation network model.

[0044] Figure 6 This is a diagram showing the segmentation results of the test set. Detailed Implementation

[0045] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments, but the present invention is not limited to the following embodiments.

[0046] exist Figure 1 This embodiment of the present invention provides an automatic segmentation method for abdominal multi-organ CT images based on deep learning, comprising the following steps:

[0047] Step 1. Establish the training sample set

[0048] Multi-organ CT scan images were obtained from publicly available abdominal multi-organ image datasets, such as medical imaging datasets, to construct a multi-organ segmentation dataset. This dataset contains several images of different abdominal organs, and the images in the dataset are uniformly sized to a fixed size of 256×256. The dataset is divided into a test set and a training set in an 8:2 ratio. The images in the test set are shown below. Figure 2 ;

[0049] To enhance the diversity of training data, data augmentation methods such as rotation, flipping, translation, scaling, random noise, and brightness adjustment are used to augment the training set, resulting in an augmented training set.

[0050] Step 2. Build an improved encoder

[0051] The improved encoder consists of a partial convolution module, an efficient channel attention module, a convolutional neural network downsampling module, and a Transformer module. The partial convolution module performs convolution calculations on a subset of channels of the input feature map, preserving the original features of unprocessed channels, reducing redundant computation, and enhancing local feature extraction capabilities. Figure 3The efficient channel attention module generates channel attention weights, weighting the channels of the feature map to enhance the features of important channels and suppress the features of unimportant channels. The convolutional neural network downsampling module uses convolutional layers to progressively reduce the resolution of the feature map, extracting features at different scales to obtain multi-scale feature maps. Specifically, the first stage outputs a feature map with a resolution of H / 2×W / 2×64, where H is the height and W is the width, capturing low-level details such as edges and textures. The second stage outputs a feature map with a resolution of H / 4×W / 4×128, extracting mid-level semantic features such as local organ contours. The third stage outputs a feature map with a resolution of H / 8×W / 8×256, generating high-level local semantic features such as organ region features.

[0052] The Transformer module first divides the feature map of resolution H / 8×W / 8×256 into P×P image blocks. After serialization, it is converted into a D-dimensional embedding vector through linear mapping and position embedding is added to preserve spatial information. It uses a self-attention mechanism to model global context information, capture long-term dependencies between different locations in the image, and enhance the understanding of relationships between complex organs.

[0053] The efficient channel attention module includes a global average pooling unit, a one-dimensional convolution, a sigmoid activation function unit, and a channel weighting unit, such as... Figure 4 ;

[0054] The global average pooling unit is used to perform global average pooling on the input feature map in the spatial dimension to obtain the global feature vector for each channel;

[0055] One-dimensional convolution is used to perform one-dimensional convolution operations on global feature vectors to capture local correlations between channels.

[0056] The Sigmoid activation function unit is used to generate channel attention weights by passing the feature vector after one-dimensional convolution through the Sigmoid activation function.

[0057] The channel weighting unit is used to multiply the generated channel attention weights with the original input feature map channel by channel, thereby enhancing the features of important channels and suppressing the features of unimportant channels to obtain the final output feature map.

[0058] Step 3. Build an improved decoder

[0059] The improved decoder consists of a spatial location information reconstruction module, a fusion module, and a transposed convolutional upsampling module. The spatial location information reconstruction module is used to correct the spatial location correlation lost after feature map serialization by the Transformer module, ensuring that the spatial distribution of the feature map corresponds to the anatomical structure position of the original input image. The fusion module is used for accurate segmentation of small organ features in multiple organs. The transposed convolutional upsampling module gradually restores the spatial resolution of the feature map by learning adjustable convolution kernel parameters, restores the high-frequency details of the image, and fuses it with the corresponding feature map in the encoder to enhance the semantic information of cross-scale feature transmission.

[0060] Step 4. Establish the PCE-TransUNet segmentation network model

[0061] The PCE-TransUNet segmentation network model is a symmetric encoder and decoder structure built on the TransUNet framework, including an improved encoder, an improved decoder, skip connection layers, and a segmentation head, such as... Figure 5 The improved encoder preprocesses the input multi-organ CT images through partial convolution to enhance local detail features while retaining the original information. It then extracts detailed features, mid-level semantics, and high-level semantics of the image through three stages of downsampling in the convolutional neural network downsampling module, gradually reducing the feature map size and increasing the number of channels. The Transformer module divides the feature map extracted by the convolutional neural network downsampling module into image blocks and converts them into embedding vectors. It then captures long-distance dependencies and models global features through a multi-head self-attention mechanism.

[0062] The skip connection layer transmits features from different levels of the improved encoder to the corresponding levels of the improved decoder, ensuring that local detail information is not lost during upsampling and is spliced ​​and fused with the features of the improved decoder.

[0063] The improved decoder first corrects the spatial location information of the features output by the Transformer module through the spatial location information reconstruction module, ensuring that the spatial distribution of the feature map corresponds one-to-one with the anatomical structure position of the original image. Then, it gradually restores the spatial resolution of the feature map through the transposed convolution upsampling module, learns and optimizes the feature space mapping end-to-end, and efficiently restores the high-frequency details of the image. Through the fusion module, it combines the features passed by skip connections, extracts global spatial features through convolution operations, performs differential enhancement on some channels through partial convolution, and introduces nonlinear activation functions to improve the feature representation capability.

[0064] The segmentation head receives the feature map output by the decoder, optimizes the feature representation through 3×3 convolution, restores the original image resolution through bilinear upsampling, performs pixel-level classification using the Softmax activation function, generates a segmentation mask for multiple abdominal organs, and completes the image segmentation task.

[0065] Step 5. Train the PCE-TransUNet segmentation network model using the training set. Optimize the model using the joint loss function Loss, which combines cross-entropy loss and Dice loss. During training, use the Adam optimizer with an initial learning rate of 0.001 and a decay factor of 0.9. Adjust the learning rate after every 10 training rounds to ensure convergence speed. Regularize the model using a dropout rate of 0.3 to obtain the trained PCE-TransUNet model.

[0066] The joint loss function of cross-entropy loss and Dice loss is:

[0067] Loss = 0.5 × (CE) Loss +Dice Loss )

[0068]

[0069] In the formula, For the segmentation result of the i-th prediction, y i Let N be the i-th ground truth image, and N be the total number of sample labels.

[0070] Step 6. Input the test set into the trained PCE-TransUNet model. The PCE-TransUNet model outputs segmented images, such as... Figure 6 .

Claims

1. A deep learning-based automatic segmentation method for abdominal multi-organ CT images, characterized in that, Comprising the following steps: Step 1. Establishing a training sample set Read the multi-organ CT data set original file, make a multi-organ segmentation data set, unify the data set size to a fixed size, and divide it into a test set and a training set according to the proportion; Step 2. Constructing an improved encoder The improved encoder is composed of a partial convolution module, an efficient channel attention module, a convolutional neural network downsampling module and a Transformer module, wherein the partial convolution module is used to perform convolution calculation on part of the channels of the input feature map, retains the original features of the untreated channels, reduces redundant calculation and enhances the local feature extraction capability; The efficient channel attention module is used to generate channel attention weights, weight the channels of the feature map, enhance the features of important channels, and suppress the features of unimportant channels; the convolutional neural network downsampling module is used to gradually reduce the resolution of the feature map using convolutional layers, extract features of different scales, and obtain multi-scale feature maps; the Transformer module uses a self-attention mechanism to model global context information, capture long-term dependencies between different positions in the image, and enhance the understanding of complex inter-organ relationships; Step 3. Constructing an improved decoder The improved decoder is composed of a spatial position information reconstruction module, a fusion module and a transposed convolution upsampling module, wherein the spatial position information reconstruction module is used to correct the spatial position correlation lost after the sequence processing of the feature map by the Transformer module, to ensure that the spatial distribution of the feature map corresponds to the anatomical structure position of the original input image; the fusion module is used for precise segmentation of small organ features in multi-organ; and the transposed convolution upsampling module gradually restores the spatial resolution of the feature map by learning adjustable convolution kernel parameters, restores the high-frequency details of the image, and fuses with the corresponding feature map in the encoder, to enhance the semantic information of cross-scale feature transmission; Step 4. Establishing a PCE-TransUNet segmentation network model The PCE-TransUNet segmentation network model is a symmetrical encoder and decoder structure based on the TransUNet framework, which includes an improved encoder, an improved decoder, a skip connection layer and a segmentation head; the improved encoder pre-processes the input multi-organ CT image through partial convolution, enhances local detail features, and retains original information, gradually extracts image detail features, middle-level semantics and high-level semantics through three stages of downsampling by the convolutional neural network downsampling module, gradually reduces the size of the feature map, and increases the number of channels, and the Transformer module divides the feature map extracted by the convolutional neural network downsampling module into image blocks, converts them into embedding vectors, captures long-distance dependencies through a multi-head self-attention mechanism, and models global features; The skip connection layer transmits the features of different levels of the improved encoder to the corresponding levels of the improved decoder, ensures that the local detail information is not lost in the upsampling process, and is fused with the features of the improved decoder; The improved decoder first corrects the spatial position information of the feature output by the Transformer module through the spatial position information reconstruction module, ensures that the spatial distribution of the feature map corresponds to the anatomical structure position of the original image one by one, and then gradually restores the spatial resolution of the feature map through the transposed convolution upsampling module, optimizes the feature space mapping through end-to-end learning, efficiently restores the high-frequency details of the image, combines the features transmitted through the skip connection through the fusion module, extracts global spatial features through convolution operation, differentiates and enhances part of the channels through part of the convolution, and introduces a nonlinear activation function to improve the feature representation ability; The segmentation head receives the feature map output by the decoder, optimizes the feature expression through 3*3 convolution, restores the original image resolution through bilinear upsampling, performs pixel-level classification using the Softmax activation function, generates a segmentation mask of the abdominal multi-organ, and completes the image segmentation task; Step 5. Train the PCE-TransUNet segmentation network model using the training set, and use the joint loss function of cross-entropy loss and Dice loss for optimization to obtain the trained PCE-TransUNet model; Step 6. Input the test set into the trained PCE-TransUNet model, and the PCE-TransUNet model outputs a segmentation image.

2. The automatic abdominal multi-organ CT image segmentation method based on deep learning according to claim 1, characterized in that, The efficient channel attention module includes a global average pooling unit, a one-dimensional convolution, a Sigmoid activation function unit and a channel weighting unit; The global average pooling unit is used for global average pooling of the input feature map in the spatial dimension to obtain a global feature vector of each channel; The one-dimensional convolution is used for one-dimensional convolution operation on the global feature vector to capture the local correlation between channels; The Sigmoid activation function unit is used for generating channel attention weights by Sigmoid activation function on the feature vector after one-dimensional convolution; The channel weighting unit is used for multiplying the generated channel attention weights with the original input feature map channel by channel, enhancing important channel features and suppressing unimportant channel features, to obtain the final output feature map.

3. The automatic abdominal multi-organ CT image segmentation method based on deep learning according to claim 1, characterized in that, The fusion module is composed of a part convolution unit, a regular convolution unit and a ReLU nonlinear activation function unit connected in sequence; the part convolution unit is used for enhancing local features and preserving original information, and reducing calculation overhead; the regular convolution unit is used for extracting global spatial features and making up for insufficient local information; and the ReLU nonlinear activation function unit is used for introducing nonlinearity to improve the expression ability of complex abdominal organ features.

4. The automatic abdominal multi-organ CT image segmentation method based on deep learning according to claim 1, characterized in that, The joint loss function of cross-entropy loss and Dice loss is: Loss = 0.5 x (CE Loss + Dice Loss ) In the formula, Loss is a joint loss function, is the i-th predicted segmentation result, y i is the i-th true value image, and N is the total number of sample labels.

Images