Three-dimensional kidney tumor image segmentation algorithm introducing residual feature enhancement

By introducing three-dimensional U-Net and residual feature enhancement modules for residual feature enhancement, the problem of insufficient sample data and uneven category distribution in image segmentation of kidney and renal tumors is solved, and high-precision segmentation of kidney and renal tumors is achieved. A better clinical diagnostic reference.

CN119027658APending Publication Date: 2024-11-26TIANJIN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310603712.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-26
Publication Date
2024-11-26

Smart Images

  • Figure CN119027658A_ABST
    Figure CN119027658A_ABST
Patent Text Reader

Abstract

The invention discloses a three-dimensional kidney tumor image segmentation method introducing residual feature enhancement, which relates to the technical field of deep learning, is used for segmenting kidneys and kidney tumors in medical abdomen CT (Computed Tomography) images, and comprises the following steps of: 1, preprocessing a kidney and tumor image data set, resampling all three-dimensional data to the same voxel pitch, and obtaining a kidney and tumor image data set; performing intensity cutting to perform de-noising processing on the image, and expanding a training data set by using technologies of rotation, zooming and the like; a second step, using a Pytorch framework to construct an image segmentation network introducing residual feature enhancement, the main body structure of the network model being divided into two parts: a three-dimensional U-Net trunk model used for feature extraction and recovery and based on a coding and decoding structure, and a residual feature enhancement module used for enhancing the feature expression ability at a decoding end; the two parts are integrated together for training; and step 3, testing the algorithm model and retaining optimal parameters, and performing kidney tumor three-dimensional medical image segmentation by using the model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the field of medical image segmentation, and in particular to a three-dimensional renal tumor CT image segmentation method introducing residual feature enhancement. Background Art

[0002] Renal cell carcinoma, also known as kidney cancer, is a highly harmful cancer. It is one of the most common cancers of the genitourinary system, and the main patient population is the middle-aged and elderly. The basic function of the kidney is to help humans clean up metabolic waste and retain useful substances, such as proteins and sugars. If the kidney is damaged, the human body environment will be seriously unbalanced, and the overall function will collapse. With the rapid development of modern society and the accelerated pace of life, many middle-aged people and even teenagers are facing the hazards and risks of kidney cancer. In clinical practice, kidney and kidney tumor images are mainly obtained through medical imaging methods such as electronic computed tomography and magnetic resonance imaging. Doctors manually read the film to determine the patient's lesion area, divide the tumor size, analyze the severity of kidney cancer, and then give treatment advice. However, this process requires a lot of labor costs, because radiologists may need to mark the target area in hundreds of slices for a patient. Therefore, in order to reduce the labor cost burden caused by manual segmentation, it is of great significance to realize the automated segmentation of medical images and help patients obtain diagnosis results, receive treatment and recover in the later stage as soon as possible.

[0003] In recent years, the field of computer vision has ushered in major breakthroughs, and its applications are mainly divided into four directions: object detection, image classification, semantic segmentation, and instance segmentation. For the task of kidney and renal tumor segmentation, distinguishing kidneys and renal tumors from CT images belongs to the category of semantic segmentation, which requires pixel-level classification. Deep learning-based methods, especially convolutional neural networks (CNNs), can extract image features through convolutional layers, pooling layers, and fully connected layers, and complete end-to-end image segmentation tasks without relying on a lot of manual effort.

[0004] Although the current algorithms based on deep learning and convolutional neural networks can achieve good segmentation goals in the kidney and renal tumor segmentation tasks, there are still the following problems: (1) There is less sample data for renal tumors. Since medical images involve privacy issues, hospitals need to keep patient information confidential and provide less data that can be used for training and verification. When training the network model, less sample data can easily lead to model overfitting, making the model's robustness and generalization worse; (2) The distribution of categories in kidney and renal tumor images is uneven. Since CT scans include many abdominal organs, the kidney and renal tumor occupy a smaller area of ​​the image, and there is a serious imbalance in the distribution of foreground and background, which makes it difficult to accurately segment the kidney and tumor. (3) The number of parameters and the amount of calculation of three-dimensional convolutional neural networks are large, and the network's ability to extract and restore foreground information is limited. The segmentation accuracy is poor and cannot be widely used in clinical practice. Summary of the invention

[0005] In view of the problem that the current kidney and renal tumor image segmentation algorithm has a large number of parameters and low segmentation accuracy, the present invention proposes a three-dimensional renal tumor image segmentation algorithm that introduces residual feature enhancement. The technical solution of the present invention is as follows:

[0006] A three-dimensional renal tumor image segmentation algorithm introducing residual feature enhancement includes the following steps:

[0007] The first step is to preprocess kidney and renal tumor images. The KiTS2019 dataset images are resampled so that each image has the same voxel spacing, and combined with intensity cropping, the images can adapt to the processing method of the three-dimensional convolutional neural network. At the same time, image enhancement methods such as random rotation and random scaling are used to expand the training dataset to prevent overfitting of network training.

[0008] In the second step, a three-dimensional image segmentation network with residual feature enhancement is constructed using the Pytorch framework, and feature learning and training are performed to complete the image segmentation task. The main network structure of the present invention includes two parts: a three-dimensional U-Net backbone model based on a coding and decoding structure for feature extraction and recovery, and a residual feature enhancement module for the decoder to enhance the feature expression capability. The method is as follows:

[0009] (1) 3D U-Net:

[0010] The network is improved by upgrading the dimensionality of the two-dimensional U-Net, upgrading the convolution kernel and pooling kernel to three dimensions. The overall structure is an encoding and decoding structure. The convolution layers in the encoding and decoding modules are used for feature extraction and recovery. LeakyReLU and instance normalization strategies are used to alleviate the gradient vanishing problem under small sample input. A deep supervision strategy is adopted to obtain multi-scale outputs at the decoding end. A combined loss function is used to assist network training, accelerate convergence speed, and solve the problem of imbalanced foreground and background classification of input and output data.

[0011] (2) Introducing the residual feature enhancement module:

[0012] The 3D U-Net is used as the baseline network, which can independently complete feature extraction and recovery. A feature enhancement module is introduced at the 3D U-Net decoding end to enhance the spatial and channel features of the feature map, improve the expressiveness of the decoding path features and reduce the noise interference around the target area. Finally, a residual connection is made with the original decoding backbone to improve the segmentation accuracy of the network model.

[0013] (3) Post-processing of connected domains:

[0014] The kidney and renal tumor images contain other abdominal organs, and the tumor segmentation results are easily interfered by noise. The present invention performs post-processing analysis based on the connected domain on the prediction result image, excludes the foreground pixels that obviously do not belong to the kidney and renal tumor, and enhances the final segmentation accuracy.

[0015] The third step is to test the network and save the best network model parameters, send the KiTS19 test set into the network for test verification, and upload the segmentation results to the KiTS2019 official online server for performance evaluation.

[0016] The present invention applies deep learning technology to the field of medical image segmentation. Aiming at the problems of low segmentation accuracy and large network parameters in the current kidney and renal tumor segmentation methods, a three-dimensional U-Net-based image segmentation network is proposed. The network extracts image features multiple times through the backbone network and the nested sub-network, and reduces the number of channels to reduce the model parameters. At the same time, a residual feature enhancement module is introduced to enhance the decoder feature expression ability and improve the model segmentation accuracy. The algorithm of the present invention achieves good segmentation accuracy while introducing fewer parameters, and has a good reference value for the segmentation and diagnosis of kidneys and renal tumors in clinical practice. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A three-dimensional segmentation network structure diagram of the residual feature enhancement module provided in the embodiment of the present invention

[0019] Figure 2 Model structure diagram of the three-dimensional U-Net provided by the embodiment of the present invention

[0020] Figure 3 Structure diagram of the residual feature enhancement decoding module provided by the embodiment of the present invention

[0021] Figure 4 A comparison visualization diagram of the segmentation results provided in an embodiment of the present invention DETAILED DESCRIPTION

[0022] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0023] The first step is to preprocess kidney and renal tumor images.

[0024] The present invention uses the official dataset KiTS19 of the KiTS2019 Kidney and Renal Tumor Segmentation Challenge for network training and reasoning. The dataset contains CT images of 300 patients with renal cancer, each of which contains a modality and two types of labels, namely kidney and renal tumor. Among them, 210 cases were divided by experts for kidney and renal tumor labels, and were divided into training set and validation set at a ratio of 8:2. The test set contains 90 patient images, and the label data is not public. The present invention submits the segmentation results to the challenge online server to obtain performance scores. Each case has different voxel spacing due to shooting environment factors, and data anisotropy will destroy the advantages of three-dimensional convolution. The present invention preprocesses the data, resamples all data to a voxel spacing of 3.22×1.62×1.62, and sets the window width to [-20, 300] according to the different intensity values ​​of different abdominal organs in medical images. The resampled CT image size is inconsistent, and the input image is cropped into a single-channel image block of 96×160×160 to maintain the uniform size of the input image and reduce GPU memory consumption. To prevent overfitting during training, the images were randomly rotated 15 degrees in the positive and negative directions and scaled to 0.85 or 1.25 times of the original data to complete data augmentation.

[0025] In the second step, the Pytorch framework is used to build a 3D image segmentation network that introduces residual feature enhancement, and feature learning and training are performed to complete the image segmentation task. Figure 1 The overall structure of the three-dimensional segmentation network with the residual feature enhancement module introduced in the present invention is shown. The network is divided into two parts: a three-dimensional U-Net backbone network for feature extraction and recovery, and a residual feature enhancement module for enhancing the feature expression capability of the decoding end. The specific introduction is as follows:

[0026] (1) 3D U-Net:

[0027] 3D U-Net is a dimensionality-enhanced 2D U-Net. Figure 2 The network structure of 3D U-Net is shown. It is still an encoding and decoding structure. There are four downsamplings in the encoder stage. Each layer still uses 3×3×3 convolution kernels to perform two convolutions to extract features (the two layers of convolution are grouped together in the last level encoder and the first level decoder). Unlike the convolution of 2D U-Net, the two convolutions of 3D U-Net increase the number of channels to make up for the information loss caused by downsampling. Downsampling uses a 2×2×2 maximum pooling operation to reduce the size of the feature map. In the decoding stage, 3D U-Net uses a deconvolution with a convolution kernel of 2×2×2 for upsampling. 3D U-Net uses skip-layer connections to perform channel fusion of the upsampled feature map with the shallow feature map of the same size. The fused feature map is further extracted through two 3×3×3 convolution kernels.

[0028] Figure 2 The network structure diagram of the three-dimensional U-Net is shown. Taking the specific channel parameters as an example, in the encoding path, the input image channel of the network is 1. After two convolutions by the first-level encoder, the number of channels is increased to 64. After the first downsampling, the size is reduced to 1 / 2 of the original size. After downsampling, two convolution layers are connected to double the number of channels. Finally, after four downsamplings, the size is reduced to 1 / 16 of the input image size, and the number of channels is increased to 1024. In the decoding path, a single upsampling expands the size to twice the size of the original feature map, and the same-size feature map is combined in the form of channel addition through a skip layer connection, and the shallow and deep features are fused. After fusion, the number of channels is adjusted to 1 / 2 of the previous level through two layers of convolution, until the last level decoder reduces the number of channels to the target classification number through 1×1×1 convolution to achieve segmentation. The present invention uses the LeakyReLU function and the instance normalization strategy in the improved three-dimensional U-Net to alleviate the gradient vanishing problem under the premise of small samples and improve the model fitting ability.

[0029] (2) Introducing the residual feature enhancement module

[0030] Figure 3The structure diagram of the residual feature enhancement module is shown. After the feature map completes a series of downsampling and dilated convolutions at the encoder end, it is used as the input of the Decoder. The original decoding module fuses the input feature map with the shallow feature map of the same size through skip-layer connection, and then expands the image size and reduces the number of channels through cubic interpolation and 3×3×3 convolution. In the improved network, a residual branch is added on the basis of the decoding module. The feature map enters the branch and passes through the global maximum pooling module (Max Pooling) and the global average pooling module (Avg Pooling) based on three dimensions to obtain two feature maps of size B×C×1×1×1. The linear perceptron MLP is fully connected. At this time, the feature map has adjusted and corrected the weights in three dimensions according to the global pooling standard. After the size is restored, the model will be easier to focus on the area of ​​interest. The two feature maps processed by the multi-layer perceptron are added, activated with the Sigmoid function, and residual connections are made and upsampling is performed in the same way as the original decoding module to obtain the feature enhancement results of the first stage.

[0031] In order to enhance the expression ability of the three-dimensional feature map in the spatial domain, the feature enhancement module in the spatial domain is introduced in the second stage. The feature map of size B×C×2D×2H×2W is passed through the channel-based global maximum pooling module and the global average pooling module respectively to obtain two feature maps of size B×1×2D×2H×2W. After the two feature maps are concatenated (Concat) by dimension, the channel dimension is reduced to 1 through a 7×7×7 convolution layer (Conv3D). After activation with the Sigmoid function, it is multiplied with the residual output of the second stage, and the number of channels is adjusted through convolution to make the number of channels consistent with the number of channels of the feature map processed by the original decoding module. The final branch enhancement feature map is obtained. The residual feature enhancement branch is element-wise added to the feature map of the same size of the original decoding module to obtain the residual feature enhancement output of the decoding module at this level. The feature enhancement module is integrated with each level of decoding modules in the decoding path. The original decoding module has four levels, so the feature enhancement module also has four levels.

[0032] (3) Connected domain post-processing

[0033] The present invention adopts a post-processing analysis method based on connected domains, and post-processes the reasoning results of the trained network on the validation set and the test set to remove the false positive point area, so as to improve the segmentation accuracy. A connected domain refers to an area in an image composed of foreground pixels with the same pixel value and adjacent positions. For example, in kidney and tumor images, the kidney and tumor pixel values ​​are different, which are connected domains with different ranges. When the network makes a prediction, due to the intervention of noise, small similar areas with the same kidney or tumor pixels will be generated, which will affect the final evaluation index. Using the connected domain analysis method, each connected domain in the image is marked, and non-target areas are eliminated according to a certain ratio.

[0034] The three-dimensional image contains spatial information. The present invention performs 26-neighborhood analysis, first scanning the foreground voxel points to determine whether the voxel values ​​of the 26 neighboring points around the single voxel point belong to the same category as the voxel point, and uses topology and optimized decision tree methods to find the optimal judgment path, avoiding similarity judgments on all neighborhoods and reducing the search time for similar pixels.

[0035] After the connected domain division is completed, the sub-regions of the kidney and the tumor are counted, and the number of voxel points in each sub-region is further counted. The region with the most voxels is selected as the discrimination standard. On the basis of the standard, the appropriate ratio is reduced to obtain a benchmark value. For example, the region with the most voxels in the connected domain of the kidney contains n voxel points, which is multiplied by the scale factor (value 0.2) to obtain a standard value of 0.2×n. Finally, the region smaller than the standard is divided into the background, and the region higher than the standard is divided into the kidney. The selection of the tumor connected domain is similar. The present invention finally uses the connected domain analysis method and the judgment standard of the three-dimensional image to process the segmented predicted image.

[0036] The third step is to test the network and save the best network model parameters.

[0037] The KiTS19 test set was sent to the network for testing and verification. The test set contained 90 image cases. After uniform preprocessing, the images were cropped into image blocks of 96×160×160 and sent to the network for forward reasoning. After the reasoning was completed, the preprocessing scheme was used to obtain the segmentation result images of the kidney and renal tumor respectively. The segmentation results were uploaded to the KiTS2019 official online server for performance evaluation, and the best parameter model was retained.

[0038] The segmentation algorithm proposed in this invention has Dice values ​​of 0.9658 and 0.8027 for kidney and renal tumor segmentation on the KiTS19 test set, respectively. Figure 4 The segmentation visualization results of the algorithm proposed in the present invention are shown, which are comparisons of the segmentation performances of the three-dimensional U-Net on the validation set, the three-dimensional U-Net with the residual feature enhancement introduced, and the label result. The white dotted line demarcates the kidney and renal tumor areas. It can be seen from the figure that the segmentation algorithm proposed in the present invention can better segment the kidney and renal tumor from the three-dimensional kidney image, and the three-dimensional U-Net with the residual feature enhancement module has a better segmentation effect than the three-dimensional U-Net.

[0039] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. The three-dimensional renal tumor image segmentation algorithm based on residual feature enhancement is introduced, which includes the following steps: The first step is to preprocess kidney and renal tumor images. The images in the KiTS2019 dataset are resampled so that each image has the same voxel spacing, and combined with intensity cropping, the images can be better adapted to the processing of the three-dimensional convolutional neural network. Image enhancement methods, including random rotation and random scaling, are used to expand the training data set to prevent overfitting of network training. In the second step, a three-dimensional image segmentation network with residual feature enhancement is constructed using the Pytorch framework to perform feature learning and training to complete the image segmentation task. The main network structure of the present invention includes two parts: a three-dimensional U-Net backbone model based on a coding and decoding structure for feature extraction and recovery, and a residual feature enhancement module for the decoder to enhance the feature expression capability. The method is as follows: (1) 3D U-Net: The network is upgraded to three dimensions based on the two-dimensional U-Net, and the convolution kernel and pooling kernel are upgraded to three dimensions. The overall structure is still the encoding and decoding structure. The convolution layers in the encoding and decoding modules are used for feature extraction and recovery. LeakyReLU and instance normalization strategies are used to alleviate the gradient vanishing problem under small sample inputs. A deep supervision strategy is used to obtain multi-scale outputs at the decoding end. A combined loss function is used to assist network training, speed up convergence, and solve the problem of imbalanced foreground and background classification of input and output data. (2) Introducing residual feature enhancement module: The 3D U-Net is used as the baseline network, which can independently complete feature extraction and recovery. A feature enhancement module is introduced at the 3D U-Net decoding end to enhance the spatial and channel features of the feature map, improve the expressiveness of the decoding path features and reduce the noise interference around the target area. Finally, a residual connection is made with the original decoding backbone to improve the segmentation accuracy of the network model. (3) Post-processing of connected domains: The kidney and renal tumor images contain other abdominal organs, and the tumor segmentation results are easily interfered by noise. The present invention performs post-processing analysis based on the connected domain on the prediction result image, excludes the foreground pixels that obviously do not belong to the kidney and renal tumor, and enhances the final segmentation accuracy. The third step is to test the network and save the best network model parameters, send the KiTS19 test set into the network for test verification, and upload the segmentation results to the KiTS2019 official online server for performance evaluation.