Image segmentation method incorporating spectral ct information
By constructing a mapping network and a main segmentation network from low-energy spectral CT images to high-energy spectral CT images, and utilizing deep learning technology, the problems of automation and accuracy in spectral CT image segmentation were solved, achieving efficient organ segmentation on ordinary computers and meeting clinical needs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-01
- Publication Date
- 2026-03-20
AI Technical Summary
Existing deep learning networks have not performed well in spectral CT image segmentation, making it difficult to achieve automated and accurate organ segmentation, especially in the presence of radiation hardening artifacts and low tissue contrast.
We constructed a mapping network from low-energy spectral CT images to high-energy spectral CT images and a main segmentation network. Using deep learning technology, we achieved automatic organ segmentation through pre-trained models and data augmentation methods.
It improves the accuracy and efficiency of spectral CT image segmentation, reduces the time required for doctors to manually segment images, and the algorithm can run on ordinary office computers, meeting the requirements of clinical applications.
Smart Images
Figure CN116091508B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of artificial intelligence medical image analysis, in particular to an image segmentation method integrated with spectral CT information. BACKGROUND
[0002] Spectral CT (Computerized Tomography), also known as dual-energy CT, is a computed tomography technique that uses two independent X-ray photon spectra, allowing detection of materials with different attenuation characteristics at different energies. Currently, conventional CT has become an important way for disease diagnosis and screening in clinical practice. Compared with conventional CT, spectral CT uses its single-energy image, basis material image, spectral curve, effective atomic number and other parameters for disease diagnosis and quantitative analysis, not only providing traditional CT with human anatomical morphology images, but also realizing material composition analysis, identification and quantification through multi-parameter imaging.
[0003] Organ segmentation of medical images refers to outlining all parts of the organ in a complete medical image, in other words, extracting the region of interest in the medical image. Segmentation of internal structures such as body organs in medical images, i.e. medical image segmentation, is an important task in clinical applications such as computer-aided diagnosis (CAD), computer-aided surgery (CAS) and radiotherapy (RT). Many subsequent studies on individual organs require the first segmentation of the region where the organ is located, such as principal component analysis and statistical shape model establishment. Detailed organ segmentation on CT is a difficult task that requires manual annotation and automatic segmentation algorithms. The reasons include multiple aspects, such as the morphological complexity of abdominal structures, the huge difference between internal and external subjects, and image features such as low contrast of soft tissue.
[0004] With the increasing popularity of spectral CT, organ segmentation of spectral CT has gradually become one of the important needs in clinical practice. Compared with conventional single-energy CT, single-energy spectral CT often has limitations such as radiation hardening artifacts and low tissue contrast. These limitations also affect the accuracy of organ segmentation, which is the basis for many medical applications such as lesion diagnosis, dose optimization and minimally invasive surgery. Like conventional single-energy CT, organ segmentation of spectral CT plays an important auxiliary role in computer-aided diagnosis and computer-aided surgery. Moreover, after organ segmentation on spectral CT, analysis of CT values of each organ at different energies can obtain the CT value range of the normal human body.
[0005] In recent years, deep learning technology has been constantly improved, and convolutional neural networks have achieved excellent performance in the field of medical images. However, the current application of neural networks in image segmentation is mainly based on conventional CT images. Some new networks with good performance cannot be directly used on spectral CT, and there is an urgent need for an algorithm that can automatically segment spectral CT in clinical practice. SUMMARY
[0006] To solve the above problems, the present application provides an image segmentation method integrating spectral CT information, which utilizes the difference information between single-energy spectral CT images for image segmentation. The method first selects two single-energy spectral CT images from a plurality of single-energy spectral CT images, designs a deep learning network, and completes the mapping of low-energy spectral CT images to high-energy spectral CT images. Then, the pre-trained model of the deep learning network is used as the pre-trained model of the main segmentation network, and part of the parameters of the main segmentation network is fixed to train the main segmentation network, thereby realizing the utilization of spectral CT single-energy image information and the segmentation task of organs, mainly suitable for automatic analysis of spectral CT images and conventional CT.
[0007] The technical scheme of the present application is an image segmentation method integrating spectral CT information, comprising the following steps:
[0008] Step A: Constructing a data set for a low-energy spectral CT image to high-energy spectral CT image mapping network and a data set for a main segmentation network;
[0009] Step A1, data selection;
[0010] According to the spectral CT reconstruction of a plurality of single-energy spectral CT images, the pixel points of each single-energy spectral CT image correspond one-to-one in space position, and the CT values are different; select a plurality of energy spectral CT image groups, each energy spectral CT image group includes single-energy spectral CT images at two CT energies, namely low-energy spectral CT images and high-energy spectral CT images;
[0011] Step A2, preparing labels for training the main segmentation network;
[0012] Manual segmentation of labels requires a lot of time and effort, and the present application uses a network coarse segmentation and fine adjustment method to prepare labels for training the main segmentation network; a coarse segmentation network is used to perform the first step of coarse segmentation of organs in the low-energy spectral CT image, and the coarse segmentation label includes left kidney, right kidney, liver, and spleen; based on the coarse segmentation label, a medical image processing software is used for further fine segmentation to achieve the gold standard;
[0013] Step B: Training of the low-energy spectral CT image to high-energy spectral CT image mapping network; the low-energy spectral CT image as the input of the mapping network; the difference image of the high-energy spectral CT image and the low-energy spectral CT image as the output of the mapping network;
[0014] Step C: Training of the main segmentation network, the low-energy spectral CT image as the input image of the main segmentation network; fixing the encoder parameters of the mapping network, the output of the main segmentation network becomes the segmentation label image. The knowledge learned from the mapping network is used for organ segmentation.
[0015] The specific steps of step B are as follows:
[0016] Step B1, image preprocessing and data enhancement;
[0017] The collected images come from different devices, and the parameters such as pixel spacing used are not the same. In order to ensure that the images input into the mapping network have consistency, it is necessary to normalize the pixel spacing of the images. The single-energy spectral CT image is preprocessed and data enhanced; in order to make the contrast of the region of interest more obvious, and improve the accuracy of the mapping network training, it is also necessary to map and normalize the image grayscale before network training.
[0018] The low-energy spectral CT image is first normalized in pixel spacing, and the three dimensions are unified; the normalized low-energy spectral CT image is subjected to image grayscale transformation, and the CT value is truncated to a set range, so as to enhance the contrast of organs and background in the low-energy spectral CT image; for the output image of the mapping network, the CT value is truncated to a set interval, and the influence of useless organs is excluded;
[0019] Step B2, create a mapping network model;
[0020] The network structure adopted by the mapping network from the low-energy spectral CT image to the high-energy spectral CT image is an adjusted 3D U-Net network structure; the adjusted 3D U-Net network structure is divided into left and right two network structures, the left half network structure is an encoder, and the right half network structure is a decoder; the left half is down-sampling, and the down-sampled part adopts a similar architecture as a full convolutional network (FCN), which uses a 3x3x3 convolution kernel for convolution and then passes through a linear rectifier activation function; the number of channels of each layer of down-sampling is set to 16, 32, 64, 128 and 256 respectively; the right half is up-sampling, which uses deconvolution to continuously reduce the number of feature maps and increase the size of the feature maps; after each up-sampling, the number of channels is halved, and the number of channels of each layer of up-sampling is set to 256, 128, 64, 32 and 16 respectively; the last part of the right half network structure is a 1x1x1 convolution kernel, which convolves the feature maps after up-sampling to generate a probability map; a skip connection is used in the adjusted 3D U-Net network, which connects the outputs of each channel number in the encoder with the outputs of the same channel number in the decoder as the input of the next layer of up-sampling of the decoder;
[0021] Step B3, mapping network model training; different organs and different tissues have different compositions, and the network learns certain anatomical knowledge in this mapping process.
[0022] The 64x64x64 small block cut from the low-energy spectral CT image in step A1 is used as the input of the mapping network, and the prediction of the mapping network is performed through the sliding window method to generate a probability image, which is used as the predicted image, compared with the difference map of the high-energy spectral CT image and the low-energy spectral CT image, and the loss function is calculated to guide the network parameter iteration;
[0023] The similarity of the predicted image and the difference map of the high-energy spectral CT image and the low-energy spectral CT image is evaluated according to the mean square error loss function, and the statistical parameter is the mean square error of the corresponding points of the predicted data and the original data;
[0024]
[0025] wherein, represents the number of samples, that is, the number of pixels of each input image, and respectively represent the pixel value of the gold standard image and the pixel value of the prediction result of the mapping network model.
[0026] The specific steps of step C are as follows:
[0027] Step C1, image preprocessing and data augmentation;
[0028] The low-energy spectral CT image is normalized in pixel spacing and grayscale; the label image is the label made in step A2, and the same pixel spacing normalization is performed thereon; in each cycle of training of the main segmentation network, small blocks at the same position of the low-energy spectral CT image and the label image are cut off as the input of the main segmentation network and the output of the main segmentation network respectively; the data augmentation operation of image random cropping and image random angle rotation is used to increase the training data;
[0029] Step C2, main segmentation model training;
[0030] The main segmentation network is a 3D-UNet network with an increased residual structure; the 3D-UNet network with an increased residual structure is a left-right symmetrical network structure; the left half of the network structure is an encoder, and the right half of the network structure is a decoder; the low-energy spectral CT image to high-energy spectral CT image mapping network is imported into the main segmentation network as a pre-training model, the parameters of the encoder part are unchanged, and only the parameters of the decoder part are trained; finally, five channels of probability maps are output, representing the segmentation probabilities of background, liver, spleen, left kidney and right kidney respectively;
[0031] The loss function of the main segmentation model is a Dice loss function;
[0032]
[0033] In the formula, X and Y represent the pixel values of the main segmentation model prediction image and the pixel values of the gold standard image respectively.
[0034] The coarse segmentation network selects an algorithm with better segmentation effect in the current organ segmentation field, such as the nnUNet network.
[0035] The single-energy spectral CT image in step A1 is subjected to Gaussian filtering processing; Gaussian filtering is a weighted average of the pixel values of the single-energy spectral CT image, a Gaussian kernel is used to perform sliding window convolution on the entire single-energy spectral CT image, and the value of each pixel point of the obtained result image is obtained by weighted average of the pixel values in the Gaussian kernel; the Gaussian kernel is obtained by Gaussian calculation function:
[0036] .
[0037] In the formula, and represent the template coordinates of the pixel, represent the standard deviation.
[0038] The CT value of the grayscale uniformization is-175HU to 250HU; the pixel spacing normalization in step B1 unifies the three dimensions to 1.5mm; and the CT value of the output image of the mapping network is set to-50HU to 50HU.
[0039] The selection of the mapping network model is not limited to 3D-UNet, and other mapping networks with good effects can also be used.
[0040] In the mapping network, the mapping relationship is inconsistent for different tissues and organs, which makes the network need to capture the boundaries of the organs as much as possible and extract the boundary features of the organs.
[0041] The present application has the beneficial effects: the present application uses the difference information between single-energy spectral CT images to assist the segmentation neural network to perform organ segmentation, and realizes accurate segmentation of spectral CT. Using the method of pre-training model, using the information between multiple single-energy spectral CT images, the accuracy of segmentation is improved, and even the segmentation accuracy in the case of few samples is improved. Using deep learning and using the spectral information in spectral CT to achieve accurate segmentation of organs, the accuracy of the results reaches the requirements that can be used in clinical or scientific research actual application scene, and the time required by the algorithm is significantly less than the time required by the doctor manual segmentation, at the same time, the algorithm can normally run on the daily office computer, and does not need high-performance server. BRIEF DESCRIPTION OF DRAWINGS
[0042] Figure 1 is the flow chart of the image segmentation method of the present application incorporating spectral CT information.
[0043] Figure 2 is the network structure diagram of the mapping network from the low-energy spectral CT image to the high-energy spectral CT image.
[0044] Figure 3 is the network structure diagram of the main segmentation network. DETAILED DESCRIPTION
[0045] The image segmentation method incorporating spectral CT information proposed by the present application, as shown in Figure 1 The present application will be further described below in combination with specific embodiments.
[0046] Step A: constructing the data set for the mapping network from the low-energy spectral CT image to the high-energy spectral CT image and the data set for the main segmentation network;
[0047] Step A1, data selection
[0048] In theory, spectral CT can reconstruct 101 single-energy spectral CT images from 40 keV to 140 keV. The single-energy spectral CT images at 70 keV and 140 keV are selected as the research objects. Because there is noise in the image acquisition process, the brightness of the image is not uniform. In order to eliminate such noise as much as possible, Gaussian filtering is performed on the single-energy spectral CT images at 70 keV and 140 keV. Gaussian filtering is a process of weighted average of image pixel values. A Gaussian kernel is used to perform sliding window convolution on the entire image. The value of each pixel point of the resulting image is obtained by weighted average of the pixel values in the Gaussian kernel. The Gaussian kernel is obtained by Gaussian calculation function:
[0049]
[0050] Step A2, preparing labels for training the main segmentation network
[0051] Manual segmentation labels require a lot of time and effort, so the present application uses a network coarse segmentation and accurate adjustment method to prepare labels. The present application uses the nnUNet coarse segmentation network to perform the first step of coarse segmentation of the organ. The coarse segmentation network used in the first step is not limited to nnUNet, and other networks with higher segmentation accuracy can also be used. nnUNet does not have a significant improvement in network structure, and the network structure used is still the traditional 3D UNet with residual structure, but it has a great innovation in the preprocessing and post-processing stages of the entire segmentation task. For different segmentation tasks, it can automatically analyze the properties of the image, and can perform different preprocessing and post-processing for different segmentation tasks. A large number of low-energy spectral CT images, i.e. 70 keV single-energy spectral CT images, are predicted using nnUNet to obtain rough segmentation labels including left kidney, right kidney, liver, and spleen. The label selection of the present application is not limited to these four segmentation labels, and they can be added or deleted according to demand and algorithm performance.
[0052] Step B, training of low-energy spectral CT image to high-energy spectral CT image mapping network
[0053] Step B1, image preprocessing and data augmentation
[0054] Before the image is input into the neural network, the image needs to be preprocessed and data enhanced. In the work of the present application, the low-energy spectral CT image is first normalized in pixel spacing, and is unified to 1.5 mm in three dimensions. Then the image is transformed in image gray scale, and the CT value is cut off between-175HU and 250HU to enhance the contrast of organs and background in the image. For the output image, the CT value is cut off between-50HU and 50HU to exclude the influence of useless organs. In order to reduce the required display memory, the input image is not the whole image, but a small piece (patch) of 64 × 64 × 64 size cut from the whole spectral CT image, and finally the prediction of the network is performed by sliding window.
[0055] Step B2, creating a main segmentation network model
[0056] As shown in Figure 2 , the network structure adopted by the low-energy spectral CT image to high-energy spectral CT mapping network is a 3D U-Net structure with residual structure. The 3D U-Net network is similar to an autoencoder (Autoencoder), and is also composed of two basically symmetrical structures. The left half or the down-sampling part adopts a similar architecture to the fully convolutional network (FCN), uses a 3 × 3 × 3 convolution kernel for convolution, and then passes through a linear rectifier (RELU) activation function. In the down-sampling part, the number of channels of each layer is set to 16, 32, 64, 128, and 256, respectively. Each layer of down-sampling has two convolution operations, and a residual structure is added to increase the depth of the network. In the right half of the 3D U-Net model or the up-sampling part, the deconvolution is used to continuously reduce the number of feature maps and increase the size of the feature maps. After each up-sampling, the number of channels is halved, and then the corresponding feature maps of the down-sampling part are spliced. In the last of the main segmentation network model, a 1 × 1 convolution kernel is used to convolve the feature map to generate a probability map. The 3D UNet network also uses a skip connection to connect the outputs of each channel number in the encoder and the outputs of the same channel number in the decoder as the input of the next layer of up-sampling of the decoder.
[0057] Step B3, model training
[0058] Using the training set in step A1, each low-energy spectral CT image (70kev monochromatic spectral CT image) is taken as the input of the mapping network to generate a probability image. In order to reduce the required display memory, the input image is not the whole image, but a small piece (patch) of 64 × 64 × 64 size cut from the whole spectral CT image, and finally the prediction of the mapping network is performed by sliding window.
[0059] Step C, Training the main segmentation network
[0060] Step C1, Image preprocessing and data augmentation
[0061] The segmentation target is a 70keV low-energy spectral CT image. The low-energy spectral CT image is first normalized in terms of pixel spacing to 1.5mm across all three dimensions. Then, the image grayscale is transformed, and the CT values are cropped to between -175HU and 250HU to enhance the contrast between organs and the background. The labels are those created in step A and require the same pixel spacing normalization. In each training loop, small patches are cropped from the same locations in both the low-energy spectral CT image and the label image, serving as the input and output of the main segmentation network, respectively. For data augmentation, random image cropping and random angle rotation are used to increase the training data.
[0062] Step C2, Model Training
[0063] like Figure 3 As shown, the main segmentation network also uses a 3D-UNet network with added residual structures. The low-energy spectral CT image to high-energy spectral CT image mapping network from the first stage is imported as a pre-trained model into the main segmentation network. The 3D-UNet network is also composed of two basically symmetrical structures, left and right. The left half is called the encoder, and the right half is called the decoder. After importing the pre-trained model of the first-stage mapping network, the encoder is fixed, and only one decoder is trained. This achieves the effect of utilizing the difference information of different single-energy spectral CT images.
[0064] The ideas and methods of this invention are applicable to other multimodal imaging methods, such as PET / CT images of the human brain.
Claims
1. An image segmentation method incorporating energy spectrum CT information, characterized in that, The image segmentation method incorporating spectral CT information includes the following steps: Step A: Construct the dataset for the network mapping low-energy spectral CT images to high-energy spectral CT images and the dataset for the main segmentation network; Step A1, Data Selection; Several single-energy spectral CT images were reconstructed based on spectral CT. The pixels of each single-energy spectral CT image corresponded one-to-one in spatial location, with different CT values. Several groups of energy spectral CT images were selected. Each group of energy spectral CT images included single-energy spectral CT images at two CT energies, namely low-energy spectral CT images and high-energy spectral CT images. Step A2: Create labels for training the main segmentation network; Labels for training the main segmentation network were created using a combination of coarse segmentation and fine adjustment. The coarse segmentation network was then used to perform initial coarse segmentation of organs in low-energy spectral CT images, with coarse segmentation labels including left kidney, right kidney, liver, and spleen. Based on the coarse segmentation labels, medical image processing software was used for further fine segmentation to achieve the gold standard. Step B: Training the mapping network from low-energy spectral CT images to high-energy spectral CT images; the low-energy spectral CT images are used as the input to the mapping network; the difference between the high-energy spectral CT images and the low-energy spectral CT images is used as the output of the mapping network. Step C: Training the main segmentation network, with low-energy spectral CT images serving as the input images for the main segmentation network.
2. The image segmentation method incorporating energy spectrum CT information according to claim 1, characterized in that, The specific steps of step B are as follows: Step B1, image preprocessing and data augmentation; Preprocessing and data enhancement of single-energy spectral CT images; The low-energy spectral CT images are first normalized in terms of pixel spacing, with all three dimensions being uniform. The normalized low-energy spectral CT images are then transformed in terms of image grayscale, and the CT values are cropped to a set range to enhance the contrast between organs and the background in the low-energy spectral CT images. For the output image of the mapping network, the CT values are cropped to a set interval to eliminate the influence of useless organs. Step B2, create the mapping network model; The network structure used for mapping low-energy spectral CT images to high-energy spectral CT images is a modified 3D U-Net network structure. This modified 3D U-Net network structure consists of two parts: a left-hand network (encoder) and a right-hand network (decoder). The left-hand part performs downsampling using 3 × 3 × 3 convolutional kernels followed by a linear rectified activation function. The number of channels in each downsampling layer is set to 16, 32, 64, 128, and 256, respectively. Each downsampling layer includes two convolutional operations, with a max pooling operation between downsampling layers to reduce the feature map size. The right-hand part performs upsampling, using deconvolution to continuously reduce the number of feature maps and increase their size. Each upsampling operation halves the number of channels, with the number of channels in each upsampling layer set to 256, 128, 64, 32, and 16, respectively. The right-hand network structure ends with a 1 × 1 × 1 convolutional kernel, which processes the upsampling feature map to generate a probability map. The modified 3D U-Net network structure consists of two parts: a left-hand network (encoder) and a right-hand network (decoder). The UNet network uses skip connections to connect the outputs of each channel in the encoder with the outputs of the same channel in the decoder, which are then used as the input for the next layer of upsampling in the decoder. Step B3: Map the network model training; The 64 × 64 × 64 block cut from the low-energy spectral CT image in step A1 is used as the input to the mapping network. The mapping network makes predictions using a sliding window to generate a probability image. This probability image is used as the predicted image and compared with the difference map of the high-energy spectral CT image and the low-energy spectral CT image. The loss function is calculated to guide the iteration of network parameters. The similarity between the predicted image and the difference maps of high-energy spectral CT images and low-energy spectral CT images is evaluated based on the mean squared error loss function. ； in, The number of samples refers to the number of pixels in each input image. and These represent the pixel values of the gold standard image and the pixel values of the prediction results from the mapping network model, respectively.
3. The image segmentation method incorporating energy spectrum CT information according to claim 1 or 2, characterized in that, The specific steps of step C are as follows: Step C1, image preprocessing and data augmentation; The low-energy spectral CT images are normalized in terms of pixel spacing and grayscale; the label images are the labels created in step A2, and the same pixel spacing is normalized on them; in each training loop of the main segmentation network, small patches at the same position are cut from the low-energy spectral CT images and the label images, which are used as the input and output of the main segmentation network, respectively; data augmentation operations such as random image cropping and random angle rotation are used to increase the training data; Step C2, main segmentation model training; The main segmentation network is a 3D-UNet network with added residual structure; the 3D-UNet network with added residual structure consists of two symmetrical network structures; the left half of the network structure is the encoder, and the right half of the network structure is the decoder. The low-energy spectral CT image to high-energy spectral CT image mapping network is imported into the main segmentation network as a pre-trained model. The encoder parameters remain unchanged, and only the decoder parameters are trained. Finally, a probability map with 5 channels is output, representing the segmentation probability of background, liver, spleen, left kidney and right kidney, respectively. The loss function of the main segmentation model is the Dice loss function; ； In the formula, X and Y represent the pixel values of the image predicted by the master segmentation model and the pixel values of the gold standard image, respectively.
4. The image segmentation method incorporating energy spectrum CT information according to claim 1, characterized in that, The coarse segmentation network is the nnUNet network.
5. The image segmentation method incorporating energy spectrum CT information according to claim 1 or 4, characterized in that, In step A1, the single-energy spectral CT image undergoes Gaussian filtering. Gaussian filtering involves weighted averaging of pixel values in the single-energy spectral CT image. A Gaussian kernel is used to perform sliding window convolution on the entire single-energy spectral CT image. The value of each pixel in the resulting image is obtained by weighted averaging of pixel values within the Gaussian kernel. The Gaussian kernel is obtained using the Gaussian calculation function. ； In the formula, and Template coordinates representing pixels, Represents standard deviation.
6. The image segmentation method incorporating energy spectrum CT information according to claim 3, characterized in that, The grayscale uniformization CT cropping range is -175HU to 250HU; the pixel spacing normalization in step B1 unifies the three dimensions to 1.5mm; the CT value cropping range of the image output by the mapping network is set to -50HU to 50HU.