Deep learning based colorectal cancer MRI image lesion segmentation system and method

Abstract

The application provides a colorectal cancer MRI image lesion segmentation system and method based on deep learning. The application includes preprocessing of MRI original data and extraction of a region of interest, well solves the defect of small amount of medical image data, and achieves better segmentation effect in the training of a deep learning model. Secondly, the UNet model is improved, five parallel channel calculation efficiencies are introduced in the decoding part, and the parameter amount is less. Secondly, the attention module along two independent dimensions (channels and space) sequentially infers the attention map, then the attention map is multiplied by the input feature map for adaptive feature refinement, and a more fine feature map is generated, so that considerable performance improvement is realized while the overhead is kept small.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing and relates to a system and method for segmenting lesions in colorectal cancer MRI images based on an improved UNet network. Background Technology

[0002] Medical imaging provides doctors with information about diseased tissues and anatomical views of specific organs, serving as a crucial foundation for medical diagnosis and treatment. With the rapid development and widespread availability of medical imaging equipment, various medical imaging techniques have emerged. Compared to other imaging technologies, MRI can provide more precise, high-resolution images without causing any harm to the human body, especially in the detection of soft tissues and tumors. MRI images are currently the best medical imaging method for diagnosing primary colorectal cancer in radiotherapy planning.

[0003] With the development of deep learning algorithms and corresponding hardware, in 2014, Long et al. replaced all fully connected layers in the CNN structure with fully convolutional layers, proposing the Fully Convolutional Neural Network (FCN) capable of semantic image segmentation. This demonstrated that deep neural networks can perform semantic segmentation on non-fixed-size images in an end-to-end manner. In 2015, Ronneberger proposed the UNet model, which used an encoder-decoder architecture. The encoding process extracted image features, and the decoding process reconstructed the image, achieving state-of-the-art performance in segmentation competitions at the time. Also in 2015, He proposed the ResNet model, which used residual connections to address the performance degradation problem of deep neural networks, making them easier to train. In 2016, Fausto Milletari proposed the VNet model, which improved upon UNet to enable training on 3D images, proposed Diceloss as the loss function, and incorporated residual connections.

[0004] This invention presents a UNet segmentation network with branch convolutional modules and attention modules, which accelerates the convergence speed of the original network and achieves higher performance. During the inference phase, the branch convolutional modules can obtain a larger field of view, which not only greatly improves the inference speed but also effectively reduces the memory occupied by the model. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing 3D segmentation models and to propose a lesion segmentation system and method for colorectal cancer MRI images based on an improved UNet network.

[0006] In a first aspect, the present invention provides a method for segmenting lesions in colorectal cancer MRI images based on an improved UNet network for non-disease diagnostic purposes, the method comprising the following steps:

[0007] S1, Medical MRI Image Preprocessing

[0008] Acquire NIFTI format images of colorectal cancer MRI, resample and N4 bias field correct the NIFTI format images; extract region of interest (ROI) images from the resampled images based on the lesion area.

[0009] S2, Constructing the dataset

[0010] Based on the ROI image obtained from the preprocessing in step S1, data augmentation is performed to generate the dataset used for model training.

[0011] The data augmentation process includes random noise, random blurring, random affine transformation, random elastic deformation, and random bias field.

[0012] S3, Network Setup and Training

[0013] The improved UNet model includes an encoding part and a decoding part, and a skip connection is used between the encoding part and the decoding part to enable the decoding part to obtain the feature map of the encoding part.

[0014] The encoding part includes five encoding modules, each of which includes two branch convolutional modules and one attention module. The first four encoding modules are amplified through four channels. The shallower layers capture local information, while the deeper layers have a much wider receptive field, thus capturing global information.

[0015] The decoding part includes a four-layer decoding module. The four-layer decoding module reduces the channel four times. Each layer of the decoding module merges the feature map after channel reduction with the feature map of the same size in the encoding part, and then passes through two branch convolution modules and one attention module in sequence.

[0016] The method for reducing channels in each decoding module layer is to use a 2x2x2 deconvolution layer with a stride of 2.

[0017] The branch convolution module includes a branch convolutional layer, a normalization layer, and a fully connected layer. The branch convolutional layer includes five parallel channels, where the first channel is a 3x3x3 depthwise convolutional layer, the second channel is an 11x1x1 depthwise convolutional layer, the third channel is a 1x11x1 depthwise convolutional layer, the fourth channel is a 1x1x11 depthwise convolutional layer, and the fifth channel is an identity mapping layer. Finally, the outputs of the five parallel channels are merged by channel. The normalization layer is a Batchnorm layer. The fully connected layer includes a convolutional layer, a ReLU layer, and another convolutional layer connected in sequence.

[0018] The five parallel channels described herein are computationally more efficient than large-kernel depthwise convolutions such as 7x7x7 convolutions in practice, while requiring fewer parameters. Compared to the original UNet network architecture, it has a larger spatial receptive field and better performance, showing improvements in both practical speed and performance.

[0019] The specific details of the branched convolutional layer are as follows:

[0020] MixLayer(X) = Concat(Conv 3x3x3 (X),Conv 11x1x1 (X),Conv 1x11x1 (X),Conv 1x1x11 (X),X) Equation (1)

[0021] Where X is the input feature map, Conv is the depthwise convolution operation, the subscript of Conv represents the size of the convolution kernel, Concat is the channel-wise merging operation, and MixLayer is the output of the branch convolutional layer.

[0022] The specific details of the branched convolution module are as follows:

[0023] X'=MLP(Norm(MixLayer(X)))+X Formula (2)

[0024] Norm is the normalization layer, MLP is the fully connected layer, and X' is the output of the branch convolutional module.

[0025] The attention module comprises a channel attention layer and a spatial attention layer; the specific implementation is as follows:

[0026] ① The channel attention layer includes a max pooling layer, a flat pooling layer, and a shared-parameter MLP layer. It generates a channel attention map by utilizing the inter-channel relationships of features, aggregating global spatial information. Specifically, it performs max pooling followed by MLP operations on the input feature map, and then average pooling followed by MLP operations. The two MLP operations share parameters. The results of the two processes are then added together, and finally processed using the Sigmoid activation function and multiplied with the original input feature map Y to obtain the result of the channel attention layer.

[0027] CA(Y)=σ(MLP(MaxPool(Y))+MLP(AvgPool(Y))) Formula (3)

[0028] Where CA represents the channel attention layer, MaxPool represents the max pooling layer, AvgPool represents the average pooling layer, σ is the sigmoid activation function, Y represents the output of the two concatenated branch convolutional modules of the current layer, and the MLP layer consists of a 1x1x1 convolutional layer with 16 channels reduced, a ReLU activation layer, and a 1x1x1 convolutional layer with 16 channels expanded.

[0029] ②The spatial attention layer includes a max pooling layer, a flat pooling layer, and a convolutional layer, which are used to extract location information;

[0030] The output Z of the channel attention layer is average pooled and max pooled separately and then merged by channel. The merged feature map is passed through a convolutional layer with 7x7x7 kernels with half the number of channels, and then through a Sigmoid layer. The resulting output feature map is multiplied by the output Z of the current channel attention layer.

[0031] SA(Z)=σ(Conv(Concat(MaxPool(Z),AvgPool(Z))) Formula (4)

[0032] Where SA represents the spatial attention layer, Z represents the output of the channel attention layer, MaxPool represents the max pooling layer, AvgPool represents the average pooling layer, Concat is the channel merging operation, Conv is the convolutional layer, and σ is the sigmoid activation function.

[0033] The entire attention module infers the attention map sequentially along two independent dimensions (channel and space), and then multiplies the attention map by the input feature map for adaptive feature refinement, generating a more refined feature map. This module achieves a significant performance improvement while maintaining low overhead.

[0034] S4. Using the trained improved UNet model, predict the colorectal cancer lesion area to obtain the image segmentation result.

[0035] In a second aspect, the present invention provides a lesion segmentation system for colorectal cancer MRI images, comprising:

[0036] The data acquisition and preprocessing module is used to preprocess medical MRI images to obtain images of the region of interest.

[0037] The segmentation module uses a trained and validated improved UNet network to predict colorectal cancer lesion regions and obtain image segmentation results.

[0038] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the system.

[0039] Fourthly, the present invention provides a computing device, including a memory and a processor, wherein the memory stores executable code, and the processor executes the executable code to implement the system.

[0040] The beneficial effects of this invention are:

[0041] 1. The data preprocessing module performs enhancement operations on the NIFTI format data of MRI for network training, thereby improving the image segmentation effect.

[0042] 2. By introducing a branched convolution module into the UNet network, the speed of algorithm application is greatly improved, the number of network parameters is reduced, and the final segmentation accuracy is improved, which solves the shortcomings of the original network such as slow convergence speed and low segmentation accuracy.

[0043] 3. Introducing an attention module into the UNet network can effectively learn and utilize information from the target object region and aggregate features from it, solving the problem of information not being shared between channels, refining the transmission of feature maps in the network, and further improving the accuracy of the network. Attached Figure Description

[0044] Figure 1 This is a flowchart of the method of the present invention;

[0045] Figure 2 This is a diagram showing the main structure of the improved UNet network proposed in this invention;

[0046] Figure 3 This is a diagram showing the main structure of the branch convolution module in the segmentation model proposed in this invention.

[0047] Figure 4 This is a diagram showing the main structure of the attention module in the segmentation model proposed in this invention. Detailed Implementation

[0048] The invention will be further described below with reference to the accompanying drawings.

[0049] Please refer to Figure 1 The diagram shows a flowchart of a method for segmenting lesions in colorectal cancer MRI images based on an improved UNet network, which includes the following steps:

[0050] S1. First, acquire NIFTI format images of colorectal cancer MRI, and resample and perform N4 bias field correction on the NIFTI format images; then, extract the region of interest (ROI) image from the resampled image based on the lesion region.

[0051] S11. Obtain the original NIFTI format three-dimensional colorectal cancer MRI image and the manually delineated lesion mask, and resample it to voxel size [1,1,1] and N4 bias field correction using the SimpleITK package.

[0052] S12. Write an algorithm to calculate the center of the lesion area outlined in the mask, and crop the region of interest image based on the center coordinates.

[0053] S2. Based on the ROI image obtained from the preprocessing in step S1, perform data augmentation to generate the dataset used for model training. Specifically:

[0054] Data augmentation methods are used to enhance ROI images, including random noise, random blur, random affine transformation, random elastic deformation, and random bias field. Voxel intensity is then normalized to between 0 and 1 for network training.

[0055] S3, Network Setup and Training:

[0056] The improved UNet model has the following network structure: Figure 2 As shown, the network structure mainly includes the encoding and decoding parts of the skip connections.

[0057] The encoding part is the process of reducing the image size and increasing the number of channels. After the image enters the network, the number of channels is increased four times. Each encoding process includes two branch convolutional modules, an attention module, and a downsampling module with a 2x2x2 convolutional kernel and a stride of 2.

[0058] The decoding part is a process of increasing the image size and decreasing the number of channels, aiming to restore the extracted features to their original size. There are four restoration processes in total. Each decoding process needs to be combined with the feature map extracted in the encoding part, which helps to improve network performance.

[0059] Structure reference of branch convolution module Figure 3 The branch convolutional layer of this module consists of five paths, the normalization layer is a Batchnorm layer, and the fully connected layers are a 3x3x3 convolutional layer, a ReLU layer, and a 3x3x3 convolutional layer in sequence.

[0060] The specific implementation of the structured branch convolution module is as follows:

[0061] ① Given an input feature map X, the branch convolution module includes branch convolutional layers, normalization layers, and fully connected layers. Each branch convolutional layer has five parallel channels: the first channel is a 3x3x3 depthwise convolutional layer, the second channel is an 11x1x1 depthwise convolutional layer, the third channel is a 1x11x1 depthwise convolutional layer, the fourth channel is a 1x1x11 depthwise convolutional layer, and the fifth channel is an identity mapping layer. Finally, the outputs of these five parallel channels are merged by channel. The normalization layer is a Batchnorm layer. The fully connected layer includes sequentially connected convolutional layers, ReLU layers, and another convolutional layer. In practice, the five parallel branches of the branch convolution are computationally more efficient than large-kernel depthwise convolutions such as 7x7x7 convolutional kernels, while also requiring fewer parameters. Compared to the original UNet network architecture, it has a larger spatial receptive field and better performance, showing improvements in both speed and performance.

[0062] The specific details of the branched convolutional layer are as follows:

[0063] MixLayer(X) = Concat(Conv 3x3x3 (X),Conv 11x1x1 (X),Conv 1x11x1 (X),Conv 1x1x11 (X),X)Equation (1)

[0064] Where X is the input feature map, Conv is the depthwise convolution operation, the subscript of Conv represents the size of the convolution kernel, Concat is the channel-wise merging operation, and MixLater is the output of the branch convolutional layer.

[0065] The specific details of the branched convolution module are as follows:

[0066] X'=MLP(Norm(MixLayer(X)))+X Formula (2)

[0067] Norm is the normalization layer, MLP is the fully connected layer, and X' is the output of the branch convolutional module.

[0068] The attention module includes a channel attention layer and a spatial attention layer in sequence; such as Figure 4 The specific implementation is as follows:

[0069] ① The channel attention layer consists of a max pooling layer, a flat pooling layer, and a shared parameter MLP layer.

[0070] Channel attention maps are generated by leveraging the inter-channel relationships of features, thus aggregating global spatial information.

[0071] CA(Y)=σ(MLP(MaxPool(Y))+MLP(AvgPool(Y))) Formula (3)

[0072] Where CA represents the channel attention layer, MaxPool represents the max pooling layer, AvgPool represents the average pooling layer, σ is the sigmoid activation function, and Y represents the output of the two concatenated branch convolutional modules in the current layer. The MLP layer consists of a 1x1x1 convolutional layer with 16 channels reduced, a ReLU activation layer, and a 1x1x1 convolutional layer with 16 channels expanded. The channel attention layer performs max pooling followed by MLP operations and average pooling followed by MLP operations on the input feature map, sharing parameters between the two MLP operations. The results of the two processes are then added together, and finally processed with the sigmoid activation function and multiplied with the original input feature map Y to obtain the result of the channel attention layer.

[0073] ② The spatial attention layer consists of a max pooling layer, a flat pooling layer, and a convolutional layer. It extracts location information.

[0074] SA(Z)=σ(Conv(Concat(MaxPool(Z),AvgPool(Z))) Formula (4)

[0075] In this diagram, SA represents the spatial attention layer, Z represents the result of the channel attention layer, MaxPool represents the max pooling layer, AvgPool represents the average pooling layer, Concat represents the channel-wise merging operation, Conv represents the convolutional layer, and σ represents the sigmoid activation function. The output of the channel attention layer serves as the input to the current layer. The spatial attention layer performs average pooling and max pooling on the input feature map and then merges them by channel. The merged feature map is then passed through a convolutional layer with 7x7x7 kernels (half the number of channels), and then through a sigmoid layer. The resulting output feature map is multiplied by the input feature map Z of the current layer, and the result is used as the output of the entire module.

[0076] The entire attention module infers the attention map sequentially along two independent dimensions (channel and space), and then multiplies the attention map by the input feature map for adaptive feature refinement, generating a more refined feature map. This module achieves a significant performance improvement while maintaining low overhead.

[0077] Model training parameters are set as follows: BatchSize is set to 1, AdamW optimizer is used, and learning rate is 0.001. The loss function is set to the sum of DICE loss and cross-entropy loss. Training is performed for 350 epochs.

[0078] S4. Using the trained improved UNet model, predict the colorectal cancer lesion area to obtain the image segmentation result.

[0079] The test data is input into the trained network to generate mask images of colorectal cancer lesion areas.

[0080] In summary, this invention relates to the field of medical image segmentation technology, and segments colorectal cancer lesion regions using deep learning methods. To address the shortcomings of slow model training convergence speed, low accuracy, and slow inference process with high memory consumption, an improved UNet network colorectal cancer lesion region segmentation method and system based on branched convolutional modules and attention modules are proposed. The following is a comparison of the final Dice similarity coefficients of the original UNet network and the improved UNet network on the same dataset:

[0081] UNet 0.723 Improve UNet 0.752

Claims

1. A lesion segmentation system for colorectal cancer MRI images, characterized in that... include: The data acquisition and preprocessing module is used to preprocess medical MRI images to obtain images of the region of interest. The segmentation module uses a trained and validated improved UNet network to predict colorectal cancer lesion regions and obtain the image segmentation results. The improved UNet network includes an encoding part and a decoding part, with skip connections between the encoding part and the decoding part. The encoding part includes five encoding modules, each of which includes two branch convolutional modules and one attention module. The decoding part includes four decoding modules. Each decoding module merges the feature map after reducing the number of channels with the feature map of the same size as the encoding part, and then passes it through two branch convolutional modules and one attention module in sequence. The branch convolution module includes a branch convolutional layer, a normalization layer, and a fully connected layer. The branch convolutional layer has five parallel channels: the first channel is a 3x3x3 depthwise convolutional layer, the second channel is an 11x1x1 depthwise convolutional layer, the third channel is a 1x11x1 depthwise convolutional layer, the fourth channel is a 1x1x11 depthwise convolutional layer, and the fifth channel is an identity mapping layer. Finally, the outputs of the five parallel channels are merged by channel. Specifically, the branch convolutional layer is: ( (X), (X), (X), (X),X) Equation (1) Where X is the input feature map. For depthwise convolution operations, For channel merging, This is the output of the branched convolutional layer; The branched convolutional module is specifically: Equation (2) Norm is the normalization layer, and MLP is the fully connected layer. This is the output of the branched convolution module; The attention module includes a channel attention layer and a spatial attention layer.

2. The system according to claim 1, characterized in that... The channel attention layer includes a max pooling layer, a flat pooling layer, and a shared parameter MLP layer. It generates a channel attention map by utilizing the inter-channel relationships of features and aggregating global spatial information. The input feature map is subjected to max pooling followed by MLP layer operations and average pooling followed by MLP layer operations, with the two MLP layer operations sharing parameters. The results of the two processing are then added together, and finally processed with the Sigmoid activation function and multiplied with the original input feature map Y to obtain the result of the channel attention layer.

3. The system according to claim 2, characterized in that... The channel attention layer is specifically: Equation (3) in This indicates the channel attention layer. Indicates the max pooling layer. Indicates the average pooling layer. It is the Sigmoid activation function. This represents the output of the two concatenated branch convolutional modules in the current layer.

4. The system according to claim 1, characterized in that... The spatial attention layer includes a max pooling layer, an average pooling layer, and a convolutional layer, used to extract positional information. It merges the output Z of the channel attention layer by average pooling and max pooling, and then passes the merged feature map through a convolutional layer with 7x7x7 convolutional kernels with half the number of channels, and then through a sigmoid layer. The resulting output feature map is multiplied by the output Z of the current channel attention layer.

5. The system according to claim 4, characterized in that... The spatial attention layer is specifically: Equation (4) in Represents the spatial attention layer. This represents the output of the channel attention layer. Indicates the max pooling layer. Indicates the average pooling layer. t represents the merge operation by channel. It is a convolutional layer. This is the Sigmoid activation function.

6. An image lesion segmentation method using the system described in any one of claims 1-5, characterized in that... The method includes the following steps: S1. Medical MRI image preprocessing to obtain images of the region of interest; S2, Constructing the dataset Based on the region of interest image obtained from the preprocessing in step S1, data augmentation processing is performed; S3. Build an improved UNet network and train and validate it using the dataset; S4. Using the trained improved UNet network, predict the colorectal cancer lesion area to obtain the image segmentation result.

7. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the system of any one of claims 1-5.

8. A computing device comprising a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the system of any one of claims 1-5.

Images