A multi-organ intelligent segmentation method and device based on an Eff-Unet-SE network

Abstract

The application discloses a kind of multi-organ intelligent segmentation method and device based on Eff-Unet-SE network, belong to medical image processing technical field.The method includes: obtaining MRI image dataset and label text data, and it is preprocessed;The image dataset after the pre-processing is carried out data division and data enhancement;Eff-Unet-SE network is constructed, and it is trained, verified and tested using the image dataset and label text data, to obtain optimal network;The MRI image data to be segmented is input to the optimal network, to obtain segmentation result.The application is analyzed to medical MRI image, on the basis of U type network model, in combination with the Efficient and SE network module of deep learning, can accurately separate out large intestine, small intestine and stomach, improve the reliability of segmentation result, also improve segmentation efficiency, greatly reduce the work burden of doctor.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a multi-organ intelligent segmentation method and device based on the Eff-Unet-SE network. Background Technology

[0002] According to the 2020 global cancer data released by the International Agency for Research on Cancer (IARC) of the World Health Organization, there were 1.089 million new cases of gastric cancer and 768,000 deaths worldwide in 2020. Among these, China had 478,000 new cases and 373,000 deaths from gastric cancer, demonstrating that gastric cancer has become a serious threat to human health. With the development of imaging technology, radiotherapy has gradually become an important treatment method for gastric cancer.

[0003] In radiotherapy, before delivering a high dose of radiation using X-rays directed at a gastric tumor, doctors need to pre-segment the stomach and surrounding large and small intestines in the imaging to prevent vital organs from being exposed to radiation. However, the number of MRI images of a patient's complete upper abdomen can range from dozens to hundreds, and a single patient may undergo approximately three MRI scans at different times. Faced with this massive amount of data, manual segmentation by doctors often requires a significant amount of effort and time, greatly increasing their workload and reducing diagnostic efficiency. Furthermore, existing segmentation methods are not very precise, sometimes failing to distinguish the boundaries between organs and tissues, and their segmentation effect on small organs is also unsatisfactory.

[0004] Therefore, given the demanding nature of this work, how to provide an automated method for analyzing MRI images and returning corresponding results to provide doctors with references and assist in treatment has become one of the urgent problems to be solved by technicians in this field. Summary of the Invention

[0005] In view of the technical problems existing in the prior art, the present invention provides a multi-organ intelligent segmentation method based on Eff-Unet-SE network. Its purpose is to reduce the cost for doctors to pre-annotate important organs such as the large intestine, small intestine and stomach of patients during gastric cancer radiotherapy, effectively improve the segmentation accuracy, and also have a good segmentation effect on small organs around the stomach.

[0006] To achieve the above and other related objectives, this invention provides a multi-organ intelligent segmentation method based on the Eff-Unet-SE network, comprising the following steps:

[0007] (1) Obtain the MRI image dataset and label text data, and preprocess them;

[0008] (2) Perform data partitioning and data augmentation on the preprocessed image dataset;

[0009] (3) Construct the Eff-Unet-SE network: Use the U-shaped network as the baseline model of the main model network, and use Efficient Net as the backbone to replace its encoder part; add the compression and excitation network SE to its decoder part to play the role of attention mechanism;

[0010] (4) Use the image dataset and label text data to train, validate and test the Eff-Unet-SE network to obtain the optimal Eff-Unet-SE network;

[0011] (5) Input the MRI image data to be segmented into the optimal Eff-Unet-SE network to obtain the segmentation result.

[0012] Furthermore, the preprocessing includes:

[0013] The MRI images were converted into PNG format images and then anonymized, with personal information removed.

[0014] The label text data is converted to obtain PNG format label images of the large intestine, small intestine, and stomach;

[0015] The processed MRI image and the labeled image are combined to obtain the corresponding NPY format file.

[0016] Furthermore, the data partitioning and data augmentation include:

[0017] The preprocessed dataset is divided into a training set, a validation set, and a test set, with a ratio of 8:1:1.

[0018] The training set data is augmented, including horizontal flipping, translation and scaling, and random discarding of rectangular regions. The validation set and test set are not augmented.

[0019] Furthermore, the Efficient Net consists of 7 blocks, each block being composed of a different number of shift-inverted bottleneck convolutional modules (MBConv); the MBConv structure is composed of a 1x1 ordinary convolution, a kxk DepthwiseConv convolution, an SE module, a 1x1 ordinary convolution, and a Dropout layer connected in sequence.

[0020] Furthermore, in the compression and excitation networks SE, the compression network plays the role of global pooling, and the excitation network adjusts the number of channels by scaling parameters to reduce the computational load. The combination of the two can achieve the effect of an attention mechanism on the image.

[0021] Furthermore, the training, validation, and testing process for the Eff-Unet-SE network includes:

[0022] (4.1) Input the training set data into the Eff-Unet-SE network, and update and adjust the hyperparameters in the network using the training set data and the loss function;

[0023] (4.2) Use the validation set data to validate the trained model and save the Eff-Unet-SE network parameters with the best validation results;

[0024] (4.3) Load the best network parameters from the verification process into the Eff-Unet-SE network, and send the test set data into the network for prediction and quality assessment.

[0025] Furthermore, the loss function of the Eff-Unet-SE network uses BCE Loss, and its calculation formula is as follows:

[0026] BCEloss(x i ,y i )=-w i [y i logx i +(1-y i log(1-x) i )]

[0027] Where: BCEloss refers to binary cross-entropy loss, x i y represents the value of a pixel in an image predicted by the network. i w represents the pixel value of the actual label image. i This indicates that the weights are scaled to the loss.

[0028] Furthermore, the quality assessment uses the Dice similarity coefficient and IOU as indicators, which are calculated as follows:

[0029]

[0030]

[0031] Where: TP is the true positive, representing the area of ​​intersection between the network prediction result and the actual image label; FP is the false positive, representing the area of ​​the actual image label after removing the TP result; FN is the false negative, representing the area of ​​the network prediction result after removing the TP result.

[0032] Furthermore, after step (3), the method further includes: evaluating the quality of the optimal Eff-Unet-SE network using an ablation experiment, and verifying the overall effectiveness of the model by controlling variables.

[0033] This invention also provides a multi-organ intelligent segmentation device based on the Eff-Unet-SE network, comprising:

[0034] Data acquisition unit: used to acquire MRI image datasets and label text data;

[0035] Data processing unit: used to preprocess the MRI image dataset and label text data, and to perform data segmentation and data augmentation on the preprocessed image dataset;

[0036] Training unit: Used to construct the Eff-Unet-SE network, and to train, validate and test the Eff-Unet-SE network using the image dataset and labeled text data to obtain the optimal Eff-Unet-SE network.

[0037] The beneficial effects of this invention are:

[0038] 1. This invention enables intelligent segmentation of the large intestine, small intestine, and stomach on MRI images without requiring excessive intervention from doctors. This reduces the time doctors need to manually segment surrounding organs for gastric cancer radiotherapy, thereby greatly alleviating their workload.

[0039] 2. This invention, through the analysis of medical MRI images, and based on a U-shaped network model, combines the Efficient and SE network modules of deep learning to greatly improve segmentation efficiency, accurately separate the large intestine, small intestine and stomach, thereby improving the reliability of the segmentation results. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating the multi-organ intelligent segmentation method based on the Eff-Unet-SE network according to an embodiment of the present invention.

[0041] Figure 2 This is a schematic diagram illustrating the conversion between label text and label image according to an embodiment of the present invention.

[0042] Figure 3 This is a schematic diagram illustrating the data enhancement effect of an embodiment of the present invention.

[0043] Figure 4 This is a schematic diagram of the Eff-Unet-Se network model structure according to an embodiment of the present invention.

[0044] Figure 5 This is a schematic diagram of the EfficientNet structure according to an embodiment of the present invention.

[0045] Figure 6This is a schematic diagram of the structure of the mobile inverted bottleneck convolution (MBConv) module according to an embodiment of the present invention.

[0046] Figure 7 This is a schematic diagram of the Squeeze and Excitation Networks (SE) structure according to an embodiment of the present invention.

[0047] Figure 8 These are the loss changes of the binary cross-entropy of the training set and validation set, and the evaluation metrics of Dice and IOU for the validation set in this embodiment of the invention.

[0048] Figure 9 The Dice and IOU evaluation metrics are used for the validation set in the ablation experiments of this invention. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be described in complete and detailed below with reference to the accompanying drawings. The embodiments described below are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. The illustrations only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0051] This invention provides a multi-organ intelligent segmentation method based on the Eff-Unet-SE network. It achieves segmentation of the large intestine, small intestine, and stomach on MRI images, and can autonomously analyze medical images related to gastric cancer without excessive intervention from doctors, thus significantly reducing their workload. This invention, through the analysis of medical MRI images, uses a U-shaped network as a baseline and combines deep learning's EfficientNet and SEnet to quickly segment MRI images with high accuracy. The invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] like Figure 1 As shown, this embodiment of the invention provides a multi-organ segmentation method based on MRI images, including the following steps:

[0053] S101. Obtain the MRI image dataset and label text data, and preprocess them;

[0054] MRI image data is acquired to obtain an MRI image dataset. First, anonymization processing is required to remove patient privacy information contained in the MRI medical files. Then, the DICOM format MRI medical files are converted into 0-255 RGB visual PNG images to facilitate model training.

[0055] To reduce the space occupied by the label images and facilitate storage, this method uses run-length encoding to text-encode the large intestine, small intestine, and stomach, such as... Figure 2 As shown in Table 1, to restore the image to its original black and white state, the text needs to be decoded.

[0056] Table 1. Label Image Conversion Flowchart

[0057]

[0058]

[0059] S102. Perform data partitioning and data augmentation on the preprocessed image dataset;

[0060] To train and validate a model, the data needs to be divided into three sets: a training set, a validation set, and a test set. The training set helps train the model by updating and adjusting its hyperparameters using the training data and loss function. The validation set is used for model selection, specifically for final optimization and parameter selection. The test set is used to test the accuracy of the trained model. Because the parameters are adjusted and fitted based on the existing training data during training, overfitting may occur. This means the parameters may only fit the training data accurately; if new data is introduced and the model is used to predict the result, the accuracy may be poor.

[0061] In this invention, the ratio of the training set, validation set, and test set is 8:1:1. It should be noted that only 60% of the MRI images have labels for the large intestine, small intestine, and stomach; unlabeled images will not be used for training.

[0062] Deep learning in medical image processing still faces considerable challenges. These include high annotation costs and limited, imbalanced medical image datasets, leading to underfitting or overfitting, ultimately impacting prediction performance. Data augmentation not only expands the dataset but also effectively improves model robustness. This invention utilizes translation, scaling, rotation, horizontal flipping, and Coarse Dropout for data augmentation. Specifically, Coarse Dropout involves discarding fixed-size, randomly positioned rectangular blocks within a certain number from the original image, setting these discarded areas to black to generate noise and achieve the dropout effect. This invention uses a random discarding of 5 to 8 blocks, with each discarded block being one-twentieth the size of the original image. Finally, translation, scaling, rotation, and horizontal flipping are randomly combined, as illustrated in the diagram. Figure 3 As shown.

[0063] S103. Construct the Eff-Unet-SE network: Use the U-shaped network as the baseline model of the main model network, and use Efficient Net as the backbone to replace its encoder part; add the compression and excitation network SE to its decoder part to play the role of attention mechanism;

[0064] The Eff-Unet-SE network model used in this invention uses a U-shaped network as the baseline network, EfficientNet as the backbone replacing the original encoder part of the U-shaped network, and SE is attached to the original decoder part of the U-shaped network to act as an attention mechanism. Its structure is as follows: Figure 4 As shown. It should be noted that the encoder performs downsampling to extract the spatial features of the original image, while the decoder performs upsampling to construct the image based on the spatial features extracted by the encoder.

[0065] EfficientNet consists of 7 blocks, and each block is composed of a different number of MBConv blocks, as shown in the following figure. Figure 5 As shown. The original UNet only uses two ordinary convolutional and pooling operations at each stage of the encoder to extract features. This feature extraction method easily leads to the model failing to extract all useful feature information, and some features may even be lost in the process. Replacing the original UNet encoder with EfficientNet improves UNet's feature extraction capabilities. The structure of MBConv is as follows. Figure 6As shown, the MBConv structure mainly consists of a 1x1 ordinary convolution (for dimensionality increase, including BN and Swish), a kxk Depthwise Conv convolution (including BN and Swish, with kxk having 3x3 and 5x5 cases), an SE module, a 1x1 ordinary convolution (for dimensionality reduction, including BN), and a Dropout layer. The SE module consists of a global average pooling layer and two fully connected layers. The first fully connected layer has 1 / 4 the number of nodes of the input MBConv feature matrix (channels) and uses the Swish activation function; the second fully connected layer has the same number of nodes as the output feature matrix (channels) of the Depthwise Conv layer and uses the Sigmoid activation function.

[0066] The decoder section, without altering the U-shaped network, adds compression and excitation modules to the upsampling module. The compression network acts as a global pooling mechanism, while the excitation network adjusts the number of channels through scaling parameters to reduce computational load. The combination of these two modules enables an attention mechanism for the image, such as... Figure 7 As shown in the diagram. Specifically, the module passes a feature map of size H×W×S×C in parallel through a global average pooling layer and a global max pooling layer, where H, W, S, and C represent the length, width, depth, and number of channels of the feature map, respectively. These two pooling layers compress the global spatial information of each channel into a tensor value, generating 1×C feature values. Then, they are subjected to 3D convolution operations with a kernel of 1×1×1 and the number of channels of C. The convolution results are passed through the non-linear activation function ReLU, and the outputs are then subjected to the same convolution operation again, finally obtaining two tensors of different values ​​but the same dimension of 1×1×1×C. After adding these two tensors, each value is normalized to the value range [0, 1] through a sigmoid layer. Multiplying this by the original feature matrix can significantly suppress information in unimportant channels while keeping the information in important channels almost unchanged, thereby indirectly improving the extraction of effective features. For image segmentation, pixel-wise spatial information provides more information, so an SE module is introduced in the decoder part of the U-shaped network, as shown in the figure. It can measure the importance of spatial location by compressing channel information, squeezing along the channels and exciting in space. For an input feature map, this module performs spatial compression through convolution, then normalizes it using sigmoid, and finally multiplies it with the original feature tensor.

[0067] S104. The Eff-Unet-SE network is trained, validated, and tested using the image dataset and labeled text data to obtain the optimal Eff-Unet-SE network.

[0068] (1) Input the training set data into the Eff-Unet-SE network, and update and adjust the hyperparameters in the network using the training set data and loss function;

[0069] To train the Eff-Unet-SE network, a loss function needs to be considered, which adjusts the model's parameters. This invention pertains to a multi-label task, so the activation function used is sigmoid instead of cross entropy, and the loss function is BCELoss (Binary Cross Entropy Loss), as shown in the following formula:

[0070] BCEloss(x i ,y i )=-w i [y i logx i +(1-y i log(1-x) i )]

[0071] Where: BCEloss refers to binary cross-entropy loss, x i y represents the value of a pixel in an image predicted by the network. i w represents the pixel value of the actual label image. i This indicates how the weights scale the loss; the default value is 1.

[0072] (2) Use the validation set data to validate the trained model and save the Eff-Unet-SE network parameters with the best validation results;

[0073] To evaluate the quality of segmentation results, the most commonly used metrics for medical image segmentation are the Dice Similarity Coefficient (DSC) and IOU (Intersection Over Union). Dice describes the similarity between the predicted and actual labels, with values ​​ranging from 0 to 1; a higher Dice indicates a better segmentation result. IOU describes the degree of overlap between the predicted and actual label locations, and its formula is shown below.

[0074]

[0075]

[0076] Where: TP is the true positive, representing the area of ​​intersection between the network prediction result and the actual image label; FP is the false positive, representing the area of ​​the actual image label after removing the TP result; FN is the false negative, representing the area of ​​the network prediction result after removing the TP result.

[0077] After the last ten iterations, the training error, validation error, Dice evaluation index, and IOU evaluation index are obtained. For example... Figure 8 As shown in AD. In the final tenth iteration, the loss function value in the training set was 0.07, and the loss function value in the validation set was 0.09. In the validation set, the Dice index was 0.91, and the IOU index was 0.88.

[0078] (3) Load the best network parameters from the verification process into the Eff-Unet-SE network, and send the test set data into the network for prediction and quality assessment.

[0079] To further evaluate the model's quality, ablation studies can be conducted based on it. Deep learning-based ablation studies investigate the impact of removing components from a deep network on the entire system. The aim is to understand the role of each component in the overall system by controlling for variables.

[0080] This invention uses a U-shaped network as a baseline. By removing individual components from Eff-Unet-Se and then applying the Dice and Iou metrics on a validation set, the effectiveness of the combined components is verified. Figure 9 AB. As can be seen from the graph, when the Eff module is removed, the initial values ​​of IOU and Dice are lower, while when the SE module is removed, the IOU and Dice metrics decrease slightly in the last iteration.

[0081] S104. Input the MRI image data to be segmented into the optimal Eff-Unet-SE network to obtain the segmentation result.

[0082] In summary, this invention provides a multi-organ intelligent segmentation method based on the Eff-Unet-SE network. This method improves upon the traditional U-shaped network, achieving automatic segmentation of the large intestine, small intestine, and stomach on MRI images. This reduces the time doctors spend manually segmenting surrounding organs when performing radiotherapy, significantly alleviating their workload. Finally, the evaluation results highlight the high accuracy and efficiency of this method.

[0083] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A multi-organ intelligent segmentation method based on Eff-Unet-SE network, characterized in that, Includes the following steps: (1) Obtain the MRI image dataset and label text data, and preprocess them; The preprocessing includes: The MRI images were converted into PNG format images and then anonymized, with personal information removed. The label text data is converted to obtain PNG format label images of the large intestine, small intestine, and stomach; The processed MRI image and the labeled image are combined to obtain the corresponding NPY format file; (2) Perform data partitioning and data augmentation on the preprocessed image dataset; (3) Constructing the Eff-Unet-SE network: Using the U-shaped network as the baseline model of the main model network, EfficientNet is used as the backbone to replace its encoder part; the compression and excitation network SE is attached to its decoder part to play the role of attention mechanism; The Efficient Net consists of 7 blocks, each block being composed of a different number of shift-inverted bottleneck convolutional modules MBConv; The MBConv structure consists of a 1x1 ordinary convolution, a kxk Depthwise Conv convolution, an SE module, a 1x1 ordinary convolution, and a Dropout layer connected in sequence. The addition of the compression and excitation network SE to its decoder part to serve as an attention mechanism includes: The decoder section, without altering the U-shaped network, adds a compression and activation network (SE) to the upsampling module. The compression network acts as a global pooling network, while the activation network adjusts the number of channels by scaling parameters to reduce computation. The combination of these two networks enables an attention mechanism for the image. Specifically, a feature map of size H×W×S×C is passed in parallel through a global average pooling layer and a global max pooling layer, where H, W, S, and C represent the length, width, depth, and number of channels of the feature map, respectively. These two pooling layers compress the global spatial information of each channel into a tensor value, generating 1×C feature values. Then, they are subjected to 3D convolution operations with a kernel of 1×1×1 and the number of channels C. The convolution results are passed through the non-linear activation function ReLU, and the outputs are then subjected to the same convolution operation again, resulting in two tensors of different values ​​but the same dimension, 1×1×1×C. These two tensors are added together and then normalized to the value range [0, 1] through a sigmoid layer before being multiplied by the original feature matrix. (4) Use the image dataset and labeled text data to train, validate and test the Eff-Unet-SE network to obtain the optimal Eff-Unet-SE network; (5) Input the MRI image data to be segmented into the optimal Eff-Unet-SE network to obtain the segmentation result.

2. The multi-organ intelligent segmentation method based on Eff-Unet-SE network according to claim 1, characterized in that, The data partitioning and data augmentation include: The preprocessed dataset is divided into a training set, a validation set, and a test set, with a ratio of 8:1:

1. The training set data is augmented, including horizontal flipping, translation and scaling, and random discarding of rectangular regions. The validation set and test set are not augmented.

3. The multi-organ intelligent segmentation method based on Eff-Unet-SE network according to claim 2, characterized in that, The training, validation, and testing process for the Eff-Unet-SE network includes: (4.1) Input the training set data into the Eff-Unet-SE network, and update and adjust the hyperparameters in the network using the training set data and the loss function; (4.2) Use the validation set data to validate the trained model and save the Eff-Unet-SE network parameters with the best validation results; (4.3) Load the best network parameters from the verification process into the Eff-Unet-SE network, and send the test set data into the network for prediction and quality assessment.

4. The multi-organ intelligent segmentation method based on Eff-Unet-SE network according to claim 3, characterized in that: The loss function of the Eff-Unet-SE network uses BCE Loss, and its calculation formula is as follows: Where: BCEloss refers to binary cross-entropy loss, xi represents the pixel value of the network-predicted image, yi represents the pixel value of the actual labeled image, and wi represents the weight scaling of the loss.

5. The multi-organ intelligent segmentation method based on Eff-Unet-SE network according to claim 3, characterized in that: The quality assessment uses the Dice similarity coefficient and IOU as indicators, which are calculated as follows: Where: TP is the true positive, representing the area of ​​intersection between the network prediction result and the actual image label; FP is the false positive, representing the area of ​​the actual image label after removing the TP result; FN is the false negative, representing the area of ​​the network prediction result after removing the TP result.

6. The multi-organ intelligent segmentation method based on Eff-Unet-SE network according to claim 1, characterized in that, Step (3) is followed by: evaluating the quality of the optimal Eff-Unet-SE network using ablation experiments and verifying the overall effectiveness of the model by controlling variables.

7. A multi-organ intelligent segmentation device based on an Eff-Unet-SE network, used to execute the multi-organ intelligent segmentation method based on an Eff-Unet-SE network as described in any one of claims 1-6, characterized in that, include: Data acquisition unit: used to acquire MRI image datasets and label text data; Data processing unit: used to preprocess the MRI image dataset and label text data, and to perform data segmentation and data augmentation on the preprocessed image dataset; Training unit: Used to construct the Eff-Unet-SE network, and to train, validate and test the Eff-Unet-SE network using the image dataset and labeled text data to obtain the optimal Eff-Unet-SE network.

Images