A global-local cascade based magnetic resonance image cervical cancer tumor segmentation method

Abstract

The application discloses a kind of based on global local cascade's magnetic resonance image cervical cancer tumor segmentation method, belong to medical image processing and computer vision field, the characteristics of this method are for the fusion of global feature and local feature, solve small lesion segmentation problem, specifically includes: training data set preprocessing;Establish global local cascade's magnetic resonance image cervical cancer tumor segmentation network model;Model inference, obtain lesion segmentation result and the like steps.Compared with prior art, the global feature and local feature of the image are considered and fused simultaneously, and the problem of precise segmentation of small cervical cancer tumors on magnetic resonance images under limited computing resources is effectively solved.

Description

Technical Field

[0001] This invention relates to the field of magnetic resonance imaging technology, and is a method for segmenting cervical cancer lesions in magnetic resonance imaging based on global-local cascading. Background Technology

[0002] Cervical cancer is the second most common cancer worldwide, affecting nearly 500,000 women annually and causing over 300,000 deaths. Magnetic resonance imaging (MRI) is the best screening tool for assessing early-stage cervical cancer treatment. Segmenting cervical lesions on MRI is a crucial step in the diagnosis of cervical cancer. Based on the marked lesion areas, doctors can accurately analyze and calculate the radiation dose, providing guidance for subsequent surgery.

[0003] However, magnetic resonance imaging (MRI) is affected by factors such as field strength, scan thickness, and field of view, which can lead to low signal-to-noise ratios and unclear lesion edges. Therefore, to help doctors improve diagnostic efficiency and reduce workload, developing accurate and repeatable automatic segmentation algorithms to detect cancer lesions is an important research direction in smart healthcare.

[0004] Since the development of deep learning, methods represented by convolutional neural networks have been widely applied in various fields of computer vision, and various network models based on encoder-decoder structures have emerged for magnetic resonance imaging (MRI) image segmentation. These methods do not differentiate between the targets to be detected, but the cervical cancer lesions to be segmented are very small, and the differences between slices are significant, making it difficult for these methods to achieve satisfactory results. Furthermore, some methods, considering the three-dimensional characteristics of MRI images, directly use 3D convolutional networks to improve segmentation performance, but this has significant drawbacks, such as high computational cost, difficulty in optimization, and poor real-time performance. Moreover, since the three-dimensional region of the lesion also occupies a small portion of the original image, it is difficult to divide it into more sub-images containing the tumor during preprocessing. Therefore, designing a deep convolutional neural network to segment cervical cancer lesions in MRI images is challenging. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by proposing a method for cervical cancer lesion segmentation based on global-local cascading in magnetic resonance imaging. This method is based on a medical image semantic segmentation neural network model with an encoder-decoder architecture. It employs a global-local cascading strategy, inputting the image to be processed into a trained model for cervical cancer tumor segmentation. The model outputs precisely segmented cervical cancer tumor regions, effectively solving the problem of inaccurate segmentation of small target lesions. This facilitates accurate analysis and radiation dose calculation during medical diagnosis, providing guidance for subsequent surgery.

[0006] The specific technical solution for achieving the objective of this invention is as follows:

[0007] A method for segmenting cervical cancer lesions using magnetic resonance imaging based on global-local cascading is characterized by the fusion of global and local features, specifically including the following steps:

[0008] Step 1: Preprocess the dataset;

[0009] Implemented on the CeTS magnetic resonance imaging dataset, it extracts slices from 3D magnetic resonance images, divides them into training and test sets, performs data augmentation on the images by random horizontal flipping, vertical flipping and scaling, and normalizes the data.

[0010] Step 2: Construct a neural network model based on global-local cascading;

[0011] Based on the PyTorch deep learning framework, a neural network model for cervical cancer tumor segmentation in magnetic resonance imaging was constructed, based on global-local cascading. The model is based on an encoder-decoder architecture, with the following modules cascaded into the encoder and decoder paths:

[0012] a) Global Feature Fusion Module: Input samples of size H×W×3, where H and W represent the height and width of the image, are sequentially passed through three 3×3 convolutional sub-blocks. Each convolutional sub-block contains a convolutional layer, a compression-activation block, and a ReLU activation function. Then, the importance of the feature map in the channel dimension is adjusted according to the attention value generated by the compression-activation block, so that the multi-scale feature map aggregates more global information. Then, the outputs are extracted from the three convolutional blocks and fused together to extract spatial features at different scales. Finally, 1×1 convolutional layers are introduced between the blocks to increase skip connections. The output of this module is a feature map of size H×W×C1, where C1 represents the number of intermediate channels.

[0013] b) Feature decomposition and recombination module: The input is the output F of the global feature fusion module. s0 Then, a 1×1 convolution is used to select the feature map F that needs local processing from the global feature map. s Then, using the eigenvalue decomposition method, F s The feature map is decomposed into four identical sub-blocks, each of which is processed using a convolution to obtain F1, F2, F3, and F4. Then, a feature reconstruction method is used to first concatenate the two adjacent blocks to obtain two feature maps F1, F2, F3, and F4. 12 and F 34 They are all half of the original input, and are applied to F using another convolutional layer. 12 and F 34 This allows for the extraction of more features within a larger pixel domain; for two adjacent blocks, the same operation is performed as for the adjacent blocks, resulting in the feature map F. 13 and F 24 Then splice F 12and F 34 Get F 1234 splicing F 13 and F 24 Get F 1324 and for F 1234 F 1324 Perform convolution; finally, F 1234 F 1324 The feature maps from the original input are concatenated along the channels and fused together using 1×1 convolutions;

[0014] The formula for the eigenvalue decomposition method is expressed as:

[0015] F1 = Conv(F s [0:C,0:H / 2+B,0:W / 2+B]),

[0016] F2=Conv(F s [0:C,0:H / 2+B,W / 2-B:W]),

[0017] F3 = Conv(F s [0:C,H / 2-B:H,0:W / 2+B]),

[0018] F4 = Conv(F s [0:C,H / 2-B:H,W / 2-B:W]),

[0019] Where the subscripts 1, 2, 3, and 4 of F represent the feature maps at the top left, top right, bottom left, and bottom right positions after partitioning, respectively, and Fs represents the output F of the global feature fusion module. s0 The result after performing a 1×1 convolution, where Conv represents the convolution operation, C, H and W represent the number of channels, height and width of the feature map, respectively, and B represents the length of the overlapping part of two adjacent sub-blocks;

[0020] The formula for the feature recombination method is expressed as follows:

[0021] F 12 =Conv(F1(s)F2),

[0022] F 34 =Conv(F3(s)F4),

[0023] F 13 =Conv(F1(s)F3),

[0024] F 24 =Conv(F2(s)F4),

[0025] Where (s) is the splicing operator, which represents splicing two adjacent feature maps in the spatial plane;

[0026] The formula for fusion is expressed as:

[0027] F 1234 =Conv(F 12 (s)F 34 ),

[0028] F 1324 =Conv(F 13 (s)F 24 ),

[0029] F out =Conv(F 1234 (c)F 1324 (c)F in ),

[0030] Where (c) represents the channel connection operator;

[0031] c) Channel spatial attention gating module: First, it utilizes decoder features... The relationship between different channels generates a channel attention map g c ∈R C×1×1 Then for g c Perform broadcast operations, extending to g c ∈R C×H×W and compare it with decoder feature x d Corresponding encoder feature x e ∈R C×H×W Perform pixel-by-pixel multiplication to obtain x c ∈R C×H×W Then use x c Spatial relationships generate spatial attention diagrams s s ∈R 1×H×W Finally, the spatial attention diagram s s With encoder feature x e Perform pixel-wise multiplication to obtain a feature map x that has undergone simultaneous attention in both the spatial domain and the channel direction. out ∈R C×H×W ;

[0032] Step 3: Train the network model;

[0033] The preprocessed training set is input into the neural network model for training. A loss function combining Dice loss and cross-entropy loss is used for constraint, and the loss function is expressed as:

[0034]

[0035]

[0036]

[0037] Where λ represents the balance factor between the two loss functions, which is set to 0.5, N represents the total number of pixels, C represents the number of channels in the prediction matrix, and p represents the pixel value labeled as a positive sample. This represents the probability that a pixel value is predicted to be a positive sample. This means performing a dot product operation between the true label of each channel and the matrix of the corresponding prediction result. This represents the square of the Euclidean norm operation; during the training process, a validation set is also defined, and the neural network model with the best performance on the validation set is stored.

[0038] Step 4: Division of cervical cancer lesions;

[0039] On the test set, the best network model saved in step 3 is used to output the visualization results of lesion segmentation.

[0040] Compared with the prior art, the present invention has the following beneficial technical effects:

[0041] 1) The constructed network model integrates global and local features of the image, ensuring that the features of small lesions are not lost during network propagation, thus achieving accurate segmentation of small target lesions;

[0042] 2) Using 2D slices as model input results in low computational cost and fast model inference speed;

[0043] 3) By cascading the global feature fusion module and the feature decomposition and recombination module, the fusion of multi-level image features is achieved, and a channel spatial attention gating mechanism is used during propagation to focus the output on specific features. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of the neural network model constructed in this invention;

[0045] Figure 2 This is a schematic diagram of the global feature fusion module;

[0046] Figure 3 This is a schematic diagram of the feature decomposition and recombination module;

[0047] Figure 4 This is a schematic diagram of the channel space attention gating module;

[0048] Figure 5 The image is a schematic diagram of the model experiment results for the example. Detailed Implementation

[0049] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0050] See Figure 1The present invention performs cervical cancer lesion segmentation according to the following steps:

[0051] Step 1: Preprocess the dataset

[0052] The method is mainly implemented on the CeTS magnetic resonance imaging dataset. It extracts slices from 3D magnetic resonance images, divides them into training and test sets, performs data augmentation on the images by random horizontal flipping, vertical flipping and scaling, and normalizes the data.

[0053] Step 2: Construct a neural network model for cervical cancer lesion segmentation based on global-local cascades in magnetic resonance imaging.

[0054] Based on the PyTorch deep learning framework, a neural network model for cervical cancer tumor segmentation in magnetic resonance imaging was constructed. The model consists of an encoder, a decoder, a global feature fusion module, a feature decomposition and recombination module, and a channel space attention gating mechanism.

[0055] Step 3: Training the network model

[0056] The preprocessed data samples from the training set are input into the neural network model for training. During the training process, a validation set is also defined, and the weights of the network model that performs best on the validation set are stored.

[0057] Step 4: Segmentation of cervical cancer lesions

[0058] On the test set, the best network model saved in step 3 is used for model inference, and the visualization results of lesion segmentation are output.

[0059] The neural network model constructed in step 2 consists of five parts: encoder, decoder, global feature fusion module, feature decomposition and recombination module, and channel space attention gating module.

[0060] The global feature fusion module and feature decomposition module are cascaded at each layer of the encoder to achieve the fusion of global and local features, while preserving both global and local features of the image and performing feature recombination.

[0061] The channel spatial attention gating module operates in the decoder stage. In each layer of the decoder, the channel spatial attention gating module is used to reduce the noise of the feature maps and make the feature maps focus on specific features.

[0062] Example

[0063] Step 1: Preprocess the dataset

[0064] This embodiment is implemented on the CeTS magnetic resonance imaging dataset. Slices are extracted from 3D magnetic resonance images, and training and testing sets are divided. Data augmentation of images is performed by random horizontal flipping, vertical flipping and scaling, and the data is normalized.

[0065] The CeTS MRI dataset selected in this embodiment is clinical data from an obstetrics and gynecology hospital. It includes 1800 sagittal T2-weighted samples, with detailed markings of neck tumor regions by experienced radiologists. Each 3D sample contains 20 to 35 2D slices, each slice being 256×256 pixels in size.

[0066] For each sample, all its slices were used, attributed to the physician's clinical diagnosis, as each slice could potentially influence the diagnosis of cervical cancer. Furthermore, unlike many other medical image segmentation datasets, most early cervical tumors in the CeTS dataset are very small. Therefore, the original image size was used directly without cropping or resampling to avoid altering or even eliminating the features of small targets after these operations. All 1800 samples were divided into training and test sets in a 7:3 ratio, and then the 3D samples were converted into 2D slices. This resulted in 7089 slices for training and 2972 ​​slices for testing. For each slice, it was cropped to between 0.5% and 99.5% based on the intensity value of the entire image. Subsequently, the mean and standard deviation of each channel were independently calculated within non-zero regions, and these means and standard deviations were used to normalize all 2D slices. No background normalization was performed because the background portion was too large, with pixel values ​​all being zero. Random horizontal flipping, vertical flipping, and scaling were also applied to the images for image enhancement. To accelerate the convergence of the model, the input is set to three channels: the middle channel is the image to be segmented, and the other two channels are slices of the middle channel image that are adjacent to each other in the original image.

[0067] Step 2: Construct a neural network model for cervical cancer lesion segmentation based on global-local cascades in magnetic resonance imaging.

[0068] See Figure 1 The cervical cancer lesion segmentation neural network model based on global-local cascade magnetic resonance imaging constructed in this invention mainly consists of five parts: encoder, decoder, global feature fusion module, feature decomposition and recombination module, and channel spatial attention gating module.

[0069] This embodiment uses an encoder-decoder architecture, and uses a global feature fusion module and a feature decomposition and reconstruction module on the encoder path, and a channel spatial attention gating module on the decoder path.

[0070] See Figure 2In each global feature fusion module, three 3×3 convolutional sub-blocks are used sequentially. Each sub-block contains a convolutional layer, a compression-activation block, and a ReLU activation function. The importance of the feature maps in the channel dimension is adjusted based on the attention values ​​generated by the compression-activation block, allowing multi-scale feature maps to aggregate more global information. The outputs are then extracted from the three convolutional blocks and fused together to extract spatial features at different scales. This series of three convolutional blocks simultaneously possesses receptive fields generated by 3×3, 5×5, and 7×7 convolutional operations, similar to the Inception block, but with fewer parameters and computational cost. Skip connections are then added between the blocks by introducing 1×1 convolutional layers.

[0071] See Figure 3 In each feature decomposition and recombination module, a 1×1 convolution is first used to select the feature map requiring local processing from the global feature map. This map is then decomposed into four identical sub-blocks, each processed using a convolution. Next, two adjacent blocks are concatenated to obtain two feature maps, each half the size of the original input. Another convolutional layer is then used to extract more features from the concatenated feature maps within a larger pixel domain. The same operation is applied to two sets of concatenated blocks. Finally, these two local feature maps are concatenated with the global features of the original input, and then fused using a 1×1 convolution.

[0072] See Figure 4 The channel spatial attention gating mechanism reduces noise response and focuses on specific features by integrating low-level feature information from the encoder and high-level feature information from the decoder. First, the inter-channel relationships of the decoder features are multiplied by the corresponding encoder features to obtain a channel attention map. Then, using the relationships between the output feature spaces from the previous step, the same encoder features are multiplied to generate a spatial attention map.

[0073] Step 3: Training the network model

[0074] This example is based on a PyTorch 1.8 environment and trained using a 24GB NVIDIA GeForce RTX 3090 graphics card. All models are trained from scratch, initialized using Kaiming, and optimized using the Adam optimizer. The initial learning rate is set to 0.0003, the batch size during training is set to 20, and the number of training epochs is set to 250. Furthermore, this example inputs three consecutive slices into the network together to learn the continuous information between slices. This example inputs preprocessed data into the neural network model for training, and during training, a validation set is created, and the weights of the network model that performs best on the validation set are saved.

[0075] This embodiment uses five common evaluation metrics—mean intersection-union ratio (mIoU), Dice coefficient (DSC), accuracy (Acc), sensitivity (Sen), and precision (Prec)—to quantify the image segmentation performance of the model. The higher the evaluation metric value, the better the network segmentation effect.

[0076] The average crossover ratio (mIoU) is calculated using the following formula:

[0077]

[0078] The Dice coefficient (DSC) is calculated using the following formula:

[0079]

[0080] The accuracy (Acc) is calculated using the following formula:

[0081]

[0082] The sensitivity (Sen) is calculated using the following formula:

[0083]

[0084] The precision (Prec) is calculated using the following formula:

[0085]

[0086] TP, TN, FP, and FN represent the positive examples correctly classified, the negative examples correctly classified, the positive examples incorrectly classified, and the negative examples incorrectly classified, respectively, in the prediction.

[0087] During training, the cross-entropy loss function is used to constrain the segmentation results at the pixel level, and the Dice loss function is used to constrain the segmentation results overall.

[0088] The cross-entropy loss function is defined as follows:

[0089]

[0090] The Dice loss function is defined as:

[0091]

[0092] The final loss function is expressed as:

[0093]

[0094] Where p represents the probability that each pixel in the image is labeled as a tumor. This represents the probability that each pixel in the segmentation result is predicted to be a tumor. This indicates that the labeled and predicted results are multiplied on the corresponding channels. λ represents the Euclidean norm operation, λ represents the balance factor between the two loss functions, which is set to 0.5 in this embodiment, N is the number of pixels, and C is the number of channels in the prediction result matrix.

[0095] Step 4: Segmentation of cervical cancer lesions

[0096] On the test set, the best network model saved in step 3 is used for model inference, and the visualization results of lesion segmentation are output.

[0097] See Figure 5 The magnetic resonance image to be processed (5a) is input into the best network model saved in step 3. After processing by the network model, the image segmentation result (5c) is obtained. Figure (5b) shows the tumor location marked by the doctor.

[0098] The neural network constructed using this invention was compared with U-Net, Attentionu-net, Unet 3+, MultiResUNet, DC-UNet, DualNorm-UNet, and EAR-U-Net. It outperformed other models in terms of mean intersection-over-union ratio (mIoU), Dice coefficient (DSC), accuracy (Acc), and sensitivity (Sen). The evaluation metrics of each network model on the CeTS test set are detailed in Table 1 below.

[0099] Table 1: Evaluation metrics of each network on the CeTS dataset

[0100]

[0101]

[0102] This invention demonstrates the effectiveness of the global feature fusion module, feature decomposition and recombination module, and channel spatial attention gating module through ablation experiments. The experimental results are shown in Table 2.

[0103] Table 2: Ablation performance of each module on the CeTS dataset

[0104]

[0105] Net1 is the backbone of the network architecture, a simple improvement on the original U-Net. Net2, Net3, and Net4 are trained using different global convolutional modules: ResBlock, MultiResBlock, and the global feature fusion module proposed in this invention, respectively. Net5 is trained using the global feature fusion module and the feature decomposition and reconstruction module. Net6 and Net7 add attention gating and the channel spatial attention gating proposed in this invention to Net5.

[0106] The above embodiments provide a detailed description of the cervical cancer lesion segmentation method based on global-local cascading magnetic resonance imaging proposed in this invention. For researchers in the art, based on the ideas of this invention, there may be changes in the specific implementation methods and application scope. For example, the encoder of the network model can be replaced with encoders such as VGG, ResNet, and EfficientNet to extract features, and the network gating mechanism can be replaced with SE blocks, CBAM blocks, etc. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A method for magnetic resonance image cervical cancer tumor segmentation based on global-local cascade, characterized in that, The method comprises the following specific steps: Step 1: preprocessing the data set; On the CeTS magnetic resonance image data set, slices are extracted from 3D magnetic resonance images, training and test sets are divided, data augmentation is performed on the images through random horizontal flipping, vertical flipping and scaling, and the data is normalized; Step 2: constructing a neural network model based on global-local cascade; Based on the PyTorch deep learning framework, a neural network model based on global-local cascade is constructed, and the model is based on an encoder-decoder architecture, and the following modules are inserted in the path of the encoder and the decoder: a) Global feature fusion module: input HxWx3 sample, H and W represent the height and width of the image, sequentially pass through three 3x3 convolution subblocks, each convolution subblock contains a convolution layer, a compression-activation block and a ReLU activation function; then adjust the importance of the feature map in the channel dimension according to the attention value generated by the compression-activation block, so that the multi-scale feature map aggregates more global information; then extract the output from the three convolution blocks and fuse them together to extract different scales of spatial features; finally, introduce a 1x1 convolution layer between the blocks to increase the skip connection; the output size of this module is a feature map of HxWxC1, and C1 represents the number of intermediate channels; b) Feature decomposition and recombination module: the input is the output F of the global feature fusion module s0 Then use 1x1 convolution to select the feature map F that needs local processing from the global feature map s , and use the feature decomposition method to decompose F into four sub-blocks of the same size s , each of which is processed using a convolution to obtain F1, F2, F3, and F4; then use the feature recombination method to first splice the two blocks adjacent to the left and right to obtain two feature maps F 12 and F 34 , which are half of the original input, and use another convolution layer to act on F 12 and F 34 to extract more features in a larger pixel domain; for the two blocks adjacent to the top and bottom, the same operation is performed as for the blocks adjacent to the left and right, i.e., to obtain feature maps F 13 and F 24 ; then splice F 12 and F 34 to obtain F 1234 , splice F 13 and F 24 to obtain F 1324 , and perform convolution on F 1234 , F 1324 ; finally, connect F 1234 , F 1324 with the feature map of the original input in the channel, and use 1x1 convolution to fuse them; The formula of the feature decomposition method is: F1 = Conv(F s [0:C, 0:H / 2+B, 0:W / 2+B] F2 = Conv(F s [0:C, 0:H / 2+B, W / 2-B:W]), F3 = Conv(F s [0:C, H / 2 - B:H, 0:W / 2 + B], F4 = Conv(F s [0:C,H / 2-B:H,W / 2-B:W]), wherein the subscripts 1, 2, 3, 4 of F represent the feature maps at the upper left, upper right, lower left and lower right positions after division, respectively, and Fs represents the output F of the global feature fusion module s0 The result after 1x1 convolution, Conv represents convolution operation, C, H and W represent the channel number, height and width of the feature map, respectively, and B represents the length of the overlapping part of two adjacent sub-blocks; The formula of the feature recombination method is: F 12 = Conv(F1(s) F2), F 34 = Conv(F3(s) F4), F 13 = Conv(F1(s) F3), F 24 = Conv(F2(s)F4), Where (s) is a splicing operator, which means splicing two adjacent feature maps in the spatial plane; The formula of the fusion is: F 1234 = Conv(F 12 (s)F 34 ), F 1324 = Conv(F 13 (s)F 24 ), F out = Conv(F 1234 (c) F 1324 (c) F in ), Where (c) represents a channel connection operator; c) Channel spatial attention gating module: first generate channel attention map g ∈R c using different inter-channel relationships of decoder features C×1×1 ; then broadcast operation is performed on g c to expand to g c ∈R C×H×W , and pixel-wise multiplication is performed between g d and the corresponding encoder features x e ∈R C×H×W to obtain x c ∈R C×H×W ; then generate spatial attention map s c ∈R s using the relationship in the space of x 1×H×W ; finally, pixel-wise multiplication is performed between spatial attention map s s and encoder features x e to obtain feature map x out ∈R C×H×W which is subjected to attention in spatial domain and channel direction simultaneously; Step 3: training the network model; The preprocessed training set is input into the neural network model for training, and a loss function that combines Dice loss and cross-entropy loss is used for constraint, and the loss function is represented as: where λ represents a balance factor between the two loss functions, set to 0.5, N represents the total number of pixels, C represents the number of channels contained in the prediction result matrix, and p represents the pixel value marked as a positive sample, represents the probability that the pixel value is predicted as a positive sample, represents the dot product operation of the real label of each channel and the matrix of the corresponding prediction result, represents the square of the Euclidean norm operation; a validation set is also divided during the training process, and the neural network model weight with the best performance on the validation set is saved; Step 4: segmentation of cervical cancer lesions; On the test set, the best neural network model saved in step 3 is used to output the visualization result of the lesion segmentation.

Images