A brain tumor segmentation method based on multi-region perceptual attention

Abstract

This invention discloses a brain tumor segmentation method based on multi-region perception and attention. The method aims to achieve precise positioning of brain tumor subregions by fully learning the intrinsic relationships between brain tumor tissue regions. The method includes the following steps: S1: preprocessing and data enhancement of brain tumor MR images to obtain a brain tumor dataset; S2: constructing a multimodal feature interaction enhancement module to extract and fuse features from different brain tumor MRI modalities (T1, T1CE, T2, Flair) to obtain global features; S3: constructing a multi-region perception and attention module to extract feature representations between multiple regions to obtain local multi-region features; S4: constructing a multi-region collaborative fusion module to fuse local multi-region features with global features to obtain fused features; S5: performing MRI brain glioma segmentation based on a trained MRI brain glioma segmentation model. By effectively extracting and fusing the intrinsic relationships between multiple brain tumor regions from MRI, the present invention achieves precise MRI brain tumor segmentation.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a brain tumor segmentation method and device based on multi-region perception attention. Background Art

[0002] Brain tumor segmentation is a crucial task in medical image analysis, relying on multimodal MRI images to accurately identify and locate different brain tumor regions. Despite significant progress in this field, existing brain tumor segmentation techniques ignore the relationships between local regions, resulting in insufficient segmentation accuracy, which in turn affects the accuracy and robustness of overall tumor segmentation.

[0003] The complexity of brain tumors lies in their multi-layered local structure, including areas of necrosis, surrounding edema, and enhanced regions. These distinct local regions exhibit distinct characteristics in multimodal images. During the segmentation process, effectively leveraging the relationships between these local regions to achieve accurate segmentation is a major challenge in brain tumor segmentation.

[0004] Existing segmentation methods fail to fully consider the intrinsic connections between these local regions, resulting in uneven distribution and inaccurate positioning of brain tumor regions in the segmentation results. This limitation is particularly evident in the fine-grained positioning of tumor regions, and is particularly pronounced in complex brain tumor structures. Therefore, existing multimodal MRI brain tumor segmentation methods have significant deficiencies in multimodal information fusion and modeling the intrinsic relationships between multiple regions.

[0005] In order to solve these problems, the present invention proposes a brain tumor segmentation method that can effectively fuse multimodal information, accurately locate the tumor area and optimize the positioning of the tumor boundary, thereby improving the accuracy of brain tumor segmentation. Summary of the Invention

[0006] In order to solve the above problems, the present invention provides a brain tumor segmentation method based on multi-region perceptual attention, which can achieve accurate MRI brain glioma segmentation by fully learning the intrinsic relationship between brain tumor tissue regions.

[0007] Specifically, the method comprises the following steps:

[0008] S1: Preprocess and data enhancement are performed on brain tumor MR images to obtain a brain tumor dataset;

[0009] S2: Construct a multimodal feature interaction enhancement module to extract and fuse features from different brain tumor MRI modalities (T1, T1CE, T2, Flair) to obtain global features;

[0010] S3: Construct a multi-region perception attention module to extract feature representations between multiple regions and obtain local multi-region features;

[0011] S4: Construct a multi-region collaborative fusion module to fuse local multi-region features and global features to obtain fused features;

[0012] S5: Perform MRI glioma segmentation based on the trained MRI glioma segmentation model;

[0013] 2. Preferably, step S1 includes the following steps:

[0014] S1.1: MRI Data Processing: Cropping the MRI images to remove irrelevant areas to ensure that all images are of the same size. Normalizing the pixel values ​​of each image to a uniform range (0 to 1). Using filtering techniques to remove noise from the MRI images, preserving edge features and some detail information, and ensuring the accuracy and usability of the MRI images in subsequent processing.

[0015] S1.2: Use data augmentation techniques to expand training data to increase sample diversity, reduce the risk of overfitting, and improve the generalization ability of the model;

[0016] S1.3: Dataset partitioning: Divide the processed dataset into training set, validation set, and test set.

[0017] 4. Preferably, step S2 includes the following steps:

[0018] S2.1: Construct a Multimodal Feature Interaction Enhancement Module (MFIE);

[0019] S2.2: Global feature extraction and calculation of MRI data are performed according to the MFIE module to obtain preliminary feature maps of the four modalities T1, T1CE, T2, and Flair, which are represented as f m (m=1,2,3,4);

[0020] S2.3: Perform global feature fusion calculation on MRI data according to the MFIE module to obtain the global feature f r .

[0021] Preferably, S2.1 includes the following steps:

[0022] A multi-branch convolutional network is used to extract different modal features and then fuse them to obtain the MFIE structure:

[0023]

[0024] m=1,2,3,4

[0025]

[0026] Among them, I m represents the input mth modality image, f m represents the mth modal feature map, Represents the feature map after upsampling of the mth modality. UpSample represents the upsampling operation, Conv 3×3×3 Indicates that 3×3×3 convolution is used to extract features, Conv 1×1×1 Used to fuse different modal features, Represents the feature map concatenation operation.

[0027] 4. Preferably, step S3 includes the following steps:

[0028] S3.1: Construct a Multi-region Perception Attention Module (MPAM);

[0029] S3.2: According to the MPAM module, the global feature f r Perform regional feature extraction to obtain feature representations of different tumor regions, including brain tumor necrosis region feature X1, brain tumor surrounding edema region feature X2, and brain tumor enhancement region feature X3;

[0030] S3.3: Use the self-attention mechanism to calculate each region feature X i The attention weights between (i=1,2,3) and other regional features are used to update each regional feature in a weighted manner to obtain the enhanced regional feature representation X' i (i=1,2,3).

[0031] Among them, S3.1 includes the following steps:

[0032] The correlation between cross-region features is calculated to obtain the MPAM structure:

[0033] X1,X2,X3=3DUNet[SoftMax(Conv 1×1×1 (f r ))]

[0034]

[0035] Among them, Q i 、 V i Respectively represent the regions from X iThe query, key, and value of X are represented by d, which represents the feature dimension of each region. SoftMax represents the application of the SoftMax activation function, and Linear represents the application of the SoftMax activation function to X. i Perform linear transformation, 3DUNet means performing coarse segmentation operation through 3DUNet network;

[0036] By assigning different attention weights, the model focuses on the tumor region and its surrounding key structures, enhancing the expressiveness of multi-region features and optimizing segmentation performance.

[0037] 5. Preferably, step S4 includes the following steps:

[0038] S4.1: Build a Multi-region Collaborative Fusion Module (MCFM);

[0039] S4.2: Combine local multi-region fusion features and global features f r Input MCFM to obtain fused features. The fusion process not only involves the features of different local areas, but also combines the global context features, which significantly improves the depth and breadth of feature fusion.

[0040] Preferably, S4.1 includes the following steps:

[0041]

[0042] Where ⊙ represents element-wise multiplication, Represents a splicing operation, is the final fusion feature map.

[0043] 6. Preferably, step S5 includes the following steps:

[0044] S5.1 Input the brain tumor MRI into the trained segmentation model to obtain a probability map of brain tumor segmentation;

[0045] S5.2 performs binarization processing on the probability map to obtain the final brain tumor segmentation result.

[0046] The beneficial effects of the method and device of the present invention are as follows: the brain tumor segmentation method provided by the present invention effectively fuses different modal information of T1, T1ce, T2 and Flair through the multimodal feature interaction enhancement module (MFIE). The multi-region perception attention module (MPAM) focuses on the intrinsic relationship between multiple local regions of brain tumors through the self-attention mechanism, optimizes feature expression and enhances the model's understanding of tumor morphology. The multi-region collaborative fusion module (MCFM) further fuses local multi-region features and global features to avoid information loss and improve the accuracy of segmentation results. The present invention achieves high precision and good visualization effects in MRI brain glioma segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the prior art and the drawings required for use in the embodiments. The following drawings are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0048] Figure 1 1 is a flow chart of a brain tumor segmentation method based on multi-region perception attention of the present invention;

[0049] Figure 2 Schematic diagram of the MFIE module structure of a brain tumor segmentation method based on multi-region perception attention of the present invention;

[0050] Figure 3 Schematic diagram of the MPAM module structure of a brain tumor segmentation method based on multi-region perception attention of the present invention;

[0051] Figure 4 Schematic diagram of the MCFM module structure of a brain tumor segmentation method based on multi-region perception attention of the present invention; Specific implementation plan

[0052] In order to make the purpose, features, and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making creative work are within the scope of protection of the present invention.

[0053] An embodiment of the present invention provides a brain tumor segmentation method based on multi-region perception attention, which achieves accurate positioning of brain tumor sub-regions by fully learning the intrinsic relationship between brain tumor tissue regions.

[0054] A typical embodiment of the present invention, referring to Figure 1 , the method comprises the following steps:

[0055] S1: Preprocess and data enhancement are performed on brain tumor MR images to obtain a brain tumor dataset;

[0056] S2: Construct a multimodal feature interaction enhancement module to extract and fuse features from different brain tumor MRI modalities (T1, T1CE, T2, Flair) to obtain global features;

[0057] S3: Construct a multi-region perception attention module to extract feature representations between multiple regions and obtain local multi-region features;

[0058] S4: Construct a multi-region collaborative fusion module to fuse local multi-region features and global features to obtain fused features;

[0059] S5: Perform MRI glioma segmentation based on the trained MRI glioma segmentation model;

[0060] Each step is described in detail below.

[0061] Furthermore, as a preferred embodiment of the present method, S1 includes the following steps:

[0062] S1.1: MRI Data Processing: Crop the MRI images and remove irrelevant areas to ensure that all images are of the same size. Normalize the pixel values ​​of each image to a uniform range (0 to 1). Use median filtering to remove noise from the MRI images, preserving edge features and some detail information to ensure the accuracy and usability of the MRI images in subsequent processing.

[0063] S1.2: Use data augmentation techniques to expand training data to increase sample diversity, reduce the risk of overfitting, and improve the generalization ability of the model;

[0064] S1.3: Dataset partitioning: Divide the processed dataset into training set, validation set, and test set.

[0065] Furthermore, the data enhancement technology step specifically includes:

[0066] First, the image is randomly rotated by 0°, 90°, 180°, or 270°, and translated horizontally and vertically. Then, the image is randomly scaled to 0.8 to 1.2 times, flipped horizontally and vertically, and Gaussian noise is added to simulate image noise interference and the brightness and contrast of the image are randomly adjusted.

[0067] Furthermore, the step of dividing the data set specifically includes:

[0068] The dataset is divided into 80% (training set), 10% (validation set), and 10% (test set).

[0069] Further, refer to Figure 2 , S2 includes the following steps:

[0070] S2.1: Construct a Multimodal Feature Interaction Enhancement Module (MFIE);

[0071] S2.2: Global feature extraction and calculation of MRI data are performed according to the MFIE module to obtain preliminary feature maps of the four modalities T1, T1CE, T2, and Flair, which are represented as f m (m=1,2,3,4);

[0072] S2.3: Perform global feature fusion calculation on MRI data according to the MFIE module to obtain the global feature f r .

[0073] Furthermore, S2.1 includes the following steps:

[0074] A multi-branch convolutional network is used to extract different modal features and then fuse them to obtain the MFIE structure:

[0075]

[0076] m=1,2,3,4

[0077]

[0078] Among them, I m represents the input mth modality image, f m represents the mth modal feature map, Represents the feature map after upsampling of the mth modality. UpSample represents the upsampling operation, Conv 3×3×3 Indicates that 3×3×3 convolution is used to extract features, Conv 1×1×1 Used to fuse different modal features, Represents the feature map concatenation operation.

[0079] Further, refer to Figure 3 , S3 includes the following steps:

[0080] S3.1: Construct a Multi-region Perception Attention Module (MPAM);

[0081] S3.2: According to the MPAM module, the global feature f r Perform regional feature extraction to obtain feature representations of different tumor regions, including brain tumor necrosis region feature X1, brain tumor surrounding edema region feature X2, and brain tumor enhancement region feature X3;

[0082] S3.3: Use the self-attention mechanism to calculate each region feature X i The attention weights between (i=1,2,3) and other regional features are used to update each regional feature in a weighted manner to obtain the enhanced regional feature representation X' i (i=1,2,3).

[0083] Furthermore, S3.1 includes the following steps:

[0084] The correlation between cross-region features is calculated to obtain the MPAM structure:

[0085] X1,X2,X3=3DUNet[SoftMax(Conv 1×1×1 (f r ))]

[0086]

[0087] Among them, Q i 、 V i Respectively represent the regions from X i The query, key, and value of X are represented by d, which represents the feature dimension of each region. SoftMax represents the application of the SoftMax activation function, and Linear represents the application of the SoftMax activation function to X. i Perform linear transformation, 3DUNet means performing coarse segmentation operation through 3DUNet network;

[0088] Further, refer to Figure 4 , S4 includes the following steps:

[0089] S4.1: Build a Multi-region Collaborative Fusion Module (MCFM);

[0090] S4.2: Combine local multi-region fusion features and global features f r Input MCFM to obtain fused features. The fusion process not only involves the features of different local areas, but also combines the global context features, which significantly improves the depth and breadth of feature fusion.

[0091] Furthermore, S4.1 includes the following steps:

[0092]

[0093] Where ⊙ represents element-wise multiplication, Represents a splicing operation, is the final fusion feature map.

[0094] Further, as a preferred embodiment of the present method, S5 includes the following steps:

[0095] S5.1 Input the brain tumor MRI into the trained segmentation model to obtain a probability map of brain tumor segmentation;

[0096] S5.2 performs binarization processing on the probability map to obtain the final brain tumor segmentation result.

[0097] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art may make other equivalent modifications or substitutions without violating the spirit of the invention, and these equivalent modifications or substitutions are included in the scope defined by the claims.

Claims

1. A brain tumor segmentation method based on multi-region perceptual attention, characterized in that: The method comprises the following steps: S1: Preprocess and data augment the brain tumor MRI images to obtain the brain tumor dataset; S2: Construct a multimodal feature interaction enhancement module to extract and fuse features of different brain tumor MRI modalities T1, T1CE, T2, and Flair to obtain global features; S3: Construct a multi-region perception attention module to extract feature representations between multiple regions and obtain local multi-region features; Among them, S3 includes the following steps: S3.1: Constructing a Multi-region Perception Attention Module (MPAM). S3.2: Global features according to MPAM module Perform regional feature extraction to obtain feature representations of different tumor regions, including features of brain tumor necrosis regions , Characteristics of edema areas around brain tumors and brain tumor enhancement regional characteristics ; S3.3: Use self-attention mechanism to calculate each region feature The attention weights between each regional feature and other regional features are weighted updated to obtain the enhanced regional feature representation. ; Among them, S3.1 includes the following steps: The correlation between cross-region features is calculated to obtain the MPAM structure: in, 、 、 Respectively represent regions Query, key, and value, Represents the characteristic dimension of each region, Represents global features; Indicates application activation function, Express Perform a linear transformation, Indicates passing The network performs a coarse segmentation operation. Represents a three-dimensional convolution operation with a convolution kernel size of 1×1×1; By assigning different attention weights, the model focuses on the tumor region and its surrounding key structures, enhancing the expressiveness of multi-region features and optimizing segmentation performance. S4: Construct a multi-region collaborative fusion module to fuse local multi-region features and global features to obtain fusion features; S5: Perform MRI glioma segmentation based on the trained MRI glioma segmentation model.

2. The brain tumor segmentation method based on multi-region perception attention according to claim 1 is characterized in that: Step S1 includes the following steps: S1.1: MRI data processing: Cropping the MRI images to remove irrelevant areas to ensure that all images are of the same size; normalizing the pixel values ​​of each image to a value between 0 and 1; and using filtering techniques to remove noise from the MRI images, preserving edge features and some detail information, to ensure the accuracy and usability of the MRI images in subsequent processing. S1.2: Use data augmentation techniques to expand training data to increase sample diversity, reduce the risk of overfitting, and improve the generalization ability of the model; S1.3: Dataset partitioning: Divide the processed dataset into training set, validation set, and test set.

3. The brain tumor segmentation method based on multi-region perception attention according to claim 1, characterized in that: Step S2 includes the following steps: S2.1: Construct a Multimodal Feature Interaction Enhancement Module (MFIE); S2.2: Global feature extraction and calculation of MRI data are performed based on the MFIE module to obtain preliminary feature maps of the four modalities T1, T1CE, T2, and Flair, which are represented as ; S2.3: Perform global feature fusion calculation on MRI data according to the MFIE module to obtain global features ; S2.1 includes the following steps: A multi-branch convolutional network is used to extract different modal features and then fuse them to obtain the MFIE structure: , in, Indicates the input modal images, Indicates the modal feature maps, Indicates the Feature maps after upsampling of modalities; represents the upsampling operation, express Convolution is used to extract features. Used to fuse different modal features, Represents the feature map concatenation operation.

4. The brain tumor segmentation method based on multi-region perception attention according to claim 1, characterized in that: Step S4 includes the following steps: S4.1: Build a Multi-region Collaborative Fusion Module (MCFM); S4.2: Fusion of local multi-region features and global features Input MCFM to obtain fused features. The fusion process not only involves the features of different local regions, but also combines the global context features, which significantly improves the depth and breadth of feature fusion. Among them, S4.1 includes the following steps: in represents element-wise multiplication, Represents a splicing operation, is the final fusion feature map.

5. The brain tumor segmentation method based on multi-region perception attention according to claim 1, characterized in that: Step S5 includes the following steps: S5.1 Input the brain tumor MRI into the trained segmentation model to obtain a probability map of brain tumor segmentation; S5.2 performs binarization on the probability map to obtain the final brain tumor segmentation result.

Images