Brain tumor segmentation method based on multi-region attention perception

Abstract

The invention discloses a brain tumor segmentation method based on multi-region attention perception, and aims to realize accurate positioning of brain tumor sub-regions by fully learning an internal relationship between brain tumor tissue regions. The method comprises the following steps: S1, carrying out preprocessing and data enhancement on a brain tumor MR image to obtain a brain tumor data set; s2, constructing a multi-mode feature interaction enhancement module to perform feature extraction and fusion on different modes (T1, T1CE, T2 and Flair) of the brain tumor MRI to obtain global features; s3, constructing a multi-region perception attention module, and extracting feature representation among multiple regions to obtain local multi-region features; s4, constructing a multi-region collaborative fusion module to fuse the local multi-region features and the global features to obtain fusion features; s5, performing MRI brain glioma segmentation according to the trained MRI brain glioma segmentation model; according to the method, the brain tumor multi-region internal relation is effectively extracted and fused from the MRI, so that accurate segmentation of the brain tumor MRI is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a brain tumor segmentation method and device based on multi-region perception attention. Background Art

[0002] In the field of medical image analysis, brain tumor segmentation is a crucial task that relies on multimodal MRI images to accurately identify and locate different areas of brain tumors. Although significant progress has been made in this field, existing brain tumor segmentation techniques ignore the relationship between local regions, resulting in insufficient segmentation accuracy, which in turn affects the accuracy and robustness of overall tumor segmentation.

[0003] The complexity of brain tumors lies in their multi-level local structures, including necrotic areas, edema areas around brain tumors, and enhanced areas of brain tumors. These different local areas show different characteristics in multimodal images. During the segmentation process, how to effectively utilize the relationship between local areas to achieve accurate segmentation is a major challenge in the brain tumor segmentation task.

[0004] Existing segmentation methods fail to fully consider the intrinsic connections between these local regions, resulting in uneven distribution and inaccurate positioning of brain tumor regions in the segmentation results, especially in the fine positioning of tumor regions. This limitation is particularly evident in complex brain tumor structures. Therefore, existing multimodal MRI brain tumor segmentation methods have obvious deficiencies in multimodal information fusion and modeling of intrinsic relationships between multiple regions.

[0005] In order to solve these problems, the present invention proposes a brain tumor segmentation method which can effectively fuse multimodal information, accurately locate the tumor area and optimize the tumor boundary positioning, thereby improving the accuracy of brain tumor segmentation. Summary of the invention

[0006] In order to solve the above problems, the present invention provides a brain tumor segmentation method based on multi-region perceptual attention, which can realize accurate segmentation of MRI glioma by fully learning the intrinsic relationship between brain tumor tissue regions.

[0007] Specifically, the method comprises the following steps:

[0008] S1: Preprocess and enhance the brain tumor MR images to obtain the brain tumor dataset;

[0009] S2: Construct a multimodal feature interaction enhancement module to extract and fuse features of different modalities (T1, T1CE, T2, Flair) of brain tumor MRI to obtain global features;

[0010] S3: Construct a multi-region perception attention module to extract feature representations between multiple regions and obtain local multi-region features;

[0011] S4: Construct a multi-region collaborative fusion module to fuse local multi-region features and global features to obtain fused features;

[0012] S5: Perform MRI glioma segmentation based on the trained MRI glioma segmentation model;

[0013] 2. Preferably, step S1 comprises the following steps:

[0014] S1.1: MRI data processing: Crop the MRI images and remove irrelevant areas to ensure that all images are of the same size. Normalize the pixel values ​​of each image to a uniform range (0 to 1). Use filtering technology to remove noise from the MRI images, maintain the edge features and some detail information of the images, and ensure the accuracy and availability of the MRI images in subsequent processing;

[0015] S1.2: Use data augmentation technology to expand the training data to increase sample diversity, reduce the risk of overfitting, and improve the generalization ability of the model;

[0016] S1.3: Dataset division: Divide the processed dataset into training set, validation set and test set.

[0017] 4. Preferably, step S2 comprises the following steps:

[0018] S2.1: Construct a Multimodal Feature Interaction Enhancement Module (MFIE);

[0019] S2.2: According to the MFIE module, the global feature extraction and calculation of the MRI data are performed to obtain the preliminary feature maps of the four modes of T1, T1CE, T2, and Flair, which are represented as f m (m=1,2,3,4);

[0020] S2.3: Perform global feature fusion calculation on MRI data according to the MFIE module to obtain the global feature f r .

[0021] Preferably, S2.1 comprises the following steps:

[0022] A multi-branch convolutional network is used to extract different modal features and then fuse them to obtain the MFIE structure:

[0023]

[0024] m=1,2,3,4

[0025]

[0026] Among them, I m represents the input mth modality image, f m represents the mth modal feature map, Represents the feature map after upsampling of the mth modality. UpSample represents the upsampling operation, Conv 3×3×3 Indicates that 3×3×3 convolution is used to extract features, Conv 1×1×1 Used to fuse features of different modalities. Represents the feature map concatenation operation.

[0027] 4. Preferably, step S3 comprises the following steps:

[0028] S3.1: Construct a Multi-region Perception Attention Module (MPAM);

[0029] S3.2: According to the MPAM module, the global feature f r Perform regional feature extraction to obtain feature representations of different tumor regions, including brain tumor necrosis region feature X 1 、Characteristics of edema area around brain tumors 2 and brain tumor enhancement region features X 3 ;

[0030] S3.3: Use the self-attention mechanism to calculate each region feature X i The attention weights between (i=1,2,3) and other regional features are added to perform weighted updates on each regional feature to obtain the enhanced regional feature representation X' i (i=1,2,3).

[0031] Among them, S3.1 includes the following steps:

[0032] The correlation between cross-regional features is calculated to obtain the MPAM structure:

[0033] X 1 ,X 2 ,X 3 =3DUNet[SoftMax(Conv 1×1×1 (f r ))]

[0034]

[0035] Among them, Q i , V i Respectively represent the regions from X i The query, key and value of X are represented by , d represents the feature dimension of each region. SoftMax represents the application of SoftMax activation function, and Linear represents the i Perform linear transformation, 3DUNet means coarse segmentation operation through 3DUNet network;

[0036] By assigning different attention weights, the model focuses on the tumor region and the key structures around it, enhancing the expressiveness of multi-region features and optimizing the segmentation performance.

[0037] 5. Preferably, step S4 comprises the following steps:

[0038] S4.1: Construct a Multi-region Collaborative Fusion Module (MCFM);

[0039] S4.2: Combine local multi-region fusion features and global features f r Input MCFM to obtain fused features. The fusion process not only involves the features of different local areas, but also combines the global context features, which significantly improves the depth and breadth of feature fusion.

[0040] Preferably, S4.1 comprises the following steps:

[0041]

[0042] Where ⊙ represents element-by-element multiplication, Represents a splicing operation, is the final fusion feature map.

[0043] 6. Preferably, step S5 comprises the following steps:

[0044] S5.1 Input the brain tumor MRI into the trained segmentation model to obtain a probability map of brain tumor segmentation;

[0045] S5.2 performs binarization processing on the probability map to obtain the final brain tumor segmentation result.

[0046] The beneficial effects of the method and device of the present invention are as follows: the brain tumor segmentation method provided by the present invention effectively fuses different modal information of T1, T1ce, T2 and Flair through the multimodal feature interaction enhancement module (MFIE). The multi-region perception attention module (MPAM) focuses on the intrinsic relationship between multiple local regions of brain tumors through the self-attention mechanism, optimizes the feature expression and enhances the model's understanding of tumor morphology. The multi-region collaborative fusion module (MCFM) further fuses local multi-region features and global features to avoid information loss and improve the accuracy of segmentation results. The present invention achieves high precision and good visualization effects in MRI brain glioma segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the prior art and the drawings required for use in the embodiments. The following drawings are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 It is a flowchart of a brain tumor segmentation method based on multi-region perception attention of the present invention;

[0049] Figure 2 It is a schematic diagram of the MFIE module structure of a brain tumor segmentation method based on multi-region perception attention of the present invention;

[0050] Figure 3 It is a schematic diagram of the structure of the MPAM module of a brain tumor segmentation method based on multi-region perception attention of the present invention;

[0051] Figure 4 It is a schematic diagram of the MCFM module structure of a brain tumor segmentation method based on multi-region perception attention of the present invention; Specific implementation plan

[0052] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all other embodiments obtained by ordinary technicians in this field without creative work based on the embodiments of the present invention are within the scope of protection of the present invention.

[0053] The embodiment of the present invention provides a brain tumor segmentation method based on multi-region perception attention, which realizes accurate positioning of brain tumor sub-regions by fully learning the intrinsic relationship between brain tumor tissue regions.

[0054] A typical embodiment of the present invention, referring to Figure 1 , the method comprises the following steps:

[0055] S1: Preprocess and enhance the brain tumor MR images to obtain the brain tumor dataset;

[0056] S2: Construct a multimodal feature interaction enhancement module to extract and fuse features of different modalities (T1, T1CE, T2, Flair) of brain tumor MRI to obtain global features;

[0057] S3: Construct a multi-region perception attention module to extract feature representations between multiple regions and obtain local multi-region features;

[0058] S4: Construct a multi-region collaborative fusion module to fuse local multi-region features and global features to obtain fused features;

[0059] S5: Perform MRI glioma segmentation based on the trained MRI glioma segmentation model;

[0060] Each step is described in detail as follows.

[0061] Further, as a preferred embodiment of the method, S1 comprises the following steps:

[0062] S1.1: MRI data processing: Crop the MRI images and remove irrelevant areas to ensure that all images are of the same size. Normalize the pixel values ​​of each image to a uniform range (0 to 1). Use median filtering technology to remove noise from MRI images, maintain edge features and some detail information of the images, and ensure the accuracy and usability of MRI images in subsequent processing;

[0063] S1.2: Use data augmentation technology to expand the training data to increase sample diversity, reduce the risk of overfitting, and improve the generalization ability of the model;

[0064] S1.3: Dataset division: Divide the processed dataset into training set, validation set and test set.

[0065] Furthermore, the step of data enhancement technology specifically includes:

[0066] First, the image is randomly rotated by 0°, 90°, 180° or 270°, and horizontally and vertically translated. Then the image is randomly scaled to 0.8 to 1.2 times, horizontally and vertically flipped, and Gaussian noise is added to simulate image noise interference and the brightness and contrast of the image are randomly adjusted.

[0067] Furthermore, the step of dividing the data set specifically includes:

[0068] The data is divided into 80% (training set), 10% (validation set), and 10% (test set).

[0069] Further, refer to Figure 2 , S2 includes the following steps:

[0070] S2.1: Construct a Multimodal Feature Interaction Enhancement Module (MFIE);

[0071] S2.2: According to the MFIE module, the global feature extraction and calculation of the MRI data are performed to obtain the preliminary feature maps of the four modes of T1, T1CE, T2, and Flair, which are represented as f m (m=1,2,3,4);

[0072] S2.3: Perform global feature fusion calculation on MRI data according to the MFIE module to obtain the global feature f r .

[0073] Further, S2.1 includes the following steps:

[0074] A multi-branch convolutional network is used to extract different modal features and then fuse them to obtain the MFIE structure:

[0075]

[0076] m=1,2,3,4

[0077]

[0078] Among them, I m represents the input mth modality image, f m represents the mth modal feature map, Represents the feature map after upsampling of the mth modality. UpSample represents the upsampling operation, Conv 3×3×3 Indicates that 3×3×3 convolution is used to extract features, Conv 1×1×1 Used to fuse features of different modalities. Represents the feature map concatenation operation.

[0079] Further, refer to Figure 3 , S3 includes the following steps:

[0080] S3.1: Construct a Multi-region Perception Attention Module (MPAM);

[0081] S3.2: According to the MPAM module, the global feature f r Perform regional feature extraction to obtain feature representations of different tumor regions, including brain tumor necrosis region feature X 1 、Characteristics of edema area around brain tumors 2 and brain tumor enhancement region features X 3 ;

[0082] S3.3: Use the self-attention mechanism to calculate each region feature X i The attention weights between (i=1,2,3) and other regional features are added to perform weighted updates on each regional feature to obtain the enhanced regional feature representation X' i (i=1,2,3).

[0083] Further, S3.1 includes the following steps:

[0084] The correlation between cross-regional features is calculated to obtain the MPAM structure:

[0085] X 1 ,X 2 ,X 3 =3DUNet[SoftMax(Conv 1×1×1 (f r ))]

[0086]

[0087] Among them, Q i , V i Respectively represent the regions from X i The query, key and value of X are represented by , d represents the feature dimension of each region. SoftMax represents the application of SoftMax activation function, and Linear represents the i Perform linear transformation, 3DUNet means coarse segmentation operation through 3DUNet network;

[0088] Further, refer to Figure 4 , S4 comprises the following steps:

[0089] S4.1: Construct a Multi-region Collaborative Fusion Module (MCFM);

[0090] S4.2: Combine local multi-region fusion features and global features f r Input MCFM to obtain fused features. The fusion process not only involves the features of different local areas, but also combines the global context features, which significantly improves the depth and breadth of feature fusion.

[0091] Further, S4.1 includes the following steps:

[0092]

[0093] Where ⊙ represents element-by-element multiplication, Represents a splicing operation, is the final fusion feature map.

[0094] Further, as a preferred embodiment of the present method, S5 comprises the following steps:

[0095] S5.1 Input the brain tumor MRI into the trained segmentation model to obtain a probability map of brain tumor segmentation;

[0096] S5.2 performs binarization processing on the probability map to obtain the final brain tumor segmentation result.

[0097] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art may make other equivalent modifications or substitutions without violating the spirit of the invention, and these equivalent modifications or substitutions are included in the scope defined by the application claims.

Claims

1. A brain tumor segmentation method based on multi-region perception attention, characterized in that: The method comprises the following steps: S1: Preprocess and enhance the brain tumor MR images to obtain the brain tumor dataset; S2: Construct a multimodal feature interaction enhancement module to extract and fuse features of different modalities (T1, T1CE, T2, Flair) of brain tumor MRI to obtain global features; S3: Construct a multi-region perception attention module to extract feature representations between multiple regions and obtain local multi-region features; S4: Construct a multi-region collaborative fusion module to fuse local multi-region features and global features to obtain fused features; S5: Perform MRI glioma segmentation based on the trained MRI glioma segmentation model.

2. The brain tumor segmentation method based on multi-region perception attention according to claim 1 is characterized in that: Step S1 includes the following steps: S1.1: MRI data processing: Crop the MRI images and remove irrelevant areas to ensure that all images are of the same size. Normalize the pixel values ​​of each image to a uniform range (0 to 1). Use filtering technology to remove noise from the MRI images, maintain the edge features and some detail information of the images, and ensure the accuracy and availability of the MRI images in subsequent processing; S1.2: Use data augmentation technology to expand the training data to increase sample diversity, reduce the risk of overfitting, and improve the generalization ability of the model; S1.3: Dataset division: Divide the processed dataset into training set, validation set and test set.

3. The brain tumor segmentation method based on multi-region perception attention according to claim 1, characterized in that: Step S2 includes the following steps: S2.1: Construct a multimodal feature interaction enhancement module Multimodal Feature Interaction EnhancementModule (MFIE); S2.2: According to the MFIE module, the global feature extraction and calculation of MRI data are performed to obtain the preliminary feature maps of the four modes of T1, T1CE, T2, and Flair, which are represented as f m (m=1,2,3,4); S2.3: Perform global feature fusion calculation on MRI data according to the MFIE module to obtain the global feature f r . Among them, S2.1 includes the following steps: A multi-branch convolutional network is used to extract different modal features and then fuse them to obtain the MFIE structure: Among them, I m represents the input mth modality image, f m represents the mth modal feature map, Represents the feature map after upsampling of the mth modality. UpSample represents the upsampling operation, Conv 3×3×3 Indicates that 3×3×3 convolution is used to extract features, Conv 1×1×1 Used to fuse features of different modalities. Represents the feature map concatenation operation.

4. The brain tumor segmentation method based on multi-region perception attention according to claim 1, characterized in that: Step S3 includes the following steps: S3.1: Construct a Multi-region Perception Attention Module (MPAM); S3.2: According to the MPAM module, the global feature f r Perform regional feature extraction to obtain feature representations of different tumor regions, including brain tumor necrosis region feature X1, brain tumor surrounding edema region feature X2, and brain tumor enhancement region feature X3; S3.3: Use the self-attention mechanism to calculate each region feature X i The attention weights between (i=1,2,3) and other regional features are added to perform weighted updates on each regional feature to obtain the enhanced regional feature representation X' i (i=1,2,3). Among them, S3.1 includes the following steps: The correlation between cross-regional features is calculated to obtain the MPAM structure: X1,X2,X3=3DUNet[SoftMax(Conv 1×1×1 (f r ))] Among them, Q i , V i Respectively represent the regions from X i The query, key and value of X are represented by , d represents the feature dimension of each region. SoftMax represents the application of SoftMax activation function, and Linear represents the i Perform linear transformation, 3DUNet means coarse segmentation operation through 3DUNet network; By assigning different attention weights, the model focuses on the tumor region and the key structures around it, enhancing the expressiveness of multi-region features and optimizing the segmentation performance.

5. The brain tumor segmentation method based on multi-region perception attention according to claim 1, characterized in that: Step S4 includes the following steps: S4.1: Construct a multi-region collaborative fusion module (MCFM); S4.2: Fusion of local multi-region features X' i (i=1,2,3) and global feature f r Input MCFM to obtain fused features. The fusion process not only involves the features of different local areas, but also combines the global context features, which significantly improves the depth and breadth of feature fusion. Among them, S4.1 includes the following steps: Where ⊙ represents element-by-element multiplication, Represents a splicing operation, is the final fusion feature map.

6. The brain tumor segmentation method based on multi-region perception attention according to claim 1, characterized in that: Step S5 includes the following steps: S5.1 Input the brain tumor MRI into the trained segmentation model to obtain a probability map of brain tumor segmentation; S5.2 performs binarization processing on the probability map to obtain the final brain tumor segmentation result.

Images