Brain tumor segmentation algorithm based on multi-path modal fusion and edge-driven attention

Abstract

The application discloses a brain tumor segmentation algorithm based on multi-path modal fusion and edge-driven attention, and belongs to the technical field of medical image analysis. The application comprises the following steps: performing two-dimensional slicing and center clipping on an original image; decomposing the input image into two independent two-channel features and extracting preliminary features through a convolution layer; inputting the decomposed image into a double-main-stem feature learning flow based on a Transformer to extract semantic features under a specific mode, and simultaneously performing deep integration through a shared fusion path to mine high-order and low-order detailed information of different modes. In the decoding stage, the upper and lower independent paths finely guide the shallow edge features through edge-driven attention; meanwhile, the shared fusion path adopts a multi-scale modal fusion module to cooperate with a skip connection to perform multi-feature fusion. The application has significant advantages in fine-grained edge processing and complex tumor morphology segmentation.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of medical image analysis, and particularly relates to a brain tumor segmentation algorithm based on multi-path modality fusion and edge-driven attention. BACKGROUND

[0002] The brain, as an important control center of the human body, controls human physiological movement, cognitive function and emotional experience. However, abnormal cell growth and canceration in the human brain can cause brain tumors. Glioma, as a primary tumor originating from tumor cells in the brain, can be roughly divided into two categories according to its grade: low-grade glioma (LGG) and high-grade glioma (HGG). High-grade glioma is characterized by canceration, and even if the patient receives timely intervention treatment, the average survival period is only 2 years or even shorter; low-grade glioma tends to show benign characteristics, although the average survival period is about 7 years, but it is inevitable that all low-grade gliomas will eventually deteriorate into high-grade gliomas and die. In the face of the threat of brain tumors, early diagnosis and accurate positioning of lesions are crucial for developing effective treatment plans.

[0003] Currently, the diagnosis and monitoring of brain tumors mainly rely on medical imaging technology. Among them, magnetic resonance imaging (MRI) as a non-invasive imaging method can produce high-resolution, non-invasive brain images, and is the main technical means for diagnosing and evaluating brain tumors. MRI can provide different imaging sequences (modes) for the same tissue under different imaging techniques, including T1 fluid attenuation inversion recovery (FLAIR), T1 weighted (T1), T1 weighted enhancement (T1c), and T2 weighted (T2). Precise brain tumor segmentation is a key step for preoperative planning diagnosis, growth rate prediction and postoperative disease monitoring. However, manual segmentation of brain tumors is a time-consuming and arduous task. Expert manual annotation is easily disturbed by subjective judgment, and due to fatigue or lack of experience, the possibility of inaccurate operation and division by doctors is large, leading to missed diagnosis and misdiagnosis of some diseases, which seriously affects the life and health of patients. Therefore, an automatic brain tumor segmentation scheme has become a key and widely studied task.

[0004] In recent years, Convolutional Neural Networks (CNNs) have been rapidly applied and developed in the field of medical image segmentation due to their excellent feature extraction capabilities. This technology has achieved outstanding results in brain tumor segmentation. CNN networks capture more feature information and global context data by stacking convolutional layers and leveraging the complementary advantages of translation invariance and shared sliding window strategies, thereby obtaining a wider receptive field. With the introduction of algorithms such as U-Net, Attention U-Net, and U-Net++, end-to-end semantic segmentation models based on U-shaped structures have become the mainstream in brain tumor segmentation. They introduce residual or recurrent structures and combine with multi-scale information interaction, attention mechanisms, and other technologies to make the model have more powerful information representation capabilities and further improve segmentation results. For example, Chen et al. introduced a cascaded binary classifier in FSENet to segment each tumor sub-region, then erased it to classify other tumor regions. Wang et al. and Yogananda et al. transformed the multi-classification problem into three binary segmentation problems and designed three different networks for complete tumor, tumor core, and enhanced tumor segmentation, effectively solving the class imbalance problem. To explore the hierarchical structure of tumor sub-regions, Xu et al. used a multi-branch network with cascaded attention mechanisms for each sub-region. Ghassemi et al. used a GAN model to generate MR images and trained a deep neural network as a discriminator to perform multi-classification of brain tumors. However, these methods mainly rely on the extraction and processing of local features, and when faced with complex and multi-modal MRI, it is difficult to fully capture the various semantic information in the image. In recent years, deep learning models based on Transformer have been gradually introduced into medical image segmentation, especially when dealing with different modal images. The self-attention mechanism of Transformer can effectively capture global information and long-range dependencies. In the task of brain tumor segmentation, how to effectively fuse the complementary information between different modalities remains a challenge to be solved.

[0005] In addition, edge information also plays a crucial role in medical image segmentation. Given the complex morphology and variable tissue characteristics of brain tumors, traditional deep learning methods often struggle to accurately extract tumor boundaries, leading to blurred boundaries or inaccurate segmentation.

[0006] Therefore, the existing brain tumor segmentation algorithms still have certain deficiencies in processing multi-modal MRI data, feature fusion and boundary details. In order to overcome these challenges, the present application proposes a brain tumor segmentation algorithm based on multi-path modal fusion and edge-driven attention. The algorithm more accurately extracts and fuses multi-modal information by using a multi-path feature learning stream based on Transformer, a multi-scale modal fusion module and edge-driven attention, while preserving detailed edge features during the segmentation process, thereby improving the accuracy and boundary clarity of brain tumor segmentation. The present application realizes accurate segmentation of brain tumors, providing an innovative and effective technical means for early diagnosis and treatment of brain tumors. SUMMARY

[0007] According to the problems existing in the prior art, the present application provides a brain tumor segmentation algorithm based on multi-path modal fusion and edge-driven attention, aiming to solve the problems of unclear brain tumor segmentation boundary, missing complementary characteristics and feature redundancy during multi-modal feature fusion.

[0008] The brain tumor segmentation algorithm based on multi-path modal fusion and edge-driven attention in the present application includes the following steps:

[0009] S1: Obtain the original brain tumor image (HxWxD), and slice the original 3D data into 2D images (HxW) along the axial direction. Use z-score normalization to determine the center of the image by statistical analysis of the background area range;

[0010] S2: The input four-channel image is decomposed into two independent two-channel features along the channel dimension, and a 3x3 convolution is used to set the output channel equal to the number of classifications;

[0011] S3: The decomposed image is simultaneously transmitted into a dual-mainstem feature learning stream based on Transformer, and feature representations are extracted from the corresponding brain tumor modalities, respectively. The modal attributes obtained after each stage in the two independent paths are simultaneously transmitted as inputs into a third shared fusion path based on Transformer, which deeply excavates the multi-modal semantic features and edge features in different modalities of MRI;

[0012] S4: In the decoding stage, the upper and lower two independent paths use edge-driven attention to receive modal features from the corresponding levels in the encoding stage and high-level prediction information in the decoding stage, as well as high-frequency features obtained through traditional digital image processing during the gradual upsampling process, so that the shallow edge feature information plays a guiding role in fine-tuning the edge details of brain tumor subregions, and a clear boundary prediction result is obtained;

[0013] S5: The decoding part of the third shared fusion path adopts a skip connection in cooperation with a multi-scale mode fusion module, fully fuses multi-scale brain tumor features, and performs upsampling through a linear two interpolation method to obtain a high-resolution prediction result, so as to better locate and distinguish the brain tumor region;

[0014] S6: A hybrid boundary loss function is used to evaluate the difference between the prediction result and the true label of the brain tumor segmentation result generated by the three paths.

[0015] The brain tumor segmentation algorithm based on multi-path modal fusion and edge-driven attention provided by the present application can effectively solve the problems of fuzzy brain tumor segmentation boundary, missing complementary characteristics and feature redundancy in multi-modal feature fusion in the prior art. Through the multi-path modal fusion strategy, the present application fully excavates the semantic information and edge features in different modal MRIs, and the multi-path Transformer architecture optimizes the fusion of multi-modal features, overcoming the redundancy and complementary missing of different modal features. In the decoding stage, the edge-driven attention can refine the edge details of the tumor sub-region to obtain a clearer segmentation boundary. At the same time, the multi-scale mode fusion module and the skip connection further enhance the multi-modal feature fusion capability, realize more accurate tumor positioning and distinction, and ensure accurate prediction with high resolution. Finally, the hybrid boundary loss function makes the difference between the prediction result and the true label more accurate, optimizing the training effect of the model. The present application can provide high-quality automatic brain tumor segmentation results, providing more reliable auxiliary support for clinical diagnosis and treatment.

[0016] Further, in S1, the original 3D brain tumor image (HxWxD) is obtained, which is sliced into 2D images (HxW) along the axial plane. The z-score normalization is used to standardize the four modal images (except the GT file) to zero mean and unit standard deviation, and the center cropping image size is specifically cropped according to the irrelevant background area.

[0017] The beneficial effect is that the slicing operation converts the originally complex 3D data into relatively simple 2D images, which helps to reduce memory occupation and computational burden, and also provides more accurate local information for subsequent processing. The z-score normalization is used to standardize the images of different modalities to zero mean and unit standard deviation, eliminating the deviation caused by the difference in image intensity between different modalities, so that the model can more fairly process the features of each modality. The center cropping can remove the irrelevant background area, reduce the computational complexity, and ensure that the brain tumor region is more concentrated in the image.

[0018] Further, in S2, the original four-channel brain tumor image The specific channel selection and concatenation operation along the channel dimension is decomposed into two independent features X1 and X2. The feature maps after feature recombination are all 2 channels. Among them, feature X1 selects the second and third channels, and feature X2 selects the first and fourth channels. The formula for the channel selection and concatenation operation of the feature map is as follows:

[0019]

[0020] Wherein represents selecting a specific channel of the feature map, i represents the specific feature map referred to, when i = 1 when i = 2 Concat represents channel concatenation along the channel dimension.

[0021] The beneficial effect is: through the specific channel selection and concatenation operation of the original four-channel brain tumor image, it is ensured that each channel has higher feature expression capability when inputting the network, the key information of each single modality is effectively extracted, the most representative features in each modality are retained, and the interference of redundant information is avoided. Using 3x3 convolution to adjust the output channel to the classification number ensures that the feature map can adapt to the subsequent classification task, and improves the pertinence and efficiency of feature representation.

[0022] Further, in the S3, the Transformer-based double-main-stem feature learning stream is responsible for extracting feature representations from different brain tumor modality images. The first feature learning stem Transformer1 extracts feature representation F1 from the first group of modality images X1 e The second feature learning stem Transformer2 extracts feature representation F2 from the second group of modality images X2

[0023]

[0024] After the feature extraction of each level in the two paths is completed, the modality attributes learned in each stage are retained and

[0025]

[0026] Wherein Attributes represents the modality attributes extracted in each stage. The attributes can be high-level semantic information (such as tissue structure, tumor area, etc.) and low-level features (such as edges, textures, etc.) contained in the image. The modality attributes extracted from each stage of the two independent paths and are used as the double input source of the third shared fusion path Transformer3, and the multi-modal feature representation is obtained:

[0027]

[0028] The beneficial effect is that the Transformer-based multi-path feature learning flow effectively captures high-level semantic features (brain tumor subregion tissue structure) and low-level visual features (edges, detailed textures, etc.) as well as long-range dependencies in brain tumor images of different modalities through its powerful global modeling capabilities and flexible feature expression.

[0029] The multi-path structure ensures the diversity and comprehensiveness of information by processing features from different modalities or different angles in parallel. Each path focuses on a specific type of feature or the details of a different modality, allowing each path to learn more refined and unique features. By assigning features to different paths, the model can avoid processing all information in one path, thereby reducing the risk of information redundancy and loss. The two independent feature extraction paths effectively extract the key information of each modality and enhance the expressive power of a single modality; the shared fusion path not only enhances the complementarity between different modalities, but also improves the efficiency and accuracy of the model when processing multimodal data, which helps to more accurately segment brain tumor areas.

[0030] Furthermore, in the above S4, the upper and lower independent paths use edge-driven attention (EDA) to guide the network to focus on the edge features of the lesion area during the step-by-step upsampling process. This module has three input sources, namely: the modal features corresponding to each level in the encoding layer or High-level prediction features generated by each stage of the decoder or High-frequency features after Laplacian pyramid processing or The specific process includes:

[0031] The decoded prediction features are weighted and decomposed to generate two attention maps: the first is non-salient attention, and the calculation formula is The attention score of the background area is combined with the encoded feature matrix or The second is the boundary attention map, which generates boundary predictions through Gaussian filtering and Laplacian operators, and combines them with or Then, the high-frequency features pre-processed by the Laplacian pyramid are multiplied element by element. or Use bilinear interpolation to adjust to or The same spatial size is multiplied to extract high-frequency information segments. In order to optimize the fusion of multimodal information, the background features are or boundary features or high-frequency features or Concatenate along the channel dimension, then element-wise compression and extraction through fusion convolutional layers. The specific operation is as follows:

[0032]

[0033] Introduce attention masks at the i-th layer, dynamically adjust the importance of different features through residual connection with the original encoder features, highlight the tumor area while suppressing irrelevant background information and noise, and guide the network to focus on the edge and shape features of the brain tumor. Then the features are transmitted into the channel depth convolution calibration block to calibrate the features from the channel and spatial dimensions, further capture the feature correlation between the boundary and the background area, and the calculation method is as follows:

[0034]

[0035] Beneficial effects are: edge-driven attention accurately guides the network to focus on the edge features of the tumor area by combining the weighted fusion of encoders, decoders and high-frequency features. By generating non-significant attention maps and boundary attention maps, background noise is suppressed and tumor boundaries are highlighted, ensuring that tumor shapes are accurately captured. At the same time, the original high-frequency boundary features after Laplacian pyramid and Gaussian denoising are used to assist the network in learning edge detail information, and the channel depth convolution calibration block is used to align the features from the spatial and channel dimensions.

[0036] Further, in S5, the decoding part of the third shared fusion path adopts a jump connection in cooperation with a multi-scale mode fusion module, and the specific process includes:

[0037] Specific modal features of the two independent path encoders and respectively multiplied by the features

[0038] Multiply the resulting features by channel concatenation and convolution operations, while preserving the original information of the shared fusion path encoding stage and focusing on the specific modal tumor region features. In order to further enhance the features of the shared fusion path encoding stage, and and also perform element-wise multiplication of matrices, and the calculation formula is as follows:

[0039]

[0040] Through the deep fusion interaction between multiple features, it is ensured that the network focuses on capturing the features of common attention from all modalities, and the calculation formula is as follows:

[0041]

[0042] Subsequently, the features of different levels of the shared fusion path encoding stage are input as input features into the cross attention to realize multi-level and multi-scale feature interaction, so that the high-level features guide the low-level features, the network focuses on the tumor area, and irrelevant tissue interference is excluded, and the final multi-modal fusion features are generated.

[0043] The beneficial effects are: the modal attributes learned by the two independent paths at each stage are input into the shared fusion path as input, and the multi-modal feature deep fusion and interaction are realized through the multi-scale modal fusion module. The modal features from the three paths are multiplied and spliced element by element to retain the original information of the shared path. The cross attention mechanism makes the high-level features in the shared fusion path guide the low-level features, obtain feature information of different scales, ensure that the network focuses on the tumor area, and excludes irrelevant background interference. In addition, the splicing operation of the feature map can retain the complementary information between different modalities, thereby enhancing the multi-modal learning ability of the model, enabling the network to better understand and fuse the semantic and structural information in different modalities, thereby realizing the deep fusion of different modal information and improving the multi-modal learning ability of the model.

[0044] Further, in the S6, a hybrid boundary loss function is used to evaluate the difference between the predicted results and the true labels of the brain tumor segmentation results generated by the three paths.

[0045] The beneficial effects are: the hybrid boundary loss function is used to evaluate the difference between the predicted results and the true labels, which can comprehensively consider the detailed information of the tumor boundary, especially in the case of fuzzy or irregular tumor edges, and can better maintain the boundary consistency and accuracy of the segmentation results, thereby reducing false positives and false negatives. This method can provide more accurate and stable results for automatic brain tumor segmentation.

[0046] Other advantages, objects, and features of the present application will be in part apparent and in part pointed out hereinafter in the specification, and will be learned from a reading of the following specification and by practicing the present application. The objects and other advantages of the present application will be realized and attained by the structure particularly pointed out in the specification. BRIEF DESCRIPTION OF DRAWINGS

[0047] In order to make the objects, technical solutions and advantages of the present application clearer, the preferred detailed description of the present application will be made below in combination with the drawings, wherein:

[0048] Figure 1 is the overall network structure schematic diagram of the brain tumor segmentation algorithm based on multi-path modal fusion and edge-driven attention of embodiment one of the application;

[0049] Figure 2 is the edge-driven attention schematic diagram of the brain tumor segmentation algorithm based on multi-path modal fusion and edge-driven attention of embodiment one of the application;

[0050] Figure 3 is the multi-scale modal fusion module schematic diagram of the brain tumor segmentation algorithm based on multi-path modal fusion and edge-driven attention of embodiment one of the application;

[0051] Figure 4 is the brain tumor segmentation target of the brain tumor segmentation algorithm based on multi-path modal fusion and edge-driven attention of embodiment one of the application;

[0052] Figure 5 is the ablation experiment visualization result diagram in embodiment two of the brain tumor segmentation algorithm based on multi-path modal fusion and edge-driven attention of the application; DETAILED DESCRIPTION

[0053] In order to make the technical solutions and advantages of the application clearer, the technical solutions in the embodiments of the application will be described clearly and completely below, and the technical solutions in the embodiments of the application will be described in further detail below through specific implementation manners.

[0054] Embodiment one

[0055] The application provides a brain tumor segmentation algorithm based on multi-path modal fusion and edge-driven attention, which captures brain tumor semantic information and edge features from multiple dimensions and multiple scales, and effectively solves the problems of unclear brain tumor segmentation boundary, missing complementary characteristics and feature redundancy in multi-modal feature fusion when segmenting the three lesion regions of complete tumor (WT), enhanced tumor (ET) and tumor core (TC).

[0056] Figure 1 The overall architecture schematic diagram of the application is shown in the figure, the framework includes two independent backbone paths, which respectively extract specific channel features for single tumor modal, and a shared modal fusion path is introduced to integrate multi-modal brain tumor information. The decoding part corresponds to the corresponding level of the encoder, wherein: the two independent decoding paths use edge-driven attention to refine tumor edge details, and the specific structure of the edge-driven attention is shown in the figure; the decoding stage of the shared fusion path receives features from the three encoding paths at each level, and uses a multi-scale modal fusion module to deeply fuse multi-modal multi-scale information, and the specific structure of the multi-scale modal fusion module is shown in the figure. Figure 2 Figure 3 ​As shown. Finally, after a series of upsampling operations, the decoder can generate high-resolution output feature maps.

[0057] The brain tumor segmentation algorithm based on multi-path modal fusion and edge-driven attention includes the following steps:

[0058] S1: Obtain the original brain tumor image (HxWxD), and slice the original 3D data into 2D images (HxW) along the axial direction. Adopt z-score normalization, and determine the center clipping image by statistical irrelevant background area range;

[0059] S2: The input four-channel image is decomposed into two independent two-channel features along the channel dimension, and a 3x3 convolution is used to set the output channel equal to the number of classifications;

[0060] S3: The decomposed image is simultaneously transmitted into the dual-main-stem feature learning stream based on Transformer, and the feature representation is extracted from the corresponding brain tumor modal, respectively. At the same time, the modal attributes obtained after the end of each stage in the two independent paths are taken as inputs and simultaneously transmitted into the third shared fusion path based on Transformer, to deeply mine the multi-modal semantic features and edge features in different modal MRIs;

[0061] S4: In the decoding stage, the upper and lower two independent paths utilize edge-driven attention to receive modal features from the corresponding levels in the encoding stage and high-level prediction information in the decoding stage, as well as high-frequency features obtained through traditional digital image processing, during the gradual upsampling process, so that the shallow edge feature information plays a guiding role in fine brain tumor sub-region edge details, and a clear boundary prediction result is obtained;

[0062] S5: The decoding part of the third shared fusion path adopts skip connection and uses multi-scale mode fusion module, fully fuses multi-scale brain tumor features, and performs upsampling through linear interpolation method to obtain high-resolution prediction results, so as to better locate and distinguish the brain tumor region;

[0063] S6: The hybrid boundary loss function is used to evaluate the difference between the prediction result and the true label for the brain tumor segmentation results generated by the three paths.

[0064] In step S2, the original four-channel brain tumor image is decomposed into two independent features X1 and X2 through specific channel selection and splicing operation along the channel dimension. The feature maps after feature recombination are both two channels. Among them, feature X1 selects the second and third channels, and feature X2 selects the first and fourth channels. The channel selection and splicing operation formula for the feature map is as follows:

[0065]

[0066] wherein represents selecting a specific channel of the feature map as c, i represents the specific feature map referred to, when i = 1 when i = 2 Concat represents channel concatenation along the channel dimension.

[0067] In step S3, the Transformer-based double-main-stem feature learning flow is responsible for extracting feature representations from different brain tumor modality images, and the first feature learning main stem Transformer 1 extracts feature representations from the first group of modality images X1 The second feature learning main stem Transformer 2 extracts feature representations from the second group of modality images X2

[0068]

[0069] After the feature extraction of each level in the two paths is completed, the modality attributes learned in each stage are retained and

[0070]

[0071] wherein Attributes represents the modality attributes extracted in each stage. The attributes can be high-level semantic information (such as tissue structure, tumor area, etc.) and low-level features (such as edges, textures, etc.) contained in the image. The modality attributes extracted from each stage in the two independent paths are and As the double-input source of the third shared fusion path Transformer 3, the multi-modality feature representation is obtained:

[0072]

[0073] In step S4, the upper and lower independent paths use edge-driven attention guidance networks to focus on the edge features of the lesion area during the gradual upsampling process. This module has three input sources, which are: the modality features corresponding to each level in the encoding layer or the high-level prediction features generated by the decoder at each stage or high-frequency features after Laplacian pyramid processing or As shown in Figure 2 , the specific process includes:

[0074] The decoded prediction features are weighted and decomposed to generate two attention maps: the first is non-significant attention, and the calculation formula is The attention score of the background region is multiplied with the encoded feature matrix to obtain the weighted background feature; the second is the boundary attention map, which is generated by Gaussian filtering and Laplacian operator to predict the boundary, and is multiplied with or element by element. Subsequently, the high-frequency features processed by the Laplacian pyramid preprocessing or are adjusted to the same spatial size by bilinear interpolation and multiplied to extract the high-frequency information segment. To optimize the fusion of multi-modal information, the background features or boundary features or high-frequency features or are concatenated along the channel dimension, and then compressed and extracted element by element by a fusion convolution layer. The specific operation is as follows: or or In the i-th layer, an attention mask is introduced, which is connected with the original encoder features by residual connection, dynamically adjusts the importance of different features, highlights the tumor region while suppressing irrelevant background information and noise, and guides the network to focus on the brain tumor edge and shape features. Subsequently, the features are transmitted to the channel depth convolution calibration block to calibrate the features from the channel and spatial dimensions, further capturing the feature correlation between the boundary and the background region, and the calculation method is as follows:

[0075]

[0076] In the i-th layer, an attention mask is introduced, which is connected with the original encoder features by residual connection, dynamically adjusts the importance of different features, highlights the tumor region while suppressing irrelevant background information and noise, and guides the network to focus on the brain tumor edge and shape features. Subsequently, the features are transmitted to the channel depth convolution calibration block to calibrate the features from the channel and spatial dimensions, further capturing the feature correlation between the boundary and the background region, and the calculation method is as follows:

[0077]

[0078]

[0079] In step S5, the decoding part of the third shared fusion path adopts a skip connection in cooperation with a multi-scale mode fusion module, and the specific process includes:

[0080] The specific modal features of the two independent path encoders and are multiplied with the features from the shared fusion path encoder, respectively, and the resulting features are concatenated and convolved in the channel to retain the original information of the shared fusion path encoding stage while focusing on the tumor region features of the specific modal. In order to further enhance the features of the shared fusion path encoding stage, and and are also multiplied element by element, and the calculation formula is as follows: ​

[0081]

[0082] Through the deep fusion interaction between multiple features, it is ensured that the network focuses on capturing the features of common attention from all modalities, and the calculation formula is as follows:

[0083]

[0084] Subsequently, the features of different levels in the shared fusion path encoding stage are input as input features into the cross attention to realize multi-level and multi-scale feature interaction, so that high-level features guide low-level features, the network focuses on the tumor area, and irrelevant tissue interference is excluded, and finally the multi-modal fusion features are generated.

[0085] Embodiment two

[0086] The BraTS2020 dataset is selected to verify the effective performance of the algorithm. The BraTS2020 dataset contains 369 cases in the training set, 125 cases in the verification set, and 166 cases in the test set. Each case data includes four 3D MRI sequences, which are T1 weighted MRI (T1), T1 weighted MRI enhancement (T1ce), T2 weighted MRI (T2), fluid attenuated inversion recovery (Flair) and one true value label. All different modal MRI data are strictly aligned and registered to the same general template, resampled to 1mm 3 uniform resolution and skull stripping. The true value label annotated by experts is composed of four types of background, tumor necrosis area and non-enhanced area (NCR / NET), peritumoral edema area (ED) and tumor enhanced area (ET). According to the evaluation standard of BraTs competition, the embodiment utilizes three nested combinations of label sub-regions to evaluate the accuracy of model prediction, as shown in Figure 4 The final segmentation target is three different types of brain tumor regions: complete tumor (the Whole Tumor, WT: NCR / NET+ED+ET), tumor core (Tumor Core, TC: NCR / NET+ET) and enhanced tumor (Enhancing Tumor, ET).

[0087] Model training and evaluation are carried out on an NVIDIA GeForce RTX 4090 GPU, and the experimental platform belongs to a Windows 116 64-bit operating system with 24G memory. The framework proposed in the application is built based on PyTorch. In the training process, Adam with a momentum of 0.9 is used as the optimizer. The initial learning rate, weight decay and batch size are set to 3e-4, 1e-4 and 16 respectively, and the experiment adopts a cosine annealing learning rate scheduler to train the model for 500 rounds.

[0088] To verify whether the algorithm model is effective. The present application carries out ablation experiment on BraTS2020 dataset, adds different modules in the multi-path model in turn, carries out experiment using the same parameter setting, and the specific results are shown in Table 1, wherein MPSE represents multi-path semantic extraction, MMFB represents multi-scale mode fusion module, BDA represents edge driven attention, and MB-Loss represents mixed boundary loss function.

[0089] Table 1 Objective evaluation results of ablation study on BraTS2020 dataset

[0090]

[0091] As can be seen from Table 1, the network architecture using multi-path feature extraction obtains good segmentation accuracy for the three classes of brain tumor sub-regions, and the Dice coefficients of WT, TC and ET are 95.3%, 94.4% and 90.6%, respectively, and the Hausdorff distances are 2.20mm, 1.59mm and 1.32mm, respectively. The performance of the model is further enhanced after introducing the multi-scale mode fusion module to fuse the features of different modalities, and the Dice coefficients are 95.6%, 95.1% and 91.2%, respectively, and the HD is 2.20mm, 1.55mm and 1.34mm. Then the edge driven attention is introduced to pay attention to the local and edge detail features, and the maximum degree is avoided to generate fuzzy images, and the Dice scores are increased to 96.0%, 95.4% and 91.8%, respectively, and the HD is also slightly improved (WT 2.15mm, TC 1.53mm, ET 1.34mm). After introducing the MB-Loss, the model reaches the best performance in all indicators, and the accuracy of the tumor boundary is significantly optimized. Figure 5 It can be seen that the ablation experiment verifies the gradual improvement of the performance of the model by the three modules of MMFB, EDA and MB-Loss, which shows that the synergistic effect between the modules is crucial for improving the segmentation quality and boundary accuracy, and the final optimization effect proves the effectiveness of the proposed algorithm.

[0092] Table 2 Objective evaluation results of related algorithms on BraTS2020 dataset

[0093]

[0094] The experimental results show that the proposed algorithm model performs best in brain tumor segmentation task, especially in Dice coefficient and Hausdorff distance (HD) indicators. Specifically, the Dice coefficients of the model in WT, ET and TC three sub-regions are 96.52%, 95.81% and 92.02% respectively, and the Hausdorff distances are 2.11mm, 1.41mm and 1.30mm respectively, which are significantly better than other methods. In contrast, other models such as UCTransNet and IVD-Net also have high performance in Dice coefficient (92.37% and 92.53% respectively), but are slightly inferior in boundary accuracy (HD). Overall, the proposed algorithm model significantly improves the comprehensive performance of brain tumor segmentation through more efficient segmentation and boundary preservation ability, proving its advantage in this task.

[0095] The above-mentioned is only an embodiment of the present application, and the well-known specific structure and characteristics in the scheme are not described too much. It should be pointed out that for those skilled in the art, without departing from the structure of the present application, a number of modifications and improvements can be made, which should be regarded as the protection scope of the present application, which will not affect the effect and practicality of the patent. The protection scope claimed in this application should be subject to the content of its claims, and the specific implementation mode and the like recorded in the specification can be used to explain the content of the claims.

Claims

1. A brain tumor segmentation algorithm based on multi-path modality fusion and edge-driven attention, characterized in that, The method comprises the following steps: S1: obtaining an original brain tumor image (HxWxD), slicing the original 3D data into 2D images (HxW) along the axial plane, performing z-score normalization, and determining the center clipping image by counting the range of irrelevant background regions; S2: the input four-channel image is decomposed into two independent two-channel features along the channel dimension, and a 3x3 convolution is used to set the output channel equal to the number of classifications; S3: the decomposed image is simultaneously transmitted into a dual-main-stem feature learning stream based on Transformer, and feature representations are extracted from the corresponding brain tumor modalities, respectively, while retaining the modal attributes obtained after each stage in the two independent paths, which are simultaneously transmitted into a third shared fusion path based on Transformer, to deeply mine the multi-modal semantic features and edge features in different modalities of MRI; S4: in the decoding stage, the upper and lower independent paths receive modal features from the corresponding levels in the encoding stage and prediction information from the decoding stage at a higher level, as well as high-frequency features obtained through traditional digital image processing, during the gradual upsampling process, using edge-driven attention, so that shallow edge feature information can play a guiding role in fine brain tumor sub-region edge details, and a clear boundary prediction result is obtained; S5: the decoding part of the third shared fusion path adopts a skip connection in combination with a multi-scale mode fusion module, fully fuses multi-scale brain tumor features, and performs upsampling through a linear two interpolation method to obtain a high-resolution prediction result, so as to better locate and distinguish the brain tumor region; S6: a hybrid boundary loss function is used to evaluate the difference between the prediction result and the true label for the brain tumor segmentation results generated by the three paths.

2. The brain tumor segmentation algorithm based on multi-path modality fusion and edge-driven attention according to claim 1, characterized in that: In the step S2, the original four-channel brain tumor image is processed The specific channel selection and splicing operation along the channel dimension is decomposed into two independent features X1 and X2. The feature maps after feature recombination are both two channels, wherein the feature X1 selects the second and third channels, and the feature X2 selects the first and fourth channels. The channel selection and splicing operation formula of the feature map is as follows: wherein represents selecting a particular channel of a feature map as c, i represents the feature map specifically referred to, when i = 1 when i = 2 Concat represents channel concatenation along the channel dimension.

3. The brain tumor segmentation algorithm based on multi-path modality fusion and edge-driven attention according to claim 2, characterized in that: In the step S3, the Transformer-based double backbone feature learning flow is responsible for extracting feature representations from different brain tumor modality images, and the first feature learning backbone Transformer 1 is used to extract feature representations F1 from the first group of modality images X1 e , and the second feature learning backbone Transformer 2 is used to extract feature representations F2 from the second group of modality images X2 After the feature extraction of each level in the two paths is completed, the modal attributes learned at each stage are retained and where Attributes denotes the modal attributes extracted in each stage, which can be high-level semantic information and low-level features contained in the image; the modal attributes extracted from each stage in two independent paths and As the dual-input source of the third shared fusion path Transformer3, the multi-modal feature representation is obtained: 。 4. The brain tumor segmentation algorithm based on multi-path modality fusion and edge-driven attention of claim 3, wherein: In step S4, the two independent paths use edge-driven attention guidance network to focus on the edge features of the lesion area in the step-by-step upsampling process. The module has three input sources, which are the modal features corresponding to each level in the encoding layer or high-level prediction features generated by each stage of the decoder or high-frequency features processed by the Laplacian pyramid or The specific process includes: The decoded prediction features are weighted and decomposed to generate two attention maps respectively: the first one is the non-significant attention, and the calculation formula is The attention score of the background region is multiplied with the feature matrix obtained by encoding or to obtain the weighted background features; the second one is the boundary attention map, and the boundary prediction is generated through Gaussian filtering and Laplacian operator, which is multiplied with or element by element; subsequently, the high-frequency features or preprocessed by the Laplacian pyramid are adjusted to the same spatial size as or by using the bilinear interpolation method and multiplied to extract the high-frequency information segment; in order to optimize the fusion of multi-modal information, the background features or the boundary features or and the high-frequency features or are spliced along the channel dimension, and then element-wise compression and extraction are performed through the fusion convolution layer, and the specific operation is as follows: Attention masks are introduced at the i-th layer, which are connected with the original encoder features through residual connection to dynamically adjust the importance of different features, highlight the tumor region while suppress irrelevant background information and noise, and guide the network to focus on the brain tumor edge and shape features; then the features are transmitted into a channel depth convolution calibration block to calibrate the features from the channel and spatial dimensions, further capture the feature correlation between the boundary and the background region, and the calculation method is as follows: 。 5. The brain tumor segmentation algorithm based on multi-path modality fusion and edge-driven attention of claim 4, wherein: In step S5, the decoding part of the third shared fusion path adopts a skip connection in combination with a multi-scale mode fusion module, and the specific process includes: Specific modal features f of two independent path encoders i 1 and f i 2 respectively with features f from the shared fusion path encoder i 3 Feature multiplication is performed pairwise, and the resulting features are channel concatenated and convolved, preserving the original information of the shared fusion path encoding stage while focusing on specific modal tumor region features; to further enhance the features of the shared fusion path encoding stage, f i 1 and f i 2 and f i 3 Element-wise matrix multiplication is also performed, and the calculation formula is as follows: Through deep fusion and interaction among multiple features, the network focuses on capturing common attention features from all modalities, and the calculation formula is as follows: Then, the features of different levels in the encoding stage of the shared fusion path are transmitted into cross-attention as input features to realize multi-level and multi-scale feature interaction, so that high-level features guide low-level features, the network focuses on the tumor region, and irrelevant tissue interference is excluded, and the final multi-modal fusion features are generated.

Images