Brain Tumor Segmentation Method and System Based on Multimodal MRI

Abstract

The present invention provides a brain tumor segmentation method and system based on multimodal MRI, including: acquiring multimodal MRI images; performing rigid registration on the multimodal MRI images using affine transformation; splicing the registered multimodal MRI images along the channel dimension to form an initial input tensor, and feeding it into the BTSN encoder of the segmentation network to extract multimodal MRI features; using the multimodal MRI features and mirror features as inputs to the Symmetric Boundary Visual Difference Module (SVDB) to extract symmetric boundary visual difference features; connecting the multimodal MRI features, mirror features, and symmetric boundary visual difference features to the corresponding-level decoding modules through skip connections for decoding, gradually restoring the resolution of the segmented image until the final convolutional layer obtains a segmentation result with the same size as the original MRI. The present invention can generate accurate tumor segmentation results, significantly improving the accuracy and robustness of tumor segmentation.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and specifically, to a brain tumor segmentation method and system based on multi-modal MRI. Background Art

[0002] Brain tumors are one of the most common diseases in the central nervous system, and their types are diverse, including gliomas, meningiomas, metastatic tumors, etc. Early diagnosis and accurate segmentation are crucial for the treatment plan and prognosis assessment of patients. Multi-modal magnetic resonance imaging (MRI), due to its non-invasiveness, high resolution, and multi-sequence imaging capabilities (such as T1-weighted contrast-enhanced imaging (T1CE) and T2-weighted fluid-attenuated inversion recovery sequence (T2Flair)), has become the main tool for brain tumor detection and segmentation. However, existing brain tumor segmentation techniques still face many challenges when dealing with complex multi-modal MRI data.

[0003] Traditional brain tumor segmentation methods mainly rely on manually designed feature extraction techniques, such as threshold-based segmentation, region growing algorithms, etc. Although these methods are simple and easy to implement, when dealing with multi-modal MRI data, it is difficult to comprehensively capture the complex information in the images, especially the complementary information between different modalities. The design of manual features often relies on experience and lacks a comprehensive understanding of the global and local features of the images, resulting in limited segmentation accuracy. In recent years, convolutional neural networks (CNNs) based on deep learning have made significant progress in the field of medical image segmentation. In particular, network structures such as U-Net have shown excellent performance in local feature extraction. However, CNNs mainly rely on local receptive fields and are difficult to capture the global context information in the images. The morphology and location of brain tumors are variable, and it is difficult to achieve accurate segmentation only relying on local features, especially when the tumor boundary is blurred or the contrast with the surrounding tissues is low. For example, the volume of brain tumors varies greatly, especially early tumors or small lesions, which are small in volume and have a low contrast with the surrounding tissues. Existing segmentation algorithms are often easily masked by the background features of large regions when dealing with small targets, resulting in a decrease in the segmentation accuracy of small target regions. In addition, multi-modal MRI data (such as T1CE, T2Flair, etc.) provides tumor information under different sequences. How to effectively fuse these multi-modal data to improve the segmentation accuracy is an important challenge. Existing methods usually adopt simple feature concatenation or weighted fusion strategies, which are difficult to fully utilize the complementary information between multi-modal data, resulting in less robust segmentation results.

[0004] Patent document CN111612754B discloses a method and system for optimizing the segmentation of MRI tumors based on multi-modal image fusion, including: Step 1: Construct a network for multi-modal image fusion of MRI tumors; Step 2: Construct a multi-modal 3D network for enhancing tumor image segmentation; Step 3: Based on GAN-based image fusion, construct a saliency loss function; Step 4: Construct a Mask attention mechanism contrast loss function; Step 5: Construct an SSIM loss function; Step 6: Perform optimized segmentation of MRI tumors according to the network for multi-modal image fusion of MRI tumors, the multi-modal 3D network, and the three loss functions. However, this patent document lacks global feature fusion and dynamic loss optimization.

[0005] Patent document CN111815624B discloses a method and system for determining the tumor stroma ratio based on an image processing algorithm, including: Step M1: Read the HE immunohistochemical image of the tumor pathological section; Step M2: Select the images with the average gray value and the degree of image blurriness within a preset range; Step M3: Perform preprocessing on the selected images based on an image preprocessing algorithm to obtain the preprocessed images; Step M4: Segment the preprocessed images; Step M5: Obtain the segmentation result and label it, and calculate the stroma ratio between tumors. However, it does not combine multi-modal data and the Transformer architecture.

[0006] Patent document CN117372687A discloses a method for segmenting multi-modal neural images of tumors based on deep learning, including: obtaining a pair of PET-CT images composed of a positron emission tomography (PET) image and its corresponding computed tomography (CT) image, preprocessing the pair of PET-CT images to obtain the preprocessed pair of PET-CT images, and inputting the preprocessed pair of PET-CT images into a trained multi-modal neural image tumor segmentation model to obtain the final predicted result of lesion segmentation.

[0007] However, for the brain tumor segmentation of PET and CT images in patent document CN117372687A, there is an influence of the order of first encoding the PET image and then encoding the CT image, and only CNN is used while ignoring the global data encoding ability of the Transformer. In addition, when this patent document is performing intermediate feature fusion, it only considers using spatial domain fusion and spatio-temporal domain fusion, and does not pay attention to encoding the brain tumor tissue structure with left-right brain pseudo-symmetry, making it difficult to obtain a more accurate lesion segmentation result.

[0008] Therefore, there is an urgent need in the market for a method and system for segmenting brain tumors based on multi-modal MRI that can combine local and global features, improve segmentation accuracy, reduce computational complexity, and have good generalization ability. Summary of the Invention

[0009] Aiming at the defects in the prior art, the purpose of the present invention is to provide a brain tumor segmentation method and system based on multimodal MRI.

[0010] A brain tumor segmentation method based on multimodal MRI provided by the present invention includes:

[0011] Step S1: Obtain multimodal MRI images;

[0012] Step S2: Perform rigid registration on the multimodal MRI images using affine transformation;

[0013] Step S3: Concatenate the registered multimodal MRI images along the channel dimension to form an initial input tensor , and send it into the BTSN encoder of the segmentation network to extract multimodal MRI features;

[0014] Step S4: Mirror-process the MRI images to obtain mirrored MRI images, and extract corresponding mirrored features through parallel mirrored encoders;

[0015] Step S5: Use the multimodal MRI features and the mirrored features as inputs to the Symmetric Boundary Visual Difference Module (SVDB) to extract the symmetric boundary visual difference features of the tumor in the left and right brains;

[0016] Step S6: Connect the multimodal MRI features, the mirrored features, and the symmetric boundary visual difference features to the corresponding-level decoding modules through skip connections for decoding, gradually restoring the resolution of the segmented image until finally obtaining a segmentation result with the same size as the original MRI through the convolutional layer.

[0017] Preferably, the MRI images include T1-weighted contrast-enhanced imaging (T1CE) and T2-weighted fluid-attenuated inversion recovery sequence (T2Flair).

[0018] Preferably, in step S2, Z-score normalization is used to eliminate the differences in scanning devices and protocols, and the formula is:

[0019]

[0020] where represents the image after Z-score normalization, represents the image before Z-score normalization, and represent the mean and standard deviation of the image intensity respectively.

[0021] Preferably, the BTSN encoder is composed of cascaded RSU (Residual U-shaped Subnetwork) modules;

[0022] The embedded U-shaped network of the RSU module is connected through jumps in the encoding-decoding layer, and finally the input features and decoded features are obtained through an addition operation. The output of the module, that is, the multi-modal MRI features , where , The module represents the th RSU module;

[0023] The mirror encoder splices and convolves the mirror image, and then extracts mirror features through RSU modules, that is, mirror features , .

[0024] Preferably, the symmetric boundary visual difference module is used to suppress redundant features and correct features related to image segmentation;

[0025] The symmetric boundary visual difference module includes a spatial context attention module SCM and a feature refinement module FRM with a multi-branch dilated convolution structure, which are used to correct and activate features related to tumor segmentation in multi-scale features.

[0026] Preferably, step S5 includes:

[0027] Step S5.1: Input the multi-modal MRI features and the mirror features , calculate the difference between the original MRI and the mirror MRI through element-wise subtraction, and perform convolution feature extraction on through a convolution layer. The formula is as follows:

[0028]

[0029] where is a convolution operation with a kernel, and is an element-wise subtraction operation;

[0030] Step S5.2: First pass through a normalization layer LayerNorm and a depth convolution layer with a convolution kernel of for channel compression, then use matrix element-wise addition to add the compressed features and the normalized input features element-wise, and finally obtain through a convolution layer. The calculation formula is as follows:

[0031]

[0032] Among them, LN is the LayerNorm operation, is the matrix corresponding addition operation;

[0033] Step S5.3: To capture a wider range of spatial context information in response to the scale change of the input image of the field, the obtained is sent into the spatial context attention module SCM to obtain , and at the same time the nuclear cluster information expression of small targets is obtained through the feature refinement module FRM with a multi-branch atrous convolution structure ;

[0034] Step S5.4: and are further passed through the concat operation and the convolutional layer to obtain the visual difference feature , and the calculation is as follows:

[0035] =

[0036] Among them, is the parameter of the convolution. The output of the symmetric boundary visual difference module, that is, the symmetric boundary visual difference feature is shown as follows:

[0037]

[0038] Among them, is the parameter of the convolution.

[0039] Preferably, in the step S5.3 after being input into the spatial context attention module SCM, is obtained, and the calculation formula is as follows:

[0040] =

[0041] In the formula, is the spatial weighted weight map; The calculation formula is as follows:

[0042]

[0043] Among them, is the matrix multiplication operation;

[0044] The the nuclear cluster information expression of small targets is obtained through the feature refinement module FRM with a multi-branch atrous convolution structure , respectively through convolutional layers and a multi-branch combination of convolutional layers to obtain features, and refine the features by merging the features of different branches. The calculation formula is as follows:

[0045]

[0046] where respectively represent the refined features of 4 branches.

[0047] Preferably, in order to fit the feature dimensions of the RSU module and the Transformer block in step S4, is resampled to to make it coincide with the features of the RSU module.

[0048] Preferably, it further includes: weighting the Dice loss function by given adaptive weight parameters;

[0049] giving higher loss weights to small target regions, and relatively balancing and reducing the loss weights of large-region ROIs. The ARW-dice loss is obtained by the following formula:

[0050]

[0051] where represents the adaptive ROI weight parameter with label index , is the label index, is calculated by the following formula:

[0052]

[0053] where represents the ROI annotation volume with label index , represents the ROI prediction volume with label index .

[0054] According to a brain tumor segmentation system based on multimodal MRI provided by the present invention, it includes:

[0055] Module M1: Obtain multimodal MRI images;

[0056] Module M2: Rigidly register the multimodal MRI images using affine transformation;

[0057] Module M3: Concatenate the registered multimodal MRI images along the channel dimension to form an initial input tensor , it is sent to the BTSN encoder of the segmentation network to extract multi-modal MRI features;

[0058] Module M4: Mirror-process the MRI image to obtain a mirrored MRI image, and extract corresponding mirrored features through parallel mirrored encoders;

[0059] Module M5: Use the multi-modal MRI features and the mirrored features as inputs to the Symmetric Boundary Visual Difference Module (SVDB) to extract the symmetric boundary visual difference features of the tumor in the left and right brains;

[0060] Module M6: Connect the multi-modal MRI features, the mirrored features, and the symmetric boundary visual difference features to the decoding modules at the corresponding levels through skip connections for decoding, gradually restoring the resolution of the segmented image until finally obtaining a segmentation result with the same size as the original MRI through the convolutional layer.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] 1. The present invention designs parallel encoders to process the features of the original image and the mirrored image respectively, and enhances the feature difference between the tumor foreground and the background through the Symmetric Boundary Visual Difference Module (SVDB). Subsequently, the local features of CNN, the global features of Transformer, and the visual difference features are fused, and an accurate tumor segmentation result is generated through the decoder introducing the deep supervision mechanism.

[0063] 2. The adaptive weighted Dice loss function (AWD-Loss) proposed by the present invention can dynamically adjust the weights, assign higher loss weights to small target regions, and appropriately reduce the weights of large-region ROIs, thereby significantly improving the accuracy and robustness of tumor segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, objectives, and advantages of the present invention will become more obvious:

[0065] Figure 1 is the workflow block diagram of the present invention;

[0066] Figure 2 is the schematic diagram of the network structure of the symmetric boundary visual difference module of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0067] The present invention will be described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that those of ordinary skill in the art can make several changes and improvements without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0068] The core components of the present invention include an encoder, a symmetric boundary visual difference module, and a decoder, and the segmentation performance is optimized by combining an adaptive weighted loss function. This method adopts a hybrid architecture that combines a parallel convolutional neural network CNN and a Transformer. It extracts CNN local features from multi-modal MRI images through a nested U-Net structure, and at the same time uses the underlying Transformer encoding module to capture global features, which is suitable for the precise detection and segmentation of brain tumors.

[0069] A method for segmenting brain tumors based on multi-modal MRI provided by the present invention, as Figure 1 shown, includes:

[0070] Step S1: Obtain multi-modal MRI images. The MRI images include T1-weighted contrast-enhanced imaging T1CE and T2-weighted fluid-attenuated inversion recovery sequence T2Flair. The high soft tissue contrast of T1CE and the high sensitivity of T2Flair to the edema area are complementary to improve the identification ability of the tumor boundary.

[0071] Step S2: Use affine transformation to perform rigid registration on the multi-modal MRI images to eliminate the displacement difference between modalities. Z-score normalization is used to eliminate the differences in scanning equipment and protocols. The formula is:

[0072]

[0073] Where represents the image after Z-score normalization, represents the image before Z-score normalization and represent the mean and standard deviation of the image intensity respectively.

[0074] Step S3: Concatenate the registered T1CE and T2Flair images along the channel dimension to form an initial input tensor , and send it into the BTSN encoder of the segmentation network. Subsequently, multi-modal MRI features are extracted through a 3×3 convolutional layer (stride 1, padding 1), where H represents the height of the image and W represents the width of the image. The BTSN encoder consists of a cascade of It consists of the output of the module , where . Additionally, during the convolution process, downsampling operations are used to encode the image into low-resolution features After passing through the features obtained after maintain relatively rich local context features with relatively low computational complexity. Then, is input into the bottom Transformer encoder of the encoder to further learn the long-range dependency information of the global receptive field is used as the input of the bottom Transformer block of the encoder .

[0075] Step S4: Mirror process the MRI image to obtain a mirrored MRI image, and extract corresponding mirrored features through a parallel mirrored encoder. The parallel mirrored encoder splices and performs convolution on the mirrored image, and then extracts mirrored features through the output of the mirrored encoding module, i.e., the mirrored features .

[0076] Step S5: Use the multi-modal MRI features and the mirrored features as the input of the Symmetric Boundary Visual Difference module SVDB to extract the symmetric boundary visual difference features of the tumor in the left and right brains . That is, features are extracted between the original MRI image and the mirrored MRI image and , and the symmetric boundary visual difference features at different levels are obtained through element-wise subtraction operations between the original features and the mirrored features obtained by the same-level encoding module . To enhance the attention to asymmetric regions in the network, a symmetric boundary visual difference module is designed between the original and mirrored image encoding paths to suppress redundant features and correct features related to image segmentation. Given the input of the symmetric boundary visual difference module (SVDB) as , then the output of, i.e., the final symmetric boundary visual difference features are:

[0077]

[0078] where represents the ​​A symmetric boundary visual difference module. The symmetric boundary visual difference module utilizes the pseudo-symmetric structure of the brain and constructs and uses the SVDB module between the features of the original MRI and the symmetric MRI.

[0079] In step S5, the SVDB module includes a spatial context attention module SCM and a feature refinement module FRM with a multi-branch atrous convolution structure, which is used to correct and activate the features related to tumor segmentation in the multi-scale features. As Figure 2 shown, step S5 includes:

[0080] Step S5.1: Input multi-modal MRI features and mirror features , calculate the difference between the original MRI and the mirror MRI through element-wise subtraction, and perform convolution feature extraction on through the convolution layer. The formula is as follows:

[0081]

[0082] where, is the convolution operation with a kernel, and is the element-wise subtraction operation. Define the input of the th SVDB module as , is the index of in the BTSN encoder in step S3, and are the output features of in the BTSN encoder and the mirror encoder that encode the original MRI respectively.

[0083] Step S5.2: First pass through a normalization layer LayerNorm and a depth convolution layer with a convolution kernel of for channel compression, then use the matrix element-wise addition operation to add the compressed features and the normalized input features element-wise, and finally obtain through the convolution layer. The calculation formula is as follows:

[0084]

[0085] where, LN is the LayerNorm operation, is the matrix element-wise addition operation.

[0086] Step S5.3: To capture more extensive spatial context information in response to the scale change of the input image, the obtained Sent to the spatial context attention module SCM to obtain Meanwhile Obtain the nuclear cluster information expression of small targets through the feature refinement module FRM with a multi-branch atrous convolution structure In the step S5.3 After inputting into the spatial context attention module SCM, obtain The calculation formula is as follows:

[0087] =

[0088] In the formula, Is the spatial weighted weight map; The calculation formula is as follows:

[0089]

[0090] Among them, Is the matrix multiplication operation.

[0091] The Obtain the nuclear cluster information expression of small targets through the feature refinement module FRM with a multi-branch atrous convolution structure Respectively pass through Convolution layer and The multi-branch combination of the convolution layer to obtain features, and refine the features by merging the features of different branches. The calculation formula is as follows:

[0092]

[0093] Among them, Respectively represent the refined features of 4 branches.

[0094] Step S5.4: Combine And Through the concat operation and The convolution layer further obtains the visual difference feature The calculation is as follows:

[0095] =

[0096] Among them, Is The parameter of the convolution. The output of the symmetric boundary visual difference module, that is, the symmetric boundary visual difference feature Is shown in the following formula:

[0097]

[0098] Among them, is the parameter of convolution.

[0099] To fit the feature dimensions of the RSU module and the Transformer block in step S4, is resampled to to match the features of the RSU module.

[0100] Step S6: The multi-modal MRI features , the mirror features and the symmetric boundary visual difference features are all decoded through skip connections into the decoding modules at the corresponding levels, gradually restoring the resolution of the segmented image until finally obtaining a segmentation result with the same size as the original MRI through the convolutional layer.

[0101] The decoded features are obtained by decoding through the th module with bilinear interpolation upsampling operation, as follows:

[0102]

[0103] where the th module.

[0104] The present invention also includes weighting the Dice loss of the loss function by a given adaptive weight parameter. Larger target regions are easier to accurately segment compared to smaller ROIs. Therefore, higher loss weights are given to smaller target regions, while relatively balancing the reduction of loss weights for larger region ROIs. As Figure 1 shown, the green area is the small target region, and the sum of the green area and the red area is the large region. The ARW-dice loss can be obtained by the following formula

[0105]

[0106] where is the adaptive ROI weight parameter with label index , is the label index, which can be calculated by the following formula

[0107]

[0108] where is the ROI annotation volume with label index , is the label index The predicted volume of the ROI.

[0109] The present invention also provides a brain tumor segmentation system based on multimodal MRI. The brain tumor segmentation system based on multimodal MRI can be implemented by executing the process steps of the brain tumor segmentation method based on multimodal MRI. That is, those skilled in the art can understand the brain tumor segmentation method based on multimodal MRI as a preferred embodiment of the brain tumor segmentation system based on multimodal MRI.

[0110] A brain tumor segmentation system based on multimodal MRI provided by the present invention includes:

[0111] Module M1: Obtain multimodal MRI images. The MRI images include T1-weighted contrast-enhanced imaging T1CE and T2-weighted fluid-attenuated inversion recovery sequence T2Flair.

[0112] Module M2: Perform rigid registration on the multimodal MRI images using affine transformation. In Module M2, Z-score normalization is used to eliminate the differences in scanning devices and protocols. The formula is:

[0113]

[0114] Where represents the image after Z-score normalization, represents the image before Z-score normalization, and represent the mean and standard deviation of the image intensity respectively.

[0115] Module M3: Concatenate the registered multimodal MRI images along the channel dimension to form an initial input tensor , and send it into the BTSN encoder of the segmentation network to extract multimodal MRI features. The BTSN encoder is composed of cascaded RSU (Residual U-shaped Subnetwork) modules. The RSU module embeds a U-shaped network, and the encoding-decoding layers are connected by jumps. Finally, the input features and the decoded features are obtained through an addition operation as the output of the module, that is, the multimodal MRI features , where .

[0116] Module M4: Mirror process the MRI images to obtain mirrored MRI images, and extract corresponding mirrored features through a parallel mirrored encoder. After the mirrored encoder performs concatenation and convolution on the mirrored images, it extracts mirrored features through RSU modules, that is, the mirrored features .

[0117] Module M5: Extract the visual difference features of the symmetric boundary of the tumor in the left and right brains by taking the multi-modal MRI features and mirror features as the input of the Symmetric Boundary Visual Difference Module (SVDB). The Symmetric Boundary Visual Difference Module is used to suppress redundant features and correct the features related to image segmentation. The Symmetric Boundary Visual Difference Module includes a Spatial Context Attention Module (SCM) and a Feature Refinement Module (FRM) with a multi-branch dilated convolution structure, which are used to correct and activate the features related to tumor segmentation in multi-scale features. Module M5 includes: Module M5.1: Input multi-modal MRI features and mirror features , calculate the difference between the original MRI and the mirror MRI through element-wise subtraction, and perform convolutional feature extraction on through a convolutional layer. The formula is as follows:

[0118]

[0119] where is the convolutional operation with a kernel, and is the element-wise subtraction operation. Module M5.2: First, pass through a normalization layer (LayerNorm) and a depth convolutional layer with a kernel for channel compression. Then, use the matrix element-wise addition operation to add the compressed features and the normalized input features element-wise. Finally, obtain through a convolutional layer. The calculation formula is as follows:

[0120]

[0121] where LN is the LayerNorm operation, and is the matrix element-wise addition operation. Module M5.3: To capture more extensive spatial context information in response to the scale changes of the input image, send the obtained to the Spatial Context Attention Module (SCM) to obtain . At the same time, obtain through the Feature Refinement Module (FRM) with a multi-branch dilated convolution structure for the expression of the nuclear mass information of small targets. Module M5.4: Concatenate and through the concat operation and further obtain the visual difference feature through a convolutional layer. The calculation is as follows:

[0122] =​

[0123] Among them, is the parameter of convolution. The output of the symmetric boundary visual difference module, that is, the symmetric boundary visual difference feature is shown as follows:

[0124]

[0125] Among them, is the parameter of convolution.

[0126] In module M5.3 after obtaining the input spatial context attention module SCM , the calculation formula is as follows:

[0127] =

[0128] In the formula, is the spatial weighted weight map. The calculation formula is as follows:

[0129]

[0130] Among them, is the matrix multiplication operation. The nuclear mass information expression of small targets is obtained through the feature refinement module FRM with a multi-branch dilated convolution structure , respectively through the convolution layer and the multi-branch combination of the convolution layer to obtain features, and the features are refined by merging different branch features. The calculation formula is as follows:

[0131]

[0132] Among them, respectively represent the refined features of 4 branches.

[0133] In order to fit the feature dimensions of the RSU module and the Transformer block of module M4, is resampled to to make it consistent with the features of the RSU module.

[0134] Module M6: The multi-modal MRI features, mirror features, and symmetric boundary visual difference features are all connected to the corresponding-level decoding module through skip connections for decoding, gradually restoring the resolution of the segmented image until finally the convolution layer of

[0135] The brain tumor segmentation system based on multi-modal MRI of the present invention further includes: weighting the Dice loss function by given adaptive weight parameters. Higher loss weights are given to small target regions, while relatively balancing the reduction of loss weights for large-region ROIs. The ARW-dice loss is obtained by the following formula:

[0136]

[0137] wherein, represents the adaptive ROI weight parameter with label index , is the label index, which is calculated by the following formula:

[0138]

[0139] wherein, represents the ROI annotation volume with label index , represents the ROI prediction volume with label index .

[0140] Those skilled in the art know that in addition to implementing the system and its various devices, modules, and units provided by the present invention in the form of pure computer-readable program code, the method steps can be logically programmed to enable the system and its various devices, modules, and units provided by the present invention to be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers, etc. to achieve the same functions. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a kind of hardware component, and the devices, modules, and units included therein for implementing various functions can also be regarded as the structures within the hardware component; the devices, modules, and units for implementing various functions can also be regarded as either software modules for implementing the method or the structures within the hardware component.

[0141] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above specific implementation manners. Those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Without conflict, the embodiments and features in the embodiments of the present application can be combined with each other arbitrarily.

Claims

1. A brain tumor segmentation method based on multimodal MRI, characterized in that, Including: Step S1: Obtain multi-modal MRI images; Step S2: Perform rigid registration on the multi-modal MRI images using affine transformation; Step S3: Concatenate the registered multi-modal MRI images along the channel dimension to form an initial input tensor , and send it into the BTSN encoder of the segmentation network to extract multi-modal MRI features; The BTSN encoder consists of cascaded RSU modules; The embedded U-shaped network of the RSU module is connected by skipping for the encoding-decoding layer, and finally the input features and the decoded features are obtained through an addition operation The output of the module, namely, the multi-modal MRI features ; The mirror encoder splices and after convolution, and then performs mirror feature extraction through RSU modules, that is, mirror features Step S4: Mirror process the MRI images to obtain mirrored MRI images, and extract corresponding mirrored features through parallel mirrored encoders; Step S5: Use the multi-modal MRI features and the mirrored features as inputs to the Symmetric Boundary Visual Difference Module (SVDB) to extract the symmetric boundary visual difference features of the tumor in the left and right brains; Step S5 includes: Step S5.1: Input multi-modal MRI features and mirrored features , calculate the difference between the original MRI and the mirrored MRI through element-wise subtraction, and perform convolutional feature extraction on through a convolutional layer. The formula is as follows: ​ Among them, is a convolution operation with a core, is an element-wise subtraction operation; Step S5.2: Take and first pass it through a Layer Normalization layer and a depthwise convolutional layer with a convolutional kernel of for channel compression. Subsequently, use matrix element-wise addition operation to add the compressed features and the normalized input features element by element. Finally, obtain through a convolutional layer. The calculation formula is as follows: ​ Among them, LN is the LayerNorm operation, is the matrix element-wise addition operation; Step S5.3: To capture more extensive spatial context information in response to the scale change of the input image of the field, the obtained is fed into the Spatial Context Attention Module (SCM) to obtain , while the expression of the nuclear cluster information of small targets is obtained through the Feature Refinement Module (FRM) with a multi-branch dilated convolution structure to obtain ; Step S5.4: Combine and through a concat operation and further obtain the visual difference feature , calculated as follows: = Among them, is the parameter of convolution, the output of the symmetric boundary visual difference module, that is, the symmetric boundary visual difference feature as shown in the following formula: Among them, is the parameter of convolution; Step S6: The multi-modal MRI features, mirror features, and symmetric boundary visual difference features are all connected through skip connections to the decoding modules at corresponding levels for decoding, gradually restoring the resolution of the segmented image until finally, through the convolutional layer, a segmentation result with the same size as the original MRI is obtained.

2. The method for brain tumor segmentation based on multimodal MRI according to claim 1, wherein The MRI images include T1-weighted contrast-enhanced imaging (T1CE) and T2-weighted fluid-attenuated inversion recovery sequence (T2Flair).

3. The brain tumor segmentation method based on multimodal MRI according to claim 1, characterized in that, In Step S2, Z-score normalization is used to eliminate differences in scanning equipment and protocols. The formula is: Among them, represents the image after Z-score normalization, represents the image before Z-score normalization, and represent the mean and standard deviation of the image intensity respectively.

4. The method for brain tumor segmentation based on multimodal MRI according to claim 1, wherein The Symmetric Boundary Visual Difference Module is used to suppress redundant features and correct features related to image segmentation; The Symmetric Boundary Visual Difference Module includes a Spatial Context Attention Module (SCM) and a Feature Refinement Module (FRM) with a multi-branch dilated convolution structure, which are used to correct and activate features related to tumor segmentation in multi-scale features.

5. The method for brain tumor segmentation based on multimodal MRI according to claim 1, wherein In the step S5.3 after inputting into the spatial context attention module SCM, obtain , and the calculation formula is as follows: = In the formula, is the spatial weighted weight map; The calculation formula is as follows: Among them, is a matrix multiplication operation; The kernel information expression of small targets is obtained through the feature refinement module FRM with a multi-branch dilated convolution structure , respectively through the convolution layer and the multi-branch combination of the convolution layer to obtain features, and refine the features by merging different branch features. The calculation formula is as follows: Among them, respectively represent the refinement features of 4 branches.

6. The method for brain tumor segmentation based on multimodal MRI according to claim 1, wherein To fit the feature dimensions of the RSU module and the Transformer block in step S4, is resampled to so that it matches the features of the RSU module.

7. The brain tumor segmentation method based on multimodal MRI according to claim 1, wherein, Also included: Weight the Dice loss function by giving an adaptive weight parameter; Give a higher loss weight to small target regions, and relatively balance the reduction of the loss weight of large-region ROIs. ARW-diceloss is obtained by the following formula: Among them, represents the adaptive ROI weight parameter with label index , is the label index calculated by the following formula: Among them, represents the ROI annotation volume with a label index of , and represents the ROI prediction volume with a label index of .

8. A brain tumor segmentation system based on multimodal MRI, characterized in that, Including: Module M1: Obtain multi-modal MRI images; Module M2: Perform rigid registration on the multi-modal MRI images using affine transformation; Module M3: Concatenate the registered multi-modal MRI images along the channel dimension to form an initial input tensor , and send it into the BTSN encoder of the segmentation network to extract multi-modal MRI features; Module M4: Mirror process the MRI images to obtain mirrored MRI images, and extract corresponding mirrored features through parallel mirrored encoders; Module M5: Use the multi-modal MRI features and the mirrored features as inputs to the Symmetric Boundary Visual Difference Module (SVDB) to extract the symmetric boundary visual difference features of the tumor in the left and right brains; Module M6: The multi-modal MRI features, mirror features, and symmetric boundary visual difference features are all connected to the decoding modules at the corresponding levels through skip connections for decoding, gradually restoring the resolution of the segmented image until finally obtaining a segmentation result with the same size as the original MRI through the convolutional layer; The BTSN encoder consists of cascaded RSU modules; The embedded U-shaped network of the RSU module is connected by jumps in the encoding-decoding layer, and finally the input features and the decoded features are obtained through an addition operation The output of the module, that is, the multi-modal MRI features ; The mirror encoder stitches and after convolution, and performs mirror feature extraction through RSU modules, that is, mirror features ; Module M5 includes: Module M5.1: Input Multimodal MRI Features and mirrored features , calculate the difference between the original MRI and the mirrored MRI through element-wise subtraction, and perform convolutional feature extraction on through the convolutional layer. The formula is as follows: ​ Among them, is a convolution operation with a core, is an element-wise subtraction operation; Module M5.2: First, pass through a Layer Normalization layer and a depthwise convolutional layer with a convolutional kernel of for channel compression. Subsequently, use the matrix element-wise addition operation to add the compressed features and the normalized input features element by element. Finally, obtain through a convolutional layer. The calculation formula is as follows: ​ Among them, LN is the LayerNorm operation, is the matrix corresponding addition operation; Module M5.3: To capture more extensive spatial context information in response to the scale changes of the input field images, the obtained is fed into the spatial context attention module SCM to obtain , while the nuclear mass information expression of small targets is obtained through the feature refinement module FRM with a multi-branch dilated convolution structure ; Module M5.4: Combine and through the concat operation and further obtain the visual difference feature through the convolutional layer , calculated as follows: = Among them, is the parameter of convolution, the output of the symmetric boundary visual difference module, that is, the symmetric boundary visual difference feature as shown in the following formula: Among them, is the parameter of convolution.

9. The brain tumor segmentation system based on multi-modal MRI according to claim 8, wherein, The MRI images include T1-weighted contrast-enhanced imaging (T1CE) and T2-weighted fluid-attenuated inversion recovery sequence (T2Flair).

10. The brain tumor segmentation system based on multimodal MRI according to claim 8, wherein The Symmetric Boundary Visual Difference Module is used to suppress redundant features and correct features related to image segmentation; The Symmetric Boundary Visual Difference Module includes a Spatial Context Attention Module (SCM) and a Feature Refinement Module (FRM) with a multi-branch dilated convolution structure, which are used to correct and activate features related to tumor segmentation in multi-scale features.

Images