A brain extraction method, system, electronic device and storage medium for enhancing perception of anatomical structure and surface information

Abstract

The present application relates to a kind of brain extraction method, system, electronic equipment, storage medium for enhancing anatomical structure and surface information perception, belong to image processing technical field.The anatomical structure perception module of the present application aims to enhance the understanding of global brain tissue, while the surface information perception module emphasizes the local mode of brain surface, to solve the challenge brought by the low level inter-class similarity of voxel intensity between brain and non-brain tissue;In addition, a novel local-global feature enhancement module is used to enhance the correlation between local surface mode and global brain tissue, and provide a robust local-global representation that helps accurate brain extraction.The present application enhances the anatomical structure and surface information perception of human brain to achieve accurate brain extraction.

Description

Technical Field

[0001] This invention relates to a method, system, electronic device, and storage medium for extracting human brain information that enhances the perception of anatomical structures and surface information, and belongs to the field of image processing technology. Background Technology

[0002] Brain extraction aims to segment the human brain from 3D MRI images to serve clinical needs such as brain volume measurement and analysis of brain morphological changes. This task was initially defined as an image processing problem. Researchers use pattern recognition techniques to distinguish between brain and non-brain structures, with common methods including histogram analysis, edge detection, and thresholding. However, these methods often heavily rely on pre-defined hyperparameters, such as edge constants and thresholds, making them sensitive to voxel intensity variations caused by different medical devices. Traditional machine learning methods attempt to learn discriminative feature representations from data, but these methods still rely on hand-designed features. Deep learning has become a major solution for many medical image segmentation tasks, providing the ability to learn effective deep representations. In particular, U-Net-based convolutional neural networks have shown excellent performance in medical image segmentation and have been used for brain extraction. Existing work has made various improvements to the U-Net architecture, including feature fusion, adversarial mechanisms, and attention mechanisms, to enhance its representation of local features and global semantic information. With the advent of large visual models, researchers have begun to improve large visual models for natural images to adapt them for medical image segmentation tasks.

[0003] Limited by receptive field and pooling operations, CNNs have insufficient ability to capture global feature information. Traditional U-Net and its variants mainly rely on convolutional operations to extract local features, resulting in insufficient fusion of global semantic information and local surface features, especially limiting their ability to segment complex anatomical regions. The human brain has a complex surface structure, and its low-level features depict the detailed features of the brain's edges, which is crucial for improving the accuracy of brain segmentation. Existing methods often do not pay attention to these features and lack a dedicated module for extracting low-level brain surface features during the decoding stage, resulting in insufficient boundary segmentation accuracy, especially unstable segmentation results in low-contrast or inter-class similar regions. In addition, medical image segmentation methods based on large models, such as SAM-Med3D and SegVol, have shown strong robustness and generalization ability. However, their effectiveness in human brain extraction remains unclear.

[0004] Figure 1 This visually demonstrates the challenges posed by inter-class similarity of voxel intensity and the complex anatomy of the head to segmentation tasks; both factors significantly affect the accuracy of brain tissue extraction. Figure 1 In (a), the voxel intensities of certain non-brain tissue regions are similar to those of brain tissue. This inter-class similarity often interferes with existing methods, leading to inaccurate brain segmentation masks. Figure 1As shown in (b), the human head has a complex anatomical structure, especially in the surface area of ​​the brain, making it particularly difficult to accurately distinguish brain tissue from the surrounding non-brain tissue.

[0005] Current brain extraction methods based on U-Net and its variants primarily rely on local convolutional operations, lacking the ability to model the global anatomical structure of the brain over a long period, leading to decreased segmentation accuracy for complex anatomical regions. Furthermore, existing methods fail to effectively extract low-level physical features of the brain surface (such as edges and textures) during the decoding stage, making it difficult to accurately distinguish between different anatomical regions with similar voxel intensities, resulting in blurred or incorrect segmentation boundaries. Meanwhile, while large-scale pre-trained medical image segmentation models developed in recent years possess strong generalization capabilities, they lack dedicated modules for brain extraction tasks, resulting in insufficient sensitivity to details of human brain boundaries and difficulty adapting to the fine segmentation requirements of complex brain anatomy. Summary of the Invention

[0006] To address the problems existing in the current technology, this invention provides a method, system, electronic device, and storage medium for human brain extraction that enhances the perception of anatomical structure and surface information. This invention solves the shortcomings of existing methods in global anatomical structure modeling and human brain boundary detail processing, thereby improving the accuracy and robustness of human brain extraction.

[0007] The technical solution of this invention is: a method for extracting human brain information that enhances the perception of anatomical structures and surface information, the method comprising:

[0008] Step 1: Construct the anatomical structure perception module AA of the encoder. The input 3D head MRI image is sent to the encoder. By setting up multiple anatomical structure perception modules AA, anatomical structure perception feature maps of different resolutions are extracted layer by layer. Then, the anatomical structure perception feature maps are sent to two recursive multilayer perceptron (RMLP) modules to capture long-range dependencies.

[0009] Step 2: Construct the surface information perception module (SA) of the decoder. Two additional RMLP modules are used in the decoder to further enhance the decoding of long-range dependencies. By using skip connections, the low-level features of different layers of the encoder in Step 1 are obtained from the human brain surface features using the 3D Sobel surface feature detector and softly fused with the high-level semantic information upsampled by the decoder to ensure the alignment of surface cues with high-level semantics and enhance the surface information perception capability.

[0010] Step 3: Construct the Local-Global Feature Enhancement (LGFE) module of the decoder. Use channel attention branch and spatial attention branch to extract global anatomical features and local surface features from the soft-fused features obtained in Step 2. Then, fuse the extracted global anatomical features and local surface features to generate enhanced feature representation.

[0011] Step 4: The outputs of each layer of the decoder are upsampled to the same size as the original input. Then, a 3D convolution is used to further fuse these upsampled features and finally generate the predicted brain tissue mask.

[0012] Furthermore, in Step 1, the encoder uses three stacked anatomical structure perception modules AA, all with the same structure. Each anatomical structure perception module AA has a dual-branch structure, including a multilayer perceptron (MLP) branch and a convolutional neural network (CNN) branch.

[0013] The multilayer perceptron (MLP) branch is used to capture global anatomical information of the human brain, enhancing the understanding of the overall anatomical structure of the human brain.

[0014] The convolutional neural network (CNN) branch is responsible for extracting local anatomical information of the human brain.

[0015] The multilayer perceptron (MLP) branch includes a spatial movement module, which moves related input features along three axes: height (H), width (W), and depth (D). After each movement, the linear layer captures long-range correlations and preserves global anatomical information of the human brain.

[0016] The CNN branch contains 3D convolutional blocks, batch normalization layers, max pooling layers, and ReLU activation functions. These modules work together to extract local anatomical information of the human brain.

[0017] Further, Step 1 includes:

[0018] Step 1.1: The convolutional neural network (CNN) branch and the multilayer perceptron (MLP) branch in layer l generate features respectively. and

[0019]

[0020] Where I is a given input head voxel block, I∈R W×H×D×1 Where W, H, and D represent width, height, and depth, and the number of channels is 1; This is the anatomical structure perception feature map output by the (l-1)th layer of the anatomical structure perception module AA. This refers to the local anatomical structure information of the human brain output by the branches of a convolutional neural network (CNN). The global anatomical structure information of the human brain output by the MLP branch of the multilayer perceptron;

[0021] Step 1.2: Use the output from the multilayer perceptron (MLP) branch. As the output of the CNN branch of the convolutional neural network Attention weights are then applied; then residual connections are applied to... This is used to enable the anatomical structure perception module AA to fully extract anatomical structure perception feature maps. Anatomical structure perception feature map The extraction process is represented as follows:

[0022]

[0023] in, This indicates element-wise multiplication.

[0024] Furthermore, in Step 2, the surface information perception module SA is integrated into the ASNet decoder, comprising three sub-modules: surface extraction sub-module SE, semantic perception sub-module SP, and soft fusion sub-module SF;

[0025] The surface extraction submodule SE is used to extract human brain surface features, and the surface extraction submodule uses a 3D Sobel surface feature detector as its core component.

[0026] The semantic awareness submodule SP is used to extract high-level semantic information;

[0027] The soft fusion submodule SF is used to softly fuse human brain surface features with high-level semantic information using cross-attention.

[0028] Furthermore, Step 2 includes:

[0029] Step 2.1: Surface Extraction Submodule SE processes low-level features. Low-level features It is obtained from the (l-1)th encoder through a skip connection; the 3D Sobel surface feature detector uses different kernel weights along three axes, namely the axial, coronal, and sagittal axes, to measure low-level features. Perform convolution; then combine the convolutional features and apply them as attention weights. The calculation process is shown below;

[0030]

[0031] Where ReLU(.) represents ReLU activation function processing, BN(.) represents batch normalization processing, Conv(.) represents convolution processing, and Sobel(.) represents 3D Sobel surface feature detector processing. This represents the surface features of the human brain output by the surface extraction submodule SE;

[0032] Low-level features It is obtained from the (l-1)th encoder layer through a skip connection. This means that the decoder needs to use the output features of the corresponding layer in the encoder. Here, the corresponding features are directly obtained to the decoder through a skip connection. The reason for l-1 is that the decoder needs to perform upsampling. The features after upsampling of the lth layer decoder have the same resolution as the features of the l-1th layer in the encoder.

[0033] Step 2.2: Use the semantic awareness submodule SP to obtain the enhanced feature representation from the local-global feature enhancement module LGFE. High-level semantic information is extracted from the l-th decoder layer;

[0034]

[0035] in, This represents the high-level semantic information output by the semantic awareness submodule SP. This represents the enhanced feature representation obtained by the Local-Global Feature Enhancement Module (LGFE).

[0036] Step 2.3: Use cross-attention and the soft fusion submodule SF to... With F l sp Soft integration:

[0037]

[0038] Where σ(·) represents the Sigmoid function, The surface information sensing module SA from the l-th decoder layer represents the features after soft fusion.

[0039] Furthermore, in Step 3, the Local-Global Feature Enhancement (LGFE) module includes a channel attention branch (CHA), a spatial attention branch (SPA), and a skip connection branch:

[0040] The channel attention branch CHA is used to extract global anatomical features and enhance the global anatomical structure information of the human brain;

[0041] The Spatial Attention Branch (SPA) is used to capture local surface features and refine boundary information;

[0042] The skip connection branch is used for cross-scale feature fusion, enabling local and global information to complement each other.

[0043] Furthermore, Step 3 includes:

[0044] Step 3.1, Regarding the features after soft fusion in Step 2 Channel attention branch CHA uses channel attention capture Implied remote dependency information:

[0045]

[0046] Where CHA(·) represents the channel attention branch, Indicates global anatomical features;

[0047] Step 3.2, Spatial Attention Branch SPA: Utilizing Spatial Attention to Capture The local dependency information contained within:

[0048]

[0049] Where SPA(·) represents the spatial attention branch, F l spa Indicates local surface features;

[0050] Step 3.3: Use element-wise multiplication to fuse global anatomical features. With local surface features F l spa Finally, through residual connections, the fused features are combined with... Adding them together generates an enhanced feature representation.

[0051]

[0052] The present invention also provides a human brain extraction system for enhancing the perception of anatomical structures and surface information, the system comprising: a module for performing the aforementioned human brain extraction method for enhancing the perception of anatomical structures and surface information.

[0053] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for enhancing the perception of anatomical structures and surface information of the human brain.

[0054] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned method for enhancing the perception of anatomical structures and surface information in human brain extraction.

[0055] The CNN in this invention stands for Convolutional Neural Network, a specially designed deep learning model.

[0056] The MLP in this invention stands for Multilayer Perceptron, which is a feedforward artificial neural network model.

[0057] Spatial attention representation of the present invention: a mechanism in deep learning that focuses on a specific region or content in the input data.

[0058] Channel attention representation in this invention: a mechanism in deep learning that emphasizes the importance of different channels in the input feature map.

[0059] This invention proposes an enhanced human brain extraction system, ASNet, which enhances the perception of anatomical structures and surface information. Specifically, it includes an Anatomical Awareness (AA) module, a Surface Awareness (SA) module, and a Local-Global Feature Enhancement (LGFE) module. These modules can capture human brain anatomical structures and surface information and effectively represent local and global features. The AA module has a dual-branch structure: a Multilayer Perceptron (MLP) branch for representing anatomical structures and a Convolutional Neural Network (CNN) branch for extracting low-level features. The MLP branch includes a spatial translation module that moves the input features along three axes. After each translation, linear layers capture long-range correlations and preserve global anatomical information. The convolutional branch includes 3D convolutional blocks, batch normalization layers, max-pooling layers, and ReLU activation functions. The SA module is integrated into the ASNet decoder and consists of three sub-modules: a Surface Extraction (SE) sub-module, a Semantic Awareness (SP) sub-module, and a Soft Fusion (SF) sub-module. The SE sub-module is responsible for extracting human brain surface features. To address this, the present invention introduces a 3DSobel surface detector as a core component; the Local-Global Feature Enhancement (LGFE) module comprises three branches: a channel attention branch (CHA) for global feature extraction, a spatial attention branch (SPA) for local feature extraction, and a skip connection branch for feature fusion. The global feature extraction branch uses channel attention to capture long-range dependencies, while the local feature extraction branch uses spatial attention to capture local dependencies.

[0060] The beneficial effects of this invention are:

[0061] 1. Based on the U-shaped network structure, this invention introduces an anatomical perception (AA) module, a surface perception (SA) module, and a local-global feature enhancement (LGFE) module to improve the accuracy and robustness of human brain extraction. The AA module adopts a dual-branch structure, capturing global anatomical information through the MLP branch to enhance the model's representation of the overall brain structure. At the same time, it uses the CNN branch to extract local anatomical details, retaining detailed anatomical structural information and improving the ability to segment complex regions.

[0062] 2. The SA module of this invention consists of a surface information extraction layer, a semantic perception layer, and a soft fusion layer. The surface extraction layer is used to extract surface features of the human brain; the semantic perception layer is used to extract high-level semantic information; and the soft fusion layer integrates low-level edge information and high-level semantic features through an attention mechanism, thereby enhancing the model's ability to perceive boundary regions. The use of this module makes up for the shortcomings of existing methods in the extraction of low-level physical features of the brain surface. In order to further improve the feature expression ability of the model;

[0063] 3. The LGFE module of this invention adopts a three-branch structure to enhance the network's fusion of local and global features. Specifically, the channel attention branch is responsible for extracting global anatomical features, the spatial attention branch focuses on capturing local surface details, and the skip connection branch is used for cross-scale feature fusion, so that local and global information complement each other, improve the model's ability to accurately depict the boundaries of brain tissue, and thus optimize the segmentation effect.

[0064] 4. The method of the present invention enhances the perception of the anatomical structure and surface information of the human brain to achieve accurate brain extraction. Specifically, the anatomical structure perception module aims to enhance the understanding of global brain organization, while the surface information perception module emphasizes local patterns on the brain surface to address the challenge posed by low-level inter-class similarity of voxel intensity between brain and non-brain tissues. In addition, a novel local-global feature enhancement module is employed to enhance the correlation between local surface patterns and global brain organization, providing robust local-global representations that facilitate accurate brain extraction.

[0065] 5. This invention has been thoroughly tested on four publicly available datasets, covering different imaging modes, imaging device parameters, and patient groups. The experimental results show that the ASNet proposed in this invention outperforms the current state-of-the-art methods in terms of both accuracy and robustness. Compared with existing methods, ASNet enhances the global anatomical structure modeling capability through the AA module, refines the human brain boundary segmentation through the SA module, and balances the fusion of local and global features through the LGFE module, thus achieving more accurate human brain extraction. Attached Figure Description

[0066] Figure 1 This diagram illustrates two challenges faced in the midbrain extraction process of the invention; among them Figure 1 (a) is a statistical diagram showing the intensity distribution of brain tissue and non-brain tissue in a local area of ​​the present invention; Figure 1 (b) is a schematic diagram of brain structures along different axes;

[0067] Figure 2 This is a structural diagram of the ASNet network in this invention;

[0068] Figure 3 This is a structural diagram of the anatomical structure sensing module (AA) in this invention;

[0069] Figure 4 This is a structural diagram of the surface information sensing module (SA) in this invention; wherein, Figure 4 (a) is a structural diagram of the surface extraction submodule SE, semantic perception submodule SP, and soft fusion submodule SF in the surface information perception module SA of the present invention; Figure 4 (b) is a structural diagram of the 3D Sobel surface feature detector in the surface information sensing module of the present invention;

[0070] Figure 5 This is a structural diagram of the Local-Global Feature Enhancement Module (LGFE module) in this invention. Detailed Implementation

[0071] Example 1: As Figures 1-5 As shown, a method for extracting human brain information to enhance the perception of anatomical structures and surface information includes:

[0072] Step 1: Construct the anatomical structure perception module AA of the encoder. The input 3D head MRI image is sent to the encoder. By setting up multiple anatomical structure perception modules AA, anatomical structure perception feature maps of different resolutions are extracted layer by layer. Then, the anatomical structure perception feature maps are sent to two recursive multilayer perceptron (RMLP) modules to capture long-range dependencies.

[0073] Furthermore, in Step 1, the anatomical perception (AA) module is integrated into the shallow coding stage to enhance the network's ability to represent information about the human brain structure, such as... Figure 3 As shown; Figure 2 In the encoder, three stacked anatomical structure perception modules AA are used, all of which have the same structure. Each anatomical structure perception module AA has a dual-branch structure, including a multilayer perceptron (MLP) branch and a convolutional neural network (CNN) branch.

[0074] The multilayer perceptron (MLP) branch is used to capture global anatomical information of the human brain, enhancing the understanding of the overall anatomical structure of the human brain.

[0075] The convolutional neural network (CNN) branch is responsible for extracting local anatomical information of the human brain.

[0076] The multilayer perceptron (MLP) branch includes a spatial movement module, which moves related input features along three axes: height (H), width (W), and depth (D). After each movement, the linear layer captures long-range correlations and preserves global anatomical information of the human brain.

[0077] The CNN branch contains 3D convolutional blocks, batch normalization layers, max pooling layers, and ReLU activation functions. These modules work together to extract local anatomical information of the human brain.

[0078] Further, Step 1 includes:

[0079] Step 1.1: The convolutional neural network (CNN) branch and the multilayer perceptron (MLP) branch in layer l generate features respectively. and

[0080]

[0081] Where I is a given input head voxel block, I∈R W×H×D×1 Where W, H, and D represent width, height, and depth, and the number of channels is 1; This is the anatomical structure perception feature map output by the (l-1)th layer of the anatomical structure perception module AA. This refers to the local anatomical structure information of the human brain output by the branches of a convolutional neural network (CNN). The global anatomical structure information of the human brain output by the MLP branch of the multilayer perceptron;

[0082] Step 1.2: To help the AA module identify anatomical structures and extract low-level features, this invention uses the output from the MLP branch of the multilayer perceptron. As the output of the CNN branch of the convolutional neural network Attention weights are then applied; then residual connections are applied to... This is used to enable the anatomical structure perception module AA to fully extract anatomical structure perception feature maps. Anatomical structure perception feature map The extraction process is represented as follows:

[0083]

[0084] in, This indicates element-wise multiplication.

[0085] Step 2: Construct the surface information perception module (SA) of the decoder. Two additional RMLP modules are used in the decoder to further enhance the decoding of long-range dependencies. By using skip connections, the low-level features of different layers of the encoder in Step 1 are obtained from the human brain surface features using the 3D Sobel surface feature detector and softly fused with the high-level semantic information upsampled by the decoder to ensure the alignment of surface cues with high-level semantics and enhance the surface information perception capability.

[0086] Furthermore, in Step 2, to enable ASNet to effectively represent human brain surface information, this invention proposes a surface information perception (SA) module. The surface information perception module SA is integrated into the ASNet decoder, such as... Figure 2As shown, integrating the SA module into the ASNet decoder serves two main purposes: (i) to enhance the representation of surface cues, and (ii) to ensure consistency between surface cues and high-level semantics. The structure of the SA module is as follows: Figure 4 Detailed description is provided. For example... Figure 4 As shown in (a), the surface information perception module SA includes three sub-modules: surface extraction sub-module SE, semantic perception sub-module SP, and soft fusion sub-module SF;

[0087] The surface extraction submodule SE is used to extract human brain surface features, and the surface extraction submodule uses a 3D Sobel surface feature detector as its core component.

[0088] The semantic awareness submodule SP is used to extract high-level semantic information;

[0089] The soft fusion submodule SF is used to softly fuse human brain surface features with high-level semantic information using cross-attention.

[0090] Furthermore, Step 2 includes:

[0091] Step 2.1: Since surface structure is a low-level physical feature, the surface extraction submodule SE processes low-level features. Low-level features It is obtained from the (l-1)th encoder through a skip connection; Figure 4 (b) shows the structure of the 3D Sobel surface feature detector on the left. Figure 4 (b) As shown on the right, the 3D Sobel surface feature detector uses different kernel weights along three axes—axial, coronal, and sagittal—to measure low-level features. Perform convolution; then combine the convolutional features and apply them as attention weights. The calculation process to enhance its ability to represent brain surface features is as follows;

[0092]

[0093] Where ReLU(.) represents ReLU activation function processing, BN(.) represents batch normalization processing, Conv(.) represents convolution processing, and Sobel(.) represents 3D Sobel surface feature detector processing. This represents the surface features of the human brain output by the surface extraction submodule SE;

[0094] Low-level features It is obtained from the (l-1)th encoder layer through a skip connection. This means that the decoder needs to use the output features of the corresponding layer in the encoder. Here, the corresponding features are directly obtained to the decoder through a skip connection. The reason for l-1 is that the decoder needs to perform upsampling. The features after upsampling of the lth layer decoder have the same resolution as the features of the l-1th layer in the encoder.

[0095] Step 2.2: Use the semantic awareness submodule SP to obtain the enhanced feature representation from the local-global feature enhancement module LGFE. High-level semantic information is extracted from the l-th decoder layer;

[0096]

[0097] in, This represents the high-level semantic information output by the semantic awareness submodule SP. This represents the enhanced feature representation obtained by the Local-Global Feature Enhancement Module (LGFE).

[0098] Step 2.3, due to From low-level features It is exported, so it may contain noise. To reduce this noise and incorporate important high-level semantic information, this invention uses cross-attention and integrates it through the soft fusion submodule SF. With F l sp Soft integration:

[0099]

[0100] Where σ(·) represents the Sigmoid function, The surface information sensing module SA from the l-th decoder layer represents the features after soft fusion.

[0101] Step 3: Construct the Local-Global Feature Enhancement (LGFE) module of the decoder. Use channel attention branch and spatial attention branch to extract global anatomical features and local surface features from the soft-fused features obtained in Step 2. Then, fuse the extracted global anatomical features and local surface features to generate enhanced feature representation.

[0102] Furthermore, in the decoding stage, the thorough fusion of local surface features and global anatomical features is crucial for generating high-precision human brain segmentation masks. To this end, this invention designs a Local-Global Feature Enhancement (LGFE) module and integrates it into each decoder layer;

[0103] In Step 3, as follows Figure 5As shown, the Local-Global Feature Enhancement (LGFE) module includes a channel attention branch (CHA), a spatial attention branch (SPA), and a skip connection branch:

[0104] The channel attention branch CHA is used to extract global anatomical features and enhance the global anatomical structure information of the human brain;

[0105] The Spatial Attention Branch (SPA) is used to capture local surface features and refine boundary information;

[0106] The skip connection branch is used for cross-scale feature fusion, enabling local and global information to complement each other.

[0107] Furthermore, Step 3 includes:

[0108] Step 3.1, Regarding the features after soft fusion in Step 2 Channel attention branch CHA uses channel attention capture Implied remote dependency information:

[0109]

[0110] Where CHA(·) represents the channel attention branch, Indicates global anatomical features;

[0111] Step 3.2, Spatial Attention Branch SPA: Utilizing Spatial Attention to Capture The local dependency information contained within:

[0112]

[0113] Where SPA(·) represents the spatial attention branch, F l spa Indicates local surface features;

[0114] Step 3.3: Use element-wise multiplication to fuse global anatomical features. With local surface features F l spa Finally, through residual connections, the fused features are combined with... Adding them together generates an enhanced feature representation.

[0115]

[0116] Step 4: The outputs of each layer of the decoder are upsampled to the same size as the original input. Then, a 3D convolution is used to further fuse these upsampled features and finally generate the predicted brain tissue mask.

[0117] The present invention also provides a human brain extraction system for enhancing the perception of anatomical structures and surface information, the system comprising: a module for performing the aforementioned human brain extraction method for enhancing the perception of anatomical structures and surface information.

[0118] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for enhancing the perception of anatomical structures and surface information of the human brain.

[0119] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned method for enhancing the perception of anatomical structures and surface information in human brain extraction.

[0120] Figure 2 The proposed ASNet network structure is illustrated in this paper. The ASNet network includes an anatomical structure awareness (AA) module, a surface information awareness (SA) module, and a local-global feature enhancement (LGFE) module. This invention enhances the ability to perceive global anatomical structures and local surface patterns, thereby improving segmentation accuracy and robustness.

[0121] In the encoder section, the ASNet network employs three stacked Anatomical Awareness (AA) modules, the structure of which is as follows: Figure 3 As shown in the diagram, the Multilayer Perceptron (MLP) branch is responsible for modeling long-range dependencies, capturing global anatomical information, and enhancing the model's understanding of the overall brain structure. The convolutional branch is responsible for extracting local anatomical details. Through the combination of MLP and convolutional branches, the AA module can effectively improve the model's ability to perceive the brain's anatomical structure. In the subsequent part of the encoder, two recursive MLP (RMLP) modules are introduced to further enhance the network's ability to model long-range correlations, enabling the model to fully integrate local and global information at different scales, providing more discriminative feature representations for the subsequent decoding stage.

[0122] In the decoder section, ASNet employs a five-layer decoding structure symmetrical to the encoder, progressively recovering features extracted during the encoding process and enhancing boundary perception capabilities. To address the issues of blurred brain surface boundaries and insufficient extraction of local anatomical patterns, the decoder introduces a surface information perception module (SA). For example... Figure 4 As shown, this module uses a 3D Sobe filter to enhance the perception of brain surface patterns, enabling it to more accurately distinguish different anatomical tissues with similar voxel intensities, thereby optimizing boundary segmentation accuracy.

[0123] like Figure 5As shown, the Local-Global Feature Enhancement (LGFE) module adopts a three-branch structure: the Channel Attention Branch (CHA) is responsible for extracting global features and enhancing anatomical structure information; the Spatial Attention Branch (SPA) is responsible for capturing local surface features and refining boundary information; and the Skip Connection Branch is responsible for fusing features at different scales.

[0124] Finally, the outputs of each layer of the decoder are upsampled to the same size as the original input (W, H, D, 4), where 4 represents the color channels. These features are then further fused using a 3D convolution (meaning the features from each layer of the decoder are upsampled, concatenated along the channel dimension, and then fused using a 3D convolution), ultimately generating the predicted brain tissue mask. Two channels correspond to the predicted probabilities of brain tissue and non-brain tissue, respectively. These two channels mean that the network output has two possible prediction values ​​for each pixel: the first channel represents the probability of background prediction (i.e., non-brain tissue prediction), and the second channel represents the probability of foreground prediction (i.e., brain tissue prediction). The probabilities of non-brain tissue and brain tissue correspond to these two channels. By comparing the probability values ​​of these two channels, it is determined whether the pixel belongs to brain tissue or non-brain tissue, thus generating a brain tissue segmentation map. Typically, for a binary classification problem, the network output corresponds to two channels; for a 3-class classification problem, the network output corresponds to three channels.

[0125] exist Figure 2 The flowchart shown illustrates the overall process of the model during the inference phase. Due to the high resolution of 3D brain tissue, it's impossible to directly feed a large-resolution image into the network for training. Therefore, during the training phase, a cropped block is used for training. However, during the inference phase, the cropped pattern cannot be used for prediction. Instead, a sliding window approach is employed, processing the large-resolution image into multiple fixed-size voxel blocks, which are then fed into the network for prediction. After all voxel blocks have been predicted, they are stitched together to obtain a complete large-resolution image. During model training, this stitching is not required; the resulting image is a cropped brain tissue mask. During the testing phase, in order to obtain a complete brain mask without cropping, it is necessary to use a sliding window to cut a complete brain into multiple voxel blocks of size (W, H, D), and then stitch these brain masks together to obtain a complete brain mask.

[0126] Specifically, given an input head voxel block I∈R W×H×D×1Where W, H, and D represent width, height, and depth, and the number of channels is 1. During inference, the entire head 3D image is uniformly divided into multiple 128×128×128 image blocks, with an overlap stride of 48 between these blocks. In the encoding stage, the output feature dimensions of each layer of the encoder are as follows: and Following the standard U-Net design, the spatial size is halved to reduce computational cost, while the number of channels is doubled (e.g., from 16 to 32 after the first layer) to ensure that the model can extract richer hierarchical information.

[0127] During the decoding phase, the output size of each layer of the decoder is matched to the corresponding encoder layer. The outputs of all layers are upsampled to (W, H, D, 4). Finally, 3D convolutional layers are used to merge all the upsampled results to generate a human brain segmentation mask.

[0128] By employing three modules—Anatomy Awareness (AA), Surface Information Awareness (SA), and Local-Global Feature Enhancement (LGFE)—ASNet, the method of this invention emphasizes the representation of anatomical structure information and human brain surface information, thereby generating a more accurate human brain segmentation mask. To verify the effectiveness of this method, experiments were conducted on four challenging datasets, comparing ASNet with other mainstream brain extraction methods. These four datasets are NFGS, SynthStrip, OASIS-1, and CC-359. MSD, RMSD, HD95, and HD99 were used as metrics. MSD represents the average surface distance, RMSD represents the root mean square surface distance, HD95 represents the 95% Hausdorff distance, and HD99 represents the 99% Hausdorff distance. The experimental results are shown in Tables 1, 2, 3, and 4. ASNet outperforms other baseline models on all four datasets. The results demonstrate that, across all datasets, the accuracy of the human brain surface contour and overall segmentation mask generated by ASNet surpasses existing baseline models.

[0129] Table 1 shows the quantitative comparison results on the NFBS dataset.

[0130]

[0131] As shown in Table 1, the method of this invention has the best performance in terms of MSD, RMSD, HD95 and HD99 on the NFBS dataset; the baseline model MedSAM has the second best performance in terms of MSD, RMSD, HD95 and HD99 on the NFBS dataset.

[0132] Table 2 presents the quantitative comparison results on the SynthStrip dataset.

[0133]

[0134]

[0135] As shown in Table 2, the present invention has the best performance in terms of MSD, RMSD, HD95 and HD99 metrics on the SynthStrip dataset; the baseline model UNETR++ has the second best performance in terms of MSD metrics on the SynthStrip dataset; and the baseline model MA-SAM has the second best performance in terms of RMSD, HD95 and HD99 metrics on the SynthStrip dataset.

[0136] Table 3 shows the quantitative comparison results on the OASIS-1 dataset.

[0137]

[0138] As shown in Table 3, the ASNet method of this invention has the best performance in terms of MSD, RMSD, HD95, and HD99 metrics on the OASIS-1 dataset; the baseline model SAM-Med3D has the second best performance in terms of MSD and RMSD metrics on the OASIS-1 dataset; the baseline model MSMHA-CNN has the second best performance in terms of HD95 metrics on the OASIS-1 dataset; and the baseline model 3D UNeXt has the second best performance in terms of HD99 metrics on the OASIS-1 dataset.

[0139] Table 4 presents the quantitative comparison results on the CC-359 dataset.

[0140]

[0141]

[0142] As shown in Table 4, the ASNet method of this invention has the best performance in terms of MSD, RMSD, HD95 and HD99 metrics on the CC-359 dataset; the baseline model MedSAM has the second best performance in terms of MSD, RMSD and HD99 metrics on the CC-359 dataset; and the baseline model 3D UNeXt has the second best performance in terms of HD95 metrics on the CC-359 dataset.

[0143] Table 5 compares ASNet with other baseline models in terms of computational complexity.

[0144]

[0145] Table 5 compares ASNet with other baseline models in terms of computational complexity. TM, IM, and IT represent training memory, inference memory, and inference time, respectively. The method of this invention shows the best performance compared with other baseline models in terms of training memory, inference memory, and inference time.

[0146] Table 5 compares the performance of the proposed method ASNet with baseline models in terms of computational resource consumption and average inference speed. The results show that the proposed method ASNet significantly outperforms existing methods in terms of computational efficiency. Specifically, the proposed method ASNet has the lowest number of parameters, only 7.73M, and achieves the lowest computational complexity, only 28.02 GFLOPs. This result is significantly better than other models, such as SegVol (716.66 GFLOPs) and MedSAM (371.99 GFLOPs). Furthermore, the proposed method ASNet requires less training memory, only 2,404M, and inference memory, only 1,106M, which is far lower than other base models, such as MedSAM (10,230M and 3,574M, respectively) and SegVol (11,146M and 3,700M, respectively). It is worth noting that the ASNet method of this invention has an inference time of only 0.21 seconds for each 3D head image with a size of 128×128×128, which is better than SAM-Med3D (0.28 seconds) and far exceeds MedSAM (10.17 seconds) and SegVol (17.92 seconds).

[0147] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A method for extracting human brain information that enhances the perception of anatomical structures and surface information, characterized in that: The method includes: Step 1: Construct the anatomical structure perception module AA of the encoder. The input 3D head MRI image is sent to the encoder. By setting up multiple anatomical structure perception modules AA, anatomical structure perception feature maps of different resolutions are extracted layer by layer. Then, the anatomical structure perception feature maps are sent to two recursive multilayer perceptron (RMLP) modules to capture long-range dependencies. Step 2: Construct the surface information perception module (SA) of the decoder. Two additional RMLP modules are used in the decoder to further enhance the decoding of long-range dependencies. By skipping connections, the low-level features of different layers of the encoder in Step 1 are obtained from the human brain surface features using the 3DSobel surface feature detector and softly fused with the high-level semantic information upsampled by the decoder to ensure the alignment of surface cues with high-level semantics and enhance the surface information perception capability. Step 3: Construct the Local-Global Feature Enhancement (LGFE) module of the decoder. Use channel attention branch and spatial attention branch to extract global anatomical features and local surface features from the soft-fused features obtained in Step 2. Then, fuse the extracted global anatomical features and local surface features to generate enhanced feature representation. Step 4: The outputs of each layer of the decoder are upsampled to the same size as the original input. Then, a 3D convolution is used to further fuse these upsampled features and finally generate the predicted brain tissue mask. In Step 1, the encoder uses three stacked anatomical structure perception modules AA, all with the same structure. Each anatomical structure perception module AA has a dual-branch structure, including a multilayer perceptron (MLP) branch and a convolutional neural network (CNN) branch. The multilayer perceptron (MLP) branch is used to capture global anatomical information of the human brain, enhancing the understanding of the overall anatomical structure of the human brain. The convolutional neural network (CNN) branch is responsible for extracting local anatomical information of the human brain. The multilayer perceptron (MLP) branch includes a spatial motion module that moves along three axes, namely height. Axis, width Axis and Depth The input features are related to axis movement; after each shift, the linear layer captures long-range correlations and preserves global anatomical information of the human brain. The convolutional neural network (CNN) branch includes 3D convolutional blocks, batch normalization layers, max pooling layers, and... Activation functions, through the collaborative work of these modules, extract local anatomical information of the human brain; The output from the multilayer perceptron (MLP) branch is used as the attention weights for the output of the convolutional neural network (CNN) branch; then residual connections are applied to the output of the CNN branch to enable the anatomical structure perception module (AA) to fully extract the anatomical structure perception feature map. In Step 2, the surface information perception module SA is integrated into the ASNet decoder, which includes three sub-modules: surface extraction sub-module SE, semantic perception sub-module SP, and soft fusion sub-module SF. The surface extraction submodule SE is used to extract human brain surface features, and the surface extraction submodule uses a 3D Sobel surface feature detector as its core component. The semantic awareness submodule SP is used to extract high-level semantic information; The soft fusion submodule SF is used to softly fuse human brain surface features with high-level semantic information using cross-attention; In Step 3, the Local-Global Feature Enhancement (LGFE) module includes a Channel Attention Branch (CHA), a Spatial Attention Branch (SPA), and a Skip Connection Branch: The channel attention branch CHA is used to extract global anatomical features and enhance the global anatomical structure information of the human brain; The Spatial Attention Branch (SPA) is used to capture local surface features and refine boundary information; The skip connection branch is used for cross-scale feature fusion, enabling local and global information to complement each other.

2. The method for extracting human brain information with enhanced anatomical structure and surface information perception according to claim 1, characterized in that: Step 1 includes: Step1.1, No. The convolutional neural network (CNN) branch and the multilayer perceptron (MLP) branch generate features respectively. and : ； ； Where I is a given input head voxel block, ,in, , and Indicates width, height, and depth; the number of channels is 1. For the anatomical structure perception module AA Anatomical structure perception feature map output by the layer. This refers to the local anatomical structure information of the human brain output by the branches of a convolutional neural network (CNN). The global anatomical structure information of the human brain output by the MLP branch of the multilayer perceptron; Step 1.2: Use the output from the multilayer perceptron (MLP) branch. As the output of the CNN branch of the convolutional neural network Attention weights are then applied; then residual connections are applied to... This is used to enable the anatomical structure perception module AA to fully extract anatomical structure perception feature maps. Anatomical structure perception feature map The extraction process is represented as follows: ； in, This indicates element-wise multiplication.

3. The method for extracting human brain information with enhanced anatomical structure and surface information perception according to claim 1, characterized in that: Step 2 includes: Step 2.1: Surface Extraction Submodule SE processes low-level features. Low-level features From the ( The encoders are obtained through skip connections; the 3D Sobel surface feature detector uses different kernel weights along three axes—axial, coronal, and sagittal—to measure low-level features. Perform convolution; then combine the convolutional features and apply them as attention weights. The calculation process is shown below; ； in, express Activation function processing, This indicates batch normalization processing. This indicates convolution processing. This indicates 3D Sobel surface feature detector processing. This represents the surface features of the human brain output by the surface extraction submodule SE; Low-level features From the first Each encoder layer is obtained through skip connections. This means that the decoder needs the output features of the corresponding layer in the encoder, and here the corresponding features are directly obtained and fed to the decoder through skip connections. The reason for this is... This is because the decoder needs to perform upsampling, the first Features upsampled by the layer decoder and those in the encoder Consistent layer feature resolution; Step 2.2: Use the semantic awareness submodule SP to obtain the enhanced feature representation from the local-global feature enhancement module LGFE. Extracting high-level semantic information, which is derived from the first... Each decoder layer is generated; ； in, This represents the high-level semantic information output by the semantic awareness submodule SP. This represents the enhanced feature representation obtained by the Local-Global Feature Enhancement Module (LGFE). Step 2.3: Use cross-attention and the soft fusion submodule SF to... and Soft integration: ； ； in, express function, From the The surface information sensing module SA of each decoder layer represents the features after soft fusion.

4. The method for extracting human brain information to enhance the perception of anatomical structure and surface information according to claim 1, characterized in that: Step 3 includes: Step 3.1, Regarding the features after soft fusion in Step 2 The channel attention branch CHA uses channel attention capture. Implied remote dependency information: ； in, Indicates the channel attention branch, Indicates global anatomical features; Step 3.2, Spatial Attention Branch SPA: Utilizing Spatial Attention to Capture The local dependency information contained within: ； in, Represents the spatial attention branch. Indicates local surface features; Step 3.3: Use element-wise multiplication to fuse global anatomical features. With local surface features Finally, through residual connections, the fused features are combined with... Adding them together generates an enhanced feature representation. : 。 5. A human brain retrieval system that enhances the perception of anatomical structures and surface information, characterized in that, The system includes a module for performing a human brain extraction method for enhancing the perception of anatomical structures and surface information as described in any one of claims 1 to 4.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements a human brain extraction method for enhancing the perception of anatomical structures and surface information as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements a method for enhancing the perception of anatomical structures and surface information of the human brain as described in any one of claims 1 to 4.

Images