A lightweight fast skull stripping network 3D unext module

Abstract

The application discloses a lightweight and rapid skull stripping network 3D UNeXt module, which takes U-Net as a basic architecture and comprises an encoder, a network bottleneck and a decoder, the encoder uses a 3D convolution module to extract local features of medical image volume data, the network bottleneck uses a 3D MLP module to learn long-range dependencies between features in cooperation with an axial shift mechanism, and the decoder uses a multi-scale feature fusion module and a skip connection to fuse shallow and deep features of different scales. The application has the beneficial effects that a new lightweight neural network 3D UneXt module for rapid brain skull stripping is designed and implemented, and the evaluation indexes such as Dice, MSD, HD95 and HD99 of the network on public and private datasets are all better than or close to current advanced networks, and the parameter quantity, FLOPs and average CPU inference time of the network are also far less than those of current advanced networks.

Description

Technical Field

[0001] This invention relates to the field of medical technology, specifically to a lightweight, rapid craniotomy network 3D UNeXt module. Background Technology

[0002] Craniotomy, also known as brain extraction, involves removing non-brain tissues such as the skull from three-dimensional medical images of the head to obtain the complete human brain. Craniotomy is fundamental to the clinical analysis and diagnosis of neuroimaging, providing diagnostic and decision-making support for subsequent brain image registration, brain region segmentation, brain volume measurement, and neurosurgical planning. The precision and speed of craniotomy are crucial factors in achieving high-quality clinical analysis and diagnosis of neuroimaging.

[0003] Currently, the clinical gold standard for craniectomy is obtained semi-manually by radiologists: they first use software such as FSL to automatically obtain initial brain images, and then manually refine the results. Typically, an experienced radiologist takes an average of 30-40 minutes to complete a craniectomy. Clearly, this process suffers from over-reliance on domain experts and is time-consuming and labor-intensive.

[0004] With the widespread research and application of deep learning in medical image analysis, researchers have begun using convolutional neural networks (CNNs) for automated skull dissection. Currently, mainstream CNNs used for skull dissection suffer from the following problems:

[0005] (1) Lack of high-level semantic information. To obtain a larger receptive field, CNNs often require deep network structures. However, as the network layers deepen, the spacing between network modules increases, which leads to the blockage of connections between them. This makes it difficult for deep features to capture the details of shallow features and to establish long-range dependencies between deep features, resulting in a lack of high-level semantic information learned by the deep layers of the network.

[0006] (2) Poor global feature representation capability. Most current CNNs used for skull dissection employ multiple convolutional kernels, thus possessing strong local feature learning capabilities. However, limited by the size of the convolutional kernels and the finite receptive field, convolutional operations struggle to capture high-quality global features. This results in existing networks lacking high-quality global feature guidance during the inference phase, thereby affecting their prediction accuracy.

[0007] (3) Complex structure and slow inference speed. To achieve higher skull dissection accuracy, current networks mostly adopt deep convolutional architectures. Furthermore, to obtain long-range dependencies between features, these networks integrate various attention mechanisms. The multiple layers of convolution and attention mechanisms result in complex network structures and a large number of parameters. During the inference phase, these networks typically need to run on GPUs, consuming significant computational resources and inference time. Summary of the Invention

[0008] The technical problem to be solved by this invention is to overcome the existing defects and provide a lightweight and fast craniotomy network 3D UNeXt module. On public and private datasets, its evaluation metrics such as Dice, MSD, HD95 and HD99 are better than or close to those of current state-of-the-art networks. Its parameter count, FLOPs and average CPU inference time are also much smaller than those of current state-of-the-art networks, which can effectively solve the problems in the background technology.

[0009] To achieve the above objectives, the present invention provides the following technical solution: a lightweight, fast craniotomy network module, 3DUNeXt, comprising an encoder, a network bottleneck, and a decoder. The encoder uses a 3D convolutional module to extract local features from medical image volume data; the network bottleneck uses a 3D MLP module in conjunction with an axial shift mechanism to learn the long-range dependencies between features; the decoder uses a multi-scale feature fusion module and skip connections to fuse shallow and deep features of different scales.

[0010] As a preferred embodiment of the present invention, the encoder comprises five modules. From top to bottom, the first three modules each consist of a 3D convolutional layer, a batch normalization layer (BN), and a max pooling layer. The last two modules each consist of a 3D convolutional layer, a 3D MLP layer, and a layer normalization layer (LN). The input 3D medical image volume data is downsampled through the first three modules to obtain local feature representations. These features are then passed through the last two modules to extract their long-range dependencies. The introduction of the max pooling layer improves the model's generalization ability and reduces the size of the feature map, thereby reducing computational overhead. With each module of the encoder, the feature map size is halved and the number of channels is doubled.

[0011] As a preferred embodiment of the present invention, the decoder also includes five modules, which correspond one-to-one with the five modules of the encoder. From bottom to top, the first two modules each consist of a 3D convolutional layer, a BN layer, a trilinear interpolation layer, a 3D MLP module, and an LN layer. The latter three modules each consist of a 3D convolutional layer, a BN layer, and a trilinear interpolation layer.

[0012] As a preferred embodiment of the present invention, in order to enhance the generalization performance of the decoder, the above modules are supplemented with rectified linear units (ReLU). For each module of the decoder, the size of the feature map doubles and the number of channels is halved.

[0013] In a preferred embodiment of the present invention, the two modules at the bottom of the encoder and the two modules at the bottom of the decoder together constitute the network bottleneck. As mentioned above, each module of this network bottleneck consists of a 3D convolutional layer, a 3D MLP layer, and an LN layer. Compared to the original input of the network, the input feature map size of the network bottleneck is significantly reduced and contains more global information and high-level semantic information. The 3D convolutional layer is used to smooth noise in the input feature map and extract global features, while the 3D MLP layer further mines the long-range dependencies between features through linear shift operations and fully connected layers.

[0014] As a preferred technical solution of the present invention, to efficiently fuse shallow and deep features of different scales without increasing network complexity, 3D UNeXt introduces a multi-scale feature fusion module between the encoder and decoder. For the outputs of the first four modules of the encoder, this feature fusion module restores the outputs to their original size using a 3×3×3 3D convolution and trilinear interpolation; then, these restored feature maps are concatenated with the output feature maps of the decoder; finally, the concatenated feature maps are subjected to a 1×1×1 3D convolution to achieve multi-scale feature fusion.

[0015] As a preferred embodiment of the present invention, the 3D MLP module comprises five parts: an embedding layer, a depth convolutional layer, a shifting layer, a linear layer, and a Gaussian error linear unit layer.

[0016] As a preferred embodiment of the present invention, the embedding layer uses a 3D convolution with a 3×3×3 kernel to extract local features and perform normalization operations. For each channel of these local features, the DWConv layer uses an independent 2D convolution to extract deep semantic information for a single channel. Compared with conventional 3D convolution, this method significantly reduces the number of parameters and computational overhead. The shifting layer shifts the feature map along the height, width, and depth axes respectively. The GELU layer introduces a non-linear activation mechanism into the 3D MLP module, improving the model's generalization ability.

[0017] As a preferred technical solution of the present invention, the displacement operation in the height axial direction is as follows: Figure 2As shown: First, the feature map is split into 5 partitions; then, each partition is moved by -2, -1, 0, 1, and 2 positions along the height axis, respectively; finally, the moved partitions are concatenated to obtain a new feature map. Next, this feature map is passed through a linear layer to obtain the long-range dependencies between features at different spatial locations.

[0018] As a preferred embodiment of the present invention, an LN layer is introduced at the end of the 3D MLP module to normalize the features. Residual connections add the original input features as residuals, avoiding gradient vanishing caused by increased network depth.

[0019] Compared with the prior art, the beneficial effects of the present invention are: to design and implement a novel lightweight neural network 3D UneXt module for rapid craniotomy, which outperforms or approaches current state-of-the-art networks in terms of evaluation metrics such as Dice, MSD, HD95 and HD99 on both public and private datasets, and whose parameter count, FLOPs and average CPU inference time are also much smaller than current state-of-the-art networks. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the structure of the present invention;

[0021] Figure 2 This is a schematic diagram showing the displacement layer of the present invention shifted along the height axis;

[0022] Figure 3 A schematic diagram of skull dissection results on different networks of IBSR;

[0023] Figure 4 A schematic diagram of skull dissection results on different networks of NFBS;

[0024] Figure 5 This is a schematic diagram of the skull dissection results using different networks on HTU-BrainMask. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] With the continuous advancement of computer vision and deep learning, deep neural networks have been widely applied to medical image analysis fields such as classification, segmentation, and assisted diagnosis (Litjens et al., 2017). Among them, CNNs are a type of neural network (Shin et al., 2016), possessing the advantage of learning local information and being able to learn discriminative visual features from a large number of training samples. Furthermore, CNNs require only sufficient training samples during training, minimizing human intervention. These advantages made CNNs the preferred network architecture for the early application of deep learning in medical image analysis tasks such as skull dissection (Dou et al., 2016; Kleesiek et al., 2016).

[0027] However, traditional CNNs have complex network structures and a large number of parameters, often requiring significant GPU computing resources. Furthermore, as network depth increases, CNNs suffer from information loss at different levels. U-Net (Ronneberger et al., 2015), with its encoder-decoder structure, is a landmark network architecture in medical image segmentation. U-Net downsamples and upsamples medical images through convolutional modules in the encoder and decoder, respectively, and uses skip connections between shallow and deep features to combat gradient vanishing and achieve the fusion of local and global features. U-Net and its variants have become the mainstream architecture for medical image segmentation and are widely used in various segmentation tasks.

[0028] In recent years, various network variants based on U-Net have been proposed. Ibtehaz et al. (2020) designed MultiResUNet, which combines the MultiRes module with residual connections to U-Net, demonstrating good segmentation performance. Wang et al. (2020) proposed Non-local U-Net, which combines attention mechanisms, encoders, and decoders to obtain a larger receptive field and solve the problem of global information loss caused by simple convolution operations.

[0029] When processing 3D medical images directly using traditional U-Net, the data can only be organized as 2D slices. This discards the spatial positional information between different views of the 3D volumetric data, leading to a decrease in segmentation accuracy. 3D U-Net is an extension of U-Net for 3D volumetric data. (e.g., 2016), it replaces the original 2D convolution and 2D deconvolution with 3D convolution and 3D deconvolution respectively. 3D U-Net learns local features of 3D volumetric data by using smaller 3D convolution kernels, and achieves global feature extraction by deepening the convolutional layers. However, the locality of convolution operations limits 3D U-Net's ability to learn long-range dependencies between features, despite its use of skip connections between the encoder and decoder.

[0030] Wang et al. (2021) proposed DA 3D-UNet based on 3D U-Net, adding a dual attention module combining spatial and channel attention in the last two layers of the decoding stage to obtain a wider range of global features and improve segmentation accuracy. Jin et al. (2020) introduced a residual attention module into 3D U-Net (Wang et al., 2017), proposing RA-UNet, a hybrid residual and attention mechanism. This network uses stacked residual modules, increasing network depth and refining attention to image details while avoiding gradient vanishing.

[0031] To overcome the shortcoming of convolutional operations in losing long-range dependencies between features, Transformer modules with integrated attention mechanisms (Dosovitskiy et al., 2020) were introduced into the U-Net architecture to acquire global semantic information. Chen et al. (2021) proposed TransUNet, which integrates Transformer and U-Net. This network uses Transformer modules to encode input information to preserve details of image segmentation results. Cao et al. (2021) proposed SwinUnet, which uses a U-Net architecture to organize Transformer modules to extract long-range dependencies between features.

[0032] The aforementioned work focuses on improving model accuracy, neglecting factors such as the number of network parameters, computational complexity, and inference time. However, in clinical applications, lightweight and efficient craniectomy models can save computational resources and improve work efficiency, thus possessing greater clinical application potential and value (Zhang et al., 2020; He et al., 2021).

[0033] Recently, Machine Learning Platforms (MLPs) (Pinkus et al., 1999) have begun to attract attention in the field of computer vision. MLPs learn long-range dependencies between features at different spatial locations through fully connected and non-linear activation functions. Simultaneously, MLPs utilize shift operations to shift local features at different axial positions, thereby achieving the fusion of feature information from different spatial locations in a linear manner. These characteristics of MLPs enable them to achieve feature representation performance comparable to Transformers, while consuming fewer computational resources.

[0034] Lian et al. (2021) designed an axial shift strategy and proposed AS-MLP to fuse features from different spatial locations, thereby obtaining more long-range dependencies between features. Valanarasu et al. (2022) proposed UNeXt, the first lightweight 2D medical image segmentation model based on CNN and MLP. UNeXt uses 2D convolutions in the shallow layers of the network to learn local features, and uses MLP modules in the deeper layers to learn long-range dependencies and global features. The MLP module effectively reduces the number of parameters in UNeXt, overcoming the shortcomings of existing CNN and Transformer-based 2D networks, such as excessive parameters and high computational complexity.

[0035] Please see Figure 1-5 This invention provides a technical solution: a lightweight, fast craniotomy network 3D UNeXt module, based on the U-Net architecture, comprising an encoder, a network bottleneck, and a decoder. The encoder uses a 3D convolutional module to extract local features from medical image volume data; the network bottleneck uses a 3D MLP module in conjunction with an axial shift mechanism to learn long-range dependencies between features; the decoder uses a multi-scale feature fusion module and skip connections to fuse shallow and deep features of different scales.

[0036] from Figure 1 (b) As can be seen, the encoder consists of five modules. From top to bottom, the first three modules each consist of a 3D convolutional layer, a batch normalization layer (BN), and a max pooling layer. The last two modules each consist of a 3D convolutional layer, a 3D MLP layer, and a layer normalization layer (LN). The input 3D medical image volume data is downsampled through the first three modules to obtain local feature representations. Then, these features are passed through the last two modules to extract their long-range dependencies. The introduction of the max pooling layer improves the model's generalization ability and reduces the size of the feature map, thereby reducing computational overhead. With each module of the encoder, the size of the feature map is halved and the number of channels is doubled.

[0037] Correspondingly, the decoder also contains five modules, each corresponding to one of the encoder's five modules. From bottom to top, the first two modules each consist of a 3D convolutional layer, a batch normalization (BN) layer, a trilinear interpolation layer, a 3D MLP module, and an linear nonlinear interpolation (LN) layer. The latter three modules each consist of a 3D convolutional layer, a BN layer, and a trilinear interpolation layer. To enhance the decoder's generalization performance, this paper adds rectified linear units (ReLUs) to each of the above modules. With each module in the decoder, the feature map size doubles and the number of channels is halved.

[0038] The two modules at the bottom of the encoder and the two modules at the bottom of the decoder together constitute the network bottleneck. As mentioned earlier, each module of this network bottleneck consists of a 3D convolutional layer, a 3D MLP layer, and an LN layer. Compared to the original input of the network, the input feature map size of the network bottleneck is significantly reduced and contains more global and high-level semantic information. The 3D convolutional layer is used to smooth noise in the input feature map and extract global features, while the 3D MLP layer further mines the long-range dependencies between features through linear shift operations and fully connected layers.

[0039] like Figure 1 As shown in (c), to efficiently fuse shallow and deep features of different scales without increasing network complexity, 3D UNeXt introduces a multi-scale feature fusion module between the encoder and decoder. For the outputs of the first four encoder modules, this feature fusion module restores the outputs to their original size using a 3×3×3 3D convolution and trilinear interpolation. Then, these restored feature maps are concatenated with the output feature maps of the decoder. Finally, the concatenated feature maps are subjected to a 1×1×1 3D convolution to achieve multi-scale feature fusion.

[0040] Figure 1 (a) The basic structure of a 3D MLP module is given, which consists of five parts: an embedding layer, a depth-wise convolution layer (DWConv), a shift layer, a linear layer, and Gaussian error linear units (GELU) (Hendrycks et al., 2016).

[0041] The embedding layer uses 3D convolutions with a 3×3×3 kernel to extract local features and performs normalization operations. For each channel of these local features, the DWConv layer uses an independent 2D convolution to extract deep semantic information for that single channel. Compared to typical 3D convolutions, this approach significantly reduces the number of parameters and computational overhead.

[0042] The shifting layer shifts the feature map along the height, width, and depth axes, respectively. The shifting operation along the height axis is as follows: Figure 2 As shown: First, the feature map is split into 5 partitions; then, each partition is moved by -2, -1, 0, 1, and 2 positions along the height axis, respectively; finally, the moved partitions are concatenated to obtain a new feature map. Next, this feature map is passed through a linear layer to obtain the long-range dependencies between features at different spatial locations. The GELU layer introduces a non-linear activation mechanism into the 3D MLP module, improving the model's generalization ability.

[0043] At the end of the 3D MLP module, this paper introduces an LN layer to normalize the features. Residual connections add the original input features as residuals, avoiding gradient vanishing caused by increased network depth. The key code of the 3D MLP module is shown in Table 1.

[0044] Table 1 Key Code of 3D MLP Module

[0045] Table 1 Key code of the proposed 3D MLP block

[0046]

[0047] Loss function:

[0048] Binary cross entropy (BCE) and dice loss (DL) are used as loss functions. The formula for calculating the BCE loss function is:

[0049]

[0050] The formula for calculating the DL loss function is:

[0051]

[0052] In the formula, L BCE Let L be the BCE loss function. DL This is the loss function for deep learning. Let i be the predicted label for the i-th voxel. Let y be the i-th voxel output by the network. i The probability value, y i ∈{0,1} represents the true label of the i-th voxel. ξ is the smoothing factor.

[0053] The total loss function L is L BCE With L DL Weighted sum:

[0054] L = L BCE +αL DL #(3)

[0055] Let α = 0.4.

[0056] Experimental section:

[0057] Dataset:

[0058] The experiment used three datasets: IBSR ( https: / / www.nitrc.org / projects / ibsr ), NFBS http: / / preprocessed-connectomes-project.org / NFB_skullstripped ) and HTU-BrainMask.

[0059] The IBSR dataset contains 18 T1-weighted 3D head medical images and their corresponding human brain labels. Each image measures 256×256×128 pixels with a resolution of 1mm×1mm×2mm. The NFBS dataset contains 125 T1-weighted 3D head medical images and their corresponding human brain labels. Each image measures 192×256×256 pixels with a resolution of 1mm×1mm×1mm. HTU-BrainMask is a private dataset collected in this paper, containing 482 T1-weighted 3D head medical images, which were obtained from the IXI dataset (…). https: / / brain-development.org / ixi-dataset The images were downloaded. Each image was 256×256×150 pixels with a resolution of 0.9mm×0.9mm×1.2mm. First, cranial dissection was performed on these images using FSL (Frontal Stratum Lens), then the dissection results were manually corrected by a radiologist to obtain the final human brain label.

[0060] Data preprocessing:

[0061] Different head 3D medical imaging devices have different imaging parameters, and different patients have different imaging positions. This paper preprocesses the raw images, projecting them into the MNI (Montreal Neurological Institute) standard space (Feindel et al., 1991) using FSL, and converting their resolution to 1mm × 1mm × 1mm. During the training phase, random cropping is used as data augmentation: for each image in the same training batch, a 128×128×128 block is randomly cropped and fed into the network. During the inference phase, a sliding window is used to divide each image into multiple 128×128×128 overlapping blocks and feeds them sequentially into the network. The segmentation results of all overlapping blocks are then combined to obtain the final skull dissection result.

[0062] Experimental environment:

[0063] The experiments were conducted using the PyTorch framework on a compute node equipped with an Nvidia A100 GPU and an Intel(R) Xeon(R) Gold 6348 @ 2.60GHz CPU. The Adam optimizer was used with an initial learning rate of 0.003, a weight decay factor of 2e-6, and a batch size of 4. Validation was performed every 50 epochs; training ended if the network failed to optimize for 500 consecutive epochs.

[0064] Evaluation indicators:

[0065] The accuracy of craniectomy using different methods was evaluated using Dice, MSD, HD95, and HD99.

[0066] The formula for calculating Dice is:

[0067]

[0068] In the formula, Y represents the real label. The predicted label is Dice. Dice describes the degree of overlap between the actual label and the network prediction result, with a value of 0 to 1, where 1 indicates the best prediction result.

[0069] The formula for calculating MSD is:

[0070]

[0071] In the formula for The average distance of each voxel on the surface from the nearest voxel on the Y surface. Distance of each voxel on the Y surface The average distance between the nearest voxels on the surface. The smaller the MSD value, the higher the precision of skull dissection.

[0072] The formula for calculating HD is:

[0073]

[0074]

[0075]

[0076] In the formula These are the true label and the predicted label, respectively. HD95 and HD99 are for Y and Y respectively. The 95% and 99% of all distances between the mid-boundary points are measured. HD describes how close the skull dissection results are to the true values; the smaller the value, the better the network's prediction.

[0077] Result comparison:

[0078] This paper conducts experiments on the IBSR, NFBS, and HTU-BrainMask datasets to compare the performance of 3D UNeXt with other mainstream networks for skull dissection. Three-fold cross-validation was performed on each dataset during the experiments.

[0079] Computational complexity:

[0080] Table 2 compares the number of parameters, FLOPs, and average CPU inference time of 3D UNeXt with other mainstream networks. Average CPU inference time refers to the total inference time for a single image, calculated by dividing the original image into multiple overlapping 128×128×128 blocks using a sliding window mechanism and feeding them sequentially into the network. The total inference time for multiple images is calculated, and the average value is taken as the average CPU inference time. For UNeXt used for 2D medical image segmentation, this paper divides the 3D volume data into 2D slices, and then feeds these slices into the network one by one.

[0081] As shown in the table, UNeXt has the lowest training parameters, FLOPs, and CPU inference time among all methods, at 1.47M, 0.31G, and 2.22s, respectively. Our proposed network has the second lowest FLOPs and CPU inference time, at 8.77G and 2.54s, respectively. This is because UNeXt uses 2D convolutions and 2D MLPs. Due to the excessive use of 3D convolutions or Transformer modules, 3D U-Net, Modified-UNet, UNETR, and MSMHA-CNN all have significantly higher FLOPs and CPU inference times than our proposed network. Conversely, our proposed network reduces the use of 3D convolutions, replacing them with 3D MLP modules, thus significantly reducing FLOPs and CPU inference time. In particular, compared to the classic 3D U-Net, our proposed network reduces FLOPs by 76 times and CPU inference time by 10 times.

[0082] Quantitative results:

[0083] Tables 3, 4, and 5 quantitatively compare the skull dissection performance of 3D UNeXt and other mainstream methods. On the IBSR dataset, the proposed network's Dice, MSD, HD95, and HD99 values ​​are 0.9792, 1.0108 mm, 2.7992 mm, and 5.7635 mm, respectively, all superior to other methods. On the NFBS dataset, the proposed network's Dice, MSD, HD95, and HD99 values ​​are 0.9892, 0.5180 mm, 1.3211 mm, and 2.5137 mm, respectively, all superior to other networks. On the HTU-BrainMask dataset, the proposed network's Dice, MSD, HD95, and HD99 values ​​are 0.9806, 0.7505 mm, 2.0334 mm, and 3.5736 mm, respectively, all close to or superior to other networks. This is because the network in this paper uses a 3D MLP module in the deep layers of the network to obtain the long-range dependencies between features, and uses a multi-scale feature fusion module in the decoding stage to achieve efficient fusion of shallow and deep features, thereby significantly improving the accuracy of skull dissection.

[0084] UNeXt takes 2D slices as input, discarding the spatial and long-range dependency information contained in 3D volume data. This results in the worst brain dissection performance, despite having the lowest computational complexity. 3D U-Net, Modified-UNet, UNETR, and MSMHA-CNN use 3D convolutions or Transformers to obtain local features or long-range dependencies between features, achieving better dissection results. However, these methods do not fully capture the aforementioned information; furthermore, their FLOPs and average CPU inference time are significantly higher than the network presented in this paper.

[0085] Table 2 Comparison of parameters, FLOPs, and CPU inference time for different networks

[0086] Table 2 Comparison of parameters, FLOPs and CPU inference time of different networks

[0087]

[0088] Note: Bold text represents the optimal value for each line, and underlined text represents the second-best value for each line.

[0089] Table 3 Comparison of skull dissection results using different methods on IBSR

[0090] Table 3 Comparison of skull stripping results of different networks IBSR

[0091]

[0092] Note: Bold text represents the best value for each column, and underlined text represents the second-best value for each column.

[0093] Table 4 Comparison of skull dissection results using different methods on NFBS

[0094] Table 4 Comparison of skull stripping results of different networkson NFBS

[0095]

[0096] Note: Bold text represents the best value for each column, and underlined text represents the second-best value for each column.

[0097] Table 5 Comparison of skull dissection results using different methods on HTU-BrainMask

[0098] Table 5 Comparison of skull stripping results of different networks HTU-BrainMask

[0099]

[0100] Note: Bold text represents the best value for each column, and underlined text represents the second-best value for each column.

[0101] Qualitative results:

[0102] Figure 3 , Figure 4 and Figure 5 Visualizations of skull dissection using 3D UNeXt and other state-of-the-art networks on the IBSR, NFBS, and HTU-BrainMask datasets are presented. The results show that the dissection results of the network presented in this paper have no significant noise and exhibit the best brain edge smoothness, with the brain extraction results closest to the true labels. This is because the 3D MLP module and multi-scale feature fusion module can fully learn the long-range dependencies between features and effectively fuse shallow and deep features. UNeXt's dissection results show significant holes, mainly because it loses the spatial and long-range dependency information contained in the 3D volume data. The dissection results of 3D U-Net, Modified-UNet, UNTER, and MSMHA-CNN exhibit edge noise and poor smoothness of the extracted brain edges. This is because although these methods can acquire rich local detail information, they still cannot fully learn the long-range dependencies between features to eliminate noise from local details outside the brain region.

[0103] Ablation experiment:

[0104] To verify the effectiveness of the 3D MLP module and the multi-scale feature fusion module, this paper conducts the following two ablation experiments on the aforementioned datasets. Ablation Experiment 1: Removing the 3D MLP module to verify its effectiveness; Ablation Experiment 2: Removing the multi-scale feature fusion module to verify its effectiveness. The ablation experiment results on the three datasets are shown in Tables 6, 7, and 8. It can be seen that the proposed 3D MLP module and multi-scale feature fusion module play important roles in 3D UNeXt.

[0105] This invention designs a novel lightweight and fast craniotomy network 3D UneXt module, with the following main contributions:

[0106] 1) A novel 3D MLP module with linear shifting along different axes is proposed. This module performs shifting operations on small-scale deep features along three different axes of width, height and depth in the deep layers of the network, and fuses features along different axes with linear computational overhead; then, a small-sized fully connected layer is used to obtain the long-range dependencies between features.

[0107] 2) Design a multi-scale feature fusion module that integrates shallow and deep features of different scales. This module achieves size alignment between features of different scales through 3D convolution and trilinear interpolation, and then achieves efficient fusion between features through small-size 3D convolution.

[0108] 3) Based on the above work, a novel lightweight neural network 3D UneXt module for rapid brain and skull dissection was designed and implemented. This network outperforms or approaches the current state-of-the-art networks in terms of evaluation metrics such as Dice, MSD, HD95 and HD99 on both public and private datasets. Its parameter count, FLOPs and average CPU inference time are also much smaller than the current state-of-the-art networks.

[0109] Skull dissection is a fundamental and crucial step in the clinical diagnosis and analysis of neuroimaging, and also an active research direction in 3D medical image segmentation. Researching fast, accurate, and lightweight neural networks for skull dissection has significant theoretical and practical value. This paper proposes 3D UNeXt, a novel lightweight network architecture for skull dissection. This network can run quickly on ordinary CPUs, effectively solving the problems of large parameter count and slow inference speed in existing mainstream networks.

[0110] Traditional 3D U-Net uses multiple convolutional layers to expand the network's receptive field and acquire richer high-level semantic information. With each layer of convolution, the number of parameters increases dramatically, significantly raising computational complexity. To improve the network's ability to learn long-range dependencies between features, a Transformer module is introduced. While this improves the network's skull-splitting accuracy, it also incurs greater computational overhead. To address these issues, 3D UNeXt uses 3D convolutions in shallow layers to extract rich local feature information, and in deeper layers, it uses a 3D MLP module with shift operations to achieve feature interactions and represent long-range dependencies across different axes. The shift operations are linear, effectively reducing the computational complexity of 3D UNeXt and improving inference speed. The multi-scale feature fusion module in the decoding stage effectively fuses shallow and deep features at different scales, further improving the network's skull-splitting accuracy.

[0111] All parts not disclosed in this invention are prior art, and their specific structures, materials, and working principles will not be described in detail. Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A lightweight, rapid craniotomy network 3D UNeXt module, characterized in that: Based on U-Net, the architecture comprises three parts: an encoder, a network bottleneck, and a decoder. The encoder uses a 3D convolutional module to extract local features from medical image volume data. The network bottleneck uses a 3D MLP module combined with an axial shift mechanism to learn long-range dependencies between features. The decoder uses a multi-scale feature fusion module and skip connections to fuse shallow and deep features at different scales. The encoder contains five modules. From top to bottom, the first three modules each consist of a 3D convolutional layer, a batch normalization layer, and a max-pooling layer. The last two modules each consist of a 3D convolutional layer, a 3D MLP layer, and a normalization layer. The MLP module comprises five parts: an embedding layer, a deep convolutional layer, a shift layer, a linear layer, and a Gaussian error linear unit layer. The embedding layer uses a 3D convolution with a kernel of 3×3×3 to extract local features and perform normalization operations. For each channel of these local features, the deep convolutional layer uses an independent 2D convolution to extract deep semantic information for a single channel. The shift layer shifts the feature map along the height, width, and depth axes. The Gaussian error linear unit layer introduces a non-linear activation mechanism to the 3D MLP module, improving the model's generalization ability. The shift operation along the height axis of the shift layer involves first splitting the feature map into 5 partitions, then shifting each partition along the height axis by -2, -1, 0, 1, and 2 positions respectively, and finally concatenating the shifted partitions to obtain a new feature map. This feature map is then processed by the linear layer to obtain the long-range dependencies between features at different spatial locations.

2. The lightweight, rapid craniotomy network 3D UNeXt module according to claim 1, characterized in that: The decoder also contains five modules, which correspond one-to-one with the five modules of the encoder. From bottom to top, the first two modules each consist of a 3D convolutional layer, a batch normalization layer, a trilinear interpolation layer, a 3D MLP module, and a normalization layer. The last three modules each consist of a 3D convolutional layer, a batch normalization layer, and a trilinear interpolation layer.

3. The lightweight, rapid craniotomy network 3D UNeXt module according to claim 2, characterized in that: Each module of the decoder has an added modified linear unit. For each module of the decoder, the size of the feature map doubles and the number of channels is halved.

4. The lightweight, rapid craniotomy network 3D UNeXt module according to claim 2, characterized in that: The two modules at the bottom of the encoder and the two modules at the bottom of the decoder together form the network bottleneck. Each module of the network bottleneck consists of a 3D convolutional layer, a 3D MLP layer and a normalization layer. Compared with the original input of the network, the input feature map of the network bottleneck is significantly reduced in size and contains more global information and high-level semantic information. The 3D convolutional layer is used to smooth the noise of the input feature map and extract global features, while the 3D MLP layer further mines the long-range dependencies between features through linear shift operations and fully connected layers.

5. The lightweight, rapid craniotomy network 3D UNeXt module according to claim 2, characterized in that: A multi-scale feature fusion module is introduced between the encoder and the decoder. For the outputs of the first four modules of the encoder, the multi-scale feature fusion module restores these outputs to their original size through a 3×3×3 3D convolution and trilinear interpolation. Then, these restored feature maps are concatenated with the output feature maps of the decoder. Finally, the concatenated feature maps are subjected to a 1×1×1 3D convolution to achieve the fusion of multi-scale features.

6. A lightweight, rapid craniotomy network 3D UNeXt module according to claim 1 or 2, characterized in that: The 3D MLP module introduces a normalization layer at the end to standardize the features, and the residual connection adds the original input features as residuals to avoid gradient vanishing caused by the increase in network depth.

Images