Three-dimensional medical image segmentation method, device and medium based on deep learning

Abstract

Three-dimensional medical image segmentation method, device and medium based on deep learning. It belongs to the field of medical image processing technology, and specifically relates to the field of three-dimensional medical image segmentation technology. It solves the problem that when existing methods face segmentation targets with different shapes, large size differences and blurred boundaries, the anatomical structure integrity is poor due to insufficient global modeling capabilities; or the edge segmentation accuracy of the target is insufficient due to the lack of local inductive bias. The method includes the following steps: collecting three-dimensional medical images, dividing the training set and the validation set according to the ratio; constructing a three-dimensional medical image segmentation model; training the three-dimensional medical image segmentation model: inputting the data of the training set into the three-dimensional medical image segmentation model constructed in step S2 for training, so as to obtain model parameters that meet the requirements, and verifying the effect through the validation set; segmenting the three-dimensional medical image by the trained three-dimensional medical image segmentation model.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image processing, and in particular to the technical field of three-dimensional medical image segmentation. Background Art

[0002] Medical image segmentation involves identifying and labeling target lesions, tissues, or organs, voxel by voxel from raw medical images, providing information about their shape and volume. As a crucial prerequisite for image-guided diagnosis and computer-assisted intervention, it provides crucial support for clinical practice. Traditional methods utilize independent 2D slice processing mechanisms, which cannot meet the spatial continuity requirements of large-scale anatomical structures (such as the spine and abdominal organs) and cannot preserve three-dimensional context. 3D segmentation methods, on the other hand, can capture the continuity of anatomical structures in three dimensions and are suitable for segmenting objects with complex three-dimensional morphology. They have been widely studied in the field of medical image analysis.

[0003] In recent years, the rapid development of deep learning has brought about tremendous changes in 3D medical image segmentation. The traditional U-Net-based encoder-decoder architecture extracts multi-level feature representations through the encoder and gradually restores the resolution through the decoder to generate pixel-by-pixel segmentation results. It is widely used in medical image segmentation tasks. However, the U-Net framework relies on fixed-scale local convolution kernels for feature extraction. Although it is beneficial for local texture modeling, it is limited by the constraints of the receptive field and cannot effectively capture long-range dependencies. The spatial relationship between pixels in medical image structures far exceeds the receptive field of a single convolution kernel. Although the receptive field can be expanded by enlarging the convolution kernel, it will significantly increase the computational complexity and memory usage in 3D high-resolution medical images.

[0004] To effectively capture long-range dependencies, many studies have applied the Transformer's self-attention mechanism to medical image segmentation. This approach achieves global context modeling through dynamic weight allocation, demonstrating excellent performance in medical image segmentation tasks. However, its computational complexity grows quadratically with the input resolution, leading to severe computational bottlenecks when processing high-resolution 3D medical images. Mamba, a novel state-space model (SSM), achieves linear computational complexity through a selective state transfer mechanism, demonstrating significant computational efficiency advantages in long-sequence modeling tasks. However, existing methods employ simple serial or parallel architectures, lack dynamic adaptive fusion of cross-scale features, and lack targeted modeling of multi-scale anatomical structures in medical images. Vision LSTM (ViL), based on the extended long short-term memory (xLSTM), addresses the parallelization and storage issues of traditional LSTMs through a matrix memory structure and exponential gating mechanism, effectively capturing dependencies in long sequences while maintaining linear complexity. ViL's use of an alternating odd-even unidirectional scanning strategy strikes a balance between computational efficiency and performance in traditional 2D vision tasks, but struggles with global spatial modeling of complex 3D medical images. Summary of the Invention

[0005] The present application provides a three-dimensional medical image segmentation method based on deep learning, thereby solving the problem that existing methods (such as convolutional neural networks and Transformer models) have poor anatomical structure integrity due to insufficient global modeling capabilities when faced with segmentation targets of various shapes, large size differences, and blurred boundaries; or insufficient edge segmentation accuracy of the target due to the lack of local inductive bias.

[0006] The method comprises the following steps:

[0007] S1. Data collection and division:

[0008] Collect 3D medical images and divide them into training and validation sets according to the ratio;

[0009] S2. Construct a 3D medical image segmentation model, specifically:

[0010] S21. Construct an encoder with a dual path, wherein the dual path encoder is composed of four identical basic modules, each basic module includes a main path and an auxiliary path, and the outputs of the main path and the auxiliary path enter a feature fusion module for fusion;

[0011] S22. Construct a decoder, wherein the decoder includes a B-ViL module and a layer-by-layer upsampling module. The output of the encoder first enters the B-ViL module and then passes through the layer-by-layer upsampling module in sequence before being output through the output layer.

[0012] S3. 3D medical image segmentation model training: inputting the data of the training set into the 3D medical image segmentation model built in step S2 for training, thereby obtaining model parameters that meet the requirements, and verifying the effect through the validation set;

[0013] S4. Segment the three-dimensional medical image using the trained three-dimensional medical image segmentation model.

[0014] Furthermore, the layer-by-layer upsampling module includes four upsampling modules, and each of the upsampling modules is connected to the feature fusion module through a residual block.

[0015] Furthermore, the backbone path is composed of a standard convolutional module, which includes two layers, each of which sequentially includes a convolutional layer, a three-dimensional instance normalization layer, and an activation layer.

[0016] Furthermore, the auxiliary path is composed of a multi-axial multi-branch depth-separable convolution module, and the multi-axial multi-branch depth-separable convolution module includes three feature extraction branches, the first feature extraction branch is a depth convolution branch, the second feature extraction branch is a directional large kernel depth convolution branch, and the third feature extraction branch is an identity mapping branch.

[0017] Furthermore, in the feature fusion module, the features output by the main path and the auxiliary path are added element by element to obtain the fused feature, and then the spatial and channel dimensions of the fused feature are compressed respectively, and the channel attention weight and the spatial attention weight are calculated. Next, the channel attention weight and the spatial attention weight are multiplied by the features output by the main path and the auxiliary path, and then the features output by the weighted auxiliary path are dynamically fused into the features output by the weighted main path through the guided attention fusion module.

[0018] Furthermore, when the channel attention weight and spatial attention weight are multiplied by the features output by the main path and the auxiliary path, four fusion features are obtained, namely: the fusion feature obtained by multiplying the channel attention weight by the features output by the main path , multiply the channel attention weight by the fusion feature obtained by the output of the auxiliary path , multiply the spatial attention weight by the fusion feature obtained by the output of the backbone path , multiply the spatial attention weight by the fusion feature obtained by the output of the auxiliary path .

[0019] Furthermore, the features output by the weighted auxiliary path are dynamically fused into the features output by the weighted main path through the guided attention fusion module. Specifically, the fused features Dynamically fuse to fusion features by guiding the attention fusion module In the fusion feature , the fusion features Dynamically fuse to fusion features by guiding the attention fusion module In the fusion feature , and then fuse the features and fusion features Fusion is performed via standard convolutional modules.

[0020] Furthermore, the B-ViL module uses a block embedding operation to split the input features into several blocks, maps these blocks to a fixed dimension and adds a learnable vector to each block, then uses these blocks to construct an input sequence, processes the sequence through a Vision-LSTM module with a bidirectional scanning mechanism, thereby capturing the relationship between feature space positions, and finally remaps the processed sequence back to the same spatial dimension and size as the input features through a block expansion operation.

[0021] The beneficial effects of the method of the present invention are:

[0022] Enhanced multi-scale feature extraction capability: The method described in the present invention uses a dual-path encoder structure, combined with standard convolution and axial multi-branch depth-separable convolution, to achieve an organic combination of local correlation modeling and multi-scale feature extraction, effectively improving the model's global modeling capability for segmentation targets of various shapes and sizes, thereby significantly improving the anatomical structure integrity of the segmentation results.

[0023] Improve edge segmentation accuracy: The method described in the present invention introduces a feature fusion module of the collaborative spatial-channel attention mechanism to dynamically fuse the multi-scale features extracted by the dual-path encoder, thereby enhancing the model's ability to capture target edge details and solving the problem of insufficient segmentation accuracy caused by the lack of local inductive bias in existing methods.

[0024] Optimizing spatial modeling and resolution restoration: The method described in the present invention designs a decoder module based on Vision-LSTM, which enhances the efficiency of feature transfer through residual blocks and skip connections, and combines a bidirectional scanning mechanism with an upsampling module to restore spatial resolution layer by layer, thereby significantly improving spatial modeling capabilities and segmentation accuracy in 3D vision tasks.

[0025] High computational efficiency and applicability: While ensuring segmentation performance, the method described in the present invention reduces the computational complexity of the model through efficient network design and computational strategies, enabling it to better adapt to the actual application scenarios of three-dimensional medical image segmentation, and providing more efficient and accurate technical support for clinical diagnosis and research. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1This is the overall architecture diagram of the deep learning-based 3D medical image segmentation model of the present invention;

[0027] Figure 2 This is the overall architecture diagram of the auxiliary path of the present invention;

[0028] Figure 3 This is the overall architecture diagram of the feature fusion module of the present invention;

[0029] Figure 4 1 is a structural diagram of the channel weight optimization module in the feature fusion module of the present invention;

[0030] Figure 5 It is a structural diagram of the spatial weight optimization module in the feature fusion module of the present invention;

[0031] Figure 6 This is a structural diagram of the GA (Guided Attention) module of the present invention;

[0032] Figure 7 1 is a block diagram of the Vision-LSTM module of the bidirectional scanning mechanism in the decoder of the present invention;

[0033] Figure 8 This is a visualization diagram comparing the segmentation effects of the model of the present invention and other advanced models in the heart segmentation task;

[0034] Figure 9 This is a visualization diagram comparing the segmentation effects of the model of the present invention and other advanced models in the spine segmentation task;

[0035] Figure 10 This is a 3D visualization effect diagram of the model of the present invention in the spine segmentation task;

[0036] Figure 11 This is a visualization diagram comparing the segmentation effects of the model of the present invention and other advanced models in the brain tumor segmentation task. DETAILED DESCRIPTION

[0037] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0038] Example 1

[0039] This embodiment provides a three-dimensional medical image segmentation method based on deep learning, the method comprising the following steps:

[0040] S1. Data collection and division:

[0041] Collect 3D medical images and divide them into training and validation sets according to the ratio;

[0042] S2. Construct a three-dimensional medical image segmentation model, the model is as follows Figure 1 As shown, specifically:

[0043] S21. Construct an encoder with a dual path, wherein the dual path encoder is composed of four identical basic modules, each basic module includes a main path and an auxiliary path, and the outputs of the main path and the auxiliary path enter a feature fusion module for fusion;

[0044] S22. Construct a decoder, wherein the decoder includes a B-ViL module and a layer-by-layer upsampling module. The output of the encoder first enters the B-ViL module and then passes through the layer-by-layer upsampling module in sequence before being output through the output layer.

[0045] S3. 3D medical image segmentation model training: inputting the data of the training set into the 3D medical image segmentation model built in step S2 for training, thereby obtaining model parameters that meet the requirements, and verifying the effect through the validation set;

[0046] S4. Segment the three-dimensional medical image using the trained three-dimensional medical image segmentation model.

[0047] Example 2

[0048] This embodiment further limits Example 1 and further introduces the three-dimensional medical image segmentation model.

[0049] The encoder with dual paths is composed of four identical basic modules. Each basic module includes a main path and an auxiliary path, and the outputs of the main path and the auxiliary path enter the feature fusion module for fusion.

[0050] The backbone path is composed of a standard convolutional module (Conv Block), which contains two layers, each of which contains Convolutional layer, 3D instance normalization layer (InstanceNorm3d), and activation layer (ReLU activation function).

[0051] The auxiliary path is composed of a multi-axial multi-branch depthwise separable convolution module (IDC Block), such as Figure 2 As shown, the multi-axial multi-branch depth-separable convolution module includes three feature extraction branches. The first feature extraction branch is The deep convolution branch is used to extract local features. The second feature extraction branch is the directional large kernel deep convolution branch, which contains three deep convolutions (DWConv) in sequence, with sizes of 、 and , used to extract features of large receptive field; the third feature extraction branch is the identity mapping branch (Identity).

[0052] In the auxiliary path, features are first split into three branches along the channel dimension before being fed into the multi-axial, multi-branch, depthwise separable convolution module. The data from these three branches is then fed into the three feature extraction branches of the multi-axial, multi-branch, depthwise separable convolution module. The features output from these three feature extraction branches are then concatenated along the channel dimension. All parallel branches do not change the number of channels, so the concatenated feature map maintains the same dimensionality as the input feature map. To improve model training efficiency, the concatenated features are normalized, and residual connections are introduced to address vanishing gradients and gradient degradation during training. These residual connections connect the normalized features with the features fed into the auxiliary path. Finally, a ReLU activation function is applied to the normalized feature map for nonlinear mapping.

[0053] The multi-axis multi-branch depthwise separable convolution module is designed for the characteristics of significant object scale differences and complex anatomical structures in 3D medical images. Its purpose is to capture multi-scale features. A channel partitioning strategy is used to balance computational efficiency and feature diversity. The input features are divided into three groups along the channel dimension, including an identity mapping branch, a local 3×3×3 convolution branch, and a group of directional large kernel depthwise convolution branches. Specifically, for a given input feature , (where B represents the batch size, C is the number of channels, and D, H, and W represent the depth, height, and width of the three-dimensional data respectively), first divide it into three branches along the channel dimension:

[0054] ;

[0055] Where, represents the identity mapping branch, which is set to 0.5 times the number of channels; Represents a set of directional large-kernel depth convolution branches, divided into 0.375 times the number of channels; Represents a local 3×3×3 convolution branch, divided into 0.125 times the number of channels.

[0056] for Branch, adopt The convolution is used to extract local features, such as formula:

[0057] ;

[0058] Where: Conv represents the regular 3×3×3 convolution, and c represents the number of channels of the branch.

[0059] Will , and sequentially apply 1D oriented convolution kernels in depth, height, and width dimensions for processing, through

[0060] ;

[0061] Get the final output .

[0062] As an identity mapping branch, it is not processed: .

[0063] Finally, the auxiliary path is further processed to obtain the final output ,

[0064] .

[0065] Where, Represents input features that have not yet been split. Concat represents concatenation along the channel dimension. All parallel branches do not change the number of channels, so the concatenated feature map maintains the same shape as the input feature map. To improve model training efficiency, the concatenated features are normalized, and residual connections are introduced to address gradient vanishing and degradation issues during training. Finally, the ReLU activation function is applied to the normalized feature map for nonlinear mapping, enhancing the model's nonlinear expression capabilities.

[0066] The feature fusion module (GAF Block) dynamically fuses the multi-scale features extracted by the dual encoder path through the collaborative space-channel attention mechanism. The overall architecture of the feature fusion module is as follows: Figure 3 shown.

[0067] In the feature fusion module, the features output by the main path and the auxiliary path are added element by element to obtain the fused features. The spatial and channel dimensions of the fused features are then compressed respectively, and the channel attention weight and spatial attention weight are calculated. Next, the channel attention weight and spatial attention weight are multiplied by the features output by the main path and the auxiliary path. The features output by the weighted auxiliary path are then added to the features output by the weighted main path through the guided attention fusion module.

[0068] pass Figure 1 and Figure 3 Further introduction to the feature fusion module:

[0069] GAF Block (Guided Attention Fusion Block): This module is the feature fusion module. Figure 3As shown in Figure 1, in order to solve the problem of effective fusion of multi-scale context information in the dual encoder path, a guided attention feature fusion module is built to dynamically integrate features with differentiated receptive fields in the dual encoder path through the collaborative spatial and channel attention mechanisms. , among which is the output feature of Conv Block, It is the feature after IDC Block processing. and Add element by element to get the fused feature map .Will As the input of the spatial attention module (Spatical Attention) and the channel attention module (Channel Attention), the channel attention weights are obtained respectively and spatial attention weights , the calculation process can be expressed as formula:

[0070] ;

[0071] Where, and Represent the output features of the two branches respectively, and add them element by element to get the fused feature map F. represents the channel attention module, Represents the spatial attention module. For the input feature , respectively with and Multiply them together to get four feature maps: , , , .

[0072] The structures of the channel attention module and the spatial attention module are as follows: Figure 4 and Figure 5 shown.

[0073] feature and As a group, and As a group, the two groups are input into the GA module (GuidedAttention). The structure of the GA module is as follows: Figure 6 As shown. Among them, the characteristics The spatial and channel reconstruction features of are used as the feature X in the GA module, called Main, and the feature The spatial and channel reconstructed feature maps are used as feature Y in the GA module, called Guidance. The feature maps processed by the two GA modules are element-wise added and then processed by a standard convolution module to obtain the final output feature map. The specific calculation process is described as follows:

[0074] ;

[0075] Where GA represents the guided attention module, Conv represents the convolution block, Represents the feature map of the final output.

[0076] Channel Attention: This module is located in Figure 3 The Channel Attention part in is the channel weight optimization module of the present invention, such as Figure 4 As shown. The pooled features are jointly modeled to enhance the ability of the two features to express in a coordinated manner. The number of output channels of MLP is half of the input, and finally the channel attention weight is obtained through Sigmoid. The calculation process of channel attention can be as follows:

[0077] ;

[0078] Where, represents global average pooling, Represents the global maximum pooling, Concat represents splicing along the channel dimension, represents Sigmoid, Represents the channel attention weight.

[0079] Spatial Attention: This module is located in Figure 3 The Spatial Attention part in is the spatial weight optimization module of the present invention, such as Figure 5 The spatial attention module generates a spatial attention map by dynamically calculating the relationship between spatial information. First, the channel dimension of the input feature is compressed by global average pooling and global maximum pooling to obtain two feature maps. and , and then splice according to the channel dimension to get a 2-channel feature Three different rate dilated convolutions are used to capture multi-scale features, which enhances the ability of ordinary convolution to capture spatial relationships while further reducing computational complexity. The convolution of fused multi-scale features and suppressed redundant information, and finally the spatial attention weight was obtained through Sigmoid. The calculation process of spatial attention is as follows:

[0080] ;

[0081] ;

[0082] Where, represents global average pooling, Represents the global maximum pooling, Concat represents splicing along the channel dimension, represents a regular 3D convolution with a kernel size of 3, represents a dilated convolution with a rate of 2. represents a dilated convolution with a rate of 3. represents a convolution with a kernel size of 1, represents Sigmoid, represents the spatial attention weight.

[0083] GA Block: This module is located in Figure 3 The GA part in is the Guided Attention module, such as Figure 6 As shown in the figure. The features processed by the IDC Block on the left encoder branch are features with a large receptive field. The features processed by the spatial and channel attention are used as guiding features to construct the global feature modeling of the main branch. The specific calculation process of the guided attention module is as follows:

[0084] ;

[0085] Where, This is a learnable parameter with an initial value of 0 that dynamically controls the weight of guided attention. ⊙ represents element-wise multiplication. GA Gate represents the Guided Attention Gate, Concat represents concatenation along the channel dimension, and "+" represents a residual connection. Represent the input backbone features and guide features respectively.

[0086] The present invention designs a decoder module B-ViL module based on Vision-LSTM (ViL), such as Figure 7 As shown in the figure, the B-ViL module uses a patch embedding operation to split the input features into several blocks, maps these blocks to a fixed dimension and adds a learnable vector to each block. Then, an input sequence is constructed using these blocks, and the sequence is processed by a Vision-LSTM module with a bidirectional scanning mechanism to capture the relationship between the feature space positions. Finally, the processed sequence is remapped back to the same spatial dimension and size as the input feature through a patch expansion operation.

[0087] Aiming to improve the spatial modeling ability and segmentation accuracy in 3D vision tasks, this module processes the features output by the encoder through residual blocks (ResBlock) to enhance information transmission and feature expression capabilities, and at the same time uses jump connections to transmit the residual-processed feature information to the decoding path, and combines multiple upsampling modules to restore the spatial resolution layer by layer; the upsampling module (Up-Sample) gradually enlarges the spatial size of the feature map and generates segmentation results through the output layer, where the number of channels of the output layer is set according to the number of task categories to ensure that the segmentation results accurately match the input image size and task requirements; in addition, the decoder module combines a bidirectional scanning mechanism with an efficient feature transmission strategy to significantly improve the performance of the network in 3D vision tasks, thereby achieving more efficient and more accurate segmentation effects.

[0088] Up-Sample Block: Figure 1 As shown in Figure 1, this module is an upsampling module that passes feature information processed by the residual module (Res Block) to the decoder via skip connections. It performs layer-by-layer upsampling to restore spatial resolution, effectively generating an output segmentation result that matches the input image size. The decoding path includes multiple upsampling blocks. Each upsampling module gradually amplifies the feature map, ultimately obtaining a segmentation result through the output layer. The number of channels in the output layer corresponds to the number of categories in the task.

[0089] Example 3

[0090] This example further illustrates the effectiveness and universality of the method of the present invention through specific experiments.

[0091] Our method was evaluated on three different datasets: the ACDC dataset for heart segmentation, the BraTS2021 dataset for brain tumor segmentation, and the VerSe2019 dataset for vertebral segmentation. To verify the effectiveness of the proposed module, ablation experiments were conducted on the IDC Block, GAF Block, and B-ViL Block. To ensure a fair comparison, the same experimental setup was used for our method and all comparison methods. Table 1 shows the basic information and partitioning of the datasets.

[0092] Table 1:

[0093]

[0094] This example uses the Dice Similarity Coefficient (DSC) and the 95th percentile Hausdorff Distance (HD95) as core evaluation metrics. These metrics quantify the segmentation results based on the degree of region overlap and boundary fit, respectively. The Dice coefficient reflects segmentation accuracy by calculating the degree of overlap between the predicted segmented region and the ground truth. It is defined as follows:

[0095] ;

[0096] Where A represents the segmentation area predicted by the model, and B represents the Ground Truth, i.e. the real standard data. The value range of DSC is The larger the percentage, the higher the overlap between the segmentation result and the Ground Truth, and the better the segmentation effect.

[0097] However, it is worth noting that this indicator is sensitive to the internal filling completeness of the segmented target, but may not be sensitive to the boundary detail deviation, which is not enough for medical image segmentation tasks with extremely high boundary accuracy requirements.

[0098] Therefore, in order to evaluate the consistency between the segmentation boundary and the true contour, HD95 is introduced to measure the distance between the boundary of the segmentation prediction and the voxel of the GT. Its definition is as follows:

[0099] ;

[0100] Where A represents the segmentation area predicted by the model, and B represents the Ground Truth. It represents the maximum distance between the surface voxel points of A and B, and the maximum value of the 95th percentile is taken as the result.

[0101] The experiment was implemented using Pytorch 2.1.2 and trained on a NVIDIA GeForce RTX 4090 24GB GPU. For the BraTS dataset, The input size is trained with a batch size of 2. For the VerSe dataset, we use , the batch size is 2. For the ACDC dataset, The input size is trained, the batch size is set to 4, and AMOS2022 adopts The input size is , and the batch size is set to 2. The same hyperparameter settings are used for all datasets. The SGD optimizer with Nesterov momentum (coefficient μ = 0.99) is used for training for 1000 epochs. The initial learning rate is set to 0.01 and the learning rate decay weight is set to 3e-5.

[0102] To fully validate the performance of our model and method, we conducted experiments on three 3D medical image datasets. Tables 2 through 4 report the quantitative comparison results with state-of-the-art models on these three datasets. Ours is the model of our invention, and the other names refer to existing models. Figure 8 、 Figure 9 、 Figure 11 They are the qualitative comparison results corresponding to Table 2 to Table 4, Figure 10 The 3D visualization of the proposed model in the spine segmentation task is shown in Table 2. Experiments using the ACDC dataset are compared with other methods, using the Dice similarity coefficient for evaluation. Another evaluation metric, HD95, is also introduced (RV: right ventricle, Myo: myocardium, LV: left ventricle, Mean: the average of all evaluation metrics).

[0103] Table 2:

[0104]

[0105] Table 3 uses the VerSe2019 dataset for experiments and compares it with other methods. The Dice similarity coefficient is used for evaluation, and another evaluation indicator HD95 is introduced (Cervical: cervical vertebra, Thoracic: thoracic vertebra, Lumbar: lumbar vertebra, Mean: the average of the three categories).

[0106] Table 3:

[0107]

[0108] Table 4 uses the BraTS2021 dataset for experiments and compares with other methods. The Dice similarity coefficient is used for evaluation, and another evaluation indicator HD95 is introduced (WT: whole tumor, TC: tumor core, ET: enhanced tumor, Mean: the average of the three categories).

[0109] Table 4:

[0110]

[0111] This embodiment conducts various ablation experiments on three data sets to verify the effectiveness of each module of the present invention.

[0112] Table 5 shows the ablation results on BraTS2021, where (1) Baseline is a standard 3D-UNet architecture, (2) Baseline+IDC means replacing the Baseline encoder with the IDC Block in our model, (3) Dual-Encoder means using two encoder branches based on Baseline, one using the IDC Block and the other using the standard 3D-UNet encoder, and using a simple element-by-element addition feature fusion method. (4) DualEncoder+GAF means using GAF for feature fusion based on (3). (5) Ours is our model.

[0113] In (2), the encoder is replaced by the proposed IDC Blcok, which can significantly improve the feature extraction capability, and the Dice similarity coefficient reaches 85.49%. (3) uses a two-path encoder, but the effect is not as good as (2). This is because the feature extraction capability of the ordinary encoder is not as good as IDC and the simple element-by-element addition cannot effectively fuse the feature map, resulting in the result of (3) (Dice similarity coefficient 84.84%) being better than (1)'s 84.05% but not as good as (2)'s 85.49%. This is a normal phenomenon. (4) The GAF Block of the present invention is added on the basis of the dual-path encoder. Because the ordinary encoder has a better ability to extract local detail features, while IDC is good at extracting multi-scale features, the ordinary element-by-element addition method cannot effectively fuse features, while the GAF Blcok of the present invention can effectively fuse the two, thereby achieving better performance of 85.87%. (5) is the method of the present invention, which adds BViL Blcok to the bottleneck block on the basis of (4), which can significantly improve the global context modeling capability and achieve a Dice similarity coefficient of 86.15. The role of this module is more obvious in the task of large anatomical structures.

[0114] Table 5:

[0115]

[0116] Table 6 shows the ablation experiment results on the ACDC dataset. The results of (1)-(4) are similar to those in Table 5, but the result of (5) (Dice similarity coefficient 90.93%) is lower than the 91.64% of (4). This is because the B-ViL module is used for long-distance modeling in three-dimensional space. However, the data in ACDC is extremely low in depth (only about 10), and is further reduced in the deep layer of the network (B-ViL is applied in the bottleneck layer), making the B-ViL module extremely susceptible to noise interference, resulting in a decrease in segmentation performance.

[0117] Table 6:

[0118]

[0119] Table 7 shows the ablation experiment results on the VerSe19 dataset, which further proves the views described in Table 5. B-ViL can significantly improve the performance on large anatomical structures (the Dice similarity coefficient increases by 2.42% after adding B-ViL).

[0120] Table 7:

[0121]

Claims

1. A three-dimensional medical image segmentation method based on deep learning, characterized in that: The method comprises the following steps: S1. Data collection and division: Collect 3D medical images and divide them into training and validation sets according to the ratio; S2. Construct a 3D medical image segmentation model, specifically: S21. Construct an encoder with a dual path, wherein the dual path encoder is composed of four identical basic modules, each basic module includes a main path and an auxiliary path, and the outputs of the main path and the auxiliary path enter a feature fusion module for fusion; The backbone path consists of a standard convolutional module, which contains two layers, each of which sequentially includes a convolutional layer, a 3D instance normalization layer, and an activation layer; The auxiliary path is composed of a multi-axial multi-branch depth-separable convolution module, which includes three feature extraction branches, the first feature extraction branch is a depth convolution branch, the second feature extraction branch is a directional large kernel depth convolution branch, and the third feature extraction branch is an identity mapping branch; In the feature fusion module, the features output by the main path and the auxiliary path are added element by element to obtain the fused feature. The spatial and channel dimensions of the fused feature are then compressed respectively, and the channel attention weight and spatial attention weight are calculated. The channel attention weight and spatial attention weight are then multiplied by the features output by the main path and the auxiliary path. The features output by the weighted auxiliary path are then dynamically fused into the features output by the weighted main path through the guided attention fusion module. S22. Construct a decoder, wherein the decoder includes a B-ViL module and a layer-by-layer upsampling module. The output of the encoder first enters the B-ViL module and then passes through the layer-by-layer upsampling module in sequence before being output through the output layer. The B-ViL module uses a block embedding operation to split the input features into several blocks, maps these blocks to a fixed dimension and adds a learnable vector to each block. It then uses these blocks to construct an input sequence, processes the sequence through a Vision-LSTM module with a bidirectional scanning mechanism to capture the relationship between feature spatial positions, and finally remaps the processed sequence back to the same spatial dimension and size as the input features through a block expansion operation; S3. Training the 3D medical image segmentation model: inputting the data of the training set into the 3D medical image segmentation model described in step S2 for training, thereby obtaining model parameters that meet the requirements, and verifying the effect using the validation set; S4. Segment the three-dimensional medical image using the trained three-dimensional medical image segmentation model.

2. The deep learning-based three-dimensional medical image segmentation method according to claim 1, characterized in that: The layer-by-layer upsampling module includes four upsampling modules, and each of the upsampling modules is connected to a feature fusion module through a residual block.

3. The deep learning-based three-dimensional medical image segmentation method according to claim 2, characterized in that: When the channel attention weight and spatial attention weight are multiplied by the features output by the main path and the auxiliary path, four fusion features are obtained, namely: the fusion feature obtained by multiplying the channel attention weight by the features output by the main path , multiply the channel attention weight by the fusion feature obtained by the output of the auxiliary path , multiply the spatial attention weight by the fusion feature obtained by the output of the backbone path , multiply the spatial attention weight by the fusion feature obtained by the output of the auxiliary path .

4. The deep learning-based three-dimensional medical image segmentation method according to claim 3, characterized in that: The features output by the weighted auxiliary path are dynamically fused into the features output by the weighted main path through the guided attention fusion module. Specifically: Dynamically fuse to fusion features by guiding the attention fusion module In the fusion feature , the fusion features Dynamically fuse to fusion features by guiding the attention fusion module In the fusion feature , and then fuse the features and fusion features Fusion is performed via standard convolutional modules.

5. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

6. A computer-readable storage medium for storing computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images