Post-decoder three-dimensional axial decoupling enhancement method for three-dimensional medical image segmentation

Abstract

The invention discloses a post-decoder three-dimensional axial decoupling enhancement method for three-dimensional medical image segmentation. The method mainly comprises two main pipelines, namely a replacement dimension enhancement pipeline and a sequence axial enhancement pipeline. PDI learns and understands different axial features from the perspective of global spatial information and performs attention enhancement on z-axis features, and SAI performs grouping processing on z-axis information directly on a time axis so as to enhance the feature expression ability of the model. The spatial-temporal features of different axial information of three-dimensional medical image data are analyzed, an attention mechanism is introduced to independently enhance the feature expression of the axial time dimension information, and through independent work and cooperative processing of a PDI pipeline and an SAI pipeline, axial decoupling enhancement can be carried out on a coarse mask of three-dimensional medical image segmentation with high complexity and high difficulty, and the accuracy of the three-dimensional medical image segmentation is improved. A more accurate and robust segmentation result is realized, the method can be adapted to any three-dimensional image segmentation deep learning model, and the wide application potential of the method in the field of three-dimensional medical image segmentation is shown.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional medical image segmentation, and specifically to a post-decoder three-dimensional axial decoupling enhancement method for three-dimensional medical image segmentation, PaR (Post-Axial Refiner). Medical image three-dimensional medical image segmentation fundus retinal tissue segmentation medical image PaR analyzes the differences in spatiotemporal features of different axes of three-dimensional medical image data, introduces an attention mechanism to enhance the model's ability to express time axis features, thereby more accurately mapping structural differences and detail changes on the image time axis, and improving the accuracy and robustness of three-dimensional medical image segmentation. This method is applicable to a variety of medical image segmentation tasks such as abdominal organ segmentation, fundus retinal tissue segmentation, and cardiac system segmentation, providing an efficient automated segmentation tool for clinical diagnosis to assist diagnosis. Background Art

[0002] Three-dimensional medical imaging data based on medical imaging technologies such as MRI and CT are extremely critical for accurate clinical diagnosis. These data contain the precise spatial structure of each tissue and organ in the patient's body and its dynamic changes over time, such as blood flow or organ movement. The three-dimensional nature of these data allows doctors to deeply analyze pathological conditions and physiological processes. In order to extract accurate biological structure information from these complex data sets, conduct effective disease diagnosis and subsequent treatment analysis, accurate three-dimensional medical image segmentation is essential. Traditional two-dimensional image segmentation technology ignores the hierarchical relationship between the time axis slices of three-dimensional medical imaging data, and cannot effectively express the spatiotemporal correlation of three-dimensional data, resulting in a lack of three-dimensional cohesion and accuracy in the segmentation results. Three-dimensional convolution directly extracts features in three-dimensional space by simultaneously expanding the convolution kernel in three dimensions for information analysis and processing, and can utilize information from each dimension in the three-dimensional medical image data, ensuring that the segmentation results can truly reflect the integrity of the three-dimensional structure, retain the continuity of the space and enhance the local correlation of information.

[0003] However, the symmetrical structure design of the 3D convolution kernel makes it process the time axis information (z axis) in the same way as the slice plane (XY plane) information, and equally collects information in three dimensions. This processing strategy ignores the differences in the physical properties and information density of 3D medical image data in the z axis and the xy plane, which will lead to inefficient use and misunderstanding of each axial information by the deep learning model. In 3D medical image data, the xy plane image information is directly collected by the sensor array, and the brightness and darkness changes of the pixels map the spatial distribution of tissue density to reflect the exact position of different density tissues in physical space. In contrast, the z axis is a scanning sequence along the time axis, which is obtained by scanning layer by layer, and the interval between each layer is usually much larger than the pixel resolution in the xy plane. Therefore, the characteristics of the z axis reflect the complex of time process and spatial structure, and the spatiotemporal characteristics of its information are fundamentally different from the pure spatial characteristics of the xy axis. However, the structure of the 3D convolution is the same in all directions, which cannot reflect the difference of this information. This axial information asymmetry makes it impossible for simple 3D convolution to effectively distinguish and utilize the characteristic information of each axis, especially when processing z axis information. Since the information density of the z-axis is much lower than that of the xy plane, traditional 3D convolution may introduce spatial deviations and errors when evenly extracting features on the z-axis and xy plane, and apply the xy plane features that reflect specific spatial relationships to the z-axis features, ultimately affecting the segmentation accuracy.

[0004] In the field of medical image analysis and processing, high-precision segmentation of three-dimensional medical imaging data is extremely critical for accurate and efficient clinical diagnosis and subsequent treatment analysis. However, most of the existing deep learning methods use three-dimensional convolution to process the information of three-dimensional medical image data in three dimensions. This method has certain limitations and ignores the differences in the physical and scanning characteristics of three-dimensional image data in different axes, resulting in poor performance in three-dimensional medical image segmentation tasks. In order to solve this problem, the present invention proposes a three-dimensional axial decoupling enhancement (Post-Axial Refiner, PaR) method for three-dimensional medical image segmentation after coarse mask optimization. PaR splits the coarse mask obtained after the deep learning model decoder along the time axis and uses the attention mechanism to enhance the feature expression, realizing three-dimensional axial decoupling to enhance the deep learning model's understanding of the association of spatiotemporal information in three-dimensional image data. PaR analyzes the differences in the features of each axis of the three-dimensional medical image data and introduces the attention mechanism to independently enhance the expression ability of the z-axis feature information, thereby more accurately mapping the structural differences and detail changes on the time axis, and improving the accuracy and robustness of three-dimensional medical image segmentation. This module is suitable for a variety of medical image segmentation tasks such as abdominal organ segmentation, fundus retinal tissue segmentation and cardiac system segmentation, providing a high-precision automated segmentation tool for clinical diagnosis to assist diagnosis. The proposed PaR is a plug-and-play module that can be easily integrated into any neural network architecture, not limited to 3D medical image segmentation models. Summary of the invention

[0005] The purpose of the present invention is to provide a post-decoder three-dimensional axial decoupling enhancement method for three-dimensional medical image segmentation in view of the deficiencies of the prior art. The method analyzes the spatiotemporal characteristics of different axial information of three-dimensional medical image data, introduces an attention mechanism to independently enhance the feature expression of axial time dimension information, thereby more accurately mapping the structural differences and detail changes on the time axis (z axis), and improving the accuracy and robustness of three-dimensional medical image segmentation. The method is suitable for a variety of medical image segmentation tasks such as abdominal organ segmentation, fundus retinal tissue segmentation, and cardiac system segmentation, and provides a high-precision automated segmentation tool for clinical diagnosis to assist diagnosis. The method is based on the idea of ​​three-dimensional axial decoupling, and enhances the feature expression ability of the model by specially processing the time axis information of three-dimensional medical image data to improve the performance of the model in three-dimensional medical image segmentation tasks. The method mainly includes two main pipelines: the permutative dimensional intensification (PDI) pipeline and the sequential axial intensification (SAI) pipeline. PDI learns and understands different axial features from the perspective of global spatial information and enhances attention to z-axis features, while SAI directly groups z-axis information on the time axis to enhance the model's feature expression capabilities. Through the independent work and collaborative processing of the PDI pipeline and the SAI pipeline, the present invention can perform axial decoupling enhancement on the coarse mask of high-complexity and high-difficulty three-dimensional medical image segmentation, achieve more accurate and robust segmentation results, and can be adapted to any three-dimensional image segmentation deep learning model, demonstrating its wide application potential in the field of three-dimensional medical image segmentation.

[0006] The technical solution included in the present invention for solving the technical problem comprises the following steps: S1: Collect 3D medical image segmentation datasets corresponding to various organ segmentation tasks from the Internet.

[0007] S2: Divide the collected 3D medical image data into a training set and a test set in proportion; S3: performing data preprocessing and data enhancement on the three-dimensional medical image data in step S2; S4: Collect advanced deep learning models widely used for 3D medical image segmentation from the Internet as benchmark models; S5: Design a post-decoder 3D axial decoupling enhancement method (PaR) that can be used for coarse mask optimization in various 3D medical image segmentation tasks; S6: Integrate the PaR approach into the collected benchmark models.

[0008] S7: Use the baseline model and the baseline model integrated with PaR to perform model training on the preprocessed and data-enhanced datasets, and save the respective best model weights according to the evaluation results on the validation set; S8: The saved best model weights are verified on the test set to evaluate the model performance. The performance differences between different benchmark models are compared according to evaluation indicators including Dice Similarity Coefficient (DSC), Mean Intersection over Union (mIoU) and 95% Hausdorff Distance (HD95), and the effectiveness of the proposed PaR method is verified.

[0009] The beneficial effects of the present invention are as follows: Based on the deep learning segmentation architecture, the present invention takes into account the problem of low efficiency in utilizing the time axis (z axis) feature information caused by the difference between the isotropy of the convolution kernel inherent in the three-dimensional convolution and the anisotropy of the three-dimensional medical image data, which is mainly used in the current deep learning model for three-dimensional medical image segmentation. This leads to poor model segmentation accuracy, which affects the quality of clinical diagnosis and subsequent treatment analysis. The present invention proposes a post-decoder three-dimensional axial decoupling enhancement (PaR) method for coarse mask optimization of three-dimensional medical image segmentation. PaR analyzes the spatiotemporal features of different axial information of three-dimensional medical image data, introduces an attention mechanism to independently enhance the feature expression of axial time dimension information, thereby more accurately mapping the structural differences and detail changes on the spatiotemporal axis (z axis), and improving the accuracy and robustness of three-dimensional medical image segmentation. The module is suitable for a variety of medical image segmentation tasks such as abdominal organ segmentation, fundus retinal tissue segmentation, and cardiac system segmentation, and provides a high-precision automated segmentation tool for clinical diagnosis to assist diagnosis. As a plug-and-play deep learning module, the invention can be easily integrated into any deep learning model to improve model performance. The application of this invention can improve the accuracy of deep learning models in three-dimensional medical image segmentation, thereby improving the efficiency and effectiveness of clinical diagnosis and subsequent treatment analysis, bringing important technological advances to the field of three-dimensional medical image segmentation, and ultimately improving doctors' diagnostic efficiency and the quality of treatment plans. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 4 is an overall structural diagram of the PaR method in an embodiment of the present invention.

[0011] Figure 2 Detailed structural diagram of the permutation dimension enhancement pipeline (PDI) in an embodiment of the present invention.

[0012] Figure 3Detailed structural diagram of the sequential axially enhanced pipeline (SAI) in an embodiment of the present invention. DETAILED DESCRIPTION

[0013] The present invention is further explained below with reference to the embodiments.

[0014] like Figure 1 As shown, the present invention proposes a post-decoder 3D axial decoupling enhancement (PaR) method for 3D medical image segmentation. The method aims to enhance and optimize the rough mask of 3D medical image segmentation obtained by a deep learning model, and specifically includes the following steps:

[0015] Step 1: Collect public 3D medical image datasets corresponding to various organ segmentation tasks from the Internet, including but not limited to high-quality datasets such as the abdomen segmentation dataset FLARE2021, the fundus retina segmentation dataset OIMHS, and the heart segmentation dataset SegTHOR. Each dataset contains the original image and annotations, which can be used for subsequent training, verification, and testing of deep learning models.

[0016] Step 2: Divide the collected multi-organ 3D medical image dataset into a training set, a validation set, and a test set in a ratio of 8:1:1.

[0017] Step 3: Preprocess the 3D medical image data in step 2 to ensure the consistency and quality of the input data for better training results. This includes re-interpolating the data and interpolating the sequence length less than 108 to 108; cropping the image pixels and applying maximum-minimum normalization. The threshold for image intensity cropping is set to [-125, 275] for the FLARE2021 dataset, [0, 400] for the OIMHS dataset, and [-1000, 300] for the SegTHOR dataset. 96x96x96 is randomly cropped for training, and the sliding window (slide_windows) method is used for verification. The overlap between images is set to 0.5 during verification. Data augmentation is performed on the experimental data, including but not limited to random flipping, random rotation, random scaling, and random image intensity offset.

[0018] Step 4. Collect advanced deep learning models widely used for 3D medical image segmentation from the Internet as benchmark models, including but not limited to 3D U-Net, V-Net, RAUNet, ResUNet, SegResNet, MultiResUNet, UNETR, SwinUNETR, TransBTS, nnFormer and 3D UX-Net.

[0019] Step 5: In order to improve the performance of the benchmark deep learning model for 3D medical image segmentation, this study proposes a post-decoder 3D axial decoupling enhancement (PaR) method. First, the decoder output of the general deep learning model for segmentation is defined as a coarse mask ,in, represents the set of real numbers, represents the number of segmentation categories, , and Represent the width, height and time axis depth of the three-dimensional medical image data respectively. Compared with two-dimensional planar images, three-dimensional medical images have richer spatiotemporal and detailed texture information. Therefore, the PaR method uses three-dimensional axial decoupling to learn the feature differences of three-dimensional medical image data in the time axis plane and slice plane, thereby optimizing the effect of the coarse mask. In order to effectively capture the differences in time axis and spatial features, it adopts two parallel three-dimensional decoupling enhancement pipelines: the permutation dimension enhancement (PDI) pipeline and the sequence axial enhancement (SAI) pipeline, extracting key spatiotemporal features from different spatial perspectives, and performing three-dimensional axial decoupling from the time axis direction, thereby comprehensively refining the segmentation coarse mask. The following is a detailed introduction to the PaR method and the structure and function of the PDI pipeline and the SAI pipeline, and gives the relevant mathematical formulas.

[0020] like Figure 1 As shown in Figure 1, the outputs of the PDI pipeline and the SAI pipeline are combined by addition, and then the sigmoid function compresses the value into the range of (0,1), and then the coarse mask is used. After multiplication, the residual connection is added to the coarse mask The final optimized and enhanced segmentation mask is obtained, and the mathematical formula is as follows: in, It is the final optimized and enhanced segmentation mask. Represents the refined segmentation mask from the global perspective output by the PDI pipeline; Represents the refined segmentation mask from the temporal perspective output by the SAI pipeline.

[0021] like Figure 2 As shown, the PDI pipeline analyzes and processes the coarse mask The key time axis feature information under different spatial distributions can improve the accuracy and robustness of 3D image segmentation.

[0022] For the decoder output of a general deep learning model, the coarse mask , the PDI pipeline reconstructs the feature space distribution through a permutation operation. This operation rearranges the coarse mask Each dimension generates multiple feature maps , these feature maps are presented in different spatial configurations, such as , as well as Etc. Among them The axes are rearranged to the last dimension to enhance the attention of the subsequent attention mechanism on the time axis (z axis) information. These rearranged feature maps are merged from five-dimensional tensors to three-dimensional tensors. , , so that its format is suitable for input into the multi-head self-attention mechanism to extract and analyze the z-axis information space features. The formula of multi-head self-attention is expressed as:

[0023] in, is the attention output, Represents multi-head self-attention.

[0024] Next, the attention output Reshape into a 5-dimensional tensor through the reshape operation , and then through the permutation operation and the coarse mask The dimension of is aligned and recorded as the segmentation mask after attention weighting The last step of the PDI pipeline is to accumulate and sum these attention-weighted segmentation masks to generate the optimized segmentation mask. , thereby optimizing the coarse mask by extracting and processing the z-axis information from a global perspective .

[0025] The inherent brevity of sequence length in 3D medical images is a fundamental drawback that is usually mitigated by interpolation in the data preprocessing stage, but this also leads to significant changes in temporal information. Since most segmentation networks usually identify segmentation targets based on slice planes, they have difficulty understanding the rich spatiotemporal information across sequences. Figure 3 As shown, the SAI pipeline starts from a coarse mask Extract spatiotemporal information from the coarse mask and learn unified features across different time axis planes, enhancing the baseline model to understand the rich spatiotemporal information across sequences. Optimize to improve the accuracy and robustness of 3D image segmentation.

[0026] For coarse mask , the SAI pipeline first passes through an initial feature extraction layer, which contains 3D convolution, group normalization (GN), ReLU activation function and residual connection. This initial feature extraction layer performs preliminary feature extraction and generates an initial feature map In order to capture the timeline features through the subsequent attention mechanism, the initial feature map along The axis is split into multiple subsets The subsets are further divided into two groups: channel group and space group .

[0027] For channel groups, Through adaptive max pooling and adaptive mean pooling Extract the time axis channel information, then merge it by addition and input it into the adaptive weighting function In the above example, we generate feature maps : in, and are all trainable hyperparameters. The initial hyperparameters of are set to 1. The initial hyperparameters of are set to 0.

[0028] Then, The sigmoid function is used to limit the value to the interval (0,1) and multiply it element by element. Fusion to extract Key channel features in .

[0029] For the space group, After a 3D convolutional layer to capture the spatial features in the local neighborhood of the feature map, it reflects These features are then weighted by an adaptive weighting function Generating spatially weighted feature maps :

[0030] in, represents the dimensional convolutional layer, and They are also trainable hyperparameters, and their initial hyperparameters are set to 1 and 0 respectively.

[0031] Likewise, After activation by the sigmoid function, its value is ensured to be in the range (0,1) and The fusion method is fused by element-wise multiplication to extract Key spatial features in .

[0032] Finally, the outputs of the channel group and spatial group are concatenated and rescaled into a coarse mask The dimension of time axis decoupling feature map is obtained .

[0033] In order to further extract and utilize the spatial information from the time-decoupled feature map, the SAI pipeline introduces a spatial redistribution method to capture shared features from different spatial states. The shape is reshaped to , then Dimensions and Dimensions are exchanged between dimensions to obtain . Then resize the transformed tensor back to , and add to The final output of the SAI pipeline is the optimized segmentation mask. By supplementing spatial information, spatial redistribution enhances the ability of the benchmark model to understand the spatiotemporal dynamics and helps capture the complex spatiotemporal relationships inherent in three-dimensional medical imaging.

[0034] Step 6. Integrate the implemented PaR method into the collected benchmark models to obtain the segmentation networks 3D U-Net+PaR, V-Net+PaR, RAUNet+PaR, ResUNet+PaR, SegResNet+PaR, MultiResUNet+PaR, UNETR+PaR, SwinUNETR+PaR, TransBTS+PaR, nnFormer+PaR and 3D UX-Net+PaR.

[0035] Step 7: Input the multi-organ 3D medical image data after data preprocessing and data enhancement in step 3 into different segmentation networks for fully supervised learning. For training on all data sets, we use As the global loss function, the AdamW optimizer with an initial learning rate of 0.0001 was used, and the training was performed for 80,000 iterations. The batch size was set to 2, and a sliding window (slide_windows) was used for validation. The image overlap was set to 0.5, and the model was evaluated on the validation set every 1,000 iterations to save the best model weights. All experiments were completed under the same software and hardware conditions. All models were trained and validated on multiple computer servers equipped with 2 NVIDIA GeForce RTX 4090 GPUs and 128G memory. Our framework was implemented using Python 3.9, PyTorch 2.0.0, monai0.9.0, and the DDP distributed training framework was used for training and evaluation.

[0036] Step 8: Verify the saved best model weights on the test set to evaluate the model performance, and compare the performance differences between different benchmark models and verify the effectiveness of the proposed axial decoupling module based on evaluation indicators including Dice Similarity Coefficient (DSC), Mean Intersection over Union (mIoU) and 95% Hausdorff Distance (HD95). The mean intersection over union can effectively reflect the overall accuracy of multi-category segmentation tasks, the Dice similarity coefficient performs better for small target areas in medical image segmentation, and the 95% Hausdorff distance is used to measure the accuracy of segmentation boundaries and is more sensitive to boundary detail errors.

[0037] The present invention has conducted extensive and comprehensive benchmark model comparison experiments on publicly available high-quality 3D medical image segmentation datasets FLARE2021, OIMHS, and SegTHOR. The experimental results are shown in Tables 1, 2, and 3. From the indicator results in the table, it can be seen that the PaR method proposed in the present invention can help all 11 advanced benchmark segmentation models collected to improve the segmentation performance. Therefore, it can be verified that PaR, as a plug-and-play module, has a wide range of application value in 3D medical image segmentation tasks.

[0038] Table 1 Benchmark experimental results of FLARE2021 dataset Method IOU Dice HD95 3D U-Net 87.92 93.08 16.31 +PaR 89.20+1.28 93.89+0.81 2.49-13.82 RAUNet 87.93 93.08 27.37 +PaR 88.32+0.39 93.28+0.20 26.71-0.66 ResUNet 87.38 92.56 30.15 +PaR 87.83+0.45 92.94+0.38 11.80-18.35 SegResNet 86.24 91.81 3.22 +PaR 88.02+1.78 93.13+1.32 2.78-0.44 V-Net 84.13 89.89 12.93 +PaR 85.99+1.86 91.49+1.60 5.98-6.95 UNETR 84.74 90.70 4.63 +PaR 85.87+1.13 91.48+0.78 3.64-0.99 Swin UNETR 88.28 93.23 3.25 +PaR 89.47+1.19 94.04+0.81 2.61-0.64 nnFormer 85.50 91.43 5.41 +PaR 88.83+3.33 93.69+2.26 2.35-3.06 TransBTS 87.63 92.84 3.54 +PaR 88.36+0.73 93.27+0.43 2.90-0.64 MultiResUNet 85.92 91.35 9.04 +PaR 86.76+0.84 91.85+0.50 3.68-5.36 3D UX-NET 88.40 93.31 8.85 +PaR 89.24+0.84 93.84+0.53 2.43-6.42 The present invention tests the performance of 11 benchmark models after integrating PaR on the FLARE dataset, and the results are shown in Table 1. As can be seen from Table 1, the performance of all basic models has been improved after integrating PaR to decouple the axial information of the coarse mask and enhance it separately. Performance indicators including average intersection-over-union ratio and Dice similarity coefficient show that the PaR method improves the accuracy of the overall segmentation performance of the model, while the 95% Hausdorff distance shows that the PaR method improves the reliability of the model boundary performance. After PaR, the average intersection-over-union ratio and Dice similarity coefficient indicators of all benchmark models have increased by 0.39% to 3.33% and 0.2% to 2.26% respectively, which reflects that the multi-head attention of the PDI pipeline to global information enhances the model's ability to analyze the overall spatial features, and can extract the most important information from complex spatial features, which improves the overall segmentation performance of the model. The integrated PaR also significantly improves the boundary segmentation performance of the baseline model. The 95% Hausdorff distance index of U-Net drops from 16.31 to 2.49, which shows that the SAI pipeline not only learns and understands the axial spatial information, but also corrects the obvious errors of independent segmentation.

[0039] Table 2. Benchmark experimental results of OIMHS dataset Method IOU Dice HD95 3D U-Net 86.60 92.49 3.40 +PaR 87.55+0.95 93.08+0.59 2.89-0.51 RAUNet 84.52 91.14 13.61 +PaR 86.20+1.68 92.25+1.11 5.37-8.24 ResUNet 84.06 90.84 3.92 +PaR 87.07+3.01 92.79+1.95 3.33-0.59 SegResNet 83.59 90.52 12.05 +PaR 84.82+1.23 91.35+0.83 5.07-6.98 V-Net 81.00 88.53 18.13 +PaR 83.21+2.21 90.26+1.73 13.52-4.61 UNETR 81.52 89.05 29.15 +PaR 83.73+2.21 90.55+1.50 7.24-21.91 Swin UNETR 87.11 92.82 5.21 +PaR 87.92+0.81 93.27+0.45 2.92-2.29 nnFormer 80.54 88.29 25.32 +PaR 85.50+4.96 91.80+3.51 7.36-17.96 TransBTS 79.40 87.39 33.52 +PaR 83.68+4.28 90.55+3.16 21.00-12.52 MultiResUNet 86.53 92.44 3.23 +PaR 88.22+1.69 93.49+1.05 2.85-0.38 3D UX-NET 87.45 93.01 4.61 +PaR 88.51+1.06 93.66+0.65 2.65-1.96 The performance on the OIMHS dataset is shown in Table 2. As can be seen from the table, the performance of all the benchmark models has been steadily improved after integrating PaR to decouple and re-enhance the axial features. After integrating PaR, the average intersection-over-union ratio and Dice similarity coefficient of the benchmark models have increased by 0.81% to 4.96% and 0.45% to 3.51%, respectively. After integrating PaR, the 95% Hausdorff distance index of UNETR has dropped from 29.15 to 7.24, reflecting the boundary correction function that comes with PaR's centralized decoupling of axial information. The PaR method still has a good improvement effect on the latest SOAT model. When the original segmentation mask of 3DUXNET was relatively high-quality (Dice similarity coefficient index of 93.01), it still achieved an average intersection-over-union ratio improvement of 1.06% and a Dice similarity coefficient improvement of 0.65% after integrating PaR. This shows that PaR's way of learning and extracting features from the segmentation map from the perspective of axial space is different from the idea of ​​analyzing and extracting features of conventional three-dimensional image segmentation models. The ability to improve performance can surpass the boundaries of traditional three-dimensional convolution, so it can still achieve certain improvements on the field SOTA model.

[0040] Table 3. Benchmark experimental results on the SegTHOR dataset Method IOU Dice HD95 3D U-Net 78.69 87.59 4.32 +PaR 82.08+3.39 89.82+2.23 2.98-1.34 RAUNet 79.34 88.13 14.58 +PaR 81.04+1.70 89.08+0.95 2.99-11.59 ResUNet 79.61 88.26 3.22 +PaR 80.23+0.62 88.71+0.45 3.14-0.08 SegResNet 77.78 86.99 3.38 +PaR 81.77+3.99 89.60+2.61 2.87-0.51 V-Net 75.17 85.12 16.10 +PaR 76.23+1.06 85.81+0.69 11.49-4.61 UNETR 73.76 84.03 4.71 +PaR 74.13+0.37 84.18+0.15 4.65-0.06 Swin UNETR 78.19 87.26 3.87 +PaR 78.51+0.32 87.42+0.16 3.62-0.25 nnFormer 77.27 86.65 5.11 +PaR 79.00+1.73 87.69+1.04 3.51-1.60 TransBTS 77.70 86.88 3.84 +PaR 81.02+3.32 89.13+2.25 3.75-0.09 MultiResUNet 79.87 88.53 26.75 +PaR 80.81+0.94 89.10+0.57 11.06-15.69 3D UX-NET 78.30 87.34 4.69 +PaR 79.01+0.71 87.77+0.43 3.61-1.08 The effect of PaR on axial feature decoupling enhancement on the SegThOR dataset is shown in Table 3. The performance of the basic model is steadily improved after integrating PaR. Among all the benchmark models, in terms of overall segmentation performance, the average intersection-over-union ratio and Dice similarity coefficient indicators are improved by 0.32% to 3.99% and 0.15% to 2.61% after integrating PaR. The boundary correction effect of three-dimensional axial decoupling enhancement can also be observed on the SegThOR dataset. After integrating PaR, the 95% Hausdorff distance indicator of RAUNet dropped from 14.58 to 2.99.

[0041] The experimental results on three public datasets, FLARE, OIMHS and SegTHOR, show that connecting PaR after the decoder of the baseline model to perform axial information decoupling and independent enhancement can achieve stable performance improvement and self-correct the results of models with poor performance. This improvement can break through the upper limit of traditional convolutional structure design and still provide performance gains in the SOTA three-dimensional segmentation method, proving the effectiveness and reliability of three-dimensional axial decoupling for three-dimensional medical image data analysis.

[0042] Table 4 Ablation experiment 1 Method mIoU Dice HD95 UNETR 81.52 89.05 29.15 SAI 81.88 89.21 13.35 PDI 82.88 89.98 22.59 PDR-cat 82.61 89.77 16.87 PDR-mult 82.07 89.42 27.31 PDR-add 83.73 90.55 7.24 Table 5 Ablation experiment 2 Method mIoU Dice HD95 UNet 87.92 93.08 16.31 SAI 88.45 93.49 20.33 PDI 88.92 93.73 10.72 PDR-cat 88.67 93.56 8.24 PDR-mult 88.58 93.48 13.37 PDR-add 89.20 93.89 2.49 In order to verify the performance difference of the proposed method on the traditional convolutional structure and Transformer structure, we used U-Net and UNETR as the base models to conduct ablation experiments on the key pipeline components and pipeline fusion methods of PaR on the FLARE2021 dataset and OIMHS dataset, as shown in Tables 4 and 5. When only the PDI pipeline or SAI pipeline is integrated, the model performance is improved. For the selection of the fusion method of the PDI pipeline and the SAI pipeline in the post-PaR method, we experimented with three fusion methods: addition, matrix element-by-element multiplication, and concentrate. The best effect was achieved when addition was used, and all indicators were improved, indicating the structural complementarity of the two key pipelines. The degree of improvement of the PDI pipeline or SAI pipeline on different datasets and different models is not the same, but after using PaR combining the two methods, the model surpasses all basic models in segmentation accuracy and boundary refinement, which proves that decoupling the axial information of 3D image data and using feature learning enhancement strategies such as attention mechanism and information redistribution can produce certain performance gains.

[0043] In summary, the PaR method proposed in the present invention extracts feature information by decoupling analysis of different axial spatial state distributions from a global perspective, spatially decouples the coarse segmentation mask of the deep learning model from the axial depth perspective, and learns and enhances the information distribution characteristics unique to the axial space, which significantly improves the segmentation accuracy of the three-dimensional medical image segmentation benchmark model and has broad application prospects.

[0044] The above contents are only preferred implementation modes of the present invention and do not limit the protection scope of the present invention. It should be emphasized that any minor adjustments and improvements made by technicians in this field without departing from the basic principles of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A post-decoder 3D axial decoupling enhancement method for 3D medical image segmentation, characterized in that: The method comprises the following steps: S1: Collect 3D medical image segmentation datasets corresponding to various organ segmentation tasks from the Internet; S2: Divide the collected 3D medical image data into a training set and a test set in proportion; S3: performing data preprocessing and data enhancement on the three-dimensional medical image data in step S2; S4: Collect advanced deep learning models widely used for 3D medical image segmentation from the Internet as benchmark models; S5: Design a post-decoder coarse mask volume axial decoupling enhancement module that can be used for coarse mask optimization in various 3D medical image segmentation tasks; the designed module includes two main pipelines: a permutation dimension enhancement pipeline and a sequence axial enhancement pipeline; S6: Integrate the PaR method into the collected benchmark models; S7: Use the baseline model and the baseline model integrated with PaR on the preprocessed and data augmented datasets to perform model training and save the respective best model weights according to the evaluation results on the validation set; S8: Validate the saved best model weights on the test set to evaluate model performance.

2. The post-decoder three-dimensional axial decoupling enhancement method for three-dimensional medical image segmentation according to claim 1, characterized in that: The post-decoder coarse mask volume axial decoupling enhancement module adopts two parallel 3D decoupling enhancement pipelines: the permutation dimension enhancement pipeline PDI and the sequence axial enhancement pipeline SAI, which extract key spatiotemporal features from different spatial perspectives and perform 3D axial decoupling from the temporal axis direction, thereby comprehensively refining the segmentation coarse mask; the permutation dimension enhancement pipeline learns and understands different axial features from the perspective of global information, while the sequence axial enhancement pipeline directly analyzes and enhances the temporal axial features from the perspective of axial spatial information; The outputs of the permutation dimension enhancement pipeline and the sequence axis enhancement pipeline are combined by addition, and then the values ​​are compressed to the range of (0,1) by the sigmoid function, and then combined with the coarse mask After multiplication, it is added to the coarse mask by residual connection The final optimized and enhanced segmentation mask is obtained, and the formula is as follows: ； in is the final optimized and enhanced segmentation map. Represents the refined segmentation mask from the global perspective output by the PDI pipeline; Represents the refined segmentation mask from the temporal perspective output by the SAI pipeline.

3. The post-decoder three-dimensional axial decoupling enhancement method for three-dimensional medical image segmentation according to claim 1 or 2, characterized in that: First, define the decoder output of the deep learning model for segmentation as a coarse mask ,in, represents the set of real numbers, represents the number of segmentation categories, , and Respectively represent the width, height and time axis depth of the three-dimensional medical image data; the PDI pipeline analyzes and processes the coarse mask The key time axis feature information under different spatial distributions is as follows: Coarse mask for decoder output of deep learning model , the PDI pipeline reconstructs the feature space distribution through a permutation operation; this operation rearranges the coarse mask Each dimension generates multiple feature maps ,in The axes are rearranged to the last dimension to enhance the subsequent attention mechanism's attention to the time axis information; The rearranged feature maps are merged from a five-dimensional tensor into a three-dimensional tensor , , so that its format is suitable for input into the multi-head self-attention mechanism to extract the information space features of the analysis time axis z axis. The formula of multi-head self-attention is expressed as: ； in, is the attention output, represents multi-head self-attention; Attention Output Reshape into a 5-dimensional tensor through the reshape operation , and then through the permutation operation and the coarse mask The dimension of is aligned and recorded as the segmentation mask after attention weighting ; Finally, the PDI pipeline converts the attention-weighted segmentation mask Perform cumulative summation to generate an optimized segmentation mask , thereby optimizing the coarse mask by extracting and processing the z-axis information from a global perspective .

4. The post-decoder three-dimensional axial decoupling enhancement method for three-dimensional medical image segmentation according to claim 3, characterized in that: The SAI pipeline starts from a coarse mask Extract spatiotemporal information from the coarse mask and learn unified features across different time axis planes, enhancing the baseline model to understand the rich spatiotemporal information across sequences. Optimize as follows: For coarse mask , the SAI pipeline first passes through an initial feature extraction layer, which contains three-dimensional convolution, group normalization, ReLU activation function and residual connection; the initial feature extraction layer performs preliminary feature extraction and generates an initial feature map , the initial feature map along The axis is split into multiple subsets , the subsets are further divided into two groups: channel group and space group , extract key channel features from the channel group, and extract key spatial features from the spatial group; The key channel features and key spatial features are concatenated and reshaped into a coarse mask The dimension of time axis decoupling feature map is obtained .

5. The post-decoder 3D axial decoupling enhancement method for 3D medical image segmentation according to claim 4, characterized in that: The key channel features of the channel group are obtained as follows: First, the channel group Through adaptive max pooling and adaptive mean pooling Extract the time axis channel information, then merge it by addition and input it into the adaptive weighting function In the above example, we generate feature maps : in, and are all trainable hyperparameters. The initial hyperparameters of are set to 1. The initial hyperparameters of are set to 0; Then, the feature map The sigmoid function is used to limit the value to the interval (0,1) and multiply the channel group by element. Fusion to extract Key channel features in .

6. The post-decoder three-dimensional axial decoupling enhancement method for three-dimensional medical image segmentation according to claim 4 or 5, characterized in that: The key spatial features of the space group are obtained as follows: First, the space group After a 3D convolutional layer to capture the spatial features in the local neighborhood of the feature map, it reflects the importance of different spatial regions in the These features are then weighted by an adaptive weighting function Generating spatially weighted feature maps : ； in, represents the dimensional convolutional layer, and They are also trainable hyperparameters, and their initial hyperparameters are set to 1 and 0 respectively; Then, the spatially weighted feature map After sigmoid function activation, ensure that its value is in the range (0,1), and The fusion method is fused by element-wise multiplication to extract Key spatial features in .

7. The post-decoder 3D axial decoupling enhancement method for 3D medical image segmentation according to claim 6, characterized in that: The SAI pipeline introduces a spatial redistribution method to capture shared features from different spatial states. The specific implementation is as follows: First, decouple the time axis from the feature map The shape is reshaped to , then Dimensions and Dimensions are exchanged between dimensions to obtain ; Then reshape the transformed tensor back to , and added to the time axis decoupling feature map The final output of the SAI pipeline is the optimized segmentation mask. .

8. The post-decoder 3D axial decoupling enhancement method for 3D medical image segmentation according to claim 1, characterized in that: The benchmark models include but are not limited to 3D U-Net, V-Net, RAUNet, ResUNet, SegResNet, MultiResUNet, UNETR, SwinUNETR, TransBTS, nnFormer and 3D UX-Net.

9. The post-decoder 3D axial decoupling enhancement method for 3D medical image segmentation according to claim 1, characterized in that: Data preprocessing in step 2 includes re-interpolation, interpolating sequence lengths less than 108 to 108; performing image intensity cropping on image pixels and applying maximum-minimum normalization. The threshold for image intensity cropping is set to [-125, 275] for the FLARE2021 dataset, [0, 400] for the OIMHS dataset, and [-1000, 300] for the SegTHOR dataset. 96x96x96 is randomly cropped for training and verified using the sliding window method. The overlap between images is set to 0.5 during verification. Data augmentation is performed on the experimental data, including but not limited to random flipping, random rotation, random scaling, and random image intensity offset.

10. The post-decoder 3D axial decoupling enhancement method for 3D medical image segmentation according to claim 8, characterized in that: Step 6 integrates the implemented PaR into the collected benchmark models to obtain the segmentation networks 3D U-Net+PaR, V-Net+PaR, RAUNet+PaR, ResUNet+PaR, SegResNet+PaR, MultiResUNet+PaR, UNETR+PaR, SwinUNETR+PaR, TransBTS+PaR, nnFormer+PaR and 3D UX-Net+PaR.

Images