Implementation of multi-aspect segmentation network for visible light medical image segmentation

Abstract

The present invention discloses a multi-aspect segmentation network implementation method for visible light medical image segmentation. The implementation of the present invention includes the following four high-level modules: the spectral spline encoder uses multi-frequency modulation and channel attention mechanism for noise suppression; the hierarchical morphological encoder captures complex geometric structures through deformable convolution; the semantic context encoder enhances semantic understanding through context analysis; the multi-attention KAN integrator integrates different types of features through multiple self-attention mechanisms and feature fusion. Through the independent processing and collaborative work of the above three encoders, the present invention can achieve more accurate and robust segmentation effects in complex medical imaging scenes, demonstrating its wide application potential in the field of medical image segmentation.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and in particular to a method for implementing a multi-aspect segmentation network for visible light medical image segmentation, the method comprising a multi-aspect segmentation network (MAS-Net) for visible light medical image segmentation. MAS-Net significantly improves the accuracy and robustness of image segmentation by combining frequency domain feature processing, geometric deformation analysis, and semantic context analysis. The network is suitable for a variety of medical image processing tasks such as skin cancer detection and endoscopic image analysis, and provides a high-precision image segmentation tool for clinical diagnosis and treatment. Background Art

[0002] In the field of medical imaging, accurate image segmentation is crucial for diagnosis and treatment. For example, in skin cancer detection, precise lesion segmentation can help doctors assess the condition more accurately; in endoscopic image analysis, clear tissue segmentation helps surgeons better identify and protect important anatomical structures during surgery. However, current image segmentation technology still has significant shortcomings when dealing with complex backgrounds, noise interference, and diverse geometric forms.

[0003] Existing segmentation models based on U-shaped encoder-decoder architecture, such as U-Net and its variants, perform well in certain specific tasks, but their segmentation accuracy and robustness still need to be improved when faced with complex medical images. Traditional frequency domain processing methods, such as Fast Fourier Transform (FFT), can extract frequency domain information of images to a certain extent, but they perform poorly in understanding complex geometric structures and semantic information. Although methods based on graph convolutional networks can handle irregular geometric shapes, they are not satisfactory when faced with high-noise environments in endoscopic images.

[0004] In order to solve the above problems, the present invention proposes a multi-aspect segmentation network (MAS-Net), which realizes high-precision segmentation of medical images by combining frequency domain feature processing, geometric deformation analysis and semantic context analysis. The three-branch encoder architecture of MAS-Net (including spectral spline encoder, hierarchical morphological encoder and semantic context encoder) can independently process and fuse frequency domain features, complex geometric structures and semantic information, thereby significantly improving the accuracy and robustness of image segmentation. Summary of the invention

[0005] In the field of medical image processing, high-precision image segmentation is essential for accurate medical diagnosis and effective therapeutic intervention, involving key tasks such as tumor detection and tissue classification. However, existing methods have limitations in dealing with noise, geometric complexity, and semantic ambiguity, resulting in poor performance in different visible light medical imaging scenarios.

[0006] The present invention proposes a multi-aspect segmentation network (MAS-Net) to solve the above problems in the prior art. Based on a robust basic model, MAS-Net improves the performance of segmentation tasks through complex feature processing techniques. The network includes the following four high-level modules: Spectral Spline Encoder (SSE) uses multi-frequency modulation and channel attention mechanism for noise suppression; Hierarchical Morphology Encoder (HDME) captures complex geometric structures through deformable convolution; Semantic Context Encoder (SCE) enhances semantic understanding through context analysis; Multi-Attention Kolmogorov-Arnold Network Integrator (MAKI): integrates different types of features through multiple self-attention mechanisms and feature fusion. Through the independent processing and collaborative work of the above three encoders, the present invention can achieve more accurate and robust segmentation effects in complex medical imaging scenes, demonstrating its wide application potential in the field of medical image segmentation.

[0007] The technical solution included in the present invention to solve the technical problem includes the following steps:

[0008] S1: Image data are collected from multiple public medical image datasets.

[0009] S2: Organize and divide the collected data into training set and test set;

[0010] S3: preprocessing the image data in step S2;

[0011] S4: Apply data enhancement technology to the image in step S2;

[0012] S5: Build a multi-aspect segmentation network for visible light medical image segmentation;

[0013] S6: Use the preprocessed data set to train the multi-aspect segmentation network model and obtain weights;

[0014] S7: Evaluate the model performance with the trained weights on the test set, and adjust the model parameters and architecture according to indicators including but not limited to Dice Similarity Coefficient (DSC), Mean Intersection over Union (mIoU), Accuracy (ACC), and 95% Hausdorff Distance (HD95) to optimize the model performance.

[0015] Beneficial effects of the present invention

[0016] Based on the deep learning framework, the present invention takes into account the main problems currently faced by medical image processing, including noise interference, geometric complexity and semantic ambiguity, which lead to low segmentation accuracy and affect the diagnosis and treatment effects. The multi-aspect segmentation network proposed in the present invention significantly improves the accuracy and robustness of medical image segmentation by integrating multi-frequency feature modulation, deformation convolution and context analysis. Its innovative multi-dimensional feature fusion technology enhances the recognition ability of complex geometric structures and semantic information, and improves the generalization ability and applicability of the model through pre-training and refinement training. The application of this invention not only improves the accuracy of medical image segmentation, but also significantly improves the efficiency and effectiveness of clinical diagnosis and treatment, bringing important technological advances to the field of medical imaging, and ultimately improving patients' medical experience and treatment outcomes. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is an overall architecture diagram of a multi-aspect segmentation network framework in an embodiment of the present invention.

[0018] Figure 2 It is a detailed structural diagram of a three-branch encoder in an embodiment of the present invention.

[0019] Figure 3 It is a detailed structural diagram of the multi-head attention KAN integrator in an embodiment of the present invention.

[0020] Figure 4 It is a detailed structural diagram of the multi-scale channel attention mechanism in an embodiment of the present invention.

[0021] Figure 5 It is a detailed structural diagram of iterative attention feature fusion in an embodiment of the present invention.

[0022] Figure 6 It is a detailed structural diagram of the multi-frequency characteristic modulator in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The present invention is further explained below with reference to the embodiments.

[0024] like Figure 1 As shown, the present invention proposes a multi-aspect segmentation network method. It aims to segment visible light medical images, which specifically includes the following steps:

[0025] Step 1: Collect image data from multiple public medical image datasets: ISIC2017, ISIC2018, PH2, CVC-ClinicDB, Kvasir-SEG, and PolypGen. Each dataset contains training, validation, and test sets, covering different types of medical images including skin cancer detection and colonoscopy images.

[0026] Step 2: Organize and divide the collected data into training set and test set.

[0027] Step 3: Preprocess the image data in step S2 to ensure the consistency and quality of the input data, including adjusting all images to a uniform size to meet the neural network input requirements. And normalize the image pixel values ​​to the range of [0,1] to accelerate convergence. Then remove blurred, incomplete or incorrectly labeled images.

[0028] Step 4: Apply data augmentation techniques to the image data: randomly scale the image and crop it to the target size, randomly flip the image horizontally, and adjust the brightness, contrast, and saturation of the image.

[0029] Step 5. In order to achieve high-precision medical image segmentation, this paper proposes a multi-aspect segmentation network (MAS-Net), which includes three parallel encoder paths: spectral spline encoder (SSE), hierarchical morphological encoder (HDME) and semantic context encoder (SCE). The following details the structure and function of each encoder and gives the relevant mathematical formulas.

[0030] like Figure 2 As shown, the spectral spline encoder is mainly used to enhance noise suppression and feature extraction capabilities in the frequency domain, including a multi-frequency feature regulator and a KAN channel attention mechanism, which converts the image from the spatial domain to the frequency domain through a fast Fourier transform for more accurate filtering and noise compensation; Figure 6 As shown in the figure, the multi-frequency feature modulator modulates the frequency domain information of the original image and performs weighted processing to obtain the modulated convolution features:

[0031]

[0032]

[0033] in, is the modulated convolution feature, is the original image, is a low-pass filtering operation, is the frequency domain information of the original image after filtering. is a learnable weight matrix, is the set of real numbers, is the number of channels, is the image width, is the image height; then perform fast Fourier transform on the convolved features:

[0034]

[0035]

[0036] in, It is the frequency domain feature obtained by fast Fourier transform. is the convolution feature of the input, is the set of real numbers, represents the fast Fourier transform, Represents convolution; then the frequency domain features after modulation are restored to the spatial domain through inverse fast Fourier transform to achieve the fusion of frequency domain and spatial domain:

[0037]

[0038] in, is the feature after inverse Fourier transformation, are the learnable coefficients, Represents inverse fast Fourier transform; the final image features are obtained by integrating the frequency domain information of the modulated image with the frequency domain information of the original spatial domain image:

[0039]

[0040] in, It is the image feature output by the multi-frequency feature modulation module.

[0041] like Figure 4 As shown in Figure 2, the KAN channel attention mechanism introduces an adaptive adjustment strategy based on the KAN layer to dynamically optimize the response ability of each channel through adaptive channel weight adjustment. For the calculation of channel attention, the image features are first The feature map of each channel is globally averaged pooled:

[0042]

[0043] in, is the global feature vector after pooling, is the feature map of each channel, and Represent the height and width of the feature map respectively, Indicates the position on the feature map The pixel value at ; then the channel attention weight is calculated through the KAN layer:

[0044]

[0045] in, is the channel attention weight, is a learnable activation function, It is a Kolmogorov-Arnold network; applying attention weights to channel features:

[0046]

[0047] in, It is the channel feature of the final output after weighting.

[0048] The hierarchical morphological encoder generates multi-scale feature representations through progressive convolution and pooling operations, covering structural details at different resolutions and providing a comprehensive understanding of the main structure. In order to enhance the understanding of complex geometric structures, the hierarchical morphological encoder introduces a variable Laplace transform module, which dynamically adjusts the shape and position of the convolution kernel by calculating the geometric offset of the feature map, so that the convolution operation can adaptively parse the complex irregular morphology in the image. The geometric offset calculation formula is as follows:

[0049]

[0050] in, represents the geometric offset of the convolution kernel, and Respectively represent The weights and biases of the operations, is the total number of operations; then through multi-layer convolution and pooling operations, feature maps of different scales are extracted and weighted addition is performed according to the weights to obtain multi-scale features:

[0051]

[0052] in, is a multi-scale feature, Indicates The weights of the layer convolution, Representative The feature map extracted by the layer, is the total number of convolutional layers.

[0053] The semantic context encoder has the global context perception ability of a large visual model, which can effectively capture and integrate the semantic information of the image at multiple scales, and dynamically capture long-distance dependencies in the global scope of the image; first, local features are extracted through a 16×16 window attention mechanism, and then these features are integrated with the global context through multiple global attention blocks to form a complete semantic representation:

[0054]

[0055] in, is the semantic context feature, is the original output image, is the window of attention, It is global attention.

[0056] like Figure 3 As shown in FIG, the multi-attention KAN integrator processes the global frequency domain features, geometric morphological features and semantic features in parallel, and fuses them through the designed KAN multiple self-attention mechanism and iterative attention feature fusion mechanism, as shown in FIG. Figure 5 As shown;

[0057]

[0058] in, is the global frequency domain feature, It is a two-dimensional convolution;

[0059] Multi-scale features are transformed and enhanced through double-layer convolution and PixelShuffle. The convolution layer is responsible for preliminary feature extraction, while the PixelShuffle technology is used for enhancement and upsampling:

[0060]

[0061] in, It is a geometric feature. is the activation function, It is a multi-scale feature output by the hierarchical morphological encoder; this processing method effectively captures the complex geometric structure in the image, making the feature more adaptable;

[0062] The semantic features are transformed through double-layer convolution and feature enhancement is performed to strengthen the semantic expression of the features:

[0063]

[0064] in, is a semantic feature, is the feature output by the semantic context encoder.

[0065] Furthermore, various features are optimized through KAN multiple self-attention mechanism to achieve dynamic interaction. First, the feature branch is processed by block-level self-attention to calculate the query, key and value (Q, K, V) to obtain the interdependence between features. Then, the KAN layer performs feature transformation on the self-attention output to enhance its representation ability:

[0066]

[0067] in, is the transformed feature, is the feature after self-attention; finally, the features of each branch are merged along the channel dimension and restored to the original number of channels through the convolution layer:

[0068]

[0069] in, is the adjusted output, They are the features of each branch after KAN processing, It is a concatenation operation; the iterative attention fusion module combines local and global attention mechanisms to achieve feature integration through weighted fusion:

[0070]

[0071] in, is the fused feature, are the learnable coefficients, is the feature of each branch output, Represents element-wise multiplication.

[0072] Step 6: Import the training set divided in step 2 and its annotated images as training data into the main function for supervised learning. The framework is implemented using PyTorch and trained on dual NVIDIA 4090 GPUs with specific hyperparameters and data augmentation techniques. The performance of the model is evaluated on six benchmark datasets: ISIC2017, ISIC2018, PH2, CVC-ClinicDB, Kvasir-SEG, and Polypgen.

[0073] Step 7. Apply the trained weights to the test set to evaluate the segmentation effect. We take the main and key evaluation indicators to evaluate the model, including Dice Similarity Coefficient (DSC), Mean Intersection over Union (mIoU), Accuracy (ACC), and 95% Hausdorff Distance (HD95). The Dice similarity coefficient is a statistical indicator used to quantify the similarity between two sample sets, especially widely used in the field of image segmentation. Its calculation formula is as follows:

[0074]

[0075] in, is the set of predicted binary segmentation results, The actual binary segmentation result set. is the number of elements in the intersection of the predicted result and the actual result. The Dice coefficient ranges from 0 to 1, with a value of 1 indicating complete overlap and a value of 0 indicating no overlap. A higher DSC value indicates a larger overlap area between the predicted result and the actual result, which means that the model has a higher accuracy in capturing the target area.

[0076] Average intersection-over-union is a standard metric for evaluating the performance of image segmentation models. The formula is as follows:

[0077]

[0078] in, is the set of predicted binary segmentation results, The actual binary segmentation result set. is the number of elements in the intersection of the predicted result and the actual result. It is the number of elements in the union of the predicted result and the actual result. In multi-class segmentation tasks, mIoU is the average of the IoUs of all categories. A higher mIoU value indicates that the segmentation performance of the model on different categories is consistent and accurate, reflecting the robustness and generalization ability of the model.

[0079] Accuracy is an indicator used to evaluate the overall prediction accuracy of the model and is defined as follows:

[0080]

[0081] in: (True Positive): True positive, predicted to be positive and actually positive. (True Negative): True negative example, predicted to be negative and actually negative. (False Positive): False positive examples are predicted to be positive but are actually negative. (False Negative): False negative examples are predicted to be negative but are actually positive. In medical image segmentation, a higher ACC value indicates that the model can accurately distinguish between the target area and the background area, which is of great significance for clinical diagnosis and treatment planning.

[0082] The Hausdorff distance is a measure of the maximum distance between two sets, while the 95% Hausdorff distance calculates the 95th percentile of the distance between two boundaries to reduce the impact of outliers. and , the Hausdorff distance is defined as:

[0083]

[0084] in: Indicate point and Point The Euclidean distance between . represents the supremum (i.e., maximum value). Represents the infimum (i.e., the minimum value). The 95% Hausdorff distance is the 95th percentile of all Hausdorff distances. A lower HD95 value indicates a smaller maximum error between the predicted boundary and the actual boundary, indicating that the model can accurately capture the shape and boundary of the target, thereby improving the credibility and clinical applicability of the segmentation results.

[0085] According to the index results, it can be seen that the multi-aspect segmentation network framework of the present invention has reached the highest level in all indicators of almost all categories. It can be concluded that the multi-aspect segmentation network is extremely competitive in visible light medical image segmentation.

[0086] Table 1 Comparison of experimental results on the ISIC 2017 dataset

[0087] method mIoU(%) DSC(%) ACC(%) UNet 76.98 86.99 - UTNetV2 77.35 87.23 - TransFuse 79.21 88.4 - MobileViTv2 78.72 88.09 - MobileNetv3 77.69 87.44 - UNeXt-S 78.26 87.8 - MALUNet 78.78 88.13 - EGE-UNet 79.81 88.77 - Mostafa et al. 74.1 83 88.5 Mostafa et al.*+proposed 76.4 85 90.9 Team Yuan 76.5 84.9 93.4 Team Berseth 76.2 84.7 93.2 CE-Net 77.54 85.61 93.51 ASCU-Net 74.2 83 92.6 FAC-Net 74.27 84.91 93.63 MS-Red* 76.32 84.83 93.1 FAT-Net* 76.92 85.01 93.64 autoSMIM 77.87 85.72 93.91 MAS-NET 83.37 90.55 95.38

[0088] The quantitative results of the ISIC2017 dataset are shown in Table 1. MASNET achieves a DSC of 90.55% and an mIoU of 83.37%, which is superior to several existing methods. It is worth noting that MAS-NET is 1.78% higher than EGE-UNet in DSC and 3.56% higher than EGE-UNet in mIoU. Its ACC is also better than EGE-UNet, reaching 95.38%.

[0089] Table 2 Comparison of experimental results on the ISIC 2018 dataset

[0090] Model MDice MIoU HD95 UTNetV2 88.25 78.97 - SANet 88.59 79.52 - MobileViTv2 88.81 79.88 - MobileNetv3 87.98 78.55 - UNeXt-S 88.33 79.09 - MALUNet 89.04 80.25 - EGE-UNet 89.46 80.94 - CE-NET 89.59 82.82 - DO-NET 89.48 82.61 - FAC-NET 91.19 83.99 - BAT 91.2 - - MS-Red 88.95 82.16 - FAT-Net 88.78 81.99 - Unet 88.52 81.99 13.4783 Unet++ 88.9 81.19 14.4803 Unet3+ 87.36 76.27 16.2877 Deeplabv3+ 87.66 80.2 13.9569 Double-Unet 88.9 80.01 15.332 HRNetv2-w18-sv2 85.69 78.72 16.535 PraNet 86.6 80.6 12.763 UACANet-S 89 81.37 13.2488 TransUnet 89 80.64 14.5543 Transfuse 82.46 82.46 13.5575 MSRF-Net 88 80.75 13.7399 SSFormer-S 87.98 83.17 11.679 MedT 87.36 80.83 15.0448 DCSAU-Net 89.04 81.37 14.8347 Atten-Unet 88.44 84.03 15.3676 MRUnet 87.06 82.62 12.9289 SwinUnet 88.39 82.62 12.6434 UCTransNet 88.44 82.07 14.7628 TGANet 89.34 82.18 12.5765 PVT-Cascade 90.02 83.16 12.78 ERSUnet 90.02 83.16 12.78 ERDUnet 90.75 84.13 11.2009 MAS-NET 91.3 84.67 6.9067

[0091] Table 2 shows the quantitative results of the ISIC2018 dataset. MAS-NET focuses on mIoU, DSC, and HD95 indicators, and achieves 91.3% mIoU, 84.67% DSC, and 6.91 HD95, outperforming various methods. MAS-NET exceeds BAT by 0.1% in DSC, EDRUnet by 0.54% in mIoU, and has the lowest HD95, indicating that the boundary delineation accuracy has improved.

[0092] Table 3 Comparison of experimental results on PH2 dataset

[0093] method IoU Dice HD95 U-Net (Ronneberger et al., 2015) 81.60 ± 0.68 88.16 ± 0.41 - AttU-Net (Oktay et al., 2018) 82.30 ± 0.45 89.76 ± 0.45 - Res-U-Net (Xiao et al., 2018) 82.24 ± 0.64 89.66 ± 0.64 - U-Net++ (Zhou et al., 2018) 82.98 ± 0.39 90.29 ± 0.39 - Deeplabv3+ (Chen et al., 2018) 83.70 ± 0.51 90.62 ± 0.51 - DenseASPP (Yang et al., 2018) 83.07 ± 0.34 90.37 ± 0.34 - BCDU-Net (Azad et al., 2019) 81.05 ± 0.60 88.91 ± 0.60 - CE-Net (Gu et al., 2019) 80.34 ± 0.42 90.34 ± 0.42 - nnU-Net (Isensee et al., 2021) 83.58 ± 0.43 90.56 ± 0.43 - DO-Net (Wang et al., 2020b) 83.28 ± 0.42 90.40 ± 0.38 - CPF-Net (Feng et al., 2020) 83.05 ± 0.23 90.02 ± 0.23 - HMRNet (Fu et al., 2022) 83.67 ± 0.60 90.61 ± 0.41 - DPAC-UNet (Hui et al., 2021) 83.91 ± 0.21 90.73 ± 0.21 - H-Net (Zhou et al., 2022) 83.84 ± 0.59 90.42 ± 0.59 - Trans-Unet (Chen et al., 2021) 82.54 ± 0.62 89.98 ± 0.62 - Swin-Unet (Cao et al., 2021) 84.29 ± 0.46 91.03 ± 0.46 - I^2U-Net* 84.92 ± 0.58 91.31 ± 0.58 - I^2U-Net 85.70 ± 0.35 91.83 ± 0.37 - Mostafa et al. 81.6 89.1 - Mostafa et al.*+proposed 84.3 90.1 - U-Net 73.91 83.66 60.12 U-Net++ 81.26 88.99 46.66 Polar Res-UNet++ 75.23 85.36 42.96 Attention U-Net 69.52 80.52 74.51 DeepLabV3+ 82.03 89.56 37.81 CE-Net 83.48 90.44 33.97 CA-Net 75.18 84.66 64.53 TransFuse 82.32 89.75 39.98 TransUNet 83.99 90.96 33.3 XBound-Former 85.38 91.8 26 MAS-NET 90.36 94.82 7.28

[0094] The quantitative results of the PH2 dataset are shown in Table III, focusing on the mIoU, DSC, and HD95 indicators. MAS-NET has an mIoU of 90.36%, a DSC of 94.82%, and an HD95 of 7.28, which is better than various methods. Specifically, MAS-NET is 4.66% and 2.99% higher than I2U-Net in mIoU and DSC, respectively, and HD95 is also significantly reduced, indicating that its boundary delineation accuracy is higher.

[0095] Table 4 Comparison of experimental results on CVC dataset

[0096]

[0097] The quantitative results on the CVC-ClinicDB dataset shown in Table IV highlight the DSC and mIoU metrics. MAS-NET achieves 95.14% DSC and 90.83% mIoU, outperforming many methods. It is worth noting that MAS-NET outperforms DSNet by 0.54% in DSC and 0.63% in mIoU.

[0098] Table 5. Comparison of experimental results on KVASIR-SEG dataset

[0099] Method mDIce mIoU Recall Precision U-Net++ 82.1 74.3 - - SFA 72.3 61.1 - - MSNet 90.7 86.2 - - EU-Net 90.8 85.4 - - SANet 90.4 84.7 - - DCRNet 88.6 82.5 - - GLFRNet 90.6 85.5 - - DSNet 92.8 88 - - U-Net 82.64 74.72 85.03 87.03 DeepLabV3+ 88.37 81.72 90.14 90.28 PraNet 89.43 82.96 90.6 91.26 MSNet 88.59 82.17 90.06 91.1 TransFuse-S 87.8 80.79 88.98 90.9 TransFuse-L 87.68 81.15 88.42 91.98 Polyp-PVT 89.6 83.28 94.4 88.11 UACANet 88.35 81.33 90.85 89.47 DuAT 89.03 82.94 91.86 90.19 CaraNet 87.07 79.58 92.03 86.21 SSFormer-S 89.94 83.63 91.94 90.86 SSFormer-L 90.6 85 92.13 91.99 UNeXt 73.18 62.84 78.4 76.56 LDNet 88.81 82.08 90.63 90.46 TGANet 89.82 83.3 91.32 91.23 PVT-Cascade 89.5 83.29 93.55 88.88 PrototypeLab 90.86 85.44 93.44 91.36 MAS-NET 93.96 89.41 96.20 92.50

[0100] The quantitative results on the Kvasir-SEG dataset described in Table V highlight the DSC, mIoU, recall, and precision metrics. MASNET outperforms various methods with DSC of 93.96%, mIoU of 89.41%, Recall of 96.20%, and Precision of 92.50%. MAS-NET outperforms DSNet by 1.16% in DSC and 1.41% in mIoU.

[0101] Table 6 Comparison of experimental results on the POLYPGEN dataset

[0102] Model MDice MIoU HD95 Unet 69.47 61.83 53.0159 Unet++ 71.67 64.02 39.8603 Unet3+ 67.37 54.07 28.6603 Deeplabv3+ 69.84 59.04 47.4551 Double-Unet 59.04 53.33 45.8987 HRNetv2-w18-sv2 57.82 52.82 61.9138 PraNet 77.76 70.39 25.4807 UACANet-S 77.26 70.39 13.2488 TransUnet 69.5 61.84 39.8603 Transfuse 62.55 56.84 27.7006 MSRF-Net 82.13 80.24 24.4807 SSFormer-S 67.04 62.75 11.679 MedT 75.83 72.75 44.6983 DCSAU-Net 74.19 67.85 32.4563 Atten-Unet 84.3 83.04 27.7146 MRUnet 64.36 65.48 45.0347 SwinUnet 75.8 72.75 40.2244 UCTransNet 83.42 80.12 12.7036 TGANet 71.3 67.85 32.822 PVT-Cascade 81.94 80.45 25.4807 ERSUnet 76.83 66.11 35.7869 ERDUnet 79.6 70.67 28.0181 MAS-NET 88.91 82.20 8.23

[0103] The quantitative results of the Polypgen dataset are shown in Table VI, focusing on the DSC, mIoU, and HD95 indicators. MAS-NET has a DSC of 88.91%, an mIoU of 82.20%, and an HD95 of 8.23, which is better than many methods. MAS-NET exceeds PVT-Cascade and PVT-Cascade by 6.97% and 4.95% in DSC and mIoU, respectively.

[0104] To verify the effectiveness of the three-path spectral hierarchical encoder, feature remodulation mechanism, and multi-level feature reconstruction module in our proposed multi-aspect segmentation network framework, we conducted extensive ablation experiments.

[0105] Table VII Ablation experiment

[0106] Pre-trained SSE HDME mIoU(%) DSC(%) MCC(%) G-mean(%) 77.85 86.78 83.38 88.20 ✓ 80.83 88.63 84.74 89.33 ✓ 82.51 89.90 85.96 90.83 ✓ ✓ 82.85 90.18 85.27 91.19 ✓ ✓ ✓ 83.37 90.55 86.06 91.74

[0107] The results of this ablation experiment are presented in Table VII above. SSE without KAN channel attention or MFFM has an IoU of 81.36%, DSC of 88.94%, MCC of 85.38%, and G-mean of 90.31%. Introducing only KAN channel attention, IoU improves by 82.36%, DSC by 89.77%, MCC by 85.56%, and G-mean by 90.96%, indicating that the importance of channels is dynamically optimized, enhancing noise suppression and key structural channels. MFFM alone further improves IoU to 82.60%, DSC to 89.92%, MCC to 85.72%, and G-mean to 91.4%, emphasizing the advantage of focusing on low-frequency components through fast Fourier transform. The IoU of the combination of KAN Channel Attention and MFFM is 83.37%, DSC is 90.55%, MCC is 86.06%, and G-mean is 91.74%, indicating that the complementary advantages are effectively utilized for segmentation performance.

[0108] In summary, the MAS-Net proposed in this invention significantly improves the accuracy and robustness of medical image segmentation by combining frequency domain feature processing, geometric deformation analysis and semantic context analysis, and has broad application prospects.

[0109] The above contents are only preferred implementation modes of the present invention and do not limit the protection scope of the present invention. It should be emphasized that any minor adjustments and improvements made by technicians in this field without departing from the basic principles of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A multi-aspect segmentation network implementation method for visible light medical image segmentation, characterized in that: This method integrates spectral spline encoder, hierarchical morphological encoder, semantic context encoder and multi-attention KAN integrator; the spectral spline encoder uses multi-frequency modulation and channel attention mechanism for noise suppression; the hierarchical morphological encoder captures complex geometric structures through deformable convolution; the semantic context encoder enhances semantic understanding through context analysis; the multi-attention KAN integrator integrates different types of features through multiple self-attention mechanisms and feature fusion; the image segmentation accuracy and robustness of the network are improved by enhancing noise suppression and feature extraction in the frequency domain, multi-scale feature encoding and deformable geometric analysis, contextual relationship enhancement, and multiple self-attention mechanisms and feature fusion; The spectral spline encoder includes a multi-frequency feature regulator and a KAN channel attention mechanism. The image is converted from the spatial domain to the frequency domain through fast Fourier transform. The multi-frequency feature regulator modulates the frequency domain information of the original image and performs weighted processing to obtain modulated convolution features; then the convolution features are fast Fourier transformed; then the modulated frequency domain features are restored to the spatial domain through inverse fast Fourier transform to achieve the fusion of frequency domain and spatial domain; the final image features are obtained by integrating the modulated frequency domain information and the frequency domain information of the original spatial domain image; The hierarchical morphological encoder generates multi-scale feature representations through progressive convolution and pooling operations, covering structural details at different resolutions and providing a comprehensive understanding of the main structure. In order to enhance the understanding of complex geometric structures, the hierarchical morphological encoder introduces a variable Laplace transform module, which dynamically adjusts the shape and position of the convolution kernel by calculating the geometric offset of the feature map, so that the convolution operation can adaptively parse the complex irregular morphology in the image. The geometric offset calculation formula is as follows: Among them, Δp represents the geometric offset of the convolution kernel, W K and O k Represent the weight and offset of the jth operation respectively, and K is the total number of operations; then through multi-layer convolution and pooling operations, feature maps of different scales are extracted, and weighted addition is performed according to the weights to obtain multi-scale features: Among them, Y HDME is a multi-scale feature, W i represents the weight of the i-th convolution layer, F i represents the feature map extracted by the i-th layer, and L is the total number of convolutional layers.

2. The multi-aspect segmentation network implementation method for visible light medical image segmentation according to claim 1, characterized in that: The spectral spline encoder includes a multi-frequency feature regulator and a KAN channel attention mechanism. The image is converted from the spatial domain to the frequency domain through fast Fourier transform. The multi-frequency feature regulator modulates the frequency domain information of the original image and performs weighted processing to obtain the modulated convolution feature. The specific formula is as follows: X1=LPF(X) Among them, Y mod is the modulated convolution feature, X is the original image, LPF is the low-pass filtering operation, X1 is the frequency domain information of the original image after filtering, w is the learnable weighting matrix, is a set of real numbers, C is the number of channels, W is the image width, and H is the image height; the convolutional features are then subjected to fast Fourier transform, and the specific formula is as follows: X2=Conv(X) Among them, Y fft is the frequency domain feature obtained by fast Fourier transform, X2 is the input convolution feature, is a set of real numbers, FFT stands for fast Fourier transform, and Conv stands for convolution. Then, the frequency domain features after modulation are restored to the spatial domain through the inverse fast Fourier transform to achieve the fusion of the frequency domain and the spatial domain. The specific formula is as follows: Among them, Y ifft is the feature after inverse Fourier transform, M is the learnable coefficient, and IFFT represents inverse fast Fourier transform. The final image feature is obtained by integrating the frequency domain information after modulation and the frequency domain information of the original spatial domain image. The specific formula is as follows: Among them, Y MFFM It is the image feature output by the multi-frequency feature modulation module.

3. The method for implementing a multi-aspect segmentation network for visible light medical image segmentation according to claim 2, characterized in that: The KAN channel attention mechanism introduces an adaptive adjustment strategy based on the KAN layer. Through adaptive channel weight adjustment, the response ability of each channel is dynamically optimized. For the calculation of channel attention, the image feature Y is first MFFM The feature map of each channel is globally averaged pooled: in, is the global feature vector after pooling, x c is the feature map of each channel, H and W represent the height and width of the feature map respectively, x c (i, j) represents the pixel value at position (i, j) on the feature map; the channel attention weight is then calculated through the KAN layer: Among them, a c is the channel attention weight, σ is a learnable activation function, and KAN is a Kolmogorov-Arnold network; apply the attention weight to the channel features: Among them, Y SSE It is the channel feature of the final output after weighting.

4. The method for implementing a multi-aspect segmentation network for visible light medical image segmentation according to claim 3, characterized in that: The semantic context encoder has the global context perception ability of a large visual model, which can effectively capture and integrate the semantic information of the image at multiple scales, and dynamically capture long-distance dependencies in the global scope of the image; first, local features are extracted through a 16×16 window attention mechanism, and then these features are integrated with the global context through multiple global attention blocks to form a complete semantic representation: Y SCE =GlobalAttention(LocalAttention(X)) Among them, Y SCE is the semantic context feature, X is the output original image, LocalAttention is the window attention, and GlobalAttention is the global attention.

5. The method for implementing a multi-aspect segmentation network for visible light medical image segmentation according to claim 4, characterized in that: The multi-attention KAN integrator processes global frequency domain features, geometric morphological features and semantic features in parallel, and fuses them through the designed KAN multiple self-attention mechanism and iterative attention feature fusion mechanism; FFT features =Conv2d(Y SSE ) Among them, FFT features It is the global frequency domain feature, Conv2d is the two-dimensional convolution; The multi-scale features are transformed and enhanced through double-layer convolution and PixelShuffle. The convolution layer is responsible for preliminary feature extraction, while the PixelShuffle technology is used for enhancement and upsampling: BOD features =PixelShuffle(Conv2d(ReLU(Conv2d(Y HDME )))) Among them, BOD features is the geometric feature, ReLU is the activation function, Y HDME It is a multi-scale feature output by the hierarchical morphological encoder; this processing method effectively captures the complex geometric structure in the image, making the feature more adaptable; The semantic features are transformed through double-layer convolution and feature enhancement is performed to strengthen the semantic expression of the features: Sat. features =Conv2d(ReLU(Conv2d(Y SCE ))) Among them, SAM features is a semantic feature, Y SCE is the feature output by the semantic context encoder.

6. The method for implementing a multi-aspect segmentation network for visible light medical image segmentation according to claim 5, characterized in that: Various features are optimized through KAN multiple self-attention mechanism to achieve dynamic interaction; first, the feature branch is processed by block-level self-attention to calculate the query, key and value (Q, K, V) to obtain the interdependence between features. Then, the KAN layer performs feature transformation on the self-attention output to enhance its representation ability: CAN feature =KANLayer(AttentionOutput) Among them, KAN feature is the transformed feature, AttentionOutput is the feature after self-attention; finally, the features of each branch are merged along the channel dimension and restored to the original number of channels through the convolution layer: CAN output =Conv(Concat(CAN feature1 ,CAN feature2 ,CAN feature3 )) Among them, KAN output is the adjusted output, KAN feature1 ,KAN feature2 ,KAN feature3 They are the features of each branch after KAN processing, and Concat is a splicing operation; the iterative attention fusion module combines the local and global attention mechanisms to achieve feature integration through weighted fusion: Among them, iAFF output is the fused feature, α is the learnable coefficient, and Y is the feature output by each branch. Represents element-wise multiplication.

7. The method for implementing a multi-aspect segmentation network for visible light medical image segmentation according to claim 6, characterized in that: The specific steps include: S1: Collect image data from multiple public medical image datasets; S2: Organize and divide the collected image data into training set and test set; S3: preprocessing the image data; S4: Apply data enhancement techniques to image data; S5: Build a multi-aspect segmentation network for visible light medical image segmentation; S6: Use the training set after the data enhancement technology in step S4 to train the multi-aspect segmentation network model and obtain weights; S7: Use the trained weights to evaluate the model performance on the test set, and adjust the model parameters and architecture based on the evaluation results to optimize the model performance.

8. The method for implementing a multi-aspect segmentation network for visible light medical image segmentation according to claim 7, characterized in that: The image data preprocessing includes adjusting all images to a uniform size, normalizing image pixel values ​​to a range of [0, 1], and removing blurred, incomplete or incorrectly labeled images.

9. The method for implementing a multi-aspect segmentation network for visible light medical image segmentation according to claim 7, characterized in that: The data augmentation techniques described include randomly scaling images and cropping them to a target size, randomly flipping images horizontally, and adjusting image brightness, contrast, and saturation.

Images