A U-KAN medical image semantic segmentation method based on multi-path Mamba fusion

Abstract

This paper presents a U-KAN medical image semantic segmentation method based on multi-path Mamba fusion, belonging to the interdisciplinary fields of computer vision and medicine. The method designs a multi-path semantic representation extraction network, obtaining multi-scale semantic representations by adjusting the input image to different resolutions and inputting them separately into global and local U-KAN encoders. A cross-path adaptive fusion network is constructed, introducing the selective state-space model Mamba to build a staged feature interaction mechanism. Cross-selective scanning and bidirectional aggregation strategies are used to achieve the fusion and enhancement of multi-scale semantic representations, improving the model's ability to capture long-range contextual information and the discriminative power of skip connection representations. A multi-stage supervised decoding network is constructed, fusing skip representations layer by layer through a U-KAN decoder architecture symmetrical to the U-KAN encoder. A deep hierarchical loss function is introduced to apply collaborative supervision, effectively promoting the alignment and optimization of high-level semantic and low-level structural information during training, enhancing the consistency of segmentation boundaries and the integrity of regions.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of computer vision and medicine, and relates to a U-KAN medical image semantic segmentation method based on multi-path Mamba fusion. Background Technology

[0002] Semantic segmentation of medical images is a crucial task in computer vision and medical image analysis. Its goal is to classify each pixel in medical images (such as magnetic resonance imaging and computed tomography scans) and assign it to predefined categories, such as specific organs, tissues, or lesion regions. Accurate semantic segmentation of medical images is of paramount importance for clinical diagnosis, treatment planning, surgical navigation, and disease monitoring.

[0003] In recent years, numerous variants of U-Net have emerged, which can be broadly categorized into three types: U-shaped methods based on convolutional neural networks (CNNs), U-shaped methods based on Transformers, and U-shaped methods based on hybrid networks. CNN-based U-shaped methods cascade convolutional blocks to construct an encoder-decoder architecture, leveraging the inherent inductive bias of convolutional operations to capture subtle changes in target boundaries and micro-texture features, thereby enhancing semantic segmentation of medical images. Transformer-based U-shaped methods cascade Transformer blocks to construct an encoder-decoder architecture, utilizing self-attention mechanisms to establish global semantic dependencies across image regions for medical image segmentation. Hybrid network-based U-shaped methods couple convolutional and Transformer blocks in parallel or sequentially to construct an encoder-decoder architecture, jointly learning global semantics and local structure to enhance semantic segmentation of medical images.

[0004] However, existing U-Net-based methods still face significant challenges in feature representation and multi-scale information integration. First, most methods rely on fixed-form activation functions for nonlinear feature mapping through linear combinations. This learning mechanism, based on stacked fixed functions, limits the model's ability to model complex nonlinear correlations in medical images, leading to performance saturation and insufficient generalization in the segmentation of morphologically varied anatomical structures. Second, existing architectures generally lack effective collaborative modeling of global semantics and local details, failing to establish interaction mechanisms between cross-scale features. This results in semantic information breaks and detail loss when restoring the target spatial structure, affecting the accuracy of segmentation boundaries and the integrity of the structure. These problems collectively reveal the inherent limitations of current methods in nonlinear representation capabilities, multi-scale context fusion, and dynamic feature adaptation, necessitating breakthroughs through novel network paradigms. Summary of the Invention

[0005] To address the problems existing in current technologies, this invention proposes a U-KAN medical image semantic segmentation method based on multi-path Mamba fusion. By introducing a Kolmogorov-Arnold network (KAN) to reconstruct the U-Net encoding and decoding structure, and through a nonlinear mapping composed of learnable parameterized activation functions, the method enhances the ability to model complex high-order relationships between pixels, thereby improving the flexibility of feature representation. This method systematically improves the U-shaped segmentation architecture from three levels: nonlinear representation, multi-path fusion, and hierarchical supervision, enabling it to better adapt to the complex structural changes in medical images.

[0006] To achieve the above objectives, the technical solution adopted by this invention is as follows: a U-KAN medical image semantic segmentation method based on multi-path Mamba fusion, comprising the following steps:

[0007] Step 1: Construct a multi-path semantic representation extraction network to extract multi-scale semantic representations of medical images: Adjust the input image to different resolutions and input them into the global and local U-KAN encoders respectively to obtain coarse-grained and fine-grained multi-scale semantic representations. Each U-KAN encoder consists of three convolutional stages and two KAN stages.

[0008] Step 2: Construct a cross-path semantic adaptive fusion network, aggregate the multi-scale semantic representations, and generate U-KAN skip connection representations:

[0009] A selective state-space model, Mamba, is introduced to construct a phased and refined interaction mechanism, adaptively aggregating the multi-scale semantic representations. The cross-path semantic adaptive fusion network includes a Mamba interaction stage and a Mamba aggregation stage. In the Mamba interaction stage, a cross-attention mechanism is simulated to enhance the semantic representation of one path by using one path semantic representation. In the Mamba aggregation stage, the semantic representations enhanced by the Mamba interaction stage are concatenated and a selective scanning mechanism is applied to obtain the final skip connection representation.

[0010] Step 3: Construct a multi-stage supervised decoding network, fusing the skip connection representation stage by stage during the image semantic representation decoding process to generate an image semantic segmentation mask:

[0011] By constructing a U-KAN encoder-symmetric U-KAN decoder architecture, skip connection representations are fused stage by stage to generate image semantic segmentation masks; at the same time, a deep hierarchical loss function is introduced to apply collaborative supervision in the three key stages of the encoder's final stage, the decoder's first stage, and the final stage.

[0012] Furthermore, step 1 specifically includes:

[0013] Step 1.1: Construct a global U-KAN encoder and a local U-KAN encoder by stacking three convolutional stages and two KAN stages respectively;

[0014] Given a medical image input ,in, Indicates image height, Indicates the image width and Indicates the number of image channels. To represent image resolution, each U-KAN encoder first stacks three convolutional stages to extract the image. The convolutional semantic representation is as follows, where the computation process of each convolutional stage is as follows:

[0015] (1)

[0016] in, Indicates that the convolution kernel is And a convolutional layer with a stride of 1, Indicates the regularization layer. Indicates size is Pooling layers;

[0017] and Let represent the output semantic representation and input semantic representation of the i-th convolutional stage, respectively; , representing the number of convolution stages;

[0018] Then, the semantic representation after the three convolutional stages The input is fed into two KAN stages; the computation flow of the j-th KAN stage is as follows:

[0019] (2)

[0020] in, Represents the vector projection layer. Indicates the KAN layer. Indicates the vector inverse projection layer; and These represent the output and input of the j-th KAN stage, respectively. , representing the number of stages in the KAN; ;

[0021] KAN layer The specific calculation method is as follows:

[0022] (3)

[0023] in, and This represents the neurons in the j-th and (j-1)-th KAN layers; Represents two neurons and Learnable nonlinear mapping function between; two KAN stages represent semantics. Transformed into the final medical image semantic representation ;

[0024] Step 1.2: Generate a multi-stage, multi-scale semantic representation set by constructing the global U-KAN encoder and the local U-KAN encoder;

[0025] The input medical images are adjusted to different resolutions and input into the global U-KAN encoder and local U-KAN encoder constructed in step 1.1 respectively to generate a multi-stage, multi-scale semantic representation set. First, the high-resolution image is input into the global U-KAN encoder to generate multi-stage semantic representations. Then, the low-resolution image is input into the local U-KAN encoder to generate multi-stage semantic representations. In each representation extraction stage, two semantic representations of different resolutions are obtained for skip connections. Finally, the multi-stage, multi-scale semantic representations of the global U-KAN encoder and the local U-KAN encoder are obtained.

[0026] Furthermore, the input resolution of the five stages of the global U-KAN encoder is defined as follows: , , , as well as ;

[0027] The input resolution of the five stages of a partial U-KAN encoder is defined as follows: , , , as well as ;

[0028] Finally, the five-stage multi-scale semantic representations of the global U-KAN encoder and the local U-KAN encoder are defined as follows:

[0029] (4)

[0030] (5)

[0031] in, and These represent the five-stage semantic representation sets captured by the global U-KAN encoder and the local U-KAN encoder, respectively.

[0032] Furthermore, step 2 specifically includes:

[0033] Step 2.1: Construct the Mamba interaction phase, simulating the cross-attention mechanism by using one path semantic representation to enhance the expression of another path semantic representation;

[0034] Given step 1.2, obtain the coarse-grained semantic representation extracted from the k-th stage in the multi-stage, multi-scale semantic representation set. and fine-grained semantic representation ;

[0035] use The operation converts a two-dimensional semantic representation into a one-dimensional semantic representation:

[0036] (6)

[0037] in, and They are respectively and One-dimensional semantic representation;

[0038] Dynamically generated using linear projection layers and Mamba parameter set of state-space model and ,in, and These are the input matrix parameters; and These are the output matrix parameters; and It controls how much information is absorbed from the current input into the state at each time step, and how much historical information is retained;

[0039] Discretization is performed to transform the continuous-time system into a discrete-time system with multiple time steps:

[0040] , (7)

[0041] , (8)

[0042] in, and It is a pre-defined learnable state transition matrix; and It is the state transition matrix after discretization; and The input matrix after discretization; Represents an exponential function;

[0043] For each time step t, by exchanging the output matrix and Achieve semantic information interaction across paths:

[0044] , (9)

[0045] , (10)

[0046] in, and This represents the hidden state vector at time step t; and This represents the hidden state vector at time step t-1; and The input is represented by a one-dimensional semantic representation of time step t; and This represents the output of the state selection model at time step t;

[0047] After T-step, utilize The operation will output a one-dimensional semantic representation. and Mapping back to two-dimensional semantic representation respectively and ;

[0048] Step 2.2: Constructing the Mamba aggregation stage, a bidirectional scanning mechanism is adopted to adaptively fuse the multi-scale semantic representation enhanced in Step 2.1 to generate the U-KAN skip connection representation;

[0049] Based on step 2.1, obtain the two-dimensional semantic representation after the interaction in the k-th stage. and ,use The operation is converted into a one-dimensional semantic representation and a concatenation operation is performed to obtain a preliminary fusion skip connection representation at the k-th stage. :

[0050] (11)

[0051] use The operation generates a reverse jump connection representation. ,Right now Information extraction is performed by applying the standard state-space model to the forward and reverse jump connection representations respectively.

[0052] , (12)

[0053] in, This represents the complete forward propagation process of the state-space model; Indicates the process Semantic representation extracted from forward processing; Indicates the process Semantic representation extracted in reverse;

[0054] Finally, and Perform average fusion and utilize The operation is mapped back to two-dimensional space to generate the final skip connection representation for the k-th stage. :

[0055] (13)

[0056] Therefore, a five-stage skip connection representation is generated through a cross-path semantic adaptive fusion network. .

[0057] Furthermore, the forward propagation process of the state-space model includes parameter generation, discretization, and sequence scanning.

[0058] Furthermore, step 3 specifically includes:

[0059] The multi-stage supervised decoding network includes a U-KAN decoder and a deep hierarchical loss. The U-KAN decoder is constructed as a symmetrical structure of the U-KAN encoder, consisting of two KAN stages and three convolutional stages connected in series. The input of each decoding stage is composed of the skip connection semantic representation of the corresponding encoding stage and the output of the previous decoding stage.

[0060] Given the output of the (k-1)th decoding stage and the semantic representation of the skip connection corresponding to the k-th decoding stage. The decoding process in the k-th decoding stage is defined as follows:

[0061] (14)

[0062] (15)

[0063] in, The output of the k-th decoding stage;

[0064] To build a deep hierarchical loss, the model maintains prediction consistency at different resolutions: supervision signals are added in three stages: the final stage output of the U-KAN encoder, the first stage output of the U-KAN decoder, and the final stage output of the U-KAN decoder, to train the overall segmentation network.

[0065] The depth-level hierarchical loss is defined as follows:

[0066] (16)

[0067] in, , and These are the hyperparameters of the three-stage supervision loss; Represents the true segmentation mask of the image; This indicates the final output from the U-KAN encoder. The generated predicted segmentation mask; This indicates the output from the first stage of the U-KAN decoder. The generated predicted segmentation mask; This indicates the final output from the U-KAN decoder. The generated predicted segmentation mask;

[0068] Each loss It consists of two sub-losses:

[0069] (17)

[0070] in, This represents the weighted IoU loss, used to measure the degree of overlap between the predicted bounding box and the ground truth bounding box; This represents the weighted cross-entropy loss;

[0071] After the entire model is trained using deep hierarchical supervised loss, medical images are input into the trained model to obtain the corresponding semantic segmentation mask.

[0072] Furthermore, the hyperparameters of the three-stage supervision loss. =0.2, =0.2 and =0.6.

[0073] The beneficial effects of this invention are as follows: By introducing KAN to reconstruct the U-Net encoding and decoding structure, this invention significantly enhances the model's ability to model complex nonlinear relationships in medical images, effectively overcoming the expression bottleneck caused by traditional fixed activation function stacking; by designing a multi-path semantic representation extraction and Mamba-driven cross-path adaptive fusion network, it achieves efficient extraction and interaction of multi-scale semantic representations, improving the model's ability to perceive long-range contextual information and discriminative power of skip connection representations; further, by combining a multi-stage supervised decoding network, it effectively promotes the collaborative optimization of high-level semantic and low-level structural information during training, significantly improving the accuracy and region integrity of segmentation boundaries. Experimental results show that this invention achieves leading performance on multiple medical image segmentation benchmarks, demonstrating superior generalization ability and robustness, providing an effective solution for accurate segmentation in complex medical scenarios, and has good clinical application value. Attached Figure Description

[0074] Figure 1 Framework diagram of the U-KAN medical image semantic segmentation method based on multi-path Mamba fusion;

[0075] Figure 2 Flowchart of the U-KAN medical image semantic segmentation method based on multi-path Mamba fusion; Detailed Implementation

[0076] The embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0077] Step 1: Construct a multi-path semantic representation extraction network to extract multi-scale semantic representations of medical images. Specifically,

[0078] The input images are resized to different resolutions and fed into global and local U-KAN encoders respectively to obtain coarse-grained and fine-grained multi-scale semantic representations. Each U-KAN encoder consists of three convolutional stages and two KAN stages. By constructing multi-path semantic representation extraction paths, rich and robust multi-scale semantic information is provided for the subsequent image decoding process, greatly enhancing segmentation performance.

[0079] Step 2: Construct a cross-path semantic adaptive fusion network, aiming to effectively aggregate the multi-scale semantic representations extracted in Step 1 and generate U-KAN skip connection representations. Specifically,

[0080] A cross-path semantic adaptive fusion network is constructed by introducing the selective state-space model Mamba to adaptively aggregate semantic representations at different scales learned in step 1. The cross-path semantic adaptive fusion network consists of a Mamba interaction stage and a Mamba aggregation stage. In the Mamba interaction stage, a cross-attention mechanism is simulated to enhance the semantic representation of one path by using the semantic representation of another path. In the Mamba aggregation stage, the semantic representations enhanced by the Mamba interaction stage are concatenated and a selective scanning mechanism is applied to obtain the final skip connection representation.

[0081] Step 3: Construct a multi-stage supervised decoding network, fusing the skip connection representations extracted in Step 2 stage by stage during the image semantic representation decoding process to generate an image semantic segmentation mask. Specifically,

[0082] By constructing a U-KAN decoder architecture symmetrical to the U-KAN encoder, skip connection representations are fused stage by stage to generate image semantic segmentation masks. Simultaneously, a deep hierarchical loss function is introduced to apply collaborative supervision in three key stages: the final stage of the encoder, the first stage of the decoder, and the final stage. This effectively promotes the alignment and optimization of high-level semantic and low-level structural information during training, enhancing the consistency of segmentation boundaries and the integrity of regions.

[0083] Example 1

[0084] This invention is a U-KAN medical image semantic segmentation method based on multi-path Mamba fusion. For example... Figure 1 As shown, firstly, the input medical images are adjusted to different resolutions and fed into global and local U-KAN encoders respectively. The global U-KAN encoder processes the original resolution image to preserve complete spatial details, while the local U-KAN encoder processes the downsampled image to obtain a more generalized semantic context. Both the global and local U-KAN encoders perform hierarchical feature extraction through three convolutional stages and two KAN stages, outputting five different stages of multi-scale semantic representations, forming a multi-stage, multi-scale semantic representation set. Subsequently, the semantic representations of the corresponding stages of the dual-path encoders are fed into a cross-path adaptive fusion network. In this network, the semantic representations from different paths are first cross-selectively scanned through a Mamba interaction stage to achieve bidirectional enhancement between semantic representations of different scales. Then, the enhanced semantic representations are concatenated, and the Mamba aggregation stage captures long-range dependencies through bidirectional state space modeling, finally outputting a skip connection representation that integrates multi-path information. Afterward, the U-KAN decoder upsamples stepwise in a bottom-up manner, concatenating the current semantic representation with the skip connection representation of the corresponding encoding stage at each decoding stage to gradually restore spatial details. During the decoding process, the encoder's final stage representation, the decoder's first stage representation, and the final stage representation all generate segmentation predictions and are subject to collaborative supervision by a deep hierarchical loss function, ensuring multi-level consistency from global semantics to local structure. Finally, the network outputs a semantic segmentation mask with the same resolution as the input image, completing the end-to-end medical image segmentation task. The entire process of the U-KAN medical image semantic segmentation method based on multi-path Mamba fusion is as follows: Figure 2 As shown.

[0085] The specific implementation steps are as follows:

[0086] Step 1: Construct a multi-path semantic representation extraction network to extract multi-scale semantic representations of medical images;

[0087] Step 1.1: Construct global and local U-KAN encoders by stacking three convolutional stages and two KAN stages respectively. Specifically,

[0088] Given a medical image input ,in, Indicates image height, Indicates the image width and Indicates the number of image channels. To represent image resolution, each encoder first stacks three convolutional stages to extract the image. The convolutional semantic representation is as follows, where the computation process of each convolutional stage is as follows:

[0089] (1)

[0090] in, Indicates that the convolution kernel is And a convolutional layer with a stride of 1, Indicates the regularization layer. Indicates size is Pooling layer. and Let represent the input semantic representation and the output semantic representation of the i-th convolutional stage, respectively. , representing the number of convolutional stages. Then, the semantic representation after three convolutional stages... The input is fed into two KAN stages. The computation flow of the j-th KAN stage is as follows:

[0091] (2)

[0092] in, Represents the vector projection layer. Indicates the KAN layer. This represents the vector inverse projection layer. and Let represent the input semantic representation and output semantic representation of the j-th KAN stage, respectively. , representing the number of stages in the KAN. . KAN layer The specific calculation method is as follows:

[0093] (3)

[0094] in, and This represents the neurons in the j-th and (j-1)-th KAN layers. Represents two neurons and KAN employs a learnable nonlinear mapping function as weights, replacing the linear weight matrix of traditional neural networks. This significantly enhances the model's ability to capture complex nonlinear relationships between pixels in medical images, while also improving model flexibility. The two KAN stages represent semantics... Transformed into the final medical image semantic representation .

[0095] Step 1.2: By constructing the global and local U-KAN encoders, a multi-stage, multi-scale semantic representation set is generated.

[0096] Medical images contain rich semantics, ranging from large-scale tissue contours to cellular-level details. Single-granularity feature extraction struggles to capture both global and local information, leading to inaccurate segmentation results. Therefore, the input medical images are adjusted to different resolutions and fed into the global and local U-KAN encoders constructed in step 1.1, generating a multi-stage, multi-scale semantic representation set. First, the high-resolution image is input into the global U-KAN encoder to generate five stages of semantic representations. The input resolution of the five stages of the global U-KAN encoder is defined as follows: , , , as well as Then, the low-resolution image is input into the local U-KAN encoder to generate a five-stage semantic representation, where the input resolution of the five stages of the local U-KAN encoder is defined as follows: , , , as well as This method allows for the acquisition of two semantic representations at different resolutions for skip connections at each semantic representation extraction stage, significantly enriching the semantic information required for the decoding process. Finally, the multi-scale semantic representations for the five stages of the global and local U-KAN encoders are defined as follows:

[0097] (4)

[0098] (5)

[0099] in, and These represent the five-stage semantic representation sets captured by the global U-KAN encoder and the local U-KAN encoder, respectively.

[0100] Step 2: Construct a cross-path semantic adaptive fusion network, aiming to effectively aggregate the multi-scale semantic representations extracted in Step 1 and generate U-KAN skip connection representations, specifically;

[0101] Step 2.1: Construct the Mamba interaction phase, simulating the cross-attention mechanism by using one path semantic representation to enhance the expression of another path semantic representation. Specifically,

[0102] Given step 1.2, obtain the coarse-grained semantic representation extracted from the k-th stage in the multi-stage, multi-scale semantic representation set. and fine-grained semantic representation First, in order to adapt to the sequence processing characteristics of the state-space model Mamba, we utilize... The operation converts a two-dimensional semantic representation into a one-dimensional semantic representation:

[0103] (6)

[0104] in, and They are respectively and A one-dimensional semantic representation is then generated using a linear projection layer. and Mamba parameter set of state-space model and ,in, and These are the input matrix parameters, which determine how the current input affects the model's internal state. and The output matrix parameters determine how the model's internal state is transformed into the final output. and It controls how much information is absorbed from the current input into the state at each time step, and how much historical information is retained. Simultaneously, it performs discretization, transforming the continuous-time system into a discrete-time system with multiple time steps.

[0105] , (7)

[0106] , (8)

[0107] in, and It is a pre-defined learnable state transition matrix. and It is the state transition matrix after discretization. and The input matrix after discretization. This represents an exponential function. Then, for each time step t, the output matrix is ​​swapped... and Achieve semantic information interaction across paths:

[0108] , (9)

[0109] , (10)

[0110] in, and This represents the hidden state vector at time step t. and This represents the hidden state vector at time step t-1. and The input is a one-dimensional sequence semantic representation at time step t. and This represents the output of the state selection model at time step t. After T steps, it utilizes... The operation will output a one-dimensional semantic representation. and Mapping back to two-dimensional semantic representation respectively and .

[0111] Step 2.2: Constructing the Mamba aggregation stage. A bidirectional scanning mechanism is used to adaptively fuse the multi-scale semantic representation enhanced in Step 2.1, generating a U-KAN skip connection representation. Specifically,

[0112] Based on the semantic representation obtained in step 2.1 after the k-th stage interaction and ,use The operation is converted into a one-dimensional semantic representation and a concatenation operation is performed to obtain a preliminary fusion skip connection representation at the k-th stage. :

[0113] (11)

[0114] Then, in order to capture bidirectional contextual information, utilize The operation generates a reverse jump connection representation. ,Right now Information extraction is performed by applying the standard state-space model to the forward and reverse jump connection representations respectively:

[0115] , (12)

[0116] in, It represents the complete forward propagation process of the state-space model, including steps such as parameter generation, discretization, and sequence scanning. Indicates the process Semantic representation extracted from forward processing. Indicates the process The semantic representation extracted in reverse. Finally, and Perform average fusion and utilize The operation is mapped back to two-dimensional space to generate the final skip connection representation for the k-th stage. :

[0117] (13)

[0118] Therefore, a five-stage skip connection representation is generated through a cross-path semantic adaptive fusion network. .

[0119] Step 3: Construct a multi-stage supervised decoding network, which integrates the skip connection representations learned in Step 2 stage by stage during the image semantic representation decoding process to generate an image semantic segmentation mask. Specifically,

[0120] The multi-stage supervised decoding network mainly consists of a U-KAN decoder and a deep hierarchical loss. The U-KAN decoder is constructed as a symmetric structure of the U-KAN encoder, composed of two KAN stages and three convolutional stages cascaded together. The input to each decoding stage is a concatenation of the skip connection semantic representation of the corresponding encoding stage and the output of the previous decoding stage. Given the output of the (k-1)th decoding stage... and the skip connection semantic representation corresponding to the decoding stage generated in step 2. The decoding process in the k-th decoding stage is defined as follows:

[0121] (14)

[0122] (15)

[0123] in, The output of the k-th decoding stage. To improve segmentation accuracy, a deep hierarchical loss is proposed, forcing the model to maintain prediction consistency across different resolutions and enhancing the co-optimization of high-level semantics and low-level structure in image segmentation. Specifically, it adds supervision signals in three stages: the final stage output of the U-KAN encoder, the first stage output of the U-KAN decoder, and the final stage output of the U-KAN decoder, to train the overall segmentation network. The deep hierarchical loss is defined as follows:

[0124] (16)

[0125] in, =0.2, =0.2 and =0.6 is the hyperparameter of the three-stage supervision loss. This represents the true segmentation mask of the image. This indicates the output from the final stage of the U-KAN encoder. The generated predicted segmentation mask. This indicates the output from the first stage of the U-KAN decoder. The generated predicted segmentation mask. This indicates the output from the final stage of the U-KAN decoder. The generated predicted segmentation mask. Each loss statement... It consists of two sub-losses:

[0126] (17)

[0127] in, This represents the weighted IoU loss, used to measure the degree of overlap between the predicted bounding box and the true bounding box. This represents the weighted cross-entropy loss.

[0128] After the entire method is trained using deep hierarchical supervised loss, the medical image is input into the trained method to obtain the corresponding semantic segmentation mask.

[0129] Example 2

[0130] Datasets: Three heterogeneous medical datasets were selected as benchmark datasets to verify the superiority and effectiveness of this invention in medical image semantic segmentation. The BUSI dataset contains 647 breast tumor images and corresponding semantic segmentation masks. The CVC-ClinicDB dataset contains 612 colonoscopy sequence images and corresponding semantic segmentation masks. The Kvasir-seg dataset contains 1000 polyp images and corresponding semantic segmentation masks. In the experiments, all three datasets were divided into training and test sets in an 8:2 ratio, with all images resized to 256×256.

[0131] Experimental Details: The entire network architecture of this invention is implemented using PyTorch and runs on a Linux system with an NVIDIA GTX A100 GPU. Each U-KAN encoder consists of three convolutional stages and two KAN stages, and the U-KAN decoder is composed of a symmetrical architecture of the U-KAN encoder. During training, the proposed method utilizes Adam to optimize the entire network, with a learning rate of 0.0001, a training epoch of 350, and a batch size of 8. Dice and mIoU are selected as evaluation metrics for semantic segmentation performance, with values ​​ranging from [0,1]. Higher Dice and mIoU values ​​indicate better segmentation performance.

[0132] Comparison Method: This invention selects nine medical image semantic segmentation models as comparison methods, including three U-net models based on convolutional neural networks (CU-Net, EU-Ne, and IU-Net), three U-net models based on Transformer (LKAFormer, TransU-Net, and MFormer), and three U-net models based on Mamba (HMAFNet, MambaNet, and VMamba-Net).

[0133] Comparative Analysis: The comparative experimental results on three heterogeneous datasets are shown in Table 1. This invention achieves state-of-the-art segmentation results, demonstrating its effectiveness and advancement in medical image segmentation. Its technological superiority stems primarily from innovative designs at three levels: First, at the infrastructure level, this invention fundamentally reconstructs the traditional U-Net using the Kolmogorov-Arnold Network. Compared to traditional convolutional or Transformer modules that rely on fixed activation functions, the KAN layer forms a more flexible nonlinear mapping mechanism through learnable parameterized activation functions, enabling it to more accurately fit the complex high-order relationships between pixels in medical images. This theory-inspired network structure not only enhances the model's ability to capture subtle tissue boundaries and complex pathological features but also improves the model's reliability through its inherent mathematical interpretability, laying a solid foundation for accurate segmentation. Second, regarding the feature fusion mechanism, the cross-path semantic adaptive fusion network designed in this invention overcomes the limitations of traditional skip connections. Through the cross-scanning and bidirectional aggregation strategy of the selective state-space model, this module achieves dynamic adaptive interaction between multi-granularity semantic representations, preserving the integrity of local details while effectively establishing long-range contextual dependencies. Compared to the local receptive field limitations of traditional convolutions or the quadratic computational complexity of Transformers, this module achieves efficient collaboration between global and local features at linear computational cost, significantly improving the model's ability to represent multi-scale anatomical structures. Finally, regarding optimization strategies, the multi-stage supervised decoding network proposed in this invention achieves fine-tuning of the training process through a deep hierarchical loss function. This loss function applies collaborative supervision to three key nodes: the encoder's final stage output, the decoder's first stage output, and the final stage output, forcing the model to maintain prediction consistency across different resolutions and effectively mitigating the semantic information decay problem common in deep networks. This multi-level supervision strategy promotes the alignment optimization between high-level semantic concepts and low-level structural features, enabling the model to maintain boundary accuracy while improving consistency within regions, thus exhibiting stronger robustness in complex medical scenarios.

[0134] Table 1. Segmentation results of three heterogeneous medical image datasets

[0135]

[0136] Ablation Analysis: To fully verify the effectiveness of each component of this invention, three ablation variants were constructed. Variant 1: This invention without MPE represents transforming the multipath encoder into a single-path encoder. Variant 2: This invention without MMF represents using traditional skip connections between the encoder and decoder. Variant 3: This invention without KAN represents using a traditional convolutional architecture between the encoder and decoder. The ablation results are shown in Table 2. The results show that the ablation of each component exhibits suboptimal semantic segmentation performance. This demonstrates the rationality and effectiveness of the design of each component of this invention.

[0137] Table 2 Ablation segmentation results of three heterogeneous medical image datasets

[0138]

[0139] Specifically, a multi-path semantic representation extraction network is designed. By adjusting the input image to different resolutions and inputting it into the global and local U-KAN encoders respectively, coarse-grained and fine-grained multi-scale semantic representations are obtained. Then, a cross-path adaptive fusion network is constructed. By introducing the selective state-space model Mamba to build a staged feature interaction mechanism, cross-selective scanning and bidirectional aggregation strategies are used to achieve efficient fusion and enhancement of multi-scale semantic representations, effectively improving the model's ability to capture long-range contextual information and the discriminativeness of skip connection representations. Finally, a multi-stage supervised decoding network is constructed. By designing a U-KAN decoder architecture symmetrical to the U-KAN encoder, skip representations are fused layer by layer, and a deep hierarchical loss function is introduced to apply collaborative supervision, effectively promoting the alignment and optimization of high-level semantics and low-level structural information during training, and enhancing the consistency of segmentation boundaries and the integrity of regions.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A U-KAN medical image semantic segmentation method based on multi-path Mamba fusion, characterized in that, Includes the following steps: Step 1: Construct a multi-path semantic representation extraction network to extract multi-scale semantic representations of medical images: Adjust the input image to different resolutions and input them into the global and local U-KAN encoders respectively to obtain coarse-grained and fine-grained multi-scale semantic representations. Each U-KAN encoder consists of three convolutional stages and two KAN stages. Step 2: Construct a cross-path semantic adaptive fusion network, aggregate the multi-scale semantic representations, and generate U-KAN skip connection representations: A selective state-space model, Mamba, is introduced to construct a phased and refined interaction mechanism, adaptively aggregating the multi-scale semantic representations. The cross-path semantic adaptive fusion network includes a Mamba interaction stage and a Mamba aggregation stage. In the Mamba interaction stage, a cross-attention mechanism is simulated to enhance the semantic representation of one path by using one path semantic representation. In the Mamba aggregation stage, the semantic representations enhanced by the Mamba interaction stage are concatenated and a selective scanning mechanism is applied to obtain the final skip connection representation. Step 3: Construct a multi-stage supervised decoding network, fusing the skip connection representation stage by stage during the image semantic representation decoding process to generate an image semantic segmentation mask: By constructing a U-KAN encoder-symmetric U-KAN decoder architecture, skip connection representations are fused stage by stage to generate image semantic segmentation masks; at the same time, a deep hierarchical loss function is introduced to apply collaborative supervision in the three key stages of the encoder's final stage, the decoder's first stage, and the final stage.

2. The U-KAN medical image semantic segmentation method based on multi-path Mamba fusion according to claim 1, characterized in that, Step 1 specifically includes: Step 1.1: Construct a global U-KAN encoder and a local U-KAN encoder by stacking three convolutional stages and two KAN stages respectively; Given a medical image input ,in, Indicates image height, Indicates the image width and Indicates the number of image channels. To represent image resolution, each U-KAN encoder first stacks three convolutional stages to extract the image. The convolutional semantic representation is as follows, where the computation process of each convolutional stage is as follows: （1）； in, Indicates that the convolution kernel is And a convolutional layer with a stride of 1, Indicates the regularization layer. Indicates size is Pooling layers; and Let represent the output semantic representation and input semantic representation of the i-th convolutional stage, respectively; , representing the number of convolution stages; Then, the semantic representation after the three convolutional stages The input is fed into two KAN stages; the computation flow of the j-th KAN stage is as follows: （2）； in, Represents the vector projection layer. Indicates the KAN layer. Indicates the vector inverse projection layer; and These represent the output and input of the j-th KAN stage, respectively. , representing the number of stages in the KAN; ; KAN layer The specific calculation method is as follows: （3）； in, and This represents the neurons in the j-th and (j-1)-th KAN layers; Represents two neurons and Learnable nonlinear mapping function between; two KAN stages represent semantics. Transformed into the final medical image semantic representation ; Step 1.2: Generate a multi-stage, multi-scale semantic representation set by constructing the global U-KAN encoder and the local U-KAN encoder; The input medical images are adjusted to different resolutions and input into the global U-KAN encoder and local U-KAN encoder constructed in step 1.1 respectively to generate a multi-stage, multi-scale semantic representation set. First, the high-resolution image is input into the global U-KAN encoder to generate multi-stage semantic representations. Then, the low-resolution image is input into the local U-KAN encoder to generate multi-stage semantic representations. In each representation extraction stage, two semantic representations of different resolutions are obtained for skip connections. Finally, the multi-stage, multi-scale semantic representations of the global U-KAN encoder and the local U-KAN encoder are obtained.

3. The U-KAN medical image semantic segmentation method based on multi-path Mamba fusion according to claim 2, characterized in that, The input resolution of the five stages of the global U-KAN encoder is defined as follows: , , , as well as ; The input resolution of the five stages of a partial U-KAN encoder is defined as follows: , , , as well as ; Finally, the five-stage multi-scale semantic representations of the global U-KAN encoder and the local U-KAN encoder are defined as follows: （4）； （5）； in, and These represent the five-stage semantic representation sets captured by the global U-KAN encoder and the local U-KAN encoder, respectively.

4. The U-KAN medical image semantic segmentation method based on multi-path Mamba fusion according to claim 3, characterized in that, Step 2 specifically includes: Step 2.1: Construct the Mamba interaction phase, simulating the cross-attention mechanism by using one path semantic representation to enhance the expression of another path semantic representation; Given step 1.2, obtain the coarse-grained semantic representation extracted from the k-th stage in the multi-stage, multi-scale semantic representation set. and fine-grained semantic representation ; use The operation converts a two-dimensional semantic representation into a one-dimensional semantic representation: （6）； in, and They are respectively and One-dimensional semantic representation; Dynamically generated using linear projection layers and Mamba parameter set of state-space model and ,in, and These are the input matrix parameters; and These are the output matrix parameters; and It controls how much information is absorbed from the current input into the state at each time step, and how much historical information is retained; Discretization is performed to transform the continuous-time system into a discrete-time system with multiple time steps: ， （7）； ， （8）； in, and It is a pre-defined learnable state transition matrix; and It is the state transition matrix after discretization; and The input matrix after discretization; Represents an exponential function; For each time step t, by exchanging the output matrix and Achieve semantic information interaction across paths: ， （9）； ， （10）； in, and This represents the hidden state vector at time step t; and This represents the hidden state vector at time step t-1; and The input is represented by a one-dimensional semantic representation of time step t; and This represents the output of the state selection model at time step t; After T-step, utilize The operation will output a one-dimensional semantic representation. and Mapping back to two-dimensional semantic representation respectively and ; Step 2.2: Constructing the Mamba aggregation stage, a bidirectional scanning mechanism is adopted to adaptively fuse the multi-scale semantic representation enhanced in Step 2.1 to generate the U-KAN skip connection representation; Based on step 2.1, obtain the two-dimensional semantic representation after the interaction in the k-th stage. and ,use The operation is converted into a one-dimensional semantic representation and a concatenation operation is performed to obtain a preliminary fusion skip connection representation at the k-th stage. : （11）； use The operation generates a reverse jump connection representation. ,Right now Information extraction is performed by applying the standard state-space model to the forward and reverse jump connection representations respectively. ， （12）； in, This represents the complete forward propagation process of the state-space model; Indicates the process Semantic representation extracted from forward processing; Indicates the process Semantic representation extracted in reverse; Finally, and Perform average fusion and utilize The operation is mapped back to two-dimensional space to generate the final skip connection representation for the k-th stage. : （13）； Therefore, a five-stage skip connection representation is generated through a cross-path semantic adaptive fusion network. .

5. The U-KAN medical image semantic segmentation method based on multi-path Mamba fusion according to claim 4, characterized in that, The forward propagation process of the state-space model includes parameter generation, discretization, and sequence scanning.

6. The U-KAN medical image semantic segmentation method based on multi-path Mamba fusion according to claim 5, characterized in that, Step 3 specifically includes: The multi-stage supervised decoding network includes a U-KAN decoder and a deep hierarchical loss. The U-KAN decoder is constructed as a symmetrical structure of the U-KAN encoder, consisting of two KAN stages and three convolutional stages connected in series. The input of each decoding stage is composed of the skip connection semantic representation of the corresponding encoding stage and the output of the previous decoding stage. Given the output of the (k-1)th decoding stage and the semantic representation of the skip connection corresponding to the k-th decoding stage. The decoding process in the k-th decoding stage is defined as follows: （14）； （15）； in, The output of the k-th decoding stage; To build a deep hierarchical loss, the model maintains prediction consistency at different resolutions: supervision signals are added in three stages: the final stage output of the U-KAN encoder, the first stage output of the U-KAN decoder, and the final stage output of the U-KAN decoder, to train the overall segmentation network. The depth-level hierarchical loss is defined as follows: （16）； in, , and These are the hyperparameters of the three-stage supervision loss; Represents the true segmentation mask of the image; This indicates the final output from the U-KAN encoder. The generated predicted segmentation mask; This indicates the output from the first stage of the U-KAN decoder. The generated predicted segmentation mask; This indicates the final output from the U-KAN decoder. The generated predicted segmentation mask; Each loss It consists of two sub-losses: （17）； in, This represents the weighted IoU loss, used to measure the degree of overlap between the predicted bounding box and the ground truth bounding box; This represents the weighted cross-entropy loss; After the entire model is trained using deep hierarchical supervised loss, medical images are input into the trained model to obtain the corresponding semantic segmentation mask.

7. The U-KAN medical image semantic segmentation method based on multi-path Mamba fusion according to claim 6, characterized in that, In step 3, the hyperparameters of the supervision loss in the three stages are... =0.2, =0.2 and =0.6.

Images